I'd just like to add my experience. This has always been an issue for me but having just upgraded to Windows 10 64bit using Acrobat DC, the problem has got worse. None of the simple workarounds work and I too do not have the option of using a different PDF
reader. I deal with PDF files in a graphics and print capacity and need to preview and delete files many times a day.
I would have to go to services and end the prevhost.exe task in order to delete the PDF, not an ideal workflow. I think that perhaps prevhost.exe does stop itself after an as yet undefined amount of time has passed after turning off the preview pane. I'm
keeping my eye on it.
I also found that I can rename one of the locked PDFs, e.g. to .txt, but still can't delete it. Maybe this snippet of info will help Microsoft fix the bug;)