I actually had to disable the "show preview handlers in a preview pane" in the Windows Explorer, options, view Tab. I was then able to easily delete all my sticky files not deleting because I do have my preview pane setting to on.
Interesting... I'll definitely give that a try and report back.
If others don't aren't clear about this:
In Explorer go to View Tab
Select Options button on right
Select the View tab on the Folder Options window
In the Advance Settings window deselect Show preview handlers in preview pane
This solved the problem in Win 8.1 for me. Now I can rename multiple folders one after the other without having to distract explorer by selecting the properties of another folder entirely in-between renames.
@DaveMaddison, Your not quite jumping ship, more like settling for a ferry.
Nup its back again. Very frustrating. I hoped I'd found a way to keep the folder thumbnails and still be able to rename files. Maybe it will be fixed in Windows 10. MS are promoting it as the productive OS. Do! rename your files without an in use error.