I've had almost the exact same problem - new Surface Pro 4 with Windows 10 (no dock or second battery). The taskbar thumbnail preview icons disappear extremely quickly despite continuing to hover my moues over them. The issue is semi intermittent in that I cant make it happen ALL the time. That said - it seems to happen a LOT if I'm plugged in and my battery is at 100%. If I'm on battery alone (not plugged in) or even if I'm plugged in and its charging (vs being at 100%) it seems to behave just fine for the most part. But once it hits 100% it starts acting up which in turns causes a huge impact to jumping between e-mails, IE windows, etc...
From everything I've read it seems like it might be related to some underlying update/change in status grabbing focus away from the taskbar (e.g. behind the scenes going from charging to 100% and back or something...?).
I've tried updating all drivers (including battery), tried my work IT support, spent time with Microsoft customer support, looked all around the web (its NOT registry entries / hover time). I wish there was some way to figure out what is making it happen, but I'm stumped.
Anyone have any ideas!?! Please help!? :-) Thanks!