What kind of graphics card do you have ?
Do you have the latest drivers installed ?
Not everything is just the Windows side of it your hardware might have a bottleneck you are unaware of or an accessory connected that is keeping your system busy.
If you could switch back to prior Windows would the problem go away ?
@R Starzuft,
See my post above for a sample configuration that shows the delay.
Also, I can confirm that using Windows 10 on the same machine the switching from dark to light and back has no delay at all.