I did find one "fix" that isn't really a fix at all, but at least it prevents one from having to log out.
- If you hit Win+E to open an explorer window, Explorer will either crash or not crash. If it doesn't crash, open Task Manager, find Explorer, right-click, and select restart. If it does crash, proceed right to step 2! Otherwise, the manual restart of Explorer should start the crash loops.
- At this point, hitting Win+E will likely cause Explorer to crash in a loop. Just keep tapping Win+E and have Explorer crash 4 or 5 times in a row, when you start seeing Explorer windows pop up en masse, stop, and try copy paste. It should work again at this point.
Kinda hilarious that crashing Explorer over and over fixes the problem, but does also reinforce that it is likely a Windows bug.
I've since given up thinking it is any of my third-party software, moved on to hardware troubleshooting for a few to see if old USB 2 input devices were causing the issue by making it trigger something, as often when logging back in, the start menu would flicker open/closed a few times like someone was hammering the Windows key. After discovering the above reproducible steps to fix copy/paste though, I don't think it is third-party software, nor the old USB hardware. With or without the hardware the bug still manifested daily.
When installing anything that has shell extensions I avoid installing them since W11 has butchered the use of the context menu, so I highly doubt it's any third-party apps interacting with Explorer, but anything's possible.
What an absolutely dumb problem. Linux is looking nicer every day.