Yes, I just tested again after updating to build 26100.4351 and I no longer see the issue. Cool. It appears that Microsoft quietly fixed this issue. Hopefully everyone else sees the same improvement after updating.
Thanks JeffWhiteAZ and HeyKeys!
Note: While testing, my address bar drop-down menu sometimes stayed on the screen after selecting a folder from the drop down. When I get a chance, I'll test that on some more Windows 11 machines to see if it's something common or something that only happens on my test machine.
I did update to build 26100.4351 today. Thanks for mentioning the build nr explicitly.
The process of creating new folders now seems to work fine (when you are working in the main windows and then press new+folder in the address bar.
However it sill faced issues with the address bar suddenly dropping history / with focus between windows and the address bar, especially when I move from monitor 1 to monitor2. I will try to formulate a clearer description of the issue in the next days.