I have to re-pair my xbox controller EVERY time i want to use it on Win 11, was also like this on Win 10. It just wont connect normally, it works perfectly fine anywhere else, from my tablet, to phone to xbox. I went so far as to buy a DualSense controller and DSX on Steam. I wholeheartedly recommend getting the DualSense and DSX program, even the haptic feedback works(which i can customize to no end) and i can change colorways and all kinds of nice stuff. It evens connect without me having to re-pair and reconfigure every time! :)
Solution for me in regards to Windows scuffed BT with Xbox controller: Buy Playstation Controller and DSX program instead.