I have the exact same problem and it can not be solved reliably.
The BT device will connect, but Windows will refuse to set up an audio device. The headset shows up as connected in the BT device settings of Windows.
I can also see the charge percentage. But no audio device in Windows becomes available. Neither Microphone nor speaker. When i forget the device, pair it again and reinstall the bluetooth audio devices in device manager it works for this session. Next time Windows is rebooted, it's back to square one: The device connects, but audio devices are not being created. Windows is fully updated and i have the most recent drivers installed.
To make matters worse, this isn't even a one-off problem. It can be reproduced on multiple computers. I am working as a systems administrator and have access to a good dozen of notebooks made by HP and most of them have this problem right after Windows 11 is set up.