Try doing the following as a temporary solution, which might be annoying:
Open Sound Settings
Select 'Manage Sound Devices'
Under Output Devices, select then Disable the AirPods Pro Hands-Free AG Audio
Under Input Devices, select then Disable the AirPods Pro Hands-Free AG Audioas well
This helps me get a more consistent connection. BUT, you will need to Re-Enable these options any time you want make use of that Device option, such as when on Zoom/Skype Call, or simply using your Mic whatsoever.
Hope this helps a bit.