Alright, this is a new twist to this issue.
My company used Skype well before Teams' existence, so some of us still use Skype to voice call each other as a habit as it is more reliable than one-to-one Teams voice call. I have noticed if I end a Teams meeting/call and then immediately use Skype to voice call someone, the audio from Bluetooth headset will not be heard on the Skype call. I can switch to handset and it will have audio.
After weeks of experimenting, I noticed if I disable "Nearby devices" permission in Teams, then the same sequence of ending a Teams call then try to make Skype voice call works on the Bluetooth headset. Interestingly, even if I remove "Nearby devices" permission for Teams, it could still route audio to my Bluetooth headset. In looking at Android specs, "Nearby devices" permission is NOT needed to access Bluetooth audio stack. This permission is to RF scan nearby devices as in discovering nearby devices, such as audio conference systems or speakers.
I am a little surprised that Teams and Skype haven't found ways to work, without interfering each other.