Managing sound and video settings during Teams meetings and calls for optimal communication
Hanging up first is the key to me (regardless of the button pressed on Teams or headphone which is possibly faster for you).
See my post with the available workarounds so far on page 9, that i've just edited with more details .