Yeah. This seems like some sort of bug. Mine gets stuck on out of office, even after the Calendar time for being out of office is over.
I.e.: 9-10: Outlook calendar is out of office
10:30: Teams still shows as out of office
Try this:
A. In Outlook go to Automatic Replies where you set the Out Of Office reply and 1) turn it on, 2) click ok to set it, then 3) turn it off, 4) click ok to set it.
B. If there are any all day calendar events (out of office events or anything else), 1) set the "Show As" to "Free", 2) set Reminders to None.
Then wait -- I do not know if it takes 20 minutes as someone else wrote -- should take effect in 1 to 2 hours worst case.
This worked for me.
Good luck.