Not so fast. Don't hold your breath...
I was also afflicted for the last few days - with this registry setting present and set to 1..
I'm also on Office 2021 Pro (perpetual) - lacking the Startup option dropdown.
The annoying popup reliably occurred every time I opened Outlook for the last several days - and then magically stopped today, until late tonight. Oh.. I didn't previously close Outlook with any open messages.
ReopenWindowsOption had been there all along, set to 1, since the issue arose - I did not need to add it.
While it was behaving today, I tried to force it using key value 0 while closing with open messages (no pop up opened on startup)... as well as closing Outlook with an open message and key value 2 (it did not open the message on startup). So it is not obeying the registry value anyway.
This quirk appears to have a complete mind of it's own regardless of the registry setting - at least for me.
Oh... I've followed Diane's work for years (over two decades actually - oh my) - and I'm confident she'll get to the bottom of it.
Cheers!
Chip