That is not the behavior I see on my installation of Word 2016: If I click Vertical and exit Word, Vertical remains activated.
Which is your specific release of Word? Look at File | Account (under "Office Updates").

Note that both Vertical and Side To Side are new options in Office 2016. Assuming that the options are indeed saved to the registry, deleting the registry keys for Word may fix the problem on your system.
Note that I'm not sure which registry entries are relevant. I'll see if I can find out.
Hi Stefan:
Thanks for your assistance. I also have the same build, BUT I am only experiencing the issue on my Surface Studio. On my Surface Pro 4, my Word has been updated to version 1702 but is not experiencing the issue.
I'm unsure why the difference. You mentioned deleting the registry keys; how would one go about doing so?
Thanks again; appreciate it!