This is not something you can "fix" this is a problem with the way windows is designed from what I can tell. The problem (for me anyway) is that all these monitors have audio over HDMI or whatever connection and so (for me) when a monitor with audio capability is connected windows automatically switches to it, by default and there is apparently no setting to disable this behavior.
Its so annoying to be completely honest. Why is it so hard to give us more control over this? Why cannot I not set it to not ever change the audio output device from a specific device (input and output) aka locked? - This would hold true unless for some reason that device became unavailable at which point it would default to the next best device. However, for this to be an actual good feature, is that if the original "locked device" became available again, in that situation it should automatically switch back to that device. I know this cannot be that hard to implement.
A lot of us use laptops and we move back and forth between workstations that have different peripherals so this is a constant annoyance. I'm sure the only hope is a 3rd party software that can make up for MS shortcomings on this issue. I do use a "tray-icon" software package called "audio switcher" which helps me deal with switching things back when they get messed up with out having to mess about in the system settings. It has a "favorite" feature but unfortunately it does not work how I just described.
The problem with the disconnecting / reconnecting devices is that even if you "uninstall" or "disable" the device to teach windows you do not want to use them, as soon as they get disconnected and reconnected any of those settings get tossed out with the bathwater and its treated as a completely new device so you cant even disable all of the "devices" you never want to use because of this...