Right- right. I went down that road. There is a way to not automagically install devices, but it has something to do with a whole _class_ of devices, I haven't found a way to get it to recognize a particular device. I'm not sure it's every time you disconnect and connect, but it does feel that way.
Have a look at my post further up- that might help you; I was able to tell Windows, "Don't use the device" both in Device Manager and in W10 settings in, it's "Manage Sound Devices"
This worked to finally get windows to stop thinking the USB Hub built into my monitor was a preferred audio output device. I said "don't use it" and it has not bothered my since.
Good luck.
<removed by Moderator>