Hello Allicit,
I can confirm that the behaviors that you're encountering are experienced the very same on my side with my one Cortana Invoke speaker: yes.
Specifically, the WiFi drops...which makes no sense to me whatsoever. I am in Washington state, U.S.A.
I'm really not noticing "slowness" to respond so much, but that could be due to the fact that we are using our Google Home Mini more and more since Smart Life dropped their service from IFTTT.
I am still holding out hope for Cortana because I am 100% integrated to MSFT for all things productivity such as Calendar (Outlook) and "To Do".
I would love to have (3) Invoke speakers as they are just incredible units.
I hope my response helps you.
Best Regards,