Hi Tracey Farrar,
Understandable this is an issue with the desktop client, that you can't add a new meeting participant very easily without sending out a notification to everyone.
This causes issues when you are organizing large meetings and you get confirmations one by one, over a period of time and you need to send them an invite as a confirmation.
I recommend you can try using a 3rd party tool like Let's Calendar - www.letscalendar.com via which you can organize these calendar invites and then add participants to them one-by-one.
The USP of www.letscalendar.com is that it will natively integrate with your O365 account, and send the invites via your calendar to the participants so they will see YOUR name as the sender and not someone else, and then because you can send 1:1 invites, you can personalize these invites as well with all the relevant details you want.
All native O365 features will work like tracking responses, editing the meeting, rescheduling it or updating it.
Do give it a try, it may help your use case out.