I agree with you @Nunya Bizness.
Copilot supports the RECALL functionality:
Microsoft states that the screenshots and database(s) are encrypted, but I think that any powerful AI APP or utility, especially one that runs in the background, lacks openness and transparency.
Because most are proprietary in nature, there is (AFAIK) going to be a great lack of viable tools to verify vendors claims for ANY such AI models.
Even the OpenAI group is NOT open about how it trains it's AI models according to Mozilla, and is even fighting a major court case brought by the New York Times – asking them to explain its use of personal data and copyrighted content in its AI models.
This shows that the big companies developing this technology are hiding behind the "proprietary" wall, to avoid any oversight and responsibility.
It only takes a minor coding error to open up a hackers paradise, and therefore I put this in the same basket as IoT firmware, as regards security and privacy.
Data Breaches make big news (Optus, AT&T, Medibank, Ticketmaster to name a few), but this does NOT help those affected.
I believe that we should ALL be aware of the inherent risks that this sort of AI poses, and do our utmost to reduce our attack surface, whether that is in a Home network, or a business setting.
So, turn off what you do not need, or what might pose a security or privacy risk. As I see it, COPILOT is more useful to the Microsoft AI LLM systems than it is to users.
See this for proof of an issue waiting to happen:
https://www.theverge.com/2024/6/3/24170305/microsoft-windows-recall-ai-screenshots-security-privacy-issues