How do you handle Noroboto-like attacks in M365 Copilot Studio Agents with Graph-grounded documents?

Microsoft 365 Copilot and Copilot Studio use multiple safety and grounding mechanisms that help mitigate risks from hidden or adversarial content in Graph-grounded documents, including “Noroboto-like” attacks. These mitigations focus on both how content is used for grounding and how prompts/responses are filtered.

Key platform-level mitigations

1. Grounding in trusted, permissioned data

• Copilot Studio and Microsoft 365 Copilot ground responses in organizational data sources that the user is authorized to access (for example, SharePoint and Microsoft Graph connectors).
• Grounding is designed to reduce ungrounded or fabricated content and keep outputs aligned with enterprise information.
• Agents only access knowledge sources and connectors that are configured and allowed by organizational policies and role-based access controls.

2. Content safety filters and responsible AI checks

• All generative AI requests in Copilot Studio are subject to content moderation policies that:
  • Detect and block harmful or offensive content (hate, sexual, violence, self-harm, etc.).
  • Protect against malicious attempts such as jailbreaking, prompt injection, and prompt exfiltration.
  • Check content twice: once on user input and again on AI-generated output before it is shown.
• Microsoft 365 Copilot also applies defense-in-depth safeguards, including:
  • First-party protections to detect jailbreak attempts and prompt injection patterns (including cross-prompt injection attacks).
  • Content harm filters on prompts and generated responses.
  • In some cases, additional safety mitigations built into the underlying model.

3. Guardrails and agent boundaries

• Copilot Studio agents use guardrails and prompt enrichment to keep behavior within defined scope and reduce misinterpretation.
• Tools and connectors are constrained to specific actions and data sources; agents cannot execute actions outside their defined scope.
• Admins can configure secure connectors and data loss prevention controls for knowledge and custom tools, which helps limit exposure to untrusted or adversarial content coming from external sources (such as emails, tickets, or externally provided documents).

4. Continuous monitoring and evaluation

• Copilot Studio includes evaluation, monitoring, and feedback mechanisms so organizations can:
  • Assess agent quality, safety, and reliability.
  • Detect unintended behaviors and performance drift.
  • Iterate on agents and safety configurations as new risks are identified.
• Microsoft applies phased release and continual evaluation for Microsoft 365 Copilot, updating mitigations as new attack patterns and misuse scenarios are discovered.

Best practices for using Graph-grounded documents (including PDFs)

Within the constraints of the documented behavior, the following practices align with the provided guidance and help reduce risk from hidden or adversarial content in documents:

1. Restrict and validate knowledge sources

• Treat external or untrusted PDFs and other documents as higher risk knowledge sources.
• Prefer grounding in trusted, internal repositories and systems that comply with organizational policies.
• Use secure connectors and data loss prevention controls in Copilot Studio to limit which external sources can be used as knowledge.

2. Apply role-based access and least privilege

• Ensure that only appropriate users and agents can access specific SharePoint locations or Graph-connected content.
• Regularly review permissions and role-based access controls so that adversarial or experimental content is not broadly available as grounding material.

3. Separate experimental from production data

• For high-risk or externally sourced documents, consider:
  • Using separate sites or libraries for evaluation and testing.
  • Validating how agents behave with those sources in a sandbox environment before enabling them in production.

4. Maintain human oversight

• For high-stakes or sensitive use cases (finance, legal, healthcare, etc.), ensure that:
  • Users review AI-generated outputs before acting on them.
  • Critical decisions are grounded in verifiable data and not solely on model-generated content.

5. Monitor for anomalous behavior

• Use available dashboards and monitoring to watch for:
  • Unexpected tool calls or actions.
  • Responses that appear to follow hidden instructions or deviate from expected behavior.
• Collect user feedback and update agents, knowledge sources, and connectors when suspicious behavior is observed.

These measures, combined with Microsoft’s responsible AI checks, content safety filters, and injection/jailbreak protections, form the primary mitigation path for Noroboto-like and similar prompt-injection attacks in Graph-grounded Copilot Studio agents.

References:

• Application card: Microsoft 365 Copilot
• Application Card: Microsoft Copilot Studio
• Extend Microsoft 365 Copilot with agents
• Copilot Studio licensing
• Data, Privacy, and Security for Microsoft 365 Copilot
• FAQ for the agent creation from natural language experience