M365 Copilot Agent built using Agent Builder fails to retrieve/stage knowledge base stored in SharePoint Site or even in the attachments

Question

M365 Copilot Agent built using Agent Builder fails to retrieve/stage knowledge base stored in SharePoint Site or even in the attachments

Rohit Budha Borse 0

We have developed a Microsoft 365 Copilot Agent using Agent Builder and configured a SharePoint-based knowledge repository to support AI-generated training and technical content.

The agent exhibits inconsistent and unreliable behaviour when accessing knowledge sources. While the same prompts occasionally succeed, they frequently fail due to apparent retrieval, indexing, or file access issues.

As a result, the agent is unable to consistently generate grounded responses based on the configured knowledge base.

Knowledge Base Characteristics

Approximately 15-20 source documents
Each document contains approximately 300-500 pages
Documents are stored in SharePoint
Documents are configured as knowledge sources for the agent
Content consists primarily of large technical manuals and reference documentation

The agent intermittently fails to access information stored in SharePoint knowledge sources.

Typical behaviours include:

Agent reports that it cannot find relevant information
Agent fails to retrieve document content
Agent appears unable to search indexed content
Agent cannot generate output from the configured knowledge source
Agent falls back to generic LLM responses rather than grounded responses
Identical prompts sometimes work and sometimes fail without any configuration changes

The agent should:

Reliably retrieve content from configured SharePoint knowledge sources
Consistently ground responses on available documentation
Return the same result for identical prompts when no source documents have changed
Successfully process large technical documents after indexing is complete

0 comments

2 answers

Your answer

Answer 1

For an Agent Builder agent that intermittently fails to ground on SharePoint knowledge, the supported causes in this scenario are:

Missing or incomplete SharePoint search indexing
- SharePoint grounding depends on Microsoft Search indexing.
- In SharePoint search, test a unique keyword from a target document. If the file does not appear, indexing is incomplete.
- Recently uploaded content can also fail until indexing finishes.
- If urgent, a SharePoint site admin can trigger reindex from Site Information > View all site settings > Search and offline availability > Re-index site.
User permission trimming
- The agent only returns content the signed-in user can read.
- Verify the affected user can open the exact SharePoint file directly.
- Test the same query with the same user in Microsoft 365 Copilot Chat. If that also fails, the issue is likely permissions or indexing.
- For Agent Builder specifically, auto-sharing has a limitation: SharePoint files and folders are only auto-shared when sharing with specific security groups, not everyone in the organization. Manual permission updates might be required for the agent to return information from those sources.
Protected or encrypted files
- Files protected by an encrypting sensitivity label, Double Key Encryption, or a password can show as Ready but still return no response.
- Verify whether the files prompt for a password, indicate DKE, or use an encrypting sensitivity label.
- If allowed by policy, remove the protection or publish an unprotected copy in a SharePoint location the agent can access.
SharePoint source or configuration limitations
- Only the top three SharePoint search results are used to summarize and generate a response.
- Only modern SharePoint pages are supported; modern pages with SPFx components are not supported.
- Content from classic .aspx pages is not used to generate answers.
- If a SharePoint file name contains null characters, the agent returns no results from that knowledge source.
Agent Builder licensing and authentication requirements
- Declarative agents grounded in SharePoint can fail at runtime with “Sorry, I wasn’t able to respond.” when the signed-in user does not have a Microsoft 365 Copilot license.
- SharePoint and OneDrive knowledge sources require an active Copilot license.
- For SharePoint grounding, the connection must use User authentication. Service principals aren’t supported.
- The user must have at least Read permission on the target SharePoint site.
Restricted SharePoint Search
- If Restricted SharePoint Search is enabled for the tenant, SharePoint as a knowledge source is blocked and no results are returned.
- Confirm with the SharePoint administrator that it is not enabled, or that the relevant sites are on the allowed list.
Large-content reliability expectations
- The available guidance does not state that large 300–500 page manuals are guaranteed to return consistent results after indexing.
- What is documented is that retrieval depends on SharePoint search results, supported formats, supported page types, permissions, and licensing.

A practical verification path for this case:

Pick one document that fails intermittently.
In SharePoint search, search for a unique phrase from that document.
- If it does not appear, wait for indexing or request a site reindex.
Open that exact file using the same signed-in account that uses the agent.
- If access fails, fix SharePoint permissions.
Check whether the file is encrypted, password-protected, or protected by DKE/sensitivity-label encryption.
- If yes, use an unprotected copy if policy allows.
Confirm the signed-in user has an active Microsoft 365 Copilot license.
Confirm the agent uses User authentication, not a service principal.
If the source is SharePoint pages, confirm they are modern pages and do not use SPFx components.
Check whether Restricted SharePoint Search is enabled in the tenant.
If the issue is limited to Agent Builder sharing behavior, manually update SharePoint file/folder permissions used by the agent.

If identical prompts sometimes succeed and sometimes fail with no content changes, the documented causes that best match are indexing state, permission trimming, licensing/authentication requirements, Restricted SharePoint Search, or unsupported/protected content.

Answer 2

BabybooHN 3,040 Independent Advisor

Dear @Rohit Budha Borse

Thank you for providing the detailed information. I understand that the agent intermittently fails to retrieve content from its configured SharePoint knowledge sources and may fall back to a generic response.

To narrow down where the issue is occurring, could you please provide the following details?

Does the issue occur with SharePoint sources only, or does it also occur when the same document is added directly as an attachment to the agent?
Please provide the name of one affected document and its file type (for example, PDF or DOCX).
Approximately how large is the affected file in MB/GB, in addition to the page count?
When the issue occurs, does the agent consistently fail to retrieve information from the same document, or does it vary between different documents?
Please provide one exact prompt that reliably reproduces the issue, along with:
- The agent’s response when retrieval succeeds
- The agent’s response when retrieval fails
Does the agent show the SharePoint source as successfully configured/available, or is there any indication that the source is still being processed or indexed?
Does the problem affect all users who can access the agent, or only a particular user?

These details will help determine whether the behavior is related to knowledge-source configuration, document indexing/retrieval, document size/content, or the agent itself, rather than treating all failures as a single issue.

If possible, please start with one specific document and one reproducible prompt. That will make it much easier to isolate the problem.

Rohit Budha Borse 0 Reputation points

2026-09-11T15:13:12.9633333+00:00
Hi @BabybooHN , thank you for your response.

The issue occurs with both:

SharePoint-based knowledge sources

Documents directly uploaded as attachments in the Agent's Knowledge/Attachment section

Knowledge Source Details

File size ranges from approximately 5 MB to 50 MB.

Documents typically contain 200 to 500 pages.

Most source documents are PDF files.

Example document name: "H-type Container Handler SOM Manual - MARUTI.pdf".

Observed Behaviour

The behaviour is intermittent and inconsistent. Success rate is 20%.

At times, the agent is able to successfully retrieve information from the document and generate the expected output. However, when the same request is executed later (for example, the next day or even a few hours later), the agent reports that it is unable to retrieve or access the source documents and hence it cant create final production ready source grounded course package.

We have also observed that the agent frequently does not follow the instructions configured in the Instructions section, even though no changes have been made to the agent configuration.

Additional Observations

The agent consistently shows that the SharePoint source has been successfully configured.

It is able to display and recognise the files available in the configured source.

It can successfully validate mappings stored in one of our Excel files.

These observations suggest that the agent can detect the knowledge sources and access certain metadata.

However, despite this, the final content generation process frequently fails.

In our specific use case, the agent is expected to generate a structured output package (ZIP file). The generation of this final ZIP output fails consistently, even when the agent appears to have successfully identified the relevant source documents and validated the associated mappings.

Impact

The agent is currently under testing and evaluation and the issue is reproducible across all users who have access to it. As a result, we are unable to determine whether the platform can reliably support our intended production use case.Thank you for your response.

The issue occurs with both:

SharePoint-based knowledge sources

Documents directly uploaded as attachments in the Agent's Knowledge/Attachment section

Knowledge Source Details

File size ranges from approximately 5 MB to 50 MB.

Documents typically contain 200 to 500 pages.

Most source documents are PDF files.

Example document name: "H-type Container Handler SOM Manual - MARUTI.pdf".

Observed Behaviour

The behaviour is intermittent and inconsistent.

At times, the agent is able to successfully retrieve information from the document and generate the expected output. However, when the same request is executed later (for example, the next day or even a few hours later), the agent reports that it is unable to retrieve or access the document.

We have also observed that the agent frequently does not follow the instructions configured in the Instructions section, even though no changes have been made to the agent configuration.

Additional Observations

The agent consistently shows that the SharePoint source has been successfully configured.

It is able to display and recognise the files available in the configured source.

It can successfully validate mappings stored in one of our Excel files.

These observations suggest that the agent can detect the knowledge sources and access certain metadata.

However, despite this, the final content generation process frequently fails.

In our specific use case, the agent is expected to generate a structured output package (ZIP file). The generation of this final ZIP output fails consistently, even when the agent appears to have successfully identified the relevant source documents and validated the associated mappings.

The agent is currently under testing and evaluation and the issue is reproducible across all users who have access to it. As a result, we are unable to determine whether the platform can reliably support our intended production use case.
BabybooHN 3,040 Reputation points Independent Advisor

2026-09-11T15:52:35.8233333+00:00
Dear @Rohit Budha Borse

Thank you for the additional details. This helps narrow the issue down significantly.

Since the same behavior occurs with both SharePoint knowledge sources and files uploaded directly to the Agent, and the Agent can recognize the files and validate the Excel mapping, this does not appear to be limited to SharePoint permissions or indexing.

The combination of large PDF files (5–50 MB and 200–500 pages), intermittent retrieval, inconsistent instruction following, and the consistent failure when generating the final ZIP suggests that we should isolate the retrieval stage from the final output-generation stage.

Could you please perform the following focused test?

Select one affected PDF, preferably “H-type Container Handler SOM Manual - MARUTI.pdf”.

Create a small test copy containing only a limited section of the document (for example, 20–30 pages) and add that copy as the Agent’s knowledge source/attachment.

Use one exact prompt that asks the Agent to retrieve specific information that is clearly present in that test document.

Run the same prompt several times and note whether the retrieval succeeds consistently.

Separately, test whether the Agent can generate a simple text response from the retrieved information without attempting to create the ZIP package.

Please also provide one example of:

The exact prompt used.

The response when it succeeds.

The response/error when it fails.

Whether the failure occurs before or only when the Agent attempts to generate the ZIP file.

This distinction is important because if the smaller document is retrieved consistently but the ZIP generation still fails, the knowledge retrieval itself is likely working and the issue would be more specific to the Agent’s output-generation workflow. Conversely, if retrieval remains intermittent even with the smaller document, we can focus on the knowledge retrieval/indexing behavior.

Also, please confirm that the affected users have active Microsoft 365 Copilot licenses and can open the source documents directly with the same accounts used to access the Agent.

At this stage, I would not recommend assuming that the 200–500-page documents are unsupported solely because of their page count. The test above should help establish whether document size/content is contributing to the behavior.

Once I have the result of this controlled test, I can narrow down the next step rather than changing multiple Agent or SharePoint settings at the same time.

M365 Copilot Agent built using Agent Builder fails to retrieve/stage knowledge base stored in SharePoint Site or even in the attachments

2 answers

Knowledge Source Details

Observed Behaviour

Additional Observations

Impact

Knowledge Source Details

Observed Behaviour

Additional Observations

Your answer