Foundry IQ knowledge base not working despite using same index and foundry completion model as the other working knowledge base.

Hello @Charles Bartholomae

Since one knowledge base is working and the other is not, despite using the same index and chat completion model, the issue is likely related to the configuration of the specific knowledge base rather than the Azure AI Search service, RBAC permissions, or network settings.

I would recommend checking the following:

• Verify that the non-working knowledge base is correctly linked to the intended LLM/project connection and has not been associated with an outdated or duplicate connection object.
• Review the knowledge source configuration for the affected knowledge base to ensure it correctly references the index and is returning data as expected.
• If the knowledge base was created recently, allow some additional time for provisioning and then test again.
• Enable diagnostic logging and review the underlying error details, as the playground message is generic and may not reflect the actual cause.
• If possible, try recreating the affected knowledge base and reattaching the Search and Azure OpenAI/Foundry connections.

If the issue persists, could you please share:

• Whether the Search service uses public or private network access
• The full error message (including any Request ID)
• Whether both knowledge bases were created from the same portal and around the same time

These details can help determine whether this is a configuration issue or a backend provisioning problem.

Thankyou!

Hello Charles Bartholomae,

Greetings! Thanks for raising this question in the Q&A forum.

Since the first knowledge base works fine and both live on the same search service, you can rule out service-level RBAC and network access as the root cause here, even though that's what the generic error message points to. That message is a catch-all the playground shows whenever the knowledge base can't complete a retrieval call, and Microsoft's own troubleshooting guidance for this exact error lists the real common causes as: incorrect project connection configuration, and empty or misconfigured knowledge sources. Because your two knowledge bases are separate objects, each one holds its own reference to the LLM connection and its own knowledge source definition, even when both ultimately point at the same underlying index and the same model deployment name. So the second knowledge base's own object-level configuration is the most likely place something is wrong, not your identity/network setup.

Work through this in order:

Check the second knowledge base's own LLM connection binding, not just which model it displays in the UI. Selecting "the same completion model" in the portal doesn't guarantee it resolved to the same underlying connection object. In the Azure portal, open Azure AI Search > Knowledge bases > [the broken one] > JSON view (or REST GET) and compare the answerModel/LLM connection block against the working knowledge base. Confirm it's pointing at a valid, still-existing Azure OpenAI connection with a live deployment, not a stale or partially-created connection.

Inspect the knowledge source(s) attached to the broken knowledge base independently. Even though you say it's the same index, each knowledge base creates or references its own knowledge source object. Query the knowledge source directly (bypassing the knowledge base) to confirm it resolves and returns data on its own:

GET https://<search-service>.search.windows.net/knowledgeSources/<name>?api-version=2026-05-01-preview

If this knowledge source object is empty, still provisioning, or has a broken pointer to the index, that alone will make the knowledge base fail even though the index itself is fine.

Recreate the project connection for this specific knowledge base rather than assuming the shared connection carries over. If the second knowledge base was created through a different flow (for example, Azure portal vs Microsoft Foundry portal, or created before a connection change), it may hold a reference to an old or duplicate connection object. Delete and recreate just this knowledge base's connection to Azure AI Search / Azure OpenAI, then retest.

Rule out propagation delay. Newly created knowledge bases and knowledge sources in Azure AI Search can take a few minutes to fully propagate before playground queries succeed reliably. If this knowledge base was created recently, wait 5-10 minutes and retry before doing deeper troubleshooting.

Enable diagnostic logging on the search service to capture the actual request/response detail behind the generic playground message, which will show the real HTTP status and failure reason rather than the sanitized UI text.

Confirm both knowledge bases are on the same API version. If the second one was created against a different Search REST API version (older preview vs 2026-05-01-preview), some fields required for agentic retrieval may be missing or incompatible even though the schema looks similar in the portal.

If none of this isolates it, capture the request ID from the chat playground error along with the knowledge base name and open a support ticket, since Foundry IQ is still a recent preview capability and object-level provisioning bugs do occasionally need backend investigation.

If this answer helps you kindly accept the answer which will help others who have similar questions.

Best Regards,

Jerald Felix.