Azure Foundry agent uses the old completions API by default. How to migrate to the newer Responses API?

Ofir.D 20 Reputation points
2026-08-02T11:30:11.6866667+00:00

Azure Foundry agents seem to be using the old Completions API by default.

Is it possible to configure it to use the newer Responses API?

Here's a test i did:

First turn - No KB usage:

I set up an agent with a connected Knowledge Base and a connected Memory. However, For this test purposes, I intentionally ask a general question which doesn't require a KB query.

Result: success - as can be seen in the below screenshot:

First turn - No KB usage - Success

Second turn - With KB usage:

With the same agent, in a subsequent turn, I ask a question whose answer requires a KB query.
This is indeed what happens, and issues an error:

Error: An error occurred invoking knowledge_base_retrieve: An error occurred invoking knowledge_base_retrieve: BadRequest Message-{"Message":"Could not complete model action. The model endpoint returned status code \u0027400\u0027 (BadRequest). Function tools with reasoning_effort are not supported for this model in /v1/chat/completions. Please use /v1/responses instead.","DataPlaneErrorCode":0,"DataPlaneErrorDetail":[]} RequestId: a6f6c6f3-85fe-4659-bb0e-103f190e9ae7. Troubleshooting guide: https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/foundry-iq-connect?tabs=foundry%2Cpython#troubleshooting

Result: failure - as can be seen in the below screenshot: Second turn - With KB usage - Failure

The error indicates the agent is using the older /v1/chat/completions api, instead of the newer /v1/responses.

Searching the web, this issue started since GPT 5.4, so I tried using GPT 5.2 and that indeed works - but I want to be able to use the latest models.Is this something I could configure to fix it?

Foundry Agent Service
Foundry Agent Service

A fully managed platform in Microsoft Foundry for hosting, scaling, and securing AI agents built with any supported framework or model

0 comments
Answer accepted by question author
Gowtham CP 8,120 Reputation points Volunteer Moderator
2026-08-04T06:26:34.7466667+00:00

Hi @Ofir.D ,

Thanks for reaching out.

Currently, there is no option to configure Azure AI Foundry Agents to use the /v1/responses API. The managed Agent Service handles the underlying API calls and currently uses /v1/chat/completions for tool invocations such as Knowledge Base retrieval.

This is why your first prompt succeeds (no tool call), while the second fails when the KB tool is invoked with newer reasoning models like GPT-5.4 and GPT-5.6.

As a workaround, you can either use GPT-5.2/GPT-4o, or call the Responses API directly and manage the orchestration yourself.

References:

https://learn.microsoft.com/azure/ai-foundry/openai/how-to/responses

https://learn.microsoft.com/azure/ai-foundry/agents/how-to/foundry-iq-connect

I hope this helps. If it answers your question, please consider accepting it to help others facing the same issue.

Was this answer helpful?

1 person found this answer helpful.
0 comments

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.