Azure Foundry - using file search tool in hosted agent

Question

Azure Foundry - using file search tool in hosted agent

Souciance Eqdam Rashti 0

Hello,

I have deployed a hosted agent to Azure Foundry.

It references a toolbox.

The toolbox has the filesearch tool

I have uploaded a simple txt file to the filesearch tool and it created a new index with status completed.

However, the agent cannot access the data in teh index and when trying to fetch it manually I simply get:

"result": {"_meta":{"tool_configuration":{"type":"file_search","name":"index_epic_whistle_ht0mz00ttd","description":"","vector_store_ids":["vs_aSG7XqAQOxrEUiq6OCW7qDZq"]}},"content":[{"type":"text","text":"ServerError[500, user=An error occurred while processing your request. You can retry your request, or contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 if the error persists. Please include the request ID 77b55650-a284-4285-9c45-02755372c66c in your message.]"}],"isError":true}

It is impossible to troubleshoot further. I have uploaded other files as well but same issue. And here is my agent code:

using Azure.AI.Projects;
using Azure.Identity;
using DotNetEnv;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Foundry.Hosting;

// Load environment variables from a .env file if present (for local development).
Env.NoClobber().TraversePath().Load();

if (string.IsNullOrEmpty(Environment.GetEnvironmentVariable("APPLICATIONINSIGHTS_CONNECTION_STRING")))
{
    Console.Error.WriteLine(
        "[WARNING] APPLICATIONINSIGHTS_CONNECTION_STRING not set — traces will not be sent " +
        "to Application Insights. Set it to enable local telemetry. " +
        "(This variable is auto-injected in hosted Foundry containers — do not declare it in agent.manifest.yaml.)");
}

var projectEndpoint = new Uri(Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT")
    ?? throw new InvalidOperationException("FOUNDRY_PROJECT_ENDPOINT environment variable is not set."));

var deployment = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME")
    ?? throw new InvalidOperationException("AZURE_AI_MODEL_DEPLOYMENT_NAME environment variable is not set.");

// Create an AIAgent backed by a Foundry model.
// The agent framework manages the LLM call, conversation sessions, and response lifecycle.
var credential = new DefaultAzureCredential();//new AzureCliCredential();

var token = credential.GetToken(
    new Azure.Core.TokenRequestContext(
        ["https://ai.azure.com/.default"]));

//Console.WriteLine($"CLI token acquired: {token.ExpiresOn}");

AIAgent agent = new AIProjectClient(projectEndpoint, credential)
    .AsAIAgent(
        model: deployment,
        instructions: """
    You are an Azure pricing assistant.

    You have access to a file_search tool containing internal Azure pricing data.

    For every pricing question:
    - Always call file_search before answering.
    - Search for the requested service name.
    - Also search for the requested Azure region if one is specified.
    - Use only information returned by file_search.
    - Never invent pricing.

    If file_search returns matching information, provide the answer using that information.

    If file_search returns no matching information, say:
    "I could not find this service or pricing information in the indexed pricing data."

    Keep responses concise.
    """,
        name: "hostedagent",
        description: "A minimal Hello World agent using the Agent Framework");

// AgentHost.CreateBuilder() auto-configures:
//   - Kestrel on port 8088 (or the PORT environment variable)
//   - GET /readiness health probe
//   - OpenTelemetry traces and metrics
//   - x-platform-server response header
var builder = AgentHost.CreateBuilder(args);
builder.Services.AddFoundryResponses(agent);
builder.Services.AddFoundryToolboxes(credential, Environment.GetEnvironmentVariable("FOUNDRY_TOOLBOX_NAME"));
builder.RegisterProtocol("responses", endpoints => endpoints.MapFoundryResponses());

var app = builder.Build();
app.Run();

0 comments

3 answers

Your answer

Answer 1

Hello Souciance Eqdam Rashti,

Greetings! Thanks for raising this question in the Q&A forum.

The ServerError[500] coming back from the toolbox's file_search tool call is a backend error on the file search execution path, not a problem with your C# hosting code itself. The vector store showing status "completed" only confirms that ingestion finished, it does not confirm that the runtime identity used by your hosted container can actually query that vector store and its underlying storage. Since the hosted C# integration relies on AddFoundryToolboxes(credential, ...) to call the toolbox MCP endpoint directly with DefaultAzureCredential, a 500 at query time most commonly comes from one of the following:

Missing role assignment on the identity used inside the hosted container. The identity that created the vector store (typically your own signed-in az login identity when testing locally, or a different identity if this was done through the Foundry portal) is not necessarily the same identity your hosted agent uses at runtime. Confirm the identity resolved by DefaultAzureCredential in your hosted container has both of these roles on the project:

Storage Blob Data Contributor on the project's storage account
- Foundry User on the Foundry project (previously named Azure AI User)
If you are using a standard agent setup with your own connected Azure AI Search resource, that identity also needs read access on the connected AI Search resource. Role assignment changes can take several minutes to propagate, so retry after waiting a bit. Region or preview availability. Hosted agents combined with toolboxes and the file_search tool are a newer capability. Confirm your Foundry project's region is listed as supporting this combination:

https://learn.microsoft.com/en-us/azure/foundry/reference/region-support

Isolate whether the issue is toolbox-specific or vector-store-specific. Call the vector store directly through the REST API using the same bearer token flow, bypassing the toolbox and hosted agent entirely:

curl --request GET \
  --url $FOUNDRY_PROJECT_ENDPOINT/openai/v1/vector_stores/vs_aSG7XqAQOxrEUiq6OCW7qDZq \
  -H "Authorization: Bearer $AGENT_TOKEN"

If this also intermittently fails or returns errors, the problem sits with the vector store or its underlying search resource rather than with the hosted agent or toolbox wiring.

Confirm the toolbox connection setup matches the documented pattern. For hosted C# agents, AddFoundryToolboxes authenticates the toolbox MCP call directly with the credential you pass in, it does not go through a "remote-tool" project connection the way the REST and TypeScript flows do. Double check you are following the maintained sample rather than mixing the REST-based connection pattern with the hosted C# pattern:

https://github.com/microsoft/agent-framework/tree/main/dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox

If the RBAC and region checks come back clean and the direct REST call to the vector store also fails or returns 500 intermittently, this points to a platform-side issue on the file search backend that cannot be diagnosed further from outside. In that case, open a Technical support request under Microsoft Foundry > Agent Service, and include:

Request ID: 77b55650-a284-4285-9c45-02755372c66c
Vector store ID: vs_aSG7XqAQOxrEUiq6OCW7qDZq
Toolbox name: index_epic_whistle_ht0mz00ttd

If this answer helps you kindly accept the answer which will help others who have similar questions.

Best Regards,

Jerald Felix.

Souciance Eqdam Rashti 0 Reputation points

2026-08-26T16:48:37.2233333+00:00

That's a lot of troubleshooting :) I am pretty sure I have given it the correct roles but will confirm once again. Then I will check the endpoints for the vector store and see what I get back. Will get back to you.
Souciance Eqdam Rashti 0 Reputation points

2026-08-26T16:54:39.3333333+00:00

@Jerald Felix I queried the index and this is the response I got:

{ "id": "vs_8D2M1HKOFLQEVB8K8WYJyN3G", "object": "vector_store", "name": "index_olden_diamond_qznrgbv023", "status": "completed", "usage_bytes": 1140, "created_at": 1787761871, "file_counts": { "in_progress": 0, "completed": 1, "failed": 0, "cancelled": 0, "total": 1 }, "metadata": {}, "expires_after": null, "expires_at": null, "last_active_at": 1787761871 }
Souciance Eqdam Rashti 0 Reputation points

2026-08-26T16:56:28.47+00:00

@Jerald Felix I have given the agent identity Contributor access to all the resources, so pretty much as higher that the specific roles you mentioned. Still getting 500 error when querying for the actual data in the index even though the status says completed.

Answer 2

Hello @Souciance Eqdam Rashti

Thanks for the detailed payloads. Two corrections to what you've already been asked to do, then a test sequence you can complete without waiting on me.

First, the RBAC guidance you were given was wrong :- File search doesn't need broad Contributor. It needs Storage Blob Data Contributor on the project's storage account and Foundry User on the Foundry project. Escalating to Contributor wouldn't have fixed a scope problem, so that result told us nothing. Sorry for the detour.

Second, the store you validated isn't the store that's failing :-
Your toolbox index_epic_whistle_ht0mz00ttd pins vs_aSG7XqAQOxrEUiq6OCW7qDZq. The GET you ran successfully was vs_8D2M1HKOFLQEVB8K8WYJyN3G (index_olden_diamond_qznrgbv023). Different objects the failing one is still untested.

That matters because file_search inside a toolbox requires the file and vector store to be created at the resource-level endpoint with the x-aml-project-id header, using the project GUID from properties.amlWorkspace.internalId. A store created at project scope can return 200 on GET and still not resolve on the toolbox query path — which matches a healthy store, a 500 with no sub-code, and RBAC changes making no difference.

Step 1:- probe the failing store at both scopes. Run both; the pair is what's diagnostic, not either alone.

AGENT_TOKEN=$(az account get-access-token --scope https://ai.azure.com/.default --query accessToken -o tsv)
VS=vs_aSG7XqAQOxrEUiq6OCW7qDZq

# project scope
curl -s -w "\nproject:%{http_code}\n" \
  "$FOUNDRY_PROJECT_ENDPOINT/openai/v1/vector_stores/$VS" \
  -H "Authorization: Bearer $AGENT_TOKEN"

# account/resource scope (the path the toolbox uses)
curl -s -w "\naccount:%{http_code}\n" \
  "$ACCOUNT_ENDPOINT/openai/v1/vector_stores/$VS" \
  -H "Authorization: Bearer $AGENT_TOKEN" \
  -H "x-aml-project-id: $PROJECT_GUID"

Read it as:

project 200 / account 404 → scope mismatch. Go to Step 3.
both 404 → the ID is stale; the index was recreated and the toolbox still points at the old object. Step 3.
account 200 → check status is completed and file_counts.failed is 0. If files failed ingestion, re-upload; that alone can produce the 500. If it's clean, config is sound → Step 4, then escalation.

Step 2:- read the toolbox's live pinned config. azd ai toolbox show <toolbox-name> --output json. vector_store_ids are immutable for a given toolbox version, so a recreated index always leaves the old version pointing at a dead ID. Confirm the default_version config carries the ID that returned 200 above.

Step 3 :- recreate at account scope and republish. POST {account_endpoint}/openai/v1/files with purpose=assistants and x-aml-project-id: {project-guid}, then POST {account_endpoint}/openai/v1/vector_stores with the returned file ID and the same header. Pin the new ID into a new toolbox version and promote it to default_version.

One catch worth knowing: your agent must be on the unversioned consumer endpoint (https://<account>.services.ai.azure.com/api/projects/<project>/toolboxes/<toolbox-name>/mcp?api-version=v1) for the promotion to take effect. If it's pinned to a versioned developer endpoint, the promotion is silently ignored — a documented symptom. Verify with a tools/call and confirm isError: false with chunk content returned.

Step 4 :- update packages and redeploy. Upgrade Microsoft.Agents.AI.Foundry.Hosting and the agentserver/responses packages, redeploy, and confirm /readiness is healthy — it reports unhealthy specifically when the host can't enumerate toolbox tools, which distinguishes a discovery failure from an execution failure. Then re-run the failing prompt.

On expectations: Step 3 is a documented configuration correction, and I'd expect it to resolve this if Step 1 shows a scope or staleness fault. Step 4 is a defensive SDK refresh. If Step 1 returns account-scope 200 with a completed store and Steps 3–4 change nothing, the fault is in the server-side file-search execution path that needs backend tracing, not more work from you, and I'll escalate with your payloads attached rather than asking you to keep testing.

References

https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/tools/file-search?pivots=python
https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/tools/use-toolbox-hosted-agent?pivots=python
https://learn.microsoft.com/en-us/azure/foundry/agents/concepts/toolbox-overview

Thanks,
Manish.

Answer 3

Souciance Eqdam Rashti 0

This issue has been resolved. The biggest problem here is that errors happening in the background, for example between Foundry UI and Foundry backend to the connected services like AI search are not shown.

In this particular case, I was using AI Search developer tier which is in preview. This tier is not compatible with AI foundry due to paging issues. However it took an incredible amount of troubleshooting to understand this.

After switching to a basic tier of AI Search, everything works as expected.

0 comments