Foundry Agent Token consuption

Thanks for sharing the detailed context and code snippet!

You’re absolutely right by default, Azure AI Foundry agents currently resend the entire thread context (including user messages, model outputs, and intermediate tool responses) with each new run. This is by design, as Foundry’s thread-based memory mechanism maintains full conversational continuity for consistent model behavior.

That said, if you’d like to optimize or control memory usage, here are a few approaches you can consider:

1. Manage conversation memory manually You can implement a custom memory handler by storing only selected parts of the conversation (for example, just the last N turns) before calling messages.create(). At the moment, Foundry doesn’t expose a public API to replace the internal memory manager directly memory threads are handled automatically.

2. Use a new thread for short or stateless interactions If you don’t need the full history every time, create a fresh thread_id per query. This avoids sending long histories to the model, which helps reduce token consumption.

3. Use lightweight summarization Some developers maintain a condensed context by periodically summarizing previous interactions and sending that summary as a system message this helps maintain continuity while keeping the payload smaller.

Currently, Foundry doesn’t yet provide an official configuration to switch or limit the internal memory manager, but this capability is under consideration as part of SDK enhancements.

To help us understand your setup better and guide you further, could you please share a bit more info?

What does your current agent configuration look like (especially how messages are being handled)?

Are you able to check how many previous turns are currently retained in the conversation history?

Which SDK version of azure-ai-agents and azure-ai-projects are you using (just confirming your pip list)?

Once we have that, we can suggest a more tailored approach for managing memory efficiently.

Hope this helps!

thanks for responding.

let me clarify;

the only way to get a better manage memory is to NOT use thread in create msg, (or creating a new one) and sending the context as a msg....

and there is no way of "inject" a memory manager to your threads.

The problem with this is;

my agent use codeinterpreter per run (generally) and CI cost are associated per thread, so im forced to pay $0.033/session for every run
https://azure.microsoft.com/en-us/pricing/details/cognitive-services/openai-service/
this make my agent probably not worth it.

am i loosing something? is someone doing this a better way?

ab my config:
nowadays im saving as pairs (question, answer) in my own db.
so im able to tell how many turns there are , and summarised them. But the main reason im using azure sdk is to be able to use code interpreter in a safe enviroment but it appears CI use is attatch to threads ("costly speaking#)

as i said earlier im using:
azure-ai-agents 1.1.0

azure-ai-projects 1.0.0

azure-core 1.35.1

Hi alejandro,
Could you try using this code to get the agent token count for the thread:

import os
import pandas as pd
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

PROJECT_ENDPOINT = "<ENDPOINT_URL>"
AGENT_ID = "AGENT_ID"
OUTPUT_CSV = "azure_agent_full_report.csv"


credential = DefaultAzureCredential()
project_client = AIProjectClient(endpoint=PROJECT_ENDPOINT, credential=credential)


def flatten_attributes(obj, parent_key=""):
   
    result = {}
    for attr in dir(obj):
        if attr.startswith("_"):
            continue
        try:
            value = getattr(obj, attr)
        except Exception:
            value = None
        key = f"{parent_key}.{attr}" if parent_key else attr
        if hasattr(value, "__dict__") and not isinstance(value, (str, int, float, bool, list, dict)):
            result.update(flatten_attributes(value, parent_key=key))
        else:
            result[key] = value
    return result

print(f"Fetching threads and runs for agent: {AGENT_ID}")

all_data = []

threads = project_client.agents.threads.list()

for thread in threads:
    thread_id = thread.id
    print(f"\nProcessing thread: {thread_id}")

    runs = project_client.agents.runs.list(thread_id=thread_id)

    for run in runs:
        full_run = project_client.agents.runs.get(thread_id=thread_id, run_id=run.id)

        run_data = flatten_attributes(full_run)
        run_data["agent_id"] = AGENT_ID
        run_data["thread_id"] = thread_id
        all_data.append(run_data)


if all_data:
    df = pd.DataFrame(all_data)
    print("\n=== Sample of flattened run attributes ===")
    print(df.head())
    df.to_csv(OUTPUT_CSV, index=False)
    print(f"\n Full agent report saved to: {OUTPUT_CSV}")
else:
    print("No runs found for this agent.")

Hope this helps!