Azure AI Foundry Agent fails to process uploaded images and logs IMDS (169.254.169.254) connection error

Question

Azure AI Foundry Agent fails to process uploaded images and logs IMDS (169.254.169.254) connection error

Mohamed, Rihan 60

Hi Team,

I'm working with an Azure AI Foundry agent that uses Azure AI Search as its knowledge source.

The agent works correctly for text-based queries, but when I upload an image and ask questions about its contents, the agent is unable to extract or process any information from the image.

What I've tried

Tested with different deployment models (including GPT-5).
Tested both Prompt Flow and Hosted Agents.
Updated the agent instructions.
Verified that the image is uploaded successfully.
Reviewed the Application Insights logs.

Error observed in Application Insights

Questions

Is this IMDS (169.254.169.254) connection error expected in Azure AI Foundry, or does it indicate a configuration issue?
Could this error prevent image processing or OCR from working?
Does Azure AI Foundry require Managed Identity or Azure Instance Metadata Service (IMDS) access for image ingestion?
Are there any additional configurations or permissions I should verify for image extraction to work correctly?

Any guidance would be greatly appreciated.

Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-07-02T20:51:29.2233333+00:00

Hello @Mohamed, Rihan

Did you get any chance to review the response.

Thankyou!
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-07-05T11:51:41.6966667+00:00

Hello @Mohamed, Rihan

Please 'Upvote'(Thumbs-up) and 'Accept' as answer if the reply was helpful. This will be benefitting other community members who face the same issue.

Thankyou!
Mohamed, Rihan 60 Reputation points

2026-07-06T06:31:43.04+00:00
Hello Anshika,

Thank you — the App Insights suggestion was helpful and we resolved several image-related bugs using it.

However, clipboard-pasted screenshots (Ctrl+V in Teams) are still not working. File uploads and text queries work fine.

What we tried: We read request.activity.attachments at the SDK layer, fetch the CDN URL using DefaultAzureCredential (scopes: Teams API → Graph API fallback), and inject the image as inline base64 before the model call.

Still need clarity on:

Does request.activity.attachments actually contain clipboard paste images in a Foundry-hosted agent?

Is https://api.spaces.skype.com/.default the correct scope for fetching Teams media CDN URLs via Workload Identity?

Are any specific permissions needed on the agent's managed identity for Teams media access?

Thank you!
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-07-08T19:03:40.45+00:00
Hello Mohamed, Rihan,

Thank you for the update, and I'm glad Application Insights helped identify and resolve several of the image-processing issues.

Regarding the remaining clipboard-pasted screenshot scenario, based on the current behavior and available documentation, there are a few important points to consider:

Clipboard-pasted images in Teams are not always surfaced the same way as standard file uploads. While uploaded files are typically exposed through activity.attachments, Microsoft has not currently documented that all Ctrl+V pasted screenshots are reliably converted into attachment objects that are forwarded to Azure AI Foundry hosted agents. Similar reports indicate that the image may be lost before it reaches the agent runtime, resulting in only the text portion being delivered.

If request.activity.attachments is empty when a screenshot is pasted, the image is likely being dropped upstream rather than by your image-processing logic. In that case, your current approach of downloading the attachment and injecting it as input_image would never be triggered because the attachment metadata is not present.

Regarding the api.spaces.skype.com/.default scope and managed identity permissions, I could not find official Microsoft documentation that explicitly confirms this as the supported scope for retrieving Teams clipboard-media CDN content from a Foundry-hosted agent. Similarly, I found no documentation describing specific managed identity roles required for direct Teams media CDN access in this scenario.

At this point, the key diagnostic step would be to log the complete incoming request.activity payload for both:

A successful file upload

A Ctrl+V pasted screenshot

If the pasted screenshot does not appear in attachments (or any other image-related field), that would confirm the image is not currently being forwarded to the Foundry-hosted agent channel.

I hope this helps clarify the current behavior. Please let us know what you observe in the raw request.activity payload, and we'll be happy to investigate further.

Once this helps resolve your issue, please consider upvoting the helpful replies and marking the accepted answer so it benefits other community members facing the same problem.

Thankyou!
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-07-09T15:39:20.26+00:00

Hello @Mohamed, Rihan

Please let me know if the issue persists after these checks. If you have any remaining questions or need additional details, I’ll be glad to provide further clarification or guidance.

If the above steps resolve your issue, please mark this answer as Accepted so others can easily find the solution.

Thankyou!
Mohamed, Rihan 60 Reputation points

2026-07-09T16:05:11.8933333+00:00
Thank you for the clarification.

If clipboard-pasted images are currently not forwarded to Azure AI Foundry hosted agents, are there any recommended alternative approaches to support this scenario?

For example:

Is there any Teams SDK or Bot Framework capability that can capture pasted images before they reach the hosted agent?

Is there a different integration pattern (custom Teams app, Microsoft 365 Agent Toolkit, etc.) that supports clipboard-pasted images?

Is this functionality planned for Azure AI Foundry hosted agents in a future release or on the product roadmap?

Our goal is to provide a seamless user experience where users can simply paste screenshots (Ctrl+V) into Teams and have the agent analyze them without requiring manual file uploads. We'd appreciate any guidance or recommendations.

This asks three important questions:

Is there another implementation approach today?

Is there another Microsoft technology that supports it?

Is this feature planned for the future?

Are there any workaround or whether it's a current platform limitation.?
Mohamed, Rihan 60 Reputation points

2026-07-10T07:25:45.05+00:00

@Anshika Varshney Can you please respond to the above message, and I will be able to close this conversation.
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-07-12T19:39:30.6033333+00:00

Hello @Mohamed, Rihan

Thank you for the detailed follow-up questions.

Based on the current documentation and publicly available information, I could not find any official confirmation that Azure AI Foundry hosted agents currently support automatic forwarding of clipboard-pasted images in the same way as standard file uploads. From the behavior you've observed, this appears to be a current platform limitation rather than a configuration issue.

Regarding alternative approaches, a custom implementation using Teams app capabilities or Bot Framework may provide greater control over incoming activities and attachments before they are sent to your AI workflow. However, I could not find official documentation that specifically confirms support for capturing clipboard-pasted screenshots and forwarding them to Azure AI Foundry hosted agents in this scenario.

At present, the most reliable, supported approach appears to be using standard file uploads, as those attachments are exposed to the agent workflow whereas clipboard-pasted images may not be surfaced through the incoming activity payload.

If Microsoft publishes future updates or roadmap information for this capability, those details would be communicated through the official Azure AI Foundry documentation and release announcements.

Thank you!
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-07-13T19:16:18.18+00:00

Hello @Mohamed, Rihan

Please let me know if the issue persists after these checks. If you have any remaining questions or need additional details, I’ll be glad to provide further clarification or guidance.

If the above steps resolve your issue, please mark this answer as Accepted so others can easily find the solution.

Thankyou!

Answer accepted by question author

Anshika Varshney 15,625 Microsoft External Staff Moderator

Hello @Mohamed, Rihan

Thank you for sharing the details.

Based on your description, the fact that text-based queries work successfully through Azure AI Search while image-based queries fail suggests that the issue may be related to image understanding or ingestion rather than the Azure AI Search knowledge source itself. The IMDS endpoint (169.254.169.254) is commonly associated with Managed Identity token retrieval, and seeing connection attempts in logs does not necessarily indicate the root cause of the image-processing issue by itself.

A few checks that may help narrow down the problem:

Verify that the model being used by the agent supports image/multimodal input and that the image is actually being passed to the model during execution.
Review Agent traces and Application Insights logs to determine whether the failure occurs during image upload, image retrieval, tool invocation, or model inference.
If Managed Identity is being used, validate that the required permissions are correctly assigned and that authentication-related errors are not appearing elsewhere in the traces. Similar agent scenarios involving Azure AI Search have been impacted by authentication and managed identity configuration issues.
Test with a simple image question using a known multimodal model and compare the traces against a successful text-only request to identify where the execution path differs.

Could you also share the exact error message from the Application Insights logs (with sensitive information removed) and confirm which model deployment is being used for image processing?

That would help determine whether this is an image-processing limitation, a permissions issue, or an authentication-related failure.

I Hope this helps. Do let me know if you have any further queries.

Thankyou!

0 comments

1 additional answer

Your answer

Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-07-02T20:51:29.2233333+00:00

Hello @Mohamed, Rihan

Did you get any chance to review the response.

Thankyou!
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-07-05T11:51:41.6966667+00:00

Hello @Mohamed, Rihan

Please 'Upvote'(Thumbs-up) and 'Accept' as answer if the reply was helpful. This will be benefitting other community members who face the same issue.

Thankyou!
Mohamed, Rihan 60 Reputation points

2026-07-06T06:31:43.04+00:00

Hello Anshika,

Thank you — the App Insights suggestion was helpful and we resolved several image-related bugs using it.

However, clipboard-pasted screenshots (Ctrl+V in Teams) are still not working. File uploads and text queries work fine.

What we tried: We read request.activity.attachments at the SDK layer, fetch the CDN URL using DefaultAzureCredential (scopes: Teams API → Graph API fallback), and inject the image as inline base64 before the model call.

Still need clarity on:

Does request.activity.attachments actually contain clipboard paste images in a Foundry-hosted agent?

Is https://api.spaces.skype.com/.default the correct scope for fetching Teams media CDN URLs via Workload Identity?

Are any specific permissions needed on the agent's managed identity for Teams media access?

Thank you!
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-07-09T15:39:20.26+00:00

Hello @Mohamed, Rihan

Please let me know if the issue persists after these checks. If you have any remaining questions or need additional details, I’ll be glad to provide further clarification or guidance.

If the above steps resolve your issue, please mark this answer as Accepted so others can easily find the solution.

Thankyou!
Mohamed, Rihan 60 Reputation points

2026-07-09T16:05:11.8933333+00:00

Thank you for the clarification.

If clipboard-pasted images are currently not forwarded to Azure AI Foundry hosted agents, are there any recommended alternative approaches to support this scenario?

For example:

Is there any Teams SDK or Bot Framework capability that can capture pasted images before they reach the hosted agent?

Is there a different integration pattern (custom Teams app, Microsoft 365 Agent Toolkit, etc.) that supports clipboard-pasted images?

Is this functionality planned for Azure AI Foundry hosted agents in a future release or on the product roadmap?

Our goal is to provide a seamless user experience where users can simply paste screenshots (Ctrl+V) into Teams and have the agent analyze them without requiring manual file uploads. We'd appreciate any guidance or recommendations.

This asks three important questions:

Is there another implementation approach today?

Is there another Microsoft technology that supports it?

Is this feature planned for the future?

Are there any workaround or whether it's a current platform limitation.?
Mohamed, Rihan 60 Reputation points

2026-07-10T07:25:45.05+00:00

@Anshika Varshney Can you please respond to the above message, and I will be able to close this conversation.
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-07-12T19:39:30.6033333+00:00

Hello @Mohamed, Rihan

Thank you for the detailed follow-up questions.

Based on the current documentation and publicly available information, I could not find any official confirmation that Azure AI Foundry hosted agents currently support automatic forwarding of clipboard-pasted images in the same way as standard file uploads. From the behavior you've observed, this appears to be a current platform limitation rather than a configuration issue.

Regarding alternative approaches, a custom implementation using Teams app capabilities or Bot Framework may provide greater control over incoming activities and attachments before they are sent to your AI workflow. However, I could not find official documentation that specifically confirms support for capturing clipboard-pasted screenshots and forwarding them to Azure AI Foundry hosted agents in this scenario.

At present, the most reliable, supported approach appears to be using standard file uploads, as those attachments are exposed to the agent workflow whereas clipboard-pasted images may not be surfaced through the incoming activity payload.

If Microsoft publishes future updates or roadmap information for this capability, those details would be communicated through the official Azure AI Foundry documentation and release announcements.

Thank you!
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-07-13T19:16:18.18+00:00

Hello @Mohamed, Rihan

Please let me know if the issue persists after these checks. If you have any remaining questions or need additional details, I’ll be glad to provide further clarification or guidance.

If the above steps resolve your issue, please mark this answer as Accepted so others can easily find the solution.

Thankyou!

Answer 1

Hello Mohamed, Rihan,

Greetings! Thanks for raising this question in the Q&A forum.

The IMDS (169.254.169.254) connection error is almost always related to Microsoft Entra ID token acquisition through Managed Identity, not to the image content itself. When your agent's underlying compute or connection tries to authenticate using DefaultAzureCredential (or an equivalent SDK-managed credential chain), it attempts several credential sources in order, and ManagedIdentityCredential is one of the first it tries. If the environment cannot reach IMDS, either because no managed identity is enabled or the network path to the metadata endpoint is blocked, that specific credential attempt fails and logs the connection error you're seeing. Whether this is harmless noise or a real blocker depends entirely on which downstream call is relying on that token.

I noticed the error block in your post came through empty, so I can't see the exact exception text or correlation ID. Based on the symptom pattern (text queries succeed, image queries fail, IMDS shows up in App Insights), here is the most likely cause and how to confirm it.

Check which resource actually processes the image. In Foundry Agent Service, image understanding for an uploaded file is typically routed either through the multimodal capability of the deployed chat model (for example GPT-5 with vision input) or through a separate connected resource such as Azure AI Vision or Content Understanding for OCR. Go to Foundry portal > Your project > Connected resources and confirm which resource is wired up for image/document processing.

Confirm the authentication mode on that connection. Under Connected resources > [resource name] > Authentication, check if it's set to API key or Microsoft Entra ID (managed identity). Only the Entra ID/managed identity path will attempt to reach IMDS. If your AI Search connection uses an API key but the vision/OCR resource is configured for Entra ID, that mismatch explains why text queries (Search-based) succeed while image queries (vision/OCR-based) fail.

Verify managed identity is actually enabled and has role assignments. Go to Foundry project > Settings > Identity and confirm a system-assigned (or user-assigned) managed identity is turned on. Then in the target resource (Azure AI Services, Computer Vision, or Content Understanding), go to Access control (IAM) and confirm the project's managed identity has at minimum the Cognitive Services User role, or Azure AI User if you're on the newer Foundry RBAC model. A missing role assignment will not stop the IMDS call from happening but will cause the subsequent authorized call to fail with a 401/403 right after.

Check network restrictions. If your Foundry project or the target Cognitive Services resource sits behind a private endpoint or has public network access disabled, and the hosted agent's compute doesn't have a route to IMDS or to the target resource's private endpoint, the token request or the downstream call will fail. Review Networking on both the Foundry resource and any connected Cognitive Services resource.

Confirm the model deployment supports vision input in your region. Even with authentication resolved, image processing will silently fail if the specific GPT-5 deployment or SKU you're using doesn't have multimodal input enabled for your subscription and region. Check Foundry portal > Model catalog > Your deployment > Capabilities to confirm vision/image input is listed.

Capture the full exception. In Application Insights, run this query to pull the complete stack trace and any CorrelationId tied to the failed image request, since that will confirm exactly which SDK call is generating the IMDS attempt.

traces
| where timestamp > ago(24h)
| where message has "169.254.169.254" or message has "IMDS"
| project timestamp, message, severityLevel, operation_Id, cloud_RoleName
| order by timestamp desc

To directly answer your four questions: the IMDS error itself is expected behavior when any part of the pipeline uses managed identity based authentication, so it is not inherently a bug, but it does indicate that a token acquisition attempt failed somewhere in that request path. Yes, it can block image processing or OCR if the specific resource handling that step depends on the failed token. Azure AI Foundry does require managed identity or IMDS access only for the connections you've explicitly configured to use Entra ID auth, not universally, so this is a configuration choice rather than a hard platform requirement. And the configuration to verify is the connection authentication type, the RBAC role assignment on the target resource, and network reachability, as outlined in steps 2 through 4 above.

If you can share the full error text and the CorrelationId from Application Insights, I can help pinpoint the exact failing call. If after these checks the issue persists, please open a support ticket through the Azure portal with the CorrelationId attached, since a persistent IMDS timeout inside a fully managed hosted agent environment (not a customer-managed VM) would point to a platform-side networking issue that needs the Foundry Agent Service team to investigate directly.

If this answer helps you kindly accept the answer which will help others who have similar questions.

Best Regards,

Jerald Felix.