Azure AI Foundry evaluation run fails with 403 UnauthorizedUserAction when account networking is restricted to Selected Networks

Question

Azure AI Foundry evaluation run fails with 403 UnauthorizedUserAction when account networking is restricted to Selected Networks

Marian Cap 25

Hello,

We are facing an issue with Azure AI Foundry evaluations when the Foundry / Azure AI Services account is configured with network restrictions.

Environment

Resource type: Microsoft.CognitiveServices/accounts
Kind: AIServices
SKU: S0
Region: eastus2
Foundry project: proj-default
Public network access: Enabled
Network mode: Selected Networks and Private Endpoints
networkAcls.defaultAction: Deny
Bypass: AzureServices
IP allowlist configured
No private endpoints
No virtual network rules

Scenario

We are trying to create and submit an Azure AI Foundry evaluation from the Foundry portal.

The evaluation setup is:

Target: Agent
Scope: Individual turns
Data source: Existing traces
Dataset: Application Insights traces
Field mapping: default / unchanged
Criteria: default / unchanged

When submitting the evaluation run, the portal fails with:

Request failed with status code 400: The action cannot be finished with reason Response status code does not indicate success: 403 (Forbidden).

The failing request is:

POST https://<foundry-account>.cognitiveservices.azure.com/api/projects/proj-default/openai/v1/evals/<eval_id>/runs

The response body contains:

{
  "error": {
    "code": "UserError",
    "message": "The action cannnot be finished with reason Response status code does not indicate success: 403 (Forbidden).",
    "innerError": {
      "code": "UnauthorizedUserAction"
    }
  },
  "componentName": "raisvc",
  "statusCode": 400
}

RBAC already validated

We already granted the project managed identity the following roles:

Foundry User on the Foundry / Azure AI Services account
Log Analytics Reader on the connected Application Insights resource
Log Analytics Reader on the linked Log Analytics Workspace

After RBAC propagation, the issue still persisted.

Network test performed

We performed a controlled test to confirm whether the issue is caused by networking restrictions.

Initial state:

publicNetworkAccess = Enabled
networkAcls.defaultAction = Deny
bypass = AzureServices

Temporary test:

Changed networkAcls.defaultAction from Deny to Allow

Result:

The evaluation Submit action worked successfully.

Rollback:

Changed networkAcls.defaultAction back to Deny

After rollback, the resource was back to the secured state and the IP allowlist was preserved.

Conclusion

The issue appears to be caused by the Foundry evaluation backend / raisvc not being able to complete the evaluation run when the Foundry account is restricted to Selected Networks.

The evaluation works only when networkAcls.defaultAction is temporarily changed to Allow.

Questions

Are Azure AI Foundry evaluations supported when the underlying Microsoft.CognitiveServices/accounts resource is configured with Selected Networks / networkAcls.defaultAction = Deny?
If this scenario is supported, what is the required network configuration for the Foundry evaluation backend / raisvc?
Is there a service tag, trusted service setting, managed private endpoint, or specific network rule that needs to be configured?
Is Allow Azure services on the trusted services list expected to allow Foundry evaluation runs? It is already enabled on our resource, but the evaluation still fails unless defaultAction is set to Allow.
What is the recommended production-safe configuration that allows Azure AI Foundry evaluations without setting the account to Allow all networks?

Thank you.

image (5)

Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-06-05T07:47:48.78+00:00

Hello @Marian Cap ,

Did you get any chance to review the response.

Thankyou!
Marian Cap 25 Reputation points

2026-06-05T12:55:07.7633333+00:00
Hello @Anshika Varshney ,

Thank you for confirming that this is a known limitation.

Regarding Option 1, Private Endpoint is not currently feasible for us directly because our Azure policy does not allow us to create Private DNS Zones. Private DNS is managed centrally by our platform/networking team.

Can you please confirm the exact Private DNS zones required for Azure AI Foundry evaluations for a Microsoft.CognitiveServices/accounts resource of kind AIServices? From the documentation, it looks like the relevant zones may include:

privatelink.cognitiveservices.azure.com

privatelink.openai.azure.com

privatelink.services.ai.azure.com

Also, can you confirm whether a Private Endpoint would actually allow the Microsoft-managed evaluation backend / raisvc to submit evaluation runs, or whether it only helps clients inside our VNet access the Foundry account?

Regarding Option 2, can you please confirm whether the AzureAIFoundry service tag can be used directly in the network ACL of a Microsoft.CognitiveServices/accounts resource? From what I can see, the Cognitive Services firewall allows IP rules and virtual network rules, while Azure service tags are typically used in NSGs, Azure Firewall, or route tables.

If service tags are not directly supported on the Foundry account firewall, can Microsoft provide the supported production-safe configuration for running Foundry evaluations with networkAcls.defaultAction = Deny?

Our goal is to avoid setting the Foundry account to Allow all networks in production.
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-06-09T19:58:42.95+00:00
Thank you for the detailed questions, these are very valid considerations for a production‑grade setup. I’ll address each point clearly.

Regarding Private DNS zones For a Microsoft.CognitiveServices/accounts resource (kind: AIServices), the commonly required Private DNS zone is:

privatelink.cognitiveservices.azure.com

The other zones you listed may be used depending on specific scenarios:

privatelink.openai.azure.com → relevant when using Azure OpenAI endpoints

privatelink.services.ai.azure.com → used by some newer AI platform endpoints (including Foundry-related experiences)

So, in practice:

The baseline requirement is privatelink.cognitiveservices.azure.com

Additional zones (openai / ai.azure.com) may be required depending on which features or endpoints your evaluation workflow actually calls

Your networking team would typically need to link these zones to the VNet hosting the private endpoint.

Regarding whether Private Endpoint enables evaluation runs This is the key point:

A Private Endpoint primarily enables client traffic from your VNet to the Cognitive Services account. However, Foundry evaluation runs are executed by Microsoft-managed backend services (for example, raisvc).

Because of this:

Private Endpoint alone does not guarantee that the Microsoft-managed evaluation backend can reach your resource

The backend service does not originate from your VNet, so it will still be subject to your network ACL restrictions

So yes, your interpretation is correct:

Private Endpoint helps your internal clients

It does not fully solve access for Microsoft-managed evaluation services

Regarding service tags / AzureAIFoundry You are also correct here:

Cognitive Services (Microsoft.CognitiveServices/accounts) firewall supports:

IP rules

Virtual network rules

It does not currently support Azure service tags directly in the resource firewall

Service tags like AzureAIFoundry are typically used in:

NSGs

Azure Firewall

Route tables

So, they cannot be used directly to allow traffic into the Cognitive Services account.

Recommended production-safe configuration Given current platform limitations, there is no fully clean allow-list mechanism for evaluation backend traffic when:

networkAcls.defaultAction = Deny

This is why the behavior is currently a known limitation.

Option A (most secure, but limited support):

Use Private Endpoint + required DNS zones

Be aware that evaluation runs may still fail due to backend access constraints

Option B (functional workaround):

Temporarily set: networkAcls.defaultAction = Allow

Run evaluation jobs

Revert back to Deny after completion

Option C (partial control):

Keep Deny

Add trusted IP ranges if guidance becomes available (not currently documented for evaluation backend)

Summary

Your understanding is accurate across all points

Private DNS zones: cognitiveservices is mandatory, others are scenario-dependent

Private Endpoint does not fully enable evaluation backend access

Service tags cannot be used directly on Cognitive Services firewall

There is currently no fully production safe “Allow selected networks” configuration for evaluation runs without some compromise

This is an active gap in the current design, and the recommendation is to plan evaluations with temporary relaxation of network restrictions where needed.

I Hope this helps. Do let me know if you have any further queries.

Thankyou!
Marian Cap 25 Reputation points

2026-06-10T07:21:28.3966667+00:00

Thanks for your answers!
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-06-10T11:57:58.86+00:00

Hello @Marian Cap

Thank you for sharing the update I appreciate you taking the time to confirm the resolution!

Since I’ve converted comment into an answer, could you please take a moment to mark it as Accepted?

This helps others in the community with the same question find the solution more easily.

Thankyou!

Answer accepted by question author

0 additional answers

Your answer

Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-06-05T07:47:48.78+00:00

Hello @Marian Cap ,

Did you get any chance to review the response.

Thankyou!
Marian Cap 25 Reputation points

2026-06-05T12:55:07.7633333+00:00

Hello @Anshika Varshney ,

Thank you for confirming that this is a known limitation.

Regarding Option 1, Private Endpoint is not currently feasible for us directly because our Azure policy does not allow us to create Private DNS Zones. Private DNS is managed centrally by our platform/networking team.

Can you please confirm the exact Private DNS zones required for Azure AI Foundry evaluations for a Microsoft.CognitiveServices/accounts resource of kind AIServices? From the documentation, it looks like the relevant zones may include:

privatelink.cognitiveservices.azure.com

privatelink.openai.azure.com

privatelink.services.ai.azure.com

Also, can you confirm whether a Private Endpoint would actually allow the Microsoft-managed evaluation backend / raisvc to submit evaluation runs, or whether it only helps clients inside our VNet access the Foundry account?

Regarding Option 2, can you please confirm whether the AzureAIFoundry service tag can be used directly in the network ACL of a Microsoft.CognitiveServices/accounts resource? From what I can see, the Cognitive Services firewall allows IP rules and virtual network rules, while Azure service tags are typically used in NSGs, Azure Firewall, or route tables.

If service tags are not directly supported on the Foundry account firewall, can Microsoft provide the supported production-safe configuration for running Foundry evaluations with networkAcls.defaultAction = Deny?

Our goal is to avoid setting the Foundry account to Allow all networks in production.
Marian Cap 25 Reputation points

2026-06-10T07:21:28.3966667+00:00

Thanks for your answers!
Anshika Varshney 15,625 Reputation points Microsoft External Staff Moderator

2026-06-10T11:57:58.86+00:00

Hello @Marian Cap

Thank you for sharing the update I appreciate you taking the time to confirm the resolution!

Since I’ve converted comment into an answer, could you please take a moment to mark it as Accepted?

This helps others in the community with the same question find the solution more easily.

Thankyou!

Answer 1

Hello @Marian Cap ,

This is a known limitation with Azure AI Foundry evaluations when network restrictions are configured. The evaluation backend service (raisvc) needs to reach your Cognitive Services account, but it's not being recognized as part of the trusted Azure services even when bypass = AzureServices is set.

Key findings from your testing:

Evaluation works when networkAcls.defaultAction = Allow
Evaluation fails when networkAcls.defaultAction = Deny (even with bypass = AzureServices)
RBAC roles are correctly configured (Foundry User, Log Analytics Reader)

Solutions:

Option 1: Use Private Endpoint (Production-safe) Configure a private endpoint for your Foundry account and ensure the evaluation backend can connect through it. Follow the configure private link documentation.

Option 2: Add Specific Service Tags The evaluation backend may require specific Azure service tags. Check if adding the AzureAIFoundry service tag (if available) to your IP allowlist resolves the issue.

Option 3: Temporary Allow with Monitoring If you must keep defaultAction = Deny, consider:

Keeping bypass = AzureServices
Adding your evaluation backend's source IPs to the allowlist (monitor Azure Firewall/NSG flow logs to identify them per this guidance)

I Hope this helps. Do let me know if you have any further queries.

Thankyou!

Dungrani, Deny 0 Reputation points

2026-07-03T09:16:32.5+00:00

Hi @Anshika Varshney , we are facing the same issue enough though the private endpoints are configured. Are there any further requirements to get evaluation to work?

openai.BadRequestError: Error code: 400 - {'error': {'code': 'UserError', 'severity': None, 'message': 'The action cannnot be finished with reason Forbidden', 'messageFormat': 'The action cannnot be finished with reason {error}', 'messageParameters': {'error': 'Forbidden'}, 'referenceCode': None, 'detailsUri': None, 'target': None, 'details': [], 'innerError': {'code': 'UnauthorizedUserAction', 'innerError': None}, 'debugInfo': None, 'additionalInfo': None}, 'correlation': {'operation': 'c325a4b2b5aad7ef1e25ad3dec165af7', 'request': 'afa695d7f6c94b32'}, 'environment': 'swedencentral', 'location': 'swedencentral', 'time': '2026-07-03T08:11:58.3217694+00:00', 'componentName': 'raisvc', 'statusCode': 400}