Azure ML batch endpoint returns 403 “Tenant mismatch” even though token and resource tenant IDs match

Ray Swan 0 Reputation points
2026-08-16T13:40:34.27+00:00

Azure ML batch endpoint returns 403 “Tenant mismatch” even though token and resource tenant IDs match

I'm seeing a reproducible Azure ML Batch Endpoint invocation failure that appears to occur at the Azure ML scoring frontdoor.

az ml batch-endpoint invoke returns:

HTTP 403
Tenant mismatch: Token tenant does not match resource tenant
server: azureml-frontdoor

The endpoint and deployment provision successfully and report healthy. The failure occurs only when invoking the endpoint.

I've completed the usual authentication, tenant, endpoint, and RBAC troubleshooting and then traced the failing request with --debug.

For the exact bearer token sent to the scoring frontdoor:

aud = https://ml.azure.com
tid = [TENANT_ID]

I compared that tid with every relevant tenant value available through Azure/ARM:

Source Result
Bearer token tid [TENANT_ID]
-------- --------
Bearer token tid [TENANT_ID]
Subscription homeTenantId [TENANT_ID]
Workspace properties.tenantId [TENANT_ID]
Workspace managed identity identity.tenantId [TENANT_ID]
Only tenant returned by az account tenant list [TENANT_ID]

All five are identical. There is no guest/B2B or multi-tenant discrepancy visible from the client or ARM side.

I've also tested:

Fresh az login --tenant [TENANT_ID]

az account clear followed by fresh authentication

User-principal and GitHub Actions OIDC/service-principal authentication

Data-asset and raw HTTPS blob inputs

Deleting and recreating the endpoint/deployment

az ml workspace sync-keys

Waiting ~9 hours after endpoint creation to eliminate propagation as an explanation

Most importantly, I reproduced the same 403 against a second, independent, older Azure ML workspace and batch endpoint in the same subscription/tenant, ruling out configuration specific to the new workspace.

The response is a literal 60-byte text/plain response from azureml-frontdoor, not a client-generated error. I traced the Azure ML CLI response handling to verify this.

At this point the observable tenant information contradicts the frontdoor response: the token says tenant A, ARM says the resource belongs to tenant A, but the scoring frontdoor rejects the request because it believes the resource belongs to a different tenant.

Has anyone encountered this with Azure ML Batch Endpoints?

In particular, is there another resource-level tenant association used internally by azureml-frontdoor that isn't exposed through the workspace ARM resource?

If someone from the Azure ML team can help investigate, I can reproduce on demand and provide a fresh UTC timestamp, endpoint/resource ID, sanitized token claims, full debug trace, and any request/correlation IDs returned by the frontdoor.

I don't currently have a paid Azure technical-support plan, so I'm posting here after completing the Azure-recommended troubleshooting.

Azure Batch
Azure Batch

An Azure service that provides cloud-scale job scheduling and compute management.

0 comments

2 answers

Sort by: Newest
Most helpful Newest Oldest
  1. Phillip Stenger 10 Reputation points
    2026-09-04T18:33:46.4866667+00:00

    For us this was solved by setting a default deployment for the batch endpoint. Not sure why it manifested as a 403 or Tenant error. But maybe double check that your endpoint has a default deployment set.

    Was this answer helpful?

    1 person found this answer helpful.
    0 comments
  2. Allan Solomon Mejia 7,915 Reputation points
    2026-08-16T16:33:00.3466667+00:00

    Hello @Ray Swan

    Based on what you’ve already validated, this does not look like a normal client-side tenant, login, or RBAC problem. Azure ML Batch Endpoints require a valid Microsoft Entra token for invocation, and the invoking identity must have the required permissions to create the batch job.

    In your case, though, the token tid, subscription homeTenantId, workspace tenantId, workspace managed identity tenant, and the tenant returned by Azure CLI all match. You also reproduced the same 403 against a second older workspace in the same subscription/tenant, after testing fresh authentication and different principals.

    The particularly important detail is that the response comes directly from:

    server: azureml-frontdoor

    Tenant mismatch: Token tenant does not match resource tenant

    rather than from the CLI itself.

    Given that evidence, I would not keep recreating endpoints, refreshing credentials, or changing RBAC roles. The failure appears to happen before the batch job is created, during the Azure ML data-plane/frontdoor authorization check.

    Microsoft’s public Batch Endpoint documentation explains the Entra token requirement and endpoint authorization model, but I don’t see a documented additional tenant identifier that customers can inspect or repair themselves which would explain this specific mismatch.

    At this point, this needs an Azure Machine Learning backend investigation. I would provide Microsoft with:

    • Workspace and batch endpoint resource IDs
    • Subscription and tenant IDs
    • Exact UTC timestamp of a fresh reproduction
    • Sanitized token claims showing aud and tid
    • CLI --debug trace
    • Any request/correlation IDs returned by azureml-frontdoor

    Since you don’t have a paid technical support plan, I would ask a Microsoft Q&A moderator to help route this to the Azure ML product team or arrange collection of those identifiers privately.

    I also wouldn’t describe this as a confirmed Azure ML platform bug yet, but your cross-workspace reproduction makes a stale or incorrect backend tenant association a reasonable hypothesis that Microsoft engineering would need to verify.

    Please "Accept the Answer" if this information helped you. This will help us and others in the community as well.

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.