Azure OpenAI GlobalStandard: streaming /responses request held 155 s before response headers with no 429 - is this the shared-pool admission gate?

Eszeki Daniel 0 Reputation points
2026-09-10T08:20:43.8233333+00:00
We stream chat responses from a gpt-5.6 deployment on Azure OpenAI, using the
Responses API (POST /responses, api-version=v1, stream: true) with reasoning
effort medium. The deployment is GlobalStandard, and x-ms-region reads France
Central on every response.

Intermittently a request is accepted and then held for a long time before any
response byte arrives. Over one 12-hour window, out of 1,257 streaming
requests, 12 took more than 30 seconds just to return response headers, and the
worst took 155 seconds and then came back HTTP 400 with a 153-byte body
(apim-request-id c0d1fdbb-7c97-44b8-b292-8327894f9708). There is no 429, no
5xx, no Retry-After, no streamed error event, and the connection is never
closed by the service. Retrying the same prompt seconds later returns headers
in under a second.

Two things make me think this is not simply slow generation. A 400 arriving
after 155 seconds does not look like request validation, which should fail in
milliseconds; every other 400 in the window returned in under a second. And the
delay does not track our request rate: the worst delays happened in our
quietest hours, at about a fifth of our daytime volume.

I have already ruled out our own egress path. Outbound NAT was at 3.5 percent
of allocated ports with zero failed connections, client CPU throttling was 0.46
percent, DNS logged no failures, and the Date response header shows the service
had the request promptly and held the response, rather than the delay being in
connection setup. The content filter on this deployment is disabled, so
filtering is not in the path either.

I have read the guidance on Azure OpenAI performance and latency, and the
existing thread about GPT-5.2 first-token latency on Global Standard, which
attributes 40 to 50 second delays to enterprise processing layers and
shared-capacity queueing. The closest match I found is this thread about the
Responses API on GPT-5.4 Data Zone Standard in the same region:

https://learn.microsoft.com/en-us/answers/questions/5915419/azure-openai-(gpt-5-4-data-zone-standard)-response

The answer there identifies a shared-pool capacity admission gate, separate
from TPM and RPM quota, that surfaces as HTTP 200 with no 429 and no
Retry-After. That is very close to what we see, except we get no error event at
all: we have zero occurrences of no_capacity or too_many_requests in the whole
window.

My question: can that same admission gate also apply to a GlobalStandard
deployment and show up as a hold before the response headers are flushed, with
no error event and no 429? If it can, is there any response header or event we
can key on to tell "queued behind admission control" apart from "generating
slowly"? The two look identical from the client today, and they need opposite
handling - wait longer, or back off and retry.

I can share per-request timings and the Azure request ids for the other
affected requests if that helps.

Azure OpenAI in Foundry Models
0 comments No comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.