Azure OpenAI in Foundry Models
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
We stream chat responses from a gpt-5.6 deployment on Azure OpenAI, using the
Responses API (POST /responses, api-version=v1, stream: true) with reasoning
effort medium. The deployment is GlobalStandard, and x-ms-region reads France
Central on every response.
Intermittently a request is accepted and then held for a long time before any
response byte arrives. Over one 12-hour window, out of 1,257 streaming
requests, 12 took more than 30 seconds just to return response headers, and the
worst took 155 seconds and then came back HTTP 400 with a 153-byte body
(apim-request-id c0d1fdbb-7c97-44b8-b292-8327894f9708). There is no 429, no
5xx, no Retry-After, no streamed error event, and the connection is never
closed by the service. Retrying the same prompt seconds later returns headers
in under a second.
Two things make me think this is not simply slow generation. A 400 arriving
after 155 seconds does not look like request validation, which should fail in
milliseconds; every other 400 in the window returned in under a second. And the
delay does not track our request rate: the worst delays happened in our
quietest hours, at about a fifth of our daytime volume.
I have already ruled out our own egress path. Outbound NAT was at 3.5 percent
of allocated ports with zero failed connections, client CPU throttling was 0.46
percent, DNS logged no failures, and the Date response header shows the service
had the request promptly and held the response, rather than the delay being in
connection setup. The content filter on this deployment is disabled, so
filtering is not in the path either.
I have read the guidance on Azure OpenAI performance and latency, and the
existing thread about GPT-5.2 first-token latency on Global Standard, which
attributes 40 to 50 second delays to enterprise processing layers and
shared-capacity queueing. The closest match I found is this thread about the
Responses API on GPT-5.4 Data Zone Standard in the same region:
https://learn.microsoft.com/en-us/answers/questions/5915419/azure-openai-(gpt-5-4-data-zone-standard)-response
The answer there identifies a shared-pool capacity admission gate, separate
from TPM and RPM quota, that surfaces as HTTP 200 with no 429 and no
Retry-After. That is very close to what we see, except we get no error event at
all: we have zero occurrences of no_capacity or too_many_requests in the whole
window.
My question: can that same admission gate also apply to a GlobalStandard
deployment and show up as a hold before the response headers are flushed, with
no error event and no 429? If it can, is there any response header or event we
can key on to tell "queued behind admission control" apart from "generating
slowly"? The two look identical from the client today, and they need opposite
handling - wait longer, or back off and retry.
I can share per-request timings and the Azure request ids for the other
affected requests if that helps.
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.