About Azure Speech Custom Endpoint scaling

Question

About Azure Speech Custom Endpoint scaling

Siarhei Lukashenka 20

Hi,

My quota for the TTS Speech service is supposedly quote good, but when I have a traffic burst (well within the quota limits) I see a lot of ResourceExhausted: No tokens available for passthrough (WebSocket Error 1013) errors. Is my understanding correct that it takes some time for the Speech endpoint to scale out, and that is why those errors appear?

Siarhei Lukashenka 20 Reputation points

2026-09-07T04:11:32.9433333+00:00

Is there any way to ask for more default capacity on the Azure site to reduce the risk of cold start for TTS?

2 answers

Your answer

Siarhei Lukashenka 20 Reputation points

2026-09-07T04:11:32.9433333+00:00

Is there any way to ask for more default capacity on the Azure site to reduce the risk of cold start for TTS?

Answer 1

Yes, that is a plausible explanation, but I would be careful about treating the ResourceExhausted: No tokens available for passthrough (WebSocket Error 1013) message as direct evidence that you've hit a quota limit.

The key distinction is that Azure Speech quotas define what you are allowed to consume, whereas a custom TTS endpoint must still have sufficient runtime capacity available to serve incoming requests. As a result, it is possible to see ResourceExhausted errors during a sudden burst even when your measured request rate is below the configured quota. [learn.microsoft.com]

One aspect worth checking is whether the burst creates a very high number of simultaneous synthesisation requests rather than simply a high requests-per-second rate. Custom endpoints have concurrency-related limits and internal capacity constraints that can become the bottleneck before an overall quota is reached.

A useful test is to compare these two scenarios:

A steady load at the same average throughput.
A short, sharp burst that reaches the throughput immediately.

If the steady load succeeds but the burst produces 1013 errors, that strongly suggests a transient capacity or scale-out effect rather than a quota exhaustion issue.

I would also monitor:

Peak concurrent synthesis requests.
Connection establishment rate for WebSocket clients.
Percentage of requests failing only during burst windows.
Whether failures disappear after a brief retry delay.

If errors occur only during sudden traffic spikes and recovery is quick, that aligns with temporary backend capacity availability while the service adjusts to the new demand. If the same errors continue at a stable and sustained load level, then the issue is more likely related to endpoint-specific concurrency or service capacity rather than autoscaling lag alone. [learn.microsoft.com]

Help make this community better for everyone: if this answer resolved your issue, please accept it or leave an upvote. If not, share more details in a comment so we can continue the discussion and find the right solution.

Siarhei Lukashenka 20 Reputation points

2026-09-07T05:46:21.76+00:00

I agree that I don't think it is a quota-related issue as we have a good one as far as I know. I attribute the issue to the cold start in the Speech Service containers that take time to scale out, so my question is whether it is possible to expand the default baseline for how many containers are on stand-by to avoid cold start errors. Is that even attainable in Azure Speech?

Answer 2

Yes. That behavior matches Azure Speech autoscaling.

A burst can produce 429-style throttling conditions even when the workload is still within the configured quota, because the service may still be scaling up to the new demand and not yet have enough backend capacity available. The documented guidance is that increasing quota does not help in that situation.

For text-to-speech specifically, the service recommends:

Implement retry logic for transient throttling conditions.
Avoid sharp workload changes; ramp traffic up gradually instead of sending sudden bursts.
Test different load-increase patterns before production.
Use multiple Speech resources in different regions and distribute traffic across them. Multiple resources in the same region do not improve performance because they use the same backend cluster.

There is also a TTS-specific note that many 429 errors for standard voices are caused by limited backend capacity for a specific voice in a region, not by quota limits. In that case, increasing quota will not resolve the issue. Using the voice in its native region or choosing a more common voice in the current region can help.

For resiliency, client code can monitor persistent errors and redirect traffic to a backup Speech resource in another region.