About Azure Speech Custom Endpoint scaling

Siarhei Lukashenka 20 Reputation points
2026-09-07T02:19:28.78+00:00

Hi,

My quota for the TTS Speech service is supposedly quote good, but when I have a traffic burst (well within the quota limits) I see a lot of ResourceExhausted: No tokens available for passthrough (WebSocket Error 1013) errors. Is my understanding correct that it takes some time for the Speech endpoint to scale out, and that is why those errors appear?

Azure Speech in Foundry Tools

2 answers

Sort by: Oldest
Most helpful Newest Oldest

  1. AI answer

    2026-09-07T02:19:41.21+00:00

    Yes. That behavior matches Azure Speech autoscaling.

    A burst can produce 429-style throttling conditions even when the workload is still within the configured quota, because the service may still be scaling up to the new demand and not yet have enough backend capacity available. The documented guidance is that increasing quota does not help in that situation.

    For text-to-speech specifically, the service recommends:

    1. Implement retry logic for transient throttling conditions.
    2. Avoid sharp workload changes; ramp traffic up gradually instead of sending sudden bursts.
    3. Test different load-increase patterns before production.
    4. Use multiple Speech resources in different regions and distribute traffic across them. Multiple resources in the same region do not improve performance because they use the same backend cluster.

    There is also a TTS-specific note that many 429 errors for standard voices are caused by limited backend capacity for a specific voice in a region, not by quota limits. In that case, increasing quota will not resolve the issue. Using the voice in its native region or choosing a more common voice in the current region can help.

    For resiliency, client code can monitor persistent errors and redirect traffic to a backup Speech resource in another region.

    AI-generated content may be incorrect. Read our transparency notes for more information.

    Was this answer helpful?

  2. Vinodh247-1375 44,556 Reputation points Volunteer Moderator
    2026-09-07T04:48:56.4366667+00:00

    Yes, that is a plausible explanation, but I would be careful about treating the ResourceExhausted: No tokens available for passthrough (WebSocket Error 1013) message as direct evidence that you've hit a quota limit.

    The key distinction is that Azure Speech quotas define what you are allowed to consume, whereas a custom TTS endpoint must still have sufficient runtime capacity available to serve incoming requests. As a result, it is possible to see ResourceExhausted errors during a sudden burst even when your measured request rate is below the configured quota. [learn.microsoft.com]

    One aspect worth checking is whether the burst creates a very high number of simultaneous synthesisation requests rather than simply a high requests-per-second rate. Custom endpoints have concurrency-related limits and internal capacity constraints that can become the bottleneck before an overall quota is reached.

    A useful test is to compare these two scenarios:

    • A steady load at the same average throughput.
    • A short, sharp burst that reaches the throughput immediately.

    If the steady load succeeds but the burst produces 1013 errors, that strongly suggests a transient capacity or scale-out effect rather than a quota exhaustion issue.

    I would also monitor:

    • Peak concurrent synthesis requests.
    • Connection establishment rate for WebSocket clients.
    • Percentage of requests failing only during burst windows.
    • Whether failures disappear after a brief retry delay.

    If errors occur only during sudden traffic spikes and recovery is quick, that aligns with temporary backend capacity availability while the service adjusts to the new demand. If the same errors continue at a stable and sustained load level, then the issue is more likely related to endpoint-specific concurrency or service capacity rather than autoscaling lag alone. [learn.microsoft.com]

    Help make this community better for everyone: if this answer resolved your issue, please accept it or leave an upvote. If not, share more details in a comment so we can continue the discussion and find the right solution.

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.