All my requests are hitting 429 today on Document Intelligence

Hello Maher,

Thank you for your question. It's very frustrating when a service that was working perfectly suddenly starts failing with 429 errors, especially when you've made no changes to your implementation.

The HTTP status code 429 Too Many Requests is a definitive message from the Document Intelligence service indicating that your application is being throttled because it has exceeded the allowed rate of requests [, ].

You mentioned that you enabled auto-scaling, which is a good step, but it's important to understand how throttling and auto-scaling work together in Azure AI services.

Why You're Getting 429 Errors

Even with auto-scaling enabled, there are still base transaction-per-second (TPS) limits on your Document Intelligence resource. For the standard pricing tier, the default limit for analysis requests (POST operations) is 15 TPS.

Auto-scaling is designed to handle gradual increases in load. If your application sends a sudden, sharp spike in requests that far exceeds your current limit, the auto-scaling mechanism might not have enough time to provision the necessary backend resources. During this "warm-up" period, the service will throttle your requests, resulting in the 429 errors you're seeing.

This can happen for a few reasons:

• A Sudden Burst in Workload: A new process or a batch job in your application might have started sending a large number of documents at once.

Aggressive Polling: If your code is checking for the results of an analysis operation (a GET request) too frequently and in a tight loop, it can also lead to throttling.

Regional Capacity Issues: While less common, there can be temporary capacity constraints in a specific Azure region that cause the service to throttle requests more aggressively than usual.

How to Fix and Prevent This Issue

Here are the best practices for handling 429 errors and ensuring your application runs smoothly.

1. Implement Exponential Backoff and Retry Logic

This is the most critical and immediate fix. Your application code should be designed to handle throttling gracefully. Instead of failing immediately, it should catch the 429 error and automatically retry the request after a short delay.

How it works: When you receive a 429 error, wait for a small, randomized amount of time (e.g., 1-2 seconds) and then retry the request. If it fails again, increase the delay before the next retry (e.g., 2-4 seconds, then 4-8 seconds, and so on). This "backs off" the pressure on the service, giving it time to catch up [].

Check the Retry-After Header: The 429 response from the Document Intelligence service often includes a Retry-After header. This header tells you exactly how many seconds you should wait before sending the next request. Using this value is the most efficient way to handle throttling.

2. Smooth Out Your Workload

Instead of sending a large batch of requests all at once, try to spread them out over a slightly longer period. A gradual increase in the workload is much less likely to trigger the auto-scaler's throttling mechanism.

3. Request a Quota Increase

If your application consistently requires a higher throughput than the default limit, and you've already implemented retry logic, the next step is to request a permanent increase in your TPS limit.

You can do this by creating a support ticket in the Azure portal and requesting a "Service and subscription limits (quotas)" increase for your Document Intelligence resource [, ]. Be prepared to explain your workload and the throughput you require.

By implementing robust retry logic in your application, you can make it resilient to temporary throttling, which will resolve the immediate issue. For a long-term solution, consider requesting a quota increase if your application's demand consistently exceeds the default limits.

Best Regards,

Jerald Felix

I don't believe it is at the api level. If I try it in Document Intelligence studio no extraction completes regardless of the model or document.

I don't believe it is at the api level. If I try it in Document Intelligence studio no extraction completes regardless of the model or document.

I have the exact same issue starting 10:17 AM UTC yesterday (sept 17)

Dived a bit into it. We are not doing more requests, running requests are taking a very long time. This is visible as we do not do more analyze calls, but only object.get calls are being done. These are done by the poller in the azure SDK (which is at 1s) so if documents are slow, a queue builds up in azure and our services keep on polling azure (through the SDK we did not build the polling ourselves). This leads to so many requests that 429 start being returned. I'm not sure why this is suddenly happing. We did not upgrade packages, change code, or change our traffic. I guess there is some capacity issue at Azure.

Does this mean we have to raise an issue on the azure sdk github, change the polling parameter ourselves, ...? Do note that it takes more than 10 minutes for documents to process now.