Function App deadlock when merge >10k entities to azure data table

Thank you! I will give it a try. Does it also work with?

TableClient.fromConnectionString()

I tested it with new TableClient, but the options object does not seem to be right.

const keepAliveAgent = new https.Agent({ keepAlive: true });
const baseHttpClient = createDefaultHttpClient();
const httpClient = {
    sendRequest: (request: any) => {
        request.agent = keepAliveAgent;
        return baseHttpClient.sendRequest(request);
    },
};
const client = new TableClient(
    `https://${accountName}.table.core.windows.net`,
    tableName,
    new DefaultAzureCredential(),
    { httpClient }
);

tried like that, but stopped after 16k, same behaviour

Hello Phillip,

Thanks for the update and for sharing your configuration! Implementing the keepAlive custom HTTP client is a great step, and doing so successfully eliminates SNAT port (socket) exhaustion from the equation.

However, since your function is an HTTP Trigger and it appears you are still iterating through the table merges sequentially (one await at a time), your function has hit a different, hardcoded architectural wall: the Azure Load Balancer timeout.

The 230-Second Limit

Let's do some quick math: If a single Azure Table merge takes roughly 15 milliseconds, processing 16,000 records sequentially will take about 240 seconds (4 minutes). This duration aligns perfectly with a known platform limit.

According to the official Azure Functions scale and hosting documentation:

"Regardless of the function app timeout setting, 230 seconds is the maximum amount of time that an HTTP triggered function can take to respond to a request. This limit exists because of the default idle timeout of Azure Load Balancer."

When your HTTP Trigger runs for more than 230 seconds without returning an HTTP response to the caller, the underlying Azure Load Balancer forcibly drops the connection. The process is terminated abruptly, which perfectly explains why the function simply "stops" at the 16k mark with no errors logged in Application Insights.

The Fix: You Must Implement Chunking

While your keepAlive agent is keeping the single connection healthy, it isn't making the code run fast enough. To safely bypass the 230-second load balancer limit, you must process the rows concurrently in batches using Promise.all(). This will compress the total execution time from several minutes down to just a few seconds.

Here is how you apply concurrency to your existing setup:

const { DefaultAzureCredential } = require("@azure/identity");
const { TableClient } = require("@azure/data-tables");
const { createDefaultHttpClient } = require("@azure/core-rest-pipeline");
const https = require("https");

// 1. Your custom keepAlive logic
const keepAliveAgent = new https.Agent({ keepAlive: true, maxSockets: 50 });
const baseHttpClient = createDefaultHttpClient();
const httpClient = {
    sendRequest: (request) => {
        request.agent = keepAliveAgent;
        return baseHttpClient.sendRequest(request);
    },
};

const client = new TableClient(
    `https://${accountName}.table.core.windows.net`,
    tableName,
    new DefaultAzureCredential(),
    { httpClient }
);

module.exports = async function (context, req) {
    const entitiesToUpload = req.body.entities; // Assuming 50k rows passed in
    
    // 2. Chunking Logic (DO NOT process 1 by 1)
    const chunkSize = 100; // Process 100 concurrently
    
    for (let i = 0; i < entitiesToUpload.length; i += chunkSize) {
        const chunk = entitiesToUpload.slice(i, i + chunkSize);
        
        // Fire 100 HTTP requests in parallel, and await the whole batch
        await Promise.all(chunk.map(entity => client.upsertEntity(entity, "Merge")));
        
        if (i % 5000 === 0) {
            context.log(`Successfully merged ${i} entities...`);
        }
    }
    
    context.res = { body: "All entities merged successfully!" };
}

By adding Promise.all() to fire off 100 requests concurrently, your 50,000 records will finish processing well before the 230-second Azure Load Balancer timeout drops the connection. Note: This code is generated with the help of AI systems.

Thank you for the answer. In my case I use a timer trigger. So the http trigger timeout should not occur. And still it would be great to get and error or a log for that.