IoT Hub – Workaround for "Maximum call stack size exceeded"

Hello @Nelson Lim

Thanks for the very thorough repro details — the fault-injection sequence and the stack frame you captured made this much easier to reason about, and your follow-up data genuinely changed the diagnosis. An uncaught RangeError: Maximum call stack size exceeded that terminates the Node.js host process is a production-severity condition, and I'm treating it as one.

Where the evidence points (likely, not yet confirmed)

The failing frame is the SDK-internal getErrorName(), and the crash happens while an error object is being processed during link recovery — not during the send itself. The leading hypothesis is that the translated error carries a self-referential cause / innerError chain that getErrorName() walks without a terminating condition, exhausting the stack.

That fits all three of your observations: it reproduces at 2 and 10 concurrent sends, it still occurs with retry count reduced and with retry disabled entirely, and the throw isn't catchable at your call site. Those results rule out the two explanations originally on the table send concurrency, and retry depth as a sufficient cause.

To be straight with you: I checked Microsoft Learn, Q&A, public issue trackers and our internal troubleshooting material, and I found no existing bug. What's still open is whether the defect sits in azure-iothub, azure-iot-amqp-base, or the underlying rhea AMQP layer, and whether it's transport-specific. The steps below are designed to close exactly that gap so this can be filed against the right package.

Step 1 — Capture the full, untruncated stack

DEBUG=azure-iot*,rhea* node --stack-trace-limit=1000 --unhandled-rejections=strict app.js

Node truncates stack traces by default, which is why the recursive cycle is currently invisible. The repeating frame block beneath getErrorName will name the exact package/function pair forming the loop. Please send me the repeating unit plus the last ~500 lines of debug output before termination.

Step 2 — Report the resolved dependency tree

npm ls azure-iothub azure-iot-amqp-base azure-iot-common rhea rhea-promise
node -v

azure-iothub@1.16.6 is only the top-level package — AMQP error translation lives beneath it, and the correct place to file depends on which transitive versions you've resolved. You should see a pinned version for each with no UNMET or (empty) entries.

Step 3 — Re-run the same fault injection over AMQP over WebSockets

Switch the client construction to Client.fromConnectionString(connStr, AmqpWs) (importing AmqpWs from azure-iothub), then repeat the test blocking port 443 instead of 5671.

Both outcomes are useful. If the crash disappears, the defect is specific to the native AMQP transport and you have a production-viable transport change today — 443 also sidesteps the firewall condition that triggered this in the first place. If it still reproduces, the defect is in shared error-handling code, which is decisive for filing.

Confirm the connection is actually on 443 mid-send with ss -tnp | grep :443 (Linux) or netstat -an | findstr 443 (Windows), then report crash/no-crash.

Step 4 — Make sure there's only one retry authority

Check whether your app wraps serviceClient.send() in its own retry while the SDK's default ExponentialBackoffWithJitter policy is also active. If so, collapse to one layer and bound it via setRetryPolicy — a shouldRetry capped at 3 attempts and a jittered nextRetryTimeout capped at 30 seconds. Azure guidance explicitly warns against duplicated retry layers and endless retry, with 3s / 12s / 30s as the reference back-off progression. Stacked retry amplifies reconnect churn even where it isn't the root cause.

Expected result with the port blocked: at most three attempts at roughly 3s, 12s and 30s, then a terminal error returned to your callback — with the process still alive.

On the workarounds already suggested — treat them as containment, not a fix

Throttling concurrency, external back-off, cancelling pending sends on disconnect, and a process.on('uncaughtException') guard all reduce exposure, but none repairs the SDK path you're hitting. One specific caution on the last one: there are documented Node.js cases where an uncaughtException handler does not intercept a stack-overflow RangeError. Please verify your handler actually catches this error before relying on it in production.

One issue in the load test itself

IoT Hub queues C2D messages server-side and holds at most 50 cloud-to-device messages per device queue (max C2D message size 64 KB). A 2,000-message burst to one device will exceed that regardless of connectivity and return 403004 DeviceMaximumQueueDepthExceeded, which will muddy your results. Worth correcting even though it isn't the crash mechanism. If you need to drain a backlog during testing, the Purge Queue API is the clean way to reset between runs.

Reference documentation

• Manage device reconnections / retry patterns: https://learn.microsoft.com/azure/iot-hub/concepts-manage-device-reconnections
• Node SDK setRetryPolicy: https://learn.microsoft.com/javascript/api/azure-iot-device/client#azure-iot-device-client-setretrypolicy
• ExponentialBackOffWithJitter (azure-iot-common): https://learn.microsoft.com/javascript/api/azure-iot-common/exponentialbackoffwithjitter
• NoRetry class: https://learn.microsoft.com/javascript/api/azure-iot-common/noretry
• Node SDK connectivity & retries wiki: https://github.com/Azure/azure-iot-sdk-node/wiki/Connectivity-and-Retries
• C2D messaging concepts (50-message queue limit): https://learn.microsoft.com/azure/iot-hub/iot-hub-devguide-messages-c2d
• IoT Hub quotas and throttling: https://learn.microsoft.com/azure/iot-hub/iot-hub-devguide-quotas-throttling#other-limits
• 403004 DeviceMaximumQueueDepthExceeded: https://learn.microsoft.com/azure/iot-hub/troubleshoot-error-codes#403xxx-forbidden-errors
• Purge C2D queue API: https://learn.microsoft.com/rest/api/iothub/service/cloud-to-device-messages/purge-cloud-to-device-message-queue
• Transient fault handling guidance: https://learn.microsoft.com/azure/architecture/best-practices/transient-faults

Once you send the Step 1 trace and the Step 2 dependency versions, I'll take it from there — including raising it with the SDK team on your behalf if the trace confirms the recursion. Given this crashes a production process, please don't hesitate to open a support case in parallel if you need a faster escalation path; I'm happy for this thread to feed straight into it.

Thanks again for the quality of the repro it's doing a lot of the work here.

Thanks,
Manish.

Hi ,

Thanks for reaching out to Microsoft Q&A.

This is a known failure pattern under heavy pending operations+unstable AMQP where the Node.js azure-iothub SDK can get into recursive error handling during reconnect storms. There is no clean bypass, so the practical workaround is to control concurrency and failure behaviour: throttle C2D sends (queue with max inflight limit instead of blasting 2K async calls), implement retry with exponential backoff+jitter outside the SDK, and cancel/timeout pending sends when the connection drops rather than letting them pile up. Also consider switching transport to AMQP over websockets (443) to avoid port blocks, and add a process level guard (uncaughtException handler+graceful restart) to prevent full crashes. If possible, upgrade or test alternate patterns (Service Bus queue ->device pull) since C2D at that scale with flaky connectivity is fragile in this SDK.

Two key questions?

1. Are you firing all 2k sends concurrently or using any bounded queue/backpressure mechanism?
2. Have you tested AMQP over websockets or reduced retry policy to limit recursive reconnect attempts?

Please 'Upvote'(Thumbs-up) and 'Accept' as answer if the reply was helpful. This will be benefitting other community members who face the same issue.