This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(169.60000000000002, 66%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EC%3C/text%3E%3C/svg%3E)
Hello,
Suddenly some SSL handshake issues started surfacing intermittently, for the outbound calls from my AKS hosted services. My AKS cluster is a private cluster and exposes the APIs via Azure Application Gateway as the ingress. This was not the case for the same services hosted in an elastic Kubernetes hosted in AWS. The detailed exception is here:
"System.ApplicationException\n ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.\n ---> System.IO.IOException: Unable to read data from the transport connection: Connection timed out.\n ---> System.Net.Sockets.SocketException (110): Connection timed out\n at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow)\n at System.Net.Sockets.NetworkStream.ReadAsync(Memory1 buffer, CancellationToken cancellationToken)\n at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken, Int32 estimatedSize)\n at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)\n at System.Net.Sockets.SocketAsyncEngine.System.Threading.IThreadPoolWorkItem.Execute()\n at System.Threading.ThreadPoolWorkQueue.Dispatch()\n at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()\n at System.Threading.Thread.StartCallback()\n--- End of stack trace from previous location ---\n\n --- End of inner exception stack trace ---\n at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken, Int32 estimatedSize)\n at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder1.StateMachineBox1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)\n at System.Net.Security.SslStream.ReceiveHandshakeFrameAsync[TIOAdapter](CancellationToken cancellationToken)\n at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](Boolean receiveFirst, Byte[] reAuthenticationData, CancellationToken cancellationToken)\n at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)\n --- End of inner exception stack trace ---\n at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)\n at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)\n at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)\n at System.Net.Http.HttpConnectionPool.InjectNewHttp11ConnectionAsync(QueueItem queueItem)\n at System.Threading.Tasks.TaskCompletionSourceWithCancellation1.WaitWithCancellationAsync(CancellationToken cancellationToken)\n at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)\n at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)\n at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)\n at System.Net.Http.SocketsHttpHandler.<SendAsync>g_CreateHandlerAndSendAsync|115_0(HttpRequestMessage request, CancellationToken cancellationToken)\n at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendCoreAsync>gCore|4_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken)\n at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendCoreAsync>gCore|4_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken)\n at System.Net.Http.HttpClient.<SendAsync>gCore|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
Need urgent fix as it is a production issue. My HTTP client configurations are verified as per the standard .net practice recommendations.
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 95%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESD%3C/text%3E%3C/svg%3E)
Hello cargobuddy
Kindly please share the details requested in the private message for further troubleshooting.
Please close this incident, as the root cause of this issue was not configuring the NAT gateway for the cluster. Earlier, we had load balancer for outbound traffic, which we have now substituted with NAT gateway, and after that we are not facing any issue.
Hello @SUNOJ KUMAR YELURU
After applying the above command, now we have started getting the below exception:
ExceptionMetadata { Error: "System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. ---> System.IO.IOException: Unable to read data from the transport connection: Connection timed out. ---> System.Net.Sockets.SocketException (110): Connection timed out at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow) at System.Net.Sockets.NetworkStream.ReadAsync(Memory1 buffer, CancellationToken cancellationToken) at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken, Int32 estimatedSize) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Net.Sockets.SocketAsyncEngine.System.Threading.IThreadPoolWorkItem.Execute() at System.Threading.ThreadPoolWorkQueue.Dispatch() at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart() at System.Threading.Thread.StartCallback() --- End of stack trace from previous location --- --- End of inner exception stack trace --- at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken, Int32 estimatedSize) at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder1.StateMachineBox1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token) at System.Net.Security.SslStream.ReceiveHandshakeFrameAsync[TIOAdapter](CancellationToken cancellationToken) at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](Boolean receiveFirst, Byte[] reAuthenticationData, CancellationToken cancellationToken) at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.InjectNewHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.SocketsHttpHandler.<SendAsync>g__CreateHandlerAndSendAsync|115_0(HttpRequestMessage request, CancellationToken cancellationToken) at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendCoreAsync>g__Core|4_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken) at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendCoreAsync>g__Core|4_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken) at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
Can you help get this fixed urgently? If required, please setup a call, as it is crucial for our production deployment to get resolved. The above exception is found with almost no traffic, with just one person triggering the APIs, so it is not related to the request load.
This is not SSL. It's TCP timeout..NET just reports it during the TLS handshake because that's where it hangs.
Quick test from inside the cluster:
kubectl run -it --rm debug --image=mcr.microsoft.com/azure-cli -- bash
inside pod
for i in {1..100}; do curl -w "%{http_code} %{time_total}\n" -o /dev/null -s https://your-external-dependency.com & done
If many curls hang, it's SNAT/firewall, not code.
Send me the output of: az aks show... --query networkProfile.loadBalancerProfile
and your total node count
Hi @SUNOJ KUMAR YELURU Just now I moved from load balancer to NAT gateway for outbound traffic. So now, we are not facing the SSL issue, but the following issue which is totally different:
System.Exception: An exception occurred during backend call: Response entity too large: DefaultHttpResponse(decodeResult: success, version: HTTP/1.1)
HTTP/1.1 200
server-timing: intid;desc=693c7d22039ac7d6
vary: accept-encoding
content-type: application/xml
content-length: 2747460
date: Fri, 07 Aug 2026 10:46:37 GMT
---> System.Exception: Supplier API 500 Error: An exception occurred during backend call: Response entity too large: DefaultHttpResponse(decodeResult: success, version: HTTP/1.1)
HTTP/1.1 200
server-timing: intid;desc=693c7d22039ac7d6
vary: accept-encoding
content-type: application/xml
content-length: 2747460
date: Fri, 07 Aug 2026 10:46:37 GMT
Do you think switching to NAT gateway as you suggested earlier have resolved the problem ? If so, then what can I do to resolve the above one?
Hello @cargobuddy,
That SSL connection could not be established is misleading. The root cause is not SSL. It's TCP.
SocketException (110): Connection timed out
at SslStream.EnsureFullTlsFrameAsync
TCP SYN went out, TCP connection was never established, so TLS ServerHello never came back. SSL is the victim.
In AKS private clusters this is almost always SNAT exhaustion or hairpin routing via App Gateway. This explains why it worked in EKS - AWS NAT Gateway has 55k ports per destination and auto-scales, Azure LoadBalancer SNAT gives you ∼1024 ports per node per public IP by default.
1. Add outbound capacity immediately:
If you are on LB outbound type:
az aks update -g <RG> -n <AKS> \
--load-balancer-managed-outbound-ip-count 3 \
--load-balancer-managed-outbound-ports 16000 \
--load-balancer-idle-timeout 4
This triples your ports from ∼1024 to ∼48k per node. Node reboot not needed but takes 5-10 min.
Best long term: switch to Managed NAT Gateway:
az aks update -g <RG> -n <AKS> --outbound-type managedNATGateway --nat-gateway-managed-v4-ip-count 2
2 IPs = 128k SNAT ports.
2. Harden .NET client - this is mandatory on Azure:
Default Azure LB idle timeout is 4 min. If .NET reuses a connection after 4 min, LB already dropped the mapping -> timeout on next handshake
Also set DOTNET_SYSTEM_NET_HTTP_SOCKETSHTTPHANDLER_HTTP2SUPPORT=false temporarily to test if HTTP/2 multiplexing is aggravating it.
If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.
@SUNOJ KUMAR YELURU just to let you know, I applied the solution 1 as you suggested. But it gave me the below error:
unrecognized arguments: --load-balancer-managed-outbound-ports 16000
Then, I removed that argument, and it worked. Hope this should work in my case.
If you remove it, AKS uses defaults (∼1024 ports per node).
If you have 10 nodes and want 16000 ports per node, you need 4 IPs: 10 nodes * 16000 ports = 160k ports needed / 64k per IP = 3 IPs minimum
Don't use 16000 with 1 IP, it will fail validation. Use the calculation above.
For a permanent fix for a private cluster with heavy outbound, migrate to NAT Gateway:
To follow-up, Please let us know if you have further query on this and don’t forget to Accept the answer
@SUNOJ KUMAR YELURU I just got an update from the team that this issue is found even if there is very light load, in which case there are sufficient ports available.