An Azure service that is used to deploy cloud workloads to run on internet of things (IoT) edge devices via standard containers.
Hello Bocar KANE,
Greetings! Thanks for raising this question in the Q&A forum.
This is a very specific and technically precise issue you have described. The truncated 44-character symmetric key in config.toml instead of the expected 88-character key is the direct cause of the persistent 401 on every DPS provisioning attempt, because the key is being written incomplete and DPS rejects it. Let me explain the root cause and the correct path to resolution.
Why this is happening
The 44-character versus 88-character discrepancy is a classic sign of a base64 encoding truncation during the provisioning call that fetches the symmetric key from the MCC backend. The provisionmcc.sh script calls out to the Azure MCC provisioning service to retrieve the symmetric key that gets written into /etc/aziot/config.toml. If that outbound HTTPS call is partially intercepted, inspected, or NAT-translated by a network device in a way that truncates the response body, the script writes only the first half of the key it received 44 characters instead of 88 and then proceeds to configure aziot with this broken value.
The BGP routing change on June 15 is the critical clue here. A BGP change that alters the egress path from your MCC server can cause traffic to flow through a different firewall, DPI appliance, or TLS inspection proxy that it was not previously traversing. If that device performs any response body modification, SSL inspection, or content filtering on HTTPS responses, the key retrieval call from provisionmcc.sh will receive a truncated payload. Since you have reproduced this consistently across multiple uninstall and reinstall cycles with fresh registration keys, the problem is almost certainly not in the key itself or the portal, but in the network path the provisioning script takes to reach the MCC backend APIs.
Here are the steps to diagnose and resolve this:
Step 1: Verify the exact network path from the MCC server post-BGP change
On the MCC server, trace the route to the MCC provisioning endpoint to see which path is now being used after the BGP change:
traceroute -T -p 443 management.azure.com
traceroute -T -p 443 global.azure-devices-provisioning.net
Compare this output with what the route looked like before June 15. If the BGP change has caused traffic to flow through a new transit or internal network device, that device is the likely culprit.
Step 2: Check for TLS inspection or proxy on the new network path
A network element that performs certificate validation or TLS inspection on the connection can interfere with the provisioning script's HTTPS calls, causing truncated or malformed responses. Run the following to check if the connection to the provisioning endpoint is being intercepted:
curl -v https://global.azure-devices-provisioning.net 2>&1 | grep -E "issuer|subject|SSL"
If the certificate issuer shown is your ISP's internal CA or a DPI vendor (e.g., Cisco, Palo Alto, Zscaler) rather than DigiCert or Microsoft, then TLS inspection is the cause of the truncation. The provisioning script does not support TLS-inspecting proxies.
Step 3: Update Docker's DNS as the first immediate fix to try
The MCC support documentation recommends this as the first step for connectivity issues after provisioning failures. Updating Docker's DNS can help resolve connectivity issues. Try updating to Google's public DNS resolver:
nano /etc/docker/daemon.json
Update the file to:
{
"log-driver": "json-file",
"log-opts": {"max-size": "10m", "max-file": "3"},
"dns": ["8.8.8.8", "8.8.4.4"]
}
Then restart Docker and rerun iotedge check:
systemctl restart docker
iotedge check --verbose
Step 4: Manually inspect and correct the truncated key in config.toml
While you investigate the root network cause, you can attempt a manual repair to verify whether a full-length key works. First, retrieve the correct full 88-character symmetric key from the Azure portal:
- Go to Azure Portal > Your MCC Resource > Settings > Cache Nodes
- Select your cache node and look for the Registration Key or Symmetric Key values
- Copy the full key value
Then manually edit /etc/aziot/config.toml on the server:
sudo nano /etc/aziot/config.toml
Find the symmetric_key line and replace the 44-character truncated value with the full 88-character key. Save the file, then restart the IoT identity service:
sudo aziotctl config apply
sudo systemctl restart aziot-keyd aziot-certd aziot-identityd aziot-tpmd
sudo iotedge system restart
Then check the status:
iotedge check --verbose
If the node comes up healthy with the manually corrected key, this confirms the truncation is happening during the provisioning script's API call, not in the portal.
Step 5: Use the Diagnose and Solve Problems tool in the portal
If the issue persists, use the Diagnose and solve problems functionality within your Connected Cache resource in the Azure portal. You can find this on the left pane within your Connected Cache resource. Select Troubleshoot under the type of problem you are facing and follow the prompts.
Step 6: Escalate to Microsoft via a support ticket
Since this issue started with a specific BGP routing change and the provisioning script is behaving consistently incorrectly across multiple fresh registration keys, this is very likely a network-path issue specific to your ISP environment after the routing change not a general MCC bug. The MCC team at Microsoft needs to be involved to assist further.
Raise a support ticket via Azure Portal > Help + Support > New support request, selecting Microsoft Connected Cache as the service. Include:
- The date the issue started (June 15, 2026)
- That the BGP routing change preceded the failure
- The truncated 44-character key being written to
config.toml - The iotedge check verbose output
- Your cache node ID and the 401 error correlation from the DPS provisioning logs
You can also reach the MCC ISP team directly at mccforispsupport@microsoft.com which is the dedicated support channel for ISP MCC deployments.
If this answer helps you kindly accept the answer which will help others who have similar questions.
Best Regards,
Jerald Felix.