Event Grid AzureFunction destination returns intermittent Unauthorized on Flex Consumption plan, zero App Insights traces, works fine on Consumption plan

Question

Event Grid AzureFunction destination returns intermittent Unauthorized on Flex Consumption plan, zero App Insights traces, works fine on Consumption plan

Umar Al-Faruq 0

Environment

Azure Function App plan: Flex Consumption
Trigger type: EventGridTrigger
Event Grid destination type: AzureFunction (resource-based, resourceId pointing to function, no embedded ?code= key)
Event Grid topic type: Custom Topic
Event delivery schema: CloudEventSchemaV1_0
AzureWebJobsStorage: full connection string (not managed identity)
alwaysReadyInstances: 0 (default)

Problem

After migrating a Function App from Consumption plan to Flex Consumption plan, Event Grid deliveries to an EventGridTrigger function begin failing intermittently with outcome=Unauthorized. The same setup worked correctly on the Consumption plan with no changes to the Event Grid subscription or function code.

Example from Event Grid delivery failure log:

outcome=Unauthorized, deliveryCount=6, probationCount=4,
state=FilteredFailingDelivery, deliveryResponse=Unauthorized,
errorCode=Unauthorized, StatusCode=Unauthorized(401)
armId=/subscriptions/.../providers/Microsoft.Web/sites/<app-name>/functions/<function-name>

The failures are intermittent some delivery attempts succeed and some fail with Unauthorized for the same event subscription and function, with no pattern tied to event type or subject.

What I already tried:

Deleted and recreated the Event Grid subscription
Checked Application Insights for any trace of the failed deliveries. Zero traces exist for the failing invocations. Successful deliveries show traces normally. This means the requests are being rejected before reaching the Functions host pipeline entirely

Siddhesh Desai 8,210 Reputation points Microsoft External Staff Moderator

2026-07-02T11:55:30.2+00:00

Hi @Umar Al-Faruq

Thanks for sharing details on Private chat.

I have emailed you from the ticket I have created with the backend team. I'll keep you updated on the solution.

2 answers

Your answer

Siddhesh Desai 8,210 Reputation points Microsoft External Staff Moderator

2026-07-02T11:55:30.2+00:00

Hi @Umar Al-Faruq

Thanks for sharing details on Private chat.

I have emailed you from the ticket I have created with the backend team. I'll keep you updated on the solution.

Answer 1

Hi @Umar Al-Faruq

Thank you for reaching out to Microsoft Q&A.

I received a response from the backend team, and this is a platform side issue reported by multiple customers.

Please find the Root cause and recommended workaround for you to resolve this issue:

Root Cause

The root cause is a system key synchronization race condition on Flex Consumption cold starts:

With alwaysReadyInstances: 0, the app scales to zero between events

When Event Grid delivers an event, FPS allocates a new pod (cold start)

During pod startup, the Functions host must load the eventgrid_extension system key from BlobStorageSecretsRepository

The request may arrive at the pod's HTTP endpoint before the system key has been loaded into memory

The webhook authentication middleware cannot validate the system key → returns 401 Unauthorized

Because the 401 occurs before the function is invoked, no App Insights traces are generated for failed deliveries

Why this worked on Consumption plan: Classic Consumption uses a placeholder/warm-standby instance model where keys are pre-loaded. Flex Consumption with alwaysReadyInstances: 0 has no warm instance, so every request after scale-to-zero hits a cold start.

Why it is intermittent: The race depends on timing — if the key loads before the request arrives, the request succeeds (200). If the request arrives first, it fails (401). Additionally, Blob Storage errors during startup exacerbate the issue.

Confidence level: High. Evidence chain: 100% cold-start correlation for 401s + system key loading on every pod startup + Blob Storage write errors during initialization + zero App Insights traces (401 before function invocation) + works on Consumption plan (warm instances).

Recommendations

Immediate workaround: Set alwaysReadyInstances to at least 1 for the http function group. This ensures a warm instance is always available with the system key pre-loaded, eliminating the cold start race condition.

az functionapp config set --name xxxx-xx-xx-txxxt --resource-group txxxxg-txxt \
  --always-ready-instances http=1

Upgrade Node.js runtime: The app is running Node.js v20, which reached EOL in April 2026. Upgrade to Node.js v22 to receive security patches and performance improvements.
Alternative workaround: If setting alwaysReadyInstances is not desired (cost), consider switching the Event Grid subscription destination type from AzureFunction (resource-based) to a Webhook destination with the function URL and ?code= key embedded directly. This may behave differently during key validation.
Set WEBSITE_RUN_FROM_PACKAGE=1: FunctionsLogs show "Set 'WEBSITE_RUN_FROM_PACKAGE' to '1' to significantly improve load times" and "Loading 'index.js' took 1067ms". This will reduce cold start time and narrow the race window.
Platform follow-up: The Flex Consumption platform should ensure system keys are loaded from Blob Storage before marking the pod as ready to receive webhook requests. This is a platform-level gap.

Node.js v20 EOL warnings: FunctionsLogs show repeated "Node.js v20 reached EOL on 2026-04. Please upgrade to a supported version" warnings on every instance.

Answer 2

Flex Consumption supports EventGridTrigger, so this is not a documented plan incompatibility. For an AzureFunction destination, Event Grid obtains and presents the Functions Event Grid system key. A cold start should cause delay or timeout, not a 401.

Verify that the event subscription resourceId and function name match migrated app, and that Microsoft.EventGrid is registered. Recreate the subscription only after the function is deployed and its system key is available. As an isolation test, create a WebHook subscription using the function endpoint and system key. If that is stable, the problem is in resource-based key retrieval or the Functions front end, not your code or AzureWebJobsStorage.

Event Grid places an unauthorized destination into five-minute probation, so failures can cluster. With no Application Insights invocation and mixed success for the same subscription, keep the backend case open and provide UTC samples, event-subscription ID, function resource ID, delivery logs, and correlation IDs.