Azure Stream Analytic at-least-once guarantees is not 100% accurate

Anonymous
2023-08-22T01:58:01.4166667+00:00

I have Azure Stream Analytic (ASA) that process data from the event hub as input.

And this job is for the finance biz so it is heavily based on Event Delivery Guarantees at least once of Azure Stream Analytic. But recently we found out that some data were not processed from the event hub and it leads to data becoming inconsistent.

The reliability of ASA can refer as below with 2 ponts:

  1. Azure Stream Analytics guarantees exactly-once event processing and at-least-once delivery of events, so events are never lost.
  2. Stream Analytics guarantees event processing with a 99.9% availability at a minute level of granularity.

https://learn.microsoft.com/en-us/azure/stream-analytics/stream-analytics-introduction#reliability

User's image

But when I asking support from the Microsoft team, a supporter from MINDTREE concluded that:

1)The probably cause of event loss is due to sudden outrage in backend issue from event hub side.  We would like to let you know that as this is a vast cloud platform and these kind of intermittent issues occurs once in a while but could happen which are very rare and our development team would be always working on improving the product to avoid such issues. Hence, the product gives 99.9% SLA for failures and at least-once concept as well, sharing the below document for your reference: https://learn.microsoft.com/en-us/azure/stream-analytics/streaming-technologies.

Is that statement correct? , as far as I know, SLA is for availability and we also have not come across any downtime information for ASA. Furthermore, during the same time frame as the missing data incident, there were a lot of successfully processed data.

And at-least-once guarantees are not 100% accurate is meanless, cause we can't rely on this service

Azure Event Hubs
Azure Stream Analytics
Azure Stream Analytics

An Azure real-time analytics service designed for mission-critical workloads.

1 answer

Sort by: Newest
Most helpful Newest Oldest
  1. Christos Panagiotidis 3,551 Reputation points
    2026-07-20T13:31:22.9766667+00:00

    Microsoft’s current documentation states that Stream Analytics provides exactly-once event processing and at-least-once delivery; 99.9% is an availability commitment. They are not interchangeable: at-least-once permits duplicate output attempts, not a percentage of lost valid events.

    For the incident window, compare Event Hubs incoming-message and partition data with Stream Analytics Input Events, Output Events, backlog, watermark delay, conversion errors, and runtime errors. Enable Execution resource logs. Check malformed input, partition-count changes, receiver conflicts, query filters, TIMESTAMP BY late or out-of-order policy, and output authentication or throttling. Malformed or policy-dropped events should create warnings or logs.

    For a financial pipeline, retain or replay the source, use a dedicated consumer group, and make the sink idempotent with unique keys. If a valid retained Event Hubs event lacks processing or output evidence, give Microsoft Support its UTC time, partition, offset, job name, and logs; backend causation cannot be established from the SLA alone.

    Was this answer helpful?

    0 comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.