Clarification on Kafka Transactional API Support and Transaction Guarantees in Azure Event Hubs

Question

Clarification on Kafka Transactional API Support and Transaction Guarantees in Azure Event Hubs

Priyanka Sundararajan 0

The Kafka Transactional API is documented as being supported by Azure Event Hubs - https://learn.microsoft.com/en-us/azure/event-hubs/apache-kafka-transactions. In Apache Kafka, transactions are managed by the Transaction Coordinator, which maintains transaction state and persists transaction metadata in the internal __transaction_state topic.

How is this maintained in Azure Event Hubs when using the Kafka protocol?

Does Event Hubs have an equivalent Transaction Coordinator or Transaction Manager? How is transaction state maintained internally?
Is there an equivalent mechanism to Kafka's __transaction_state topic for persisting transaction metadata?

Is there detailed technical documentation available describing the transaction architecture and guarantees in Azure Event Hubs while using Kafka Transactional API? We are particularly interested in documentation covering the transaction lifecycle, transaction state management, persistence, commit/abort behavior, failure recovery, and consumer visibility of transactional records.

We are looking for information comparable to Kafka's transaction architecture documentation, which explains how the Transaction Coordinator maintains transaction state and how transactional guarantees are enforced.

Any guidance, documentation, architecture references, or insights from the Azure Event Hubs team or community would be greatly appreciated.

Peram Venkateswara Reddy 260 Reputation points Microsoft External Staff Moderator

2026-08-11T12:41:01.34+00:00
Hi @Priyanka Sundararajan ,

Based on the architecture is not publicly documented, and there is no Event Hubs equivalent of Kafka's transaction-architecture documentation covering coordinator internals, __transaction_state, or state-persistence mechanics. Event Hubs exposes a Kafka protocol endpoint over its own service implementation rather than running Kafka brokers, so Kafka's internal topics aren't part of the surface area you can inspect. Nothing in the published documentation set describes what replaces them internally.

Since that's the case, here is what you can build a design against.

Feature status — read this first

From the transactions article: "Kafka Transactions is currently in Public preview in Premium, and Dedicated tier." Two implications for your planning:

It is not GA, so it is not covered by a production SLA for the transactional behavior specifically.

It is unavailable on Basic and Standard — Premium or Dedicated only.

The documented contract

Transactional producer requires enable.idempotence=true and a unique transactional.id; register with initTransactions(), then beginTransaction() / commitTransaction() / abortTransaction().

Exactly-once requires consumers set isolation.level=read_committed, and the producer must call sendOffsetsToTransaction(offsetsToCommit, group) before committing.

Timeout: "If the transaction is not committed or aborted before the max.transaction.timeout.ms, the transaction is aborted by Event Hubs automatically. The default max.transaction.timeout.ms is set to 15 minutes by Event Hubs, but the producer can override it to a lower value by setting the transaction.timeout.ms property."

A known behavioral divergence you should factor into your design

This is probably the most useful thing I can give you, because it speaks directly to your questions about commit/abort behavior, failure recovery, and consumer visibility.

There's an existing thread — Azure Event Hubs with Kafka, read_committed consumers stuck (Apr 2026) — where a customer ran Event Hubs side by side against Strimzi. Test: send message 1, sleep longer than transaction.timeout.ms, then send message 2 on the same producer.

Strimzi: the second send is rejected with InvalidProducerEpochException. Broker state stays consistent.

Event Hubs: the second send succeeds. It never receives an abort marker from either the broker or the producer. The result is a message with no transactional marker — the LSO stops advancing and read_committed consumers block indefinitely on that offset.

The customer's read was that "validation of the transactional state appears to occur late, at the EndTxn (commit/abort) stage rather than at Produce time." The Microsoft moderator on that thread described the original blocked-consumer symptom as "not expected" and "a potential service-side issue rather than a client misconfiguration."

If your design depends on producer-fencing being enforced at Produce time — which it is in native Kafka — validate that specific path before committing to the architecture. The mitigations suggested on that thread were to call abortTransaction() explicitly in every failure path and keep transaction.timeout.ms low.

How to get the architecture detail you're actually after

The community can't produce internal design documentation that hasn't been published. Two routes that can:

Azure support ticket against your Event Hubs namespace, asking specifically for transaction-architecture and guarantee detail for a design review. This is the route the engineering team can respond on, and it's what was recommended on the thread above for service-side transaction-state inspection.

Documentation feedback via the Was this page helpful? / feedback control at the bottom of the transactions article — requesting an architecture section covering lifecycle, state persistence, and recovery. Given the feature is in preview, that's a well-timed ask.

References

Transactions in Apache Kafka for Azure Event Hubs — https://learn.microsoft.com/azure/event-hubs/apache-kafka-transactions

What is Azure Event Hubs for Apache Kafka? (conceptual mapping, key differences) — https://learn.microsoft.com/azure/event-hubs/azure-event-hubs-apache-kafka-overview

Apache Kafka troubleshooting guide for Event Hubs (documented Kafka-vs-Event Hubs divergences) — https://learn.microsoft.com/azure/event-hubs/apache-kafka-troubleshooting-guide

Migrate to Azure Event Hubs for Apache Kafka (feature compatibility table) — https://learn.microsoft.com/azure/event-hubs/apache-kafka-migration-guide

If you can share which tier you're on and whether you're evaluating this for a new design or troubleshooting existing behavior, I can point you more precisely.

Kind Regards,
Microsoft Support Team.

1 answer

Your answer

Answer 1

Hi @Priyanka Sundararajan
In addition to the information already provided, a few practical points may be useful when evaluating Kafka transactions with Azure Event Hubs:- Azure Event Hubs provides Kafka protocol compatibility, but it is not a Kafka broker deployment. Therefore, Kafka broker internals such as __transaction_state and the implementation details of the Transaction Coordinator are not exposed to customers.

The application should rely on the documented Event Hubs transaction behavior and configuration, rather than depending on Kafka-internal topics or broker implementation details.
For transactional producers, ensure enable.idempotence=true and configure a unique transactional.id.
Consumers that need transactional visibility should use isolation.level=read_committed.
Applications should explicitly handle transaction failures and call abortTransaction() where appropriate rather than relying only on transaction timeout behavior.
The documented maximum transaction timeout is 15 minutes. The producer can configure a lower transaction.timeout.ms value.
Kafka Transactions in Event Hubs are currently Public Preview and available on Premium and Dedicated tiers, so please consider the preview status when evaluating the feature for production workloads.

For your specific questions regarding the internal transaction coordinator, transaction-state persistence, and an equivalent of Kafka's __transaction_state topic, there is currently no public Azure Event Hubs documentation describing those internal implementation details.

If you are evaluating this for a specific application architecture, the Event Hubs tier, Kafka client/library version, producer/consumer configuration, and whether exactly once processing is required would be useful to determine the applicable guidance.

I hope this helps. Please let me know if you have any further questions.