OutGoing message more than InComing messages in EventHub

Question

OutGoing message more than InComing messages in EventHub

Mahendra Sawarkar 60

I am using Kafka Protocol for polling the records from EventHub

Below is the dependency I am using

        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-clients</artifactId>
            <version>3.9.1</version>
            <exclusions>
                <exclusion>
                    <groupId>org.xerial.snappy</groupId>
                    <artifactId>snappy-java</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

For each partition i am spawning the new java consumer thread. But the partition assignment for each java consumer thread is automatic. (we are not manually assigning the partition to KafkaConsumer)

After a small load test we could see below graph in the EventHub monitoring

As you can see in the image around ~5M messages are consumed again

Screenshot 2026-09-02 at 12.40.28 PM

What could be issue?

Himaja Y 375 Reputation points Microsoft External Staff Moderator

2026-09-03T14:29:00.68+00:00

Hi @Mahendra Sawarkar ,

I hope you had a chance to review the information shared earlier, and I hope this information has been helpful! If you still have questions, please let us know what is needed in the comments so the question can be answered.

3 answers

Your answer

Himaja Y 375 Reputation points Microsoft External Staff Moderator

2026-09-03T14:29:00.68+00:00

Hi @Mahendra Sawarkar ,

I hope you had a chance to review the information shared earlier, and I hope this information has been helpful! If you still have questions, please let us know what is needed in the comments so the question can be answered.

Answer 1

Welcome to Microsoft Q&A,

@Mahendra Sawarkar I hope you are doing well,

The higher outgoing message count does not necessarily mean Event Hubs is generating duplicate messages. Since you are consuming through the Kafka protocol, I would first check the consumer offset and rebalance behavior.

Event Hubs provides at-least-once delivery, so a record can be delivered again if the consumer loses its partition assignment or the offset is not committed after processing.

Since partition assignment is automatic, I recommend checking:

Make sure all consumer threads that should share the workload use the same group.id.
Check for frequent consumer rebalances.
Review max.poll.interval.ms if message processing takes significant time.
Verify your offset commit configuration (enable.auto.commit, commitSync, or commitAsync) and ensure offsets are not committed before processing completes.
Log the Event Hubs/Kafka partition and offset for each processed record. If the same partition/offset is processed multiple times, that confirms reprocessing rather than new messages.

Also, having one consumer thread per partition is not inherently incorrect. Kafka will assign the partitions automatically within the consumer group.

Microsoft documentation: https://learn.microsoft.com/en-us/azure/event-hubs/azure-event-hubs-apache-kafka-overview https://learn.microsoft.com/en-us/azure/event-hubs/apache-kafka-configurations https://learn.microsoft.com/en-us/azure/event-hubs/apache-kafka-troubleshooting-guide

Answer 2

Hi @Mahendra Sawarkar ,

Thank you for reaching out to the Microsoft Q&A forum.

The most likely cause is that the consumers are re-reading events rather than Event Hubs generating extra messages.

Since you are using automatic partition assignment, Kafka consumer group rebalancing can occur when consumer threads start, stop, or lose connection. If offsets are not committed successfully before a rebalance or restart, the consumer will read messages again from the last committed offset.

Key things to check:

Verify offset commits (enable.auto.commit or manual commits).
Check consumer logs for rebalance events.
Ensure all consumers use the same consumer group ID.
Verify that the number of consumer threads does not exceed the number of Event Hub partitions.
Review application restarts or connection interruptions during the load test.

Based on the graph, the spike in Outgoing Messages is most likely due to duplicate consumption caused by consumer rebalancing or offset replay, resulting in approximately 5 million events being consumed again.

Answer 3

Mahendra Sawarkar 60

thank you @Himaja Y @Jose Benjamin Solis Nolasco

The information you shared was of really help.

0 comments