ASA parallel JOIN drops matches when the two Event Hubs inputs use different partition-key hashing

Question

ASA parallel JOIN drops matches when the two Event Hubs inputs use different partition-key hashing

Andrei Josephsen 0

ASA job (compat 1.2) joins two Event Hubs on EntityId (both 4 partitions) into a third hub. With the input Partition key column declared (embarrassingly-parallel), the JOIN silently loses ~78% of matches — no errors. Measured: parallel ≈ 22% match, non-parallel ≈ 100%.

Cause: the two inputs are produced by clients that hash the key differently, so the same EntityId lands on different partitions: EventHubBufferedProducerClient uses client-side lookup3, EventHubProducerClient/gateway uses another hash, and ASA output uses a third.

Q: Can I force all publishers (buffered producer, regular producer, and ASA output) to use the same partition-key hashing function, so the same EntityId always lands on the same partition and a parallel JOIN matches?

SIVASANKAR YEDDULA 165 Reputation points Microsoft External Staff Moderator

2026-08-25T12:55:26.5066667+00:00

Hi @Andrei Josephsen ,

Thank you for reaching out to the Microsoft Q&A forum.

To investigate this issue further, I have sent you a private message. Could you please provide the following details through the private message?

Error or failure screenshots

Your email address

Time zone

Your availability for a Microsoft Teams meeting

These details will help us perform a deeper investigation and engage the appropriate engineering team if required.

Once you've shared the requested information, we will continue our analysis and provide further guidance.
SIVASANKAR YEDDULA 165 Reputation points Microsoft External Staff Moderator

2026-08-26T08:47:01.2333333+00:00

Hi @Andrei Josephsen ,

I hope you are doing well.

This is a gentle follow-up regarding the information requested in my previous message Could you please share the requested details when you have a chance.

This will help us continue our investigation and assist you further.

Thank you for your time and cooperation.
Andrei Josephsen 0 Reputation points

2026-08-26T11:00:46.51+00:00

The reason is fairly clear ... events with a partitionID get routed to different partitions depending on how they were published .. example with four partitions

event with PartitionKey: 8898d388-f719-461f-b26d-2035fbb732f8ends on partition 1 using ASA

ends on partition 2 using sdk EventHub BufferedProducerClient

This basically ruins out combining ASA events with BufferedProducerClient events for high throughput scenarios
SIVASANKAR YEDDULA 165 Reputation points Microsoft External Staff Moderator

2026-08-31T09:55:13.7033333+00:00

Hi @Andrei Josephsen ,

I hope you are doing well.

This is a gentle follow-up regarding the information requested in my previous message Could you please share the requested details when you have a chance.

This will help us continue our investigation and assist you further.

Thank you for your time and cooperation.
SIVASANKAR YEDDULA 165 Reputation points Microsoft External Staff Moderator

2026-09-02T06:45:13.28+00:00

Hi @Andrei Josephsen ,

Thank you for reaching out to the Microsoft Q&A forum.

While reviewing your support request, we were unable to locate an active entitlement/subscription associated with the account, which is preventing us from creating the DFM case.

Could you please confirm whether you have an active subscription linked to the affected resource? If available, kindly share the Subscription ID or any relevant subscription details so that we can proceed further with the case creation.

We look forward to your response.

Thanks & Regards,
SIVASANKAR YEDDULA 165 Reputation points Microsoft External Staff Moderator

2026-09-02T07:13:24.33+00:00

Hi @Andrei Josephsen ,

Thank you for reaching out to the Microsoft Q&A forum.

Based on the current Azure Event Hubs and Azure Stream Analytics design, it is not possible to force EventHubProducerClient, EventHubBufferedProducerClient, and Azure Stream Analytics outputs to use the same partition-key hashing function.

The partition key hash algorithm used to map events to partitions is an internal implementation detail and is not exposed as a configurable setting. As a result, different producers or services may route the same partition key value to different physical partitions.

For Azure Stream Analytics parallel JOINs, the engine assumes that records with the same partition key are co-located in corresponding partitions across all inputs. If the upstream producers use different partitioning behaviors and the same EntityId ends up in different partitions, matching events may never be processed together, causing the reduced match rate you observed.

As a workaround, consider one of the following approaches:

• Disable the partition-key optimization and run the JOIN without declaring the Partition key column. This typically restores correctness but may reduce throughput.

• Ensure both input streams are produced through a single consistent publishing path so that identical EntityId values are partitioned consistently before reaching Stream Analytics.

• Repartition or normalize the streams upstream (for example, through an intermediary processing layer) before they are consumed by the ASA job.

Therefore, there is currently no supported mechanism to configure a common partition-key hashing function across EventHubBufferedProducerClient, EventHubProducerClient, and Azure Stream Analytics output destinations.

If maintaining an embarrassingly parallel JOIN is a requirement, the recommended approach is to ensure partition alignment upstream rather than relying on configurable hashing behavior.
SIVASANKAR YEDDULA 165 Reputation points Microsoft External Staff Moderator

2026-09-09T08:31:04.5566667+00:00

Hi @Andrei Josephsen ,

I hope the information provided has been helpful. If you have any further questions or need additional assistance, please let us know in the comments, and we will be glad to help. If this answer helps, please consider clicking Accept Answer so that other community members with similar questions can benefit as well.

Your answer

SIVASANKAR YEDDULA 165 Reputation points Microsoft External Staff Moderator

2026-08-25T12:55:26.5066667+00:00

Hi @Andrei Josephsen ,

Thank you for reaching out to the Microsoft Q&A forum.

To investigate this issue further, I have sent you a private message. Could you please provide the following details through the private message?

Error or failure screenshots

Your email address

Time zone

Your availability for a Microsoft Teams meeting

These details will help us perform a deeper investigation and engage the appropriate engineering team if required.

Once you've shared the requested information, we will continue our analysis and provide further guidance.
SIVASANKAR YEDDULA 165 Reputation points Microsoft External Staff Moderator

2026-08-26T08:47:01.2333333+00:00

Hi @Andrei Josephsen ,

I hope you are doing well.

This is a gentle follow-up regarding the information requested in my previous message Could you please share the requested details when you have a chance.

This will help us continue our investigation and assist you further.

Thank you for your time and cooperation.
Andrei Josephsen 0 Reputation points

2026-08-26T11:00:46.51+00:00

The reason is fairly clear ... events with a partitionID get routed to different partitions depending on how they were published .. example with four partitions

event with PartitionKey: 8898d388-f719-461f-b26d-2035fbb732f8ends on partition 1 using ASA

ends on partition 2 using sdk EventHub BufferedProducerClient

This basically ruins out combining ASA events with BufferedProducerClient events for high throughput scenarios
SIVASANKAR YEDDULA 165 Reputation points Microsoft External Staff Moderator

2026-08-31T09:55:13.7033333+00:00

Hi @Andrei Josephsen ,

I hope you are doing well.

This is a gentle follow-up regarding the information requested in my previous message Could you please share the requested details when you have a chance.

This will help us continue our investigation and assist you further.

Thank you for your time and cooperation.
SIVASANKAR YEDDULA 165 Reputation points Microsoft External Staff Moderator

2026-09-02T06:45:13.28+00:00

Hi @Andrei Josephsen ,

Thank you for reaching out to the Microsoft Q&A forum.

While reviewing your support request, we were unable to locate an active entitlement/subscription associated with the account, which is preventing us from creating the DFM case.

Could you please confirm whether you have an active subscription linked to the affected resource? If available, kindly share the Subscription ID or any relevant subscription details so that we can proceed further with the case creation.

We look forward to your response.

Thanks & Regards,
SIVASANKAR YEDDULA 165 Reputation points Microsoft External Staff Moderator

2026-09-09T08:31:04.5566667+00:00

Hi @Andrei Josephsen ,

I hope the information provided has been helpful. If you have any further questions or need additional assistance, please let us know in the comments, and we will be glad to help. If this answer helps, please consider clicking Accept Answer so that other community members with similar questions can benefit as well.