ConversationTranscriber compatibility with Custom Speech

Hello Tabarcea, Georgiana,

Greetings! Thanks for raising this question in the Q&A forum.

The documentation listing endpointId as a property on ConversationTranscriber is technically accurate in that the property exists and can be set, but it does not behave the same way it does on SpeechRecognizer. In practice, ConversationTranscriber does not respect a custom endpoint for a Custom Speech model. It continues to use the default base model endpoint regardless of what you set for endpointId or SpeechConfig.endpointId, and this has been confirmed as a limitation in other Q&A threads reporting the exact same mismatch between the documented property and actual behavior. This is why you are seeing conflicting information: the property is present in the SDK surface, but the underlying real-time diarization pipeline behind ConversationTranscriber was built against the base model service and was never wired up to honor Custom Speech endpoints, unlike SpeechRecognizer.

Here is what this means for your scenario, and your options:

Confirm the behavior in your own testing first, since this is not always called out clearly. Set SpeechConfig.endpointId on your ConversationTranscriber instance to a known custom endpoint, then run a phrase that only your custom model would transcribe correctly (a domain-specific term, product name, or acronym). If it comes back using the base model's interpretation rather than your custom one, that confirms the endpoint is being ignored, consistent with the pattern others have reported.

If you need both diarization and Custom Speech, you cannot get both from a single ConversationTranscriber call today. Your practical options are:

• Run SpeechRecognizer against your Custom Speech endpoint for accurate transcription, and separately run diarization logic (either your own speaker-segmentation approach, or a secondary pass) to attribute segments to speakers. This is more work but keeps your custom vocabulary accuracy.
  • Use ConversationTranscriber for diarization only, accept base model accuracy, and treat any domain-specific vocabulary gaps as a post-processing correction step (for example, a phrase list correction pass, or PhraseListGrammar which is supported on ConversationTranscriber and can help nudge recognition toward known terms without a full custom model).
    • Consider Fast Transcription API or batch transcription, which does support diarization together with custom models for offline/asynchronous scenarios, if your use case does not require real-time output.
    File this as a documentation gap. Since the endpointId property is listed without a caveat that it is not honored on ConversationTranscriber, it is worth reporting through the Speech SDK GitHub repository so the docs page gets a note added, since other developers are hitting the same confusion.

If this answer helps you kindly accept the answer which will help others who have similar questions.

Best Regards,

Jerald Felix.