The voice of Azure OpenAI is robotic. How to have customer voice in Realtime 4o Mini

Hello @Divjot Singh

Thank you for reaching out to Microsoft Q&A,

What you are observing is expected with the current Azure OpenAI Realtime API voice capabilities.

Current Voice Options in Azure OpenAI Realtime

The gpt-4o-mini-realtime model currently supports six built-in voices:

• alloy

echo fable onyx nova shimmer

These voices are optimized for low-latency, real-time conversations. However, they currently offer limited regional accent coverage, and none are specifically tuned for Indian English. As a result:

Indian-accented voice output is not natively available Prompting can sometimes influence pronunciation temporarily Accent and speaking style may drift over time and revert to the default voice characteristics

This is why prompt-based attempts to maintain an Indian accent are often inconsistent across longer conversations.

Azure AI Speech for Voice Synthesis

For telephony and customer-facing voice applications, the recommended approach is to use:

Azure OpenAI Realtime API for conversation handling and reasoning Azure AI Speech Text-to-Speech (TTS) for voice synthesis

This hybrid architecture provides a much more natural, consistent, and regionally appropriate voice experience.

Why Azure AI Speech Is the Better Choice

Azure AI Speech offers:

More than 500 prebuilt neural voices Multiple Indian English and Hindi voices High-quality, natural speech synthesis SSML support for fine-grained control over pronunciation, pitch, rate, and pauses Low latency, typically suitable for real-time conversational scenarios

Examples of Indian voices include:

en-IN-NeerjaNeural en-IN-PrabhatNeural en-IN-PriyaNeural hi-IN-SwaraNeural

These voices provide significantly better Indian pronunciation and a more natural customer experience.

Integration with Your Existing Realtime Workflow

You can easily integrate Azure Speech into your current solution by replacing only the text-to-speech portion of the pipeline:

Caller speaks → Azure OpenAI Realtime processes audio and generates text → Azure AI Speech synthesizes the response → audio is played back to the caller

This allows you to retain the conversational intelligence of Realtime while delivering a superior voice experience.

The Speech service can be accessed via:

REST API Speech SDK for C#, Python, JavaScript, and other languages

It also supports streaming over WebRTC or SIP, making it well suited for telephony scenarios.

Option for a Fully Customized Brand Voice

If you require a unique, branded, or highly specific Indian-accented voice, Azure AI Speech also offers Custom Neural Voice

With Custom Neural Voice, you can:

Provide approximately 1–2 hours of professionally recorded speech samples Train a custom voice model tailored to your brand or spokesperson Deploy the voice through Azure Speech for production use

This is ideal for organizations that want a distinctive and consistent voice identity.

Please refer this

What are OpenAI text to speech voices? (voice comparison table) https://learn.microsoft.com/azure/ai-services/speech-service/openai-voices?wt.mc_id=knowledgesearch_inproduct_azure-cxp-community-insider#openai-text-to-speech-voices-via-azure-openai-or-via-azure-speech

Neural voices supported by Azure Speech https://docs.microsoft.com/azure/cognitive-services/speech-service/language-support#neural-voices

Get started with Custom Neural Voice https://docs.microsoft.com/azure/cognitive-services/speech-service/how-to-custom-voice

Create and use your Custom Neural Voice model https://docs.microsoft.com/azure/cognitive-services/speech-service/how-to-custom-voice-create-voice

I hope this helps, do let me know if you have further queries.

Thank you!

I think 4o‑mini’s voice layer is wired into the model itself, u just can’t drop a new one in…so the only steady path is lettin’ it spit txt and havin’ Azure Speech en‑IN speak it on the line.

Hi @Divjot Singh

Did you get any chance to review the above response. Do let me know if you have any further queries.

Thank you!

Thanks for inputs.

We have already adopted the approach of using Azure speech layer with 4o mini text output.

But it is causing issues of latency. As sending the response to TTS engine and generating voice from it adds some latency.

Hi @Divjot Singh

Thank you for the update. I understand your concern regarding the additional latency introduced when using Azure AI Speech for text-to-speech synthesis.

This is a common tradeoff when choosing between the built-in Realtime voices and external neural voice synthesis.

Why This Happens

The built-in voices in gpt-4o-mini-realtime are tightly integrated with the model, which is why they offer very low latency. However, as you’ve observed, they currently provide only a limited set of voices and do not yet include region-specific options such as Indian English.

When you use Azure AI Speech as a separate TTS layer, the workflow becomes:

User audio → Realtime model → text generation → Azure Speech TTS → audio playback

This additional synthesis step naturally introduces some extra latency compared to the native Realtime voices.

Current Recommendation

At this time, there is no supported way to replace or customize the built-in Realtime voices directly within the gpt-4o-mini-realtime model. The available Realtime voices are fixed and managed by the service.

If your priority is:

Lowest possible latency: the built-in Realtime voices remain the best option.

Higher voice quality and regional accent fidelity: Azure AI Speech is the recommended approach.

Options to Minimize TTS Latency

You can often reduce the added latency by optimizing your Speech integration:

Use the Speech SDK streaming APIs instead of waiting for the entire text response before synthesis.

Enable text streaming from Azure OpenAI so TTS can begin synthesizing partial responses as soon as tokens are generated.

Use low-latency neural voices optimized for real-time scenarios.

Deploy both Azure OpenAI and Azure AI Speech in the same Azure region to minimize network round-trip time.

Reuse persistent connections for both services rather than creating new connections per request.

With streaming enabled end-to-end, many customers achieve significantly lower perceived latency.

Best Practice for Telephony Scenarios

For production voice applications, the recommended architecture is:

Caller → Speech-to-Text → Azure OpenAI Realtime/Text Model → Streaming Azure AI Speech TTS → Caller

This balances:

• Natural, regionally appropriate voice quality
• Consistent accent and pronunciation
• Acceptable real-time responsiveness

Built-in Realtime voices cannot currently be customized or replaced.

Indian-accented native Realtime voices are not yet available.

Azure AI Speech remains the best option for natural Indian voices.

Latency can be significantly reduced by implementing a fully streaming pipeline.

Thank you!