what is the secure way to use Azure voice live service(from foundry) from browser

Prasanna Sai Kommineni 90 Reputation points
2026-08-13T21:03:59.2+00:00

My concern is about the token being visible in browser .

We tried relaying the service but avatar features are not that goood from pythin sdk

Azure Speech in Foundry Tools

3 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. SRILAKSHMI C 19,730 Reputation points Microsoft External Staff Moderator
    2026-09-01T17:36:22.63+00:00

    Hello @Prasanna Sai Kommineni

    Thank you for reaching out to Microsoft Q&A.

    Your concern about the authentication token being visible in the browser is valid. The recommended approach depends on whether you are using Voice Live directly or integrating it with Microsoft Foundry Agent Service.

    Authentication for Voice Live

    Microsoft recommends Microsoft Entra ID authentication for Voice Live rather than exposing an API key. For Microsoft Foundry resources, the token should be requested with the https://ai.azure.com/.default scope. The legacy https://cognitiveservices.azure.com/.default scope is also supported.

    For a browser WebSocket connection, the standard browser WebSocket API cannot set the Authorization header during the WebSocket handshake. Voice Live therefore supports passing the Entra bearer token as an Authorization query-string parameter for browser scenarios. Microsoft documents that query-string parameters are encrypted when transmitted over wss://.

    However, this does not mean the token is hidden from the browser. If the browser receives the token, the user can potentially see it through browser developer tools. Therefore, if your security requirement is that the browser should never have access to a service credential, a backend-brokered/proxy architecture is the preferred approach.

    If you are using Foundry Agent Service

    For the current Voice Live Agent Service integration, Microsoft explicitly states that agent mode does not support key-based authentication. Microsoft Entra ID is required for agent invocation. The documented samples use credentials such as DefaultAzureCredential/AzureCliCredential on the application side.

    Therefore, an architecture such as the following is recommended when you want to keep credentials server-side:

    Browser --> Your backend --> Microsoft Entra ID --> Voice Live / Foundry Agent Service

    Your backend can authenticate using a managed identity or another appropriate Entra credential, while the browser communicates with your application rather than holding a long-lived Azure service credential.

    About the 24-hour token

    The token obtained for https://cognitiveservices.azure.com/.default is an Microsoft Entra access token; it should not be treated as a Voice Live API key. Its lifetime is controlled by Microsoft Entra ID and applicable token policies.

    If the browser receives an access token, it should be treated as sensitive and kept only for the required session. Do not store Azure API keys or long-lived service credentials in browser code or persistent client-side storage.

    Recommended options

    1. Direct browser WebSocket

    Use Microsoft Entra ID and pass the bearer token through the documented Voice Live browser WebSocket authentication mechanism.

    This is supported, but the token is necessarily available to the browser.

    2. Backend broker/proxy - preferred when token exposure is unacceptable

    Keep the Azure credential on your backend and have the backend establish/broker the Voice Live connection. This prevents the browser from receiving the Azure service credential.

    3. WebRTC

    For browser-based real-time scenarios, WebRTC can also be considered. Microsoft documents browser-oriented real-time architectures where the server can procure session credentials and the browser receives only the credential/session information necessary for the connection.

    Since you also mentioned that the Python SDK does not provide the avatar experience you need, you don't necessarily need to abandon the browser Voice Live approach. Voice Live itself supports avatar-related WebSocket events, including session.avatar.connect, so the browser-based Voice Live API can be used for scenarios requiring avatar functionality.

    Using an Entra ID token directly in the browser is supported by the Voice Live API, but the token will be visible to the browser because the browser is using it. If your requirement is specifically "the browser must never see the Azure credential", use a backend-brokered architecture instead of exposing the token to JavaScript.

    Please refer this Microsoft documentation:

    I Hope this helps. Do let me know if you have any further queries.

    If this answers your query, please do click Accept Answer and Yes for was this answer helpful.

    Thank you!

    Was this answer helpful?

    0 comments
  2. Prasanna Sai Kommineni 90 Reputation points
    2026-08-13T21:06:21.7466667+00:00

    I am generating a token for https://cognitiveservices.azure.com/.default. but it is valid for 24 hours it is not short lived

    Was this answer helpful?

    0 comments

  3. AI answer

    2026-08-13T21:04:14.96+00:00

    The most secure browser pattern is to avoid exposing an API key and avoid giving the browser a long-lived Microsoft Entra access token.

    For Voice Live from Microsoft Foundry:

    1. Use Microsoft Entra ID authentication rather than an API key when possible. Voice Live supports Entra ID and API key authentication, and Entra ID is the recommended method.
    2. If the client is a browser, do not rely on connection headers for auth. For browser scenarios, Voice Live supports passing auth on the WebSocket request URI because prehandshake headers are not available in browsers.
    3. Use a backend token broker/service:
      • The backend acquires the Entra token with scope https://ai.azure.com/.default or legacy https://cognitiveservices.azure.com/.default.
      • The backend returns only a short-lived token or otherwise brokers session setup.
      • The browser uses that token on the Voice Live WebSocket request.
    4. Keep tokens in memory only in the browser. Do not store them in persistent browser storage.
    5. If stronger isolation is needed, use a backend to proxy session negotiation so the browser does not directly hold the token. A related real-time audio pattern explicitly calls out proxying browser session negotiation through a service as more secure because the browser does not have access to the ephemeral token.

    For the WebSocket connection itself:

    • Use the Voice Live endpoint:
      • wss://<your-ai-foundry-resource-name>.services.ai.azure.com/voice-live/realtime?api-version=2026-04-10
    • In browser scenarios, auth can be passed as a query parameter:
      • Authorization=Bearer <token> for Entra ID, URL-encoded as needed
      • or api-key=<key>
    • Query string parameters are encrypted by the wss:// transport, but that does not remove the browser exposure concern if the token is present in client code or browser tooling.

    Given the concern that the token is visible in the browser, the recommended practical approach is:

    • Best security: backend-brokered flow so the browser does not directly hold a reusable credential.
    • Acceptable direct browser approach: Entra token instead of API key, short-lived, scoped correctly, and kept only in memory.
    • Least preferred: putting an API key in the browser.

    Also note:

    • Voice Live is optimized for Microsoft Foundry resources.
    • Azure AI Speech resources do not support Microsoft Foundry Agent Service integration or bring-your-own-model for Voice Live.
    AI-generated content may be incorrect. Read our transparency notes for more information.

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.