CRITICAL ISSUE Azure AI Speech SDK – Numbers getting Added , Deleted and Substituted and sometimes Exceeds too much time while using the microsoft realtime speech to text conginitve services API

Hi Aravind,​

Good day. Extremely sorry for replying late.

I have optimized my previous python code to handle audio stream issue too. Output format

[LEXICAL] one one one one one two three four three two one one two three four five four three two one four three two one [DISPLAY] 111112343211234543214321. [CONFIDENCE] 0.820 [CLEAN] 111112343211234543214321 [SESSION STOPPED] [FINAL TRANSCRIPT] 111112343211234543214321

Fixes applied

Fix 1 — Disable ITN (tackles phone-number formatting) ITN (Inverse Text Normalization) is the Azure layer that was converting "one one one" into "1-111-111-1111". Disabling it via set_service_property stops Azure from ever applying phone/date/number formatting patterns.

Fix 2 — Use Detailed Output Format (enables Lexical access) Switched from the default simple output to OutputFormat.Detailed, which exposes the raw NBest JSON array containing the Lexical field — the unformatted word-by-word transcript before any Azure post-processing.

Fix 3 — Tune Silence Timeouts (tackles mid-number cutoffs) Two timeouts were increased — segmentation silence to 2000ms and initial silence to 5000ms. This prevents Azure from prematurely cutting off a recognition segment while the user is still speaking digits, which was causing deletions and substitutions.

Fix 4 — Read Lexical Field Instead of DisplayText (core bypass) Instead of using evt.result.text (which is ITN-formatted), the code now parses NBest[0]["Lexical"] directly from the JSON. This gives raw spoken words like "one two three" before Azure distorts them.

Fix 5 — Custom Digit Word Mapping (fully controlled conversion) A convert_spoken_digits() function replaces Azure's ITN entirely for numeric content. It uses a deterministic word-to-digit dictionary — including common mishears like "to"→2, "won"→1, "niner"→9 — so the conversion is predictable and auditable, not a black box.

#Cause- Timeout -issue
import azure.cognitiveservices.speech as speechsdk
import wave
import time

speech_key = "****RUOgVBYmdSmBVZ5uU0HLfoiPMuaX1t4MavwUH8JQQJ99CDAC1i4TkXJ3w3AAAYACOGvqe6"
speech_endpoint = "https://centralus.api.cognitive.microsoft.com/"

speech_config = speechsdk.SpeechConfig(subscription=speech_key, endpoint=speech_endpoint)

# Optional: Set output format for better accuracy
speech_config.speech_recognition_language = "en-US"
speech_config.set_property(
    speechsdk.PropertyId.Speech_SegmentationSilenceTimeoutMs, "1500"
)


def from_audio_stream(audio_file_path: str):
    """
    Recognize speech from an audio file stream.
    Replaces microphone input with a push/pull audio stream.
    Handles: word substitutions, deletions, insertions, and timeout issues.
    """

    # ── 1. Audio stream config (must match your WAV: 16kHz, 16-bit, mono) ──
    stream_format = speechsdk.audio.AudioStreamFormat(
        samples_per_second=16000,
        bits_per_sample=16,
        channels=1
    )

    # ── 2. PushAudioInputStream: you control when bytes are fed ──
    push_stream = speechsdk.audio.PushAudioInputStream(stream_format)
    audio_config  = speechsdk.audio.AudioConfig(stream=push_stream)

    # ── 3. Recognizer wired to the stream ──
    recognizer = speechsdk.SpeechRecognizer(
        speech_config=speech_config,
        audio_config=audio_config
    )

    results        = []
    done_event     = False

    # ── 4. Callbacks ──
    def recognized_cb(evt):
        if evt.result.reason == speechsdk.ResultReason.RecognizedSpeech:
            print(f"[RECOGNIZED] {evt.result.text}")
            results.append(evt.result.text)
        elif evt.result.reason == speechsdk.ResultReason.NoMatch:
            print(f"[NO MATCH]   {evt.result.no_match_details}")

    def recognizing_cb(evt):
        # Interim / partial result – useful for debugging substitutions
        print(f"[PARTIAL]    {evt.result.text}", end="\r")

    def canceled_cb(evt):
        nonlocal done_event
        print(f"\n[CANCELED]   Reason : {evt.reason}")
        if evt.reason == speechsdk.CancellationReason.Error:
            print(f"             Code   : {evt.error_code}")
            print(f"             Detail : {evt.error_details}")
        done_event = True

    def session_stopped_cb(evt):
        nonlocal done_event
        print("\n[SESSION STOPPED]")
        done_event = True

    recognizer.recognized.connect(recognized_cb)
    recognizer.recognizing.connect(recognizing_cb)
    recognizer.canceled.connect(canceled_cb)
    recognizer.session_stopped.connect(session_stopped_cb)

    # ── 5. Start continuous recognition ──
    recognizer.start_continuous_recognition()

    # ── 6. Push audio bytes in chunks (mimics real-time streaming) ──
    chunk_size = 3200  # 100 ms worth of audio at 16kHz/16-bit/mono

    with wave.open(audio_file_path, "rb") as wf:
        assert wf.getframerate()    == 16000, "WAV must be 16 000 Hz"
        assert wf.getsampwidth()    == 2,     "WAV must be 16-bit"
        assert wf.getnchannels()    == 1,     "WAV must be mono"

        while True:
            data = wf.readframes(chunk_size // 2)   # readframes takes frame count
            if not data:
                break
            push_stream.write(data)
            time.sleep(0.1)   # pace the feed; remove for as-fast-as-possible

    # ── 7. Signal end-of-stream, then wait ──
    push_stream.close()

    timeout = 30  # seconds to wait after all audio is pushed
    elapsed = 0
    while not done_event and elapsed < timeout:
        time.sleep(0.5)
        elapsed += 0.5

    recognizer.stop_continuous_recognition()

    # ── 8. Return combined transcript ──
    full_transcript = " ".join(results)
    print(f"\n[TRANSCRIPT] {full_transcript}")
    return full_transcript


# ── Entry point ──
if __name__ == "__main__":
    from_audio_stream("./ITNfixwithnumber.wav")

Have shared working code and files via email for reference.

Thank you