Azure AI services multi-service account Conversation Summary is extremely slow (>30 minutes to response)

Eric Richards 91 Reputation points
2026-06-22T15:24:41.2266667+00:00

We've been using the older-style Azure Cognitive Services multi-service account resource to do keyword extraction and sentiment analysis and conversation summarization for several years without issue.

In the past month or so, we've seen that response times from the same resource for these requests, especially the Conversation Summarization API, have exploded.

I know that these are slated for deprecation and retirement by 2029, but even though that is a long time off, they have reached a point where they are effectively unusable. I suspect that I'm going to have to swap this around to using GPT models and prompts to achieve the same capabilities and something approaching reasonable performance.

I am not running particularly large chat conversations through the summarization jobs, but the average time to a response with 5-10 chat messages and various summarization aspects like [Issue, Resolution, Follow-up Tasks, Recap] is averaging 30 minutes to one and a half hours for the past two months. When we originally implemented this, circa October 2024, performance for these kinds of requests was averaging less than 60 seconds.

We have been using Azure.AI.Language.Conversations 2.0.0-beta library as new versions are released. Currently using the latest 2.0.0-beta.5 version from NuGet.

Our Cognitive Services account is the Standard tier, which is the only SKU option that is available to select; I don't see that there is any way to scale it up or out.

Azure Language in Foundry Tools
Azure Language in Foundry Tools

An Azure service that provides natural language capabilities including sentiment analysis, entity extraction, and automated question answering.

1 answer

Sort by: Oldest
Most helpful Newest Oldest
  1. Karnam Venkata Rajeswari 5,255 Reputation points Microsoft External Staff Moderator
    2026-06-22T16:21:45.18+00:00

    Hello @Eric Richards ,

    Welcome to Microsoft Q&A .Thank you for reaching out to us.

    Based on the information provided, the current latency pattern represents a significant change from the previously observed behavior.Since Conversation Summarization is implemented as an asynchronous Long Running Operation (LRO), the overall completion time includes job submission, backend processing and result retrieval.

    Regarding the queries asked

    1. If the observed latency related to the retirement timeline Current documentation does not indicate that latency degradation is expected because of the retirement timeline. Conversation Summarization remains a supported feature until March 31, 2029. Documentation recommends planning migration of existing workloads and directing new summarization projects toward Foundry-based models, which provide enhanced capabilities and lower latency characteristics.
    2. If Is migration to Azure OpenAI / Foundry-based models is recommended Yes, migration should be considered for long-term modernization and future planning. Benefits include:
      • Enhanced summarization quality
      • More flexible prompt-driven scenarios
      • Improved scalability options
      • Lower latency characteristics compared to legacy summarization workloads
    3. Are scale-up or scale-out options available As of now , no customer-controlled scale-up or scale-out options were identified for Conversation Summarization within the Standard tier experience. Throughput and execution capacity are primarily managed by the service layer.

    The following would have resulted in the observed latency

    1. Service-side processing delays occurring after job submission
    2. Regional service conditions affecting execution timing
    3. Additional processing overhead when multiple summarization aspects are requested in a single job
    4. Client-side polling intervals affecting perceived completion time
    5. Behavior specific to the preview SDK version (Azure.AI.Language.Conversations 2.0.0-beta.5)

    Please check if the following steps help-

    1. Comparing REST API and SDK Execution Run the same request using:
      • Direct REST API
      • Azure.AI.Language.Conversations 2.0.0-beta.5
      Compare:
      • Time to receive the operation-location
      • Total job completion time
      This helps determine whether the delay is occurring within the SDK path or within the service execution layer itself.
    2. Analyzing the Full LRO Lifecycle Consider Capturing:
      • Request submission timestamp (UTC)
      • Operation-location receipt timestamp
      • Polling timestamps
      • Final completion timestamp
      This helps determine whether latency is introduced during:
      • Job admission
      • Job processing
      • Result retrieval
    3. Validating Request Complexity Perform tests using:
      • A single conversation per request
      • Individual summarization aspects separately:
      • Issue
      • Resolution
      • Follow-up Tasks
      • Recap
      Comparing completion times across configurations can help identify whether specific processing paths are contributing to the delay
    4. Reviewing Service Signals Please check for:
      • HTTP 429 responses
      • Retry-After headers
      • Correlation between request IDs and long-running jobs
      This may help identify throttling or transient capacity-related conditions.
    5. Performing a Region Comparison If feasible:
      • Please deploy a separate Azure AI Language resource in another supported region
      • Execute identical workloads
      • Compare completion times
    6. Evaluating SDK Version Impact Compare behavior between:
      • Current beta SDK (2.0.0-beta.5)
      • Direct REST API
      • Latest generally available SDK path (if applicable)
      This helps determine whether SDK-specific behavior contributes to the observed latency.

    The following references might be helpful , please check them out

    Please let us know if the response was helpful

     

    Thank you

    Was this answer helpful?

    0 comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.