Diagnosing spikes of 15-20s in "Success Server Latency"

Hi Tony Ross,

I hope you had a chance to review the information shared earlier, and I hope this information has been helpful! If you still have questions, please let us know what is needed in the comments so the question can be answered.

Thanks for sharing. We actually switched to the Azure.Data.Tables SDK at the start of this month. Unfortunately, it did not help.

Hi Tony Ross,

Thanks again for the details and your patience while we reviewed this.

• You’re seeing intermittent spikes of 15–20 seconds in Success Server Latency for Azure Table Storage across multiple accounts, even though normal latency is ~5ms.
• The issue started recently after years of stable performance, and it does not correlate with peak load.
• Your table design, partitioning strategy, and query patterns follow best practices, and no throttling or service‑side issues were observed in our telemetry.
• You updated to the newer Azure.Data.Tables SDK, but the spikes still occurred.
• Since this latency affects your gameplay experience, you moved the production workload to Cosmos DB, which has improved consistency.
• While we couldn’t identify a single definitive root cause from available logs, all your feedback has been shared with the product team for further review and future improvements.

If you'd like to share additional input or track progress, you can follow this feedback item:
https://feedback.azure.com/d365community/idea/ac44f0e9-fd10-f111-83da-7c1e52aa87b8

Please let us know if you have any more questions we’re here to help.

Hi Tony Ross,

Just checking in to see if the information provided earlier was helpful. Please let us know if you need any additional details or assistance, and we’ll gladly continue supporting you.

Thanks @Manoj Kumar Boyini

I don't think there's more we can do here at this point. I hope the feedback to the product team leads to a resolution at some point in the future as we'd love to move some of our traffic back to Table Storage. In the meantime, Cosmos DB is working well for us.

Thanks for the response! To follow up:

1. Partitioning Strategy: yes, we use randomly hashed keys along with uniqueness to avoid balancing problems with range partitions. All reads/writes are done using both partition key and row key - no scanning.
2. Monitoring Metrics: yes, it's the SuccessServerLatency that's spiking so we know this isn't a client issue.
3. Storage Analytics Logs: nothing has stood out to us yet, but some of your linked queries are new to use so we'll take another look here...
4. Client-Side Performance: we aren't experiencing any CPU/memory/etc. bottlenecks on our clients at these times.
5. Network Configuration: our servers are co-located in the same region as our storage account (EastUS2).
6. Throttling and Rate Limiting: our transactions count does not spike during these times, and the issue typically occurs during non-peak usage.
7. Consult Azure Documentation: we've reviewed the recommendations from those links. We've exhausted the issues/checklist recommendations but still need to try a few of the linked diagnostic queries - we'll follow up with what we find.

And to clarify:

1. We've yet to notice patterns in timing. It seems somewhat random.
2. The operations we use are primarily QueryEntity and UpdateEntity with a few InsertEntity calls mixed in - that's it.
3. No, we haven't seen this linked to specific queries/transactions. We only use the three operations above and all of them are done with an exact partition key and row key to read/write a single entity.

One additional note: our entities are typically between 3-12KB in size. However, we haven't seen a correlation between the size of the entity and these delays either.

Hi @Tony Ross

I hope you had a chance to review the information shared earlier, and I hope this information has been helpful! If you still have questions, please let us know what is needed in the comments so the question can be answered.

We haven't had any luck resolving this yet. We're still combing through Storage Analytics Logs to see if we can find any additional insight.

We're debating moving this particular data to premium block blobs or cosmos to avoid the issue since we're not having much luck isolating a cause. Not sure yet what we'll do for our other affected tables. The odd part is this worked with sufficiently low latency (under 5s) for years until the end of November (and our volume of transactions hasn't increased).

Can anyone answer what an expected max latency would be for a case like ours in table storage (frequent reads/writes but well-balanced, unique partition keys)?

Hi Tony Ross,

Thank you for the update. Based on the additional details you provided, the latency spikes you are seeing (15–20 seconds SuccessServerLatency on successful Table Storage operations) are not expected for a well‑designed Azure Table Storage workload.

To address your questions directly:

Expected latency for your scenario

For workloads with:

Well‑balanced, unique partition keys

Frequent reads/writes (point lookups)

No hot partitions

Entity sizes in the ~3–12 KB range

…the expected server‑side latency for Azure Table Storage is:

Typical: < 10–50 ms

During background maintenance: occasional spikes up to 500 ms – 1–2 seconds

Not expected: recurring 15–20 second latencies on successful operations

Spikes at the 15–20 second level indicate that the delay is coming from backend storage node operations, not from the client, not from partitioning, and not from your workload design.

Why you’re still seeing the issue (even after splitting tables/accounts)

You already confirmed:

No throttling signals (429/503)

No hot partitions

No abnormal traffic spikes

Randomized partition keys

Same behavior across multiple tables and accounts

Issue began suddenly around late November

This pattern points to backend storage node behavior, such as:

Partition movement or rebalancing

Background compaction / index merges

Internal retries before returning 200 (SuccessServerLatency captures server time)

Regional cluster maintenance events

These events do not appear in Storage Analytics Logs, which is why combing through them hasn’t revealed the root cause.

Because the latency occurs even after splitting tables across accounts, the issue is cluster‑level, not table‑level.

About moving to Premium Block Blobs or Cosmos DB

It’s important to note:

Premium Block Blobs do not provide entity‑level operations or server‑side indexing—they are not a functional replacement for Table Storage.

Cosmos DB Table API is the direct successor when you require:

Predictable low latency (P99 in single‑digit ms to low tens of ms)

  Automatic partition management
  
     SLA‑backed performance
     
        Scale‑out with negligible spikes
        

If your application depends on consistent low‑latency key/value access, Cosmos DB may be the right long‑term option.

Hi Tony Ross,

I hope you had a chance to review the information shared earlier, and I hope this information has been helpful! If you still have questions, please let us know what is needed in the comments so the question can be answered.