Progress report in DevOps Test plan does not synchronize with the test plan

Hello Mårten Bergström,

Could you please provide us with the below requested information to understand and pinpoint the issue.

1. Are all requirement-based test suites linked to work items of the same type (e.g., all user stories or all PBIs)?
2. Have any of the requirements or test cases been recently renamed, moved, or modified (which can desync the progress report cache)? If test cases are properly associated with each requirement?
3. When you open the test plan in the Web UI → Progress report → Configuration, do you see any filters applied?
4. Are you using query-based suites or only requirement-based suites?
5. Has the test plan been cloned or copied recently from another project or iteration? (This can cause partial linkage loss.)
6. Are all the requirements correctly linked to test cases via “Tests” or “Tested By” relation in the backlog view?

Meanwhile I am providing you few workarounds please check this if it helps.
Workarounds:

Step-by-step checks:

1. Verify requirement links:
   • Go to each test suite → select a test case → check the “Requirement” link.
   • Ensure the link type is “Tests” or “Tested By”, not custom.
2. Rebuild the test plan cache:
   • Sometimes, progress report sync lags behind.
   • Try refreshing the Test Plan Progress report → “Regenerate” or reopen in a new session.
3. Check for filtering or missing iterations:
   • In the Progress report, ensure no hidden filters (like Area Path, Iteration Path, or State).
4. Avoid mixed suite types:
   • If a test plan mixes requirement-based and query-based suites, results can appear incomplete or random.
   • Try isolating to requirement-based only for validation.
5. Force synchronization workaround:
   • Unlink and re-link one affected requirement, then save and refresh the progress report.
   • This often triggers reindexing of test result coverage.
6. Known behavior:
   • The progress report only reflects requirements that have linked test cases in the current test plan, not globally across all suites.
   • So, if a requirement’s test case is linked but not included in the plan, it won’t appear.

References:

• Understanding test plans and suites
• Azure DevOps Progress Report Documentation
• View progress report

I hope the above helps. Please do let us know if you have any further questions. Thank you.

Q:Are all requirement-based test suites linked to work items of the same type (e.g., all user stories or all PBIs)?

A: Yes, I have only based the test suites on Requirement Based Suites, filtering only User Stories 

Q: Have any of the requirements or test cases been recently renamed, moved, or modified (which can desync the progress report cache)? If test cases are properly associated with each requirement?

A: Yes, the user stories have test cases linked (tested by). They have been created during the last months but not changed since creation.

Q: When you open the test plan in the Web UI → Progress report → Configuration, do you see any filters applied?

A: No filters at all in addition to the selected test plan

Q: Are you using query-based suites or only requirement-based suites?

A: Only requirement-based suites

Q: Has the test plan been cloned or copied recently from another project or iteration? (This can cause partial linkage loss.)

A: No, this is a new test plan

Q: Are all the requirements correctly linked to test cases via “Tests” or “Tested By” relation in the backlog view?

A: I overall yes, all I have inspected have test cases linked. There might be a few single requirements missing test cases that I have not found. There are 	approximately 800 requirements in 10 suites in the test plan. Some requirements show in the progress report, and some don't, for some suites. I believe all 	requirements show, but in some, only a low percentage is shown. I tried your workaround with removing and adding requirements, the single requirements then show in 	the progress report, but I don't want to do that for ~200 requirements.

Hello Martin,

Thank you for confirming those details. Based on your inputs, the behavior you’re seeing could occur if some user stories in the test plan don’t have any linked test cases, the Progress Report will only display requirements that have linked tests in the same plan.

Could you please verify the following:

1. Make sure every User Story in the plan has at least one linked test case (Tested By relationship). You can check this via Boards → Queries using a filter for stories missing test links.
2. Try removing and re-adding one affected suite to trigger a metadata refresh, then reload the Progress Report (this helps if the issue is due to a stale analytics cache).
3. If the issue persists, please confirm whether the Analytics extension is enabled for your project. The Progress Report depends on it for synchronization.
4. If all stories are properly linked and the Analytics service is active, but the issue continues, it may be a known sync inconsistency in that case, I can help raise this& check with engineering team for further investigation.

References:

• Track test progress and coverage in Azure DevOps
• Manage test suites and test plans

I hope this helps. Please do let us know if you have any further questions. Thank you!

Hello Martin, following up to see if you have a chance to check my previous response and help us with requested information to check and assist you further on this. Thank you!

Hi Anurag, thanks for your help. See my comments to your suggestions below.

1. Make sure every User Story in the plan has at least one linked test case (Tested By relationship). You can check this via Boards → Queries using a filter for stories missing test links.
   1. Yes, done that.
2. Try removing and re-adding one affected suite to trigger a metadata refresh, then reload the Progress Report (this helps if the issue is due to a stale analytics cache).
   1. This adds the specific item to the progress report but not any of the others.
3. If the issue persists, please confirm whether the Analytics extension is enabled for your project. The Progress Report depends on it for synchronization.
   1. Is should be enabled, it is not an option to turn it off in my version
4. If all stories are properly linked and the Analytics service is active, but the issue continues, it may be a known sync inconsistency in that case, I can help raise this& check with engineering team for further investigation.
   1. I waited over night to see if it would sync, but nothing happens. What I can see is that once a test is executed it is also added to the progress report, which is sort of a comfort, but I don't get a good overview of how muck remains since a lot of requirements/test cases are not included in the progress report, they only show in the plan. Please go ahead an raise this to the engineering team.