Data Quality signal's "Violations over time" chart stays empty despite multiple completed monitor runs

Ashutosh Singh 20 Reputation points
2026-08-12T07:26:48.2633333+00:00

I've set up an Azure ML model monitor (via the Python SDK, MonitorSchedule/MonitorDefinition) for a binary classification model deployed to an online endpoint, with three signals configured: data_drift_signal, data_quality_signal, and prediction_drift_signal, all using model_inputs/model_outputs collector data as production data and an MLTable as reference data.

After manually triggering the schedule 5+ times (all runs show Completed status), I compared the three signals' detail pages in Studio:

  • data_drift_signal: the "Violations over time" / feature drift trend chart renders correctly, showing a line graph across the run history.
  • prediction_drift_signal: same — trend chart renders correctly.
  • data_quality_signal: the Overview, Violation summary, and Feature breakdown table all populate correctly with accurate numbers (pass rate, null value rate, out-of-bounds rate per feature), but the "Violations over time" panel at the top of the page is completely blank — no chart, no error, just empty space.

I also noticed the data_quality_signal detail page header displays "data_quality_signal (Generation quality) PREVIEW" — but per the official docs (Model monitoring in production), "Generation quality" refers to a separate GenAI/LLM evaluation signal type (groundedness, relevance, fluency, etc.), not the classic tabular Data Quality signal (null value rate / data type error rate / out-of-bounds rate) that I've actually configured. This looks like a mislabeling in the UI.

Cross-referencing the underlying blob output (model_monitor_metrics_output/signals/), I found that data_drift_signal and prediction_drift_signal each have their own subfolder containing per-feature histogram/time-series JSON files, while data_quality_signal only produces a single flat data_quality_signal.json with no equivalent per-run historical folder — which would explain why there's no data for the trend chart to plot.

Questions:

  1. Is the missing trend chart for Data Quality an expected/documented limitation, or a bug?
  2. Is the "(Generation quality) PREVIEW" label on a standard tabular Data Quality signal a known UI mislabeling issue?
  3. Is there a supported way to get historical Data Quality trend data (either via the UI or by requesting the underlying per-run time-series data via SDK/API)?User's image
Azure Machine Learning
Answer accepted by question author
Pavan Kumar Purilla 410 Reputation points Microsoft External Staff Moderator
2026-08-26T11:59:48.4033333+00:00

HiAshutosh Singh,

We checked this with the Product Group (PG), and they confirmed that the behavior you are seeing is expected with the current implementation.

  1. Data Quality – “Violations over time” chart The chart is currently blank because the Data Quality signal does not emit the required run-level metric (value time series) that the chart uses. This is a current product limitation/feature gap, not an issue with your configuration or monitor runs.
  2. “Generation quality – PREVIEW” label The Product Group confirmed that this is a Studio UI issue. The tabular Data Quality signal is currently being mapped to the “Generation quality” detail page. This is separate from the missing trend chart and is handled by the Azure ML Studio UI team.
  3. Historical Data Quality trend data Data Quality does not currently generate the per-run time-series data required for the “Violations over time” chart. Therefore, there is currently no supported SDK/API method to retrieve this specific DQ trend data.
  4. Blob output The flat data_quality_signal.json file without a per-run historical folder is expected behavior with the current implementation. The per-run trend data available for drift signals is not generated for Data Quality.
  5. Future fix / alternative At this time, there is no committed fix or planned feature to add this DQ trend functionality.

Therefore, the completed monitor runs and the Data Quality metrics shown in the Overview, Violation Summary, and Feature Breakdown are valid; the limitation is specifically with the historical trend visualization and its underlying time-series data.

MOMO will be retiring soon so no new feature in planning.

Was this answer helpful?

1 person found this answer helpful.
0 comments

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. bhavana8284 0 Reputation points
    2026-08-26T13:30:25.79+00:00

    Data Quality Signals → “Violations over time” is empty even though monitor runs completed, the most likely issue is that the runs completed but didn’t produce violation events that the chart can plot.

    Check these in order:

    Verify the monitor actually detected violations

    Open a completed run.

    Check the run details/results for failed rules or violations.

    A successful/completed run with 0 violations can leave the chart empty.

    Check the chart’s time range/filter

    Make sure the selected date range includes the completed runs.

    Try expanding it to something like Last 30/90 days.

    Check whether a timezone/date boundary is excluding the runs.

    Check that the monitored asset/rules are still active

    Confirm the chart is looking at the same dataset/table and monitor.

    If rules were recently changed, the historical runs may not correspond to the currently selected rules.

    Look for a distinction between “runs” and “violations”

    The chart generally needs violation records/events, not merely execution history.

    For example:

    Run 1 → completed, 0 violations

    Run 2 → completed, 0 violations

    Run 3 → completed, 2 violations

    → only Run 3 should contribute to the violations-over-time series.

    Check whether the UI/data pipeline is delayed

    If violations are visible inside individual run results but not in the chart, this points more strongly to a chart aggregation/indexing issue rather than your monitor configuration.

    Refresh/reopen the signal and check again after a new run.

    Was this answer helpful?

    0 comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.