AIJudgeLoopEvaluator Class
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
A LoopEvaluator that uses a separate judge chat client to decide whether the user's original request has been fully addressed, continuing the loop (with the judge's gap analysis as feedback) while the answer is "no".
public sealed class AIJudgeLoopEvaluator : Microsoft.Agents.AI.LoopEvaluatortype AIJudgeLoopEvaluator = class
inherit LoopEvaluatorPublic NotInheritable Class AIJudgeLoopEvaluator
Inherits LoopEvaluator- Inheritance
Remarks
After each iteration the judge is queried directly (without any agent tools, session, or middleware) with the original request and the agent's latest response, and asked for a structured Microsoft.Agents.AI.JudgeVerdict. If the judge client does not honor structured output, the verdict falls back to parsing the raw text for the non-overlapping DoneVerdictMarker / MoreVerdictMarker markers (with MoreVerdictMarker winning, so the loop keeps running, when the verdict is ambiguous or absent).
When the request is not yet answered, the evaluator returns feedback built from FeedbackMessageTemplate with the judge's gap analysis substituted for GapAnalysisPlaceholder. How that feedback is delivered to the agent (and whether the session is reset) is decided by the LoopAgent that consumes this evaluator.
The judge instructions act as a template: any occurrence of CriteriaPlaceholder is replaced with the rendered Criteria (or removed when no criteria are supplied), letting callers add bespoke standards the response must satisfy.
LLM-judged loops are costly and probabilistic, so consider setting a stricter MaxIterations on the owning LoopAgent.
Constructors
| Name | Description |
|---|---|
| AIJudgeLoopEvaluator(IChatClient, AIJudgeLoopEvaluatorOptions) |
Initializes a new instance of the AIJudgeLoopEvaluator class. |
Fields
| Name | Description |
|---|---|
| CriteriaPlaceholder |
The placeholder token within DefaultInstructions (or a custom Instructions) that is replaced with the rendered Criteria. When no criteria are supplied, the placeholder is removed. |
| DefaultFeedbackMessageTemplate |
The default template used to build the feedback produced when the request is not yet answered. |
| DefaultInstructions |
The default system instructions used to prompt the judge. |
| DoneVerdictMarker |
The verdict marker the judge is asked to emit (for clients that do not honor structured output) when the original request has been fully addressed. |
| GapAnalysisPlaceholder |
The placeholder token within DefaultFeedbackMessageTemplate (or a custom FeedbackMessageTemplate) that is replaced with the judge's gap analysis. |
| MoreVerdictMarker |
The verdict marker the judge is asked to emit (for clients that do not honor structured output) when more work is still required. Takes precedence over DoneVerdictMarker when both (or neither) are present. |
Methods
| Name | Description |
|---|---|
| EvaluateAsync(LoopContext, CancellationToken) |
Evaluates the loop state after an iteration and decides whether to re-invoke the wrapped agent and what feedback to provide. |