How Do You Organize Prompt Versioning in Azure AI Foundry?

A practical pattern is to treat prompt versioning like code versioning, with evaluation attached to every change.

Recommended workflow:

1. Create immutable prompt versions In the Databricks MLflow prompt registry flow shown in the context, prompt versions are immutable after creation. Any edit creates a new version, which preserves history and supports rollback. That model works well for teams because it avoids silent changes to production prompts.
2. Track prompts alongside application versions Keep prompt versions tied to app versions so prompt changes are not separated from the code and configuration that use them. This makes it easier to understand whether a quality change came from the prompt, the app logic, or both.
3. Document more than prompt text For each version, record:
   • prompt text
   • hyperparameters such as temperature, top-p, and max tokens
   • evaluation metric results across the test set
   • changes from the previous version and the reason for those changes
   This is explicitly recommended in the RAG prompt engineering guidance and is the clearest way to track prompt text, system instructions, parameter changes, and outcomes over time.
4. Use a consistent evaluation dataset Evaluate all prompt versions against the same dataset for fair comparison. Include expected facts or other expected outputs so results are comparable across versions.
5. Run controlled experiments Change one variable at a time: for example, only the grounding instruction wording, only the context ordering, or only the number of few-shot examples. This makes it possible to attribute metric changes to a specific prompt change instead of guessing.
6. Use evaluation metrics to guide edits Useful prompt-related signals include:
   • Groundedness: whether the prompt strongly instructs the model to use only provided context
   • Completeness: whether the prompt tells the model to address all parts of the question
   • Utilization: whether the prompt encourages use of all relevant context
   • Relevance: whether vague instructions are causing drift
   • Correctness: whether the prompt explains how to handle ambiguity or conflicts
7. Keep prompts specific and structured Effective prompt authoring practices in Azure OpenAI guidance include:
   • be specific
   • be descriptive
   • repeat critical instructions when needed
   • pay attention to ordering
   • give the model an “out,” such as returning not found when the answer is absent
8. Use platform features for comparison and iteration Prompt flow in Foundry supports creating prompt variants, comparing performance, and using built-in evaluation flows. That makes it useful for experimentation and team iteration. In the Databricks MLflow flow, prompt versions can also be compared directly in the UI.

What tends to work well in practice for frequently changing prompts:

• Source control for review and traceability: use version control for the surrounding app and prompt assets.
• Registered prompt versions for runtime use: use immutable registered versions when testing and deploying.
• Evaluation as a release gate: do not promote a new prompt version unless it was evaluated on the same dataset as the current version.
• Meaningful change notes: record why the prompt changed, not just what changed.
• Production monitoring: continue evaluating after deployment to catch degradation.

A strong operating model is:

• draft and refine prompts iteratively
• register each meaningful revision as a new version
• evaluate every version on the same dataset
• compare results side by side
• deploy only the best-performing version
• keep the full history of prompt text, parameters, and results

References:

• Prompt flow in Microsoft Foundry portal (classic)
• Track prompt versions alongside application versions
• RAG prompt engineering
• Evaluate and compare prompt versions
• Prompt engineering techniques
• Create and edit prompts