Clarification on Azure AI Services Architecture, LLM Usage, and Data Security Controls

Question

Clarification on Azure AI Services Architecture, LLM Usage, and Data Security Controls

Vijaya Ahire 20

Hi Team,

We are currently evaluating Azure AI services for a client project and would appreciate clarification on a few security and architecture-related topics.

Our client has strict requirements regarding data security, privacy, and potential data leakage when using AI services. In particular, we would like to better understand the underlying architecture and security controls for Azure AI Foundry and Azure AI/Cognitive Services.

Could you please help clarify the following:

For services such as Azure-Language-Text-PII-redaction and other Azure AI Language/Cognitive Services, are requests processed using Large Language Models (LLMs), or are they based on traditional machine learning/NLP models?
If any underlying LLMs are used, could you provide details on the model family and processing architecture?
Is customer data used for model training, retraining, or service improvement by Microsoft?
What safeguards are in place to prevent data leakage across tenants?
How is customer data handled during processing (storage, retention, logging, encryption, etc.)?
Are there any Microsoft documents or official references that describe:
- Data privacy and security controls
  - Data residency and retention policies
    - Compliance certifications
      - LLM usage (if applicable) for Azure AI services

We would appreciate any relevant documentation, whitepapers, or architecture references that can help address our client's security and governance concerns.

0 comments

Answer accepted by question author

Arif Ali Mughal 85

Hi Vijaya,

For Azure AI Language / PII redaction, Microsoft documentation does not publicly identify a specific underlying LLM family. The safer answer is to treat each Azure AI service based on its own documented privacy and security controls rather than assuming LLM usage.

Microsoft states that Azure Language request data may be temporarily stored for up to 48 hours and then purged, and for PII/health endpoints, LoggingOptOut is true by default. Microsoft also documents encryption, RBAC, private networking options, customer-managed keys, and secure deployment controls.

For Azure AI Foundry models sold by Azure, Microsoft states that prompts, completions, embeddings, and fine-tuning data are not used to train foundation models without the customer’s permission or instruction.

References: https://learn.microsoft.com/en-us/azure/ai-services/language-service/secure-deployment https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/language-service/data-privacy https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/data-privacy https://learn.microsoft.com/en-us/azure/foundry/concepts/customer-managed-keys

Vijaya Ahire 20 Reputation points

2026-09-19T17:58:47.1033333+00:00
Hi Arif,

Thank you for the details on the privacy and security controls.

We have one follow-up question specifically related to the underlying model architecture.

For Azure AI Language services such as PII Redaction, Healthcare Text Analytics, and similar Cognitive Services capabilities, can Microsoft explicitly confirm whether these services use any Large Language Models (LLMs) as part of the request processing pipeline?

Specifically, we are looking for clarification on the following:

Are these services powered by LLMs, traditional NLP/ML models, or a combination of both?

If LLMs are used, can you share the model family (GPT, Phi, proprietary Microsoft model, etc.) or any available architecture documentation?

If Microsoft cannot disclose the specific model, can you at least confirm whether customer data is processed by an LLM at any stage for PII/Healthcare Language Service requests?

Our client's security review requires us to understand whether submitted data is processed by an LLM or by non-generative AI models, independent of the existing privacy and security controls.

We would appreciate any official Microsoft documentation or statement that can help clarify this point.

Thank you for your support.

Best Regards,
Vijaya Ahire
Senior Associate Data Science

2 additional answers

Your answer

Vijaya Ahire 20 Reputation points

2026-09-19T17:58:47.1033333+00:00

Hi Arif,

Thank you for the details on the privacy and security controls.

We have one follow-up question specifically related to the underlying model architecture.

For Azure AI Language services such as PII Redaction, Healthcare Text Analytics, and similar Cognitive Services capabilities, can Microsoft explicitly confirm whether these services use any Large Language Models (LLMs) as part of the request processing pipeline?

Specifically, we are looking for clarification on the following:

Are these services powered by LLMs, traditional NLP/ML models, or a combination of both?

If LLMs are used, can you share the model family (GPT, Phi, proprietary Microsoft model, etc.) or any available architecture documentation?

If Microsoft cannot disclose the specific model, can you at least confirm whether customer data is processed by an LLM at any stage for PII/Healthcare Language Service requests?

Our client's security review requires us to understand whether submitted data is processed by an LLM or by non-generative AI models, independent of the existing privacy and security controls.

We would appreciate any official Microsoft documentation or statement that can help clarify this point.

Thank you for your support.

Best Regards,
Vijaya Ahire
Senior Associate Data Science

Answer 1

For Azure AI Language services in the provided material, the documented guidance describes them as Azure Language capabilities and does not identify PII redaction, NER, CLU, question answering, summarization, or related Language features as being processed by a specific LLM family. No supported documentation here names an underlying LLM architecture for Azure Language PII redaction or other Language/Cognitive Services features.

Supported security and data-handling points are:

Customer data use for training or service improvement
- For Azure AI Foundry models sold by Azure, prompts, completions, embeddings, and training data are not available to other customers, are not available to OpenAI or other model providers, and are not used to train foundation models without permission or instruction.
- For Azure Language, service-specific behavior varies by capability. For example:
  - PII detection temporarily stores and encrypts submitted input for up to 48 hours to support troubleshooting by default.
  - Setting loggingOptOut=true prevents that temporary storage.
  - Text Analytics for health has temporary input logging disabled by default with loggingOptOut=true.
  - Custom NER and custom text classification store labeled datasets in the customer’s own Azure Blob Storage account, and trained models are retained until deleted.
  - For custom NER and custom text classification, text submitted for extraction/classification is not stored by the service.
  - Question answering stores knowledge bases and chat logs in the customer’s own Azure resources such as Azure AI Search and, when diagnostic logging is enabled, Azure Monitor.
  - CLU training utterances, tags, trained models, and deployed models persist until the customer deletes them.
Safeguards against data leakage and cross-tenant exposure
- Microsoft recommends defining data boundaries based on user access levels and application scope.
- Use dataset isolation so each AI workload operates in its own storage or data environment.
- Apply Azure RBAC and least-privilege access controls.
- For CLU, restrict access to projects, models, and training utterances.
- For question answering, apply RBAC to Azure AI Search and Azure Monitor resources and enable chat logging only when needed.
- For highly secure or isolated scenarios, Azure Language containers can be used for controlled or disconnected environments.
- Additional security controls called out in the material include customer-managed keys, VNET support for some AI services, and Azure AD / Microsoft Entra ID authentication with RBAC.
Storage, retention, logging, and encryption
- Azure Language data at rest is automatically encrypted with FIPS 140-2 compliant 256-bit AES.
- Customer-managed keys (CMK) are supported for Language resources when using a supported pricing tier and Azure Key Vault.
- For PII detection, default temporary retention is up to 48 hours unless loggingOptOut=true is set.
- For CLU:
  - Example utterances and tags remain until deleted by the customer.
  - Trained and deployed models remain until deleted by the customer.
- For question answering, chat logs are stored only when diagnostic logging is enabled.
- For sensitive document workflows, secure storage containers, access controls, and encryption are recommended.
Data residency and governance
- Azure Language guidance recommends selecting the designated region for the Language resource to meet data residency requirements and using supported regions.
- Governance guidance recommends Microsoft Purview for data discovery, classification, lineage, and compliance management across AI workloads.
Compliance and security references available in the material
- The material points to the Microsoft Trust Center for Microsoft privacy and security commitments.
- The provided content does not list specific compliance certifications for Azure Language in detail.
Direct answer on LLM usage
- No supported source here states that Azure Language PII redaction or the other listed Azure Language capabilities use an LLM.
- No supported source here identifies a model family or architecture for those services.
- For client assurance, the strongest supported position from this material is to rely on the documented service-specific privacy, retention, encryption, access-control, and isolation behaviors rather than assume LLM usage.

Answer 2

Hi Vijaya,

Based on Microsoft’s public documentation, I do not see an official statement that confirms Azure AI Language PII Redaction or Healthcare Text Analytics is processed by a specific LLM family such as GPT, Phi, or another named model. Microsoft documents these as Azure AI Language / NLP features, but the underlying model architecture is not publicly disclosed in the referenced docs.

For a client security review, the safest position is: Microsoft has documented the data handling, retention, logging, encryption, and training-use controls, but has not publicly disclosed the exact model family or confirmed LLM involvement for these specific Language APIs. If explicit confirmation is required, I recommend raising a Microsoft support request through the Azure portal so Microsoft can provide an official product-team response.

Relevant Microsoft references:

https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/language-service/data-privacy https://learn.microsoft.com/en-us/azure/ai-services/language-service/secure-deployment https://learn.microsoft.com/en-us/azure/ai-services/language-service/personally-identifiable-information/how-to/redact-text-pii https://learn.microsoft.com/en-us/azure/foundry/responsible-ai/openai/data-privacy