Helm package best practices

Helm is a package manager for Kubernetes that helps simplify application lifecycle management. Helm packages are called charts and consist of YAML configuration and template files. Upon execution of a Helm operation, the charts are rendered into Kubernetes manifest files to trigger the appropriate application lifecycle actions. For the most efficient integration with Azure Operator Service Manager, follow these recommended best practices when developing Helm charts.

Considerations for registryPath and imagePullSecrets

Every Helm chart generally requires registryPath and imagePullSecrets parameters. Most commonly, you expose these parameters in the values.yaml file. At first, Azure Operator Service Manager depended on publishers managing these values in a strict manner (legacy approach), to be substituted for the proper Azure values during deployment. But not all publishers could easily comply with the strict management of these values. Some charts hide registryPath and/or imagePullSecrets behind conditionals, or other value restrictions, which weren't always met. Some charts declare registryPath and/or imagePullSecrets as an array instead of as the expected named string.

To reduce the compliance requirements on publishers, Azure Operator Service Manager introduced two improved methods: injectArtifactStoreDetail and cluster registry. These newer methods don't depend on registryPath or imagePullSecrets appearing in the Helm package. Instead, these methods use a webhook to inject proper Azure values directly into pod operations.

Method summary for registryPath and imagePullSecrets

All three methods are presently supported as described in this article. Choose the best option for your network function (NF) and use case.

Legacy:

• Requires you to parameterize registryPath and imagePullSecrets in Helm values and deployment templates for substitution.
• Hosts images in Azure Container Registry.

InjectArtifactStoreDetail:

• Uses a webhook to inject registryPath and imagePullSecrets directly into pod operations, with minimal dependencies on Helm.
• Hosts images in Azure Container Registry.

Cluster registry:

• Uses a webhook to inject registryPath and imagePullSecrets directly into pod operations, with no dependency on Helm.
• Hosts images in the local network function operator (NFO) extension.

In all three cases, Azure Operator Service Manager substitutes Azure values for whatever values you expose in templates. The only difference is the method of substitution.

Legacy requirements for registryPath and imagePullSecrets

Azure Operator Service Manager uses the Azure Network Function Manager service to deploy containerized network functions (CNFs). With the legacy method, Azure Network Function Manager substitutes the Azure Operator Service Manager container's registryPath and imagePullSecrets values into the Helm operation during the deployment of network functions.

Example of the legacy method

The following Helm deployment template shows an example of how you should expose registryPath and imagePullSecrets:

apiVersion: apps/v1 
kind: Deployment 
metadata: 
  name: nginx-deployment 
  labels: 
    app: nginx 
spec: 
  replicas: 3 
  selector: 
    matchLabels: 
      app: nginx 
  template: 
    metadata: 
      labels: 
        app: nginx 
    spec: 
      {{- if .Values.global.imagePullSecrets }} 
      imagePullSecrets: {{ toYaml .Values.global.imagePullSecrets | nindent 8 }} 
      {{- end }} 
      containers: 
      - name: contosoapp 
        image:{{ .Values.global.registryPath }}/contosoapp:1.14.2 
        ports: 
        - containerPort: 80 

The following values.yaml template shows an example of how you can provide the registryPath and imagePullSecrets values:

global: 
   imagePullSecrets: [] 
   registryPath: "" 

The following values.schema.json file shows an example of how you can define the registryPath and imagePullSecrets values:

{ 
  "$schema": "http://json-schema.org/draft-07/schema#", 
  "title": "StarterSchema", 
  "type": "object", 
  "required": ["global"], 
  "properties": { 
      "global" : {
          "type": "object",
          "properties": {
              "registryPath": {"type": "string"}, 
              "imagePullSecrets": {"type": "string"}, 
          }
          "required": [ "registryPath", "imagePullSecrets" ], 
      } 
   } 
} 

The following network function definition version (NFDV) request payload shows an example of how you can provide the registryPath and imagePullSecrets values at deployment:

"registryValuesPaths": [ "global.registryPath" ], 
"imagePullSecretsValuesPaths": [ "global.imagePullSecrets" ], 

In the preceding examples:

• The registryPath value is set without any prefix such as https:// or oci://. If necessary, define a prefix in the Helm package.
• imagePullSecrets and registryPath must be provided during NFDV onboarding.

Other considerations

Consider the following recommendations when you're using the legacy method.

Avoid references to an external registry

References to an external registry can cause validation problems. For example, if deployment.yaml uses a hard-coded registry path or external registry references, it fails validation.

Perform manual validations

Review the images and container specifications to ensure that the images have a prefix of registryPath and that imagePullSecrets is populated with secretName:

 helm template --set "global.imagePullSecrets[0].name=<secretName>" --set "global.registry.url=<registryPath>" <release-name> <chart-name> --dry-run

Here's another example:

 helm install --set "global.imagePullSecrets[0].name=<secretName>" --set "global.registry.url=<registryPath>" <release-name> <chart-name> --dry-run
 kubectl create secret <secretName> regcred --docker-server=<registryPath> --dockerusername=<regusername> --docker-password=<regpassword>

Use a static image repository and tags

Each Helm chart should contain a static image repository and tags. You set the static values through one of the following methods:

• In the image line
• In values.yaml, without exposing these values in the NFDV

An NFDV should map to a static set of Helm charts and images. You update the charts and images only by publishing a new NFDV, as shown in the following examples:

 image: "{{ .Values.global.registryPath }}/contosoapp:1.14.2"

 image: "{{ .Values.global.registryPath }}/{{ .Values.image.repository }}:{{ .Values.image.tag}}"
 
YAML values.yaml
image:
  repository: contosoapp
  tag: 1.14.2

 image: http://myUrl/{{ .Values.image.repository }}:{{ .Values.image.tag}}

injectArtifactStoreDetails requirements for registryPath and imagePullSecrets

In some cases, third-party Helm charts might not be fully compliant with Azure Operator Service Manager requirements for registryPath. In these cases, you can use injectArtifactStoreDetails to avoid making compliance changes to Helm packages.

With injectArtifactStoreDetails enabled, you use a webhook method to inject the proper registryPath and imagePullSecrets dynamically during the pod operations. This method overrides the values that are configured in the Helm package. You still must use legal dummy values where registryPath and imagePullSecrets are referenced, usually in the global section of values.yaml.

The following values.yaml example shows how you can provide the registryPath and imagePullSecrets values for compatibility with the injectArtifactStoreDetails approach:

global: 
   registryPath: "azure.io"
   imagePullSecrets: ["abc123"] 

Using the injectArtifactStoreDetails method

To enable injectArtifactStoreDetails, set the installOptions parameter in the NF resource's roleOverrides section to true, as shown in the following example:

resource networkFunction 'Microsoft.HybridNetwork/networkFunctions@2023-09-01' = {
  name: nfName
  location: location
  properties: {
    nfviType: 'AzureArcKubernetes'
    networkFunctionDefinitionVersionResourceReference: {
      id: nfdvId
      idType: 'Open'
    }
    allowSoftwareUpdate: true
    nfviId: nfviId
    deploymentValues: deploymentValues
    configurationType: 'Open'
    roleOverrideValues: [
      // Use inject artifact store details feature on test app 1
      '{"name":"testapp1", "deployParametersMappingRuleProfile":{"helmMappingRuleProfile":{"options":{"installOptions":{"atomic":"false","wait":"false","timeout":"60","injectArtifactStoreDetails":"true"},"upgradeOptions": {"atomic": "false", "wait": "true", "timeout": "100", "injectArtifactStoreDetails": "true"}}}}}'
    ]
  }
}

Cluster registry requirements for registryPath and imagePullSecrets

With a cluster registry, images are copied from Azure Container Registry to a local Docker repository on the Nexus Kubernetes cluster. You use a webhook method to inject the proper registryPath and imagePullSecrets values dynamically during the pod operations. This method overrides the values that are configured in the Helm package. You still must use legal dummy values where registryPath and imagePullSecrets are referenced, usually in the global section of values.yaml.

The following values.yaml example shows how you can provide the registryPath and imagePullSecrets values for compatibility with the cluster registry approach:

global: 
   registryPath: "azure.io"
   imagePullSecrets: ["abc123"] 

For more information on using a cluster registry, see the concept documentation.

Recommendations for immutability restrictions

Immutability restrictions prevent changes to a file or directory. For example, an immutable file can't be changed or renamed. You should avoid using mutable tags such as latest, dev, or stable. For example, if deployment.yaml uses latest for .Values.image.tag, the deployment fails.

 image: "{{ .Values.global.registryPath }}/{{ .Values.image.repository }}:{{ .Values.image.tag}}"

Recommendations for CRD declaration and usage split

We recommend splitting the declaration and usage of customer resource definitions (CRDs) into separate Helm charts to support updates. For detailed information, see the Helm documentation about separating charts.

Recommendations for Image Version Tagging

To ensure consistent and predictable deployments, we recommend the following for all container images:

• Avoid using :latest in production environments.
  • Using latest can cause unexpected behavior because the actual image behind latest can change without notice.
  • In a cluster registry setup, if the tag value changes but the tag name stays the same, the cluster registry will not re-download the updated image.
  • This can lead to running outdated or inconsistent images.
• Instead, always use immutable tags like :1.4.2
• Ensure every build produces a unique tag, do not overwrite existing tags.

These practices help prevent deployment issues and improve traceability, rollback safety, and security compliance.

Recommendations for nfApplication sequential ordering

By default, CNF applications are installed or updated based on the order in which they appear in the NFDV. For the deletion operation, the CNF applications are deleted in the specified reverse order. If you need to define a specific order of CNF applications that's different from the default, use dependsOnProfile to define a unique sequence for installation, update, and deletion operations.

How to use dependsOnProfile

You can use dependsOnProfile in the NFDV to control the sequence of Helm executions for CNF applications. In the example that follows:

• During an installation operation, the CNF applications are deployed in the following order: dummyApplication1, dummyApplication2, dummyApplication.
• During an update operation, the CNF applications are updated in the following order: dummyApplication2, dummyApplication1, dummyApplication.
• During a deletion operation, the CNF applications are deleted in the following order: dummyApplication2, dummyApplication1, dummyApplication.

{
    "location": "eastus",
    "properties": {
        "networkFunctionTemplate": {
            "networkFunctionApplications": [
                {
                  "dependsOnProfile": {
                        "installDependsOn": [
                            "dummyApplication1",
                            "dummyApplication2"
                        ],
                        "uninstallDependsOn": [
                            "dummyApplication1"
                        ],
                        "updateDependsOn": [
                            "dummyApplication1"
                        ]
                    },
                    "name": "dummyApplication"
                },
                {
                  "dependsOnProfile": {
                        "installDependsOn": [
                        ],
                        "uninstallDependsOn": [
                            "dummyApplication2"
                        ],
                        "updateDependsOn": [
                            "dummyApplication2"
                        ]
                    },
                    "name": "dummyApplication1"
                },
                {
                    "dependsOnProfile": null,
                    "name": "dummyApplication2"
                }
            ],
            "nfviType": "AzureArcKubernetes"
        },
        "networkFunctionType": "ContainerizedNetworkFunction"
    }
}

Common errors with dependsOnProfile

Currently, if the dependsOnProfile code provided in the NFDV is invalid, the NF operation fails with a validation error. The message for the validation error appears in the operation status resource and looks similar to the following example:

 {
  "id": "/providers/Microsoft.HybridNetwork/locations/EASTUS2EUAP/operationStatuses/ca051ddf-c8bc-4cb2-945c-a292bf7b654b*C9B39996CFCD97AB3A121AE136ED47F67BB13946C573EF90628C47628BC5EF5F",
  "name": "ca051ddf-c8bc-4cb2-945c-a292bf7b654b*C9B39996CFCD97AB3A121AE136ED47F67BB13946C573EF90628C47628BC5EF5F",
  "resourceId": "/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/resourceGroups/xinrui-publisher/providers/Microsoft.HybridNetwork/networkfunctions/testnfDependsOn02",
  "status": "Failed",
  "startTime": "2023-07-17T20:48:01.4792943Z",
  "endTime": "2023-07-17T20:48:10.0191285Z",
  "error": {
    "code": "DependenciesValidationFailed",
    "message": "CyclicDependencies: Circular dependencies detected at hellotest."
  }
}

Best practices for adopting Helm 4

Helm has been the standard package manager for Kubernetes since its initial release in 2016. Its evolution has closely tracked Kubernetes itself:

• Helm v2 (2016–2019): Introduced chart-based application packaging but relied on a server-side component (Tiller), which created security and multitenancy concerns.
• Helm v3 (2019–2025): Removed Tiller, shifting to a client-only model with improved security and usability. This version became the industry standard and accumulated incremental enhancements while maintaining backward compatibility.

After nearly six years of Helm v3, the project accumulated technical debt, architectural limitations, and security challenges that it couldn't address without introducing breaking changes. This situation led to the release of Helm v4 in late 2025.

What Helm 4 Represents

Helm 4 is a significant architectural evolution rather than an incremental upgrade. Its primary goals are to:

• Align with modern Kubernetes deployment patterns
• Remove legacy Helm v3 behaviors
• Improve extensibility, maintainability, and security

Key changes introduced with Helm 4 include:

• Server-Side Apply (SSA): Replaces the legacy three-way merge approach and aligns deployments with Kubernetes-native reconciliation semantics.
• Redesigned plugin system: Introduces a more extensible architecture, including optional WebAssembly-based plugins for improved isolation and flexibility.
• Improved resource tracking: Leverages newer Kubernetes status mechanisms, such as kstatus, to provide more accurate deployment state reporting.
• Internal modernization: Removes technical debt and lays the foundation for future innovation and performance improvements.

Importantly, Helm 4 maintains compatibility with existing Helm v3 charts, enabling organizations to adopt Helm 4 gradually without requiring immediate changes to charts or deployment artifacts.

Relevance to AOSM Publishers

The AOSM team plans to support Helm 4 through two key milestones:

• First, the AOSM team releases an NFO version that includes Helm 4.1.4 operating in a "compatibility mode." This mode preserves Helm 3.18 behavior, so publishers can adopt Helm 4 without modifying existing charts or artifacts.
  • You can preview test this NFO version today in the UKSouth lab.
• Second, the AOSM team releases an NFO version that removes compatibility customizations and enables full Helm 4 behavior. Publishers can adopt this version when ready, understanding that chart and artifact changes might be required.
  • The AOSM team plans this NFO version for publisher testing in Q4 CY2026.

Publishers continue to have flexibility when selecting Helm behavior during NFO installation. NFO defaults to "compatibility mode," while providing an installation option to enable full Helm 4 behavior. This capability is cluster-scoped, meaning all deployments within a cluster must use the same Helm operating mode.

Compatibility mode details

The following settings preserve Helm 3 behavior when running Helm 4 in "compatibility mode":

• Stricter schema validation
  • Helm 4 introduces stricter validation that rejects Go-typed slices, such as []map[string]interface{}, when validating JSON arrays. This behavior can cause failures when NFO injects imagePullSecrets values.
  • NFO updates value injection logic to use []interface{} instead and audits similar code paths to ensure compatibility.
• Server-Side Apply (SSA) enabled by default
  • Helm 4 validates rendered manifests against the cluster OpenAPI schema before applying resources. Charts containing invalid field definitions that Helm 3 previously tolerated might fail validation.
  • Compatibility mode disables SSA during install and upgrade operations to preserve Helm 3 behavior.
• New wait model
  • Helm 4 defaults to an event-driven waiting model that requires Kubernetes watch permissions. This behavior can fail on Nexus clusters where the necessary RBAC permissions aren't available.
  • Compatibility mode pins wait behavior to LegacyStrategy, preserving Helm 3 polling semantics.
• Recreate removed
  • Helm 4 removes support for Upgrade.Recreate. While runtime impact is expected to be low, customer-configured values in the CRD would otherwise no longer have any effect.
  • Compatibility mode preserves the CRD field for backward compatibility but ignores it when executing Helm 4 operations.
• Schema metaschema validation
  • Helm 4 validates values.schema.json against the JSON Schema metaschema. Charts containing non-compliant schema definitions are rejected before values validation occurs. This behavior is known to impact some publisher charts.
  • Compatibility mode sets SkipSchemaValidation=true during install and upgrade operations.