Azure Data Factory – DeleteNotInSource not deleting pipelines in TEST/UAT/PROD environments
We are using an Azure Data Factory CI/CD deployment process with the Publish-AdfV2FromJson PowerShell command.
The deployment option is configured as:
However, pipelines that have been removed from the source repository are not being deleted from the target Azure Data Factory in higher environments (TEST, UAT and PROD).
The same deployment/deletion mechanism is expected to remove ADF pipelines that are no longer present in the source.
We are using:
Expected behaviour: When an ADF pipeline is removed from the source repository and the deployment is executed, the corresponding pipeline should be deleted from the target ADF environment.
Actual behaviour: The pipeline remains in TEST/UAT/PROD even though DeleteNotInSource is set to $true.
We need Microsoft to confirm whether there are any limitations or known issues with DeleteNotInSource when used with Includes, GetObjectsByFolderName, and IncrementalDeployment, particularly for higher environments.
Please also advise how we can reliably delete ADF pipelines from TEST/UAT/PROD when they are removed from the source.