An Azure machine learning service for building and deploying models.
Unable to change Azure ML system datastore authentication from AccountKey to Identity for UAI-only workspace
Mustapha Yussif
5
Reputation points Microsoft Employee
We have an existing Azure Machine Learning workspace created with only a user-assigned managed identity. We need to change its default storage access type from AccountKey to Identity without recreating the workspace.
Current configuration
- Workspace identity type:
UserAssigned -
primaryUserAssignedIdentityis configured -
systemDatastoresAuthMode:accesskey - Storage account shared-key access is disabled
- Workspace uses an AML managed VNet
- Blob and File private endpoints are approved
- The UAI has the required permissions, including:
- Contributor and Key Vault Administrator on the workspace Key Vault
- Storage Account Contributor
- Storage Blob Data Contributor
- Storage File Data Privileged Contributor
The system datastores have inconsistent authentication:
-
workspaceblobstore:NoneCredentialConfiguration/identity-based -
workspacefilestore:AccountKey
Error
When changing the workspace storage access type to Identity in the Azure portal, Azure ML attempts to update workspacefilestore and fails:
Credential Service error due to user error.
This can occur if the system assigned managed identity of the workspace
does not have access to the key vault.
Please add workspace managed identity as Contributor for the key vault
associated with the workspace.
Please make sure that you are passing valid secret names.
The inner response is 404 NotFound from the Azure ML datastore Credential Service.
Troubleshooting performed
- Verified the UAI's Key Vault and storage role assignments.
- Changed the datastore credentials to
NoneCredentialConfiguration. It failed with the same error. - Created another Azure Files datastore and made it the default. The workspace update still reconciles the fixed system datastore named
workspacefilestore. - Deleted
workspacefilestorethrough the SDK and ARM REST API. DELETE returns204, but the datastore remains visible through both ARM and Azure ML. Its original creation timestamp is unchanged. - Attempted to add a system-assigned identity alongside the existing UAI. Azure ML rejected the update:
Not supporting updating workspace identity to
SystemAssigned,UserAssigned when workspace's identity is UserAssigned
Questions
- Is changing
systemDatastoresAuthModefromaccesskeytoIdentitysupported for an existing UAI-only workspace? - Is there a supported way to repair or reset the internal credential associated with the platform-managed
workspacefilestore? - Can Microsoft perform a backend migration without recreating the workspace?
- Why does Credential Service require a system-assigned identity when the workspace has a configured primary UAI with all required permissions?
Recreating the workspace is not currently an acceptable option.
Azure Machine Learning
Azure Machine Learning
Sign in to answer