Databricks: After metastore storage root location removal cannot change default storage root for new __databricks_internal catalog

Hi @Jurijs Ņemcevs , Hope you are doing well.

Thanks for the detailed write-up — what you're seeing is expected behaviour, and there's a clear explanation plus a couple of things to try.

Why this happened

When you remove metastore-level storage, Unity Catalog doesn't discard the old root — it pushes it down. Per the documentation: "Existing catalogs that have no storage root specified are given the metastore storage root's cloud storage location as their catalog-level managed storage location."

__databricks_internal had no explicit storage root of its own, so it inherited the old path. Your other catalogs kept their own roots, which is why only this one is pointing at the legacy location.

Check for prior_metastore_root_location

The same operation also creates an external location and storage credential for the old path if one didn't already exist — named prior_metastore_root_location by default. This is almost certainly the external location you were considering force-editing. Confirm it exists in Catalog Explorer → External Data → External Locations.

Try this first

ALTER CATALOG ... SET MANAGED LOCATION is generally available in Databricks SQL and DBR 18.1+ (it may not have been when the older threads you found were written):

ALTER CATALOG __databricks_internal
SET MANAGED LOCATION 'abfss://<container>@<storage-account>.dfs.core.windows.net/<path>';

You'll need OWNER or MANAGE on the catalog, plus CREATE MANAGED STORAGE on the target external location. Note this affects only new managed objects — existing ones aren't relocated, which is fine given your path is empty.

__databricks_internal is a system-managed catalog, so this may be rejected. If it is, please share the exact error — that's what determines the next step.

On the force-flag approach — I'd advise against it

Editing the URL of an in-use external location with force is risky here: __databricks_internal backs materialized views, streaming tables, and pipeline state. Repointing the location doesn't move any data — it silently breaks the mapping between Unity Catalog metadata and the underlying files. Don't do this before the ALTER CATALOG attempt.

Before you delete the storage account or subscription

Please don't delete either while Unity Catalog still references the path. Sequence it as:

1. Repoint or confirm nothing references the old path.
2. Drop the prior_metastore_root_location external location and its storage credential — a dangling reference to a deleted subscription is much harder to clean up afterwards.
3. Then delete the storage account and subscription.

Also worth checking: if any managed tables were dropped recently, their data may still be within the retention window in that location.

References

• Manage Unity Catalog metastores — Remove metastore-level storage
• Specify a managed storage location in Unity Catalog
• ALTER CATALOG

Let me know how the ALTER CATALOG attempt goes and we'll take it from there.

Kind regards,
Microsoft Support Team.

Hello Peram,

Thank you for your answer! I checked the sql cmd from the Databricks SQL warehouse and received following error:
ALTER CATALOG SET MANAGED LOCATION command is not enabled here..
Tried it from both serverless and non-serverless sql warehouses.

Regarding the content - seems like not. All of the schemas under this __databricks_internal catalog point to different storage accounts and external locations (which will remain since they also serve as a storage location for common catalogs). So even if any materialized views or anything existed / exists - they would be located in other storage accounts. I also checked the storage account itself - its empty, no tables, volumes or anything.

Kind regards,
Jurijs

Hi @Jurijs Ņemcevs ,

Thanks for testing and coming back with the exact error — that's what we needed.

What the error tells us

command is not enabled here reads as feature gating rather than a permissions problem. A permissions failure would name the missing privilege — OWNER/MANAGE on the catalog, or CREATE MANAGED STORAGE on the target external location — so this isn't something you can grant your way past.

There are two possible causes, and one statement separates them. ALTER CATALOG ... SET MANAGED LOCATION requires Databricks SQL or DBR 18.1 and above. Please run the same command against a standard catalog you own, pointing at any external location where you hold CREATE MANAGED STORAGE:

ALTER CATALOG <your_test_catalog>
SET MANAGED LOCATION 'abfss://<container>@<account>.dfs.core.windows.net/<path>';

• Works there, fails on __databricks_internal → the restriction is specific to the system-managed catalog. The documented path covers standard and foreign catalogs only; I can't find any published guidance on __databricks_internal at all, which is itself a strong indicator this needs a platform-side change.
• Fails on both → it's your compute version. Check your SQL warehouse channel (Current vs. Preview) and retry, or run it from a notebook on a DBR 18.1+ cluster.

It's also worth capturing the current state for the record:

DESCRIBE CATALOG EXTENDED __databricks_internal;

The Location field will show exactly what storage root is attached — useful evidence either way.

For completeness: Catalog Explorer's Add storage location option won't help, as that UI is documented for foreign catalogs. And re-setting a metastore root at account level won't retract a location already pushed down onto a catalog.

Good news from your own findings

Unity Catalog resolves managed storage bottom-up — schema location first, catalog location only if the schema has none, metastore only if neither. You've confirmed every schema under __databricks_internal has its own location, so nothing writes to that catalog root today. It also can't start: once metastore-level storage is removed, every new catalog must be given a dedicated storage location. This is stale metadata, not a live write target.

The force flag — not just risky, ineffective

Rewriting an in-use external location's URL doesn't move data, and more to the point it doesn't remove the catalog's storage-root reference, which is the thing actually blocking you. Please leave it alone.

Before you delete the subscription

A Unity Catalog reference pointing into a deleted subscription is considerably harder to unwind than one cleaned up while it still exists. I'd sequence it like this:

1. In Catalog Explorer → External Data → External Locations, look for prior_metastore_root_location. When metastore-level storage is removed, Unity Catalog creates an external location and storage credential for the old path under that default name — so it very likely still points at the account you want to delete, even though you've confirmed the account is empty.
2. Confirm no other external location targets that storage account.
3. Check dropped managed tables. The default recovery period is 7 days, configurable to 0 hours or 7–30 days. Your storage reads empty, so this is probably already clear, but worth confirming against the date you removed the metastore root.
4. Drop prior_metastore_root_location and its storage credential.
5. Then delete the storage account and subscription. References: Specify a managed storage location in Unity Catalog — https://learn.microsoft.com/azure/databricks/connect/unity-catalog/cloud-storage/managed-storage ALTER CATALOG — https://learn.microsoft.com/azure/databricks/sql/language-manual/sql-ref-syntax-ddl-alter-catalog

Hello,

Thank you for your answer. I will be on vacation for next 2 weeks, so will be able to check right after it. Will inform about the results as soon as possible!

Kind regards,
Jurijs