OneDrive on mac is keeping two copies of all downloaded files

Anonymous
2022-07-14T09:40:57+00:00

Environment:

MacOS: 12.4 (21F79)

Mac filesystem type hosting OneDrive folder: APFS Data Volume: /dev/disk1s1 on /System/Volumes/Data (apfs, local, journaled, nobrowse)
OneDrive: 22.131.0619.0001 (Standalone)
OneDrive home account as part of Microsoft 365 Personal subscription
OneDrive folder: ~/OneDrive, symlink to /Users/mikaelkjaerulff/Library/CloudStorage/OneDrive-Personal
OneDrive “Files On-Demand” is off by selecting “Download all OneDrive files now”
OneDrive “File collaboration” is off
OneDrive status is “Your files are synced” and there are no warnings/errors
OneDrive storage: 134 GB used of 1.01 TB (as reported by https://onedrive.live.com/?v=managestorage)

I have discovered that OneDrive keeps 2 local copies of all the synced files.  So instead of taking up 134 GB of disk space it uses 268 GB – that can’t be right!  The (I assume) primary copy is in the OneDrive folder:

~/Library/CloudStorage/OneDrive-Personal/

And then there is the (I assume) secondary (or internal?) copy in:

~/Library/Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/

This is true copies of the files, not hard-linked files – I checked that it is different inodes:

> cd ~/Library

> ls -i CloudStorage/OneDrive-Personal/Gravhund/ActivitiesData.xlsx Group\ Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/ActivitiesData.xlsx
8965664803 CloudStorage/OneDrive-Personal/Gravhund/ActivitiesData.xlsx
8965664807 Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/ActivitiesData.xlsx

There are some differences. If I run a diff on the two locations, like:

> cd ~/Library

> diff -rq CloudStorage/OneDrive-Personal/ Group\ Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive | grep -v .DS_Store > ~/tmp/diff.txt

Note that I exclude Mac Spotlight files (.DS_Store) – Spotlight does not run on the secondary copy. I get:

> cat -v ~/tmp/diff.txt

Only in CloudStorage/OneDrive-Personal/: .Trash

Only in Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-01-14 ?M-^Ersoversigt.PDF
Only in CloudStorage/OneDrive-Personal/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-01-14 A?M-^Jrsoversigt.PDF
Only in Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-02-17 SEAS-NVE - årsopgørelse.pdf
Only in CloudStorage/OneDrive-Personal/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-02-17 SEAS-NVE - a?M-^Jrsopgørelse.pdf
Only in Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-03-22 Novafos - årsregning 2020.pdf
Only in CloudStorage/OneDrive-Personal/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-03-22 Novafos - a?M-^Jrsregning 2020.pdf
Only in Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-12-31 ?M-^Ersoversigt.PDF
Only in CloudStorage/OneDrive-Personal/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-12-31 A?M-^Jrsoversigt.PDF
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/IMPO/sed0.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed0.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/IMPO/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/KULD/sed0.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/KULD/sed0.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/KULD/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/KULD/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/PROV/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/PROV/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/PROV.KRITIK/sed0.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/PROV.KRITIK/sed0.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/PROV.PRID/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/PROV.PRID/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/RYGI/sed0.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/RYGI/sed0.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/RYGI/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/RYGI/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/RYGS/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/RYGS/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/UDST/sed1.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/UDST/sed1.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/UDST/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/UDST/sed2.txt differ
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk: index.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk: tilmeld.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk: tilmeld_ver.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk/traen: lydighed.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk/traen: ring.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk/traen: spor.htm~

I see the following main differences:

  1. Symlinks in primary are converted into normal files with content of the symlink in the secondary.
  2. Some file names are converted (like ?M-^Ersoversigt.PDF to årsregning 2020.pdf).
  3. Looks like some emacs backup-files are not copied to the secondary location and not uploaded to the cloud.

(1)

 Symlinks are converted to normal files:

> ls -l "CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/IMPO/sed0.txt" "Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed0.txt"
lrwxrwxr-x  1 mikaelkjaerulff  staff  18 Jun 17 20:01 CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/IMPO/sed0.txt -> ../common/sed0.txt
-rwxr-xr-x@ 1 mikaelkjaerulff  staff  18 Jun 17 20:01 Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed0.txt
> cat -v "Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed0.txt"
../common/sed0.txt

I do understand that it often makes little to no sense synchronizing symbolic links to a remote location as-is – they will often point to an invalid location.  But if the symlink is relative (like above), then it might work (if the remote filesystem supports the concept of symbolic links).  The real annoyance is that if you do a fresh download from the OneDrive cloud the symlinks are “lost” – which is the case when you do a reset of OneDrive.  The symlinks are restored as normal files with the content of the symlink.  As far as I can tell there is no easy way to find just these files and convert them back to symlinks.

(2)

Apple’s APFS file system encodes file and directory names using Unicode but is not a “normalizing file system”. I assume the secondary OneDrive copy does some normalization (from NFC to NFD) of file and directory names before uploading to the cloud.  At least that is the only explanation I can come up with.

Ref on APFS and normal forms:

(3)

I don’t understand why emacs backup files are not copied to the secondary and subsequently uploaded to the cloud.  I have a suspicion it could be due to the way emacs create the backup file.  Nevertheless, OneDrive does not pick up the emacs backup files – this must be a bug in OneDrive.

I could understand if OneDrive has the need for a temporary location (to normalize for example) for files and directories before being uploaded to the cloud – but with emphasis on “temporary”.  The secondary copies live forever – that can’t be right.

As a side note: Still OneDrive goes to 100+ % CPU for a minute or so every time I save a file in my OneDrive location.  At least the newest version of OneDrive does not cause kernel panics anymore (fingers crossed, knock on wood…).

What – if anything – can I do to avoid having the secondary copies?

/Mikael

Microsoft 365 and Office | OneDrive | For home | MacOS

Locked Question. This question was migrated from the Microsoft Support Community. You can vote on whether it's helpful, but you can't add comments or replies or follow the question.

0 comments

45 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Anonymous
    2022-10-01T01:09:05+00:00

    Since I started to use Monterey the OneDrive shown problems to keep updated on my Mac. As I use little, I was using it on the browser. However, a few days ago there might have been an app or OS update and from then on OneDrive started a sync which caused all my files to duplicate.

    It seems that on the Monterey we can't choose the path used by the app and there too many bugs on it.

    It became a garbage useless.

    Was this answer helpful?

    1 person found this answer helpful.
    0 comments
  2. Anonymous
    2022-09-30T01:11:04+00:00

    Just a few comments on the current activity in this thread:

    In my case everything was on the system disk, i.e. internal disk, and even in the same file system.

    In my case there were no symbolic links involved.

    My duplicates were true duplicates, each consuming the exact same amount of disk space.

    Given they were on the same file system I checked they were not hard linked, i.e. same i-node. It was two different i-nodes and thus true duplicates.

    The "File on demand" is a relatively new feature in the OneDrive app and did cause a lot of noise when introduced - I think about two years back. Therefore, many of us turned it off. Also, if you have a flakey - or even no internet connection at times - it is nice to have a local copy. Especially for lage files it is annoying to wait the extra time for OneDrive to download it.

    I have been using the rclone & macFuse alternative for 2 months now. I have not experienced any issues, so I can recommend it as an alternative to OneDrive app. It has saved many CPU hours.

    /Mikael

    Was this answer helpful?

    1 person found this answer helpful.
    0 comments
  3. Anonymous
    2022-09-29T22:02:06+00:00

    Thanks: Per your 1) and 2) above, testing with my paths for sync and cache, du -ks * | sort -n reveals similar sizes in both locations, though not identical for some items.

    The only symlink I see is in my home folder. /Users/mymac

    MYMACHINE:~ mymac$ ls -lA
    lrwxr-xr-x    1 mymac  staff     51 Jul 21 14:59 OneDrive -> /Users/mymac/Library/CloudStorage/OneDrive

    From /Users/mymac
    du -ks OneDrive does show a size of 0 as expected.

    I can't find any other symlinks in the paths to either location. It seems that both locations contain real files, are not symlinks, and that there are actually 2 files consuming space. I am not seeing anything that looks like a stub.

    Again, note that I have no external drive. Both sync and cache are on the MBP's internal drive.

    And correct, there is no "- Personal" in my path, rather there is a "- CompanyName" that I have removed from the copy/paste to this thread, and changed my userid to "mymac"

    Was this answer helpful?

    1 person found this answer helpful.
    0 comments
  4. Anonymous
    2022-09-29T14:58:32+00:00

    Curious and sorry if I missed this. Does everyone on this thread fully synch their OneDrives to their Mac? (no files on demand usage)?

    The underlying mechanics are not fully understood by me but I do know the following:

    1. My OneDrive is approximately 325GB
    2. When I unlink my account, delete the OneDrive app *and* ensure the two locations already referenced referenced above, I free up approximately 650GB of space on the drive. (i do not use an external drive)

    Was this answer helpful?

    1 person found this answer helpful.
    0 comments
  5. Anonymous
    2022-09-28T21:41:16+00:00

    I just went through the exercise of removing OneDrive, deleting the Container copy, re-installing OneDrive and letting everything sync. I selected the option to keep all files on my drive. Once again, I am seeing two copies of everything: my "working drive" and the one Microsoft so thoughtfully crated for me that consumes storage real-estate:

    My files:
    /Users/mymac/Library/CloudStorage/OneDrive

    Microsoft-created copy:

    /Users/mymac/Library/Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive

    I don't see any symlinks anywhere, actual duplicate files exist.

    (Note for formatting on this Forum, jump to or insert a new line with "SHIFT-Return")

    Was this answer helpful?

    1 person found this answer helpful.
    0 comments