OneDrive on mac is keeping two copies of all downloaded files

Anonymous
2022-07-14T09:40:57+00:00

Environment:

MacOS: 12.4 (21F79)

Mac filesystem type hosting OneDrive folder: APFS Data Volume: /dev/disk1s1 on /System/Volumes/Data (apfs, local, journaled, nobrowse)
OneDrive: 22.131.0619.0001 (Standalone)
OneDrive home account as part of Microsoft 365 Personal subscription
OneDrive folder: ~/OneDrive, symlink to /Users/mikaelkjaerulff/Library/CloudStorage/OneDrive-Personal
OneDrive “Files On-Demand” is off by selecting “Download all OneDrive files now”
OneDrive “File collaboration” is off
OneDrive status is “Your files are synced” and there are no warnings/errors
OneDrive storage: 134 GB used of 1.01 TB (as reported by https://onedrive.live.com/?v=managestorage)

I have discovered that OneDrive keeps 2 local copies of all the synced files.  So instead of taking up 134 GB of disk space it uses 268 GB – that can’t be right!  The (I assume) primary copy is in the OneDrive folder:

~/Library/CloudStorage/OneDrive-Personal/

And then there is the (I assume) secondary (or internal?) copy in:

~/Library/Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/

This is true copies of the files, not hard-linked files – I checked that it is different inodes:

> cd ~/Library

> ls -i CloudStorage/OneDrive-Personal/Gravhund/ActivitiesData.xlsx Group\ Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/ActivitiesData.xlsx
8965664803 CloudStorage/OneDrive-Personal/Gravhund/ActivitiesData.xlsx
8965664807 Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/ActivitiesData.xlsx

There are some differences. If I run a diff on the two locations, like:

> cd ~/Library

> diff -rq CloudStorage/OneDrive-Personal/ Group\ Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive | grep -v .DS_Store > ~/tmp/diff.txt

Note that I exclude Mac Spotlight files (.DS_Store) – Spotlight does not run on the secondary copy. I get:

> cat -v ~/tmp/diff.txt

Only in CloudStorage/OneDrive-Personal/: .Trash

Only in Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-01-14 ?M-^Ersoversigt.PDF
Only in CloudStorage/OneDrive-Personal/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-01-14 A?M-^Jrsoversigt.PDF
Only in Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-02-17 SEAS-NVE - årsopgørelse.pdf
Only in CloudStorage/OneDrive-Personal/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-02-17 SEAS-NVE - a?M-^Jrsopgørelse.pdf
Only in Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-03-22 Novafos - årsregning 2020.pdf
Only in CloudStorage/OneDrive-Personal/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-03-22 Novafos - a?M-^Jrsregning 2020.pdf
Only in Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-12-31 ?M-^Ersoversigt.PDF
Only in CloudStorage/OneDrive-Personal/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-12-31 A?M-^Jrsoversigt.PDF
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/IMPO/sed0.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed0.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/IMPO/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/KULD/sed0.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/KULD/sed0.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/KULD/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/KULD/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/PROV/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/PROV/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/PROV.KRITIK/sed0.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/PROV.KRITIK/sed0.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/PROV.PRID/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/PROV.PRID/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/RYGI/sed0.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/RYGI/sed0.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/RYGI/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/RYGI/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/RYGS/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/RYGS/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/UDST/sed1.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/UDST/sed1.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/UDST/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/UDST/sed2.txt differ
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk: index.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk: tilmeld.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk: tilmeld_ver.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk/traen: lydighed.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk/traen: ring.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk/traen: spor.htm~

I see the following main differences:

  1. Symlinks in primary are converted into normal files with content of the symlink in the secondary.
  2. Some file names are converted (like ?M-^Ersoversigt.PDF to årsregning 2020.pdf).
  3. Looks like some emacs backup-files are not copied to the secondary location and not uploaded to the cloud.

(1)

 Symlinks are converted to normal files:

> ls -l "CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/IMPO/sed0.txt" "Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed0.txt"
lrwxrwxr-x  1 mikaelkjaerulff  staff  18 Jun 17 20:01 CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/IMPO/sed0.txt -> ../common/sed0.txt
-rwxr-xr-x@ 1 mikaelkjaerulff  staff  18 Jun 17 20:01 Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed0.txt
> cat -v "Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed0.txt"
../common/sed0.txt

I do understand that it often makes little to no sense synchronizing symbolic links to a remote location as-is – they will often point to an invalid location.  But if the symlink is relative (like above), then it might work (if the remote filesystem supports the concept of symbolic links).  The real annoyance is that if you do a fresh download from the OneDrive cloud the symlinks are “lost” – which is the case when you do a reset of OneDrive.  The symlinks are restored as normal files with the content of the symlink.  As far as I can tell there is no easy way to find just these files and convert them back to symlinks.

(2)

Apple’s APFS file system encodes file and directory names using Unicode but is not a “normalizing file system”. I assume the secondary OneDrive copy does some normalization (from NFC to NFD) of file and directory names before uploading to the cloud.  At least that is the only explanation I can come up with.

Ref on APFS and normal forms:

(3)

I don’t understand why emacs backup files are not copied to the secondary and subsequently uploaded to the cloud.  I have a suspicion it could be due to the way emacs create the backup file.  Nevertheless, OneDrive does not pick up the emacs backup files – this must be a bug in OneDrive.

I could understand if OneDrive has the need for a temporary location (to normalize for example) for files and directories before being uploaded to the cloud – but with emphasis on “temporary”.  The secondary copies live forever – that can’t be right.

As a side note: Still OneDrive goes to 100+ % CPU for a minute or so every time I save a file in my OneDrive location.  At least the newest version of OneDrive does not cause kernel panics anymore (fingers crossed, knock on wood…).

What – if anything – can I do to avoid having the secondary copies?

/Mikael

Microsoft 365 and Office | OneDrive | For home | MacOS

Locked Question. This question was migrated from the Microsoft Support Community. You can vote on whether it's helpful, but you can't add comments or replies or follow the question.

0 comments

45 answers

Sort by: Newest
Most helpful Newest Oldest
  1. Anonymous
    2022-09-01T22:02:14+00:00

    I am observing something similar. I don't want to go through the steps only to end up with the same result as Mikael had.

    Summary:

    I have my OneDrive folder with all of the files I want OneDrive to back up:

    /Users/Me/Library/Cloud Storage/OneDrive. (This is 42.9 Gig of data)

    I see the same file structure plus additional data size in:

    /Users/Me/Library/Group Containers/UBF8T346G9/OneDriveStandAloneSuite/OneDrive.noindex/OneDrive (this is 64.6 Gig).

    I haven't clicked anything under Files on Demand in OneDrive Preferences. (And I would prefer that all files are always present on my local drive).

    It looks like /Users/Me/Library/Cloud Storage/OneDrive is the "managed" copy where some files are stored in the cloud and others are downloaded, but /Users/Me/Library/Group Containers/UBF8T346G9/OneDriveStandAloneSuite/OneDrive.noindex/OneDrive contains all of my files.

    I want to fix this to free up space, but don't want to lose all of my files. Any advice?

    Was this answer helpful?

    9 people found this answer helpful.
    0 comments
  2. Anonymous
    2022-07-24T09:19:13+00:00

    I have given up on the OneDrive App for Mac.

    I think I have tried all suggestions and the ultimate was to make a local copy on my NAS of all my OneDrive data, and then vacuum my Mac for all traces of the OneDrive App (stop OneDrive, run the reset, remove the OneDrive App, using find to find all traces of the OneDrive App…), and doing a total cleanup of OneDrive storage in the cloud and finally a restart of MacOS.  This left me in a “virgin” situation with respect to OneDrive.  Then installed the OneDrive standalone App, configured, and then copied the data into the local OneDrive from my local backup.  And then waiting for hours while it synced to the cloud and burned tons of CPU.

    I ended in the exact same situation: files duplicated and the CPU going to 2-300% for minutes every time I just touch a file in OneDrive.

    So, bye bye OneDrive App.

    I have moved a smaller set of working files to iCloud Drive and that works like a charm.  Not burning CPU for no apparent reason and MS Office Apps working excellent.  My Finder has become much more responsive, and I save energy (important in these troubled times). 

    I could choose to pay for more iCloud storage (I’m currently on 1$/month for 50GB) – but that seems stupid when I have 1TB on OneDrive.  I now use OneDrive as a cloud archive.  Initially I used the web-interface for upload/download – but that often fails for large transfers: Google Chrome claims it is the server (i.e., OneDrive cloud) that becomes unresponsive or fails. 

    Instead, I have now installed rclone and macFUSE. Rclone is truly "The Swiss army knife of cloud storage".  I now have OneDrive mounted as a remote filesystem, providing (slow) access via Finder, command line… to all my files in the OneDrive cloud. Although slow, it is much faster transfers than the web-interface – and not to mention uses way less CPU.  A side benefit is if you have other cloud storages, chances are that rclone would be able to connect to those.

    I’m looking into using some of the rclone commands for syncing data to the OneDrive cloud.  For the time being I copy data to OneDrive manually, either when a project is done or periodically.

    I can truly recommend this solution and save frustration and energy for the good of the environment.

    /Mikael

    Was this answer helpful?

    10+ people found this answer helpful.
    0 comments
  3. Anonymous
    2022-07-18T20:44:40+00:00

    Hi Mikael,

    Sorry that information failed resolving the issue. I understand that you don't have more than one user logins in the mac. Sometimes if you changed the password, the account may end up out of sync if the password saved temporarily. Please find the information from the third party sites mentioned below. This workaround fixed the issue for most of the users.

    https://www.downloadsource.net/how-to-stop-onedrive-creating-duplicate-files-and-folders/n/19093/
    https://tweaklibrary.com/onedrive-creating-duplicate-files/
    https://businesstechplanet.com/onedrive-automatically-duplicating-files-causes-and-fixes/

    Could you please try removing the duplicates once by using the tool mentioned below?

    Once this is done, unlink OneDrive from all the machines and delete the app and try reinstalling it after a safe reboot.

    Note: This is a non-Microsoft website. The page appears to be providing accurate, safe information. Watch out for ads on the site that may advertise products frequently classified as a PUP (Potentially Unwanted Products). Thoroughly research any product advertised on the site before you decide to download and install it.

    Hope this information helps. Please feel free to get back if you have any questions.

    Thank you!

    Was this answer helpful?

    0 comments
  4. Anonymous
    2022-07-18T08:07:47+00:00

    Sarah,

    As stated above I have turned “File collaboration” off. Therefor there is no option for Sync conflicts options.

    /Mikael

    Was this answer helpful?

    2 people found this answer helpful.
    0 comments
  5. Anonymous
    2022-07-18T07:54:13+00:00

    Hello Dearwolf,

    Welcome to Microsoft Community and hope you are doing well.

    I'm Sarah and I'd be happy to help you out with your questions.

    Can you try to use the RESTART Mode on Onedrive MAC> Look for the OneDrive icon on the right side of the menu bar. Restart OneDrive. Select the OneDrive icon on the menu bar, then select Quit OneDrive. Use Spotlight to search for OneDrive and reopen it. Update the OneDrive sync app for Mac. Click the OneDrive icon> Settings> Preferences> under Office tab, check the Sync conflicts options, tick the option “Let me choose to merge changes or keep both copies”.

    Hope the above information helps. Reply if you need more assistance and I'll be happy to help you.

    Best Regards,

    Sarah

    Was this answer helpful?

    0 comments