OneDrive on mac is keeping two copies of all downloaded files

Anonymous
2022-07-14T09:40:57+00:00

Environment:

MacOS: 12.4 (21F79)

Mac filesystem type hosting OneDrive folder: APFS Data Volume: /dev/disk1s1 on /System/Volumes/Data (apfs, local, journaled, nobrowse)
OneDrive: 22.131.0619.0001 (Standalone)
OneDrive home account as part of Microsoft 365 Personal subscription
OneDrive folder: ~/OneDrive, symlink to /Users/mikaelkjaerulff/Library/CloudStorage/OneDrive-Personal
OneDrive “Files On-Demand” is off by selecting “Download all OneDrive files now”
OneDrive “File collaboration” is off
OneDrive status is “Your files are synced” and there are no warnings/errors
OneDrive storage: 134 GB used of 1.01 TB (as reported by https://onedrive.live.com/?v=managestorage)

I have discovered that OneDrive keeps 2 local copies of all the synced files.  So instead of taking up 134 GB of disk space it uses 268 GB – that can’t be right!  The (I assume) primary copy is in the OneDrive folder:

~/Library/CloudStorage/OneDrive-Personal/

And then there is the (I assume) secondary (or internal?) copy in:

~/Library/Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/

This is true copies of the files, not hard-linked files – I checked that it is different inodes:

> cd ~/Library

> ls -i CloudStorage/OneDrive-Personal/Gravhund/ActivitiesData.xlsx Group\ Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/ActivitiesData.xlsx
8965664803 CloudStorage/OneDrive-Personal/Gravhund/ActivitiesData.xlsx
8965664807 Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/ActivitiesData.xlsx

There are some differences. If I run a diff on the two locations, like:

> cd ~/Library

> diff -rq CloudStorage/OneDrive-Personal/ Group\ Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive | grep -v .DS_Store > ~/tmp/diff.txt

Note that I exclude Mac Spotlight files (.DS_Store) – Spotlight does not run on the secondary copy. I get:

> cat -v ~/tmp/diff.txt

Only in CloudStorage/OneDrive-Personal/: .Trash

Only in Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-01-14 ?M-^Ersoversigt.PDF
Only in CloudStorage/OneDrive-Personal/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-01-14 A?M-^Jrsoversigt.PDF
Only in Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-02-17 SEAS-NVE - årsopgørelse.pdf
Only in CloudStorage/OneDrive-Personal/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-02-17 SEAS-NVE - a?M-^Jrsopgørelse.pdf
Only in Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-03-22 Novafos - årsregning 2020.pdf
Only in CloudStorage/OneDrive-Personal/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-03-22 Novafos - a?M-^Jrsregning 2020.pdf
Only in Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-12-31 ?M-^Ersoversigt.PDF
Only in CloudStorage/OneDrive-Personal/Ejerforeningen matr. nr. 146 Hareskov By, Værløse/Billag/2021: 2021-12-31 A?M-^Jrsoversigt.PDF
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/IMPO/sed0.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed0.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/IMPO/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/KULD/sed0.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/KULD/sed0.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/KULD/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/KULD/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/PROV/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/PROV/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/PROV.KRITIK/sed0.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/PROV.KRITIK/sed0.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/PROV.PRID/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/PROV.PRID/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/RYGI/sed0.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/RYGI/sed0.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/RYGI/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/RYGI/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/RYGS/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/RYGS/sed2.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/UDST/sed1.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/UDST/sed1.txt differ
Files CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/UDST/sed2.txt and Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/UDST/sed2.txt differ
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk: index.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk: tilmeld.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk: tilmeld_ver.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk/traen: lydighed.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk/traen: ring.htm~
Only in CloudStorage/OneDrive-Personal/Gravhund/WWW/nsdgk/traen: spor.htm~

I see the following main differences:

  1. Symlinks in primary are converted into normal files with content of the symlink in the secondary.
  2. Some file names are converted (like ?M-^Ersoversigt.PDF to årsregning 2020.pdf).
  3. Looks like some emacs backup-files are not copied to the secondary location and not uploaded to the cloud.

(1)

 Symlinks are converted to normal files:

> ls -l "CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/IMPO/sed0.txt" "Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed0.txt"
lrwxrwxr-x  1 mikaelkjaerulff  staff  18 Jun 17 20:01 CloudStorage/OneDrive-Personal/Gravhund/GravDB/ETL/IMPO/sed0.txt -> ../common/sed0.txt
-rwxr-xr-x@ 1 mikaelkjaerulff  staff  18 Jun 17 20:01 Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed0.txt
> cat -v "Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/Gravhund/GravDB/ETL/IMPO/sed0.txt"
../common/sed0.txt

I do understand that it often makes little to no sense synchronizing symbolic links to a remote location as-is – they will often point to an invalid location.  But if the symlink is relative (like above), then it might work (if the remote filesystem supports the concept of symbolic links).  The real annoyance is that if you do a fresh download from the OneDrive cloud the symlinks are “lost” – which is the case when you do a reset of OneDrive.  The symlinks are restored as normal files with the content of the symlink.  As far as I can tell there is no easy way to find just these files and convert them back to symlinks.

(2)

Apple’s APFS file system encodes file and directory names using Unicode but is not a “normalizing file system”. I assume the secondary OneDrive copy does some normalization (from NFC to NFD) of file and directory names before uploading to the cloud.  At least that is the only explanation I can come up with.

Ref on APFS and normal forms:

(3)

I don’t understand why emacs backup files are not copied to the secondary and subsequently uploaded to the cloud.  I have a suspicion it could be due to the way emacs create the backup file.  Nevertheless, OneDrive does not pick up the emacs backup files – this must be a bug in OneDrive.

I could understand if OneDrive has the need for a temporary location (to normalize for example) for files and directories before being uploaded to the cloud – but with emphasis on “temporary”.  The secondary copies live forever – that can’t be right.

As a side note: Still OneDrive goes to 100+ % CPU for a minute or so every time I save a file in my OneDrive location.  At least the newest version of OneDrive does not cause kernel panics anymore (fingers crossed, knock on wood…).

What – if anything – can I do to avoid having the secondary copies?

/Mikael

Microsoft 365 and Office | OneDrive | For home | MacOS

Locked Question. This question was migrated from the Microsoft Support Community. You can vote on whether it's helpful, but you can't add comments or replies or follow the question.

0 comments No comments

45 answers

Sort by: Oldest
  1. Anonymous
    2022-07-14T10:30:04+00:00

    Hi, Dearwolf. Thank you for your question and reaching out. My name is John and I’d be more than happy to help you with your query.

    I am so sorry about that. You can fix this issue by following these steps:

    1. From Spotlight search type “keychain” and select "Keychain Access”. If it's not visible in Spotlight, go to the Utilities folder and open Keychain Access from there.
    2. Search for “OneDrive.”
    3. Delete “OneDrive Cached Credential .”
    4. Run OneDrive.

    If it still did not work, I highly suggest you to uninstall and reinstall the OneDrive.
    You can redownload it by clicking https://go.microsoft.com/fwlink/p/?LinkID=248255

    Best regards,
    John

    Was this answer helpful?

    1 person found this answer helpful.
    0 comments No comments
  2. Anonymous
    2022-07-14T13:08:43+00:00

    John,

    Are you sure about this???? How can a keychain item of type "application password" have anything to do with having downloaded files stored twice?

    And, by the way: I have 2 items:

    I assume the first is from when I was running the App Store version. I switched to the standalone version on recommendation from MS support when my kernel experienced several panics due to OneDrive.

    Which one to delete?

    /Mikael

    Was this answer helpful?

    1 person found this answer helpful.
    0 comments No comments
  3. Anonymous
    2022-07-14T21:47:25+00:00

    Hi Dearwolf,

    I'm Sneha and I'd be happy to help you out with your question.

    It's happening because of Sync issue with OneDrive and your Mac.

    To fix it you'll need to go through following steps:-

    First in order to verify if your personal account is fine, in Safari internet browser on incognito mode go to below link:
    https://onedrive.live.com

    Access with your personal account, verify if you are allowed to access.

    If so, you can proceed with the reset process:

    1. Quit OneDrive. (Select the cloud icon in the top tray, then select Settings > Quit OneDrive.)
    2. Find OneDrive in your Applications folder.
    3. Right-click OneDrive and select Show Package Contents.
    4. Browse to the Contents > Resources folder.
    5. Double-click ResetOneDriveApp.command (or ResetOneDriveAppStandalone.command, if you're using the standalone app).
    6. Start OneDrive and finish the setup process.

    See more information in below link: https://support.microsoft.com/en-us/office/reset-onedrive-34701e00-bf7b-42db-b960-84905399050c#ID0EBCAAA=Mac&ID0EDF=Mac

    I hope this information helps. If you have any questions, please let me know and be glad to assist you further.

    Best Regards,
    Sneha

    Was this answer helpful?

    8 people found this answer helpful.
    0 comments No comments
  4. Anonymous
    2022-07-15T16:35:03+00:00

    Hi ,

    I'm Sneha and I'd be happy to help you out with your question.

    Sorry that my response was not helpful, For better assistance to fix your issue, Can you update that what issue persisted after trying to fix sync issue in your system.

    If you have any questions, please let me know and be glad to assist you further.

    Best Regards,
    Sneha

    Was this answer helpful?

    0 comments No comments
  5. Anonymous
    2022-07-16T11:39:40+00:00

    Sneha

    I don’t understand how you can claim there are sync issues.  There are no errors, warnings, or any of that sort.  One would expect that the OneDrive app would report if there were sync issues.

    In short: The reset did not solve the issue.  OneDrive on mac is keeping two copies of all downloaded files. The reset resulted in yet another copy of all files (in "~/ OneDrive-Personal (Archive)") and lost all my symbolic links.  I burned a lot of CPU energy (not very environmentally friendly) for no apparent reason.

    The detailed story:

    I did check with Safari in private browsing mode that I can log in and access my documents on https://onedrive.live.com. As expected, I had no issues and full access to my documents.

    Before doing the OneDrive reset, I suggest running:

    > cd ~/OneDrive
    
    > find . -type l -exec ls -l {} \;
    

    This will provide a list of all symlinks and what they point to.  Save the list, this will be needed to reestablish your symbolic links after the reset.

    I then performed the reset (for the 4^th^ time in the last two months):

    > cd /Applications/OneDrive.app/Contents/Resources
    
    > ./ResetOneDriveAppStandalone.command > ~/tmp/reset.txt 2>&1
    

    The command ran for ~5 minutes and did not indicate any issues in the output.

    I then started the OneDrive app and did the initial configuration, and it started to build the OneDrive (empty) structure.  Selected “Download all OneDrive files now” and did set “File collaboration” to off and waited… 

    The CPU went to 250 - 350% (total of OneDrive, OneDrive File Provider and fileproviderd processes).  The first 15 minutes was spent on building the empty structure and then it started to download files.

    While downloading I did a few comparisons of disk used in ~/OneDrive (the primary) and ~/Library/Group Containers/UBF8T346G9.OneDriveStandaloneSuite/OneDrive.noindex/OneDrive/.  To little surprise the disk usage at both locations grew slowly and were the same – so even during initial initialization OneDrive is keeping two copies of all downloaded files.

    After about 8 hours the download was completed, but not fully.  I had to select “Download all OneDrive files now” one more time and the remaining 9 GB was downloaded.

    And to little surprise I am again in the exact same situation.  All downloaded files exist in 2 copies, one in the “primary” and one in the “secondary”.  Once bitten, twice shy I now have a script to restore my symlinks which were “lost”.

    After downloading was completed, it was clear that nothing had changed - OneDrive on Mac is keeping two copies of all downloaded files.

    In reality I think this is just one of several bugs and deficiencies in the OneDrive app lately.  I have been a user of OneDrive on Mac for at least 10 years.  In the early years there were some hick-ups, but for many years it performed its job without much fuss.  Then came the change to default to “Files On-Demand” – and since then it has been a problem app, lately causing kernel panics and now keeping two copies of all downloaded files and using way too much CPU. 

    I would strongly urge MS to perform an in-depth review of this app on the Mac.  Both structure and algorithms and a detailed code-review.  I would suggest focusing on the parts related to “Files on Demand” – there must be something very wrong there.

    Also, it is mind-blowing that there are two versions of the “same” application and that they are so different.  I cannot comprehend that you have an Appel App store version and the Standalone version. Dev-ops would be so much simpler if you only had one version of the app – and that would likely lead to fewer bugs. Havoc will happen if you switch from one to the other. Suggest that you build a proper uninstall function for each version.

    Once the review is done, the development team should be solely focused on reliability, availability, serviceability, usability, and installability (RASUI) for some time and postpone new features.  Once you have a solid and stable app, then new features can be added under rigorous testing.

    OneDrive is a professional commercial app we are paying for – it is the least you can do.

    What to do now?

    /Mikael

    Was this answer helpful?

    20+ people found this answer helpful.
    0 comments No comments