So I opened a support case. Support suggested to deactivate the feature. I deemed this unacceptable, so the poor support subcontractor from Shanghai wanted me to collect network tracing logs. I stopped there and decided to actually deactivate the feature.
They do not officially admit it, but it is a bug in the SharePoint Lists Sync Engine (Nucleus). But the Product Group considers it a "cosmetic" issue.
To put this into context: they have 3884 issues in status "OPEN" over at the SharePoint Feedback Portal. The Top issue has over 500 upvotes and over 100 comments.
SharePoint · Community (microsoft.com)
This might never get fixed.
If you want to get it fixed, go ahead, open your own support case and continue where I left off.
BUT. Do you really need SharePoint Lists sync/offline usage? I figured List offline usage is not working in my environment and I am almost NEVER offline anyways, so I might just as well get rid of it.
You can read all about lists troubleshooting here:
Common issues syncing SharePoint lists offline - SharePoint | Microsoft Learn
but you basically just need to execute this:
- CMD.exe>Run as Administrator>
- REG ADD HKLM\SOFTWARE\Policies\Microsoft\OneDrive /v DisableNucleusSync /t REG_DWORD /d 1 /f

Prompt "Sign in to turn on offline access to your lists" should no longer appear.
It worked for me.
Have a successful day.