FsLogix - Unclean logoff causing locked files until server reboot

Much R 101

Problem is described by M4deman under unclean-logoff-causing-locked-files-until-server-reboot

It seems to have something to do with the 2009 version.
The latest version of FSLogix is installed whats-new

Description

After a user logoff, the "System" Process (PID 4) locks the following folders:

C:\Users\local_username\AppData\Local\Microsoft\Credentials
C:\Users\local_username\AppData\Roaming\Microsoft\Credentials

The user is completely logged of, according to Task Manager.

In the FSLogix Profile Log file I can see the following:

[07:53:55.601][tid:00000c90.0000ce44][ERROR:00000020] Delete profile failed for sid S-1-5-21-3364776539-3721753400-1968955100-1179, Cleaning up manually. (Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird.)
The last sentence means that the process cannot access the file, because another process already uses it.

Also the whole "local_username" folder cannot be deleted:

[08:23:15.479][tid:00000c90.0000bcc4][WARN: 00000005] Failed to delete C:\Users\local_usename (Access is denied)
Access Denied

Does someone have any info on this behaviour?

Anonymous

2021-01-14T03:03:58.32+00:00

Hello @Much R

Did you check affected user has full control and Owner over their FSLogix Folder and the VHDX files ?

Thanks
Karlie
Much R 101 Reputation points

2021-02-02T08:02:21.997+00:00

Hello,

Where is my answer from yesterday?
Why was this deleted?

How can I open a ticket at Microsoft now?

Thank you
Anonymous

2021-02-02T08:16:39.607+00:00

Hello @Much R

sorry I was going to delete my comment as it wasn't helpful, and your comment is below mine, so it seems they were deleted together. I want to restore but it doesn't has this option.

FSLogix is supported for use with versions of Windows and Office that are still included in the Microsoft Support Lifecycle.

How to open a FSLogix Support Request
https://social.msdn.microsoft.com/Forums/en-US/14ea5f6f-760b-4ad2-83cf-23c3cbc1bcc4/how-to-open-a-fslogix-support-request?forum=FSLogix

Best regards
Karlie
Germain 11 Reputation points

2021-02-19T11:44:10.117+00:00

We are currently experiencing the same issue. The user VHDX is locked by system and can only be unlocked by rebooting the server.
Has Microsoft acknowledged this issue?

Currently using: 2.9.7654.46150
Bracko Ewald 31 Reputation points

2021-02-19T12:10:09.137+00:00

Did you activate the registry value HKEY_LOCAL_MACHINE\SOFTWARE\FSLogix\Apps\CleanupInvalidSessions (REG_DWORD, 0x1)?
Do you use Cloud Cache or not?

Also, maybe you can check with the frx command line tool if there are still some redirects pointing to the use VHDX file. You can then use that tool to delete them manually and detach the VHDX files afterwards using diskpart.
Marius Neuhaus 1 Reputation point

2021-03-09T10:31:38.737+00:00

Hello,

we have the same problem.
We use on-premise terminalserver.:
Windows Server 2019
Citrix 1912 CU2 LTSR
FSLogix 2.9.7621.30127

I've tried the CleanupInvalidSessions RegKey, but it doesn't effect.
We don't use Cloud Cache.
FSLogix Configuration only local and not per gpo.
The frx command line tool dont show redirects pointing to the use VHDX file.
Permission are set correct. Users work correctly with the containers.
Luckily the problem occurs only randomly and not at every logoff.

In the FSLogix Configuration we include the group domain-users and exclude the domain-administrators.
Could it be a Problem?

Best Regards,
Marius
itsupport 1 Reputation point

2021-05-26T12:40:37.147+00:00

When this happens are you left with multiple <USERNAME>.original folders? I've seen it go as high as <USERNAME>.original11 before I just manually delete all the folders.
Guido Racamato 26 Reputation points

2021-09-28T17:39:34.337+00:00

Hello Everyone, For the Group Policy Access Denied, once you get an affected user, could you please modify the following registry? There's no reboot required.

HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileService\References\<SID affected user>
REG_BINARY: RefCount
Value: 0
Jon McLeod 1 Reputation point

2021-11-19T14:56:08.267+00:00

This is the answer. Thank you.
Farooq 5 Reputation points

2024-02-14T12:02:36.9533333+00:00

We are having the same issue. We have an RDS environment and time to time users get logged in with Temporary profile, when I check the FSlogix logs I am seeing the following error - Failed to open virtual disk: \server\Profiles\S-1-5-21-26711-2174_user\Profile_user.vhd (The process cannot access the file because it is being used by another process.)
What we done to try resolve this or at least try find the cause and reduce downtime is, we have removed AV from all RDS host, we have created a script which logs out all users at midnight but issue is still occurring. We have a case opened with Microsoft but unfortunately there has been no resolution. Another issue we had is where users could not sign into their Outlook, teams, Office profile on the host. Has anyone else experiencing this issue and has anyone managed to resolve this? Any help or suggestion would be much appreciated.

107 answers

BananaBread-1757 1 Reputation point

2021-09-27T23:25:16.967+00:00

We too are facing this issue with a recent implementation. We have other sites setup with fsl running on the older versions without issues.

In addition to the black screen logons, we also experience users where they cannot connect to any 365 services. Appears to be related to aad/ssso in cominbation with fsl.

We have a case logged with MS, and still waiting to hear back from them..
Was this answer helpful?

0 comments No comments
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
Robert Berggren 1 Reputation point

2021-09-28T16:53:52.21+00:00

Today I got the "The Group Policy Client service failed the sign-in" Access denied.

I found this workaround to at least let the user in again. HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileService\References\<sid affected user>
RefCount set the value to 0. Then when the user logged in a new local_username was created. Would it be possible to zero out this key at log off somehonw?
Was this answer helpful?
Guido Racamato 26 Reputation points

2021-09-28T17:30:50.477+00:00

anonymous userBerggren-1701 This going to be fixed in the next FSLogix release. We hope to get the new FSLogix Public preview on November.

Андрей Михалевский 3,476 Reputation points

2021-09-28T18:41:43.003+00:00

Give an answer on the official blog please.

https://techcommunity.microsoft.com/t5/fslogix-blog/fslogix-2105-version-is-ga/ba-p/2542845

Андрей Михалевский 3,476 Reputation points

2021-09-28T18:44:39.727+00:00

Also fix the deletion of the local folder. It is not always deleted and duplicates appear.

Also, in the latest version, sometimes a local profile was created and it is not deleted, ignoring the registry key to delete.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
Андрей Михалевский 3,476 Reputation points

2021-10-12T06:57:06.257+00:00

Has anyone tried to write a script for a temporary solution ? There will be a fix in November. https://learn.microsoft.com/en-us/answers/questions/229309/fslogix-unclean-logoff-causing-locked-files-until.html?childToView=568300#answer-568300
Was this answer helpful?
Brandon Clark 1 Reputation point

2021-10-12T14:26:05.697+00:00

Why do you think there will be a fix in November? Can you link to anything to support that?

I have an open ticket with support now on this and they are pointing to our AV solution (SEP).

Андрей Михалевский 3,476 Reputation points

2021-10-13T07:43:33.18+00:00

I think RobertBerggren-1701 is one of the developers

Robert Berggren 1 Reputation point

2021-10-13T11:45:29.587+00:00

I am not a developer! :D

Андрей Михалевский 3,476 Reputation points

2021-10-13T11:47:18.997+00:00

Where is the information that the fix will be in November?

Martijn Kools 171 Reputation points

2021-10-19T11:03:58.48+00:00

It's not AV. We used to run SEP on Windows 2016 without this issue. Now we are running Windows 2019 with Windows Defender and the issue is there. We have all the correct exclusions. I think it's more of an OS thing. Win2016 --> Win2019 and the issue started.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
Brandon Clark 1 Reputation point

2021-10-19T13:10:26.007+00:00

Hey all, just so you're aware. I troubleshot this error along with Microsoft Support for 48 hours straight. In that time I got to learn FsLogix pretty well. In OUR case, it was Symantec DLP, the Data Loss Protection software. The issue was our DLP software was OUT OF DATE. DLP (from Symantec/Broadcom) uses a FILTER DRIVER. This out of date filter driver was causing ONEDRIVE to hang, which in turn caused the Office Container to hang because of a stuck handle. Once we ugpraded our Symnatec DLP from 15.5 to 15.7, the issue went away. This was confirm with an open ticket to Broadcom as well.
Was this answer helpful?
Андрей Михалевский 3,476 Reputation points

2021-10-19T13:17:11.19+00:00

We only use microsoft office 2019 pro and google chrome + adobe acrobat pdf.

We face this problem all the time. I even recorded it a couple of times when outlook hung up and I ended the session. After that I couldn't log in anymore.

In some cases, FSLogix can't properly terminate the process.

Is this a Microsoft Office, Windows Server 2019 or FSLogix problem ?

Brandon Clark 1 Reputation point

2021-10-19T13:31:42.78+00:00

Open CMD as Admin and run this: 'fltmc filters'

This will show you the filesystem filter drivers in place. It's likely to be one of those.

But yes, if a process is hung and cannot be stopped and that process is writing to an FsLogix container, then you need to find out WHY that process is getting hung. It's not FsLogix fault the process can't exit, likely.

Андрей Михалевский 3,476 Reputation points

2021-10-19T13:39:49.43+00:00

What should I do after running this command?

All applications fail in one way or another. I had no such problems with UPD.

Is there no way to force all processes to terminate on exit?

If it is so, it is not possible to fix this problem. In my 4 years of experience, terminal servers are always unstable. Including Microsoft applications.

Bracko Ewald 31 Reputation points

2021-10-19T13:42:03.99+00:00

You don't use any third party AV scanners, correct?
Then usually the Windows Defender is activated by default.
Have you set the recommended AV exclusions for the Windows Defender?
Also, keep in mind that you don't need to activate the Search Database related settings in FSLogix because Windows Server 2019 natively stores the user-related search database within the user profile now. In fact you should turn them off now to avoid issues there.

Brandon Clark 1 Reputation point

2021-10-19T13:42:07.203+00:00

Track down each one of the items listed in that command's output and ensure those filter drivers are up to date. These drivers are usually installed from software you've added.

I've been working in multi-session environments for 20 years. Trust me, it's always been hard.

Андрей Михалевский 3,476 Reputation points

2021-10-19T13:47:18.71+00:00

As I said, I only have 3 products on the terminal server. Office, Google Chrome and Adobe Acrobat. 64 architecture and the latest updates. Even with this configuration, 11 servers and 500 users, often create requests that they can't re-logon.

Андрей Михалевский 3,476 Reputation points

2021-10-19T13:48:14.107+00:00

Yes, I use these recommendations. It doesn't help.

Bracko Ewald 31 Reputation points

2021-10-19T14:46:52.333+00:00

But the FSLogix Apps Service is still running and doesn't suddenly stop, correct?
Can you see the FSLogix disks remaining in the Disk Management? Does it take quite long until the Disk Management opens?
We had recently a weird issue with one of our customers. The FSLogix disks frequently wouldn't disconnect and in that case often the FSLogix Apps Service crashed. This crash wasn't even properly reported in the System Log. Only the FSLogix service logs reported the shutdown of the FSLogix Apps Service. And you often also couldn't start this service anymore until the next reboot. A manual health check of the affected FSLogix disks would show that those disks were all without errors.
The RDS hosts are VMs running on VMware vSphere. It turned out that the physical servers the vSphere is running on had a NIC from the vendor installed that wasn't supported for that specific server model. After we replaced those NICs and updated the firmware on the physical servers everything runs smoothly now.
So sometimes it can be something deeper in the system that can cause those issues.

Андрей Михалевский 3,476 Reputation points

2021-10-19T14:55:11.45+00:00

As I understand it, we'll have to live with it. It is practically impossible to diagnose it on your own. And I do not want to pay to microsoft.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
Martijn Kools 171 Reputation points

2021-10-19T14:13:13.96+00:00

I have two different Citrix environments. Both are pretty much identical. Same FSLogix version, same GPOs, same patch level. Almost same everything except for the language and some apps.

Both run on Citrix PVS, Windows Server 2019 with XenApp 1912 LTSR CU3, FSLogix 2105. Both vdisks are monthly rebuilt from scratch, fully scripted.

On one env I have the issue:

[01:20:09.192][tid:00001990.00001ff8][WARN: 00000005] Failed to delete C:\Users\local_x (Access is denied.)
[01:20:09.192][tid:00001990.00001ff8][WARN: 00000005] Failed to delete C:\Users\local_x (Access is denied.)
[01:20:09.192][tid:00001990.00001ff8][WARN: 00000005] Failed to delete C:\Users\local_x (Access is denied.)
[01:20:09.192][tid:00001990.00001ff8][WARN: 00000005] Failed to delete C:\Users\local_x (Access is denied.)
[01:20:09.192][tid:00001990.00001ff8][WARN: 00000005] Failed to delete C:\Users\local_x (Access is denied.)

On the other one I don't:

[15:17:25.851][tid:000017d4.000068e0][INFO] Successfully removed C:\Users\local_x

Both run Windows Defender with the EXACT same settings (they even share the same GPO).

I checked BrandonClark's command fltmc filters and they give the same output on both envs.

So why does it not work on env A and why does it work on env B?

Luckily, I don't have the issue where the users can't re-logon and the vdisk is locked. The vdisk actually is not locked any longer after logoff and people can just relogon. If they happen to hit the same server it will usually create a different folder like local_username_1 or something like that.
Was this answer helpful?

0 comments No comments
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

FsLogix - Unclean logoff causing locked files until server reboot

107 answers

Your answer