FsLogix - Unclean logoff causing locked files until server reboot

Much R 101

Problem is described by M4deman under unclean-logoff-causing-locked-files-until-server-reboot

It seems to have something to do with the 2009 version.
The latest version of FSLogix is installed whats-new

Description

After a user logoff, the "System" Process (PID 4) locks the following folders:

C:\Users\local_username\AppData\Local\Microsoft\Credentials
C:\Users\local_username\AppData\Roaming\Microsoft\Credentials

The user is completely logged of, according to Task Manager.

In the FSLogix Profile Log file I can see the following:

[07:53:55.601][tid:00000c90.0000ce44][ERROR:00000020] Delete profile failed for sid S-1-5-21-3364776539-3721753400-1968955100-1179, Cleaning up manually. (Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird.)
The last sentence means that the process cannot access the file, because another process already uses it.

Also the whole "local_username" folder cannot be deleted:

[08:23:15.479][tid:00000c90.0000bcc4][WARN: 00000005] Failed to delete C:\Users\local_usename (Access is denied)
Access Denied

Does someone have any info on this behaviour?

Anonymous

2021-01-14T03:03:58.32+00:00

Hello @Much R

Did you check affected user has full control and Owner over their FSLogix Folder and the VHDX files ?

Thanks
Karlie
Much R 101 Reputation points

2021-02-02T08:02:21.997+00:00

Hello,

Where is my answer from yesterday?
Why was this deleted?

How can I open a ticket at Microsoft now?

Thank you
Anonymous

2021-02-02T08:16:39.607+00:00

Hello @Much R

sorry I was going to delete my comment as it wasn't helpful, and your comment is below mine, so it seems they were deleted together. I want to restore but it doesn't has this option.

FSLogix is supported for use with versions of Windows and Office that are still included in the Microsoft Support Lifecycle.

How to open a FSLogix Support Request
https://social.msdn.microsoft.com/Forums/en-US/14ea5f6f-760b-4ad2-83cf-23c3cbc1bcc4/how-to-open-a-fslogix-support-request?forum=FSLogix

Best regards
Karlie
Germain 11 Reputation points

2021-02-19T11:44:10.117+00:00

We are currently experiencing the same issue. The user VHDX is locked by system and can only be unlocked by rebooting the server.
Has Microsoft acknowledged this issue?

Currently using: 2.9.7654.46150
Bracko Ewald 31 Reputation points

2021-02-19T12:10:09.137+00:00

Did you activate the registry value HKEY_LOCAL_MACHINE\SOFTWARE\FSLogix\Apps\CleanupInvalidSessions (REG_DWORD, 0x1)?
Do you use Cloud Cache or not?

Also, maybe you can check with the frx command line tool if there are still some redirects pointing to the use VHDX file. You can then use that tool to delete them manually and detach the VHDX files afterwards using diskpart.
Marius Neuhaus 1 Reputation point

2021-03-09T10:31:38.737+00:00

Hello,

we have the same problem.
We use on-premise terminalserver.:
Windows Server 2019
Citrix 1912 CU2 LTSR
FSLogix 2.9.7621.30127

I've tried the CleanupInvalidSessions RegKey, but it doesn't effect.
We don't use Cloud Cache.
FSLogix Configuration only local and not per gpo.
The frx command line tool dont show redirects pointing to the use VHDX file.
Permission are set correct. Users work correctly with the containers.
Luckily the problem occurs only randomly and not at every logoff.

In the FSLogix Configuration we include the group domain-users and exclude the domain-administrators.
Could it be a Problem?

Best Regards,
Marius
itsupport 1 Reputation point

2021-05-26T12:40:37.147+00:00

When this happens are you left with multiple <USERNAME>.original folders? I've seen it go as high as <USERNAME>.original11 before I just manually delete all the folders.
Guido Racamato 26 Reputation points

2021-09-28T17:39:34.337+00:00

Hello Everyone, For the Group Policy Access Denied, once you get an affected user, could you please modify the following registry? There's no reboot required.

HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileService\References\<SID affected user>
REG_BINARY: RefCount
Value: 0
Jon McLeod 1 Reputation point

2021-11-19T14:56:08.267+00:00

This is the answer. Thank you.
Farooq 5 Reputation points

2024-02-14T12:02:36.9533333+00:00

We are having the same issue. We have an RDS environment and time to time users get logged in with Temporary profile, when I check the FSlogix logs I am seeing the following error - Failed to open virtual disk: \server\Profiles\S-1-5-21-26711-2174_user\Profile_user.vhd (The process cannot access the file because it is being used by another process.)
What we done to try resolve this or at least try find the cause and reduce downtime is, we have removed AV from all RDS host, we have created a script which logs out all users at midnight but issue is still occurring. We have a case opened with Microsoft but unfortunately there has been no resolution. Another issue we had is where users could not sign into their Outlook, teams, Office profile on the host. Has anyone else experiencing this issue and has anyone managed to resolve this? Any help or suggestion would be much appreciated.

107 answers

Chris Pickens 1 Reputation point

2021-03-26T19:19:53.663+00:00
Hi All, I figured I'd share my (potential) resolution in our case to this issue:
- We have a Windows 10 Multi-Session VM (WVD) deployment.
- Microsoft engineering examined our environment and could not see any issue with FSLogix.
- GP client service error - Access Denied, rebooting server daily would be only temp fix. Our root cause appears (at this time) to be related to one or both of the following:
- AV - I excluded the local directory on our server that stored user profile disks from real-time scan only, in this case, it was E:\Profiles. (Yes, I had already implemented all FSLogix recommended AV exclusions) We use Sophos EndPoint.
- Moved our backup window outside of business hours for both the server hosting the profiles volume and our WVD servers. ( We use SolarWinds Backup)
After making these changes, I have not had an incident since 3/18/2021. Fingers crossed.

I'll report back if the problem returns.

Not sure, but I hope this helps someone.
Was this answer helpful?
Dlo 96 Reputation points

2021-03-29T15:24:11.647+00:00

Chris,

When you say you excluded the local directory from real time scan do you me the local_username folder?
Trying to get a bit more info so I can look into this as well and try it.

Chris Pickens 1 Reputation point

2021-03-29T15:26:38.077+00:00

Sry, I mean the local disk \ volume the . vhdx files live on, In our case, this was E:\Profiles\ on our domain controller.

Hope that makes sense.

Gianluca Melis 101 Reputation points

2021-04-05T21:58:43.367+00:00

Hi, how did you get direct help from MS engineering?
I've also would like to have em take a look!

Chris Pickens 6 Reputation points

2021-04-06T15:24:01.41+00:00

Hi, We managed to get help via our CSP. To be honest, MS support was initially not great, we had to fight to get the ticket escalated until we had someone from the FSLogix team check our WVD servers. Basically, MS logged in remotely, said they could not see any issues, and had to collect logs and the next incidnet.

Fortunately, there was not the next incident after moving our backup schedule and making the additional AV exlusions. In our scenario, MS thinks the issue was related to AV.

I'm not 100% which change helped but one did make a difference for our env.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
CitrixMember 6 Reputation points

2021-04-07T16:15:53.663+00:00

For those of you experiencing this problem, do you have one environment working and one not working?
In non working environment, do you have this in profile logs in Fslogix?

Configuration setting not found: SOFTWARE\FSLogix\Profiles\GroupPolicyState. Using default: 0

In working environment, do you have this in profile logs in Fslogix?

Configuration setting not found: SOFTWARE\FSLogix\Profiles\GroupPolicyState. Using default: 1

Thats what I have, GroupPolicyState dword registry is not much described or mentioned in the registry keys list of fslogix.
I am testing this out now in non working environment: SOFTWARE\FSLogix\Profiles GroupPolicyState Dword:0001

Further in the same log
[07:09:48.134][tid:000014ac.00000958][ERROR:0000139f] Handle could not be recovered. DetachVirtualDisk will fail (The group or resource is not in the correct state to perform the requested operation.)
[07:09:48.134][tid:000014ac.00000958][ERROR:0000139f] DetachVirtualDisk failure (The group or resource is not in the correct state to perform the requested operation.)
[07:09:48.134][tid:000014ac.00000958][INFO] ErrorCode set to 5023 - Message: The group or resource is not in the correct state to perform the requested operation.

I assuming this "bug" or error has something to do related to same problem where the "include group" entries in policy had to changed from replace to update. No clue, but think it worth a try. When this grouppolicystate registry is used with default values "0" then it sometime could have problems locate user in the same group or something, and then not able to release the locks of file...... not sure.

Any explanation according to this registry setting or feedback on testing, is highly appreciated!
Was this answer helpful?
CitrixMember 6 Reputation points

2021-04-21T11:49:20.88+00:00

GroupPolicyState dword:0001 didnt work.
Problem occured again today.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
Lee Hendrix 1 Reputation point

2021-04-12T17:58:49.413+00:00

So, I think I have found somewhat of a workaround for this, ([WARN: 00000005] Failed to delete C:\Users\local_usename (Access is denied)). Feel free to try and see if you have any luck.

My Redirections.xml file was fairly robust to try and keep our profile disks as small as possible. It carried over partially from our UPM exclusions. I rebuilt it again based off of : https://stealthpuppy.com/fslogix-profile-containers-redirections/, and then put in an AV exclusion for every path listed in the Redirections file for "userprofile\path listed in exclusion file". We were getting 30 to 40 locked profiles from hung sessions a day, and now we have been 48 hours without 1. I will update if the beast returns, but would be curious if this helps anyone else out.
Was this answer helpful?
Andy Walman 6 Reputation points

2021-04-13T15:15:23.027+00:00

Lee, could you give some examples of how you formatted the exclusions, maybe a screenshot of the GP Windows? I'm unclear whether your excluding the Fslogix redirected path or the original profile path, thanks.

Lee Hendrix 1 Reputation point

2021-04-13T15:46:25.197+00:00

AV Exclusions are for the Original Profile Path, I can't dynamically do the "Local_" folders with wildcards with my AV .

Redirections.xml contains: "<Exclude Copy="0">AppData\Local\Microsoft\Edge\User Data\Default\Cache</Exclude>" then my AV has an exclusion for [USER_PROFILE]\AppData\Local\Microsoft\Edge\User Data\Default\Cache* ,

Also added the TEMP and INETCACHE directories that are set in the GP, not the Redirections.xml.

EslieRey 6 Reputation points

2021-04-13T18:09:24.497+00:00

greate way to let a virus/ransomeware in.. exclude the user tempdir (or any dir that is userwritable ) from AV.

not an option overhere

Lee Hendrix 1 Reputation point

2021-04-13T18:16:01.987+00:00

Agreed, I am still expecting MS to fix this, as there are several open cases for this particular issue., This is just what I have found that stops the session lock issue and I have other mitigations in place in the meantime.

Gianluca Melis 101 Reputation points

2021-04-22T15:37:19.23+00:00

look, i don't know if i am just overthinking it here but i think those Av Exclusions make no sense.
When you are excluding a folder (lets say AppData\Local\Microsoft\Edge\User Data\Default\Cache) is landing on local_user.name\appdata(...) and if you set your AV to exclude AppData\Local\Microsoft\Edge\User Data\Default\Cache it's going to exclude user.name\appdata(...).
So.. there is no such folder in that directory, because you've excluded it...
I am testing exactly your config now btw, just updated the redirect and pushed the AV exclusions GPO.. let's see what happens!

Gianluca Melis 101 Reputation points

2021-04-26T07:24:56.447+00:00

sadly still not working.. trying a rollback to 2004 today
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
Dlo 96 Reputation points

2021-04-19T11:43:23.967+00:00

So the test package they gave me was for windows 10 1809 not 1909 so it wouldn't install.
I am waiting for them to update it for 1909 which they did but they said they got some errors they need to fix now.
Once they get it I will let you guys know what happens.

I also tried enabling CleanupInvalidSessions and DeleteLocalProfileWhenVHDShouldApply to see if maybe that helps with the folders not being deleted off the server for the time being.

the VHD's aren't getting stuck as much as they use to but not sure why but still have all the folders not being deleted and I believe its related to search.

I will update you once I get a working package and test.
Was this answer helpful?

2 people found this answer helpful.
CitrixMember 6 Reputation points

2021-04-21T11:50:15.147+00:00

What package did they gave you? New version of the fslogix? or just a "fix" ?

Dlo 96 Reputation points

2021-04-21T11:51:58.397+00:00

Was a hotfix but still have not got the new version of the hotfix.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
CitrixMember 6 Reputation points

2021-05-03T11:15:13.297+00:00

Anyone here who has found out more regarding the Fslogix locks problem?
Was this answer helpful?

0 comments No comments
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

FsLogix - Unclean logoff causing locked files until server reboot

107 answers

Your answer