I think I know what breaks this.
I saw that the users that were having this issue were using IrfanView. I just installed it and tried to force file association through it's built in setup which operates on the registry. This worked fine on Windows 10 but not so much on Windows 11.
As soon as I did it, I could no longer see bmp, jpg, png, etc. on my file associations list. The powershell command fixed it just nicely, but I'm pretty sure that irfanview's registry setup is what's breaking it. Probably renaming some keys or adding new ones, can't really tell.