Say when I start from C: and then go to Users, find my name double click on that, all the folders under User X are shown ie instead of bottom of the screen, the folder user X moves in the middle to show all the folders,
What you describe here has nothing to do with the problem. Double clicking a folder in the right-hand pane will open the folder in the left-hand pane, revealing it's subfolders (if "Expand to current folder" is checked) just as you described. That's how we expect it to work.
then I select My documents (which has a lot of folders) and click on the arrow on the side of it, and My Documents now moves on the top of the screen and displays all folders, and then I continue the same way well down deep and Windows Explorer attempts to show all the folders in the navigation pane as there are by moving the top level up as much as possible.
Clicking on the arrow to expand a folder in the left-hand pane is working as expected too in my opinion.
However, try clicking on the folder itself rather than the arrow. The folder will jump to the bottom of the list so that the subfolders are outside the viewable area, which means that further scrolling is required to view/access the subfolders. That's the problem we are talking about here.