It is because your computer hasn't gotten around to loading all the icons for shortcuts. If you wait a second for your computer to load all the way. Watch the task manager for the disk to calm down then open the folder it should show up. After it finishes loading you can also hit f5 to refresh the file explorer and the shortcut will appear.
I am guessing that those are in the end of the load order since you don't really need an icon to use the PC. It still lists it as a shortcut. It could be that the theme hasn't finished loading but it is not a bug. Maybe a faster PC would solve it, mine is slow so I can see it others might not be able to because it loads so fast :)
Hope that info helps
Waiting for a while after a boot doesn't bring the arrow icons back. I doubt that it has something to do with a slow pc since I have a quite a new PC (i5 3570K at 3.4 Ghz). Also, I didn't have this problem before Windows 8.1 so a bug seems much more likely.
The thing is, that even without hitting F5 these arrow icons should just come up after a boot automatically.