I seem to have found a solution, at least in my case.
The cause seems to be SensorService. Disabling it and restarting resolves the issue for all the subsequent fastboots. The issue is also reliably reproduced by Re-activating the service then restarting.
The downside of disabling this service is that you lose auto rotation on tablet mode.
Hope this works for others, gotta sleep.