This is precisely the behavior I discovered.
When a music file is located in the "Music" directory parallel to downloads/documents/pictures etc. The embedded Art will display. As soon as you play from -any- other directory the album art comes up blank. The only solution is to have it 1.) In a microsoft OneDrive folder or 2.) In the music directory exclusively.
This is a major design failure in my opinion.
I switched my default player back to WMP as it displays the embedded art no matter where the file is located.