I've been battling with Groove for the last few weeks and have had a similar issue with album art not displaying, or displaying old art even after wiping the tags and starting from fresh using mp3tag.
The solution I found was to do the following, I'm assuming this is music that you're storing in OneDrive and appears in Groove under the filter "On OneDrive":
- Take a copy of the album folder from OneDrive to your local machine.
- In Groove, right click the album and delete it, make sure it deletes from the OneDrive folder and your local synced version.
- Update the tags in mp3tag to your liking. Personally I select all files and do a "Tag Cut" then "Tag Paste" to force a re-write of all of them.
- Copy the updated folder back into OneDrive.
- Wait for it to sync up again and for the new album to appear back in Groove.
It's a bit of a pain but so far it seems to work for me.