I've had so many conversations with Microsoft Tech Support on this subject over the last 18 months that I've literally lost count, and gave up a few months ago.
Every call, dozens of them, would last upward of 30 minutes, and precisely none of them resulted in a solution. Many hours lost to this bug.
The last call was this:
"We're rolling out an update in June. This will allow you to delete categories."
I haven't seen any update so far.
Like your customer, I also have the problem where I've lost categories, recreated them, and suddenly the lost ones come back, so now I have nearly a thousand duplicate and triplicate categories, to the point where it's completely unusable.
I spent two years meticulously creating and maintaining a client/supplier database using categories, only to have Microsoft blow it apart in 24 hours.