Organizing and accessing contact information in Outlook.com
Something that worked for me was that I noticed that some of the contacts were linked (this appears in the menu bar at the top of the screen along side New, Manage, Delete etc.). If you break the links between the contacts by clicking on the cross next to each linked account, you then edit (or delete) each contact individually.
I think this may have occurred on my account where I have multiple outlook accounts set up on my iphone, that have then been uploaded to the server.