Organizing and accessing contact information in Outlook.com
If you cannot delete the contact in People, try this:
Go to your Outlook Email (Outlook Live) and find an email with that particular contact name/address in it. Open the email and click on the name/email address (could be the sender or other recipients) - you should see a pane at the right hand side. Click on the 3 dots symbol - you will see option to Edit contact or Delete. (Alternatively just hover the mouse over the sender's coloured circle or other recipient address - you wiil see a small window with the 3 dots.)