I'm at 80 hours uptime and my svchost.exe is at 17, 108K. I apparently do still have a small memory leak after the IE10 uninstall. Now I'm not sure if I should install IE11 or I need to go through my last updates to find a problem. While it is certainly allot slower and less of an issue, it's still more than the 8.5MB it normally runs at.
Perhaps I'll just restart first and see what it does again. I'm just getting tired of this issue as it seems to be the only problem I have.