I also only use IE if I have a problem with my default browser (Firefox), and even then just to check whether the problem is with Firefox or the website I am having trouble with.
I have no problem ignoring ALL updates to IE11, and when I do have to use it, doing so in a sandbox.
With a few exceptions, Microsoft is no longer supporting back-level versions of IE. Only IE 11 is supported.
Even if you never use IE 11, it is important to keep it up to date and install security patches for IE 11. The reason for this is that IE is built in to the Windows operating system, and all versions of Windows (7, 8, and 10) use various functions of IE (regardless of whether you use IE as a browser). So to keep your version of Windows protected, you have to keep IE up to date. Microsoft knows this, and thus appears to be using IE security updates to pull in non-security updates that will coerce the installation of Windows 10. Ugh!