Hi Andrew,
Thanks. I finally found the solution to my problem here: https://forums.iis.net/t/1229869.aspx. The Wireless broadcom driver, even after uninstalling leave some registry entries that caused the problem. I de-activated the Wireless device in BIOS, removed the driver completely, did not allowed Windows Update to update the driver and finally went thorugh the registry and deleted every reference to Broadcom I could find. "After a bunch of digging, I found my problem was that my Broadcom Wireless driver had added some registry keys to HKLM\SYSTEM\CurrentControlSet\Control\Cryptography\Providers and iissetup.exe was just picking the first provider from the list, which was the Broadcom one, and failing."
After i did that, I also proceed with the USB-ISO created installation(Setup) without any connection to Internet (unplug Ethernet cable) and I also clicked to not make updates before installing(which is the recommended option during setup).
Then I am now with 1709 with latest January updates. Wireless Broadcom driver is back enabled up and running as well.
My 2012 Optiplex has been failing the 1709 update for the last 5 months and this finally fixed it. I have not used a broadcom wireless adapter in years and the software had already been removed but the registry entries above were present. Once I removed them the 1709 update finally went through. Thank you for figuring this one out and posting to the community!