Thank you Eddie, this makes a lot of sense and explains the issues I was having. I was able to fix my error and get upgraded just a couple days after the dev branch was released. I was able to see what was causing the error and the setupdiag tool And it was caused by a faulty driver for a realtek network controller. I had tried many times to get the latest driver from the manufacturer's website but my problem was that Microsoft was pulling in a new driver in place of the one I had already installed on my system and in doing so caused the update to fail. The driver Microsoft pulled was from 2015 and it completely wiped out my 2021 driver. I was lucky enough to have two NICs on my board so I disabled the realtek NIC in BIOS and I was able to update using my Intel nic.
So even if you have the latest drivers the upgrade can still fail if Microsoft decides that it wants to pull an old driver to replace your current driver.