After troubleshooting this issue I wanted to add my experience as it might be helpful to those still having issues. We were also seeing NDRs as described by @ProfessionalGeek who opened this thread. Long story short....
Root cause: Thanks to DMARC reporting (and a good baseline) we found the domain had recently been used in a mass spam mail out due to a misconfigured SPF record. We saw spammers sending out mail from the domain (passing SPF & DMARC) which spiked for 3 days and then stopped but then google started rejecting the domain due to low reputation of the domain. Kinda makes sense right?
Takeaway #1 - Setup DMARC reporting!
Takeaway #2 - Check DMARC and SPF records DMARC should not be p=none!! and SPF record should -all
Takeaway #3 - Are you 100% sure your domain is really not sending spam? Spammers can be like trying to find a water leak *sad face*
The IPv4 Work Around
As suggested we adjusted to hit the IPv4 addresses rather than IPv6. The result was 1 email out of 3 would deliver. For me that is totally strange but the inconsistencies don't stop there. After a week all mail via IPv4 connectors worked but any attempts via IPv6 continue to fail to this day.
My Take Away
Is Microsoft at fault? No I don't think so.
Is Google at fault? No I don't think so. We confirmed for other domains IPv6 worked without issues between Microsoft and Google.
Are you are fault? Maybe... I'll let you answer that :)
But I would add that we saw clear inconstancies with google mail servers. I have a the feeling the IPv6 server farm is either not seeing enough traffic to train it's spam engine or is more harden for blocking these types of events. But it's the sort of thing that only a Google engineer can answer. *insert another sad face here*