(1) I think the USB drive should have shown up as it did for you before. Maybe try other USB ports.
(2) Right, System Recovery in the Startup Menu was for Win 8.1. Your site recommends a Win 10 Reset instead, which can keep files. But you've also got your system images which have files, settings & apps. However, you can't do either of them unless you can
get to a recovery environment. But here is a good article describing Reset...
https://support.hp.com/us-en/document/c04742289
HP PCs - Resetting Your Computer (Windows 10)
(2) Well, OMG, at your computer's site, I'm getting led by the nose from article to article starting with this one...
https://support.hp.com/us-en/product/hp-pavilion-21-h000-touchsmart-all-in-one-desktop-pc-series/6528604/model/6529764/document/c03801890
HP PCs and Tablets - About UEFI and the Startup Menu
It says, your Boot Device Options menu normally should include the Windows Boot Manager (whether or not a USB flash drive is plugged in)...

It makes me wonder again whether your HDD is getting detected. Maybe the System Diagnostics reported no error because it, too, found no HDD. Here is an article that should settle the issue...
https://support.hp.com/us-en/document/c06298595
HP Desktop PCs - Advanced Startup Troubleshooting (Windows 10, 8)
Do the things in the section "Check if the hard drive is detected in BIOS". Then do either "Hard drive is not detected in BIOS" or "Hard drive is detected in BIOS, but computer does not start".
Or just read through all that, & report back with any further questions.