Yes you can, SSH 8.3ish uses FIDO2 and doesn't do anything Yubikey-specific. That means you don't have to bother with all the agent stuff, and it works with any dirt-cheap FIDO2 key.
EDIT: I'm going to post a writeup tomorrow detailing how to do this, because it's wonderful and super secure.
EDIT: I'm going to post a writeup tomorrow detailing how to do this, because it's wonderful and super secure.