I had the same issue with a drive formatted exFAT with an EFI (GPT) boot record.
I don’t know what the proper approach is - whether this would have worked with secure boot on or off - but I popped in a regular old FAT32 MBR USB stick and it worked like a charm.
Note: it did recognize the EFI/exFAT drive, it just wouldn’t boot from it. When I selected it nothing happened.
It will recognize and boot from it now that I disabled secure boot.
I suppose the thing to do is get into the BIOS first, disable secure boot, then try to boot from the drive. But the first boot menu came up before I could get into the BIOS.
@Fraoch I think it depends on what program you use to make the flashdrive bootable. I have used Rufus and you have to disable secure boot, but if you use the Windows Media Creation Tool then it will work with secure boot.