I had the similar issues with usb passthru, but i my case it start working but randomly crashes the host FreeBSD machine during write access to the usb device. Nec USB controllers had most issues. After then I have decide to move towards qemu (qemu9 has some memory issues, so use qemu8). A bit slow, but acceptable on modern PCs for generic work.
Pros:
- redirecting even single usb device connected to any port is very easy and cause 0 issues.
- Implementation of VNC server is amazing
- in general way more polished then bhyve
Cons:
- I didn't manage to run hardware virtualization acceleration ( kvm, the port has no maintainer since 2016 afasik )