No guest interrupts for passthru device - how do I troubleshoot?

D

dvlee

I am attempting to pass-through an i210 based network adapter to an OpenBSD 6.3 guest running under bhyve.

There is nothing amiss in dmesg; the card shows up on the guest as expected:
em0 at pci0 dev 10 function 0 "Intel I210" rev 0x03: msi, address 68:05:ca:82:a9:64

But, the pass-through device never gets an interrupt (as reported on the guest by vmstat -iz). The virtio block and network devices are running properly.

How can I troubleshoot this?

The only errors I see come from the em0 watchdog on the guest. Nothing else - not from bhyve or the host, not from the guest - indicates any error or warning. I have fiddled with every option to the point of insanity. So, I am looking for general ideas on how to gather more information.

The host is FreeBSD 11.2-RELEASE-p3, running a custom kernel with the patch for bug PR 229852applied (and nothing else). The network card, when not in passthru mode, is loaded by the host as follows:
igb0: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> mem 0xf7d00000-0xf7dfffff,0xf7e00000-0xf7e03fff irq 18 at device 0.0 on pci2
igb0: Using MSIX interrupts with 5 vectors

Thank you.
 
I could be totally shooting in the dark, but I was having all kinds of issues on different OSes with PCI passthrough devices, and source for all of them was MSI-X support. Try disabling MSI-X globally or per-device, and sorry, I don't know how you would do that on OpenBSD (FreeBSD equivalent would be hw.pci.enable_msix and dev.igb.0.iflib.disable_msix, respectively).
 
dvlee Did you find a solution for this ?
I have the same NIC with exactly the same symptoms. I tried a bunch of options, different slots etc, nothing worked.
Since I don't have permanent IPMI for this server, looks like I have to give up...
 
Hi. I had no luck. I am certain interrupts are not getting through to the guest em driver. But beyond that, the tools simply aren't available (or aren't publicized) to isolate the problem. For the record, the machine is a Dell T20 (which has a C226 chipset). I did disable MSI-X on the host, but there is no corresponding setting in OpenBSD and em reported it was using MSI.

I found the experience quite frustrating. Bhyve is fragile, not well documented, mysterious when it fails.
 
You must log in or register to reply here.
Back
Top