vm-bhyve network troubleshooting

peterchris

peterchris

I've just started using vm-bhyve, and none of vm's have network connectivity. Here's what everything looks like, following the simple example in the vm-bhyve wiki to install a FreeBSD guest with default settings:


Code: 
[peter@orca] /vms/.config# vm switch list
NAME            TYPE       IDENT       VLAN      NAT          PORTS
public          auto       bridge0     -         -            igb2


[peter@orca] /vms/.config# vm switch info public
------------------------
Virtual Switch: public
------------------------
  type: auto
  ident: bridge0
  vlan: -
  nat: -
  physical-ports: igb2
  bytes-in: 313918 (306.560K)
  bytes-out: 71767 (70.084K)

  virtual-port
    device: tap0
    vm: myguest

[peter@orca] /vms/.config# ifconfig igb2
igb2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2500b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6>
        ether ac:1f:6b:20:63:0e
        hwaddr ac:1f:6b:20:63:0e
        inet6 fe80::ae1f:6bff:fe20:630e%igb2 prefixlen 64 scopeid 0x3 
        inet6 2605:6000:f491:c300:ae1f:6bff:fe20:630e prefixlen 64 autoconf 
        inet 192.168.0.44 netmask 0xffffff00 broadcast 192.168.0.255 
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active

Here's what the FreeBSD guest spits out as it's starting up during the install:

Code: 
Starting Network: lo0 vtnet0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128 
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 
        inet 127.0.0.1 netmask 0xff000000 
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo 
vtnet0: flags=8902<BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
        ether 58:9c:fc:0c:cb:a4
        hwaddr 58:9c:fc:0c:cb:a4
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
Starting devd.
Starting Network: vtnet0.
vtnet0: flags=8902<BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
        ether 58:9c:fc:0c:cb:a4
        hwaddr 58:9c:fc:0c:cb:a4
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
add host 127.0.0.1: gateway lo0 fib 0: route already in table
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Generating host.conf.
eval: cannot create /etc/host.conf: Read-only file system
eval: cannot create /etc/host.conf: Read-only file system
eval: cannot create /etc/host.conf: Read-only file system

Then it appears to take a long time once it gets to DHCP configuration, leading me to believe it didn't work; then, it can't download any of the files it needs and the installation fails. If I drop into a shell I see this:

Code: 
root@myguest:~ # ifconfig
vtnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
        ether 58:9c:fc:0c:cb:a4
        hwaddr 58:9c:fc:0c:cb:a4
        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128 
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 
        inet 127.0.0.1 netmask 0xff000000 
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo 

root@myguest:~ # netstat -r
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
0.0.0.0/8          link#1             U        vtnet0
localhost          link#2             UH          lo0

Internet6:
Destination        Gateway            Flags     Netif Expire
::/96              localhost          UGRS        lo0
localhost          link#2             UH          lo0
::ffff:0.0.0.0/96  localhost          UGRS        lo0
fe80::/10          localhost          UGRS        lo0
fe80::%lo0/64      link#2             U           lo0
fe80::1%lo0        link#2             UHS         lo0
ff02::/16          localhost          UGRS        lo0

So it looks like I've got to configure manually. If so, this is what I've got:

Code: 
ifconfig vtnet0 inet 192.168.0.77 netmask 0xffffff00 broadcast  192.168.0.255

route add default 192.168.0.1

Now my netstat on the guest looks much like that on the host:

Code: 
root@myguest:~ # netstat -r
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.0.1        UGS      vtnet0
localhost          link#2             UH          lo0
192.168.0.0/24     link#1             U        vtnet0
192.168.0.77       link#1             UHS         lo0

Internet6:
Destination        Gateway            Flags     Netif Expire
::/96              localhost          UGRS        lo0
localhost          link#2             UH          lo0
::ffff:0.0.0.0/96  localhost          UGRS        lo0
fe80::/10          localhost          UGRS        lo0
fe80::%lo0/64      link#2             U           lo0
fe80::1%lo0        link#2             UHS         lo0
ff02::/16          localhost          UGRS        lo0

However, there is no connectivity; I can't ping anything successfully.

I have similar troubles getting an Ubuntu VM to network.

How can I get it to work?
 
Following this nice post from Roma's blog I've got a VM created with the bhyve "vmrun.sh" script to network. DHCP still didn't work, so I still had to follow the procedure above to assign a static IP. But then it worked! I guess I will try vm-bhyve's "manual" option for bridged networking and report back.
 
I learned about PCI passthru from the aforementioned Roma's blog. If you don't know, this means giving your VM some hardware, so its not quite 100% virtual. Remembering my mobo has 4 ethernet interfaces, I just got Ubuntu to install with (drum roll) DHCP working perfectly during the install process-- I didn't have to jack with network config whatsoever. I did it with the help of this other blog. (Replace the virtio-net stuff with your passthru as detailed in Roma's blog).

I couldn't get the passthru working with vm-bhyve. So it looks like i'm goin' native... bhyve... yeah baby.

Actually if anyone knows how to configure vm-bhyve to work with a network interface PCI passthru, please share.
 
You must log in or register to reply here.
Back
Top