Solved vm-bhyve don't initialize tap0 after an upgrade freeBSD-13.3 -> 13.4

dervishe

dervishe

Hello,
recently i upgrade my freebsd box from 13.3 to 13.4 and suddenly my debian vm networking broke.
My freebsd box is build from the sources, thus the upgrade too.
Here are the logs of vm-bhyve (before the upgrade and after the upgrade)

Rich (BB code): 
Jan 18 01:44:36: initialising
Jan 18 01:44:36:  [loader: grub]
Jan 18 01:44:36:  [cpu: 4]
Jan 18 01:44:36:  [memory: 16G]
Jan 18 01:44:36:  [hostbridge: standard]
Jan 18 01:44:36:  [com ports: com1]
Jan 18 01:44:36:  [uuid: 9e96e501-62c8-11ec-8df0-7c10c99c1764]
Jan 18 01:44:36:  [debug mode: no]
Jan 18 01:44:36:  [primary disk: /dev/zvol/zroot/vm/office_storage]
Jan 18 01:44:36:  [primary disk dev: custom]
Jan 18 01:44:36: initialising network device tap0
Jan 18 01:44:36: booting
Jan 18 01:44:36: create file /vm/office/device.map
Jan 18 01:44:36:  -> (hd0) /dev/zvol/zroot/vm/office_storage
Jan 18 01:44:36: /usr/local/sbin/grub-bhyve -c /dev/nmdm-office.1A -m /vm/office/device.map -M 16G -r hd0,1 -d /boot/grub office
Jan 18 01:44:44:  [bhyve options: -c 4 -m 16G -AHP -U 9e96e501-62c8-11ec-8df0-7c10c99c1764 -u]
Jan 18 01:44:44:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/dev/zvol/zroot/vm/office_storage -s 5:0,virtio-net,tap0,mac=XX:XX:XX:XX:XX:XX]
Jan 18 01:44:44:  [bhyve console: -l com1,/dev/nmdm-office.1A]
Jan 18 01:44:44: starting bhyve (run 1)

Code: 
Jan 18 03:27:08: initialising
Jan 18 03:27:08:  [loader: grub]
Jan 18 03:27:08:  [cpu: 4]
Jan 18 03:27:08:  [memory: 16G]
Jan 18 03:27:08:  [hostbridge: standard]
Jan 18 03:27:08:  [com ports: com1]
Jan 18 03:27:08:  [uuid: 9e96e501-62c8-11ec-8df0-7c10c99c1764]
Jan 18 03:27:08:  [debug mode: no]
Jan 18 03:27:08:  [primary disk: /dev/zvol/zroot/vm/office_storage]
Jan 18 03:27:08:  [primary disk dev: custom]
Jan 18 03:27:08: booting
Jan 18 03:27:08: create file /vm/office/device.map
Jan 18 03:27:08:  -> (hd0) /dev/zvol/zroot/vm/office_storage
Jan 18 03:27:08: /usr/local/sbin/grub-bhyve -c /dev/nmdm-office.1A -m /vm/office/device.map -M 16G -r hd0,1 -d /boot/grub office
Jan 18 03:27:16:  [bhyve options: -c 4 -m 16G -AHP -U 9e96e501-62c8-11ec-8df0-7c10c99c1764 -u]
Jan 18 03:27:16:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/dev/zvol/zroot/vm/office_storage]
Jan 18 03:27:16:  [bhyve console: -l com1,/dev/nmdm-office.1A]
Jan 18 03:27:16: starting bhyve (run 1)
As you can see, there is the part "-s 5:0,virtio-net,tap0,mac=XX:XX:XX:XX:XX:XX" which is missing during the boot process and of course the "initialising network device tap0" part during the initialisation.

My r.conf relevant part is:
Code: 
cloned_interfaces="bridge0 bridge1 tap0"
ifconfig_bridge1_name="bridgepriv"
ifconfig_bridgepriv="inet A.B.C.D.1/24 addm tap0 up"
vm_enable="YES"
vm_dir="zfs:zroot/vm"
vm_list="office"

My vm config is:
Code: 
loader="grub"
cpu="4"
memory=16G
network0_type="virtio-net"
network0_device="tap0"
disk0_name="/dev/zvol/zroot/vm/office_storage"
disk0_type="virtio-blk"
disk0_dev="custom"
grub_run_partition="1"
grub_run_dir="/boot/grub"
uuid="9e96e501-62c8-11ec-8df0-7c10c99c1764"
network0_mac="XX:XX:XX:XX:XX:XX"

The relevant part in the kernel conf is:
Code: 
# VirtIO support
device          virtio                  # Generic VirtIO bus (required)
device          virtio_pci              # VirtIO PCI device
device          vtnet                   # VirtIO Ethernet device
device          virtio_blk              # VirtIO Block device
device          virtio_scsi             # VirtIO SCSI device
device          virtio_balloon          # VirtIO Memory Balloon device

The ifconfig part:
Code: 
bridgepriv: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether YY:YY:YY:YY:YY:YY
        inet A.B.C.1 netmask 0xffffff00 broadcast A.B.C.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 5 priority 128 path cost 2000000
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
tap0: flags=8903<UP,BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        ether XX:XX:XX:XX:XX:XX
        groups: tap
        media: Ethernet autoselect
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

My freeBSD version: "13.4-RELEASE-p2 FreeBSD 13.4-RELEASE-p2 releng/13.4-n258268-3f40d5821eca CLOUD amd64"
vm-bhyve version: 1.6.0

I have to admit that i'm a little bit lost here...
Do you have any clues ?
Thanks a lot for your answers :)
 
dervishe said:
Code: 
cloned_interfaces="bridge0 bridge1 tap0"
ifconfig_bridgepriv="inet A.B.C.D.1/24 addm tap0 up"
Click to expand...
Remove tap0 here, the interface doesn't exist yet.

dervishe said:
Code: 
network0_device="tap0"
Click to expand...
Remove this line, it will dynamically add the first available interface. And add network0_switch to tie it to a specific switch/network.

dervishe said:
The relevant part in the kernel conf is:
Click to expand...
They are included with the GENERIC kernel. And they're for guests, not the host.
 
Thanks a lot for your answer :)
It's solve my problem but there's at least one last slight problem, i already have two bridges running for some jails. I would like the tap interface added to one of them.
When i try to put:
* network0_bridge="bridge1" or
* network0_bridge="bridgepriv"
(bridge1 is defined in rc.conf and bridgepub is the renamed interface)
The vm-bhyve logs says: "failed to find virtual switch 'bridgepriv'"
How can i map the bridge with the virtual switch ?
(I'm not very skilled with bhyve as you can see)
 
Either create a "switch" (bridge(4)) with vm switch .., or import your custom bridge in vm.

github.com

Virtual Switches

Shell based, minimal dependency bhyve manager. Contribute to churchers/vm-bhyve development by creating an account on GitHub.
github.com github.com
 
You must log in or register to reply here.
Back
Top