Solved vm-bhyve share wlan0 with archlinux

codedmart · Feb 3, 2020

I am trying to get archlinux running with bhyve using vm-bhyve. I followed a mix of the quick start guide here and NAT instructions here. Then in arch I just set a manual ip of 192.168.8.2, but not internet connection still in my gues archlinux system.

Code:

# vm conf file:
loader="grub"
cpu=4
memory=8192M
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
grub_install0="linux /arch/boot/x86_64/vmlinuz archisobasedir=arch archisolabel=ARCH_202002 ro"
grub_install1="initrd /arch/boot/x86_64/archiso.img"
graphics="yes"

Code:

# /etc/rc.conf
vm_enable="YES"
vm_dir="zfs:zroot/vm"
gateway_enable="yes"
pf_enable="yes"
ifconfig_vm-public="DHCP" # <- I didn't add this line

Code:

# /etc/pf.conf
nat on wlan0 from {192.168.8.0/24} to any -> (wlan0)

Code:

➜  ~ sudo vm switch list
NAME    TYPE      IFACE      ADDRESS         PRIVATE  MTU  VLAN  PORTS
public  standard  vm-public  192.168.8.1/24  no       -    -     -

I also keep seeing this error whenever running any related to `vm` commands or `netif`

Code:

/etc/rc.conf: ifconfig_vm-public=DHCP: not found

.

Any help would be appreciated.

SirDice · Feb 3, 2020

Remove that ifconfig_vm-public line from rc.conf.

Then vm switch public add wlan0, so the vm-public bridge is actually connected to something.

codedmart · Feb 3, 2020

SirDice said:
Remove that ifconfig_vm-public line from rc.conf.

Then vm switch public add wlan0, so the vm-public bridge is actually connected to something.

I keep removing the ifconfig_vm-public line, but it keeps showing back up. vm or something is adding it back in.

I did vm switch public add wlan0. Still no luck. I started over as well, cleared everything, and tried again. Still no internet.

SirDice · Feb 3, 2020

codedmart said:
I keep removing the ifconfig_vm-public line, but it keeps showing back up. vm or something is adding it back in.

I've been using sysutils/vm-bhyve for quite some time and it never did that. None of its scripts modify rc.conf.

codedmart · Feb 3, 2020

Hmm... I am not sure then what is causing it. I know I keep deleting it and it keeps showing back up.

codedmart · Feb 3, 2020

Could that be causing my guest network issues?

codedmart · Feb 3, 2020

That ifconfig_vm-public gets added to rc.conf immediately after I run vm switch create -a 192.168.8.1/24 public.

SirDice · Feb 3, 2020

Well, your vm-public bridge(4) interface is just dangling, not attached to anything besides the VM. So it's never going to be able to talk to anything else.

It's as if you attached a machine to a single switch and this switch isn't attached to your network. So it's just a separate, stand-alone, network.

I have two (virtual) networks, both are attached to an upstream lagg0 interface, one uses VLAN 10, the other VLAN 11:

Code:

root@hosaka:~ # vm switch list
NAME     TYPE      IFACE       ADDRESS  PRIVATE  MTU   VLAN  PORTS
servers  standard  vm-servers  -        no       9000  11    lagg0
public   standard  vm-public   -        no       9000  10    lagg0

This lagg0 interface is on the host and consists of igb1 and igb2. Those interfaces are physically attached to the rest of the network. VMs are connected to either servers or public.

codedmart · Feb 3, 2020

I am not trying to do anything extra or special here. I just want internet in my guest. I am running on a laptop with a wlan0 wireless connection. I don't know much about networking other then some basics.

I do know that regardless of what I name the switch with vm switch add ... a corresponding line shows up in /etc/rc.conf just after. Then I start seeing errors about that. So I am not sure what I am doing wrong here.

Thank you for your insight and help as well.

SirDice · Feb 3, 2020

Can you post your /vm/.config/system.conf? Maybe something sneaked in there.

codedmart · Feb 3, 2020

 ➜  ~ cat /zroot/vm/.config/system.conf

switch_list=""

codedmart · Feb 3, 2020

Also to be upfront I did not follow or do anything from the handbook and bhyve section like Preparing the host. I assumed vm-bhyve could handle everything for me. Not sure if that matters.

SirDice · Feb 3, 2020

Your config is still pristine.

Ok, make sure the VM is stopped. Just stop all VMs; vm stopall. Then look for bridge(4) interfaces with ifconfig(8). Destroy them all. I assume you don't have any other bridge(4) interfaces.

Then run vm switch create public. And vm switch public add wlan0.

Quickstart

Shell based, minimal dependency bhyve manager. Contribute to churchers/vm-bhyve development by creating an account on GitHub.

github.com

codedmart · Feb 3, 2020

OK so I ran sudo vm stopall then checked ifconfig

Code:

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 08:be:ac:04:ed:9b
        inet 192.168.1.186 netmask 0xffffff00 broadcast 192.168.1.255
        groups: wlan
        ssid blmartin channel 1 (2412 MHz 11g ht/20) bssid 4c:ed:fb:ac:08:28
        regdomain FCC country US authmode WPA2/802.11i privacy ON
        deftxkey UNDEF AES-CCM 2:128-bit AES-CCM 3:128-bit txpower 30 bmiss 7
        scanvalid 60 protmode CTS ht20 ampdulimit 64k ampdudensity 4 shortgi
        -stbc -ldpc wme roaming MANUAL
        media: IEEE 802.11 Wireless Ethernet MCS mode 11ng
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

then

 sudo vm switch create public

sudo vm switch add public wlan0

Again checked ifconfig

Code:

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vm-public: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 22:86:5c:53:97:e8
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        groups: bridge vm-switch viid-4c918@
        nd6 options=9<PERFORMNUD,IFDISABLED>
wlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 08:be:ac:04:ed:9b
        inet 192.168.1.186 netmask 0xffffff00 broadcast 192.168.1.255
        groups: wlan
        ssid blmartin channel 1 (2412 MHz 11g ht/20) bssid 4c:ed:fb:ac:08:28
        regdomain FCC country US authmode WPA2/802.11i privacy ON
        deftxkey UNDEF AES-CCM 2:128-bit txpower 30 bmiss 7 scanvalid 60
        protmode CTS ht20 ampdulimit 64k ampdudensity 4 shortgi -stbc -ldpc
        wme roaming MANUAL
        media: IEEE 802.11 Wireless Ethernet MCS mode 11ng
        status: associated
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

And again at the bottom of my /etc/rc.conf file is ifconfig_vm-public="DHCP"

SirDice · Feb 3, 2020

What's in /etc/wpa_supplicant? Is that configured to use wlan0? If you look closely you'll notice your wlan0 is now offline too, this shouldn't happen.

usdmatt · Feb 3, 2020

You shouldn't need to add wlan0 to the switch. Bridging to a wireless interface often doesn't work which is why I'd usually suggest nat.

Create the virtual switch and make sure it is created correctly. It should add it to the vm-bhyve config file, and should not change anything in /etc/rc.conf. Is it possible something else on the system is automatically trying to enable DHCP for new interfaces?

You should end up with something like the following -

Code:

# vm switch list
NAME    TYPE      IFACE      ADDRESS         PRIVATE  MTU  VLAN  PORTS
public  standard  vm-public  192.168.8.1/24  no       -    -     -


# ifconfig vm-public
vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 06:26:a9:7d:96:30
        inet 192.168.8.1 netmask 0xffffff00 broadcast 192.168.8.255

# cat vm/.config/system.conf
switch_list="public"
type_public="standard"
addr_public="192.168.8.1/24"

Start a guest attached to that switch. It should automatically connect a tap interface to that bridge. At this point there is no reason the guest should not be able to ping 192.168.8.1 from a 192.168.8.0/24 address, assuming no firewall on the host is stopping it.

Get to that point before worrying about Internet access otherwise you're trying to run before you can walk.

SirDice · Feb 3, 2020

usdmatt said:
Bridging to a wireless interface often doesn't work which is why I'd usually suggest nat.

I've never used it with wireless interface, but I assume it's the same sort of problem with trying to set up a bridging WAP and trying to bridge the ethernet and wireless interfaces?

codedmart · Feb 3, 2020

SirDice said:
What's in /etc/wpa_supplicant? Is that configured to use wlan0? If you look closely you'll notice your wlan0 is now offline too, this shouldn't happen.

Yeah the wireless network restarts. I edited my post to show the proper ifconfig after the wireless had come back.

 network={

  ssid="blmartin"

  psk="super-secret-passkey"

}

codedmart · Feb 3, 2020

usdmatt said:
You shouldn't need to add wlan0 to the switch. Bridging to a wireless interface often doesn't work which is why I'd usually suggest nat.

Create the virtual switch and make sure it is created correctly. It should add it to the vm-bhyve config file, and should not change anything in /etc/rc.conf. Is it possible something else on the system is automatically trying to enable DHCP for new interfaces?

You should end up with something like the following -

Code:

# vm switch list NAME TYPE IFACE ADDRESS PRIVATE MTU VLAN PORTS public standard vm-public 192.168.8.1/24 no - - - # ifconfig vm-public vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 06:26:a9:7d:96:30 inet 192.168.8.1 netmask 0xffffff00 broadcast 192.168.8.255 # cat vm/.config/system.conf switch_list="public" type_public="standard" addr_public="192.168.8.1/24"

Start a guest attached to that switch. It should automatically connect a tap interface to that bridge. At this point there is no reason the guest should not be able to ping 192.168.8.1 from a 192.168.8.0/24 address, assuming no firewall on the host is stopping it.

Get to that point before worrying about Internet access otherwise you're trying to run before you can walk.

That's the thing. Just a simple vm switch create -a 192.168.8.1/24 public adds it to /etc/rc.conf. I am using:

 ➜  ~ vm version

vm-bhyve: Bhyve virtual machine management v1.3 (rev. 103101)

usdmatt · Feb 3, 2020

vm-bhyve is not changing rc.conf. Something else on the system is seeing a new interface and trying to enable DHCP on it.

That line shouldn't really actually affect anything, but also shouldn't be there and as you say in the original post, appears to be producing an error whenever rc.conf is read

codedmart · Feb 3, 2020

usdmatt said:
vm-bhyve is not changing rc.conf. Something else on the system is seeing a new interface and trying to enable DHCP on it.

That line shouldn't really actually affect anything, but also shouldn't be there and as you say in the original post, appears to be producing an error whenever rc.conf is read

That makes sense. Just not sure what is causing it to change. For reference if it helps here are my loader.conf and rc.conf

Code:

# /boot/loader.conf
aesni_load="YES"
geom_eli_load="YES"
security.bsd.allow_destructive_dtrace=0
opensolaris_load="YES"
zfs_load="YES"
vfs.zfs.min_auto_ashift=12
#hw.psm.synaptics_support=1
atp_load="YES"
asmc_load="YES"

# /etc/rc.conf
clear_tmp_enable="YES"
syslogd_flags="-ss"
sendmail_enable="NONE"
hostname="free"
keymap="us.macbook.kbd"
local_unbound_enable="YES"
sshd_enable="YES"
moused_enable="YES"
ntpdate_enable="YES"
ntpd_enable="YES"
powerdxx_enable="YES"
dbus_enable="YES"
cupsd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
zfs_enable="YES"
wlans_rtwn0="wlan0"
ifconfig_wlan0="WPA SYNCDHCP"
kld_list="/boot/modules/i915kms.ko"
allscreens_flags="-f /usr/local/share/vt/fonts/terminus-b24"
vm_enable="YES"
vm_dir="zfs:zroot/vm"
gateway_enable="yes"
pf_enable="yes"

codedmart · Feb 18, 2020

I managed to get this working fine just manually adding a bridge with ifconfig. Thanks for all the help and insights.

jcamos · Apr 8, 2023

codedmart said:
I managed to get this working fine just manually adding a bridge with ifconfig. Thanks for all the help and insights.

Hi codedmart,

Same problem here. Can you elaborate how you were able to make it work?
Thanks a lot!

dilyanb · Apr 17, 2023

+1
I've been trying to make bhyve work with wla0 w/o success. Can you please share how you solved it?

dilyanb · Apr 18, 2023

I managed to make it work with all the hints above +

when the vm switch was created, ifconfig was not showing it up and running. So, I had to do:

ifconfig vm-public inet 192.168.1.1/24 up

and then the virtual machines were pinging the gateway 192.168.8.1. Then I added the following line to /etc/pf.conf to NAT the 192.168.8.0 network to the wireless and it started working:

nat on wlan0 from 192.168.8.0/24 to any -> (wlan0)

Solved vm-bhyve share wlan0 with archlinux

codedmart

SirDice

Administrator

codedmart

SirDice

Administrator

codedmart

codedmart

codedmart

SirDice

Administrator

codedmart

SirDice

Administrator

codedmart

codedmart

SirDice

Administrator

Quickstart

codedmart

SirDice

Administrator

usdmatt

SirDice

Administrator

codedmart

codedmart

usdmatt

codedmart

codedmart

jcamos

dilyanb

dilyanb