bhyve VNET interconnect vm-bhyve + jails

[freeBSDNew]

Hello

Unfortunately, due to a lack of suitable real hardware at the moment, I can't try it out myself or experiment, and so far I haven't found the necessary information to answer the following question:

I want to set up a FreeBSD 14 host running various containers and virtual machines.

I plan to set up a virtual network. I want "private" and "public" bridges to allow the appropriate connections between purely internal servers (e.g. database), segregated from external traffic (e.g. reverse proxy).

What I can't try is how to connect bhyve guests (e.g. created/administered using vm-bhyve) to these bridges, as vm-bhyve uses a different nomenclature.

However, I want to design my VNET and my rc.conf, etc. from the outset in such a way that I don't have to make any changes to it later in real operation to make this possible. And I don't want to reserve a physical network interface for Bhyve guests, nor do I want internal connections from these guests directed to jails/vms on the same server to have to go through the external network first.

I'm currently using a FreeBSD hosted in VirtualBox and on the physical machine (MacOS Ventura 13.6 / Intel Core i9, 6 Cores) running this, I haven't found a straightforward (if at all possible) way to allow VT-x for virtual machines, so I can't set up a VM inside the FreeBSD guest to test and develop this scenario.

So: (How) Can I connect Bhyve guests to the internal VNET e.g. a bridge (ifconfig create bridge0) and vice versa?


Thank you very much (and sorry for the may be stupid question)

 

[SirDice]

VNET has nothing to do with this. It's for jails.

I plan to set up a virtual network. I want "private" and "public" bridges to allow the appropriate connections between purely internal servers (e.g. database), segregated from external traffic (e.g. reverse proxy).

vm switch create public and vm switch create private.

And I don't want to reserve a physical network interface for Bhyve guests,

Then don't add one. It's not a requirement for a bridge(4) to work.

However, I want to design my VNET and my rc.conf, etc. from the outset in such a way that I don't have to make any changes to it later in real operation to make this possible.

Confusing. I'd just create the "switches" (those are bridge(4) interfaces) with vm(8). But if you want you could create them through rc.conf and import them as "custom" switches.

Virtual Switches

Shell based, minimal dependency bhyve manager. Contribute to churchers/vm-bhyve development by creating an account on GitHub.

github.com

 

[markus]

I know it's a little late, but this is how I solved the problem:

Create a switch in /etc/rc.conf ...

cloned_interfaces="bridge0"
ifconfig_bridge0_name="re1bridge"
ifconfig_re1bridge="inet 172.31.12.1/24"

Define the switch in Bhyve and call it e.g. "private" ...

# vm switch create -t manual -b re1bridge private

Add the switch to the VM ...

# vm add -d network -s private ghostbsd01

In my case I had to restart the vm service in order to apply the changes. This stops all running VMs!!! ...

# service vm restart

Now your VM will start with an additional interface connected to "re1bridge"...

# vm start ghostbsd01

On the host you can see a tap interface of the VM connected to the switch ...

# ifconfig re1bridge
re1bridge: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=0
        ether 58:9c:fc:10:89:4e
        inet 172.31.12.1 netmask 0xffffff00 broadcast 172.31.12.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 19 priority 128 path cost 2000000
        member: e0a_hugo02 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 12 priority 128 path cost 2000
        member: e0a_hugo01 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 8 priority 128 path cost 2000
        groups: bridge vm-switch viid-fdecc@
        nd6 options=9<PERFORMNUD,IFDISABLED>