I am currently experimenting with jails. My testing machine has a Mellanox ConnectX-4 NIC. Following the Nvidia procedure and forum posts, I upgraded the firmware and successfully enabled and created VFs. Now, I have several mceX interfaces (X=0, 1 are the PFs, and 2 – 3 are the VFs).
Following the handbook, I created a native thick jail with the mce2 interface, and it worked as expected. The jail does not have an IP by default, and the VF is controlled by the jail, so it does not appear on the host anymore.
However, since I need to type multiple commands to create a jail, I am looking for a suitable jail manager that can do the same job using a formatted command or config file. Unfortunately, I haven’t found a way to do this with BastilleBSD or CBSD.
For BastilleBSD, I used the following command to create a jail. There is no option to passthrough the NIC to the jail, and I must set an IP or use DHCP to create the jail.
For CBSD, I tried the jconstruct-tui method to create a jail. It allows me not to set an IP, but it seems that I must create a vnet epair bridge in Networking.
Any suggestions are welcome.
Code:
root@freebsd0:~ # cat /etc/iovctl.conf
PF {
device : "mlx5_core0";
num_vfs : 2,
}
DEFAULT {
passthrough : false;
}
VF-0 {
mac-addr : "aa:88:44:00:02:01";
}
VF-1 {
mac-addr : "aa:88:44:00:02:02";
}Following the handbook, I created a native thick jail with the mce2 interface, and it worked as expected. The jail does not have an IP by default, and the VF is controlled by the jail, so it does not appear on the host anymore.
Code:
root@freebsd0:~ # cat /etc/jail.conf
exec.start += "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;
classic {
# STARTUP/LOGGING
exec.consolelog = "/var/log/jail_console_${name}.log";
host.hostname = "${name}";
path = "/usr/local/jails/containers/${name}";
vnet;
#vnet.interface = "mce2.160";
vnet.interface = "mce2";
devfs_ruleset="7";
allow.raw_sockets;
}
root@freebsd0:~ # jexec -u root classic
root@classic:/ # cat /etc/rc.conf
ifconfig_mce2="mtu 9000 UP"
vlans_mce2="160"
ifconfig_mce2_160="SYNCDHCP"
root@classic:/ # ifconfig
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
mce2: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
options=7eef07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,NV,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,HWRXTSTMP,MEXTPG,TXTLS4,TXTLS6,VXLAN_HWCSUM,VXLAN_HWTSO,RXTLS4,RXTLS6>
ether aa:88:44:00:02:01
media: Ethernet 10GBase-CR1 <full-duplex,rxpause,txpause>
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
mce2.160: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9000
options=1c680703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6,MEXTPG,TXTLS4,TXTLS6>
ether aa:88:44:00:02:01
inet xxx.xxx.xxx.xxx netmask 0xffffff00 broadcast xxx.xxx.xxx.255
groups: vlan
vlan: 123 vlanproto: 802.1q vlanpcp: 0 parent interface: mce2
media: Ethernet 10GBase-CR1 <full-duplex,rxpause,txpause>
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>However, since I need to type multiple commands to create a jail, I am looking for a suitable jail manager that can do the same job using a formatted command or config file. Unfortunately, I haven’t found a way to do this with BastilleBSD or CBSD.
For BastilleBSD, I used the following command to create a jail. There is no option to passthrough the NIC to the jail, and I must set an IP or use DHCP to create the jail.
Code:
root@freebsd0:~ # bastille create -T bjail 14.1-RELEASE DHCP mce3For CBSD, I tried the jconstruct-tui method to create a jail. It allows me not to set an IP, but it seems that I must create a vnet epair bridge in Networking.
Any suggestions are welcome.
Thanks for the case. ( 2024-11-23 update: added in upcoming CBSD 14.2.0 )