Solved slow network speeds with jail (and vnet) on FreeBSD 14.0-RELEASE

uchman

uchman

I upgraded my home server to FreeBSD 14.0-RELEASE yesterday and since then i'm experiencing really slow speeds to my jails (bhyve guests are not affected).

I use iocage and vm to handle my jails and here is a sample from ifconfig:

Code: 
em0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=4e504bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
    ether 1c:69:7a:6b:dd:d6
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
em0.100: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    description: vm-vlan/dmz/em0.100
    options=4200001<RXCSUM,RXCSUM_IPV6,MEXTPG>
    ether 1c:69:7a:6b:dd:d6
    groups: vlan vm-vlan viid-ffdc0@
    vlan: 100 vlanproto: 802.1q vlanpcp: 0 parent interface: em0
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vm-dmz: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    options=0
    ether 42:bd:e6:76:4c:eb
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 17 priority 128 path cost 2000
    member: em0.100 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 5 priority 128 path cost 20000
    groups: bridge vm-switch viid-6ffdb@
    nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.1: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
    description: associated with jail: web as nic: epair0b
    options=8<VLAN_MTU>
    ether 94:de:80:e7:ba:31
    hwaddr 02:cf:c5:d1:c6:0a
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

As I said other bridges that have bhyve guests connected to then seem to work without a problem. So I thing the problem might be vnet.
 
I'm on FreeBSD 14.0-RELEASE and have a vnet jail running. I'm getting these speeds from the jail to a truenas host on my network:

Command from from my vnet jail to my truenas machine
Code: 
$ dd if=/dev/zero bs=1048576 count=1024 | nc -N 192.168.1.7 12345
1024+0 records in
1024+0 records out
1073741824 bytes transferred in 9.275860 secs (115756574 bytes/sec)

$ ifconfig
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0 metric 0 mtu 33152
        options=0
        groups: pflog
epair4b: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether 02:42:01:88:fa:0b
        inet 192.168.1.4 netmask 0xffffff00 broadcast 192.168.1.255
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Yes, to my jail from my truenas server the speed dropped off quite a bit, getting about 74 MB/s vs 115MB/s the other way. 192.168.1.4 is my jail

Code: 
dd if=/dev/zero bs=1048576 count=1024 | nc -Nv 192.168.1.4 12345
Connection to 192.168.1.4 12345 port [tcp/*] succeeded!
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.5302 s, 73.9 MB/s
 
The old "works for me"

Below are transfers from my laptop client to three different destinations.
Code: 
## host machine
klein:~ peter$ scp 100MB.zip hyper:

100MB.zip                                                                                                                                                   100%  100MB  25.1MB/s   00:03   

## individual jail on host machine
klein:~ peter$ scp 100MB.zip web:

100MB.zip                                                                                                                                                     7% 7936KB  59.7KB/s   26:22 ETA

^Cklein:~ peter$

## bhyve on host machine
klein:~ peter$ scp 100MB.zip docker:

100MB.zip                                                                                                                                                   100%  100MB  24.9MB/s   00:04

Even between jails on the same machine I see these slow speeds.

I cant really get my head around this.
 
I experienced the same issue after upgrading to 14.0-RELEASE. Download speeds from within a jail dropped to kBps.

I have two ixl interfaces, joined in a lagg, with tagged VLANs on top. I then bridge to different VNET jails. The VNET jails run their own pf.

Setting -lro on the two ixl interfaces resolved the issue immediately.
 
You must log in or register to reply here.
Back
Top