Solved vnet jail vlan setup, on lagg interface

S

stimi

Hi all,

I'm trying to setup a separate subnet for my jails, and I think I have everything setup correctly, but am unsure.

Here is the Switch:
Code: 
trunk 45-48 Trk2 LACP
Code: 
vlan 23
   name "Jails"
   tagged 23,Trk1-Trk2
   no ip address
   exit
Trunk 1 is the firewall, Trunk 2 is the server.
I know my switch and lagg work because my server gets internet on its default route

On the server, the lagg interface is setup as such, with the tagged vlan interface of my jails:
Code: 
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=81049b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LRO,VLAN_HWFILTER>
    ether 00:1b:21:8b:ac:f9
    laggproto lacp lagghash l2,l3,l4
    laggport: em0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
    laggport: em1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
    laggport: em2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
    laggport: em3 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
    groups: lagg
    media: Ethernet autoselect
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Code: 
lagg0.23: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=401<RXCSUM,LRO>
    ether 00:1b:21:8b:ac:f9
    inet 10.13.13.1 netmask 0xffffff00 broadcast 10.13.13.255
    groups: vlan
    vlan: 23 vlanpcp: 0 parent interface: lagg0
    media: Ethernet autoselect
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Then we have the bridge for the repair interfaces with my lagg interface, all tagged:
Code: 
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:41:ff:d9:22:00
    inet 10.13.13.12 netmask 0xffffff00 broadcast 10.13.13.255
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: epair1a.23 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 17 priority 128 path cost 2000
    member: epair0a.23 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 16 priority 128 path cost 2000
    member: lagg0.23 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 14 priority 128 path cost 2000000
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>

And the epair interface for the jail I'm trying to setup, on the host side:
Code: 
epair1a.23: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:b3:7a:1a:c8:0a
    inet 10.13.13.3 netmask 0xffffff00 broadcast 10.13.13.255
    groups: vlan
    vlan: 23 vlanpcp: 0 parent interface: epair1a
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

In the host /etc/jail.conf:
Code: 
allow.raw_sockets;
vnet;

www {      
    vnet.interface = "epair0b";
    path = "/usr/jails/www";                  
    persist;
    mount.devfs;              
    exec.start = "/bin/sh /etc/rc";          
    exec.stop = "/bin/sh /etc/rc.shutdown";
}

And in the jail:
Code: 
epair1b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=8<VLAN_MTU>
    ether 02:b3:7a:1a:c8:0b
    groups: epair
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
epair1b.23: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:b3:7a:1a:c8:0b
    inet 10.13.13.101 netmask 0xffffff00 broadcast 10.13.13.255
    groups: vlan
    vlan: 23 vlanpcp: 0 parent interface: epair1b
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

The routes from inside the jail:
Code: 
default            fw                 UGS    epair1b.
10.13.13.0/24      link#3             U      epair1b.

The /etc/resolv.conf file on the host:
Code: 
nameserver 10.11.11.11
nameserver 10.13.13.13
nameserver 10.20.20.20
nameserver 127.0.0.1

The jails only have the second line, and are set to use that "10.13.13.13" as the default gateway.

I think this is all correct, however im unable to ping the host:
Code: 
# ping 10.13.13.1
PING 10.13.13.1 (10.13.13.1): 56 data bytes
ping: sendto: Host is down
ping: sendto: Host is down
^C
--- 10.13.13.1 ping statistics ---
7 packets transmitted, 0 packets received, 100.0% packet loss
#

On the host the /etc/rc.conf file has these lines:
Code: 
defaultrouter="10.11.11.11"
gateway_enable="YES"
static_routes="server jails vm"
route_server="-net 10.11.11.0/24 10.11.11.11"
route_jails="-net 10.13.13.0/24 10.13.13.13"
route_vm="-net 10.20.20.0/24 10.20.20.20"

I was pretty sure this would work.
I have tried no IP address on the bridge, and tried to tag it but it wouldn't create the interface.

Thanks for your time.
 
Hello,

I would try a traceroute, it looks like a route is missing somewhere in the path from the computer sending the ping request. (host is down is a message from a gateway)
 
Traffic on your bridge0 is untagged. So don't VLAN tag the epair interfaces.
 
SirDice said:
Traffic on your bridge0 is untagged. So don't VLAN tag the epair interfaces.
Click to expand...
Wow. I had thought I had already tried that but looking back through the console buffer I tried it with the interfaces down. So removing the tags from the other bridge members fixed the issue. Thank you!
 
You must log in or register to reply here.
Back
Top