I know I've started a few threads here about odd networking experiments that I am running in VNET jails. Still haven't figured anything out about those when using epair(4) interfaces. So I thought I'd try a different strategy using VLAN interfaces instead. And this time, include a (janky) diagram of what I think I want this to look like.
Basically, I want two jails, one for my wireless network and another for my wired network. The jail host has six wired interfaces, em0 through em5, and one wireless interface, wlan0 (via wlandev ath0). Interface em0 is connected directly to my cable modem, which already operates in a bridged mode and itself does no routing of any kind. The jail host will manage em0 itself, pulling a WAN IP from my ISP via dhclient at startup, before any of the jails are started. Likewise, it will also partially configure wlan0 before the jails are started, and eventually hand off the physical wlan0 interface to the wireless jail.
The wifi network jail will run hostapd, isc-dhcpd, and unbound to service wireless stations on subnet 10.2.0.0/24, with the physical wlan0 interface assigned gateway IP 10.2.0.1/32. Wireless stations seeking to reach the internet will go through wlan0, an IPFW NAT firewall will rewrite their private IP to the ISP public one, and then my hope is that traffic will transparently pass through the em0 VLAN interface (em0.2, vlanid 2) and reach the WAN via the jail host (I'll worry about that VLAN tag later). I initially tried this idea using an epair(4) interface, but could not find a way to effectively make epairXb in the wireless jail into a transparent layer 2 tunnel back to the jail host.
The wired network is similar, though a tad more complex (I think?). The jail host itself is connected to a Netgear smart switch via em1 with IP 10.1.0.2. Another physical interface, em2, is connected directly to the switch with no IP address assigned. The jail host binds em2 to bridge0 at startup.
The wired router jail will service wired systems on subnet 10.1.0.0/24. It will receive an epair(4) interface and an em0 VLAN device (em0.1, vlanid 1) from the jail host on startup. Its epairXb interface will be assigned gateway IP 10.1.0.1/32, with the corresponding host-side epairXa interface bound to bridge0 by the jail host when the jail starts. Wired systems seeking to reach the internet will send packets to 10.1.0.1 via em2 --> bridge0 --> epairXa --> (router jail) --> em0.1 --> WAN. The jail host itself will, when attempting to reach the internet, send packets out em1 back to itself through em2 and then out the previously-defined path.
The kink here is I need to find a way to make the em0 VLANs transparent in some manner, meaning, I need them to route packets without any layer three network addresses assigned. Similar to the wireless jail, I previously tried this using epair(4) interfaces, but I could not find a way to make them act as layer two tunnels back to the jail host. I am trying VLANs now, because some reading seems to suggest this is more possible, that children VLAN devices off of a parent device can inherit the parent's assigned IP address, adding their respective VLAN tag on top (just after the Ethernet II layer). That said, I really don't know if VLANs are meant to work this way with FreeBSD's standard networking tools. I basically want em0, with the WAN IP from my ISP to be "cloned" (but not in the
So far, this isn't working in small-scale experiments. Both em0.1 and em0.2, when added to the jails, don't have an address assigned, even if the jail host has pulled an IP from the ISP and assigned it to the parent em0 interface. The issue seems to be that FreeBSD doesn't know how or where to route packets if an interface doesn't have an IP address assigned. I tried using
Is anything that I am describing possible using FreeBSD's standard networking tools? Do I need to incorporate tap(4)/tun(4)? Or is this where I will need to investigate using netgraph(4) or vxlan(4)? I'll also add that I am not employing any firewall scripts in either jail yet, I prefer to use IPFW over PF, and I've mostly done small-scale experiments using one of the other unused emX interfaces on the jail host, so that I don't interfere with my currently-working router setup.
Code:
10.1.0.0/24
___________
| wired |_
| network |_|-----------------------------+-------+
|___________| | |
| |
___________ | |
| bridge0 | | |
|___________| | |
| \ | |
| \ | |
| \-em2-| |
__________ _ | \ | |
| router |_|-epair0-\ \ | |
| jail |_ \ \ | |-em1 (10.1.0.2)
|__________|_|-em0.1----\ \_|_ _|_
10.1.0.1 \ |___|___|___| __________
(epair0b) \ | jail |_ _| ISP |
|-| host |_|-em0 --> |_| Modem |
/ |___________| |__________|
__________ /
_| wireless |_ /
10.2.0.1-wlan0-|_| jail |_|-em0.2---/
| |__________|
|
|
_____|_____
| wifi |
| network |
|___________|
10.2.0.0/24The wifi network jail will run hostapd, isc-dhcpd, and unbound to service wireless stations on subnet 10.2.0.0/24, with the physical wlan0 interface assigned gateway IP 10.2.0.1/32. Wireless stations seeking to reach the internet will go through wlan0, an IPFW NAT firewall will rewrite their private IP to the ISP public one, and then my hope is that traffic will transparently pass through the em0 VLAN interface (em0.2, vlanid 2) and reach the WAN via the jail host (I'll worry about that VLAN tag later). I initially tried this idea using an epair(4) interface, but could not find a way to effectively make epairXb in the wireless jail into a transparent layer 2 tunnel back to the jail host.
The wired network is similar, though a tad more complex (I think?). The jail host itself is connected to a Netgear smart switch via em1 with IP 10.1.0.2. Another physical interface, em2, is connected directly to the switch with no IP address assigned. The jail host binds em2 to bridge0 at startup.
The wired router jail will service wired systems on subnet 10.1.0.0/24. It will receive an epair(4) interface and an em0 VLAN device (em0.1, vlanid 1) from the jail host on startup. Its epairXb interface will be assigned gateway IP 10.1.0.1/32, with the corresponding host-side epairXa interface bound to bridge0 by the jail host when the jail starts. Wired systems seeking to reach the internet will send packets to 10.1.0.1 via em2 --> bridge0 --> epairXa --> (router jail) --> em0.1 --> WAN. The jail host itself will, when attempting to reach the internet, send packets out em1 back to itself through em2 and then out the previously-defined path.
The kink here is I need to find a way to make the em0 VLANs transparent in some manner, meaning, I need them to route packets without any layer three network addresses assigned. Similar to the wireless jail, I previously tried this using epair(4) interfaces, but I could not find a way to make them act as layer two tunnels back to the jail host. I am trying VLANs now, because some reading seems to suggest this is more possible, that children VLAN devices off of a parent device can inherit the parent's assigned IP address, adding their respective VLAN tag on top (just after the Ethernet II layer). That said, I really don't know if VLANs are meant to work this way with FreeBSD's standard networking tools. I basically want em0, with the WAN IP from my ISP to be "cloned" (but not in the
cloned_interfaces sense) and copies given to each of the two described jails. In other words, both jails should be able to see their respective em0.X interfaces as gateways to the Internet, with the jail host actually managing the parent em0 device's IP address.So far, this isn't working in small-scale experiments. Both em0.1 and em0.2, when added to the jails, don't have an address assigned, even if the jail host has pulled an IP from the ISP and assigned it to the parent em0 interface. The issue seems to be that FreeBSD doesn't know how or where to route packets if an interface doesn't have an IP address assigned. I tried using
route add default -iface <vlan interface>, but that doesn't work. Trying to ping systems on the wired network or the Internet doesn't work, and ping just says "network unreachable" or "sendto: host is down".Is anything that I am describing possible using FreeBSD's standard networking tools? Do I need to incorporate tap(4)/tun(4)? Or is this where I will need to investigate using netgraph(4) or vxlan(4)? I'll also add that I am not employing any firewall scripts in either jail yet, I prefer to use IPFW over PF, and I've mostly done small-scale experiments using one of the other unused emX interfaces on the jail host, so that I don't interfere with my currently-working router setup.