Jails different VLANs can still communicate?

Farhan Khan

Farhan Khan

I first created two vlans, as follows:


ifconfig vlan1 create vlan 1 vlandev em0
ifconfig vlan1 10.1.0.1/24
ifconfig vlan2 create vlan 2 vlandev em0
ifconfig vlan2 10.2.0.1/24


I then created two jails as follows:
Code: 
vlan1 {
    path = /usr/jail/vlan1;
    allow.mount;
    mount.devfs;
    ip4.addr = 10.1.0.2/24;
    interface = vlan1;
    allow.raw_sockets;
    allow.sysvipc;
    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
}

vlan2 {
    path = /usr/jail/vlan2;
    allow.mount;
    mount.devfs;
    ip4.addr = 10.2.0.2/24;
    interface = vlan2;
    allow.raw_sockets;
    allow.sysvipc;
    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
}

When I enter either jail, I expected it not to be able to ping or connect to the other jail, but it can. For example, why can vlan1 (10.1.0.0/24), talk to vlan2 (10.2.0.0/24). Is it because they both share em0? If so, doesn't that defeat the purpose of creating a VLAN to begin with? Please advise.

Thank you!
 
For testing purposes, I removed the raw_sockets and sysvipc options. Made no difference.
traceroute(8) shows 1 hop, so the base does not appear to be functioning as a router.
 
actually, having a Vlan does not mean there is no route between them. does either of the jails have a default route in their routing table? if they do (and it is the host) they might connect to the host, then the host routes the packets to the other jail.
 
Farhan Khan said:
For example, why can vlan1 (10.1.0.0/24), talk to vlan2 (10.2.0.0/24). Is it because they both share em0?
Click to expand...
More accurately, they both share the host. It's the host that does the actual routing, not the jails. And for the host these are two directly connected networks and so it'll happily route between them.
 
Just double-checked, there was no route. The IP was a /24, so it would not even know how to reach out.
The only routing rule in the first jail is this:
Code: 
Destination        Gateway            Flags     Netif Expire
10.1.0.2           link#4             UHS         lo0
If VLAN traffic can trivially cross from one VLAN to another, that would defeat the purpose of VLANing to begin with. The entire purpose is segmentation. So if traffic can cross over because they share the same host, that seems like a serious security hole in jails. what am I missing?
 
I still have not figured this out. The default routing rules creates this:

Code: 
# netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
0.0.0.0/8          link#1             U           em0
default            192.168.1.1        UGS         em1
10.1.0.0/24        link#4             U         vlan1
10.1.0.1           link#4             UHS         lo0
10.2.0.0/24        link#5             U         vlan2
10.2.0.1           link#5             UHS         lo0
127.0.0.1          link#3             UH          lo0
192.168.1.0/24     link#2             U           em1
192.168.1.13       link#2             UHS         lo0
Notice 10.1.0.0/24 via vlan1 and 10.2.0.0/24 via vlan2. I am not able to delete these two routes.

So, when a Jail on VLAN1 pings outward, it is routed by the base OS to VLAN2. This still defeats the purpose of having VLANs.
The only way to prevent that is using pf, but then the only thing separating traffic is the firewall.

The FreeBSD documentation does not seem to cover this. Any assistance would be greatly appreciated.
 
SirDice said:
More accurately, they both share the host. It's the host that does the actual routing, not the jails. And for the host these are two directly connected networks and so it'll happily route between them.
Click to expand...
The above message was to you. What then is the proper way to do this?
 
Hi all, just an update for anyone following this thread: I just attempted to utilize epair, I am still getting Layer 3 routing. Still have not found a solution to this. Very strange.
 
Note that jails provide process separation, not network separation. If you really want to separate them both on network and process using bhyve(8) will be a better solution.
 
here's a basic view of what happens when you ping from one jail to the other.
jail1 on vlan1 who has the IP 10.1.0.2/24 and the gateway is 10.1.0.1 (which is the host) ping the jail2 on vlan2 (ICMP from 10.1.0.2 to 10.2.0.2). now, Jail1 does not know where is 10.2.0.2, so it sends the packets to the default gateway, the host. the hosts receives the packets and sees that the destination is 10.2.0.2, and guess what? it does know who is 10.2.0.2! so the packet is sent to jail2.

this is very normal act.

the basic way to "fix" this is to use a firewall. basically, using IPFW you can do.
ipfw add 1000 drop from 10.1.0.1/24 to 10.2.0.1/24
and from the other network as well
ipfw add 1001 drop from 10.2.0.1/24 to 10.1.0.1/24
 
First off, thank you for all your comments.

As SirDice said, I think this is just my fundamental misunderstanding of how FreeBSD's network stack operates. I got this working if I use FreeBSD as a switch (which was the goal to begin with). I have not tested it with Bhyve, but since Bhyve creates a completely separate network stack, it should work as I expected.

I could have also given a jail a VIMAGE stack.
 
You must log in or register to reply here.
Back
Top