Vnet jail with IPFW NAT outbound traffic no longer works after upgrade from 12.2-RELEASE to 13.0-RELEASE

IPFW has a lot of ridiculous quirks which should not be in a firewall in 2021. It cannot match an xmit interface on an incoming packet. It cannot correctly keep state on UDP and ICMP packets (it completely opens both ends). Yes those are stateless protocols, but other firewalls handle these things properly. I had a look at pf and was glad I did. After 15 minutes I understood pf more than I did in 20 years of IPFW.
 
I agree with covacat. You need to simplify your firewall configuration.
A couple of thoughts: I don't know why, but when I initially create a jail its firewall denies everything, even though the ipfw service is down. You need to open the firewall explicitly (or configure the jail to use a firewall and open up the relevant ports).
You figured that out.

Second - why do you need NAT at all? If your jail's interface is bridged to your physical one - the jail participates directly on your host's LAN, routing through the host is unnecessary.
I suggest that you remove anything related to NAT from the host's IPFW configuration and do what iocage recommends for its jails: https://iocage.readthedocs.io/en/latest/networking.html#vimage-vnet
You can disable bridge member filtering in your firewall, so you would not need the "skip" rules for bridge0 and epair0 (103-106 and 114-117):
Bash:
# Add these tunables to /etc/sysctl.conf:
net.inet.ip.forwarding=1       # Enable IP forwarding between interfaces
net.link.bridge.pfil_onlyip=0  # Only pass IP packets when pfil is enabled
net.link.bridge.pfil_bridge=0  # Packet filter on the bridge interface
net.link.bridge.pfil_member=0  # Packet filter on the member interface

Keep in mind that in this scenario the jail is connected to your LAN directly, so treat it as any other physical machine - setup a firewall to block incoming traffic, open port 80.

If you really need to NAT, you would not need to bridge the jail's epair to your em0. You could route through the host from and to the jail's epair directly.
Then, try to create a simpler NAT configuration. I based my own NAT on this post: https://lists.freebsd.org/pipermail/freebsd-ipfw/2015-August/005928.html
Delete the rules you don't need.

You could think of routing (with or without NAT) and bridging as alternatives to each other, bridging connects multiple interfaces on Layer 2 (Ethernet) and they see each other's packets automatically. Routing is an explicit forwarding of the packets from one interface to the other by the kernel. You could do both but it is usually unnecessary and complicates your configuration.

Also, all deny rules should also be "log"-ging, so you know what's happening. Especially when you debug.
Go ahead and enable "log" for the NAT rules also. I have found it very helpful (when debugging only).

And keep in mind that whatever packets get denied, everything is logged in your host's /var/log/security, even the jails' firewalls log there (because they are using the host's running kernel).
Remember to put different numbers for the jail's firewall rules and the host's. Then when you look at your log you can tell if the jail denied the packet or the host.
 
Back
Top