IPFW ipfw nat stateful redirect of a port

B

bmth

Hello everyone!

I have few network services running in jailed configuration on a server, and I use ipfw to protect the server against possible attacks, and to provide its local clients with access to internet.

The goal I want to achieve is redirection of some ports of jailed services to the server external interface. And I actually did it, but I don't really like the way it looks.

Here're the essential part of my ipfw script :

Code: 
# configure nat
$ipfw nat 1 config if $wanif redirect_port tcp $jail1ip:80 8080 reset

# translate addresses
$ipfw add nat 1 ip from any to any via $wanif

# check dynamic rules
$ipfw add check-state

# allow port redirects
$ipfw add allow tcp from any to $jail1ip 80 in via $wanif keep-state

# !!! here's the rule i don't like !!!
$ipfw add allow tcp from me 8080 to any out via $wanif keep-state
# ^^^ here's the rule i don't like ^^^

# deny fake established tcp packets
$ipfw add deny log tcp from any to any established

I did some testing when I was writing these rules, and it appeared that :
  1. When a packet comes from external network to port 8080 of wan interface, nat engine translates the port of that packet, so the source ip and port of the incoming packet are remained the same, and destination ip and port changes, and then ipfw continues processing the packet according to the ruleset after nat rule.
  2. The rule 'allow port redirects' passes the packet and creates a dynamic rule for future responses for the packet.
  3. Then something happens in jail, I just skip it.
  4. The outgoing response packet is allowed using dynamic rule created at point 2.
  5. Then nat engine translates source ip and port of outgoing packet to those specified in rule 'configure nat', and puts it back into processing with ipfw.
  6. Then the outgoing packet is processed with the rule I don't like =(
So, my question is "is there a way to produce dynamic rule for packet incoming to external interface port 8080, so the responses can be processed with this rule?"

uname: freebsd 10.3
the rest of the config i consider irrelevant
 
In accordance to the Handbook chapter 29.4.4 many of us define two NAT rules, thus separating inbound and outbound traffic in a stateful NAT'ing ipfw(8) setup. By this way you can avoid the rule which you don't like. The general template for such a ruleset is:
Code: 
# Definitions and Configurations
...

# Rules for everything within the LAN - interface with heaviest traffic shall come first.
...

# Anti-spoofing rule.
...

# NAT rule for incoming packets - IPv4 only, IPv6 ain't work with NAT.
/sbin/ipfw -q add 100 nat 1 ip4 from any to any in recv $WAN
/sbin/ipfw -q add 101 check-state

# Rules for outgoing traffic - allow everything that is not explicitely denied.
/sbin/ipfw -q add 1000 deny ip from not me to any 25,53 out xmit $WAN
/sbin/ipfw -q add 1010 deny ip from any to any 5353 out xmit $WAN

# Allow all other outgoing connections, i.e. skip processing to the outbound NAT rule #10000
/sbin/ipfw -q add 2000 skipto 10000 tcp from any to any out xmit $WAN setup keep-state
/sbin/ipfw -q add 2010 skipto 10000 udp from any to any out xmit $WAN keep-state

# Rules for incomming traffic - deny everything that is not explicitely allowed.
/sbin/ipfw -q add 5000 allow tcp from any to me 22,25,80,587,993,995 in recv $WAN setup keep-state
/sbin/ipfw -q add 5010 allow udp from any to me 500,4500 in recv $WAN keep-state

# Rules for allowing packets to services which are listening on a LAN interface behind the NAT
/sbin/ipfw -q add 6000 skipto 10000 tcp from any to any 8080 in recv $WAN setup keep-state

# Catch any other tcp/udp packet, but don't touch gre, esp, icmp, etc...
/sbin/ipfw -q add 9998 deny tcp from any to any via $WAN
/sbin/ipfw -q add 9999 deny udp from any to any via $WAN

# NAT rule for outgoing packets.
/sbin/ipfw -q add 10000 nat 1 ip4 from any to any out xmit $WAN

# Allow anything else - just in case ipfw is not configured as open firewall.
/sbin/ipfw -q add 65534 allow ip from any to any
According to ipfw(8), in-kernel NAT does not play well with TSO, see: man ipfw | grep -C1 TSO. Therefore, I add the -tso flag to all my ifconfig_xxx directives in file /etc/rc.conf where appropriate.

In this scenario, we don't immediately allow outbound traffic, but skip the processing to the outbound NAT rule.

Stateful NAT'ing requires some packages to be processed more than once by the firewall, for this reason I have net.inet.ip.fw.one_pass=0 in my file /etc/sysctl.conf.
 
obsigna said:
In accordance to the Handbook chapter 29.4.4 many of us define two NAT rules, thus separating inbound and outbound traffic in a stateful NAT'ing ipfw(8) setup. By this way you can avoid the rule which you don't like.
Click to expand...
But this does the same and even worse. If my "bad" rule allows outgoing traffic only from one specific port to any destination, then the handbook variant allows anything outgoing from nat. My idea was to allow outgoing traffic only to the destination where a connection request came from, like if it was an ordinary open port with keep-state option on the wan interface.
 
Are you referring to the following rule?
/sbin/ipfw -q add 6000 skipto 10000 tcp from any to any 8080 in recv $WAN setup keep-state

This one does allow only incoming traffic (... in recv ...) and generates dynamic rules for the corresponding outgoing traffic. For me this seems to be exactly the behaviour that you are looking for, or am I missing something?
 
obsigna said:
This one does allow only incoming traffic (... in recv ...) and generates dynamic rules for the corresponding outgoing traffic. For me this seems to be exactly the behaviour that you are looking for, or am I missing something?
Click to expand...
Yes, it looks like, thank you. I didn't test it yet, but I think that rule will do what I want.

By the way, I wanna share an idea of automatic enumeration of rules, since all the ipfw config script examples I've seen on the Internet have these "hardcoded" rule numbers.
Here's an example of the script
Code: 
#!/bin/sh

# == FIREWALL CONFIGURATION PARAMETERS == #

# -- ipfw command -- #
ipfw="ipfw -q"

# -- ipfw rule number and step -- #
n=0; s=10;

# -- wan interface -- #
wanif="eth0"

# -- lan subnet -- #
lan="172.16.0.0/24"

# == FIREWALL CONFIGURATION BLOCKS == #
firewall_reset()
{
  # -- flush firewall rules -- #
  $ipfw -f flush

  # -- filter packets returned from other subsystems -- #
  $ipfw disable one_pass
}

allow_loopback()
{
  # -- allow loopback -- #
  $ipfw add $((n=n+s)) allow all from any to any via lo0
}

nat_config()
{
  # -- nat config -- #
  $ipfw nat 1 config if $wanif reset
}

nat_in()
{
  # -- deny private networks -- #
  $ipfw add $((n=n+s)) deny ip from any to 169.254.0.0/16
  $ipfw add $((n=n+s)) deny ip from 169.254.0.0/16 to any
  $ipfw add $((n=n+s)) deny ip from any to 192.168.0.0/16
  $ipfw add $((n=n+s)) deny ip from 192.168.0.0/16 to any
  $ipfw add $((n=n+s)) deny ip from any to 172.16.0.0/12
  $ipfw add $((n=n+s)) deny ip from 172.16.0.0/12 to any
  $ipfw add $((n=n+s)) deny ip from any to 10.0.0.0/8
  $ipfw add $((n=n+s)) deny ip from 10.0.0.0/8 to any

  # -- network address translation -- #
  $ipfw add $((n=n+s)) nat 1 ip from any to any
}

nat_out()
{
  # -- network address translation -- #
  $ipfw add $((n=n+s)) nat 1 ip from any to any
}

allow_dhcp()
{
  # -- dhcp -- #
  $ipfw add $((n=n+s)) allow udp from any dhcpc,dhcps to any dhcpc,dhcps
}

allow_ssh()
{
  # -- ssh -- #
  $ipfw add $((n=n+s)) allow tcp from any to me ssh
  $ipfw add $((n=n+s)) allow tcp from me ssh to any
}

allow_lan()
{
  # -- allow lan -- #
  $ipfw add $((n=n+s)) allow all from $lan to $lan
}

allow_all()
{
  # -- allow all -- #
  $ipfw add $((n=n+s)) allow all from any to any
}

deny_all()
{
  # -- deny all -- #
  $ipfw add $((n=n+s)) deny all from any to any
}

# == FIREWALL CONFIGURATION == #

# -- flush all rules, configure firewall options -- #
firewall_reset

# -- nat configuration -- #
nat_config

# -- basic setup -- #
allow_loopback
allow_lan
allow_dhcp
allow_ssh

# -- nat -- #
skipto_wan_in=$((n=n+s))
skipto_wan_out=$((n=n+s))
deny_all

$ipfw add $skipto_wan_in skipto $((n+s)) ip from any to any in via $wanif
nat_in
allow_all

$ipfw add $skipto_wan_out skipto $((n+s)) ip from any to any out via $wanif
nat_out
allow_all

This example shows the way for automatic enumeration of firewall rules.
Command
skipto_wan_in=$((n=n+s))
reserves rule number that is used there later in
$ipfw add $skipto_wan_out skipto $((n+s)) ip from any to any out via $wanif
From my personal point of view this makes ipfw scripts quite readable and easier to maintain.
 
The ugliness of IPv4 NAT is driving me to move to IPv6 and, in the process, rewriting my ipfw rules, so I've had to think through this mess once again.

As you note, there is no "good" way to manage both the "in" and the "out" with stateful rules the same way you might for a directly connected host.

As others have pointed out, thinking about "in" separately from "out" is helpful, and explicitly writing rule paths for in and out helps me maintain my sanity.


Before going into how to solve the problem, it's helpful to know just what the problem is. Adding complexity to solve one problem can easily introduce others.

If the desire is to keep the service host from "talking" to any one other than the client, then one solution is to use a VNET/VIMAGE jail, put your firewall rules there, and crank up kern.securelevel in the jail to 3 once things are running.

Understanding the concern behind

allow src-ip ${outside_ip} src-port 8080 out xmit ${outside_if}

will help as well.

Is there a real threat that you see from that? If so, how could it occur?


You certainly can tighten it up a bit with requiring that the packet at least come in from your service host's interface

allow src-ip ${outside_ip} src-port 8080 out recv ${service_if} xmit ${outside_if}



If you want to take it a step even further, packet tagging may help.

If 1.2.3.4 is the remote client, 5.6.7.8 is your outside IP, and 10.0.0.10 is the server

For the incoming packets:
  • 1.2.3.4:nnn => 5.6.7.8:8080 at the interface
  • 1.2.3.4:nnn => 10.0.0.10:80 after NAT
  • 1.2.3.4:nnn => 10.0.0.10:80 "in" from the remote client (after NAT)
  • 1.2.3.4:nnn => 10.0.0.10:80 "out" to the service host
For the outgoing packets
  • 10.0.0.10:80 => 1.2.3.4:nnn "in" from the service host
  • 5.6.7.8:8080 => 1.2.3.4:nnn after NAT
  • 5.6.7.8:8080 => 1.2.3.4:nnn "out" to the remote client
The setup packet on the way in can be used to create dynamic rule 1:
1.2.3.4:nnn <=> 10.0.0.10:80

It also uses the same rule on the way out to the service host, as do return packets as they come in the firewall host.

Now when the return packet comes in, tag it so you know not only did it come in the right interface, but also from the right host and port. Assuming you've split the in and out paths, something along the lines of

count tag 666 src-ip ${service_ip} src-port 80 in recv ${service_if}
check-state // for "in" path
deny log src-ip ${service_ip} src-port 80 in

Since you shouldn't get to that "deny" rule if it matched the dynamic rule from the inbound setup, you've now got a packet that is associated with the setup flow that has the tag 666, which will survive the NAT operation. You can then further "improve" that bothersome rule to be something like:

allow src-ip ${outside_ip} src-port 8080 out recv ${service_if} xmit ${outside_if} tagged 666

You certainly could use a similar tagging approach on the way in from the remote client as well, if you believe that there is a potential for a hole there.

There's a balancing point you need to chose for yourself, though, between complexity of the rules and readability and understandability. The fancier you get, the harder it is to think through all the paths clearly and correctly.
 
You must log in or register to reply here.
Back
Top