IPFW IPFW and natd causing huge packet load

Londo

Londo

Hello,
First I wanted to give a shout out to everyone who has posted other problems and responses over the years - the knowledge base has helped tremendously.

I'm here today to ask for pointers regarding IPFW and natd. Some background: I have a very old server that runs a business and is due for replacement, but needs to operate for a few more months. It runs an obsolete version of FreeBSD - 8.4 - and has, until now, been connected to the Internet via DSL using PPPoE. The server serves up a few network utilities like running a website and accepting connections for SSH and email, as well as using NAT to allow desktop computers to reach the Internet. The services this server provides will take time to be configured and migrated elsewhere.

With that said, we are trying to move the system from DSL to a cable connection. With DSL, PPP has always taken care of NAT functions and the system has run without a problem. I have updated the rc.conf file to reflect the new network settings and have noted the following:
  • When an external connection is made directly to the server, large amounts of traffic begin to bounce between two IPFW rules
    • divert nat ip4 from any to any via bce0
    • allow tcp from any to any established
  • When the divert rule is moved after the 'allow tcp from any to any established' rule, external connections return to normal but office computers on the private network can no longer share the external connection via natd
I further split up the divert rule into two parts:
  • divert nat ip from any to any in via bce0
  • divert nat ip from any to any out via bce0
From this I found that 'divert nat ip from any to any out via bce0' was logging a huge number of packets, along with 'allow tcp from any to any established'

I tried moving these two nat rules in relationship to themselves and 'allow tcp from any to any established' and found two operating modes:
  • The server shares its connection via NAT fine - but external request result in looping packets that result in high CPU usage and lost connections
  • The server does not share its connection via nat - but external requests run perfectly and there is no detrimental CPU activity

I am unable to understand what is going wrong, since the rc.firewall setting being used is a FreeBSD built-in option (firewall_type="simple") and works fine on other systems I have.

My kernconf options:

Code: 
options         IPFIREWALL
options         IPDIVERT
options         IPFIREWALL_VERBOSE
options         IPFIREWALL_VERBOSE_LIMIT=10
options         IPFIREWALL_DEFAULT_TO_ACCEPT

My sample IPFW output:

Code: 
00100 1564814   87629584 allow ip from any to any via lo0
00200       0          0 deny ip from any to 127.0.0.0/8
00300       0          0 deny ip from 127.0.0.0/8 to any
00400       0          0 deny ip from any to ::1
00500       0          0 deny ip from ::1 to any
00600       0          0 allow ipv6-icmp from :: to ff02::/16
00700       0          0 allow ipv6-icmp from fe80::/10 to fe80::/10
00800       0          0 allow ipv6-icmp from fe80::/10 to ff02::/16
00900       0          0 allow ipv6-icmp from any to any ip6 icmp6types 1
01000       0          0 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136
01100       0          0 deny ip from 192.168.0.1 to any in via bce0
01200       0          0 deny ip from (InternetInterface) to any in via em0
01300       0          0 deny ip from any to 10.0.0.0/8 via bce0
01400       0          0 deny ip from any to 172.16.0.0/12 via bce0
01500       0          0 deny ip from any to 192.168.0.0/16 via bce0
01600       0          0 deny ip from any to 0.0.0.0/8 via bce0
01700       0          0 deny ip from any to 169.254.0.0/16 via bce0
01800       0          0 deny ip from any to 192.0.2.0/24 via bce0
01900       0          0 deny ip from any to 224.0.0.0/4 via bce0
02000       0          0 deny ip from any to 240.0.0.0/4 via bce0
02100  782749 2304121703 divert 8668 ip from any to any out via bce0
02150     238      31015 divert 8668 ip from any to any in via bce0
02200       0          0 deny ip from 10.0.0.0/8 to any via bce0
02300       0          0 deny ip from 172.16.0.0/12 to any via bce0
02400       0          0 deny ip from 192.168.0.0/16 to any via bce0
02500       0          0 deny ip from 0.0.0.0/8 to any via bce0
02600       0          0 deny ip from 169.254.0.0/16 to any via bce0
02700       0          0 deny ip from 192.0.2.0/24 to any via bce0
02800       0          0 deny ip from 224.0.0.0/4 to any via bce0
02900       0          0 deny ip from 240.0.0.0/4 to any via bce0
03000  783166 2304189472 allow tcp from any to any established
03100       0          0 allow ip from any to any frag
03200       0          0 allow tcp from any to me dst-port 25 setup
03300       0          0 allow tcp from any to me dst-port 26 setup
03400       0          0 allow tcp from any to me dst-port 53 setup
03500       0          0 allow udp from any to me dst-port 53
03600       0          0 allow udp from me 53 to any
03700       0          0 allow tcp from any to me dst-port 80 setup
03800       2         80 deny log logamount 10 ip4 from any to any in via bce0 setup proto tcp
03900       4        208 allow tcp from any to any setup
04000      12        958 allow udp from me to any dst-port 53 keep-state
04100       0          0 allow udp from me to any dst-port 123 keep-state
65535      27       2240 allow ip from any to any

My sample rc.conf file:
Code: 
# -- Network configuration
defaultrouter=“(RouterAddress)”
gateway_enable="YES"
hostname=“mainserver”
ifconfig_bce0="inet (InternetAddress)  netmask 255.255.255.252"
ifconfig_em0="inet 192.168.0.1  netmask 255.255.255.0"
named_enable="YES"
#named_auto_forward="YES"

# -- Server Setup
inetd_enable="YES"
sshd_enable="YES"
mysql_enable="YES"
apache22_enable="YES"

# -- Sendmail Setup
sendmail_enable="YES"
saslauthd_enable="YES"

# -- Enable NFS Server and Client
rpcbind_enable="YES"
nfs_server_enable="YES"
nfs_client_enable="YES"

# -- Misc. services and options
accounting_enable="YES"
saver="daemon"
ntpdate_enable="YES"
ntpdate_flags="-b time.nist.gov"

# -- Hardware Support
usbd_enable="YES"

# -- Firewall configuration
firewall_enable="YES"
firewall_type="simple"
firewall_simple_iif="em0"
firewall_simple_inet="192.168.0.1"
firewall_simple_oif="bce0"
firewall_simple_onet=“(InternetAddress)”
natd_enable="YES"
natd_interface="bce0"
natd_flags=""

When I split the divert rule into two halves, packet accounting looks like this:
Code: 
00100 16258588   910548228 allow ip from any to any via lo0
00200        0           0 deny ip from any to 127.0.0.0/8
00300        0           0 deny ip from 127.0.0.0/8 to any
00400        0           0 deny ip from any to ::1
00500        0           0 deny ip from ::1 to any
00600        0           0 allow ipv6-icmp from :: to ff02::/16
00700        0           0 allow ipv6-icmp from fe80::/10 to fe80::/10
00800        0           0 allow ipv6-icmp from fe80::/10 to ff02::/16
00900        0           0 allow ipv6-icmp from any to any ip6 icmp6types 1
01000        0           0 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136
01100        0           0 deny ip from 192.168.0.1 to any in via bce0
01200        0           0 deny ip from (InternetInterface) to any in via em0
01300        0           0 deny ip from any to 10.0.0.0/8 via bce0
01400        0           0 deny ip from any to 172.16.0.0/12 via bce0
01500        0           0 deny ip from any to 192.168.0.0/16 via bce0
01600        0           0 deny ip from any to 0.0.0.0/8 via bce0
01700        0           0 deny ip from any to 169.254.0.0/16 via bce0
01800        0           0 deny ip from any to 192.0.2.0/24 via bce0
01900       14         504 deny ip from any to 224.0.0.0/4 via bce0
02000        0           0 deny ip from any to 240.0.0.0/4 via bce0
02100  8137944 24145826519 divert 8668 ip from any to any out via bce0
02150     7050     2208785 divert 8668 ip from any to any in via bce0
02200        0           0 deny ip from 10.0.0.0/8 to any via bce0
02300        0           0 deny ip from 172.16.0.0/12 to any via bce0
02400        0           0 deny ip from 192.168.0.0/16 to any via bce0
02500        0           0 deny ip from 0.0.0.0/8 to any via bce0
02600        0           0 deny ip from 169.254.0.0/16 to any via bce0
02700        0           0 deny ip from 192.0.2.0/24 to any via bce0
02800        0           0 deny ip from 224.0.0.0/4 to any via bce0
02900        0           0 deny ip from 240.0.0.0/4 to any via bce0
03000  8155320 24151527236 allow tcp from any to any established
03100        0           0 allow ip from any to any frag
03200        1          60 allow tcp from any to me dst-port 25 setup
03300        0           0 allow tcp from any to me dst-port 26 setup
03400        0           0 allow tcp from any to me dst-port 53 setup
03500        0           0 allow udp from any to me dst-port 53
03600        0           0 allow udp from me 53 to any
03700        9         524 allow tcp from any to me dst-port 80 setup
03800       90        3744 deny log logamount 10 ip4 from any to any in via bce0 setup proto tcp
03900      421       21940 allow tcp from any to any setup
04000      656       54335 allow udp from me to any dst-port 53 keep-state
04100        3         228 allow udp from me to any dst-port 123 keep-state
65535     1785      430704 allow ip from any to any

Perhaps someone can tell me what I'm doing wrong and how I would prevent packet loops like what appears to be going on here. My apologies in advance if I forgot to post a critical piece of information - I'll try to correct my post right away if it is found deficient.

A big Thank You in advance for all replies!
 
Success! Hours of searching later, I found this forum post.

In short, the default firewall rules do not route traffic around natd even if it is destined for the server itself. I merged the added entries, found in this example, with my firewall rules for a successful outcome:

Code: 
#!/bin/sh

cmd="ipfw -q add"
pif=bce0
lif=bce1

ipfw -q -f flush

#############################################################################
# Exempt local traffic from NAT

$cmd 00010 allow all from any to any via lo0
$cmd 00020 allow all from any to any via $lif

#############################################################################
# Bad: This would prevent return packets from being diverted through NAT
# to local machines. Many of them in reality don't need to be altered,
# but we need NATD to determine which ones do.
# $cmd 00046 allow all from not me to me via $pif

#############################################################################
# Exempt traffic originating on this host from NAT
$cmd 00047 allow all from me to not me via $pif

#############################################################################
# Divert outgoing packets from local machines and all incoming packets
# from the public interface through NAT

# This would be good enough, assuming all the rules above exist
# $cmd 00050 divert 8668 ip4 from any to any via $pif

# These rules limit the use of natd regardless of preceding rules
$cmd 00050 divert 8668 all from 192.168.0.0/24 to any via $pif
$cmd 00050 divert 8668 all from any to me via $pif

The most important entries for me were:

Code: 
allow ip from any to any via [LAN]
allow ip from me to not me via [WAN]
divert 8668 ip from 192.168.0.0/24 to any via [WAN]
divert 8668 ip from any to me via [WAN]

In my case, the internal network is 192.168.0.0/24 - change for your own application.

Once again, thanks to this forum for help with issues like these!
 
You must log in or register to reply here.
Back
Top