IPFW Forward all traffic arriving on a specific IP through VPN

O

oliver@

Developer
Hi,

I have the following setup:

- Server with 1 interface having 5 IPs assigned, running OpenVPN-Server
- Client with 1 interface behind a router, running OpenVPN-Client

The OpenVPN-Connection works fine. I now want to route all traffic arriving on one of the 5 IPs of the server through the OpenVPN Tunnel to my Client.

The server and the clients run both ipfw(8).

I hesitate to "play around" as I fear to make the server unreachable which would involve a one hour car drive to fix this. That's why I want to discuss this first.

I guess I have to
a) Run natd(8) on the server like natd -a <one-of-the-5-IPs>
b) ipfw add divert natd from any to <one-of-the-5-IPs> in via <iface>
c) ipfw add divert natd from <one-of-the-5-IPs> to any out via <iface>
I fear that I break something because my current ipfw(8) setup uses keep-state which makes it a bit tricky with natd(8).

Right now I have

Code: 
00100 count ip from any to any in via re0
00101 count ip from any to any out via re0
00102 count ip from any to <ip1> in via re0
00103 count ip from <ip1> to any out via re0
00104 count ip from any to <ip2> in via re0
00105 count ip from <ip2> to any out via re0
00106 count ip from any to <ip3> in via re0
00107 count ip from <ip3> to any out via re0
00108 count ip from any to <ip4> in via re0
00109 count ip from <ip4> to any out via re0
00110 count ip from any to <ip5> in via re0
00111 count ip from <ip5> to any out via re0
00200 deny log ip from me to any in via re0
00201 deny log tcp from any to any in via re0 tcpflags syn,fin
00202 deny ip from any to 0.0.0.0/8 via re0
00203 deny ip from any to 10.0.0.0/8 via re0
00204 deny ip from any to 127.0.0.0/8 via re0
00205 deny ip from 127.0.0.0/8 to any via re0
00206 deny ip from any to 169.254.0.0/16 via re0
00207 deny ip from any to 172.16.0.0/12 via re0
00208 deny ip from any to 192.0.2.0/24 via re0
00209 deny ip from any to 192.168.0.0/16 via re0
00210 deny ip from any to 224.0.0.0/4 via re0
00211 deny ip from any to 240.0.0.0/4 via re0
00212 allow ip from any to any via lo0
00300 allow ip from any to any via tun0
10000 deny ip from table(1) to me via re0
20300 check-state
20400 deny tcp from any to any in established
20500 allow tcp from me to any out via re0 setup keep-state
20501 allow udp from me to any out via re0 keep-state
20502 allow ip from me to any out via re0
20600 allow tcp from any to <ip2> dst-port 22 in via re0 setup keep-state
20602 allow tcp from any to <ip2> dst-port 25 in via re0 setup keep-state
20604 allow tcp from any to <ip1> dst-port 53 in via re0 setup keep-state
20606 allow tcp from any to <ip2> dst-port 80 in via re0 setup keep-state
20610 allow tcp from any to <ip1> dst-port 443 in via re0 setup keep-state
20612 allow tcp from any to <ip2> dst-port 11965 in via re0 setup keep-state
20700 allow udp from any to <ip1> dst-port 53 in via re0 keep-state
20702 allow udp from any to any dst-port 1194 in via re0 keep-state
20703 allow udp from any to <ip5> dst-port 9987 in via re0 keep-state
20800 allow icmp from any to me in via re0 icmptypes 0,3,8,11
20801 allow icmp from me to any out via re0 icmptypes 4,8,12
21000 deny ip from any to any dst-port 135 in via re0
21002 deny ip from any to any dst-port 137 in via re0
21004 deny ip from any to any dst-port 138 in via re0
21006 deny ip from any to any dst-port 139 in via re0
21008 deny ip from any to any dst-port 445 in via re0
21010 deny udp from any to any dst-port 520 in via re0
21012 deny udp from any to any dst-port 694 in via re0
22000 allow ip from any to <ip1> in via re0 frag
60000 deny log ip from any to any
65535 deny ip from any to any

How would you set this up? Goal: access <ip4> of the server from somewhere and the access is made to the client (routed through the VPN)
 
OK, something does not work.

I have a working natd(8) on the Client side. When I telnet 10.3.0.10 80 on the server, I'm arriving on the client (10.3.0.10) and being NATed to 10.0.0.13:80. This works perfectly fine.

Then I tried to configure the server so all traffic arriving on port 80 on it's external IP is NATed to 10.3.0.10 (where it is NATed again). What I did is:

1. Started natd(8):
natd -a <external-ip> -redirect_port tcp 10.3.0.10:80 80 -v

2. Adjusted ipfw(8):
Code: 
15000  94  4752 divert 8668 ip from any to <external-ip> in via re0
15001  75  3800 allow tcp from any to 10.3.0.10 dst-port 80 in via re0
15003  0  0 divert 8668 ip from 10.3.0.10 to any out via re0
When I now access <external-ip> in a Browser, my verbose server natd(8) outputs
Code: 
In  {default}[TCP]  [TCP] <browser-ip>:50002 -> <external-ip>:80 aliased to
  [TCP] <browser-ip>:50002 -> 10.3.0.10:80
tcpdump(1) on my OpenVPN client (10.3.0.10) shows an incoming packet so the routing / NAT on my server seems correct as the traffic from <external-ip> is rerouted to 10.3.0.10
Code: 
[NOPARSE]23:02:32.877690 IP <browser-ip>.50002 > 10.3.0.10.http: Flags [S], seq 2885211578, win 8192, options [mss 1350,nop,wscale 2,nop,nop,sackOK], length 0[/NOPARSE]
But no answer is captured by tcpdump(1). Also no log entry in /var/log/security....

Any hints?
 
I think the tricky thing here is by the time the packet arrives at the client the source address would still be the browser's IP. Depending on how things are set up, the reply would likely go out a default gateway on that OpenVPN client over the Internet rather than back over the tunnel.

Some thoughts:
1. Are you familiar with the iroute directive in OpenVPN? You can use that to tell the OpenVPN server that the 10.0.0.0/24 network you mentioned is behind that OpenVPN client. With that you can save needing to NAT on the client.
2. Do you think the in kernel NAT would be a better fit? See the bottom of ipfw(8). With the requirement you mentioned for getting all traffic to redirect and not conflicting with the keep state stuff, this might be an option to toy around with. Something like this perhaps based off the man page.
Code: 
ipfw nat 1 config ip <external_ip> reset same_ports redirect_addr 10.0.0.13 <external_ip>
ipfw add nat 1 log all from any to any via <external_interface>
3. An option if you don't really need all traffic redirected is to look at a reverse proxy like net/haproxy. That does get around traffic using a default gateway and going a different way than you intended. I do have some examples for TCP proxy config if you would like to look at that route.

On the tcpdump(1) you ran, did you check to see if reply traffic left out the physical interface rather than the tunnel?
 
Hi,

First, let me thank you for your posting.

Indeed, the HTTP traffic arriving on the clients tun0 is answered via its defaultrouter through igb3. Of course it is this way - I wonder why I missed this.
Is there anything which can be done here to force the system to answer it back via tun0? Now come the crazy part... what about a second NAT on the OpenVPN server?

Client (10.0.0.0/24) -> 10.3.0.10 ===[OpenVPN]===> 10.3.0.1 [NATd] Server [NATd] <external-ip>

This way the incoming traffic on <external-ip> will be redirected to 10.0.0.13 and sent via tun0 to the Client. But because of the natd running on tun0, the sender is modified to 10.3.0.1 instead of <browser-ip> and because of that, the Client should respond back via the OpenVPN tunnel - correct? ;)

Regarding iroute - nice point, I switched the client from natd to route/iroute and it works at least the same way ;)

Do you see any more generic way to fix this? It would be nice if you could share some net/haproxy configs but if there is a more generic way I would prefer it...

Edit: with a second NAT on the OpenVPN server handling tun0, the client indeed sends back the HTTP answer to the OpenVPN-IP. But then it is retranslated to the original-browser-ip by the tun0-natd but it is not accepted by the browser.

Code: 
In  {default}[TCP]  [TCP] <browser-ip>:18691 -> <external-ip>:80 aliased to
  [TCP] <browser-ip>:18691 -> 10.0.0.13:80
Out {vpn}[TCP]  [TCP] <browser-ip>:18691 -> 10.0.0.13:80 aliased to
  [TCP] 10.3.0.1:18691 -> 10.0.0.13:80
In  {vpn}[TCP]  [TCP] 10.0.0.13:80 -> 10.3.0.1:18691 aliased to
  [TCP] 10.0.0.13:80 -> <browser-ip>:18691

I guess it would be accepted if it would run through the "default" nat to and so changed back to <external-ip> -> <browser-ip>
 
Without any natd but route/iroute via OpenVPN, and net/haproxy I can now access the web pages on the client side via the external ip of the server just fine.
But anyway... if there is a more generic solution around I would be still interested.
 
1. The routing solution. The net.fibs=2 in /boot/loader.conf would be the easy part. Conceivably, you can run a script after OpenVPN connects that uses the setfib(1) command to add a default route on the second FIB to the VPN remote destination. You can then potentially have the 10.0.0.13 client you mentioned attached to an interface that is tagged with fib 1 in /etc/rc.conf

I haven't something like this, but in trying to use FIBs to get a machine that has a NIC in a LAN and a NIC in a DMZ, it took some experimenting to get the behavior I was looking for with jails that used the FIB I assigned talking only to the default gateway I wanted them to.

2. The firewall solution. The second NAT would make sense here. Basically you would want to NAT the destination to get to your 10.0.0.13 client and NAT the source to have the reply to come back to the 10.3.0.1 server. I've done this at one point on Linux but never ran into a situation where I needed to look at doing it on FreeBSD.

3. The proxy solution. It sounds like you got this figured out but I'll attach an example anyway for TCP proxy of XMPP traffic. The packets in transit look the same as far as who they go to/from but it's add on service rather than natd(8) or in kernel NAT doing the change.


Code: 
global
        maxconn 2048
        chroot /var/empty
        user www
        group www
        daemon
        stats socket /var/run/haproxy.sock
defaults
        log global
        retries 3
        option redispatch
        maxconn 2000
        timeout connect 5s
        timeout client 600s
        timeout server 600s
frontend frontend-jabber-c2s
        mode tcp
        option tcplog
        bind *:5222
        default_backend backend-jabber-c2s
        timeout client 8h
backend backend-jabber-c2s
        mode tcp
        option tcplog
        balance roundrobin
        server jabber.example.com jabber.example.com:5222
        timeout server 1h
 
1. fib feels not like a good solution to me... somehow it feels more dirty than other solutions I can think of :)
2. The dual-NAT solution was what I tried but did not worked.

I had a natd running on the external-ip of my server with redirecting port 80 to one of the Client-IPs.
I had a natd running on tun0 of the server with no redirecting just for changing the source-ip to the OpenVPN-Server-IP.

Packages came through the first natd, where sent through the second natd to the Client-IP, and even the answer of the client arrived through the second natd and where translated back but then, the first natd didn't picked them up and they where just..... lost on the server. This was what I posted:

Code: 
In {default}[TCP] [TCP] <browser-ip>:18691 -> <external-ip>:80 aliased to
[TCP] <browser-ip>:18691 -> 10.0.0.13:80
Out {vpn}[TCP] [TCP] <browser-ip>:18691 -> 10.0.0.13:80 aliased to
[TCP] 10.3.0.1:18691 -> 10.0.0.13:80
In {vpn}[TCP] [TCP] 10.0.0.13:80 -> 10.3.0.1:18691 aliased to
[TCP] 10.0.0.13:80 -> <browser-ip>:18691

"default" is the NAT running on the external ip, and "vpn" is the NAT running on tun0.

3. yeah the proxy works damn good right now.
 
You must log in or register to reply here.
Back
Top