setfib not working with tun interfaces?

L

lucdig

Hello, I've a 12.0 STABLE kernel compiled with:

Code: 
options IPFIREWALL
options IPFIREWALL_DEFAULT_TO_ACCEPT
options IPFIREWALL_VERBOSE
options IPDIVERT
options DUMMYNET
options HZ=1000
and:
Code: 
root@freebsd:~ # cat /boot/loader.conf
net.fibs=2
My network configuration:
Code: 
root@freebsd:~ # ifconfig hn0
hn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8051b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,LRO,LINKSTATE>
        ether 00:15:5d:64:09:0f
        inet 172.19.167.89 netmask 0xfffffff0 broadcast 172.19.167.95
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
My default gateway:
Code: 
root@freebsd:~ # setfib 0 netstat -nr4
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            172.19.167.81      UGS         hn0
127.0.0.1          link#1             UH          lo0
172.19.167.80/28   link#2             U           hn0
172.19.167.89      link#2             UHS         lo0

---------------------------------

root@freebsd:~ # setfib 1 netstat -nr4
Routing tables (fib: 1)

Internet:
Destination        Gateway            Flags     Netif Expire
127.0.0.1          link#1             UH          lo0
172.19.167.80/28   link#2             U           hn0
I add a fake default gateway to fib=1:
Code: 
root@freebsd:~ # setfib 1 route add default 172.19.167.82
----------
root@freebsd:~ # setfib 1 netstat -nr4
Routing tables (fib: 1)

Internet:
Destination        Gateway            Flags     Netif Expire
default            172.19.167.82      UGS         hn0
127.0.0.1          link#1             UH          lo0
172.19.167.80/28   link#2             U           hn0

Then I tell ipfw to use fib=1 for all traffic:
Code: 
root@freebsd:~ # ipfw add setfib 1 ip from any to any out via hn0
------
root@freebsd:~ # ipfw list
00100 setfib 1 ip from any to any out via hn0
65535 allow ip from any to any
Now a ping 8.8.8.8 and I have no response, because 172.19.167.82 is fake:
Code: 
root@freebsd:~ # ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
ping: sendto: No route to host
Everything OK so far. Packets are routed with routing table with fib=1.

Now I flush ipfw and flush routes in fib=1, and set up a tun100 interface (peer to peer) vith SSH (the <remote host> is Linux - using ip ... commands):
Code: 
ssh -x -l root -p 22 -f -T -N -M -S /tmp/test -o Tunnel=yes -w 100:100 <remote_host>
ssh -x -l root -p 22 -T -S /tmp/test <remote_host> 'ip link set dev tun100 up'
ssh -x -l root -p 22 -T -S /tmp/test <remote_host> 'ip addr add 10.255.255.4/32 peer 10.255.255.3/32 dev tun100'
ifconfig tun100 10.255.255.3 10.255.255.4
Pinging the peer address is OK:
Code: 
root@freebsd:~ # ping 10.255.255.4
PING 10.255.255.4 (10.255.255.4): 56 data bytes
64 bytes from 10.255.255.4: icmp_seq=0 ttl=64 time=73.775 ms
Now I add the peer tun100 remote address as default gateway in routing table with fib=1:
Code: 
root@freebsd:~ # setfib 1 route add default 10.255.255.4
add net default: gateway 10.255.255.4 fib 1
-----
root@freebsd:~ # setfib 1 netstat -nr4
Routing tables (fib: 1)

Internet:
Destination        Gateway            Flags     Netif Expire
default            10.255.255.4       UGS      tun100
10.255.255.4       link#3             UH       tun100
127.0.0.1          link#1             UH          lo0
172.19.167.80/28   link#2             U           hn0
Then I tell ipfw to use fib=1 for all traffic via tun100:
Code: 
root@freebsd:~ # ipfw add setfib 1 ip from any to any out via tun100
00100 setfib 1 ip from any to any out via tun100
-----
root@freebsd:~ # ipfw list
00100 setfib 1 ip from any to any out via tun100
65535 allow ip from any to any

Now I ping again 8.8.8.8, I expect to use fib=1 with default gateway 10.255.255.4 via tun100, but...
Code: 
root@freebsd:~ # ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=119 time=5.476 ms
------
root@freebsd:~ # tcpdump -n -i tun100
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun100, link-type NULL (BSD loopback), capture size 262144 bytes

#### NO PACKETS ####
-------
root@freebsd:~ # tcpdump -n -i hn0 icmp and host 8.8.8.8
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on hn0, link-type EN10MB (Ethernet), capture size 262144 bytes


10:45:39.091148 IP 172.19.167.89 > 8.8.8.8: ICMP echo request, id 50439, seq 0, length 64
10:45:39.099847 IP 8.8.8.8 > 172.19.167.89: ICMP echo reply, id 50439, seq 0, length 64
10:45:40.092844 IP 172.19.167.89 > 8.8.8.8: ICMP echo request, id 50439, seq 1, length 64
10:45:40.098097 IP 8.8.8.8 > 172.19.167.89: ICMP echo reply, id 50439, seq 1, length 64

It seems that ipfw setfib 1 doesn't work with tun100... do you know why?

Thanks in advance, regards
 
Last edited by a moderator:
Hello,

  1. packets from ping 8.8.8.8 are in fib 0.
  2. they choose default gateway of fib 0, that is 172.19.167.81.
  3. 172.19.167.81 is on hn0, so they are going out through hn0.
  4. packets going out through hn0 are not trapped by rule 100, but 65535.

So, the packets you ping'ed are never set to fib 1. See how the packet count of each rule changes while you are ping'ing.
setfib 1 ping 8.8.8.8 should work.
 
Hi Hiroo Ono

  1. packets from ping 8.8.8.8 are in fib 0.
Why? I set ipfw add setfib 1 ip from any to any out via tun100, is this the right command? Am I missing any other configuration/command?

I would like to send all the packets to fib=1, without putting setfib 1 before every command.

Thanks, regards
 
Because ping 8.8.8.8 is equivalent to setfib 0 ping 8.8.8.8. It is just an explanation of how packets behave under your configuration.

If what you want to do is that all packets (except the encapsulated packets that ssh sends and receives) goes through tun100 to be encapsulated by ssh, you do not need to use multiple fibs and ipfw.
Code: 
route add <remote host> 172.19.167.81
route change default 10.255.255.4
The problem with this setting is that if you ping <remote host>, it passes hn0 not encapsulated by ssh. You have to connect to 10.255.255.4 of the remote host.

If you want to use multiple fibs, and my guess of your intention is correct, the folloing configuration should work. You do not need ipfw to set fib.
Code: 
setfib 1 route add default 172.19.167.81

setfib 1 ssh -x -l root -p 22 -f -T -N -M -S /tmp/test -o Tunnel=yes -w 100:100 <remote_host>
setfib 1 ssh -x -l root -p 22 -T -S /tmp/test <remote_host> 'ip link set dev tun100 up'
setfib 1 ssh -x -l root -p 22 -T -S /tmp/test <remote_host> 'ip addr add 10.255.255.4/32 peer 10.255.255.3/32 dev tun100'
ifconfig tun100 10.255.255.3 10.255.255.4

setfib 0 route change default 10.255.255.4
 
Yes, it works, the "secret" is to use the fib=1 for the ssh traffic on port 22, with setfib 1 ssh -x ....

Then, with:
Code: 
natd -interface tun0

ipfw add divert natd ip from any to not <remote_host> out via tun100
ipfw add divert natd ip from any to not <remote_host> in via tun100
I have a Poor Man's VPN for my internal network to my remote host (with gateway_enable="YES" in /etc/rc.local as well).

I want to use fib=1 because I can simply distinguish the special route for ssh, and the routes for the rest of the the traffic.

Actually, instead of setfib 0 route change default 10.255.255.4, I set:
Code: 
setfib 0 route add 0.0.0.0/1 10.255.255.4
setfib 0 route add 128.0.0.0/1 10.255.255.4

Doing this way, when I kill the process and the tun100 is destroyed, I don't have to configure again the original default route in fib=0.

Thank you very much for your help
 
You must log in or register to reply here.
Back
Top