Solved Dual-homed IPv6 via FreeBSD gateway with pf(4)

[Incnis Mrsi]

Hopefully my experience and analysis will be appreciated here.
IPv6 was developed having multihoming in mind, and it can be beneficial when no single “good” IPv6 supply exists locally. In many parts of the world an ISP can linger in business without offering IPv6 connectivity. The shortage of native IPv6 prompts network admins to deploy workarounds. Imagine that one workaround is robust but has low bandwidth, whereas another one is fast but not very reliable. You might opt to keep both connections online simultaneously. But if two active connections are routed via the same host, then won’t the OS be confused? Won’t packets depart via a route incompatible to their source addresses (to be eventually discarded)? The regular destination-based routing is inapplicable here.

What is solved?
This article describes a solution for routing outbound IPv6 packets for two connections. Namely, how to make each packet depart via the interface its source address pertains to. This can be used for IPv6 deployment in LANs.

What is not in focus?
The article will not consider how applications choose between their IPv6 addresses (or IPv4). It also does not devolve into general IPv6 setup questions.

Conditions
In the example below the LAN administrator controls two IPv6 address ranges implemented as 6in4 encapsulation. First one is 6to4 − an IPv6 deployment mechanism where the 2002::/16 block is mapped onto v4 Internet. If your ISP gives you a public IP, has a route towards 192.88.99.1 and does not mangle carrier packets, then FreeBSD will enable it for you (under stf0) if /etc/rc.conf states:

ipv6_enable="YES"
stf_interface_ipv4addr="▀▀.▄▄.◥◥.◣◣"  # Placeholder for the public IP
ipv6_defaultrouter="2002:c058:6301::"

IETF considers 6to4 deprecated since 2015, but its advantage is that you available IPv6 range is determined by the public IP address (which in my case is static). Its main disadvantage is low bandwidth; also, there may be some latency (e.g. when a 6to4 packet from Russia does to the native v6 Internet via Amsterdam).

I obtained the second IPv6 connection from Hurricane Electric (operating under tunnelbroker.net); it provides broad bandwidth and its endpoint is somewhat closer than Amsterdam is to Russia. The kernel calls the respective interface gif0; I set it up with a rc.d script which is out of scope of this article; here you can find necessary commands. Disadvantages of tunnelbroker.net are (relatively) meagre range (namely, /64) for a general user and absence of contractual duties on the part of HE − the service is free of charge.

On pf④
If you don’t know already, /etc/pf.conf contains the text of a program for the pf(4) packet filter (which is supplied with FreeBSD). The text consists of definitions (macros) and rules of the sort “for each packet satisfying given predicates (source, destination, port, interface, etc.) do so and so”. When /etc/rc.conf has pf_enable="YES", FreeBSD startup feeds the rules to the kernel. You can also enable the (new) rules with
pfctl -e -f /etc/pf.conf
Note that -e has no effect if pf④ is already enabled.

Solution
Assume that our /etc/pf.conf defines

myv6_6to4 = "2002:▀▀▄▄:◥◥◣◣::/48"
myv6_he   = "2001:470:◆◆:●●●::/64"
v6_noglobal = "fc00::/6"

Only two rules are sufficient now:

# routing related to tunnels
pass quick from $myv6_he to {$myv6_6to4, $myv6_he, $v6_noglobal}
pass out route-to gif0 from $myv6_he to any

The pass out route-to ⋯ rule performs the bulk of job; it directs outbound packets to pass via gif0 ignoring the routing table (remind that the latter has default route to 6to4).

The pass quick ⋯ rule quickly terminates execution on packets originating from the range in question destined to the same range ($myv6_he), another “our” range ($myv6_6to4), as well as non-global addresses. Such traffic shouldn’t be forced into gif0.

Note that pass rules of pf.conf must go after definitions and redirection rules (rdr, nat).

Remarks
If you specify pass in route-to ⋯ instead of pass out route-to ⋯, then the rule will deal with transit traffic but ignore traffic originating locally.

For only two connections we can route 6to4 by default, using regular routing. What for more connections: can we route 6to4 explicitly? We could add respective rules for stf0:

pass quick from $myv6_6to4 to {$myv6_6to4, $myv6_he, $v6_noglobal}
pass out route-to stf0 from $myv6_6to4 to any

but there is a gotcha: 6to4 is not technically a tunnel like gif0 is. Packets sent to stf0 can be directed to different points. Makes a problem if, as we supposed, the default route is something other than 6to4. Although pf④ permits to specify the immediate routing point with route-to (interface, gateway), 6to4 has different routing paths for 2002::/16 and the rest of v6 Internet. One can try two steps (instead of one above):

pass out quick route-to stf0 from $myv6_6to4 to 2002::/16
pass out route-to (stf0, 2002:c058:6301::) from $myv6_6to4 to any

but I am not willing to fix a working system in such way.

 

[Incnis Mrsi]

Incidentally discovered that the ruleset proposed above interferes with stateful processing of traffic as described by www.openbsd.org/faq/pf/filter.html#state . This can be fixed, in this presumed minimal setup, with one rule suffixed with “no state” and another dummy rule. Namely:

# Routing related to tunnels
pass quick from $myv6_he to {$myv6_6to4, $myv6_he, $v6_noglobal}
pass out route-to gif0 from $myv6_he to any no state

# Other filtering rules

# dummy rule
pass out route-to gif0 from $myv6_he to any no state

See Thread stateful-gotcha-in-pf-4.86008 to explain where this workaround is necessary.