Zerotier rc.conf issues

W

Whiskers

Hi all - really appreciate any help people can please provide.

I've been trying to set up zerotier (https://www.zerotier.com/) on a freebsd box that serves as my home (multi purpose) server, so that I can VPN into my home lan remotely.

I can get the whole thing to work but it requires some bizarre weirdness. After a reboot, I need to run "service netif restart" exactly twice before the VPN will work correctly! Then it works fine.

The broad architecture is as follows:
  • My home network is on 192.168.1.0/24
  • The FreeBSD server is 192.168.1.50
  • I've been running zerotier in bridge mode: I create a cloned interface called "bridge0" on 192.168.1.55 and bridge re0 (the main nic) with the device the zerotier creates: "ztbc1svef37f9s2")
  • To do this, my rc.conf goes through the steps of creating bridge0, settings its IP to 192.168.1.55, and putting re0 and ztbc1svef37f9s2 on the bridge. Zerotier defaults to an MTU of 2800 so rec.conf also needs to change that to 1500 before it adds it as a member of the bridge
Here is an extract from my rc.conf (full file attached below)

ifconfig_re0="inet 192.168.1.50 netmask 255.255.255.0"

...

defaultrouter="192.168.1.1"

gateway_enable="YES"

ipv6_gateway_enable="YES"

...

zerotier_enable="YES"

ifconfig_ztbc1svef37f9s2="mtu 1500 up"

cloned_interfaces="bridge0"

autobridge_interfaces="bridge0"

autobridge_bridge0="re0 ztbc1svef37f9s2"

ifconfig_bridge0="inet 192.168.1.55/24 addm re0 addm ztbc1svef37f9s2 up"
Click to expand...

When I first boot up, I can see that zerotier has created the ztbc1svef37f9s2 interface but the MTU has not been updated. Also, the bridge0 innterface does not have the IP address of 192.168.1.55 set, and it is missing ztbc1svef37f9s2 as a member. For these reasons the VPN does not work at this point:

re0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>
ether 4c:cc:6a:68:54:ea
inet 192.168.1.50 netmask 0xffffff00 broadcast 192.168.1.255
media: Ethernet autoselect (1000baseT <full-duplex,master>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 58:9c:fc:10:49:31
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: re0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 55
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
ztbc1svef37f9s2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 5000 mtu 2800
options=80000<LINKSTATE>
ether 82:48:7a:ca:49:f2
hwaddr 58:9c:fc:10:bd:30
groups: tap
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Opened by PID 1031
Click to expand...

At this point I run service netif restart. This manages to update the MTU on ztbc1svef37f9s2 to the desired figure of 1500 (partial success!). However, bridge0 still has the wrong IP and is missing ztbc1svef37f9s2 as a member.

ztbc1svef37f9s2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 5000 mtu 1500
options=80000<LINKSTATE>
ether 82:48:7a:ca:49:f2
hwaddr 58:9c:fc:10:bd:30
groups: tap
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Opened by PID 1031
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 58:9c:fc:10:49:31
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: re0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 20000
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
Click to expand...

So I do one more service netif restart and voila(!) we get to what what want - the details of bridge0 are updated to reflect the desired ip address and the addtion of the ztbc1svef37f9s2 as a member.

ztbc1svef37f9s2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 5000 mtu 1500
options=80000<LINKSTATE>
ether 82:48:7a:ca:49:f2
hwaddr 58:9c:fc:10:bd:30
groups: tap
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Opened by PID 1031
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 58:9c:fc:10:49:31
inet 192.168.1.55 netmask 0xffffff00 broadcast 192.168.1.255
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: ztbc1svef37f9s2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 4 priority 128 path cost 2000000
member: re0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 20000
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
Click to expand...

At this point the whole set up works. I have the zerotier service running on the freebsd box and a mobile phone and I can VPN into the home lan from the phone (on the cellular network) and it is all great.

But it feels silly to have to go in and artifically restart the network exactly twice after a power cycle to get back to this state. I'm sure I'm doing something fairly silly so greatly appreciate any direction people are prepared to offer me.

Kind regards,
Simon
 

Attachments

You must log in or register to reply here.
Back
Top