Can't Get ipv6 Working (reliably) On Contabo VPS Freebsd 13.1

PatchyFog

PatchyFog

Happy New Year, all.

I've spun up a Freebsd 13.1 VPS at Contabo using their image and then I freebsd-update-ed it to p3. They give me a /64 6 subnet and I'd like to statically allocate 2 of my 18 quintillion addresses for mail and other services. (For this, I have chosen nodes 5 and 23 in honor of Discordia and Max Headroom, respectively.) But even though my ifconfig seemed to be coming out OK, I could not reach any other hosts. Anyway, I was in the middle of writing a long, detailed screed here to ask for help when suddenly everything started working! I mean, after 2 days of NOTHING, and I really hadn't changed anything. (Contabo support has (so far) provided nothing but cut-and-paste non-responsive responses.)

At this point, I think their gateway router at link-local fe80::1 is not up consistently or I'm having trouble with discovery. When everythinig works ndp -a shows the router present at link addr 00:c0:1d:c0:ff:ee (top of list):

Code: 
Neighbor                             Linklayer Address  Netif Expire    S Flags
fe80::1%eth0                         00:c0:1d:c0:ff:ee   eth0 23h59m5s  S R
2605:a140:2114:2385::23              00:50:56:49:64:a7   eth0 permanent R
2605:a140:2114:2385::5               00:50:56:49:64:a7   eth0 permanent R
fe80::250:56ff:fe49:64a7%eth0        00:50:56:49:64:a7   eth0 permanent R
But when it doesn't, fe80::1 is missing and it's just my 2 static globals and my 1 auto link-local showing:
Code: 
Neighbor                             Linklayer Address  Netif Expire    S Flags
2605:a140:2114:2385::23              00:50:56:49:64:a7   eth0 permanent R
2605:a140:2114:2385::5               00:50:56:49:64:a7   eth0 permanent R
fe80::250:56ff:fe49:64a7%eth0        00:50:56:49:64:a7   eth0 permanent R
If I run ndp -a in a loop with a 2 second sleep when it is not working, sometimes I can catch it in some intermediate state:
Code: 
Neighbor                             Linklayer Address  Netif Expire    S Flags
fe80::1%eth0                         (incomplete)        eth0 expired   I  1
2605:a140:2114:2385::23              00:50:56:49:64:a7   eth0 permanent R
2605:a140:2114:2385::5               00:50:56:49:64:a7   eth0 permanent R
fe80::250:56ff:fe49:64a7%eth0        00:50:56:49:64:a7   eth0 permanent R
I would love to blame their gateway, but when it is broken I have noticed that just doing this:
Code: 
ping6 -v fe80::1%eth0
PING6(56=40+8+8 bytes) fe80::250:56ff:fe49:64a7%eth0 --> fe80::1%eth0
32 bytes from fe80::1%eth0: Neighbor Advertisement
32 bytes from fe80::1%eth0: Neighbor Advertisement
^C
--- fe80::1%eth0 ping6 statistics ---
11 packets transmitted, 0 packets received, 100.0% packet loss
...usually causes the router to magically show up in ndp -aand everything starts working! (I mean, I've never gotten a ping response from it, even when things are working:
Code: 
[shn ~] -> ping6 www.kame.net
PING6(56=40+8+8 bytes) 2605:a140:2114:2385::23 --> 2001:2f0:0:8800:226:2dff:fe0b:4311
16 bytes from 2001:2f0:0:8800:226:2dff:fe0b:4311, icmp_seq=0 hlim=47 time=164.888 ms
16 bytes from 2001:2f0:0:8800:226:2dff:fe0b:4311, icmp_seq=1 hlim=47 time=164.951 ms
16 bytes from 2001:2f0:0:8800:226:2dff:fe0b:4311, icmp_seq=2 hlim=47 time=164.620 ms
^C
--- mango.itojun.org ping6 statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 164.620/164.820/164.951/0.143 ms
[shn ~] -> ping6 fe80::1%eth0
PING6(56=40+8+8 bytes) fe80::250:56ff:fe49:64a7%eth0 --> fe80::1%eth0
^C
--- fe80::1%eth0 ping6 statistics ---
7 packets transmitted, 0 packets received, 100.0% packet loss
...but seeing the discovery packets come in with the -v switch to ping seems to indicate/change something. (Tells the router my host exists?)

So...is the problem on their end or mine? Is running ndp -a just reporting or does it change the system state? Does the ping6 -v "fix" give any indication what might be going on?

Requisite configs below. They don't change whether I have global connectivity or not.

Thanks all.

-- Ward

My rc.conf for this static configuration looks like this:
Code: 
hostname=vmi1142385.contaboserver.net
ifconfig_DEFAULT="DHCP inet6 accept_rtadv"
growfs_enable="YES"
cloudinit_enable="YES"
# ------------------------------------------------------------------------------
ifconfig_vtnet0_name=eth0
ipv6_enabled="YES"
firewall_enable="NO"

defaultrouter=207.244.224.1
ifconfig_eth0="inet 207.244.238.92 netmask 255.255.240.0"
ifconfig_eth0_alias0="inet 207.244.238.92 netmask 255.255.240.0"
ifconfig_eth0_alias1="inet 144.126.129.136 netmask 255.255.240.0"


ipv6_defaultrouter="fe80::1%eth0"
ifconfig_eth0_ipv6="inet6 2605:a140:2114:2385::23 prefixlen 64"
ifconfig_eth0_alias2="inet6 2605:a140:2114:2385::23 prefixlen 64"
ifconfig_eth0_alias3="inet6 2605:a140:2114:2385::05 prefixlen 64"
and produces this interface:
Code: 
[/etc] -> ifconfig eth0

eth0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6>
         ether 00:50:56:49:64:a7
        inet6 fe80::250:56ff:fe49:64a7%eth0 prefixlen 64 scopeid 0x1
        inet6 2605:a140:2114:2385::23 prefixlen 64
        inet6 2605:a140:2114:2385::5 prefixlen 64
        inet 207.244.238.92 netmask 0xfffff000 broadcast 207.244.239.255
        inet 144.126.129.136 netmask 0xfffff000 broadcast 144.126.143.255
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
And the routes:
Code: 
Internet6:

Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
default                           fe80::1%eth0                  UGS        eth0
::1                               link#2                        UHS         lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
2605:a140:2114:2385::/64          link#1                        U          eth0
2605:a140:2114:2385::5            link#1                        UHS         lo0
2605:a140:2114:2385::23           link#1                        UHS         lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%eth0/64                    link#1                        U          eth0
fe80::250:56ff:fe49:64a7%eth0     link#1                        UHS         lo0
fe80::%lo0/64                     link#2                        U           lo0
fe80::1%lo0                       link#2                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0
 
I just commented out all Contabo pre-filled stuff from /etc/rc.conf and statically configured my IPv6 network interface vtnet0.
Code: 
ipv6_defaultrouter="fe80::1%vtnet0"
ifconfig_vtnet0_ipv6="inet6 2a02:c206:xxxx:xxxx::1 prefixlen 64"
rtsold_enable="YES"
rtsold_flags="-Fa"
As you can see I am starting the router solicitation daemon at boot.
 
facedebouc said:
I just commented out all Contabo pre-filled stuff from /etc/rc.conf and statically configured my IPv6 network interface vtnet0.
Code: 
ipv6_defaultrouter="fe80::1%vtnet0"
ifconfig_vtnet0_ipv6="inet6 2a02:c206:xxxx:xxxx::1 prefixlen 64"
rtsold_enable="YES"
rtsold_flags="-Fa"
As you can see I am starting the router solicitation daemon at boot.
Click to expand...
Ah, okay. I see you dropped the eth0 alias for vtnet0 -- but is rtsold the secret sauce? My thinking was that my host only needed to handle router solicitation messages (and rtsold) if I was doing SLAAC and being all dynamic and shit. No? I need it even with "defaultrouter" manually set? (BTW, my host has been up for hours since I did the ping -v thing and kicked fe80::1 into life.)

As you can see, I'm no expert. Thanks for your reply.
 
PatchyFog said:
Ah, okay. I see you dropped the eth0 alias for vtnet0 -- but is rtsold the secret sauce? My thinking was that my host only needed to handle router solicitation messages (and rtsold) if I was doing SLAAC and being all dynamic and shit. No? I need it even with "defaultrouter" manually set? (BTW, my host has been up for hours since I did the ping -v thing and kicked fe80::1 into life.)

As you can see, I'm no expert. Thanks for your reply.
Click to expand...
That's the way it works for me.
Default configuration from Contabo with cloud-init does not work for me so I made some cleaning after installation of their FreeBSD 13.1 image.
I removed cloud-init configuration and package as well as Linuxisms like eth0 and /etc/timezone.
I statically assign IPv6 addresses to the host and his jails.
And yes rtsold(8) does the job.
 
Well, firing up rtsold at boot didn't change anything for me (nor did enabling ifconfig_eth0_ipv6="inet6 accept_rtadv" -- but I still kinda think I should not need either one since I am forcing it with defautrouter. But a quick ping6 fe80::1%eth0 makes the router pop right up as a neighbor in the ndp -a list and everything sings and dances after that. (So I mean, I could script that as a boot-time workaround if it comes to that, but would sure like to not have to.)

I followed your lead and dropped all their other cruft (and I agree with you about sshd and sudo -- I have my own config for the former I drag around.

Anyway, thanks -- it was worth a shot.
 
Don't need to ping6, I get the same result as yours after reboot with ndp -an, but after a first IPv6 request it's OK, the defaultrouter appears in the NDP table.
The first IPv6 DNS lookup does the job :
host -v -6 www.freebsd.org
I didn't see a loss of IPv6 connection after that.
 
PatchyFog said:
Happy New Year, all.

I've spun up a Freebsd 13.1 VPS at Contabo using their image and then I freebsd-update-ed it to p3. They give me a /64 6 subnet and I'd like to statically allocate 2 of my 18 quintillion addresses for mail and other services. (For this, I have chosen nodes 5 and 23 in honor of Discordia and Max Headroom, respectively.) But even though my ifconfig seemed to be coming out OK, I could not reach any other hosts. Anyway, I was in the middle of writing a long, detailed screed here to ask for help when suddenly everything started working! I mean, after 2 days of NOTHING, and I really hadn't changed anything. (Contabo support has (so far) provided nothing but cut-and-paste non-responsive responses.)

At this point, I think their gateway router at link-local fe80::1 is not up consistently or I'm having trouble with discovery. When everythinig works ndp -a shows the router present at link addr 00:c0:1d:c0:ff:ee (top of list):

Code: 
Neighbor                             Linklayer Address  Netif Expire    S Flags
fe80::1%eth0                         00:c0:1d:c0:ff:ee   eth0 23h59m5s  S R
2605:a140:2114:2385::23              00:50:56:49:64:a7   eth0 permanent R
2605:a140:2114:2385::5               00:50:56:49:64:a7   eth0 permanent R
fe80::250:56ff:fe49:64a7%eth0        00:50:56:49:64:a7   eth0 permanent R
But when it doesn't, fe80::1 is missing and it's just my 2 static globals and my 1 auto link-local showing:
Code: 
Neighbor                             Linklayer Address  Netif Expire    S Flags
2605:a140:2114:2385::23              00:50:56:49:64:a7   eth0 permanent R
2605:a140:2114:2385::5               00:50:56:49:64:a7   eth0 permanent R
fe80::250:56ff:fe49:64a7%eth0        00:50:56:49:64:a7   eth0 permanent R
If I run ndp -a in a loop with a 2 second sleep when it is not working, sometimes I can catch it in some intermediate state:
Code: 
Neighbor                             Linklayer Address  Netif Expire    S Flags
fe80::1%eth0                         (incomplete)        eth0 expired   I  1
2605:a140:2114:2385::23              00:50:56:49:64:a7   eth0 permanent R
2605:a140:2114:2385::5               00:50:56:49:64:a7   eth0 permanent R
fe80::250:56ff:fe49:64a7%eth0        00:50:56:49:64:a7   eth0 permanent R
I would love to blame their gateway, but when it is broken I have noticed that just doing this:
Code: 
ping6 -v fe80::1%eth0
PING6(56=40+8+8 bytes) fe80::250:56ff:fe49:64a7%eth0 --> fe80::1%eth0
32 bytes from fe80::1%eth0: Neighbor Advertisement
32 bytes from fe80::1%eth0: Neighbor Advertisement
^C
--- fe80::1%eth0 ping6 statistics ---
11 packets transmitted, 0 packets received, 100.0% packet loss
...usually causes the router to magically show up in ndp -aand everything starts working! (I mean, I've never gotten a ping response from it, even when things are working:
Code: 
[shn ~] -> ping6 www.kame.net
PING6(56=40+8+8 bytes) 2605:a140:2114:2385::23 --> 2001:2f0:0:8800:226:2dff:fe0b:4311
16 bytes from 2001:2f0:0:8800:226:2dff:fe0b:4311, icmp_seq=0 hlim=47 time=164.888 ms
16 bytes from 2001:2f0:0:8800:226:2dff:fe0b:4311, icmp_seq=1 hlim=47 time=164.951 ms
16 bytes from 2001:2f0:0:8800:226:2dff:fe0b:4311, icmp_seq=2 hlim=47 time=164.620 ms
^C
--- mango.itojun.org ping6 statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 164.620/164.820/164.951/0.143 ms
[shn ~] -> ping6 fe80::1%eth0
PING6(56=40+8+8 bytes) fe80::250:56ff:fe49:64a7%eth0 --> fe80::1%eth0
^C
--- fe80::1%eth0 ping6 statistics ---
7 packets transmitted, 0 packets received, 100.0% packet loss
...but seeing the discovery packets come in with the -v switch to ping seems to indicate/change something. (Tells the router my host exists?)

So...is the problem on their end or mine? Is running ndp -a just reporting or does it change the system state? Does the ping6 -v "fix" give any indication what might be going on?

Requisite configs below. They don't change whether I have global connectivity or not.

Thanks all.

-- Ward

My rc.conf for this static configuration looks like this:
Code: 
hostname=vmi1142385.contaboserver.net
ifconfig_DEFAULT="DHCP inet6 accept_rtadv"
growfs_enable="YES"
cloudinit_enable="YES"
# ------------------------------------------------------------------------------
ifconfig_vtnet0_name=eth0
ipv6_enabled="YES"
firewall_enable="NO"

defaultrouter=207.244.224.1
ifconfig_eth0="inet 207.244.238.92 netmask 255.255.240.0"
ifconfig_eth0_alias0="inet 207.244.238.92 netmask 255.255.240.0"
ifconfig_eth0_alias1="inet 144.126.129.136 netmask 255.255.240.0"


ipv6_defaultrouter="fe80::1%eth0"
ifconfig_eth0_ipv6="inet6 2605:a140:2114:2385::23 prefixlen 64"
ifconfig_eth0_alias2="inet6 2605:a140:2114:2385::23 prefixlen 64"
ifconfig_eth0_alias3="inet6 2605:a140:2114:2385::05 prefixlen 64"
and produces this interface:
Code: 
[/etc] -> ifconfig eth0

eth0: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6>
         ether 00:50:56:49:64:a7
        inet6 fe80::250:56ff:fe49:64a7%eth0 prefixlen 64 scopeid 0x1
        inet6 2605:a140:2114:2385::23 prefixlen 64
        inet6 2605:a140:2114:2385::5 prefixlen 64
        inet 207.244.238.92 netmask 0xfffff000 broadcast 207.244.239.255
        inet 144.126.129.136 netmask 0xfffff000 broadcast 144.126.143.255
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
And the routes:
Code: 
Internet6:

Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
default                           fe80::1%eth0                  UGS        eth0
::1                               link#2                        UHS         lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
2605:a140:2114:2385::/64          link#1                        U          eth0
2605:a140:2114:2385::5            link#1                        UHS         lo0
2605:a140:2114:2385::23           link#1                        UHS         lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%eth0/64                    link#1                        U          eth0
fe80::250:56ff:fe49:64a7%eth0     link#1                        UHS         lo0
fe80::%lo0/64                     link#2                        U           lo0
fe80::1%lo0                       link#2                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0
Click to expand...

My server at Netcup had the same issue as well. The v6 route to our VPS would just suddenly not work. It's as though their stateful firewall keeps track of active v6 sessions to our VPS for XXs then kills the session, and when this happens I'd have to ping6 the VPS remotely which won't reply right away, but after a couple of ping attempts it suddenly replies then cycles through killing the session again after 30s - 1 min. Their 'fix' was to buy an additional /64 subnet which is ridiculous so I had a script that ping6 the defaultrouter fe80::1 every 20s until one day magically they fix it and things just started to work again after that.
 
Recently Contabo has been migrating VPS(s) from their DC Nuremberg to DC Hub Europe (Lauterbourg), hence the pre-configured setting static_ndp_gw is no longer valid.

Code: 
#DC Nuremberg
static_ndp_gw="fe80::1%vtnet0 28:99:3a:4d:30:af"

If your VPS has moved to DC Hub Europe use the below settings instead.

Code: 
#DC Hub Europe
static_ndp_gw="fe80::1%vtnet0 00:c0:1d:c0:ff:ee"

In case it has moved somewhere else, and ndp -a shows fe80::1%vtnet0 on a list of neighbors, yet IPv6 is not working, check facedebouc's suggestion. You can either find what gw should pair with and set it statically through static_ndp_gw or just let it run with the value discovered.
 
Just after testing the static setting, as I have described above, and unfortunately for Contabo's DC Hub Europe it does not work, even after reboot of VPS, until you perform the first ping6. So for now I will stick with rtsold.
 
On a vps hosted at Nuremberg, the 'fe80::1' gateway respond to neighbor solicitation:

10:58:48.354686 IP6 fe80::250:56ff:fe56:122f > fe80::1: ICMP6, neighbor solicitation, who has fe80::1, length 32
10:58:48.358550 IP6 fe80::1 > fe80::250:56ff:fe56:122f: ICMP6, neighbor advertisement, tgt is fe80::1, length 32

But when the vps is hosted at Hub Europe:

11:10:36.488969 IP6 fe80::250:56ff:fe57:2593 > fe80::1: ICMP6, neighbor solicitation, who has fe80::1, length 32
11:10:37.545248 IP6 fe80::250:56ff:fe57:2593 > fe80::1: ICMP6, neighbor solicitation, who has fe80::1, length 32
11:10:38.589000 IP6 fe80::250:56ff:fe57:2593 > fe80::1: ICMP6, neighbor solicitation, who has fe80::1, length 32

After reading this bug report:

263288 – IPv6 system not responding to Neighbor Solicitation

bugs.freebsd.org bugs.freebsd.org

I solve the problem with the following sysctl:

$ grep icmp6 /etc/sysctl.conf
net.inet6.icmp6.nd6_onlink_ns_rfc4861=1

and a "crappy hack" to generate initial ipv6 traffic:

$ cat /etc/cron.d/contabo_ipv6
@reboot root:wheel /sbin/ping6 -c 1 2a02:c207::1:53
 
bsdsx said:
and a "crappy hack" to generate initial ipv6 traffic:
Code: 
$ cat /etc/cron.d/contabo_ipv6 
@reboot root:wheel /sbin/ping6 -c 1 2a02:c207::1:53
Click to expand...
/etc/rc.d/netwait might be useful.

Code: 
# The netwait script helps handle two situations:
#  - Systems with USB or other late-attaching network hardware which
#    is initialized by devd events.  The script waits for all the
#    interfaces named in the netwait_if list to appear.
#  - Systems with statically-configured IP addresses in rc.conf(5).
#    The IP addresses in the netwait_ip list are pinged.  The script
#    waits for any single IP in the list to respond to the ping.  If your
#    system uses DHCP, you should probably use synchronous_dhclient="YES"
#    in your /etc/rc.conf instead of netwait_ip.
# Either or both of the wait lists can be used (at least one must be
# non-empty if netwait is enabled).

Something like:
Code: 
netwait_enable="YES"
netwait_ip="2a02:c207::1:53"

I use it to wait for the PPPoE connection to come up before resuming the boot process on my firewall.
 
You must log in or register to reply here.
Back
Top