IPv6 connectivity lost after some time

B

basbebe

Since a few weeks I'm having a problem with the IPv6 connectivity of my FreeBSD 11.0 server. There had been no software updates or configuration changes that I know of that could cause the problem.

The server only has IPv6 connectivity (incoming and outgoing) for a short time (approx. 40-80 minutes) after a reboot. Then it is gone and I don't know why.

It worked like a charm for months before.

This is my config:

/etc/rc.conf

Code: 
#IPv4

ifconfig_vtnet0_name="em0"
ifconfig_em0="inet YY.YY.YY.YY netmask 255.255.252.0"
defaultrouter="188.TT.TT.TT"

# IPv6

ifconfig_em0_ipv6="inet6 accept_rtadv"
rtsold_enable="YES"
ifconfig_em0_alias0="inet6 2aXX:XXXX:XXXX:XXXX::1 prefixlen 64"
ipv6_defaultrouter="fe80::1%em0"

gateway_enable="YES"
ipv6_gateway_enable="YES"

ifconfig

Code: 
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=6c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    ether ZZ:ZZ:ZZ:ZZ:ZZ:ZZ
    inet YY.YY.YY.YY netmask 0xfffffc00 broadcast 188.DD.DD.DD
    inet6 fe80::KKKK:KKKK:KKKK:1822%em0 prefixlen 64 scopeid 0x1
    inet6 2aXX:XXXX:XXXX:XXXX::1 prefixlen 64
    inet6 2aXX:XXXX:XXXX:XXXX::103 prefixlen 128
    nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
    media: Ethernet 10Gbase-T <full-duplex>
    status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
    inet 127.0.0.1 netmask 0xff000000
    inet 10.0.0.1 netmask 0xffffffff
    inet 10.0.0.4 netmask 0xffffffff
    inet 10.0.0.3 netmask 0xffffffff
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: lo
pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33160
    groups: pflog

The routing tables stay the same after the fail:

netstat -rn

Code: 
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            188.TT.TT.TT        UGS         em0
10.0.0.1           link#2             UH          lo0
10.0.0.2           link#2             UH          lo0
10.0.0.3           link#2             UH          lo0
10.0.0.4           link#2             UH          lo0
10.0.0.10          link#2             UH          lo0
10.0.0.11          link#2             UH          lo0
10.0.0.12          link#2             UH          lo0
10.0.0.13          link#2             UH          lo0
127.0.0.1          link#2             UH          lo0
188.TT.TT.0/22     link#1             U           em0
YY.YY.YY.YY       link#1             UHS         lo0

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
default                           fe80::1%em0                   UGS         em0
::1                               link#2                        UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
2aXX:XXXX:XXXX:XXXX::/64             link#1                        U           em0
2aXX:XXXX:XXXX:XXXX::1               link#1                        UHS         lo0
2aXX:XXXX:XXXX:XXXX::102             link#1                        UHS         lo0
2aXX:XXXX:XXXX:XXXX::103             link#1                        UHS         lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%em0/64                     link#1                        U           em0
fe80::KKKK:KKKK:KKKK:1822%em0     link#1                        UHS         lo0
fe80::%lo0/64                     link#2                        U           lo0
fe80::1%lo0                       link#2                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0


What could be causing this?
Any logs I should check? Anything I might be missing?
Thanks!
 
basbebe said:
Code: 
ifconfig_em0_ipv6="inet6 accept_rtadv"
rtsold_enable="YES"
ifconfig_em0_alias0="inet6 2aXX:XXXX:XXXX:XXXX::1 prefixlen 64"
ipv6_defaultrouter="fe80::1%em0"
Click to expand...
You typically use SLAAC or static addresses, not both.

Also note the discrepancy:
Code: 
    inet6 2aXX:XXXX:XXXX:XXXX::1 prefixlen 64
    inet6 2aXX:XXXX:XXXX:XXXX::103 prefixlen 128
The SLAAC received address has a prefixlen of 128 and your static address has 64.
 
SirDice said:
You typically use SLAAC or static addresses, not both.

Also note the discrepancy:
Code: 
    inet6 2aXX:XXXX:XXXX:XXXX::1 prefixlen 64
    inet6 2aXX:XXXX:XXXX:XXXX::103 prefixlen 128
The SLAAC received address has a prefixlen of 128 and your static address has 64.
Click to expand...

Thanks for pointing this out.

This does not work, however:

/etc/rc.conf
Code: 
#IPv4

ifconfig_vtnet0_name="em0"
ifconfig_em0="inet YY.YY.YY.YY netmask 255.255.252.0"
defaultrouter="188.TT.TT.TT"

# IPv6


rtsold_enable="YES"
ifconfig_em0_ipv6="inet6 2aXX:XXXX:XXXX:XXXX::1 prefixlen 64"
ipv6_defaultrouter="fe80::1%em0"

gateway_enable="YES"
ipv6_gateway_enable="YES"

After a restart there is no IPv6 connectivity. Ifconfig and netstat seem to show the same results like before (except the missing ACCEPT_RTADV) but the server is unreachable from the outside and I can neither ping6 nor traceroute6 from the inside.

The ::102 and ::103 addresses are addresses of jails that got added by ezjail's config.
 
Settings look fine but I have no way of knowing what your provider uses. So I would suggest asking them for the correct settings. At the very least you should be able to ping the default gateway, if that doesn't work there's something wrong with addresses you're using.
 
SirDice said:
Settings look fine but I have no way of knowing what your provider uses. So I would suggest asking them for the correct settings. At the very least you should be able to ping the default gateway, if that doesn't work there's something wrong with addresses you're using.
Click to expand...
Pinging the local Gateway (fe80::1) always works, also traceroute6 fe80::1%em0.
But anything else doesn't work.
As I said, this configuration worked fine for over a year (FreeBSD 10.x and 11.0) and now I keep having this problems.
The Provider says he doesn't know of any problems and that it must be an error in my server's config…

Disabling pf doesn't change anything as well.

Could adding a static route change anything?
Since it worked the way it did I don't know where to start.

And I have no cron jobs on the host system besides the default ones (newsy slog, atrium, etc) and a weekly check for backups…

Pinging fe80::KKKK:KKKK:KKKK:1822 doesn't work as well by the way (after IPv6 stops working)
 
basbebe said:
The Provider says he doesn't know of any problems and that it must be an error in my server's config…
Click to expand...
Don't ask them if there are problems, ask them what the correct settings should be. Then compare that info with the information you've been using.
 
SirDice said:
Don't ask them if there are problems, ask them what the correct settings should be. Then compare that info with the information you've been using.
Click to expand...
The information they provide (my IPs, my Gateway) haven't changed and are correct (fe80::1)
 
basbebe said:
Pinging the local Gateway (fe80::1) always works, also traceroute6 fe80::1%em0.
Click to expand...
Does a traceroute6 -n 2001:4860:4860::8888 not go any further than fe80::1? If that's the case the issue is probably with the provider. If you can ping/traceroute the default gateway your setup is correct and issue is somewhere beyond the gateway (and on the provider's network).
 
SirDice said:
Does a traceroute6 -n 2001:4860:4860::8888 not go any further than fe80::1? If that's the case the issue is probably with the provider. If you can ping/traceroute the default gateway your setup is correct and issue is somewhere beyond the gateway (and on the provider's network).
Click to expand...

trace routes after IPv6 stopped working don't give any answer at all:

Code: 
❯ traceroute6 -n 2001:4860:4860::8888                                                                                                                                                    ⏎
traceroute6 to 2001:4860:4860::8888 (2001:4860:4860::8888) from 2aXX:XXXX:XXXX:XXXX::1, 64 hops max, 12 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  * * *
 
In hindsight that might not be a good test address, I'm not getting anything on 2001:4860:4860::8888 either, I'm not even getting a response from anything in between:
Code: 
root@armitage:~ # traceroute6 -n 2001:4860:4860::8888
traceroute6 to 2001:4860:4860::8888 (2001:4860:4860::8888) from 2a01:7c8:aaaf:2dd::1, 64 hops max, 12 byte packets
(nothing happening)
Ping works though:
Code: 
root@armitage:~ # ping6 -c4 2001:4860:4860::8888
PING6(56=40+8+8 bytes) 2a01:7c8:aaaf:2dd::1 --> 2001:4860:4860::8888
16 bytes from 2001:4860:4860::8888, icmp_seq=0 hlim=55 time=4.852 ms
16 bytes from 2001:4860:4860::8888, icmp_seq=1 hlim=55 time=5.128 ms
16 bytes from 2001:4860:4860::8888, icmp_seq=2 hlim=55 time=4.901 ms
16 bytes from 2001:4860:4860::8888, icmp_seq=3 hlim=55 time=5.308 ms

--- 2001:4860:4860::8888 ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 4.852/5.047/5.308/0.183 ms
And I don't have any IPv6 related issues.
 
Just after a restart it works (as well as ipv6.google.com):

Code: 
~ # ❯❯❯ traceroute6 -n 2001:4860:4860::8888
traceroute6 to 2001:4860:4860::8888 (2001:4860:4860::8888) from 2aXX:XXXX:XXXX:XXXX::1 , 64 hops max, 12 byte packets
 1  2a03:4000:6:f000::2  0.530 ms  0.174 ms  0.629 ms
 2  2a03:4000:0:10::3  3.980 ms  18.043 ms  4.051 ms
 3  2a00:11c0::1:0:0:371  4.215 ms  4.222 ms  4.161 ms
 4  2001:7f8::3b41:0:2  4.548 ms  4.638 ms  4.560 ms
 5  2001:4860::1:0:d0c6  4.889 ms
    2001:4860::1:0:d0d9  4.703 ms
    2001:4860::1:0:d0c5  4.620 ms
 6  2001:4860::8:0:cb95  11.580 ms
    2001:4860::c:4000:f874  5.021 ms
    2001:4860::8:0:cb95  11.342 ms
 7  2001:4860::8:4000:d325  11.570 ms
    2001:4860::8:0:cc40  11.383 ms
    2001:4860::8:4000:d324  30.013 ms
 8  2001:4860::2:0:a94a  12.771 ms
    2001:4860::2:0:76e8  11.404 ms
    2001:4860::2:0:76e7  11.364 ms
 9  * * *
10  *

~ # ❯❯❯ ping6 -c4 2001:4860:4860::8888
PING6(56=40+8+8 bytes) 2aXX:XXXX:XXXX:XXXX::1 --> 2001:4860:4860::8888
16 bytes from 2001:4860:4860::8888, icmp_seq=0 hlim=50 time=11.315 ms
16 bytes from 2001:4860:4860::8888, icmp_seq=1 hlim=50 time=11.456 ms
16 bytes from 2001:4860:4860::8888, icmp_seq=2 hlim=50 time=11.404 ms
16 bytes from 2001:4860:4860::8888, icmp_seq=3 hlim=50 time=12.084 ms

--- 2001:4860:4860::8888 ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 11.315/11.565/12.084/0.304 ms

~ # ❯❯❯ ping6 -c 5 ipv6.google.com
PING6(56=40+8+8 bytes) 2aXX:XXXX:XXXX:XXXX::1 --> 2a00:1450:4001:81e::200e
16 bytes from 2a00:1450:4001:81e::200e, icmp_seq=0 hlim=58 time=4.576 ms
16 bytes from 2a00:1450:4001:81e::200e, icmp_seq=1 hlim=58 time=4.562 ms
16 bytes from 2a00:1450:4001:81e::200e, icmp_seq=2 hlim=58 time=4.479 ms
16 bytes from 2a00:1450:4001:81e::200e, icmp_seq=3 hlim=58 time=4.572 ms
16 bytes from 2a00:1450:4001:81e::200e, icmp_seq=4 hlim=58 time=4.796 ms

--- ipv6.l.google.com ping6 statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 4.479/4.597/4.796/0.106 ms

~ # ❯❯❯ traceroute6 -n ipv6.google.com
traceroute6 to ipv6.l.google.com (2a00:1450:4001:81e::200e) from 2aXX:XXXX:XXXX:XXXX::1 , 64 hops max, 12 byte packets
 1  2a03:4000:6:f000::2  0.245 ms  0.215 ms  0.244 ms
 2  2a03:4000:0:10::3  19.065 ms  4.109 ms  29.701 ms
 3  2a00:11c0::1:0:0:371  4.214 ms  4.126 ms  4.166 ms
 4  2001:7f8::3b41:0:2  4.626 ms  4.465 ms  4.432 ms
 5  2001:4860:0:11e0::1  4.796 ms  4.694 ms
    2001:4860:0:11e2::1  4.785 ms
 6  2001:4860:0:1::2043  4.603 ms
    2001:4860:0:1::2041  4.566 ms
    2001:4860:0:1::2043  4.480 ms
 7  2a00:1450:4001:81e::200e  4.428 ms  4.476 ms  4.475 ms
 
and after it doesn't:

Code: 
~ # ❯❯❯ traceroute6 -n 2001:4860:4860::8888
traceroute6 to 2001:4860:4860::8888 (2001:4860:4860::8888) from 2aXX:XXXX:XXXX:XXXX::1 , 64 hops max, 12 byte packets
 1  * * *
 2  * * *
 3  * * *
 4  * * *
 5  * * *
 6  * * *


~ # ❯❯❯ ping6 -c4 2001:4860:4860::8888                                                                                                                                               ⏎
PING6(56=40+8+8 bytes) 2aXX:XXXX:XXXX:XXXX::1 --> 2001:4860:4860::8888

--- 2001:4860:4860::8888 ping6 statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
 
Provider says that if I still can access the gateway, my system seems to loose the route and if it was a problem in their infrastructure then I couldn't access the gateway at all.
 
That's a really bad analysis. If you can access the gateway routing is obviously not a problem on your end. The only way to lose a default route is when the interface goes down (which I presume doesn't happen). And just because you can access the gateway it doesn't mean you can access anything beyond that gateway. So their infrastucture could still be the problem (anything beyond the gateway is part of their infrastructure).
 
basbebe said:
Provider says that if I still can access the gateway, my system seems to loose the route and if it was a problem in their infrastructure then I couldn't access the gateway at all.
Click to expand...
Not from their POV. Presumable basbebe isn't the only one using the gateway, and the provider is monitoring both the gateway and its internet link. So from their PoV, the gateway is green and once basbebe gets there, he or she should be green as well.

Issues to look at
- are you really loosing that route?
- what about neigbour discovery?
- why run a router solicitation demon if you don't need it. What does it do?
- what default route did the provider give you? On default routes there is the fe80 school of thought, and the use the real address school. I admit preferring the latter, did you try that?

We do run a few older BSD servers with route solicitation, but we needed major testing to find a reliable config for all components involved. These days people use hard configured addresses and hsrp/vrrp. Or use real routing protocols fore more complex situations (quagga and ospfv3/ospf6 work just fine on FreeBSD) - but that would be overkill for a "just find the uplink" situation.
 
Sorry to bump an old thread, but I have the same problem and it's not been solved to this date. The only system that fails here is FreeBSD and it's well known by my hoster.

I think it has something to do when the router is `fe80::1`. We have this in common. The routing problem disappeared for some people by getting a fixed ("real") router address.

The problem is not that the network configuration is wrong (ok, you don't need rtsold at this point, but you probably just tried everything just to get it work). The problem is that the networking on IPv6 stops working occasionally and then comes back again. It's around 70% of the time offline for me.

My assumption is that FreeBSD does not support this kind of routing when the router address and the neighbor address do not correspond. It appears to me that `fe80::1` is some kind of virtual router address.

There are many hints that point at a missing feature in FreeBSD. The setup works for Linux and Windows as far as I am being told. I just got a free IPv6 address by hurricane electric to work around the issue, but it results in bad latency, of course.
 
You must log in or register to reply here.
Back
Top