Help understanding crash logs

BaronBS · 2025-02-07T00:40:15+0000

Hello forum,

I had a unique crash today (never happened before).

The OS just froze and rebooted a few min latter. I was browsing with firefox open and a few others desktop apps running, nothing fancy.

Code:

doas cat /var/log/all.log
......
Feb  6 15:40:00 BattleStar-Lat5420 /usr/sbin/cron[17237]: (root) CMD (/usr/libexec/atrun)
Feb  6 15:44:00 BattleStar-Lat5420 /usr/sbin/cron[69307]: (operator) CMD (/usr/libexec/save-entropy)
Feb  6 15:45:00 BattleStar-Lat5420 /usr/sbin/cron[83532]: (root) CMD (/usr/libexec/atrun)
Feb  6 15:50:00 BattleStar-Lat5420 /usr/sbin/cron[50141]: (root) CMD (/usr/libexec/atrun)
Feb  6 15:55:00 BattleStar-Lat5420 /usr/sbin/cron[16853]: (root) CMD (/usr/libexec/atrun)
Feb  6 15:56:07 BattleStar-Lat5420 kernel: ugen0.2: <Microchip USB5806 Smart Hub> at usbus0 (disconnected)
Feb  6 15:56:07 BattleStar-Lat5420 kernel: uhub2: at uhub1, port 2, addr 1 (disconnected)
Feb  6 15:56:07 BattleStar-Lat5420 kernel: ugen0.3: <Realtek USB 10/100/1000 LAN> at usbus0 (disconnected)
Feb  6 15:56:07 BattleStar-Lat5420 kernel: ure0: at uhub2, port 3, addr 2 (disconnected)
Feb  6 15:56:07 BattleStar-Lat5420 kernel: rgephy0: detached
Feb  6 15:56:07 BattleStar-Lat5420 kernel: miibus0: detached
Feb  6 15:56:07 BattleStar-Lat5420 kernel: ure0: detached
Feb  6 15:59:15 BattleStar-Lat5420 syslogd: restart
Feb  6 15:59:15 BattleStar-Lat5420 syslogd: kernel boot file is /boot/kernel/kernel
Feb  6 15:59:15 BattleStar-Lat5420 kernel:
Feb  6 15:59:15 BattleStar-Lat5420 syslogd: last message repeated 1 times
Feb  6 15:59:15 BattleStar-Lat5420 kernel: Fatal trap 12: page fault while in kernel mode
Feb  6 15:59:15 BattleStar-Lat5420 kernel: cpuid = 4; apic id = 04
Feb  6 15:59:15 BattleStar-Lat5420 kernel: fault virtual address        = 0x458
Feb  6 15:59:15 BattleStar-Lat5420 kernel: fault code           = supervisor read data, page not present
Feb  6 15:59:15 BattleStar-Lat5420 kernel: instruction pointer  = 0x20:0xffffffff80b1ac29
Feb  6 15:59:15 BattleStar-Lat5420 kernel: stack pointer                = 0x28:0xfffffe0170f03760
Feb  6 15:59:15 BattleStar-Lat5420 kernel: frame pointer                = 0x28:0xfffffe0170f037e0
Feb  6 15:59:15 BattleStar-Lat5420 kernel: code segment         = base 0x0, limit 0xfffff, type 0x1b
Feb  6 15:59:15 BattleStar-Lat5420 kernel:                      = DPL 0, pres 1, long 1, def32 0, gran 1
Feb  6 15:59:15 BattleStar-Lat5420 kernel: processor eflags     = interrupt enabled, resume, IOPL = 0
Feb  6 15:59:15 BattleStar-Lat5420 kernel: current process              = 0 (netlink_socket (PID)
Feb  6 15:59:15 BattleStar-Lat5420 kernel: rdi: fffff8000b6882e0 rsi: 0000000000000004 rdx: 0000000000000000
Feb  6 15:59:15 BattleStar-Lat5420 kernel: rcx: 0000000000000001  r8: fffff803865ac520  r9: fffffe0170f04000
Feb  6 15:59:15 BattleStar-Lat5420 kernel: rax: 0000000000000000 rbx: 0000000000000012 rbp: fffffe0170f037e0
Feb  6 15:59:15 BattleStar-Lat5420 kernel: r10: 0000000000001388 r11: 0000000082d39ec6 r12: 0000000000000000
Feb  6 15:59:15 BattleStar-Lat5420 kernel: r13: fffff803865ac000 r14: fffffe0170f03788 r15: fffff8000b6882e0
Feb  6 15:59:15 BattleStar-Lat5420 kernel: trap number          = 12
Feb  6 15:59:15 BattleStar-Lat5420 kernel: panic: page fault
Feb  6 15:59:15 BattleStar-Lat5420 kernel: cpuid = 4
Feb  6 15:59:15 BattleStar-Lat5420 kernel: time = 1738853767
Feb  6 15:59:15 BattleStar-Lat5420 kernel: KDB: stack backtrace:
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #0 0xffffffff80b8d13d at kdb_backtrace+0x5d
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #1 0xffffffff80b3ef01 at vpanic+0x161
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #2 0xffffffff80b3ed93 at panic+0x43
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #3 0xffffffff8102cb1f at trap_pfault+0x3af
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #4 0xffffffff81003748 at calltrap+0x8
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #5 0xffffffff8094e778 at usbd_do_request_flags+0x8a8
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #6 0xffffffff8094e85e at usbd_do_request_proc+0x5e
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #7 0xffffffff849cf4b5 at ure_miibus_readreg+0xb5
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #8 0xffffffff806e8cd3 at rgephy_linkup+0xc3
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #9 0xffffffff806e8248 at rgephy_status+0x28
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #10 0xffffffff806e81d3 at rgephy_service+0x323
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #11 0xffffffff806e4177 at mii_pollstat+0x57
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #12 0xffffffff849d4924 at ure_ifmedia_sts+0x184
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #13 0xffffffff80c6e406 at ifmedia_ioctl+0x176
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #14 0xffffffff80d8c735 at dump_iface+0x145
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #15 0xffffffff80d8ed1c at dump_cb+0x1c
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #16 0xffffffff80c65077 at if_foreach_sleep+0x227
Feb  6 15:59:15 BattleStar-Lat5420 kernel: #17 0xffffffff80d8dc0d at rtnl_handle_getlink+0x24d
Feb  6 15:59:15 BattleStar-Lat5420 kernel: Uptime: 13h20m34s
Feb  6 15:59:15 BattleStar-Lat5420 kernel: Dumping 4888 out of 32272 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
Feb  6 15:59:15 BattleStar-Lat5420 kernel: Dump complete
Feb  6 15:59:15 BattleStar-Lat5420 kernel: Automatic reboot in 15 seconds - press a key on the console to abort
Feb  6 15:59:15 BattleStar-Lat5420 kernel: ---<<BOOT>>---
Feb  6 15:59:15 BattleStar-Lat5420 kernel: Copyright (c) 1992-2023 The FreeBSD Project.
Feb  6 15:59:15 BattleStar-Lat5420 kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Feb  6 15:59:15 BattleStar-Lat5420 kernel:      The Regents of the University of California. All rights reserved.
Feb  6 15:59:15 BattleStar-Lat5420 kernel: FreeBSD is a registered trademark of The FreeBSD Foundation.
Feb  6 15:59:15 BattleStar-Lat5420 kernel: FreeBSD 14.2-STABLE stable/14-n270273-3d30774f0056 GENERIC amd64
Feb  6 15:59:15 BattleStar-Lat5420 kernel: FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
.......

What do I need to look for here?

atax1a · 2025-02-07T00:54:30+0000

walking the stack trace backwards, it panicked when frame 5, usbd_do_request_flags took a page fault in kernel mode. going further up, frame 8 says rgephy_linkup and frame 12 says ure_ifmedia_sts, implicating the ure(4) driver. Ergo, you likely have a problem with your USB ethernet device. We'd start looking there, and also check memory, just to be sure.

BaronBS · 2025-02-07T01:01:38+0000

atax1a said:
walking the stack trace backwards, it panicked when frame 5, usbd_do_request_flags took a page fault in kernel mode. going further up, frame 8 says rgephy_linkup and frame 12 says ure_ifmedia_sts, implicating the ure(4) driver.

Can you elaborate on how did you arrived on that conclusion?

atax1a said:
Ergo, you likely have a problem with your USB ethernet device. We'd start looking there,

Like a usb dongle? If so, I don't have one.

atax1a said:
and also check memory, just to be sure.

How?

atax1a · 2025-02-07T01:31:57+0000

BaronBS said:
Can you elaborate on how did you arrived on that conclusion?

Reading the stack trace from the logs you posted, which forms a record of function calls that lead to the panic. the function names mention ure twice. here's the logs with running commentary.

Code:

kernel: #0 0xffffffff80b8d13d at kdb_backtrace+0x5d            We are here, printing the backtrace.
kernel: #1 0xffffffff80b3ef01 at vpanic+0x161                  Internals of panic
kernel: #2 0xffffffff80b3ed93 at panic+0x43                    panic!  the next call is what decided to panic
kernel: #3 0xffffffff8102cb1f at trap_pfault+0x3af             This is the page fault handler.
kernel: #4 0xffffffff81003748 at calltrap+0x8                  This is some kind of internal handler.
kernel: #5 0xffffffff8094e778 at usbd_do_request_flags+0x8a8   Here, we are processing a USB request
kernel: #6 0xffffffff8094e85e at usbd_do_request_proc+0x5e     Same here
kernel: #7 0xffffffff849cf4b5 at ure_miibus_readreg+0xb5       Trying to read from a `ure`
kernel: #8 0xffffffff806e8cd3 at rgephy_linkup+0xc3            while taking a link up, probably.
kernel: #9 0xffffffff806e8248 at rgephy_status+0x28            ...
kernel: #10 0xffffffff806e81d3 at rgephy_service+0x323         ...
kernel: #11 0xffffffff806e4177 at mii_pollstat+0x57            ...
kernel: #12 0xffffffff849d4924 at ure_ifmedia_sts+0x184        here's `ure` again, this time involved in a media query
kernel: #13 0xffffffff80c6e406 at ifmedia_ioctl+0x176          as called via ioctl()
kernel: #14 0xffffffff80d8c735 at dump_iface+0x145             and the rest of this just looks like a story of how we got here
kernel: #15 0xffffffff80d8ed1c at dump_cb+0x1c
kernel: #16 0xffffffff80c65077 at if_foreach_sleep+0x227
kernel: #17 0xffffffff80d8dc0d at rtnl_handle_getlink+0x24d

dunno what to say about not having a ure device, because that stack trace is visibly in the middle of trying to talk to some kind of USB device. the immediate prior logs also say that a USB hub and a ure device just detached.

As far as doing memory tests, we'd download the "Linux iso" from https://memtest.org and dd it onto a flash drive, reboot to it, and let it run at least one pass.

BaronBS · 2025-02-07T04:07:22+0000

atax1a said:
dunno what to say about not having a ure device, because that stack trace is visibly in the middle of trying to talk to some kind of USB device. the immediate prior logs also say that a USB hub and a ure device just detached.

Can USB mean something internal? because the only thing connected via usb is a nvme external storage device, it is a laptop, I don't need a usb ethernet dongle. wtf

atax1a said:
As far as doing memory tests, we'd download the "Linux iso" from https://memtest.org and dd it onto a flash drive, reboot to it, and let it run at least one pass.

Will do it.

And thank you very much for the commented log, it really helped me.

ralphbsz · 2025-02-07T04:32:13+0000

BaronBS said:
Can USB mean something internal?

Absolutely could. Run some commands to diagnose what USB hardware you have, for example usbconfig.

On "big" computers, USB-connected built in Ethernet ports seem unusual, but on small ones (like Raspberry Pi class) they are common. In between, for example NUCs, I don't know.

atax1a · 2025-02-07T04:43:50+0000

yeah, if you haven't plugged one in, it's likely that your motherboard ethernet device is the ure.

BaronBS said:
kernel: ugen0.2: <Microchip USB5806 Smart Hub> at usbus0 (disconnected)
kernel: uhub2: at uhub1, port 2, addr 1 (disconnected)
kernel: ugen0.3: <Realtek USB 10/100/1000 LAN> at usbus0 (disconnected)
kernel: ure0: at uhub2, port 3, addr 2 (disconnected)
kernel: rgephy0: detached
kernel: miibus0: detached
kernel: ure0: detached

on the one hand, your hub and ethernet device shouldn't be falling off the bus, but their reattachment also shouldn't panic the system.

ralphbsz · 2025-02-07T06:37:13+0000

atax1a said:
... but their reattachment also shouldn't panic the system.

I think this bears repeating: A kernel panic is always either a hardware fault (typically a memory problem), or a bug in the kernel. If both hardware and OS were perfect, a user would not be able to cause a panic. Alas ...

fmc000 · 2025-02-07T07:55:04+0000

Feb 6 15:59:15 BattleStar-Lat5420 kernel: #17 0xffffffff80d8dc0d at rtnl_handle_getlink+0x24d

Did you have a link flap or something?