Solved Upgrade from 12.0 to 12.1 results in kernel panic

R

Relictrix

Hi,

When upgrading from 12.0 to 12.1, the upgrade itself does not return any errors. When rebooting the system, the startup with new kernel results in a kernel panic.

* What are the next steps to see in the logs ? I cannot startup so what's the procedure to read the logs?
* I tried to restart the old kernel but that results in going to the mountroot prompt.
* Why does the server end in the mountroot prompt?
* What does he need to know from the mountroot?

Code: 
Mounting from zfs:interno/ROOT/default failed with error 5

Loader variables:
  vfs.root.mountfrom=zfs:interno/ROOT/default

Manual root filesystem specification:
  <fstype>:<device> [option]
      Mount <device> using filesystem <fstype> and with the specified (optional) option list.

    eg. ufs:/dev/da0s1a
        zfs:tank
        cd9660:/dev/cd0 ro

?                      List valid disk bootdevices
.                       Yield 1 second (for background tasks)
<empty line>  Abort manual input

mountroot>

Best Regards,

R
 
Hi Tingo,

Good idea, I did not. Just checked it, but I don't think my error is specifically mentioned in the Errata. Unless i overlooked.

For your information, as I thought perhaps the upgrade failed. I rolled back to snapshot and gave it another shot. Of course after doing the upgrade again, it's exactely the same error. I now doubt that it was error 5 or 45. But at the end I come back to mountroot. I must say I don't really know what's the purpose of mountroot. I read some other threads here related to all kind of mountroot issues. But some people then give statements to execute, but not why. That's why i decided to make a new thread. My experience with mountroot is low , very low. I never used it before :). But the more trouble the more I learn. And I don't just want to try things on that machine, i still need that data.
 
Relictrix said:
Just checked it, but I don't think my error is specifically mentioned in the Errata. Unless i overlooked.
Click to expand...
I'm going to bet it's this:
[2019-11-04] A late issue was discovered where systems using the graphics/drm-kmod port built on FreeBSD 12.0-RELEASE will crash during boot. It is advised to remove the port prior to upgrading, and build the port instead of using the upstream binary package.
Click to expand...
 
Hi SirDice,

I will check that one soon , don't remember by heart if I installed it.

As your approach to finding this issue you really base yourself to the errata. I mean is there a way to find the error from the logs. I cannot open the 12.1 kernel, old kernel gives mountroot. Perhaps i could try to open the pool readonly and check some log. Or you got another, a better approach?

Best Regards,

R
 
It's a common problem with users that have additional kernel modules installed. The packages in the repositories are still being built for 12.0. For 99% of the packages it's not a problem installing those on 12.1. Notable exceptions however are kernel modules, these will often cause the system to crash during boot.

This is one of the kernel modules that's been giving lots of users grief after they upgraded their systems. So it's my first guess it's been causing problems for you too.
 
Ok i see, I just noticed that i read the release notes not the errata. I want to go to 12.1, I will check the coming days and will respond back the result here. I already rolledback, will make a snapshot + backup externally. Then I will try again.
 
Another common cause for crashes during boot after an upgrade (besides the aforementioned drm-kmod kernel module) are the VirtualBox kernel modules. So before you attempt the upgrade again make sure to disable any (third-party) kernel modules to avoid any potential issues. Kernel modules that come with the OS itself are not a problem.
 
Hi, the port graphics/drm-kmod was there, I deleted it, and run the upgrade. Will update here when the upgrade has finished.
 
Hi,

The upgrade finished after reboot i still have a kernel error. I deleted the port with pkg delete graphics/drm-kmod and also from the /usr/ports/graphics/drm-kmod afterwards I did : make deinstall clean. This resulted in no port found. After reboot I still have a kernel panic and a auto reboot happens.

After restarting the pool in read only mode by starting the server in Single Boot mode via a USB disk, I could not see any files in /var/log or /var/crash. I wanted to search in detail for the error. Seems this got something to do with the mount, i mean normally there should be files there.

Just let me know if anybody has any thoughts, after a while I will probably rollback everything again because I need the server.

Best Regards,

R
 
Relictrix said:
The upgrade finished after reboot i still have a kernel error. I deleted the port with pkg delete graphics/drm-kmod and also from the /usr/ports/graphics/drm-kmod afterwards I did : make deinstall clean. This resulted in no port found. After reboot I still have a kernel panic and a auto reboot happens.
Click to expand...
Do a pkg autoremove, graphics/drm-kmod is a so-called meta-port, it doesn't install anything of its own and pulls in one of the other drm-*-kmod ports/packages. And it's that dependency that actually has the kernel module.

Alternatively, just remove the kld_list from rc.conf. Or from loader.conf if you used that.
 
Hi,

I did that still does not work. Perhaps I have to be more drastic and should delete a lot more ports?

Best Regards,

R
 
My pool is ok if i rollback to snapshot. After the upgrade I cannot open my pool. I did one more time, this time with
Code: 
pkg delete -a
followed by a
Code: 
pkg autoremove
. Result is still an error.

More details about the error are: fatal trap 12: page fault while in kernel mode.
 
Last edited:
Hi, I can check my /etc/rc.conf next time I rollback again. But from what i remember, I don't have anything special in there. It's more or less a standard empty rc.conf.
 
This is my rc.conf

Code: 
root@nuc1:~ # cat /etc/rc.conf
hostname="nuc1"
keymap="be.iso.kbd"
#ifconfig_em0="DHCP"
ifconfig_em0="DHCP"
#ifconfig_em0="inet 192.168.1.10 netmask 255.255.255.0"
#ifconfig_em0="inet 169.254.123.10 netmask 255.255.0.0"
defaultrouter=""
sshd_enable="YES"
ntpd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"

#firewall
ipfilter_enable="YES"
ipfilter_rules="/etc/ipf.rules"
ipmon_enable="YES"
ipmon_flags="-Ds"

#kde
#dbus_enable="YES"
#hald_enable="YES"
#sddm_enable="YES"

#xrdp_enable="YES"

#gnome
#dbus_enable="YES"
#hald_enable="YES"
#gdm_enable="YES"
#gnome_enable="YES"

#xfce
dbus_enable="YES"


#veracrypt

#virtualization
#vm_enable="YES"
#vm_dir="zfs:zroot/vms"
#vm_list="myubuntu"
#vm_delay="5"
#cloned_interfaces="bridge0 tap0"
#ifconfig_bridge0="addm em0 addm tap0"

#virtualization virtualbox
vboxnet_enable="YES"

#wireless
wlans_iwm0="wlan0"
ifconfig_wlan0="WPA SYNCDHCP"
 
Hi,

Just tried it. Commented the vboxnet_enable line and included the zfs_enable. Still the same kernel panic after upgrading from 12.0 to 12.1 :)

Was a good attempt , I thought you found it.

Best Regards,

R
 
Hi,

Just did another attempt. This time started from a minimal /etc/rc.conf and from a minimal /boot/loader.conf.

The result is again a kernel panic.

Best Regards,

R
 
Hi,

Taking a picture the error is this:
Fatal trap 12: page fault while in kernel mode

Before the fatal trap i see the ipfilter is enabling, worth to disable this as well?
ipfilter_enable="YES"
ipfilter_rules="/etc/ipf.rules"
ipmon_enable="YES"
ipmon_flags="-Ds"

Best Regards,
 
I would disable almost everything and gradually enable the services ... unfortunately costs time because of lots of reboots, but maybe you will find the problem.
 
Hi,

Finally! ipfilter was the root cause! After disabling that the upgrade worked.

Thanks rootbert for pointing out the zfs_enable="YES". I had to mount some points manually and while looking into detail setting on the mount itself: legacy and stuff, I saw the zfs_enable did the job.

This thread can be marked as solved.
 
You must log in or register to reply here.
Back
Top