Solved Upgrading to 11.1-RELEASE, kernel/userland mismatch?

F

forquare

Morning all,

I upgraded work host to 11.1-RELEASE this morning using the following commands:
Code: 
beadm create 11.1-RELEASE_2017-05-26
beadm mount 11.1-RELEASE_2017-05-26 /mnt
freebsd-update upgrade -b /mnt -d /mnt/var/db/freebsd-update -r 11.1-RELEASE
freebsd-update install -b /mnt -d /mnt/var/db/freebsd-update
beadm umount 11.1-RELEASE_2017-05-26
beadm activate 11.1-RELEASE_2017-05-26
reboot

When my system came back up, I did the following:
Code: 
root@fbsd-bil:~ # freebsd-update install
src component not installed, skipped
No updates are available to install.
Run '/usr/sbin/freebsd-update fetch' first.
root@fbsd-bil:~ # freebsd-update fetch
src component not installed, skipped
Looking up update.FreeBSD.org mirrors... 4 mirrors found.
Fetching metadata signature for 11.1-RELEASE from update5.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

No updates needed to update system to 11.1-RELEASE-p0.
root@fbsd-bil:~ # freebsd-version -ku
11.1-RELEASE
11.0-RELEASE-p11

It seems that my userland didn't get upgraded. I'm not sure why, and I'm not sure how to proceed - though I think for the time being I will revert to my previous BE.

If anyone can shed light on this I'd be much appreciated :)

Ben
 
You need to run freebsd-update install a total of three times. The first time only updates the kernel, then you need to reboot. Then run it again to update the rest of the OS, reboot again. Then one final last time to clean up.
 
When I rebooted I did run freebsd-update install, but as you can see from my second lot of output:
Code: 
No updates are available to install.
Or am I being mislead by this message and it is actually doing something?
 
This directory is odd: /mnt/etc/var/db/freebsd-update. I get you're mounting the new file system on /mnt but the directory should be /mnt/var/db/freebsd-update, not /mnt/etc/var/db/freebsd-update. There is no /etc/var/ directory. So after the reboot it's trying to update from /var/db/freebsd-update but you stored everything in /etc/var/db/freebsd-update.
 
I beg your pardon, I'd typed the first set of commands separately into the post. Looking back at my history I did use /mnt/var/db/freebsd-update. I've updated my original post to correct this error.
 
It's been a while since I played with beadm. But I would just clone the current version, boot that and simply run the updates there (the 'old' version would still be available as a fall-back). It looks like freebsd-update(8) gets confused if you first do it 'remote' (with -d and -b) then switch to 'local'.

Another thing that came to mind, is /var/ separate? You may be running it off of the 'wrong' filesystem.
 
SirDice said:
It's been a while since I played with beadm. But I would just clone the current version, boot that and simply run the updates there.
Click to expand...

May do that later then :)

SirDice said:
Another thing that came to mind, is /var/ separate? You may be running it off of the 'wrong' filesystem.
Click to expand...

/var is a separate dataset, but it is not mounted, so /var/db is actually in my zroot/ROOT/default dataset:
Code: 
root@fbsd-bil:/var/db # zfs list -o name,mounted,mountpoint | grep var
zroot/var                                no  /var
zroot/var/audit                         yes  /var/audit
zroot/var/crash                         yes  /var/crash
zroot/var/log                           yes  /var/log
zroot/var/mail                          yes  /var/mail
zroot/var/tmp                           yes  /var/tmp

root@fbsd-bil:/var/db # pwd && df -h .
/var/db
Filesystem            Size    Used   Avail Capacity  Mounted on
zroot/ROOT/default     14G    1.5G     12G    11%    /

Thanks for your advice, SirDice
 
Right, but if you boot the new environment /var/ may be mounted again which would 'overlap' with whatever was in the original /var/ directory.
 
SirDice said:
Right, but if you boot the new environment /var/ may be mounted again which would 'overlap' with whatever was in the original /var/ directory.
Click to expand...

Very true. When I try again later I shall investigate :)

EDIT: Not had time to do an upgrade yet, but did manage to reboot into the BE I created and it certainly looks like /var/db/freebsd-update is coming from the new BE's dataset:
Code: 
root@fbsd-bil:/var/db # pwd && df -h .
/var/db
Filesystem            Size    Used   Avail Capacity  Mounted on
zroot/11.1-RELEASE_2017-07-26/default     14G    1.5G     12G    11%    /
 
On one of my system I've encountered the same issue.
I've just downgraded all things freebsd-update -r 11.0-RELEASE and after all I've started to upgrade again freebsd-update -r 11.01-RELEASE.
 
forquare said:
Morning all,

I upgraded work host to 11.1-RELEASE this morning using the following commands:
Code: 
beadm create 11.1-RELEASE_2017-05-26
beadm mount 11.1-RELEASE_2017-05-26 /mnt
freebsd-update upgrade -b /mnt -d /mnt/var/db/freebsd-update -r 11.1-RELEASE
freebsd-update install -b /mnt -d /mnt/var/db/freebsd-update
beadm umount 11.1-RELEASE_2017-05-26
beadm activate 11.1-RELEASE_2017-05-26
reboot

When my system came back up, I did the following:
Code: 
root@fbsd-bil:~ # freebsd-update install
src component not installed, skipped
No updates are available to install.
Run '/usr/sbin/freebsd-update fetch' first.
root@fbsd-bil:~ # freebsd-update fetch
src component not installed, skipped
Looking up update.FreeBSD.org mirrors... 4 mirrors found.
Fetching metadata signature for 11.1-RELEASE from update5.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

No updates needed to update system to 11.1-RELEASE-p0.
root@fbsd-bil:~ # freebsd-version -ku
11.1-RELEASE
11.0-RELEASE-p11

It seems that my userland didn't get upgraded. I'm not sure why, and I'm not sure how to proceed - though I think for the time being I will revert to my previous BE.

If anyone can shed light on this I'd be much appreciated :)

Ben
Click to expand...

Just tested this way of upgrading but ran
Code: 
freebsd-update install -b /mnt -d /mnt/var/db/freebsd-update
3 times in a row as always.
Everything worked just fine.

Code: 
odin@asgard ~ % freebsd-version -ku
11.1-RELEASE
11.1-RELEASE

Thanks for beadm way of upgrade! Really a killer protip to reduce downtime.
Cheers
 
pentago said:
Just tested this way of upgrading but ran
Code: 
freebsd-update install -b /mnt -d /mnt/var/db/freebsd-update
3 times in a row as always.
Everything worked just fine.
Click to expand...

Ah! So you did the freebsd-update install... three times before rebooting? Interesting!

pentago said:
Thanks for beadm way of upgrade! Really a killer protip to reduce downtime.
Click to expand...

This is how I learnt to do Solaris upgrades, the two biggest wins for boot environments was 1) easy backout if things went wrong; 2) minimal downtime, should only require a single reboot.
Being relatively novice at more advance FreeBSD operations (only used it for ~2 years) I'm still learning how things are done :)
 
SirDice said:
You need to run freebsd-update install a total of three times. The first time only updates the kernel, then you need to reboot. Then run it again to update the rest of the OS, reboot again. Then one final last time to clean up.
Click to expand...

Just had the thought that if common practise (without boot environments) is to:
  1. upgrade
  2. install
  3. reboot
  4. install
  5. reboot
  6. install
  7. reboot
Then by doing what I did above (upgrade, install, install, install; all inside my BE) there might be some issues. However I suppose because it's effectively a dormant system there is nothing using any of the files, and so this should be OK…?

I'd be interested to know people's thoughts.
 
It should be fine. If I recall correctly there's only ever been one occasion where you absolutely had to boot the new kernel first, before installing the rest of the world, or you would end up with a non-working system. This was with FreeBSD 5.0 and it was because of the UFS to UFS2 switch.
 
Good news :)

However, earlier past night, after I booted to fresh 11.1 system after this way of upgrading i noticed that I could do another, 4th

freebsd-update install


After noticing this I had no idea what was installed at that point so I checked my

freebsd-version


and it shows 11.0-p10.
At this, it came to me that ton of mismatches will definitelly going to happen for some reason so I rebooted and boot hung with bunch of crap in console. What's interesting is that after going to bootloader and choosing previous BE, boot hung again for some reason, probably more mismatches.

I had to boot to singleuser mode and activate another, older BE via beadm directly.
This time, I repeated upgrade process in traditional way, by rebooting between each freebsd-update install and reinstalling all my freshly built Poudriere packages.

Now its all good.

forquare if you want to test this behavior by yourself, make necessary precautions not to suffer like I did :)
 
pentago said:
Good news :)

However, earlier past night, after I booted to fresh 11.1 system after this way of upgrading i noticed that I could do another, 4th

freebsd-update install
Click to expand...

Interesting, how did you see that you could do a fourth? Did you just try? Or did you see a message somewhere?

pentago said:
forquare if you want to test this behavior by yourself, make necessary precautions not to suffer like I did :)
Click to expand...

Thank you for the warning. I've upgraded two hosts already using this method, one is my "everyday work" VM which I have a snapshot of from Monday, and daily snapshots held on another machine. I shall be cautious of any actions over the next few days.
 
forquare said:
Interesting, how did you see that you could do a fourth? Did you just try? Or did you see a message somewhere?



Thank you for the warning. I've upgraded two hosts already using this method, one is my "everyday work" VM which I have a snapshot of from Monday, and daily snapshots held on another machine. I shall be cautious of any actions over the next few days.
Click to expand...

Yeah, I just tried, had some weird hunch about it and it was correct, to my disappointment :D

On the other topic, I'm interested in learning more about ZFS recovery options on my disposal after failures like this.
For instance, I had access to my mfsBSD 11 ISO but didn't know how exactly to rollback previously made snapshots.

I booted it, did zpool import -o altroot=/mnt tank and at this point I was able to browse my files i /mnt.
Then I tried
Code: 
zfs rollback -r
for some of earlier snapshots but that didn't do what I expected it will.
Boot was still hung after that so obviously rollback from other system either doesn't work or I did it in a wrong way.

I'd be grateful to heavens if someone could teach me how to rollback ZFS snapshots from livecd's such as mfsBSD or official FreeBSD one.

Thanks in advance.
 
You must log in or register to reply here.
Back
Top