ZFS ZFS Partition vs Whole Disk and Bootcode

[dave]

I have an old machine that runs on ZFS. I have just replaced one of the disks in a mirror. I used the whole disk, but the existing disk looks like it's partitioned.

I replaced thie disk with a command like,

sudo zfs replace zroot [id of old device] ada1

When I replaced the disk, I got a message like this:

This is what my `zpool status` looks like:

  pool: zroot
 state: ONLINE
  scan: resilvered 14.3G in 0 days 00:05:42 with 0 errors on Sun Jan 26 15:30:33 2020
config:

    NAME        STATE     READ WRITE CKSUM
    zroot       ONLINE       0     0     0
      mirror-0  ONLINE       0     0     0
        ada1    ONLINE       0     0     0
        ada0p3  ONLINE       0     0     0

errors: No known data errors

Now I'm confused about what command to use to fix the bootcode.

Secondly, is there a way I can now replace the partitioned drive while remaining online? I.e. can I just dd zeros onto it and then replace with the same whole disk? Or does it even matter?

 

[Emrion]

Normally, the idea behind such a mirror is to be able to boot the system even if one of the disks fails.
You shouldn't use the whole disk as mirror of a partition because it can't boot your system.

What you'd better do is:
- Remove ada1 from the mirror and wipe the disk (zeroing totally with dd).
- Copy partition scheme: gpart backup ada0 | gpart restore ada1.
- If the disk ada1 is bigger than ada0, you probably need to run: gpart recover ada1. Nothing to do if disks have the same size.
- Note where your freebsd-boot partition is (should be the first one in your case but you have to verify and note the number n).
- Copy boot codes: gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i n ada1
- Add the partition to the mirror: zpool attach zroot ada0p3 ada1p3
- Wait for the resilvering.

I hope I didn't forget something.

 

[memreflect]

Normally, the idea behing such a mirror is to be able to boot the system even if one of the disks fails.
You shouldn't use the whole disk as mirror of a partition because it can't boot your system.

What you'd better do is:
- Remove ada1 from the mirror and wipe the disk (zeroing totally with dd).

Do you really need to use a potentially time-consuming dd for this? For example, have a look at bsdinstall's zfsboot script used for 12.1 (I changed the command references to actual commands, removed the informational messages, and specifically tailored it to use ada1, but that's all):

gpart destroy -F "ada1"
graid delete "ada1"
zpool labelclear -f "/dev/ada1"
gpart create -s gpt "ada1"
gpart destroy -F "ada1"

Much faster than using dd to zero the drive!

- Copy partition scheme: gpart backup ada0 | gpart restore ada1.
- If the disk ada1 is bigger than ada0, you probably need to run: gpart recover ada1. Nothing to do if disks have the same size.
- Note where your freebsd-boot partition is (should be the first one in your case but you have to verify and note the number n).
- Copy boot codes: gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i n ada1
- Add the partition to the mirror: zpool attach zroot ada0p3 ada1p3
- Wait for the resilvering.

I hope I didn't forget something.

Looks good to me, assuming the "old machine" isn't booting in UEFI mode. If UEFI mode is used, the EFI System Partition (ESP) needs duplicated anytime after using gpart recover ada1, which can be done using dd as usual. Things may change in a later release, but for right now, I'm mostly certain that should be all you need to do; if you have trouble booting from the volume, you can try something like the following, where $esp is the duplicated ESP that exists on ada1:

mount_msdosfs /dev/ada1p$esp &&
efibootmgr --create --label "FreeBSD (ada1)" --loader /mnt/EFI/BOOT/BOOTX64.EFI &&
efibootmgr --activate $(efibootmgr | grep "FreeBSD (ada1)" | awk '{print $1}' | tr -dc '[[:digit:]]') &&
umount /mnt

You may also want to fix the boot order that has now been changed using efibootmgr once more. See efibootmgr(8) and uefi(8) for more information.

 

[dave]

Thanks everyone for the advice, this is really helpful! In my case, no UEFI. Emrion's commands worked perfectly, and memreflect was correct about how to quickly "reset" the new drive.

Man ZFS rules. What was Linus talking about?!?

 

[Emrion]

gpart destroy -F "ada1"
graid delete "ada1"
zpool labelclear -f "/dev/ada1"
gpart create -s gpt "ada1"
gpart destroy -F "ada1"

It's maybe what I missed when I once decided to zeroing my disk for a dark story of remaining vdev label. I mean zpool labelclear -f /dev/ada1. Thanks.