Tips about root mirroring

S

Sam The Ripper

I wanted to boot my FreeBSD server from a mirror, but I wanted to mirror both ZFS pool and the EFI partition.

So I performed this kind of configuration:
Code: 
# gpart create -s gpt nvd0
# gpart create -s nvd1
# gpart add -t efi -s 200M nvd0
# gpart add -t efi -s 200M nvd1
# gpart add -t freebsd-zfs nvd0
# gpart add -t freebsd-zfs nvd1
# kldload geom_mirror
# gmirror label -v gm0 nvd0p1 nvd1p1
# newfs_msdos -F32 /dev/mirror/gm0
# zpool create rpool mirror /dev/nvd0p2 /dev/nvd1p2
[... classic manual FreeBSD installation ...]

The system is booting fine, but since I don't have any experience with GEOM mirror, I'd like to ask you if configuring gmirror for EFI alongside ZFS mirroring for the root pool is something reliable, or if there is a more smart method to achieve the same result.

Thank you
 
This forum is for howto guides not to ask questions about how to do things.

You can't mirror the EFI. The UEFI/BIOS doesn't have recognize and doesn't load gmirror so it can have only option to boot from single disk /file or if there's hardware Raid controller it can boot from it where it can have the mirror option.

It's enough to have copy of the ESP partition on both disk so if one of them die your system will still boot from the other disk.

So you can install the entire system normally it will create for you both ESP partitions on the both disks and during the install you have to select ZFS mirror and select both nvd0 and nvd1. After you complete the installation mount the ESP partition from nvd1 format as FAT32 and copy the loader.efi as /EFI/BOOT/BOOTx64.efi
 
VladiBG said:
This forum is for howto guides not to ask questions about how to do things.

You can't mirror the EFI. The UEFI/BIOS doesn't have recognize and doesn't load gmirror so it can have only option to boot from single disk /file or if there's hardware Raid controller it can boot from it where it can have the mirror option.

It's enough to have copy of the ESP partition on both disk so if one of them die your system will still boot from the other disk.

So you can install the entire system normally it will create for you both ESP partitions on the both disks and during the install you have to select ZFS mirror and select both nvd0 and nvd1. After you complete the installation mount the ESP partition from nvd1 format as FAT32 and copy the loader.efi as /EFI/BOOT/BOOTx64.efi
Click to expand...
Thank you for answering.

That said, sorry, but I’m not quite understanding your point about not being able to mirror an ESP.

Yes, the BIOS does not recognize GEOM, but that’s true for every software RAID out there.
BIOS can recognize only hardware RAID.
But this does not prevent to access to the single disk to read the boot loader and launch it, as GEOM mirroring is not tampering with the partition or the file system metadata, making the ESP unreadable from the EFI boot manager, because the RAID superblock is allocated to the end of the partition.

As I said, the system is currently booting.
I tested it both with two disks running and even with just one disk (emulating a degraded mirror), and it’s booting fine.
 
If I may, you likely shouldn't mirror the EFI. That way during an upgrade you can update your primary EFI boot, and afterward you can choose to upgrade your secondary EFI boot. If you hit a problem, you can reboot back to the secondary EFI that wasn't updated.
 
ZFS already does mirroring, so while a single ZFS partition on top of a gmirror gives "the same result" I'm not sure it's better.
As for mirroring EFI I have no opinion.
 
RussellASC said:
If I may, you likely shouldn't mirror the EFI. That way during an upgrade you can update your primary EFI boot, and afterward you can choose to upgrade your secondary EFI boot. If you hit a problem, you can reboot back to the secondary EFI that wasn't updated.
Click to expand...
Yeah, thinking about it, this could be a valid reason.

mer said:
ZFS already does mirroring, so while a single ZFS partition on top of a gmirror gives "the same result" I'm not sure it's better.
As for mirroring EFI I have no opinion.
Click to expand...
I'm not configuring a ZFS pool on top of a GEOM mirror.
I'm using GEOM mirror for the EFI partition and ZFS mirror for the root partition.
 
The only system I have with a gmirror BIOS boots. All I can say is that gmirror does a fine job of replicating the MBR. I believe what you're doing will work as you expect.

Maybe disconnect one of the disks as a test?
 
I use a mirrored EFI. See my HowTo for example. Many naysayers but wheres the beef?
forums.freebsd.org

Install UEFI FreeBSD on gmirror

I wanted to post instruction for installing FreeBSD on a GEOM_MIRROR -aka- gmirror(8). This is an advanced topic so I assume you are capable of determining that your two chosen disks are empty. UFS RAID1 on FreeBSD is enabled with the geom_mirror module. I am using ada0 and ada1 as examples...
forums.freebsd.org forums.freebsd.org

I think some of this comes down to EFI BIOS.
Some need exact disk to boot and some boot with a UEFI OS label/uid of mirror.

Jose said:
Maybe disconnect one of the disks as a test?
Click to expand...
On my firewall I use a cold spare. 3 disk mirror. Rotate spare through to keep disk wear even.. Small 16GB SSDs from wayyy back.
Instead of yanking the drive just try and boot off a single volume. It should prove itself.
 
Phishfry said:
Many naysayers but wheres the beef?
Click to expand...
My understanding is that Microsoft and Intel wrote the EFI standard, and, since Microsoft didn't support software mirrors, they assumed that nobody did.

The EFI boot mechanism allows for the EFI partition to be modified outside the knowledge of the operating system (prior to booting the O/S) using the EFI shell. So a hardware mirror (with a stand-alone RAID controller) keeps EFI mirrors in sync, but a software mirror, like GEOM, does not.

If you never modify your EFI partition using the EFI shell, I think that a GEOM mirror should be fine -- and this is why lots of people say it works. This is likely to work well in home systems, where one person is in charge.

[Ubuntu understands the EFI partition dilemma, and provides for multiple EFI partitions to be mounted and maintained, separately and automatically. This does not prevent EFI shell modifications to just one partition, but all of the other EFI partitions are kept current and in a usable state.]
 
Jose said:
The only system I have with a gmirror BIOS boots. All I can say is that gmirror does a fine job of replicating the MBR. I believe what you're doing will work as you expect.

Maybe disconnect one of the disks as a test?
Click to expand...
Yeah, I made exactly the same test to emulate a degraded mirror, and the system still booted properly.
 
Phishfry said:
I use a mirrored EFI. See my HowTo for example. Many naysayers but wheres the beef?
forums.freebsd.org

Install UEFI FreeBSD on gmirror

I wanted to post instruction for installing FreeBSD on a GEOM_MIRROR -aka- gmirror(8). This is an advanced topic so I assume you are capable of determining that your two chosen disks are empty. UFS RAID1 on FreeBSD is enabled with the geom_mirror module. I am using ada0 and ada1 as examples...
forums.freebsd.org forums.freebsd.org

I think some of this comes down to EFI BIOS.
Some need exact disk to boot and some boot with a UEFI OS label/uid of mirror.


On my firewall I use a cold spare. 3 disk mirror. Rotate spare through to keep disk wear even.. Small 16GB SSDs from wayyy back.
Instead of yanking the drive just try and boot off a single volume. It should prove itself.
Click to expand...
Yeah your tutorial was really helpful. I searched in the forum for similar scenarios and I found your thread.
Thank you for writing it.

The reason why I asked for opinions about the reliability of my configuration was because I saw that you, in your tutorial, made a mirror before partitioning the disk, where I did the opposite, because of the ZFS pool.
 
Phishfry said:
UEFI looks only for /EFI/BOOT/ directory not a particular drive.
Click to expand...

Now this information is not correct. I have learned much since that post.
EFI uses UUID of disk to boot.

efibootmgr hides it by calling disk0001, disk0002 ect.ect.

EDK2 UEFI shows you the actual UUIDs to boot from.

Which brings me to this point. Can't a gmirror have a UUID?

I think that is what happens. "UEFI OS" moniker in BIOS has the UUID of the gmirror.

Not positive one bit. But my first observation above from 2019 is incorrect.(I think!!)
 
I wonder when FreeBSD installer bsdinstall runs the efiootmgr portion of setup. I know it writes an entry during install.
Perhaps that entry to UEFI variables is what makes this work. I have not tried to manually install a gmirror to see.
 
At Stage 0 the UEFI firmware runs at power up and searches for the OS loader in the FAT32 filesystem on an EFI File System Partition (ESP). The UEFI ESP partition has C12A7328-F81F-11D2-BA4B-00A0C93EC93B UUID.
Click to expand...

So maybe I got it wrong. The EFI partition UUID is written to UEFI BIOS variables not the gmirror itself.
 
Phishfry Your usage example for gmirror is for entire disk with UFS which is OK. Here the OP is talking about gmirror only for the ESP partition + ZFS partitions with mirror.

In your example you are missing the swap.
 
Phishfry said:
I wonder when FreeBSD installer bsdinstall runs the efiootmgr portion of setup. I know it writes an entry during install.
Perhaps that entry to UEFI variables is what makes this work. I have not tried to manually install a gmirror to see.
Click to expand...
I wonder that too.
I avoided efibootmgr right because I didn’t want to write an entry in my bootloader.

I just installed the boot loader as /EFI/BOOT/BOOTX64.EFI, and I let the firmware loading it automatically.

I didn’t test a scenario that involves efibootmgr, because I think it is unnecessary complicated for my server, since I only want FreeBSD, and everything else will be virtualized/jailed.
 
Phishfry said:
Klara with the deep read.

klarasystems.com

The FreeBSD Boot Process - Klara Systems

Understand the FreeBSD boot process, including differences between UEFI and legacy BIOS, GPT and MBR partitioning, and filesystem options like ZFS and UFS.
klarasystems.com klarasystems.com
Click to expand...
BIOS still works pretty well for the FreeBSD boot process, but it’s a bit slower than UEFI, and doesn’t support booting from NVMe devices...
Click to expand...

Huh? This very machine BIOS boots Freebsd from an NVMe device. I must be imagining things.
 
Phishfry said:
I wonder when FreeBSD installer bsdinstall runs the efiootmgr portion of setup. I know it writes an entry during install.
Perhaps that entry to UEFI variables is what makes this work. I have not tried to manually install a gmirror to see.
Click to expand...
Here:
github.com

freebsd-src/usr.sbin/bsdinstall/scripts/bootconfig at master · freebsd/freebsd-src

The FreeBSD src tree publish-only repository. Experimenting with 'simple' pull requests.... - freebsd/freebsd-src
github.com github.com

Called from here:
github.com

freebsd-src/usr.sbin/bsdinstall/scripts/zfsboot at master · freebsd/freebsd-src

The FreeBSD src tree publish-only repository. Experimenting with 'simple' pull requests.... - freebsd/freebsd-src
github.com github.com
 
You must log in or register to reply here.
Back
Top