Solved gptzfsboot error 128 after adding new disks

[Aayla Secura]

I'm running 11.1-RELEASE (I think, will double check when I have access to machine) with zfs as the root system in mirror-0 configuration. There are three SSDs as part of the mirror that were partitioned automatically during installation: each has three partitions labelled gptboot<n>, freebsd-swap<n>, freebsd-zfs<n>.

In addition, there is one other SSD with a single UFS partition, used for storage. I later added two more SSDs with the intention of using them as additional storage (single UFS partition each). When I did that FreeBSD wouldn't boot:

gptzfsboot: error 128 lba <some sector number>

I thought that the partitions got renumbered (despite plugging in the new SSDs in the highest SATA port numbers), so I booted from a rescue CD, I enabled kern.geom.label.gptid.enable="1" in loader.conf, I imported the zroot zfs pool specifying the device directory as /etc/gptid and replaced /boot/zfs/zpool.cache with the newly created cache from the rescue CD. I confirmed the filesystem names as given by zpool status were now the GPT UIDs rather than the disk-partition numbers before exporting the zpool and rebooting. Same error during boot.

If I simply unplug the SSDs it boots fine. I'm starting to think it has nothing to do with the zfs pool. What's going on?

EDIT:
So after a lot of experimenting I figured out what the problem was. It looks like FreeBSD was probing all disks and was throwing the error when one of them didn't have a valid partition table. Don't know why it would do that rather than ignore it, as the Live CD does. All I had to do was boot the Live CD and format the new disks, then my installation booted happily.

 

[p.r.]

I have the same problem (tried both freeNAS [11.2-U4 though 11.3-RC2] and freeBSD 12.1), however nothing helps to get rid of the gptzfsboot: error 128 lba errors. The system boots eventually but the boot takes cca N x 7 minutes (there are several gptzfsboot: error 128 lba some_sector_number lines per disk and each takes some time), where N is the number of data disks where gptzfsboot is trying to analyze.

Is there a way to tell gptzfsboot NOT to analyze other disks?

According to gptzfsboot manpage:
"gptzfsboot looks for ZFS device labels on all visible disks and in discovered supported partitions for all supported partition scheme types. The search starts with the disk from which gptzfsboot itself was loaded. Other disks are probed in BIOS defined order. After a disk is probed and gptzfsboot determines that the whole disk is not a ZFS pool member, the individual partitions are probed in their partition table order. Currently GPT and MBR partition schemes are supported. With the GPT scheme, only partitions of type freebsd-zfs are probed. The first pool seen during probing is used as a default boot pool.
The filesystem specified by the bootfs property of the pool is used as a default boot filesystem. If the bootfs property is not set, then the root filesystem of the pool is used as the default".

From the above I assume omitting the phase where gptzfsboot tries to probe other disks would solve the problem. (The installer CD boots the system just fine. Also, the system works just fine, including the data disks, once it has booted).

Anyway, should this be reported as a bug?