ZFS Catching zpools before mounted during startup

I would like to be able to run a script at startup that can modify ZFS filesystem properties on "non-root" filesystems prior to the zfs mount -a in the normal boot sequence. The application is a "fail-safe" for backup of root filesystems from other systems that might not have already been marked canmount=noauto.

Examining the output of rcorder suggested that between zpool which runs zpool import -c $cachefile -a -N and zfs which runs zfs mount -va would be appropriate. My expectation was that only the critical root filesystem would be mounted.

Dumping the output of mount to a file shows that most everything is mounted. Watching the screen (no Scroll Lock key on my keyboards) suggests that my script is running after Mounting local filesystems:, even with # BEFORE: zfs mountcritlocal

I'm puzzled as to how to get my script to run after the pools are imported but before they are mounted.

Bonus points for either or both a cleaner way to log output from "early" scripts and how to watch the console output during boot without a Scroll Lock key. As those are arguably off-topic for the sub-forum, let me know if you answer in another thread.


Ordering puzzle resolved in #2, below


While the documentation isn't very clear on this, the script has to be in /etc/rc.d/, not /usr/local/etc/rc.d/

Even if /usr/local/etc/rc.d/ is mounted during early boot, /etc/rc will not look there until after the early-late divider, usually FILESYSTEMS

/usr/local/etc/rc.d/zfs_defang

sh:
#!/bin/sh
#
#

# PROVIDE: zfs_defang
# REQUIRE: zpool root
# BEFORE: zfs mountcritlocal

. /etc/rc.subr

name="zfs_defang"
desc="Override canmount=on for backup pools"
rcvar="zfs_defang_enable"
required_modules="zfs"

start_cmd="zfs_defang_run"

load_rc_config $name

: ${zfs_defang_enable:=NO}
: ${zfs_defang_filesystems:=}  # list of filesystems to process
: ${zfs_defang_hostid:=$hostname}
: ${zfs_defang_hostid:=$(sysctl -n kern.hostuuid)}


zfs_defang_run()
{
    startmsg
    info "hostid is |$zfs_defang_hostid|"
    if [ $PPID -eq 1 ] ; then
    (
        echo
        date
        rcorder /etc/rc.d/* /usr/local/etc/rc.d/zfs_defang | head -n 50
        egrep "^# (PROVIDE|REQUIRE|BEFORE|KEYWORD):" /usr/local/etc/rc.d/zfs_defang
        echo "hostid is |$zfs_defang_hostid|"
        mount
    ) >> /root/defang.log
    fi
}


run_rc_command "$1"

Code:
Tue Feb 18 12:26:40 PST 2025
/etc/rc.d/natd
/etc/rc.d/dhclient
/etc/rc.d/sysctl
/etc/rc.d/dnctl
/etc/rc.d/dumpon
/etc/rc.d/ddb
/etc/rc.d/hostid
/etc/rc.d/ccd
/etc/rc.d/geli
/etc/rc.d/gbde
/etc/rc.d/swap
/etc/rc.d/zpool
/etc/rc.d/zpoolupgrade
/etc/rc.d/zfskeys
/etc/rc.d/fsck
/etc/rc.d/zpoolreguid
/etc/rc.d/zvol
/etc/rc.d/growfs
/etc/rc.d/root
/etc/rc.d/growfs_fstab
/etc/rc.d/serial
/etc/rc.d/mdconfig
/etc/rc.d/hostid_save
/usr/local/etc/rc.d/zfs_defang
/etc/rc.d/mountcritlocal
/etc/rc.d/var_run
/etc/rc.d/zfsbe
/etc/rc.d/tmp
/etc/rc.d/kldxref
/etc/rc.d/zfs
/etc/rc.d/sysvipc
/etc/rc.d/kld
/etc/rc.d/devmatch
/etc/rc.d/var
/etc/rc.d/linux
/etc/rc.d/localpkg
/etc/rc.d/nuageinit
/etc/rc.d/cfumass
/etc/rc.d/cleanvar
/etc/rc.d/FILESYSTEMS
/etc/rc.d/ipsec
/etc/rc.d/rctl
/etc/rc.d/geli2
/etc/rc.d/autounmountd
/etc/rc.d/adjkerntz
/etc/rc.d/hostname
/etc/rc.d/ip6addrctl
/etc/rc.d/ippool
/etc/rc.d/netoptions
/etc/rc.d/opensm
# PROVIDE: zfs_defang
# REQUIRE: zpool root
# BEFORE: zfs mountcritlocal
hostid is |<redacted>|
backup-host-2025/ROOT/default on / (zfs, local, noatime, nfsv4acls)
devfs on /dev (devfs)
/dev/gpt/efiboot0 on /boot/efi (msdosfs, local)
backup-host-2025 on /zroot (zfs, local, noatime, nfsv4acls)
backup-host-2025/var/log on /var/log (zfs, local, noatime, noexec, nosuid, nfsv4acls)
backup-host-2025/tmp on /tmp (zfs, local, noatime, nosuid, nfsv4acls)
backup-host-2025/home on /home (zfs, local, noatime, nfsv4acls)
backup-host-2025/var/mail on /var/mail (zfs, local, nfsv4acls)
backup-host-2025/var/audit on /var/audit (zfs, local, noatime, noexec, nosuid, nfsv4acls)
backup-host-2025/var/crash on /var/crash (zfs, local, noatime, noexec, nosuid, nfsv4acls)
backup-host-2025/usr/ports on /usr/ports (zfs, local, noatime, nosuid, nfsv4acls)
backup-host-2025/usr/src on /usr/src (zfs, local, noatime, nfsv4acls)
x9-xfer/front-h2/front-h2-boot/BOOT/debian on /boot (zfs, local, nfsv4acls)
backup-2025 on /backup-2025 (zfs, local, nfsv4acls)
backup-host-2025/var/tmp on /var/tmp (zfs, local, noatime, nosuid, nfsv4acls)

[...]
 
Maybe a red herring, but it looks like Root mount waiting for: usbus0 may be part of the puzzle. The root filesystem is on NVMe.

The screenshot suggests my code is running after
  • hostid_save
  • fsck
  • mountcritical
  • devmatch
Pasted image 20250218134020.jpg



boottrace confirms this. Working through "why?"

Code:
CPU      msecs      delta process                  event                                      PID CPUtime IBlks OBlks
  0      18032          0 kernel                   sysinit 0x2100001                            0    0.00     0     0
  0      18032          0 kernel                   sysinit 0x2110000                            0    0.00     0     0
  0      18032          0 kernel                   sysinit 0x2140000                            0    0.00     0     0
  0      18032          0 kernel                   sysinit 0x2160000                            0    0.00     0     0
  0      18032          0 kernel                   sysinit 0x2180000                            0    0.00     0     0
  0      18032          0 kernel                   sysinit 0x21d0000                            0    0.00     0     0
  0      18032          0 kernel                   sysinit 0x21e0000                            0    0.00     0     0
  0      18032          0 kernel                   sysinit 0x2200000                            0    0.00     0     0
  0      18032          0 kernel                   sysinit 0x2300000                            0    0.00     0     0
  0      18033          1 kernel                   sysinit 0x2400000                            0    0.00     0     0
  0      18033          0 kernel                   sysinit 0x2480000                            0    0.00     0     0
  0      18033          0 kernel                   sysinit 0x24c0000                            0    0.00     0     0
  0      18033          0 kernel                   sysinit 0x2500000                            0    0.00     0     0
  0      18033          0 kernel                   sysinit 0x2600000                            0    0.00     0     0
  0      18033          0 kernel                   sysinit 0x2700000                            0    0.00     0     0
  0      18033          0 kernel                   sysinit 0x2800000                            0    0.00     0     0
  0      18034          1 kernel                   sysinit 0x2880000                            0    0.00     0     0
  0      18034          0 kernel                   sysinit 0x2888000                            0    0.00     0     0
  0      18034          0 kernel                   sysinit 0x28fffff                            0    0.00     0     0
  0      18034          0 kernel                   sysinit 0x2900000                            0    0.00     0     0
  0      18039          5 kernel                   sysinit 0x2900001                            0    0.00     0     0
  0      18039          0 kernel                   sysinit 0x2a00000                            0    0.00     0     0
  0      18039          0 kernel                   sysinit 0x2f00000                            0    0.00     0     0
  0      18039          0 kernel                   sysinit 0x3000000                            0    0.00     0     0
  0      18039          0 kernel                   sysinit 0x3100000                            0    0.00     0     0
  0      18041          2 kernel                   sysinit 0x3800000                            0    0.00     0     0
  3      23141       5100 kernel                   sysinit 0x4000000                            0    0.00     0     0
  3      23144          3 kernel                   sysinit 0x4800000                            0    0.00     0     0
  3      24222       1078 kernel                   sysinit 0x6400000                            0    0.00     0     0
  3      24222          0 kernel                   sysinit 0x6800000                            0    0.00     0     0
  3      24222          0 kernel                   sysinit 0x6c00000                            0    0.00     0     0
  3      24222          0 kernel                   sysinit 0x6e00000                            0    0.00     0     0
  3      24222          0 kernel                   sysinit 0x7000000                            0    0.00     0     0
  3      24222          0 kernel                   sysinit 0x7400000                            0    0.00     0     0
  3      24222          0 kernel                   sysinit 0x8100000                            0    0.00     0     0
  3      24222          0 kernel                   sysinit 0x8400000                            0    0.00     0     0
  3      24222          0 kernel                   sysinit 0x8600000                            0    0.00     0     0
  3      24222          0 kernel                   sysinit 0x8700000                            0    0.00     0     0
  3      24222          0 kernel                   sysinit 0x8800000                            0    0.00     0     0
  3      24223          1 kernel                   sysinit 0x8808000                            0    0.00     0     0
  3      24223          0 kernel                   sysinit 0xa000000                            0    0.00     0     0
  3      24223          0 kernel                   sysinit 0xa800000                            0    0.00     0     0
  0      24474        251 kernel                   sysinit 0xaffffff                            0    0.00     0     0
  0      24482          8 kernel                   sysinit 0xb000000                            0    0.00     0     0
  0      24482          0 kernel                   sysinit 0xd000000                            0    0.00     0     0
  0      24482          0 kernel                   sysinit 0xd800000                            0    0.00     0     0
  0      24482          0 kernel                   sysinit 0xdc00000                            0    0.00     0     0
  0      24482          0 kernel                   sysinit 0xdffff9c                            0    0.00     0     0
  0      24482          0 kernel                   sysinit 0xe000000                            0    0.00     0     0
  0      24482          0 kernel                   sysinit 0xe400000                            0    0.00     0     0
  0      24482          0 kernel                   sysinit 0xe800000                            0    0.00     0     0
  0      24482          0 kernel                   sysinit 0xea00000                            0    0.00     0     0
  0      24482          0 kernel                   sysinit 0xec00000                            0    0.00     0     0
  0      24482          0 kernel                   sysinit 0xee00000                            0    0.00     0     0
  0      24482          0 kernel                   sysinit 0xf100000                            0    0.00     0     0
  0      24482          0 kernel                   sysinit 0xfffffff                            0    0.00     0     0
  0      24482          0 swapper                  mi_startup done                              0    0.00     0     0
  2      29606       5124 init                     init(8) starting...                          1    0.00     0     0
  2      29606          0 init                     /etc/rc starting...                          1    0.00     0     0
  3      29643         37 boottrace                /etc/rc.d/sysctl start                      29    0.00     0     0
  0      29653         10 boottrace                /etc/rc.d/sysctl done                       29    0.00    11     0
  0      29653          0 boottrace                /etc/rc.d/dumpon start                     290    0.00     0     0
  2      29664         11 boottrace                /etc/rc.d/dumpon done                      290    0.00    42     0
  2      29664          0 boottrace                /etc/rc.d/dnctl start                     2235    0.00     0     0
  1      29670          6 boottrace                /etc/rc.d/dnctl done                      2235    0.00     0     0
  1      29670          0 boottrace                /etc/rc.d/hostid start                    4983    0.00     0     0
  1      29678          8 boottrace                /etc/rc.d/hostid done                     4983    0.00     7     0
  0      29678          0 boottrace                /etc/rc.d/ddb start                       9216    0.00     0     0
  2      29684          6 boottrace                /etc/rc.d/ddb done                        9216    0.00     0     0
  2      29684          0 boottrace                /etc/rc.d/gbde start                     11755    0.00     0     0
  2      29688          4 boottrace                /etc/rc.d/gbde done                      11755    0.00     0     0
  2      29689          1 boottrace                /etc/rc.d/ccd start                      12921    0.00     0     0
  2      29692          3 boottrace                /etc/rc.d/ccd done                       12921    0.00     0     0
  2      29693          1 boottrace                /etc/rc.d/geli start                     14782    0.00     0     0
  2      29697          4 boottrace                /etc/rc.d/geli done                      14782    0.00     0     0
  2      29697          0 boottrace                /etc/rc.d/swap start                     16906    0.00     0     0
  2      29702          5 boottrace                /etc/rc.d/swap done                      16906    0.00     2     0
  2      29702          0 boottrace                /etc/rc.d/zpool start                    17571    0.00     0     0
  1      32288       2586 boottrace                /etc/rc.d/zpool done                     17571    0.66   360     0
  0      32288          0 boottrace                /etc/rc.d/zfskeys start                  20093    0.00     0     0
  0      32292          4 boottrace                /etc/rc.d/zfskeys done                   20093    0.00     0     0
  1      32292          0 boottrace                /etc/rc.d/fsck start                     22387    0.00     0     0
  1      32295          3 boottrace                /etc/rc.d/fsck done                      22387    0.00     0     0
  0      32296          1 boottrace                /etc/rc.d/zvol start                     23015    0.00     0     0
  0      33345       1049 boottrace                /etc/rc.d/zvol done                      23015    0.88   437     0
  1      33345          0 boottrace                /etc/rc.d/root start                     25451    0.00     0     0
  2      33352          7 boottrace                /etc/rc.d/root done                      25451    0.00     5     0
  3      33352          0 boottrace                /etc/rc.d/serial start                   29270    0.00     0     0
  3      33353          1 boottrace                /etc/rc.d/serial done                    29270    0.00     0     0
  3      33353          0 boottrace                /etc/rc.d/mdconfig start                 30202    0.00     0     0
  1      33358          5 boottrace                /etc/rc.d/mdconfig done                  30202    0.00     0     0
  0      33358          0 boottrace                /etc/rc.d/hostid_save start              33081    0.00     0     0
  0      33362          4 boottrace                /etc/rc.d/hostid_save done               33081    0.00     0     0
  0      33362          0 boottrace                /etc/rc.d/mountcritlocal start           34691    0.00     0     0
  3      33368          6 boottrace                /etc/rc.d/mountcritlocal done            34691    0.00    12     1
  2      33368          0 boottrace                /etc/rc.d/tmp start                      36289    0.00     0     0
  1      33374          6 boottrace                /etc/rc.d/tmp done                       36289    0.00    11     0
  1      33375          1 boottrace                /etc/rc.d/kldxref start                  39376    0.00     0     0
  1      33379          4 boottrace                /etc/rc.d/kldxref done                   39376    0.00     6     0
  1      33379          0 boottrace                /etc/rc.d/zfsbe start                    41520    0.00     0     0
  3      33387          8 boottrace                /etc/rc.d/zfsbe done                     41520    0.00     0     0
  2      33387          0 boottrace                /etc/rc.d/var_run start                  44636    0.00     0     0
  1      33391          4 boottrace                /etc/rc.d/var_run done                   44636    0.00     1     0
  0      33391          0 boottrace                /etc/rc.d/sysvipc start                  46502    0.00     0     0
  0      33394          3 boottrace                /etc/rc.d/sysvipc done                   46502    0.00     0     0
  0      33395          1 boottrace                /etc/rc.d/zfs start                      48074    0.00     0     0
  0      35175       1780 boottrace                /etc/rc.d/zfs done                       48074    1.79   167     0
  1      35175          0 boottrace                /etc/rc.d/kld start                      50468    0.00     0     0
  1      35178          3 boottrace                /etc/rc.d/kld done                       50468    0.00     0     0
  0      35179          1 boottrace                /etc/rc.d/var start                      52786    0.00     0     0
  0      35183          4 boottrace                /etc/rc.d/var done                       52786    0.00     2     0
  1      35183          0 boottrace                /etc/rc.d/devmatch start                 55715    0.00     0     0
  2      35208         25 kldload                  ichsmb.ko: sysinit 0x3100000             61323    0.00     0     0
  2      35216          8 kldload                  if_iwlwifi.ko: sysinit 0x1b00000         61744    0.00     0     0
  2      35216          0 kldload                  if_iwlwifi.ko: sysinit 0x3100000         61744    0.00     0     0
  2      35216          0 kldload                  if_iwlwifi.ko: sysinit 0xaffffff         61744    0.00     0     0
  3      35220          4 kldload                  if_iwm.ko: sysinit 0x 700000             62155    0.00     0     0
  3      35220          0 kldload                  if_iwm.ko: sysinit 0x3100000             62155    0.00     0     0
  1      35236         16 firmware taskq           iwm8265fw.ko: sysinit 0x3100000              0    0.00     0     0
  1      35314         78 boottrace                /etc/rc.d/devmatch done                  55715    0.08    37     0
  1      35314          0 boottrace                /etc/rc.d/linux start                    62639    0.00     0     0
  1      35317          3 boottrace                /etc/rc.d/linux done                     62639    0.00     0     0
  0      35318          1 boottrace                /etc/rc.d/localpkg start                 64397    0.00     0     0
  2      35323          5 boottrace                /etc/rc.d/localpkg done                  64397    0.00    11     0
  3      35324          1 boottrace                /etc/rc.d/cleanvar start                 66095    0.00     0     0
  1      35329          5 boottrace                /etc/rc.d/cleanvar done                  66095    0.00     5     0
  1      35330          1 boottrace                /etc/rc.d/cfumass start                  69234    0.00     0     0
  1      35333          3 boottrace                /etc/rc.d/cfumass done                   69234    0.00     0     0
  0      35333          0 boottrace                /etc/rc.d/FILESYSTEMS start              71638    0.00     0     0
  0      35334          1 boottrace                /etc/rc.d/FILESYSTEMS done               71638    0.00     0     0
  3      35341          7 boottrace                /usr/local/etc/rc.d/zfs_defang start     73619    0.00     0     0
  3      35344          3 boottrace                /usr/local/etc/rc.d/zfs_defang done      73619    0.00     0     0
  3      35345          1 boottrace                /etc/rc.d/netoptions start               75660    0.00     0     0
  0      35353          8 boottrace                /etc/rc.d/netoptions done                75660    0.00     1     0
  0      35354          1 boottrace                /etc/rc.d/opensm start                   80891    0.00     0     0
  3      35358          4 boottrace                /etc/rc.d/opensm done                    80891    0.00     0     0
  2      35359          1 boottrace                /etc/rc.d/random start                   82472    0.00     0     0
  1      35373         14 boottrace                /etc/rc.d/random done                    82472    0.01    10     2
 
 [...]



While the documentation isn't very clear on this, the script has to be in /etc/rc.d/, not /usr/local/etc/rc.d/

Even if /usr/local/etc/rc.d/ is mounted during early boot, /etc/rc will not look there until after the early-late divider, usually FILESYSTEMS

sh:
# Do a first pass to get everything up to $early_late_divider so that
# we can do a second pass that includes $local_startup directories
#
unset system_rc
find_system_scripts
files=`rcorder ${skip} ${skip_firstboot} ${system_rc} 2>/dev/null`

_rc_elem_done=' '
for _rc_elem in ${files}; do
        run_rc_script ${_rc_elem} ${_boot}
        _rc_elem_done="${_rc_elem_done}${_rc_elem} "

        case "$_rc_elem" in
        */${early_late_divider})        break ;;
        esac
done
 
Back
Top