Other Autoboot for zroot on geli

DomXY · 2025-04-30T23:58:17+0100

Dear community!

I want to give FreeBSD a try. My first mile stone is a fully encrypted headless system, i.e. which can do boots without interference.
The idea is to have an USB stick holding an unencrypted ufs with /boot-dir and the key file, which decrypts the main drive's partition holding a encrypted zfs root.
This way each storage element on its own is easily made useless. (Without forensic restore-ability on the same drive.)

But I got an issue at the auto boot mechanism:
I set it up as stated at geli(8). However, it is not working as expected. It is still asking for a passphrase and seems to fail to use the key file only.
I added the -p switch (*_flags) later in desperation, but it didn't help. How can I stop geli asking for the passphrase?

I attach my actions, maybe containing typo, because I was unable to copy it directly, but retyping by hand. (The system is running and and can be started using the passphrase. This applies for boots from vtbd0 as well as for vtbd1, too.)
I also attach the messages from a boot.

Currently it is a test setup VM.

Thank you in beforehand.

Dominik

PS I tested, using the DVD, if geli attach -p -k /mnt/key/boot/keys/bsdroot0.key vtbd0p3 works. It did, as expected. The issue appears only at boot time.

sh:

# Prepare environment
FSTAB_FMT="%s\t\t%s\t%s\t%s\t\t%s\t%s\n"
zpool import # just load zfs driver for sysctl
sysctl vfs.zfs.min_auto_ashift=12
kldload aesni
# Partition table
gpart destroy -F "vtbd0"
zpool labelclear -f "/dev/vtbd0"
gpart create -s gpt "vtbd0"
gpart destroy -F "vtbd0"
gpart create -s gpt "vtbd0"
# BOOT PARTITION START
# Legacy boot
gpart add -a 4k -l gptboot0 -t freebsd-boot -s 512k "vtbd0"
gpart bootcode -b "/boot/pmbr" -p "/boot/gptzfsboot" -i 1 "vtbd0"
## EFI
#gpart add -a 4k -l EFI -s 260M -t efi "vtbd0"
#newfs_msdos -F 32 -c 1 /dev/vtbd0p1
#mount -t msdosfs -o longnames /dev/vtbd0p1 /mnt
#mkdir -p /mnt/EFI/BOOT
#cp -a /boot/loader.efi /mnt/EFI/BOOT/BOOTX64.efi
#umount /mnt
#printf "$FSTAB_FMT" "/dev/vtbd0p1" "/boot/efi" "msdosfs" "rw" "0" "0" >> "/tmp/bsdinstall_etc/fstab"
# BOOT PARTITION END
# Swap
gpart add -a 1m -l swap0 -t freebsd-swap -s 2147483648b "vtbd0"
zpool labelclear -f "/dev/vtbd0p2"
printf "$FSTAB_FMT" "/dev/vtbd0p2.eli" "none" "swap" "sw" "0" "0" >> "/tmp/bsdinstall_etc/fstab"
# Root file system
gpart add -a 1m -l bsdroot0 -t freebsd-zfs "vtbd0"
zpool labelclear -f "/dev/vtbd0p3"
geli init -bg -e AES-XTS -l 256 -s 4096 "vtbd0p3" # Change -s' argument for sectors?
geli attach "vtbd0p3"
zpool create -R /mnt -O compress=zstd -O atime=off -m none -f "bsdtest"   vtbd0p3.eli
zfs create -o mountpoint=none "bsdtest/ROOT"
zfs create -o mountpoint=/ "bsdtest/ROOT/default"
zfs create -o mountpoint=/home -o compress=zstd-19 "bsdtest/home"
zfs create -o mountpoint=/tmp -o compress=zle -o exec=on -o setuid=off "bsdtest/tmp"
zfs create -o mountpoint=/usr "bsdtest/usr"
#zfs create -o setuid=off -o compress=zstd-19 "bsdtest/usr/ports"
#zfs create -o compress=zstd-19 "bsdtest/usr/src"
zfs create -o mountpoint=/var "bsdtest/var"
zfs create -o exec=off -o setuid=off "bsdtest/var/audit"
zfs create -o exec=off -o setuid=off "bsdtest/var/crash"
zfs create -o exec=off -o setuid=off "bsdtest/var/log"
zfs create -o atime=on "bsdtest/var/mail"
zfs create -o setuid=off -o compress=zle "bsdtest/var/tmp"
zfs set "mountpoint=/bsdtest" "bsdtest"
mkdir -p "/mnt/tmp"
chmod 1777 "/mnt/tmp"
mkdir -p "/mnt/var/tmp"
chmod 1777 "/mnt/var/tmp"
zpool set bootfs="bsdtest/ROOT/default" "bsdtest"
mkdir -p "/mnt/var/cache/zfs"
ln -sf "../var/cache/zfs" "/mnt/boot/zfs"
zpool set cachefile="/mnt/var/cache/zfs/zpool.cache" "bsdtest"
zfs set "canmount=noauto" "bsdtest/ROOT/default"
# Prepare configuration
echo 'zfs_enable="YES"' >> "/tmp/bsdinstall_etc/rc.conf.zfs"
echo 'kern.geom.label.disk_ident.enable="0"' >> "/tmp/bsdinstall_boot/loader.conf.zfs"
echo 'kern.geom.label.gptid.enable="0"' >> "/tmp/bsdinstall_boot/loader.conf.zfs"
echo "vfs.zfs.min_auto_ashift=12" >> "/tmp/bsdinstall_etc/sysctl.conf.zfs"
echo 'aesni_load="YES"' >> "/tmp/bsdinstall_boot/loader.conf.aesni"
echo 'geom_eli_load="YES"' >> "/tmp/bsdinstall_boot/loader.conf.geli"
echo 'verbose_loading="YES"' >> "/tmp/bsdinstall_boot/loader.conf"
^D
# finished bsdinstall

## Auto-Unlock
# Create seaparate boot partition
gpart create -s gpt /dev/vtbd1
gpart add -a 512 -l gptboot1 -t freebsd-boot -s 512k "vtbd1"
gpart bootcode -b "/boot/pmbr" -p "/boot/gptboot" -i 1 "vtbd1"
# Legacy boot
gpart add -a 512 -l autoboot0 -t freebsd-ufs "vtbd1"
newfs -Uj /dev/gpt/autoboot0
printf "$FSTAB_FMT" "/dev/gpt/autoboot0" "/boot-automatic" "ufs" "rw" "1" "2" >> "/etc/fstab"
mount -a
# Create keyfile
mkdir /etc/keys
chmod go= /etc/keys
dd if=/dev/urandom of=/etc/keys/bsdroot0.key count=512 bs=1
geli setkey -n 0 -P -K /etc/keys/bsdroot0.key /dev/vtbd0p3 # The passphrase was already moved to -n 1 at this stage.
cp -a /etc/keys /boot/keys
# Configure loader.conf
disk=vtbd0
echo 'zroot_load="YES"' >> "/boot/loader.conf"
echo "zroot_type=\"${disk}:geli_keyfile\"" >> "/boot/loader.confi"
echo 'zroot_name="/boot/keys/bsdroot0.key"' >> "/boot/loader.conf"
echo 'zroot_flags="-p"' >> "/boot/loader.conf"
echo 'vfs.root.mountfrom="zfs:bsdtest/ROOT/default"' >> "/boot/loader.conf"
# Deploy to autoboot
rm -rf /boot-automatic/boot
cp -a /boot /boot-automatic/boot

PPS This belongs to my first post. I missed how the attachments work here, so the messages was not attached. (I uploaded it, but missed that I had to insert it afterwards into the text. Though, here it goes.)

Code:

Apr 30 21:41:48 bsdtest kernel: ---<<BOOT>>---
Apr 30 21:41:48 bsdtest kernel: Copyright (c) 1992-2023 The FreeBSD Project.
Apr 30 21:41:48 bsdtest kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Apr 30 21:41:48 bsdtest kernel:     The Regents of the University of California. All rights reserved.
Apr 30 21:41:48 bsdtest kernel: FreeBSD is a registered trademark of The FreeBSD Foundation.
Apr 30 21:41:48 bsdtest kernel: FreeBSD 14.2-RELEASE releng/14.2-n269506-c8918d6c7412 GENERIC amd64
Apr 30 21:41:48 bsdtest kernel: FreeBSD clang version 18.1.6 (https://github.com/llvm/llvm-project.git llvmorg-18.1.6-0-g1118c2e05e67)
Apr 30 21:41:48 bsdtest kernel: VT(vga): text 80x25
Apr 30 21:41:48 bsdtest kernel: CPU: Intel(R) Celeron(R) N5105 @ 2.00GHz (1996.86-MHz K8-class CPU)
Apr 30 21:41:48 bsdtest kernel:   Origin="GenuineIntel"  Id=0x906c0  Family=0x6  Model=0x9c  Stepping=0
Apr 30 21:41:48 bsdtest kernel:   Features=0xf83fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,SS>
Apr 30 21:41:48 bsdtest kernel:   Features2=0xcff8a223<SSE3,PCLMULQDQ,VMX,SSSE3,CX16,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,RDRAND,HV>
Apr 30 21:41:48 bsdtest kernel:   AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
Apr 30 21:41:48 bsdtest kernel:   AMD Features2=0x101<LAHF,Prefetch>
Apr 30 21:41:48 bsdtest kernel:   Structured Extended Features=0x21940283<FSGSBASE,TSCADJ,SMEP,ERMS,RDSEED,SMAP,CLFLUSHOPT,CLWB,SHA>
Apr 30 21:41:48 bsdtest kernel:   Structured Extended Features2=0x18400124<UMIP,WAITPKG,GFNI,RDPID,MOVDIRI,MOVDIR64B>
Apr 30 21:41:48 bsdtest kernel:   Structured Extended Features3=0xac000400<MD_CLEAR,IBPB,STIBP,ARCH_CAP,SSBD>
Apr 30 21:41:48 bsdtest kernel:   XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
Apr 30 21:41:48 bsdtest kernel:   IA32_ARCH_CAPS=0x6b<RDCL_NO,IBRS_ALL,SKIP_L1DFL_VME,MDS_NO>
Apr 30 21:41:48 bsdtest kernel:   AMD Extended Feature Extensions ID EBX=0x100d000<IBPB,IBRS,STIBP,SSBD>
Apr 30 21:41:48 bsdtest kernel:   VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
Apr 30 21:41:48 bsdtest kernel: Hypervisor: Origin = "KVMKVMKVM"
Apr 30 21:41:48 bsdtest kernel: real memory  = 2147483648 (2048 MB)
Apr 30 21:41:48 bsdtest kernel: avail memory = 2040156160 (1945 MB)
Apr 30 21:41:48 bsdtest kernel: Event timer "LAPIC" quality 600
Apr 30 21:41:48 bsdtest kernel: ACPI APIC Table: <BOCHS  BXPC    >
Apr 30 21:41:48 bsdtest kernel: random: registering fast source Intel Secure Key RNG
Apr 30 21:41:48 bsdtest kernel: random: fast provider: "Intel Secure Key RNG"
Apr 30 21:41:48 bsdtest kernel: random: unblocking device.
Apr 30 21:41:48 bsdtest kernel: ioapic0 <Version 1.1> irqs 0-23
Apr 30 21:41:48 bsdtest kernel: random: entropy device external interface
Apr 30 21:41:48 bsdtest kernel: kbd1 at kbdmux0
Apr 30 21:41:48 bsdtest kernel: vtvga0: <VT VGA driver>
Apr 30 21:41:48 bsdtest kernel: kvmclock0: <KVM paravirtual clock>
Apr 30 21:41:48 bsdtest kernel: Timecounter "kvmclock" frequency 1000000000 Hz quality 975
Apr 30 21:41:48 bsdtest kernel: kvmclock0: registered as a time-of-day clock, resolution 0.000001s
Apr 30 21:41:48 bsdtest kernel: smbios0: <System Management BIOS> at iomem 0xf5990-0xf59ae
Apr 30 21:41:48 bsdtest kernel: smbios0: Version: 2.8, BCD Revision: 2.8
Apr 30 21:41:48 bsdtest kernel: aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS,SHA1,SHA256>
Apr 30 21:41:48 bsdtest kernel: acpi0: <BOCHS BXPC>
Apr 30 21:41:48 bsdtest kernel: acpi0: Power Button (fixed)
Apr 30 21:41:48 bsdtest kernel: cpu0: <ACPI CPU> on acpi0
Apr 30 21:41:48 bsdtest kernel: atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
Apr 30 21:41:48 bsdtest kernel: atrtc0: registered as a time-of-day clock, resolution 1.000000s
Apr 30 21:41:48 bsdtest kernel: Event timer "RTC" frequency 32768 Hz quality 0
Apr 30 21:41:48 bsdtest kernel: Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
Apr 30 21:41:48 bsdtest kernel: acpi_timer0: <24-bit timer at 3.579545MHz> port 0x608-0x60b on acpi0
Apr 30 21:41:48 bsdtest kernel: pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
Apr 30 21:41:48 bsdtest kernel: pci0: <ACPI PCI bus> on pcib0
Apr 30 21:41:48 bsdtest kernel: isab0: <PCI-ISA bridge> at device 1.0 on pci0
Apr 30 21:41:48 bsdtest kernel: isa0: <ISA bus> on isab0
Apr 30 21:41:48 bsdtest kernel: atapci0: <Intel PIIX3 WDMA2 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc160-0xc16f at device 1.1 on pci0
Apr 30 21:41:48 bsdtest kernel: ata0: <ATA channel> at channel 0 on atapci0
Apr 30 21:41:48 bsdtest kernel: ata1: <ATA channel> at channel 1 on atapci0
Apr 30 21:41:48 bsdtest kernel: pci0: <bridge> at device 1.3 (no driver attached)
Apr 30 21:41:48 bsdtest kernel: vgapci0: <VGA-compatible display> mem 0xfd000000-0xfdffffff,0xfebf0000-0xfebf0fff at device 2.0 on pci0
Apr 30 21:41:48 bsdtest kernel: vgapci0: Boot video device
Apr 30 21:41:48 bsdtest kernel: em0: <Intel(R) Legacy PRO/1000 MT 82540EM> port 0xc000-0xc03f mem 0xfebc0000-0xfebdffff irq 11 at device 3.0 on pci0
Apr 30 21:41:48 bsdtest kernel: em0: Using 1024 TX descriptors and 1024 RX descriptors
Apr 30 21:41:48 bsdtest kernel: em0: Ethernet address: 52:54:00:53:39:ec
Apr 30 21:41:48 bsdtest kernel: em0: netmap queues/slots: TX 1/1024, RX 1/1024
Apr 30 21:41:48 bsdtest kernel: uhci0: <Intel 82801I (ICH9) USB controller> port 0xc0c0-0xc0df irq 11 at device 4.0 on pci0
Apr 30 21:41:48 bsdtest kernel: usbus0 on uhci0
Apr 30 21:41:48 bsdtest kernel: uhci1: <Intel 82801I (ICH9) USB controller> port 0xc0e0-0xc0ff irq 10 at device 4.1 on pci0
Apr 30 21:41:48 bsdtest kernel: usbus1 on uhci1
Apr 30 21:41:48 bsdtest kernel: uhci2: <Intel 82801I (ICH9) USB controller> port 0xc100-0xc11f irq 10 at device 4.2 on pci0
Apr 30 21:41:48 bsdtest kernel: usbus2 on uhci2
Apr 30 21:41:48 bsdtest kernel: ehci0: <Intel 82801I (ICH9) USB 2.0 controller> mem 0xfebf1000-0xfebf1fff irq 11 at device 4.7 on pci0
Apr 30 21:41:48 bsdtest kernel: usbus3: EHCI version 1.0
Apr 30 21:41:48 bsdtest kernel: usbus3 on ehci0
Apr 30 21:41:48 bsdtest kernel: virtio_pci0: <VirtIO PCI (legacy) Block adapter> port 0xc040-0xc07f mem 0xfebf2000-0xfebf2fff,0xfe000000-0xfe003fff irq 10 at device 5.0 on pci0
Apr 30 21:41:48 bsdtest kernel: vtblk0: <VirtIO Block Adapter> on virtio_pci0
Apr 30 21:41:48 bsdtest kernel: vtblk0: 10240MB (20971520 512 byte sectors)
Apr 30 21:41:48 bsdtest kernel: virtio_pci1: <VirtIO PCI (legacy) Balloon adapter> port 0xc120-0xc13f mem 0xfe004000-0xfe007fff irq 10 at device 6.0 on pci0
Apr 30 21:41:48 bsdtest kernel: vtballoon0: <VirtIO Balloon Adapter> on virtio_pci1
Apr 30 21:41:48 bsdtest kernel: ahci0: <Intel ICH9 AHCI SATA controller> port 0xc140-0xc15f mem 0xfebf3000-0xfebf3fff irq 11 at device 7.0 on pci0
Apr 30 21:41:48 bsdtest kernel: ahci0: AHCI v1.00 with 6 1.5Gbps ports, Port Multiplier not supported
Apr 30 21:41:48 bsdtest kernel: ahcich0: <AHCI channel> at channel 0 on ahci0
Apr 30 21:41:48 bsdtest kernel: ahcich1: <AHCI channel> at channel 1 on ahci0
Apr 30 21:41:48 bsdtest kernel: ahcich2: <AHCI channel> at channel 2 on ahci0
Apr 30 21:41:48 bsdtest kernel: ahcich3: <AHCI channel> at channel 3 on ahci0
Apr 30 21:41:48 bsdtest kernel: ahcich4: <AHCI channel> at channel 4 on ahci0
Apr 30 21:41:48 bsdtest kernel: ahcich5: <AHCI channel> at channel 5 on ahci0
Apr 30 21:41:48 bsdtest kernel: virtio_pci2: <VirtIO PCI (legacy) Block adapter> port 0xc080-0xc0bf mem 0xfebf4000-0xfebf4fff,0xfe008000-0xfe00bfff irq 11 at device 8.0 on pci0
Apr 30 21:41:48 bsdtest kernel: vtblk1: <VirtIO Block Adapter> on virtio_pci2
Apr 30 21:41:48 bsdtest kernel: vtblk1: 1537MB (3147776 512 byte sectors)
Apr 30 21:41:48 bsdtest kernel: acpi_syscontainer0: <System Container> on acpi0
Apr 30 21:41:48 bsdtest kernel: acpi_syscontainer1: <System Container> port 0xaf00-0xaf0b on acpi0
Apr 30 21:41:48 bsdtest kernel: acpi_syscontainer2: <System Container> port 0xafe0-0xafe3 on acpi0
Apr 30 21:41:48 bsdtest kernel: acpi_syscontainer3: <System Container> port 0xae00-0xae17 on acpi0
Apr 30 21:41:48 bsdtest kernel: uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
Apr 30 21:41:48 bsdtest kernel: atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
Apr 30 21:41:48 bsdtest kernel: atkbd0: <AT Keyboard> irq 1 on atkbdc0
Apr 30 21:41:48 bsdtest kernel: kbd0 at atkbd0
Apr 30 21:41:48 bsdtest kernel: atkbd0: [GIANT-LOCKED]
Apr 30 21:41:48 bsdtest kernel: psm0: <PS/2 Mouse> irq 12 on atkbdc0
Apr 30 21:41:48 bsdtest kernel: psm0: [GIANT-LOCKED]
Apr 30 21:41:48 bsdtest kernel: WARNING: Device "psm" is Giant locked and may be deleted before FreeBSD 15.0.
Apr 30 21:41:48 bsdtest kernel: psm0: model IntelliMouse Explorer, device ID 4
Apr 30 21:41:48 bsdtest kernel: fdc0: <floppy drive controller (FDE)> port 0x3f2-0x3f5,0x3f7 irq 6 drq 2 on acpi0
Apr 30 21:41:48 bsdtest kernel: fdc0: does not respond
Apr 30 21:41:48 bsdtest kernel: device_attach: fdc0 attach returned 6
Apr 30 21:41:48 bsdtest kernel: orm0: <ISA Option ROM> at iomem 0xea800-0xeffff pnpid ORM0000 on isa0
Apr 30 21:41:48 bsdtest kernel: vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid PNP0900 on isa0
Apr 30 21:41:48 bsdtest kernel: attimer0: <AT timer> at port 0x40 on isa0
Apr 30 21:41:48 bsdtest kernel: Timecounter "i8254" frequency 1193182 Hz quality 0
Apr 30 21:41:48 bsdtest kernel: Event timer "i8254" frequency 1193182 Hz quality 100
Apr 30 21:41:48 bsdtest kernel: attimer0: non-PNP ISA device will be removed from GENERIC in FreeBSD 15.
Apr 30 21:41:48 bsdtest kernel: Timecounter "TSC" frequency 1996799729 Hz quality 800
Apr 30 21:41:48 bsdtest kernel: Timecounters tick every 10.000 msec
Apr 30 21:41:48 bsdtest kernel: usbus0: 12Mbps Full Speed USB v1.0
Apr 30 21:41:48 bsdtest kernel: usbus1: 12Mbps Full Speed USB v1.0
Apr 30 21:41:48 bsdtest kernel: usbus2: 12Mbps Full Speed USB v1.0
Apr 30 21:41:48 bsdtest kernel: usbus3: 480Mbps High Speed USB v2.0
Apr 30 21:41:48 bsdtest kernel: ugen3.1: <Intel EHCI root HUB> at usbus3
Apr 30 21:41:48 bsdtest kernel: uhub0 on usbus3
Apr 30 21:41:48 bsdtest kernel: uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
Apr 30 21:41:48 bsdtest kernel: ugen2.1: <Intel UHCI root HUB> at usbus2
Apr 30 21:41:48 bsdtest kernel: uhub1 on usbus2
Apr 30 21:41:48 bsdtest kernel: uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
Apr 30 21:41:48 bsdtest kernel: ugen1.1: <Intel UHCI root HUB> at usbus1
Apr 30 21:41:48 bsdtest kernel: uhub2 on usbus1
Apr 30 21:41:48 bsdtest kernel: uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
Apr 30 21:41:48 bsdtest kernel: ugen0.1: <Intel UHCI root HUB> at usbus0
Apr 30 21:41:48 bsdtest kernel: uhub3 on usbus0
Apr 30 21:41:48 bsdtest kernel: uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
Apr 30 21:41:48 bsdtest kernel: ZFS filesystem version: 5
Apr 30 21:41:48 bsdtest kernel: ZFS storage pool version: features support (5000)
Apr 30 21:41:48 bsdtest kernel: Trying to mount root from zfs:bsdtest/ROOT/default []...
Apr 30 21:41:48 bsdtest kernel: cd0 at ahcich0 bus 0 scbus2 target 0 lun 0
Apr 30 21:41:48 bsdtest kernel: cd0: <QEMU QEMU DVD-ROM 2.5+> Removable CD-ROM SCSI device
Apr 30 21:41:48 bsdtest kernel: cd0: Serial Number QM00005
Apr 30 21:41:48 bsdtest kernel: cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes)
Apr 30 21:41:48 bsdtest kernel: cd0: 4602MB (2356644 2048 byte sectors)
Apr 30 21:41:48 bsdtest kernel: uhub1: 2 ports with 2 removable, self powered
Apr 30 21:41:48 bsdtest kernel: uhub2: 2 ports with 2 removable, self powered
Apr 30 21:41:48 bsdtest kernel: uhub3: 2 ports with 2 removable, self powered
Apr 30 21:41:48 bsdtest kernel: Enter passphrase for vtbd0p3: uhub0: 6 ports with 6 removable, self powered
Apr 30 21:41:48 bsdtest kernel: ugen3.2: <QEMU QEMU USB Tablet> at usbus3
Apr 30 21:41:48 bsdtest kernel: GEOM_ELI: Device vtbd0p3.eli created.
Apr 30 21:41:48 bsdtest kernel: GEOM_ELI: Encryption: AES-XTS 256
Apr 30 21:41:48 bsdtest kernel: GEOM_ELI:     Crypto: accelerated software
Apr 30 21:41:48 bsdtest kernel: GEOM_ELI: Device gpt/swap0.eli created.
Apr 30 21:41:48 bsdtest kernel: GEOM_ELI: Encryption: AES-XTS 128
Apr 30 21:41:48 bsdtest kernel: GEOM_ELI:     Crypto: accelerated software
Apr 30 21:41:48 bsdtest kernel: intsmb0: <Intel PIIX4 SMBUS Interface> irq 9 at device 1.3 on pci0
Apr 30 21:41:48 bsdtest kernel: intsmb0: intr IRQ 9 enabled revision 0
Apr 30 21:41:48 bsdtest kernel: smbus0: <System Management Bus> on intsmb0
Apr 30 21:41:48 bsdtest kernel: em0: link state changed to UP
Apr 30 21:41:48 bsdtest kernel: lo0: link state changed to UP
Apr 30 21:41:48 bsdtest kernel: em0: link state changed to DOWN
Apr 30 21:41:48 bsdtest kernel: em0: link state changed to UP
Apr 30 21:41:48 bsdtest kernel: uhid0 on uhub0
Apr 30 21:41:48 bsdtest kernel: uhid0: <QEMU QEMU USB Tablet, class 0/0, rev 2.00/0.00, addr 2> on usbus3
Apr 30 21:41:48 bsdtest ntpd[835]: ntpd 4.2.8p18-a (1): Starting
Apr 30 21:41:48 bsdtest kernel: Security policy loaded: MAC/ntpd (mac_ntpd)
Apr 30 21:41:48 bsdtest ntpd[835]: Command line: /usr/sbin/ntpd -p /var/db/ntp/ntpd.pid -c /etc/ntp.conf -f /var/db/ntp/ntpd.drift
Apr 30 21:41:48 bsdtest ntpd[835]: ----------------------------------------------------
Apr 30 21:41:48 bsdtest ntpd[835]: ntp-4 is maintained by Network Time Foundation,
Apr 30 21:41:48 bsdtest ntpd[835]: Inc. (NTF), a non-profit 501(c)(3) public-benefit
Apr 30 21:41:48 bsdtest ntpd[835]: corporation.  Support and training for ntp-4 are
Apr 30 21:41:48 bsdtest ntpd[835]: available at https://www.nwtime.org/support
Apr 30 21:41:48 bsdtest ntpd[835]: ----------------------------------------------------
Apr 30 21:41:48 bsdtest ntpd[836]: leapsecond file ('/var/db/ntpd.leap-seconds.list'): good hash signature
Apr 30 21:41:48 bsdtest ntpd[836]: leapsecond file ('/var/db/ntpd.leap-seconds.list'): loaded, expire=2025-06-28T00:00:00Z last=2017-01-01T00:00:00Z ofs=37

T-Daemon · 2025-05-01T09:22:29+0100

DomXY said:
Currently it is a test setup VM.

Rich (BB code):

# Prepare environment geli init -bg -e AES-XTS -l 256 -s 4096 "vtbd0p3" # Change -s' argument for sectors?

Here you initialize the geli provider with the -b -g options. Those option will always ask for a passphrase, -b during boot, -g before the loader is loaded: geli(8)

Rich (BB code):

     init       Initialize providers which need to be encrypted. ...
              
                -b                Try to decrypt this partition during boot,
                                  before the root partition is mounted.  ...

                -g                Enable booting from this encrypted root
                                  filesystem.  The boot loader prompts for the
                                  passphrase and loads loader(8) from the
                                  encrypted partition.

You need to remove those options with the "configure" sub-command, display the configuration (Flags) before and after:

Code:

# geli list | grep -e name -e Flags

# geli configure -BG vtbd0p3

# geli list | grep -e name -e Flags

Question to your setup: Why didn't you initialize the geli provider with the keyfile in the first place but choose to use a passphrase for the initial configuration first, then "setkey" after? Is it you want a backup (passphrase) in case the keyfile gets lost?

Second question: Is this a scripted installation setup?

DomXY · 2025-05-01T11:00:10+0100

T-Daemon said:
You need to remove those options with the "configure" sub-command, display the configuration (Flags) before and after:

Before I tried this only exclusively one by one. It didn't make a difference. Removing both at the same time renders the system unable to boot: It doesn't find the zpool anymore:

Addition: The request for the passphrase comes from /boot/kernel/geom_eli.ko according to a grep for the "Enter passphrase" string.
According to these insights: I think, if -B and -G are really working, I messed something up and the key file was never used at boot time anyway. But I can't figure out, where I went wrong. (-BG means removing -bg as you suggested.)
(The keyfile is working on its own, I tested geli attach -p -k /path/to/file.key /dev/vtbd0p3 and it worked as expected.)

T-Daemon said:
Question to your setup: Why don't you initialize the geli provider with the keyfile in the first place but choose to use a passphrase for the initial configuration first, then "setkey" after? Is it you want a backup (passphrase) in case the keyfile gets lost?

There are two answers for this question.

I followed what I read from the bsdinstall_log, because when I did this by hand the first time the system didn't boot at all, so I let bsdinstall do its magic and took from the log, what I missed. Long story short: I'm pretty new to FreeBSD, and diving deeply at the beginning is very difficult. So I just stick to things I already knew working, and then continuing to set up more complex things until I understand (not just copy) what I figured the best might be. So just for simplicity I started a default encrypted system and afterwards tried to set up a separate disk for "autoboot".
In the later stage: Yes, especially in case, the USB stick is no longer available I want be able to start the machine using a passphrase. So the actual storage can't be decrypted without the Stick, but used as a regular encryption.
Therefore: It wouldn't be a big problem if I had to set the -b -g settings in this situation, and replacing the key file, but as I stated before: without -b or -g the system is no longer booting in any way.

However, this procedure might be refined and changed for a productive system and is currently just what "happend" at my testing environment. The testing environment's primary purpose is to get a good, working "manual" which can be reproduced and adapted to more specific requirements later. That said: What you see is a heavy work-in-progress stage.

(Please go easy with me.)

T-Daemon said:
Second question: Is this a scripted setup?

Not really. Anything before

Code:

## Auto-Unlock

it maybe could be used as a script.
But this was my manually written setup if you select Partitioning->Shell for the Partition Setup within bsdinstall instead of any automatic or manual setups. I just took much from a former bsdinstall_log and adapted it so it may look like a script now. (And maybe could be used this way, but I never tried.) Most of the time I was copying this manually by typing word by word from file to console.

PS I missed how the attachments work here, so the messages was not attached. I added it now.

T-Daemon · 2025-05-01T16:11:57+0100

I did some testing in a VM [1], the -b option is needed to take effect of the "geli_keyfile" configuration in /boot/loader.conf. The -g option definitively can not be set.

The problem seems to be the -b option and init/"setkey -n X" a passphrase besides a keyfile in two different master key slots are incompatible. Once a provider is passphrase initialized or "setkey" configured, it is is non-reversible, even if the passphrase slot is overwritten with the same key file that is stored in the other slot. I can't tell if this works as intended or a bug.

The geli provider, in a /boot/loader.conf "geli_keyfile" configuration needs to be initialized with a keyfile only. There can't be a "backup" passphrase in one of the master key file slots.

For backup purposes, best is to save the keyfile on multiple high quality portable media, and tested in resonable intervals.

[1]
For simplisity, I've created two disks in a VM (vbox). The first disk simulates the USB stick with a full kernel/base installed system. The second disk is configured as a data storage device.

In your setup, for the "bsdtest" Root-on-ZFS pool, you need geli init -b -P -K /boot/keys/bsdroot0.key -l 256 -s 4096 vtbd0p3, no passphrase, now or later.