FreeBSD 14/15-CURRENT SMP Hangs on Boot with OpenSBI Built from Source

Hello!
I am building OpenSBI from source (1.5.1) with no changes. The host is Ubuntu 24 (ARM64) running QEMU 9.1.2. When I attempt to boot FreeBSD, the kernel hangs in the kernel init process, somewhere in the CPU init machdep chain. It seems to hang on vm_fault_deallocate() but I am having trouble tracing this back to a source. It only hangs if the QEMU VM is configured to have multiple processors (harts). Every other hart except the boot hart is in WFI, idling, while the boot hart launches the kernel.

I'm unsure if this is a FreeBSD kernel issue with handling SMP or if this is an OpenSBI issue related to some delta between the OpenSBI typically used to boot FreeBSD versus the OpenSBI 1.5.1 built from source. The GCC version is 14.2.0 if it helps. Here is the most useful information that I can provide:
Code:
Copyright (c) 1992-2023 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 14.1-RELEASE releng/14.1-n267679-10e31f0946d8 GENERIC riscv
FreeBSD clang version 18.1.5 ([URL]https://github.com/llvm/llvm-project.git[/URL] llvmorg-18.1.5-0-g617a15a9eac9)
VT: init without driver.
SBI: OpenSBI v1.5
SBI Specification Version: 2.0
CPU 0  : Vendor=Unspecified Core=Unspecified (Hart 0)
  marchid=0, mimpid=0
  MMU: 0x7<Sv39,Sv48,Sv57>
  ISA: 0x112d<Atomic,Compressed,Double,Float,Mult/Div>
  S-mode Extensions: 0x1<Sstc>
real memory  = 2147483648 (2048 MB)
avail memory = 2036912128 (1942 MB)
QEMU 9.1.2 monitor - type 'help' for more information
(qemu) info registers

CPU#0
 V      =   0
 pc       ffffffc00058c972
 mhartid  0000000000000000
 mstatus  8000000a00006180
 hstatus  0000000200000000
 vsstatus 0000000a00000000
 mip      0000000000000000
 mie      0000000000000008
 mideleg  0000000000001666
 hideleg  0000000000000000
 medeleg  0000000000f0b509
 hedeleg  0000000000000000
 mtvec    00000000800004f0
 stvec    ffffffc000589850
 vstvec   0000000000000000
 mepc     ffffffc00058c81e
 sepc     00000000f5e00142
 vsepc    0000000000000000
 mcause   0000000000000009
 scause   000000000000000c
 vscause  0000000000000000
 mtval    0000000000000000
 stval    00000000f5e00142
 htval    0000000000000000
 mtval2   0000000000000000
 mscratch 000000008004e000
 sscratch 0000000000000000
 satp     80000000000f64f2
 x0/zero  0000000000000000 x1/ra    ffffffc00058c910 x2/sp    ffffffc000003d50 x3/gp    ffffffc0007e2198
 x4/tp    ffffffc0009b1680 x5/t0    ffffffc0009b4d98 x6/t1    0000000080200000 x7/t2    00ff000000000000
 x8/s0    ffffffc000003dc0 x9/s1    ffffffc0007e5920 x10/a0   0000000000000000 x11/a1   0000000000000001
 x12/a2   0000000000000001 x13/a3   ffffffc000975a40 x14/a4   0000000000000000 x15/a5   0000000000000001
 x16/a6   000000000000003f x17/a7   ffffffc000003b28 x18/s2   0000000000000001 x19/s3   0000000000000001
 x20/s4   ffffffc00099e958 x21/s5   0000000100000000 x22/s6   0000000000000001 x23/s7   ffffffc0007e2ba0
 x24/s8   ffffffc0005f9dfb x25/s9   0000000000000001 x26/s10  ffffffc00099e958 x27/s11  0000000000000002
 x28/t3   000000000000ff00 x29/t4   fefefefefefefeff x30/t5   8080808080808080 x31/t6   000000000000062a
 f0/ft0   ffffffff00000000 f1/ft1   ffffffff00000000 f2/ft2   ffffffff00000000 f3/ft3   ffffffff00000000
 f4/ft4   ffffffff00000000 f5/ft5   ffffffff00000000 f6/ft6   ffffffff00000000 f7/ft7   ffffffff00000000
 f8/fs0   ffffffff00000000 f9/fs1   ffffffff00000000 f10/fa0  ffffffff00000000 f11/fa1  ffffffff00000000
 f12/fa2  ffffffff00000000 f13/fa3  ffffffff00000000 f14/fa4  ffffffff00000000 f15/fa5  ffffffff00000000
 f16/fa6  ffffffff00000000 f17/fa7  ffffffff00000000 f18/fs2  ffffffff00000000 f19/fs3  ffffffff00000000
 f20/fs4  ffffffff00000000 f21/fs5  ffffffff00000000 f22/fs6  ffffffff00000000 f23/fs7  ffffffff00000000
 f24/fs8  ffffffff00000000 f25/fs9  ffffffff00000000 f26/fs10 ffffffff00000000 f27/fs11 ffffffff00000000
 f28/ft8  ffffffff00000000 f29/ft9  ffffffff00000000 f30/ft10 ffffffff00000000 f31/ft11 ffffffff00000000
(qemu)

Any thoughts? I am stuck...
Thank you,
Don B
 
Ahh.. forgot to add the correct CPU and posted an idle CPU instead. Here is the active CPU...
Code:
CPU#3
 V      =   0
 pc       ffffffc00058c972
 mhartid  0000000000000003
 mstatus  8000000a00006180
 hstatus  0000000200000000
 vsstatus 0000000a00000000
 mip      0000000000000000
 mie      0000000000000008
 mideleg  0000000000001666
 hideleg  0000000000000000
 medeleg  0000000000f0b509
 hedeleg  0000000000000000
 mtvec    00000000800004f0
 stvec    ffffffc000589850
 vstvec   0000000000000000
 mepc     ffffffc00058c81e
 sepc     00000000f5e00142
 vsepc    0000000000000000
 mcause   0000000000000009
 scause   000000000000000c
 vscause  0000000000000000
 mtval    0000000000000000
 stval    00000000f5e00142
 htval    0000000000000000
 mtval2   0000000000000000
 mscratch 0000000080048000
 sscratch 0000000000000000
 satp     80000000000f64f2
 x0/zero  0000000000000000 x1/ra    ffffffc00058c910 x2/sp    ffffffc000003d50 x3/gp    ffffffc0007e2198
 x4/tp    ffffffc0009b1680 x5/t0    ffffffc0009b4d98 x6/t1    0000000080200000 x7/t2    00ff000000000000
 x8/s0    ffffffc000003dc0 x9/s1    ffffffc0007e5920 x10/a0   0000000000000000 x11/a1   0000000000000001
 x12/a2   0000000000000001 x13/a3   ffffffc000975a40 x14/a4   0000000000000000 x15/a5   0000000000000001
 x16/a6   000000000000003f x17/a7   ffffffc000003b28 x18/s2   0000000000000001 x19/s3   0000000000000000
 x20/s4   ffffffc00099e958 x21/s5   0000000100000000 x22/s6   0000000000000001 x23/s7   ffffffc0007e2ba0
 x24/s8   ffffffc0005f9dfb x25/s9   0000000000000000 x26/s10  ffffffc00099e958 x27/s11  0000000000000001
 x28/t3   000000000000ff00 x29/t4   fefefefefefefeff x30/t5   8080808080808080 x31/t6   0000000000000316
 f0/ft0   ffffffff00000000 f1/ft1   ffffffff00000000 f2/ft2   ffffffff00000000 f3/ft3   ffffffff00000000
 f4/ft4   ffffffff00000000 f5/ft5   ffffffff00000000 f6/ft6   ffffffff00000000 f7/ft7   ffffffff00000000
 f8/fs0   ffffffff00000000 f9/fs1   ffffffff00000000 f10/fa0  ffffffff00000000 f11/fa1  ffffffff00000000
 f12/fa2  ffffffff00000000 f13/fa3  ffffffff00000000 f14/fa4  ffffffff00000000 f15/fa5  ffffffff00000000
 f16/fa6  ffffffff00000000 f17/fa7  ffffffff00000000 f18/fs2  ffffffff00000000 f19/fs3  ffffffff00000000
 f20/fs4  ffffffff00000000 f21/fs5  ffffffff00000000 f22/fs6  ffffffff00000000 f23/fs7  ffffffff00000000
 f24/fs8  ffffffff00000000 f25/fs9  ffffffff00000000 f26/fs10 ffffffff00000000 f27/fs11 ffffffff00000000
 f28/ft8  ffffffff00000000 f29/ft9  ffffffff00000000 f30/ft10 ffffffff00000000 f31/ft11 ffffffff00000000
Thanks,
D
 
***NOTE***
Last night's pull of master branch fixed whatever build issue was causing master to fail build (at least on my environment). Current OpenSBI master as of Dec 6 2024 boots SMP FreeBSD 14.1/14.2/15-CURRENT without issue. Unsure what the change is, but that resolves this thread for *my* purposes. Would be nice to know what caused the fix.
 
Glad you got it fixed.
Including a link to what OpenSBI is would be helpful. I have no idea what it is.
So its RISC related. Probably vision five.


 
Back
Top