Solved Confused about kernel configuration options

kr0m · Aug 20, 2022

I have been compiling kernel but i am a little bit confused about how it chooses to include functionalities and drivers inside kernel or as external module.
I have readed the handbook section and the "Absolute freebsd" compilation kernel book section, but i have some unresolved questions.

From my tests i have arrived to the following conclusion:

The configuration file enabled options/device are compiled inside kernel and as module too.
The configuration file disabled options/device are compiled only as module.
To disable completely an options/device it has to be disabled in configuration file and compile kernel with the option: WITHOUT_MODULES indicating the modules to be disabled.

I have understood that there are some options that cant be externalized as module, how can i know which option can be compiled as module and which not? Is there a list where i can query that kind of information?

Commenting a kernel options/device has the same result as configuring it as nooptions/nodevice?

Of course if i have understood something in the wrong way, please correct me.

kr0m · Aug 21, 2022

I found a configuration file which includes all inkernel options that cant be externalized as modules:

/usr/src/sys/amd64/conf/MINIMAL

I suppose that if a configuration option doesnt appear in that configuration file, it can be compiled as module.

And a complete list of kernel parameters can be obtained in that way:

cd /usr/src/sys/$(uname -m)/conf && make LINT
cat /usr/src/sys/$(uname -m)/conf/LINT

include "../../conf/NOTES"
include "../../x86/conf/NOTES"
include NOTES

diizzy · Aug 21, 2022

Why are you trying change the GENERIC kernel config in the first place?

mer · Aug 21, 2022

kr0m said:
I suppose that if a configuration option doesnt appear in that configuration file, it can be compiled as module.

It's been a while since I've looked at building from source, but seem to recall that in general GENERIC builds as much as possible into modules. To me that makes a lot of sense for a kernel named GENERIC to run on a wide variety of hardware.

diizzy said:
Why are you trying change the GENERIC kernel config in the first place?

Why not? Building from source and mucking with kernel configs is an excellent way to learn. Trim out too much and you get an extraordinary opportunity to practice "how to recover from a busted crashing kernel".

If one is building for a true embedded system, traditionally one trims away all things that are unneeded. That's why busybox is used in a lot of places: it lets you eliminate a lot of different executables, keeping overall image size down.

Going through the process also raises your appreciation for the beauty of GENERIC.

My opinion only

kr0m · Aug 21, 2022

Hello, answering the question:
"Why are you trying change the GENERIC kernel config in the first place?"

I am reading "Absolute FreeBSD" book and i am experimenting as much as possible of the readed content in a home server, as the book says:

For most of us, stripping unnecessary drivers and features out of a kernel to shrink it is a waste of time and energy, but I would encourage you to do it once

I think compiling a tiny-custom kernel will be a good experience for learning FreeBSD internals and how kernel works.

kr0m · Aug 21, 2022

"but seem to recall that in general GENERIC builds as much as possible into modules."

I think most of the options are compilled inside kernel and as module, for example in my main system with GENERIC kernel i can see virtio support inside kernel:

Garrus # ~> uname -a
FreeBSD Garrus.alfaexploit.com 13.1-RELEASE-p1 FreeBSD 13.1-RELEASE-p1 GENERIC amd64

Garrus # ~> sysctl kern.conftxt|grep virtio
device virtio

And the module too:

Garrus # ~> ls -la /boot/kernel/virtio.ko
-r-xr-xr-x 2 root wheel 29800 May 23 19:42 /boot/kernel/virtio.ko

Why its in both locations? It doesnt make sense for me.

zirias@ · Aug 21, 2022

kr0m said:
Why its in both locations? It doesnt make sense for me.

A FreeBSD kernel config (in contrast to e.g. Linux) doesn't control which modules are built, it only controls what is built directly into the kernel.

Many drivers and components support both, direct inclusion in the kernel and building a kernel module. But for modules, the default is to just build all of them. You can control this behavior in /etc/src.conf if you want (unfortunately it's documented in make.conf(5), which doesn't really make sense as these knobs are specific to the src tree...):

Code:

     MODULES_OVERRIDE
           (str) Set to    a list of modules to build instead of all of
           them.

[...]

     NO_MODULES       (bool) Set to not build modules with    the kernel.

[...]

     WITHOUT_MODULES
           (str) Set to    a list of modules to exclude from the build.
           This    provides a somewhat easier way to exclude modules you
           are certain you will    never need than    specifying
           MODULES_OVERRIDE.  This is applied after MODULES_OVERRIDE.

kr0m · Aug 21, 2022

"A FreeBSD kernel config (in contrast to e.g. Linux) doesn't control which modules are built, it only controls what is built directly into the kernel.", ok i suppose im so influenced from Linux world. I will continue reading about it but your comment is very illuminating.

jbo@ · Aug 21, 2022

kr0m said:
"A FreeBSD kernel config (in contrast to e.g. Linux) doesn't control which modules are built, it only controls what is built directly into the kernel.", ok i suppose im so influenced from Linux world. I will continue reading about it but your comment is very illuminating.

kr0m You can directly quote other people's messages (like shown above). This is way more readable than just copy-pasting and encapsulating it in quotation marks

kr0m · Sep 3, 2022

I think that i have understood:

A FreeBSD kernel config (in contrast to e.g. Linux) doesn't control which modules are built, it only controls what is built directly into the kernel.

If you want to control which modules to build you can use that two parameters:

MODULES_OVERRIDE
(str) Set to a list of modules to build instead of all of
them.

WITHOUT_MODULES
(str) Set to a list of modules to exclude from the build.
This provides a somewhat easier way to exclude modules you
are certain you will never need than specifying
MODULES_OVERRIDE. This is applied after MODULES_OVERRIDE.

Alain De Vos · Sep 4, 2022

zirias@ said:
A FreeBSD kernel config (in contrast to e.g. Linux) doesn't control which modules are built, it only controls what is built directly into the kernel.

Many drivers and components support both, direct inclusion in the kernel and building a kernel module. But for modules, the default is to just build all of them. You can control this behavior in /etc/src.conf if you want (unfortunately it's documented in make.conf(5), which doesn't really make sense as these knobs are specific to the src tree...):

Code:

MODULES_OVERRIDE (str) Set to a list of modules to build instead of all of them. [...] NO_MODULES (bool) Set to not build modules with the kernel. [...] WITHOUT_MODULES (str) Set to a list of modules to exclude from the build. This provides a somewhat easier way to exclude modules you are certain you will never need than specifying MODULES_OVERRIDE. This is applied after MODULES_OVERRIDE.

It would be nice to have a file with a complete list modules that you can specify in MODULES_OVERRIDE in src.conf.

Phishfry · Sep 4, 2022

Alain De Vos said:
It would be nice to have a file with a complete list modules that you can specify in MODULES_OVERRIDE in src.conf.

Use the source Luke.
/usr/src/sys/modules/Makefile
/usr/src/tools/build/options/

zirias@ · Sep 4, 2022

The default is to build all modules. This should normally be just fine:

Code:

$ du -hs /boot/kernel
 96M    /boot/kernel

If you are worried about "security", be aware that an attacker could just bring their own kernel module. For control about that, see kern_securelevel in rc.conf(5) (and an extensive description of the levels in security(7)).

Other than that, the only reason I could see to fiddle with which modules are built would be some embedded system with very limited storage space.

cy@ · Sep 6, 2022

The only reason a person might want to remove devices from their default kernel is to not load unneeded code thereby reducing the kernel memory footprint. Four approaches are:

1) Use MINIMAL and load all necessary KLDs in loader.conf.

2) Build your own.

3) Build your own using nooptions and nodevice.

4) Hybrid.

This might make sense on a memory constrained system but would be a PITA because you will need to monitor kernel development to include/exclude new devices and options that you may or may not want. At some point a kernel will not boot following an upgrade because you have not included or loaded a needed module and it will be up to you to figure that out -- it would be unfair to ask developers or others to figure out your custom problem you have put yourself into.

OTOH for people who are accustomed to doing this and have worked through the problems themselves, following kernel development, reading commit log messages and adjusting their configuration as needed, fill your boots.

Typically one can by removing almost all network devices from the kernel, relying on KLDs only because ifconfig(8) will in most cases load the required device driver modules if not already in memory. But for others one can get into trouble very quickly unless one intimately knows what they're doing and can solve problems they themselves create.

Generally, running kernels other than GENERIC is frowned upon. When you open a PR, start a discussion on one of the mailing lists or here about a problem the first thing you will be asked to do is try it out using GENERIC first.

Alain De Vos · Sep 6, 2022

This is MYKERNCONF,

Code:

cat MYKERNCONF
include GENERIC
ident MYKERNELOPT
########################
options     TSLOG
#nomakeoption    DEBUG           # Build kernel with gdb(1) debug symbols
#nomakeoption    WITH_CTF        # Run ctfconvert(1) for DTrace support
nooption    KDB            # Enable kernel debugger support.
nooption    KDB_TRACE        # Print a stack trace for a panic.
########################
nooption XENHVM                  # Xen HVM kernel infrastructure
nooption NFSCL
nooption NFSD
nooption NFSLOCKD
nooption NFS_ROOT
##########################
#Floppy Drives
nodevice fdc
# SCSI Controllers
nodevice      ahc         # AHA2940 and onboard AIC7xxx devices
nodevice      ahd         # AHA39320/29320 and onboard AIC79xx devices
nodevice      esp         # AMD Am53C974 (Tekram DC-390(T))
nodevice      hptiop          # Highpoint RocketRaid 3xxx series
nodevice      isp         # Qlogic family
nodevice     ispfw           # Firmware for QLogic HBAs- normally a module
nodevice      mpt         # LSI-Logic MPT-Fusion
nodevice      mps         # LSI-Logic MPT-Fusion 2
nodevice      mpr         # LSI-Logic MPT-Fusion 3
nodevice      sym         # NCR/Symbios Logic
nodevice      isci            # Intel C600 SAS controller
nodevice      ocs_fc          # Emulex FC adapters
nodevice      pvscsi          # VMware PVSCSI
# RAID controllers interfaced to the SCSI subsystem
nodevice      amr         # AMI MegaRAID
nodevice      arcmsr          # Areca SATA II RAID
nodevice      ciss            # Compaq Smart RAID 5*
nodevice      iir         # Intel Integrated RAID
nodevice      ips         # IBM (Adaptec) ServeRAID
nodevice      mly         # Mylex AcceleRAID/eXtremeRAID
nodevice      twa         # 3ware 9000 series PATA/SATA RAID
nodevice      smartpqi        # Microsemi smartpqi driver
nodevice      tws         # LSI 3ware 9750 SATA+SAS 6Gb/s RAID controller
# RAID controllers
nodevice      aac         # Adaptec FSA RAID
nodevice      aacp            # SCSI passthrough for aac (requires CAM)
nodevice      aacraid         # Adaptec by PMC RAID
nodevice      ida         # Compaq Smart RAID
nodevice      mfi         # LSI MegaRAID SAS
nodevice      mlx         # Mylex DAC960 family
nodevice      mrsas           # LSI/Avago MegaRAID SAS/SATA, 6Gb/s and 12Gb/s
nodevice      pmspcv          # PMC-Sierra SAS/SATA Controller driver
#XXX pointer/int warnings
nodevice     pst         # Promise Supertrak SX6000
nodevice      twe         # 3ware ATA RAID
# NVM Express (NVMe) support
nodevice      nvme            # base NVMe driver
nodevice      nvd         # expose NVMe namespaces as disks, depends on nvme
# Intel Volume Management Device (VMD) support
nodevice      vmd         # base VMD device
nodevice      vmd_bus         # bus for VMD children
# Parallel port
nodevice      ppc
nodevice      ppbus           # Parallel port bus (required)
nodevice      lpt         # Printer
nodevice      ppi         # Parallel port interface device
nodevice     vpo         # Requires scbus and da
nodevice      puc         # Multi I/O cards and multi-channel UARTs
# PCI/PCI-X/PCIe Ethernet NICs that use iflib infrastructure
nodevice      iflib
nodevice      em          # Intel PRO/1000 Gigabit Ethernet Family
nodevice      ix          # Intel PRO/10GbE PCIE PF Ethernet
nodevice      ixv         # Intel PRO/10GbE PCIE VF Ethernet
nodevice      ixl         # Intel 700 Series Physical Function
nodevice      iavf            # Intel Adaptive Virtual Function
nodevice      ice         # Intel 800 Series Physical Function
nodevice      vmx         # VMware VMXNET3 Ethernet
nodevice      axp         # AMD EPYC integrated NIC
# PCI Ethernet NICs.
nodevice      bxe         # Broadcom NetXtreme II BCM5771X/BCM578XX 10GbE
nodevice      le          # AMD Am7900 LANCE and Am79C9xx PCnet
nodevice      ti          # Alteon Networks Tigon I/II gigabit Ethernet
# Nvidia/Mellanox Connect-X 4 and later, Ethernet only
# mlx5ib requires ibcore infra and is not included by default
nodevice      mlx5            # Base driver
nodevice      mlxfw           # Firmware update
nodevice      mlx5en          # Ethernet driver
# PCI Ethernet NICs that use the common MII bus controller code.
# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs!
device      miibus          # MII bus support
nodevice      ae          # Attansic/Atheros L2 FastEthernet
nodevice      age         # Attansic/Atheros L1 Gigabit Ethernet
nodevice      alc         # Atheros AR8131/AR8132 Ethernet
nodevice      ale         # Atheros AR8121/AR8113/AR8114 Ethernet
nodevice      bce         # Broadcom BCM5706/BCM5708 Gigabit Ethernet
nodevice      bfe         # Broadcom BCM440x 10/100 Ethernet
nodevice      bge         # Broadcom BCM570xx Gigabit Ethernet
nodevice      cas         # Sun Cassini/Cassini+ and NS DP83065 Saturn
nodevice      dc          # DEC/Intel 21143 and various workalikes
nodevice      et          # Agere ET1310 10/100/Gigabit Ethernet
nodevice      fxp         # Intel EtherExpress PRO/100B (82557, 82558)
nodevice      gem         # Sun GEM/Sun ERI/Apple GMAC
nodevice      jme         # JMicron JMC250 Gigabit/JMC260 Fast Ethernet
nodevice      lge         # Level 1 LXT1001 gigabit Ethernet
nodevice      msk         # Marvell/SysKonnect Yukon II Gigabit Ethernet
nodevice      nfe         # nVidia nForce MCP on-board Ethernet
nodevice      nge         # NatSemi DP83820 gigabit Ethernet
device      re          # RealTek 8139C+/8169/8169S/8110S
device      rl          # RealTek 8129/8139
nodevice      sge         # Silicon Integrated Systems SiS190/191
nodevice      sis         # Silicon Integrated Systems SiS 900/SiS 7016
nodevice      sk          # SysKonnect SK-984x & SK-982x gigabit Ethernet
nodevice      ste         # Sundance ST201 (D-Link DFE-550TX)
nodevice      stge            # Sundance/Tamarack TC9021 gigabit Ethernet
nodevice      vge         # VIA VT612x gigabit Ethernet
nodevice      vr          # VIA Rhine, Rhine II
nodevice      xl          # 3Com 3c90x (``Boomerang'', ``Cyclone'')
# Sound support
device      sound           # Generic sound driver (required)
nodevice      snd_cmi         # CMedia CMI8338/CMI8738
nodevice      snd_csa         # Crystal Semiconductor CS461x/428x
nodevice      snd_emu10kx     # Creative SoundBlaster Live! and Audigy
nodevice      snd_es137x      # Ensoniq AudioPCI ES137x
device      snd_hda         # Intel High Definition Audio
device      snd_ich         # Intel, NVidia and other ICH AC'97 Audio
nodevice      snd_via8233     # VIA VT8233x Audio
# HyperV drivers and enhancement support
nodevice      hyperv          # HyperV drivers
# NOTE: XENHVM depends on xenpci.  They must be added or removed together.
nodevice      xenpci          # Xen HVM Hypervisor services driver
nodevice      iwifw
nodevice      iwnfw
 
nooption     COMPAT_FREEBSD4        # Compatible with FreeBSD4
nooption     COMPAT_FREEBSD5        # Compatible with FreeBSD5
nooption     COMPAT_FREEBSD6        # Compatible with FreeBSD6
nooption     COMPAT_FREEBSD7        # Compatible with FreeBSD7
nooption     COMPAT_FREEBSD9        # Compatible with FreeBSD9
nooption     COMPAT_FREEBSD10    # Compatible with FreeBSD10
 
nooption     COMPAT_FREEBSD32    # Compatible with i386 binaries
#nooption     COMPAT_FREEBSD11    # Compatible with FreeBSD11
#nooption     COMPAT_FREEBSD12    # Compatible with FreeBSD12
 
options    IPFIREWALL            # enables IPFW
options    IPFIREWALL_VERBOSE        # enables logging for rules with log keyword
options    IPFIREWALL_VERBOSE_LIMIT=5    # limits number of logged packets per-entry
options    IPFIREWALL_DEFAULT_TO_ACCEPT # sets default policy to pass what is not explicitly denied
HOST:root: /root/Root/Make_kernel #

Note the, include GENERIC , nooptions and nodevice.

kr0m · Sep 6, 2022

Thak you very much for your tips all are very valuable, finally i have fine tunned my own kernel to the minimal expression, i started with MINIMAL and compiled only necessary modules, i am so satisfied with the result:

du -h /boot/kernel.kr0m-minimal/kernel
6.8M /boot/kernel.kr0m-minimal/kernel

du -sch /boot/kernel.kr0m-minimal/*.ko
209K total

I understand that support with custom kernels are more troubleshooting, so if i detect any anomally i will reboot with GENERIC before complainig about anything or opening any forum thread.