Hi gang!
Editorial
I've been using FreeBSD forroughly 8 or 10 years now I think (edit =>) nope: for 12+ years and trust me: that's hardly an impressive feat because I know of people who are even "older" than me when it comes to being a "beastie". So the one thing that has always impressed me is that FreeBSD is one of those projects that truly elevates open source as a whole by not merely providing its source code... nah: it also does this in a way which makes it relatively easy on us users & admins to (re)build this mighty beast.
Even though there's no direct need I'm actually rebuilding my copy at the time of writing for the sole reason of "because I can..." (and because I think it's a fun experience!).
Now.... this whole thing is pretty well documented I think but at the same time I also think it's fair to say that if you're still somewhat new some things will probably only start making more sense after your first go (and maybe with a bit of trial & error?).
So I figured... why don't we look into this process from another perspective... mine?
I hope you enjoy, and that this can be useful for some of you.
Simple reasoning: you'll always have the most up to date version of the OS. Oh, and not just that: you'll get everything => a few Git commands are all you need to switch to a previous version, or... a bleeding edge version perhaps?
This, ladies and gentlemen, is Git being used to its true potential if you ask me!
Now, before we begin I suggest that you try to reserve a dedicated file system for the source tree. This is relatively easy with ZFS, but I could imagine that it'll be a bit trickier with UFS. Still =>
Why you'd want to do this? Safety & comfort:
First: because it's mostly (ASCII) source code we're dealing with here I made sure to turn compression on which will save me some precious diskspace. More importantly though: notice the readonly status? So even if someone were to log onto my server to snoop around a bit and maybe even find an exploit to bypass local security... then there's still only so much they could do here because it's not just an issue of permission bits and what not... no, this whole filesystem (or "dataset") is in full lockdown.
Best part? You can even keep it this way during building, but now I'm getting ahead of myself, sorry
So for now, execute this command:
When the download ("cloning") is done you can lock things down using:
First we need to realize that we're dealing with 2 parts rolled into 1 source tree: the base system, and the kernel.
The base system
This part is relatively easy: consult src.conf(5) to familiarize yourself with all the options at your disposal to customize the base system. You can place the options you need in /etc/src.conf and be done with it. Careful here: this is still related to the build process, as such you need to honor the syntax as expected by make: "option=".
So, for example... let's say for the sake of argument that we don't want to include the BSD installer, we don't want to keep freebsd-update around, if pkg is not around it should not be asking us to "just" download it and most of all: we can't have any fun here according to the manager (duh!) so "absolutely no games!".
We could use something like this:
(I can already hear the Karen manager): "Why doesn't it give me any tips during login anymore?!" Us: "Well, you told us no more games, remember? So... no more /usr/bin/fortune." "WHAT?!! I want to see your manage... wait.. uhm nvm... <mutters>".
When in doubt: don't change a working setup, ok?
Important: if you're upgrading to a newer version then obviously the local src.conf(5) manualpage may not include all the new options (?) at our disposal. So if you're upgrading then be sure to check the latest ("relevant") manualpage, use:
The kernel... there be dragons here!
The next part is going to be a bit more tricky... because now we're going to focus on the very heart of our operating system: the kernel. You see... there are a few very important details we need to keep in mind right now: the kernel is the literal interface between our hardware and FreeBSD itself.
Meaning? If we don't make darn sure that our upcoming kernel knows about and/or supports our current hardware then we could end up with a broken ("non-bootable") system. That's no good!
So... we have 2 things we need to consider: kernel modules, and embedded support. AND... the mixture between them.
Identifying our system
Considering that we're using our favorite operating system right now it's safe to assume that, well, "things work(ed)". So we start with "cheating" by taking a closer look at what our current kernel has found so far during the boot process. Use the dmesg(8) command and/or look at /var/log/dmesg.today has to say to get more info on this.
Now, I'm pretty sure that just staring at those logs at this time won't be of much use to you, but don't worry... it'll make more sense soon enough, and obviously we're doing this together right now.
See... this leads us to step 2: the actual kernel configuration:
Notice the mention of GENERIC and amd64 up there? And what do you know....
Fun fact: as one could expect the kernel source is obviously part of the main source tree, and its location is /usr/src/sys (notice my prompt above?). It has configuration options on a per-architecture basis (!) and as I showed you earlier.. I'm using amd64 which leads us to this specific location: sys => amd64 => conf. We already know that our kernel is GENERIC so guess what? =>
This is what defines our current kernel, so.. obviously this should also provide a good foundation to build our own kernel against. Now, once again we have multiple options...
We could reference this GENERIC config file by including it in our own configuration and then simply remove any options and/or devices which we don't want to use.
Or...
We could also copy this GENERIC file and then just edit it by removing (or commenting out) everything that we don't need.
Both methods have their pro's and cons => Referencing (or "including") will ensure that any upcoming additions will automatically become part of our current setup. Safety first? Of course that argument can easily be turned around as well: copying and editing our now dedicated (and static) config file could also ensure that no "new bloat" finds its way into our upcoming kernel config by default.
I've used both methods, there is no "right" or "wrong" here.
But whatever you do I strongly suggest to copy the GENERIC for this version out of the way, and to also keep your own custom config out of the source tree as well. The 'why' should be obvious enough: it'd be a waste to back up your source tree because one simple Git command is enough to reproduce it. Your config file otoh... not so much. Just link your own config file back into the main source tree, easy.
You can reference this on the command line, or... add it to /etc/make.conf using KERNCONF. Please check make.conf(5) for more on this. No worries: I'll refer to this again later on.
I prefer keeping both files somewhere within /root which is also part of my active backup:
Oh, and in case you're wondering why you'd ever want to keep the original GENERIC config? Easy: /usr/bin/diff. When 14.3 rolls out all I have to do is 'diff' both GENERIC default configs and I'll know right away what differences I have to keep in mind.
Sure: I'm well aware that it'll also be well documented, but.. since we have the source, why not use it?
Remember me mentioning dmesg(8) and how it would all make sense? Why don't we put this to the test...
Notice cd0 and da0 up there?
Now let's take a look at a part of my kernel config file:
See what I mean?
Cross referencing is the name of the game here or... when in doubt: just don't remove stuff?
However, this is only one part of our kernel configuration. For example: you won't find any references to ZFS in here.
Kernel modules
This leads us to part 2 of our kernel configuration: kernel modules.
Once again, we can 'cheat' by peeking at what we currently have set up:
And once again: when in doubt... just leave it be and it'll work out.
Of course that's not why we're doing this. So we have 2 solid options to control our upcoming modules: MODULES_OVERRIDE or WITHOUT_MODULES. These options can go into /etc/make.conf and either define that you only want to include a specific selection of kernel modules ("MODULES_OVERRIDE") or that you want to build everything except for those defined in "WITHOUT_MODULES".
It's a make definition so include an '=' and a quoted list of all the modules which you either want to include or exclude. This is my setup:
Now, I suppose I made this sound easy but ... as I said earlier: there be dwagonz here. As you can see above there are two specific modules loaded: dtrace.ko and opensolaris.ko. Oh, and: zfs.ko. THREE specific modules loaded! Fun fact: 'opensolaris' is part of ZFS, not DTrace, despite both features having the same origin. Oh, and: problem! It's still a separate kernel module.
If you want to fully tune this then I suggest many studies of /usr/src/sys/modules, as well as the use of
But seriously, isn't this kinda exciting? Everything you need is presented right before you, you just need to use the right commands and utilize the right info at your disposal.
But what ... if => what if we had a few drinks, got super smart (tm) and thus we "obviously" knew what the cause of the problem was. So we re-build our kernel, installed it again (note: also generating a new kernel.old(!)) and then... it still fails.
"oopsie", now we don't have any option of booting our system left at all.
...unless... => one of the things which I always do on my servers is to grab kernel.txz, extract it into /boot/kernel.bck and then add this line to my /boot/loader.conf: "kernels="kernel kernel.old kernel.bck". I also make sure to apply
Point being: no matter how much I mess up I'll always have a clean kernel which I can use to boot from so that I can try again. And this isn't just about failed builds. What if you optimized your kernel for your hardware, then a component b0rk down and got replaced. Are you really going to build a whole new kernel while everyone is waiting, or would you rather use your failsave, and customize things later, without all the heat on you?
...
Hello?! build, build... boot, boot... get to it already, what's keeping you?!
Wait, you're expecting me to tell you what commands you need to use? Why ask me?!! oh wait.. it's probably because of this guide of mine, eh? oopsie, my bad!
More seriously => yah, I am poking some fun here but I'm also dead serious at the same time. About what? About not having a clue from mind about what commands to use is what... because why bother?
When I say "elevating open source" I wasn't only talking about the stuff above... check THIS out:
Seriously, isn't this simply superior by design? Last time I checked it mentioned mergemaster up there
Don't bother with trying to remember commands, just remember to check out the Makefile as soon as you're ready! Follow by (good) example, easy!
And there you have it....
Coming up in a future guide: some generic tips & tricks for the ports collection... have you ever peeked around /usr/ports/Mk? If not, then you're missing out big time, more about that later.
For now... thanks to the mods for approving ((?) I can only hope!), thanks to you dear random reader for reading and I hope this was useful.
Editorial
I've been using FreeBSD for
Even though there's no direct need I'm actually rebuilding my copy at the time of writing for the sole reason of "because I can..." (and because I think it's a fun experience!).
Now.... this whole thing is pretty well documented I think but at the same time I also think it's fair to say that if you're still somewhat new some things will probably only start making more sense after your first go (and maybe with a bit of trial & error?).
So I figured... why don't we look into this process from another perspective... mine?
I hope you enjoy, and that this can be useful for some of you.
Step 1 => Getting (and protecting!) the source code
If you install FreeBSD from a local image then there's a good chance that it also provided you with the option to install the source code as well (referring to src.txz). Now, there's nothing really wrong with this approach (!) but even so... I suggest not relying on such archives but instead to use the actual official Git repository.Simple reasoning: you'll always have the most up to date version of the OS. Oh, and not just that: you'll get everything => a few Git commands are all you need to switch to a previous version, or... a bleeding edge version perhaps?
Code:
peter@bsd:/usr/src $ git branch -r
origin/releng/13.5
origin/releng/14.0
origin/releng/14.1
origin/releng/14.2
[SNIP]
origin/stable/12
origin/stable/13
origin/stable/14Now, before we begin I suggest that you try to reserve a dedicated file system for the source tree. This is relatively easy with ZFS, but I could imagine that it'll be a bit trickier with UFS. Still =>
# zfs create -o mountpoint=/usr/src zroot/src, assuming of course that your main ZFS pool/dataset is zroot.Why you'd want to do this? Safety & comfort:
Code:
peter@bsd:/home/peter $ zfs get compression,mountpoint,readonly,used zroot/src
NAME PROPERTY VALUE SOURCE
zroot/src compression on local
zroot/src mountpoint /usr/src local
zroot/src readonly on local
zroot/src used 2.61G -Best part? You can even keep it this way during building, but now I'm getting ahead of myself, sorry
So for now, execute this command:
# git clone https://git.freebsd.org/src.git /usr/src. Be sure to install devel/git if you haven't already!When the download ("cloning") is done you can lock things down using:
# zfs set readonly=on zroot/src. Or... remount the filesystem if you're using UFS: # mount -ur /usr/src.Step #2 => Customizing the upcoming build process
So now that we have our source tree (and locked it down!) it's time to start thinking about how we're going to put this to some good use.First we need to realize that we're dealing with 2 parts rolled into 1 source tree: the base system, and the kernel.
The base system
This part is relatively easy: consult src.conf(5) to familiarize yourself with all the options at your disposal to customize the base system. You can place the options you need in /etc/src.conf and be done with it. Careful here: this is still related to the build process, as such you need to honor the syntax as expected by make: "option=".
So, for example... let's say for the sake of argument that we don't want to include the BSD installer, we don't want to keep freebsd-update around, if pkg is not around it should not be asking us to "just" download it and most of all: we can't have any fun here according to the manager (duh!) so "absolutely no games!".
We could use something like this:
Code:
WITHOUT_BSDINSTALL=
WITHOUT_FREEBSD_UPDATE=
WITHOUT_PKGBOOTSTRAP=
WITHOUT_GAMES=When in doubt: don't change a working setup, ok?
Important: if you're upgrading to a newer version then obviously the local src.conf(5) manualpage may not include all the new options (?) at our disposal. So if you're upgrading then be sure to check the latest ("relevant") manualpage, use:
man -M /usr/src/share/man src.conf (<= c'mon... isn't this level of design just awesome?!).The kernel... there be dragons here!
The next part is going to be a bit more tricky... because now we're going to focus on the very heart of our operating system: the kernel. You see... there are a few very important details we need to keep in mind right now: the kernel is the literal interface between our hardware and FreeBSD itself.
Meaning? If we don't make darn sure that our upcoming kernel knows about and/or supports our current hardware then we could end up with a broken ("non-bootable") system. That's no good!
So... we have 2 things we need to consider: kernel modules, and embedded support. AND... the mixture between them.
Identifying our system
Considering that we're using our favorite operating system right now it's safe to assume that, well, "things work(ed)". So we start with "cheating" by taking a closer look at what our current kernel has found so far during the boot process. Use the dmesg(8) command and/or look at /var/log/dmesg.today has to say to get more info on this.
Now, I'm pretty sure that just staring at those logs at this time won't be of much use to you, but don't worry... it'll make more sense soon enough, and obviously we're doing this together right now.
See... this leads us to step 2: the actual kernel configuration:
Code:
peter@bsd:/home/peter $ uname -a
FreeBSD bsd.intranet.lan 14.2-RELEASE FreeBSD 14.2-RELEASE releng/14.2-n269506-c8918d6c7412 GENERIC amd64
Code:
peter@bsd:/usr/src/sys/amd64/conf $ ls
DEFAULTS GENERIC-KCSAN GENERIC.hints LINT-NOINET6 MINIMALUP
FIRECRACKER GENERIC-KMSAN HYPER-KERN@ LINT-NOIP NOTES
GENERIC GENERIC-MMCCAM LINT MINIMAL
GENERIC-KASAN GENERIC-NODEBUG LINT-NOINET MINIMAL-NODEBUG
Code:
peter@bsd:/usr/src/sys/amd64/conf $ less GENERIC
# GENERIC -- Generic kernel configuration file for FreeBSD/amd64
#
# For more information on this file, please read the config(5) manual page,
# and/or the handbook section on Kernel Configuration Files:
[CLIP]
makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols
makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support
options SCHED_ULE # ULE scheduler
options NUMA # Non-Uniform Memory Architecture support
options PREEMPTION # Enable kernel thread preemptionWe could reference this GENERIC config file by including it in our own configuration and then simply remove any options and/or devices which we don't want to use.
Or...
We could also copy this GENERIC file and then just edit it by removing (or commenting out) everything that we don't need.
Both methods have their pro's and cons => Referencing (or "including") will ensure that any upcoming additions will automatically become part of our current setup. Safety first? Of course that argument can easily be turned around as well: copying and editing our now dedicated (and static) config file could also ensure that no "new bloat" finds its way into our upcoming kernel config by default.
I've used both methods, there is no "right" or "wrong" here.
But whatever you do I strongly suggest to copy the GENERIC for this version out of the way, and to also keep your own custom config out of the source tree as well. The 'why' should be obvious enough: it'd be a waste to back up your source tree because one simple Git command is enough to reproduce it. Your config file otoh... not so much. Just link your own config file back into the main source tree, easy.
You can reference this on the command line, or... add it to /etc/make.conf using KERNCONF. Please check make.conf(5) for more on this. No worries: I'll refer to this again later on.
I prefer keeping both files somewhere within /root which is also part of my active backup:
Code:
# ls -lh
total 50
-rw-rw-r-- 1 root wheel 15K Apr 5 16:11 generic.14_2
-rw-rw-r-- 1 root wheel 6.2K Apr 11 22:16 hyper_kernelSure: I'm well aware that it'll also be well documented, but.. since we have the source, why not use it?
Remember me mentioning dmesg(8) and how it would all make sense? Why don't we put this to the test...
Code:
root@bsd:~ # dmesg | grep "relevant parts" | less (<= totally (kinda) fictional)
VT(efifb): resolution 1024x768
Hyper-V Version: 10.0.26100 [SP3]
Features=0x2e7f<VPRUNTIME,TMREFCNT,SYNIC,SYNTM,APIC,HYPERCALL,VPINDEX,REFTSC,IDLE,TMFREQ>
PM Features=0x0 [C2]
Features3=0xbed7b2<DEBUG,XMMHC,IDLE,NUMA,TMFREQ,SYNCMC,CRASH,NPIEP>
<SNIP>
Trying to mount root from zfs:zroot []...
cd0 at storvsc0 bus 0 scbus0 target 0 lun 1
cd0: <Msft Virtual DVD-ROM 1.0> Removable CD-ROM SPC-3 SCSI device
cd0: 300.000MB/s transfers
cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed
da0 at storvsc0 bus 0 scbus0 target 0 lun 0
da0: <Msft Virtual Disk 1.0> Fixed Direct Access SPC-3 SCSI device
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 163840MB (335544320 512 byte sectors)
hvhid0: <Hyper-V HID device> on vmbus0Now let's take a look at a part of my kernel config file:
Code:
# ATA/SCSI peripherals
device da # Direct Access (disks)
device cd # CDCross referencing is the name of the game here or... when in doubt: just don't remove stuff?
However, this is only one part of our kernel configuration. For example: you won't find any references to ZFS in here.
Kernel modules
This leads us to part 2 of our kernel configuration: kernel modules.
Once again, we can 'cheat' by peeking at what we currently have set up:
Code:
root@bsd:~/kernel.etc # kldstat
Id Refs Address Size Name
1 51 0xffffffff80200000 1f3c6c0 kernel
2 1 0xffffffff8213d000 5da658 zfs.ko
3 1 0xffffffff82719000 35e0 dtraceall.ko
4 2 0xffffffff8271d000 4258 profile.ko
5 11 0xffffffff82722000 1e2a8 opensolaris.ko
6 11 0xffffffff82741000 5dc50 dtrace.ko
7 2 0xffffffff8279f000 194d8 systrace_freebsd32.ko
8 2 0xffffffff827b9000 194f0 systrace.ko
9 2 0xffffffff827d3000 3e68 sdt.ko
10 2 0xffffffff827d7000 8538 kinst.ko
11 2 0xffffffff827e0000 5fdc8 fasttrap.ko
12 2 0xffffffff82840000 5da0 fbt.ko
13 2 0xffffffff82846000 62a0 dtnfscl.ko
14 2 0xffffffff8284d000 3518 dtmalloc.ko
15 1 0xffffffff83313000 21b8 hv_hid.ko
16 1 0xffffffff83316000 21e8 hms.ko
17 1 0xffffffff83319000 30a8 hidmap.ko
18 1 0xffffffff8331d000 3560 fdescfs.koOf course that's not why we're doing this. So we have 2 solid options to control our upcoming modules: MODULES_OVERRIDE or WITHOUT_MODULES. These options can go into /etc/make.conf and either define that you only want to include a specific selection of kernel modules ("MODULES_OVERRIDE") or that you want to build everything except for those defined in "WITHOUT_MODULES".
It's a make definition so include an '=' and a quoted list of all the modules which you either want to include or exclude. This is my setup:
Code:
peter@bsd:/home/peter $ tail -5 /etc/make.conf
## Kernel options
KERNCONF="HYPER-KERN"
MODULES_OVERRIDE="dtrace cd9660 dummynet fdescfs firmware fusefs hid hyperv if_bridge if_gre if_tuntap if_vlan ipsec kdbmux nfsd netgraph netwatch netlink opensolaris pf pflog procfs pseudofs syscons tmpfs ufs usb x86bios zfs"If you want to fully tune this then I suggest many studies of /usr/src/sys/modules, as well as the use of
dmesg and kldstat.But seriously, isn't this kinda exciting? Everything you need is presented right before you, you just need to use the right commands and utilize the right info at your disposal.
Step #2b => Setting up our failsave
As much as I want to believe that it'll all work out, we have to keep the possibility in mind that our efforts could also easily blow up in our face. Then what? Sure: if we install a new kernel then our old kernel gets copied to kernel.old, easy!But what ... if => what if we had a few drinks, got super smart (tm) and thus we "obviously" knew what the cause of the problem was. So we re-build our kernel, installed it again (note: also generating a new kernel.old(!)) and then... it still fails.
"oopsie", now we don't have any option of booting our system left at all.
...unless... => one of the things which I always do on my servers is to grab kernel.txz, extract it into /boot/kernel.bck and then add this line to my /boot/loader.conf: "kernels="kernel kernel.old kernel.bck". I also make sure to apply
# chflags schg /etc/boot/kernel.bck/*", but that's a story for another guide Point being: no matter how much I mess up I'll always have a clean kernel which I can use to boot from so that I can try again. And this isn't just about failed builds. What if you optimized your kernel for your hardware, then a component b0rk down and got replaced. Are you really going to build a whole new kernel while everyone is waiting, or would you rather use your failsave, and customize things later, without all the heat on you?
Step #3 => It's time to build!
We edited /etc/make.conf, we created /etc/src.conf and I'm assuming you set up your kernel config, so now it's time to start! So get to it!...
Hello?! build, build... boot, boot... get to it already, what's keeping you?!
Wait, you're expecting me to tell you what commands you need to use? Why ask me?!! oh wait.. it's probably because of this guide of mine, eh?
oopsie, my bad!More seriously => yah, I am poking some fun here but I'm also dead serious at the same time. About what? About not having a clue from mind about what commands to use is what... because why bother?
When I say "elevating open source" I wasn't only talking about the stuff above... check THIS out:
Code:
root@bsd:/usr/src # less Makefile
(scroll down! =>)
# For individuals wanting to upgrade their sources (even if only a
# delta of a few days):
#
# 1. `cd /usr/src' (or to the directory containing your source tree).
# 2. `make buildworld'
# 3. `make buildkernel KERNCONF=YOUR_KERNEL_HERE' (default is GENERIC).
# 4. `make installkernel KERNCONF=YOUR_KERNEL_HERE' (default is GENERIC).
# [steps 3. & 4. can be combined by using the "kernel" target]
# 5. `reboot' (in single user mode: boot -s from the loader prompt).
# 6. `etcupdate -p'
# 7. `make installworld'
# 8. `etcupdate -B'
# 9. `make delete-old'
# 10. `reboot'
# 11. `make delete-old-libs' (in case no 3rd party program uses them anymore)
#Don't bother with trying to remember commands, just remember to check out the Makefile as soon as you're ready! Follow by (good) example, easy!
Bonus => speed up by using (c)cache!
Now... that was a nice sales pitch but truth be told: the speedup will only happen once you need to rebuild your stuff again. Even so... devel/ccache can be a solid option to help you reduce rebuild times. Just make sure to enable this in your /etc/make.conf by adding: "WITH_CCACHE_BUILD=yes". See also /usr/local/share/doc/ccache/cache-howto-freebsd.txt.And there you have it....
In conclusion
How you build your own custom awesome Unix-like operating system, powered by FreeBSD. With failsave!Coming up in a future guide: some generic tips & tricks for the ports collection... have you ever peeked around /usr/ports/Mk? If not, then you're missing out big time, more about that later.
For now... thanks to the mods for approving ((?) I can only hope!), thanks to you dear random reader for reading and I hope this was useful.