Check out my custom built FreeBSD

K

kpect

Hi.

If someone would like to check out my custom built FreeBSD 11.2, welcome)
https://mega.nz/#!uq4jzDpA!g7gBWRjbzT-w2uMT3LO62I2S93etwZep6p2odepGTOk
- Built for IvyBridge and higher
- Most of server services are removed, but could be installed from ports if required
- Locales are only for Russian and English
- Added rsync, mksh, mandoc, doas, beadm, rdate, pkg & portmaster
- rc.conf, sysctl.conf, loader.conf, make.conf, src.conf are customized for Thinkpad T430 and my taste
- ... and many more
 
Why? Does it really save memory, run faster, or have fewer bugs?

By the way, I have done lots of these types of things myself: building something from scratch, or using a complicated solution when a simply one would suffice. My answer would be: Because I like doing it. I don't suffer from OCD, I enjoy it.
 
kpect said:
Hi.

If someone would like to check out my custom built FreeBSD 11.2, welcome)
https://mega.nz/#!uq4jzDpA!g7gBWRjbzT-w2uMT3LO62I2S93etwZep6p2odepGTOk
- Built for IvyBridge and higher
- Most of server services are removed, but could be installed from ports if required
- Locales are only for Russian and English
- Added rsync, mksh, mandoc, doas, beadm, rdate, pkg & portmaster
- rc.conf, sysctl.conf, loader.conf, make.conf, src.conf are customized for Thinkpad T430 and my taste
- ... and many more
Click to expand...

Very cool. Are you planning on documenting your procedure? I have wanted to build a custom release for a while but I am confused about a few things and it has prevented me from doing so yet. Specifically, How does one pre-install packages onto the release before creating the ISO? I have read release(7), and have an idea, but need some clarification.
 
alexseitsinger said:
Very cool. Are you planning on documenting your procedure? I have wanted to build a custom release for a while but I am confused about a few things and it has prevented me from doing so yet. Specifically, How does one pre-install packages onto the release before creating the ISO? I have read release(7), and have an idea, but need some clarification.
Click to expand...

Hi.
I haven't been tought to do the stuff the right way so came to everything by my own. I edit the base sources using sed, the build script you can find here:
https://gist.github.com/kpect/c1a103bd0b0294e7e966243c694ed156


The code that you are interested in is in /usr/src/release/Makefile:

Code: 
    cp -f /etc/resolv.conf ${.OBJDIR}/${DISTDIR}/base/etc
    cp -f /var/run/ld-elf.so.hints ${.OBJDIR}/${DISTDIR}/base/var/run
    mkdir ${.OBJDIR}/${DISTDIR}/base/var/cache/distfiles
    for port in ports-mgmt/pkg ports-mgmt/portmaster shells/mksh net/rsync security/doas textproc/mdocml sysutils/rdate sysutils/beadm; \
        do make -C ${PORTSDIR}/$${port} DESTDIR=${.OBJDIR}/${DISTDIR}/base DESTDIR_ENV_LIST=STRIPBIN DESTDIR_MOUNT_LIST='PORTSDIR DISTDIR:/var/cache/distfiles:/var/cache/distfiles PORT_DBDIR:/var/db/ports' install clean || exit; \
    done
    rm -rf ${.OBJDIR}/${DISTDIR}/base/etc/resolv.conf \
        ${.OBJDIR}/${DISTDIR}/base/var/run/ld-elf.so.hints \
        ${.OBJDIR}/${DISTDIR}/base/var/cache/distfiles \
        ${.OBJDIR}/${DISTDIR}/base/usr/obj/*
    echo -e 'AUTOCLEAN = true;\nCONSERVATIVE_UPGRADE = false;' > ${.OBJDIR}/${DISTDIR}/base/usr/local/etc/pkg.conf
    sed -E '/^(#|$$)/d' < ${.OBJDIR}/${DISTDIR}/base/usr/local/etc/pkg.conf.sample >> ${.OBJDIR}/${DISTDIR}/base/usr/local/etc/pkg.conf
    echo -e 'NO_BACKUP=Bopt\nALWAYS_SCRUB_DISTFILES=dopt\nPM_VERBOSE=vopt\nPM_PACKAGES_BUILD=pmp_build' > ${.OBJDIR}/${DISTDIR}/base/usr/local/etc/portmaster.rc
    echo -e 'PM_DEL_BUILD_ONLY=pm_dbo\nPM_NO_CONFIRM=pm_no_confirm\nPM_NO_TERM_TITLE=pm_no_term_title' >> ${.OBJDIR}/${DISTDIR}/base/usr/local/etc/portmaster.rc
    echo -e 'manpath /usr/share/man\nmanpath /usr/local/man\nmanpath /usr/X11R6/man\nmanpath /usr/share/openssl/man' > ${.OBJDIR}/${DISTDIR}/base/etc/man.conf
    echo '/usr/local/sbin/rdate -4s pool.ntp.org >/dev/null 2>&1' > ${.OBJDIR}/${DISTDIR}/base/usr/local/etc/periodic/daily/500.rdate
    chmod 555 ${.OBJDIR}/${DISTDIR}/base/usr/local/etc/periodic/daily/500.rdate
    echo 'permit nopass keepenv :wheel' > ${.OBJDIR}/${DISTDIR}/base/usr/local/etc/doas.conf
    chroot ${.OBJDIR}/${DISTDIR}/base pkg autoremove -y && /usr/local/sbin/makewhatis
 
Beastie said:
I haven't tried this, but theoretically you should be able to mount your ISO (mdconfig(8)) and use pkg(8)'s -r/--rootdir option:
Code: 
-r    <root directory>, --rootdir <root directory>
         pkg will install all packages within the specified    <root
         directory>.
Click to expand...

I'm not sure that this would work because the ISO comes with an installation filesystem that wraps the actual archives (base.txz, src.txz, etc.) that get installed. According to your suggestion, I think installing the packages onto the mounted ISO would only result in the installation filesystem having the packages, not actual filesystem to be installed on the computer.
 
alexseitsinger said:
According to your suggestion, I think installing the packages onto the mounted ISO would only result in the installation filesystem having the packages
Click to expand...
Yes, precisely. I must have misunderstood your question then.
alexseitsinger said:
How does one pre-install packages onto the release before creating the ISO?
Click to expand...


alexseitsinger said:
not actual filesystem to be installed on the computer.
Click to expand...
If the above is what you're after, then you can include the .txz files in the ISO. Either let the user install them manually by issuing a pkg install command or install them automatically through some shell script that does that for them instead.
 
ralphbsz said:
Why? Does it really save memory, run faster, or have fewer bugs?
Click to expand...
  • Yes, It should run faster as It's compiled not for generic x86_64 but for specific CPU - IvyBridge and higher
  • Yes, It has fewer bugs as It doesn't include a bunch of code shipped by default
  • As for memory, if It doesn't suffice your requirements, buy some RAM or use something else.
 
lebarondemerde said:
kpect

This kind of custom build does not get too much attention in here since the vastly majority of users do they own custom build when they want one. :)
Click to expand...

I'm happy for all those capable users) But this build is for those who cannot do they own. It may be useful for someone...
 
I'm just going to be a devil's advocate:

kpect said:
Yes, It should run faster as It's compiled not for generic x86_64 but for specific CPU - IvyBridge and higher
Click to expand...
Did you measure it? In my experience, most code today is not CPI constrained, but memory bandwidth constrained. And changing the compile target doesn't change the memory access patterns significantly. And the operating system kernel is not at all CPU constrained; OS operations use a very small fraction of the overall CPU. I bet if you measure the overall performance of your system on realistic workload (not on a micro-benchmark that only exercises kernel functions), it would make no appreciable different.

Yes, It has fewer bugs not including a bunch of code which comes by default in base.
Click to expand...
But the bugs are in code which you wouldn't execute anyway. Who cares if the kernel you are running contains a driver for the foo-2000 card which you don't have, and that driver is buggy? Or contains a buggy implementation of the blatz file system, which you don't use? Code that is there but unused can have bugs, and you will not notice.

As for memory, if It doesn't suffice your requirements, buy some RAM or use something else.
Click to expand...
The kernel will become smaller when you do a custom build. But is the difference appreciable? I think the kernel itself has a memory footprint of a few dozen megabytes (not counting data areas). You might be able to shrink that by say a factor of two. Will a dozen MB make a different to a machine that has many GB?

Note that all the arguments I made above are not completely true; they are generalizations. But seriously, you should think about whether the savings of custom kernels are really worth your effort. And all the time you've invested in it, and will continue to invest after every upgrade.

On the other hand, the only way to actually measure those savings is to buold a custom kernel, and measure the performance, reliability and memory usage. Once you have done that, why go back?
 
ralphbsz said:
I'm just going to be a devil's advocate:


Did you measure it? In my experience, most code today is not CPI constrained, but memory bandwidth constrained. And changing the compile target doesn't change the memory access patterns significantly. And the operating system kernel is not at all CPU constrained; OS operations use a very small fraction of the overall CPU. I bet if you measure the overall performance of your system on realistic workload (not on a micro-benchmark that only exercises kernel functions), it would make no appreciable different.


But the bugs are in code which you wouldn't execute anyway. Who cares if the kernel you are running contains a driver for the foo-2000 card which you don't have, and that driver is buggy? Or contains a buggy implementation of the blatz file system, which you don't use? Code that is there but unused can have bugs, and you will not notice.


The kernel will become smaller when you do a custom build. But is the difference appreciable? I think the kernel itself has a memory footprint of a few dozen megabytes (not counting data areas). You might be able to shrink that by say a factor of two. Will a dozen MB make a different to a machine that has many GB?

Note that all the arguments I made above are not completely true; they are generalizations. But seriously, you should think about whether the savings of custom kernels are really worth your effort. And all the time you've invested in it, and will continue to invest after every upgrade.

On the other hand, the only way to actually measure those savings is to buold a custom kernel, and measure the performance, reliability and memory usage. Once you have done that, why go back?
Click to expand...

I could spend some time and reply all you arguments above, but just don't want to waste time explaining & arguing. The thing I don't understand is: what are you advocating for? For using default FreeBSD release? ... You're free to do so! The thread is for those who may find the build useful, or may implement something in their OS. I, for example, lacked this information when started writing the build script. You, lebarondemerde etc. just waste your time in this thread, no?
 
At the end of the day, you are asking people to run an O/S that you have customized yourself. Why would anybody do this rather than use one downloaded from a trusted source. You could put keyloggers or any kind of malware/rootkit into that.
 
xtaz said:
At the end of the day, you are asking people to run an O/S that you have customized yourself. Why would anybody do this rather than use one downloaded from a trusted source. You could put keyloggers or any kind of malware/rootkit into that.
Click to expand...

Virtualbox? KVM? Bhyve? or better just don't run it, at the end of the day.
For brave enough, you can build your own ISO using the build script provided above, preliminarily peruse the script for trojans))
 
kpect said:
The thing I don't understand is: what are you advocating for? For using default FreeBSD release?
Click to expand...

Good question, so let me be clear what I'm arguing for: For most situations, a custom kernel build (or a custom world build with different build configuration) will not make the system run faster, be more reliable, or give it more memory. It takes quite a bit of work, and has some risk of breaking things if one make mistakes. From the point of view of an engineering cost-benefit analysis, it is not a good investment: the cost and the risk outweighs the benefit.

Now, I completely understand that some people (including me) want to do it anyway. I used to always create custom kernels. From the early days of Linux (I started with kernel version 0.99.13) until about 2005 I used to download the source from kernel.org and build my own kernels regularly. And within the last 10 years on FreeBSD, I have hand-built a few applications that use a lot of CPU power for better optimization (LAME and the NetPBM toolkit). Honestly, I didn't measure the performance gain before and after, and it wasn't a huge improvement (the difference was not immediately obvious). I just want the average user to understand that this is a hobby, not a sensible engineering choice.

Again, this is not meant at criticism of your setup. It is fun and instructive to custom build everything.
 
I don't see any problem with making such customizations. Sure many of them are redundant and unnecessary if you look at the actual value of them but for many tinkering with an OS like FreeBSD is the learning tool to advance to bigger and better things.

This is just my personal opinion but I don't see FreeBSD as an end user operating system, I see it instead as a toolkit for system integrators (look at the pfSense project for example), IT pros, hobbyists etc. to build their operating system their way and either make real products based of FreeBSD or just learn about all the gory stuff that goes into to putting together a beast known as a modern operating system.
 
From a system hardening perspective, this process makes a lot of sense. By removing the unused binaries, and explicitly defining variables in the configuration files you effectively reduce the number of attack vectors. Also, by reducing the number of files on the system, it makes it slightly easier to navigate. But, for the most part, it makes for a good learning experience. Lots of us are really familiar with FreeBSD and enjoy using it, and by learning to build our own releases, we learn to use it better. I think custom configurations should be encouraged, so thanks again for sharing this kpect.
 
You must log in or register to reply here.
Back
Top