Hi gang!
Prerequisites: I am assuming that you know how to install software on FreeBSD and also have some basic understanding about FreeBSD jails.
Editorial
I'm going to be very honest here: I started disliking Linux for a while, and I've worked with it for a long time. For various reasons, but one of them being that it has turned into a commercial company driven project (you do realize that RedHat and Canonical (= company behind Ubuntu) are enterprise sized companies, right?). Arguments? Simple: systemd. A piece of shit software service which goes directly against the Unix philosophy and worse yet: makes Linux pretty much incompatible with others (you'll see later on). Isn't it annoying that because of projects like systemd (and several other changes) many people can't keep up with those developments anymore and need extra study or training? How convenient that RedHat also happens to be one of the bigger companies which provides Linux training! And no: not for free of course
There's a good reason why IBM bought them.
However...
Despite my dislike I'm also the first to acknowledge all the effort that goes into those projects. And although I definitely have my (bias) ideas about stuff such as systemd (and the distributions using it) fact of the matter is that many don't have a choice. Also: many projects are still the inclusive distributions they always were. Debian still gives life to many derivatives for example. And well, even Linux can be somewhat fun
1 - Linux compatibility
FreeBSD actually has a Linux compatibility layer which allows it to run Linux (ELF) binaries without a problem. Keep in mind though that this isn't full proof, but many things can be made to work without issues. You're going to need 2 things: support in the kernel and a userland to provide optionally required libraries (the latter isn't needed for the jail but I'd still install it anyway in case you're also going to use this setup for more purposes).
Kernel modules
If you run a GENERIC kernel then you already have everything you need. If you run a more customized kernel (like I do) then you'll need to make sure that it supports these kernel modules:
... to your /etc/rc.conf file. And there's also the issue of FreeBSD being able to dynamically load kernel modules whenever it needs one.
Userland
Linux binaries often have specific dependencies on libraries (and maybe other binaries) in which case you'll need some kind of userland which is what emulators/linux_base-c7 can provide. This will install a CentOS userland within the /compat/linux directory structure. Don't worry, it won't be a resource hog; on my system the (uncompressed) ZFS filesystem uses up around 254Mb.
Congratulations, you are now Linux compatible!
2 - Adding a full Linux userland
But which Linux? See, there is a problem...
As you may (or should!) know a Jail is nothing more but a FreeBSD userland which gets started by the kernel. You can even make it 'do' stuff by having the kernel initialize your rc.d structure:
And although the FreeBSD Linux compatibility layer can do a lot, it's not perfect. Crapola like systemd doesn't properly run on it for example (which I actually consider a pro
).
Fortunately there are still plenty of people who grasp and still honor the ideology which Linux once stood for and one of those projects is Devuan. What's Devuan? It's a Debian derivative which does not include systemd so we'll be fully able to use this as a Jail just fine.
So why all this interest for Debian even though we already have a CentOS userland you ask? Two reasons: Although /compat/linux does indeed provide a userland it's not complete. It was set up to emulate, not fully simulate. So don't expect an init.d structure.
But the second reason is much cooler... See: Debian's package system is, in my humble opinion, superior by design (within the context of Linux!). It's really neatly set up and unlike RPM it's even doable (though still a bit of a drag) to maintain your own packages. Something which you might want to do if you prefer to build your own software. Debian's packages are provided as both binaries and source, how convenient is that?
And they didn't stop there.. Eventually they created debootstrap which does just what the name implies: it will grab all the packages needed for a base system and set that up. So to get to the finale of this exciting build up: it has even been included in the FreeBSD ports collection as: sysutils/debootstrap, now that is cool I think. So quickly install this critter because we're going to need it!
Bootstrapping Devuan
If you're using ZFS then I strongly suggest to set up a dedicated filesystem. At the very least this will help you to keep track of the space your jail(s) are consuming:
This can also help for security measures, but that's beyond the scope of this guide. So, as seen above we'll be using /opt/jails/devuan in my examples.
Devuan is deviously clever See; every Debian distribution has a specific name. Debian's latest (at the time of writing!) is stretch whereas Devuan is called ascii. Now, the "problem" is that debootstrap uses scripts of the same name to help it separate between the distributions. And ascii isn't the same as stretch. Fortunately for us Devuan also honors the Debian standard distribution names: stable, testing and unstable. And unlike on FreeBSD Stable is honestly just that
Sidenote: Yes, I am aware that you can also specify an individual script when using debootstrap, but I like to keep things as simple as possible.
Important: We're going to perform the bootstrap process in 2 stages. Why? debootstrap can set up the hierarchy, but it won't be able to utilize things such as linprocfs or linsysfs, so we're going to have to set that up ourselves.
C'mon, doesn't this make your geekish powerlevels raise to beyond Super Saiyan levels? 
I dub this the devu-ascii-ha, but don't worry: yelling is not required
Anyway, this will process quite a list. What is happening is that debootstrap is retrieving all the packages that make up the Devuan base system and places those in our jail directory after which it'll extract them.
Jail preparations
Depending on your system this is probably going to take a while so now would be a good time to start preparing our upcoming jail by setting up our upcoming special Linux filesystems. Create a file called /etc/fstab.devuan and add the following:
If you're using ZFS like I do then you're going to need late because otherwise these filesystems will get mounted before ZFS is ready (in which case only your root filesystem would be available). Otherwise you obviously don't have to worry.
Around this time the first stage of debootstrap should be ready and I urge you to look around in your new Devuan environment. Points of interest:
Since we're here anyway we can now mount some required directories for the next stage:
So now we've "started" what I'd like to call the "shadow jail". Unfortunately we can't use debootstrap again to perform the second stage because it'll try to create devices (= assumption on my end) which isn't supported. And don't be fooled: despite having a bit of a userland we actually got nothing, the only thing debootstrap did was extract some packages. But we also want those packages to get registered so that we can fully utilize dpkg:
This will take a while and you'll see plenty of warnings pass your screen, you can safely ignore those for now. Another advantage here is that packages don't only get installed, they get configured as well. Get ready to specify your timezone.
Inconsistent (but usable) state
Now, you're probably going to end up with a few error messages. As I mentioned earlier Linux compatibility isn't perfect:
But keep well in mind: just because there were errors doesn't mean the package didn't got installed:
This package is somewhat alright, but make sure to run:
There really isn't much which we can do here but all in all the 'damage' isn't too bad:
See? Only 2 packages which have a problem. However: the r status on sysvinit isn't good, try and force a re-install if you see this:
Speaking of which... do you know why a FreeBSD base system is far superior than a Devuan Linux base system?
However, you can fix this by running: 
).
3 - Setting up the jail
So now that we have our Devuan environment setup it's time to actually use it within FreeBSD. Edit /etc/jail.conf and add this section:
You can do this in two ways. Instead of lo0 you could also use your public network interface and assign a 'real' IP address, I used this approach with my Psi jail because it made things easier on me. However, I don't trust Linux anymore (especially after all those details which surfaced about Ubuntu adding numerous "phone home" options, all opt-out obviously), so no way that I'll allow it network access "just like that". Ergo: setup on localhost which means that you'll need to set up a NAT solution on your firewall which will allow your Linux jail access to your network.
My reasoning is simple: if you really need quick access then you can always rely on chroot for now, just as I did earlier.
When this is done then all you have to do is fire up the jail:
4 - Summing up
Best of both worlds, what's there not to like?
Prerequisites: I am assuming that you know how to install software on FreeBSD and also have some basic understanding about FreeBSD jails.
Editorial
I'm going to be very honest here: I started disliking Linux for a while, and I've worked with it for a long time. For various reasons, but one of them being that it has turned into a commercial company driven project (you do realize that RedHat and Canonical (= company behind Ubuntu) are enterprise sized companies, right?). Arguments? Simple: systemd. A piece of shit software service which goes directly against the Unix philosophy and worse yet: makes Linux pretty much incompatible with others (you'll see later on). Isn't it annoying that because of projects like systemd (and several other changes) many people can't keep up with those developments anymore and need extra study or training? How convenient that RedHat also happens to be one of the bigger companies which provides Linux training! And no: not for free of course
There's a good reason why IBM bought them.However...
Despite my dislike I'm also the first to acknowledge all the effort that goes into those projects. And although I definitely have my (bias) ideas about stuff such as systemd (and the distributions using it) fact of the matter is that many don't have a choice. Also: many projects are still the inclusive distributions they always were. Debian still gives life to many derivatives for example. And well, even Linux can be somewhat fun
1 - Linux compatibility
FreeBSD actually has a Linux compatibility layer which allows it to run Linux (ELF) binaries without a problem. Keep in mind though that this isn't full proof, but many things can be made to work without issues. You're going to need 2 things: support in the kernel and a userland to provide optionally required libraries (the latter isn't needed for the jail but I'd still install it anyway in case you're also going to use this setup for more purposes).
Kernel modules
If you run a GENERIC kernel then you already have everything you need. If you run a more customized kernel (like I do) then you'll need to make sure that it supports these kernel modules:
- linux.ko & linux64.ko; Speaks for itself, right?
- linux_common.ko; This one should also be an obvious one.
- linprocfs.ko; Although FreeBSD doesn't use /proc filesystem by default you can set this up if you want to. Linux on the other hand needs /proc to be present and to make matters worse it'll also work somewhat differently than FreeBSD does things. This module will take care of that.
- linsysfs.ko; When you build a FreeBSD base environment you'll end up with /usr/obj/usr/src which basically contains the binary structure which you build. On FreeBSD you can remove this if you want but Linux actually keeps ties into those binaries & libraries whenever you build the kernel. On Linux /sys often links to /usr/src/linux/sys (from the top of my head, I could be slightly off but the argument is fully correct). So how to cope with that? Well, that's what this module is for, it'll simulate the effect.
- linuxkpi.ko; As far as I know this one simulates the Kernel Programming Interface, but I have no idea what this exactly does. However, when I check /usr/src/sys/modules/linuxkpi/Makefile then my theory quickly becomes that this module provides access to the Linux hardware layer. So if a program tries to access USB, PCI or even a Linux kernel module then this is the FreeBSD kernel module which handles all that.
- fdescfs.ko; A file descriptor provides a method for a program to communicate with the OS. Good examples are stdin, stdout and stderr. And you guessed it: there is a difference between Linux & FreeBSD which is what this kernel module will solve.
- tmpfs.ko; Chances are high that your system already uses this but because it's still a requirement I'm listing it nonetheless. tmpfs is basically what used to be a ramdrive on DOS & Windows: a space in memory which is reserved to be used as a temporary filesystem. This is often used to provide /tmp and/or /var/tmp.
Code:
linux_enable="YES"Userland
Linux binaries often have specific dependencies on libraries (and maybe other binaries) in which case you'll need some kind of userland which is what emulators/linux_base-c7 can provide. This will install a CentOS userland within the /compat/linux directory structure. Don't worry, it won't be a resource hog; on my system the (uncompressed) ZFS filesystem uses up around 254Mb.
Congratulations, you are now Linux compatible!
2 - Adding a full Linux userland
But which Linux? See, there is a problem...
As you may (or should!) know a Jail is nothing more but a FreeBSD userland which gets started by the kernel. You can even make it 'do' stuff by having the kernel initialize your rc.d structure:
sh /etc/rc (see also ports(7)). On Linux we'd normally have /etc/init.d/rc at our disposal but guess what? systemd didn't only take over the init process, nooo that wasn't good enough: it's also spreading its tentacles into other areas such as the booting structure and mounting options itself. On most Linux environments /etc/fstab is simply a systemd emulation.And although the FreeBSD Linux compatibility layer can do a lot, it's not perfect. Crapola like systemd doesn't properly run on it for example (which I actually consider a pro
).Fortunately there are still plenty of people who grasp and still honor the ideology which Linux once stood for and one of those projects is Devuan. What's Devuan? It's a Debian derivative which does not include systemd so we'll be fully able to use this as a Jail just fine.
So why all this interest for Debian even though we already have a CentOS userland you ask? Two reasons: Although /compat/linux does indeed provide a userland it's not complete. It was set up to emulate, not fully simulate. So don't expect an init.d structure.
But the second reason is much cooler... See: Debian's package system is, in my humble opinion, superior by design (within the context of Linux!). It's really neatly set up and unlike RPM it's even doable (though still a bit of a drag) to maintain your own packages. Something which you might want to do if you prefer to build your own software. Debian's packages are provided as both binaries and source, how convenient is that?
And they didn't stop there.. Eventually they created debootstrap which does just what the name implies: it will grab all the packages needed for a base system and set that up. So to get to the finale of this exciting build up: it has even been included in the FreeBSD ports collection as: sysutils/debootstrap, now that is cool I think. So quickly install this critter because we're going to need it!
Bootstrapping Devuan
If you're using ZFS then I strongly suggest to set up a dedicated filesystem. At the very least this will help you to keep track of the space your jail(s) are consuming:
Code:
zfpeter@zefiris:/home/peter $ zfs list -r zroot/opt/jails
NAME USED AVAIL REFER MOUNTPOINT
zroot/opt/jails 1.32G 88.4M 162M /opt/jails
zroot/opt/jails/devuan 262M 28.9G 262M /opt/jails/devuan
zroot/opt/jails/psi 928M 28.9G 928M /opt/jails/psiDevuan is deviously clever
See; every Debian distribution has a specific name. Debian's latest (at the time of writing!) is stretch whereas Devuan is called ascii. Now, the "problem" is that debootstrap uses scripts of the same name to help it separate between the distributions. And ascii isn't the same as stretch. Fortunately for us Devuan also honors the Debian standard distribution names: stable, testing and unstable. And unlike on FreeBSD Stable is honestly just that Sidenote: Yes, I am aware that you can also specify an individual script when using debootstrap, but I like to keep things as simple as possible.
Important: We're going to perform the bootstrap process in 2 stages. Why? debootstrap can set up the hierarchy, but it won't be able to utilize things such as linprocfs or linsysfs, so we're going to have to set that up ourselves.
Code:
root@zefiris:/home/peter # debootstrap --foreign --arch=amd64 stable /opt/jails/devuan http://deb.devuan.org/merged/
W: Cannot check Release signature; keyring file not available /usr/share/keyrings/debian-archive-keyring.gpg
I: Retrieving InRelease
I: Retrieving Packages
I: Validating Packages
I: Resolving dependencies of required packages...
I: Resolving dependencies of base packages...Anyway, this will process quite a list. What is happening is that debootstrap is retrieving all the packages that make up the Devuan base system and places those in our jail directory after which it'll extract them.
Jail preparations
Depending on your system this is probably going to take a while so now would be a good time to start preparing our upcoming jail by setting up our upcoming special Linux filesystems. Create a file called /etc/fstab.devuan and add the following:
Code:
$ cat /etc/fstab.devuan
## Mountpoint(s) for the Devuan jail
# Dev Mountpoint FS Options Dump / Check
linprocfs /opt/jails/devuan/proc linprocfs rw,late 0 0
linsysfs /opt/jails/devuan/sys linsysfs rw,late 0 0
tmpfs /opt/jails/devuan/tmp tmpfs rw,late,mode=1777 0 0Around this time the first stage of debootstrap should be ready and I urge you to look around in your new Devuan environment. Points of interest:
- ./debootstrap; this is where the native 'binary' got placed, but also where you'll find debootstrap.log (as created by 'our' version) which shows you exactly what the system did.
- ./etc/init.d/rc; no systemd crapola for us! This is the key to booting our upcoming jail.
- ./root; I share a lot of criticism towards Linux so I also think it's important to give credit where credit's due: setting /root to 700 is in my opinion a solid option. No intruders, especially not during these early stages.
- ./var/cache/apt/archives; and this is where our new software collection resides
Since we're here anyway we can now mount some required directories for the next stage:
Code:
root@zefiris:/opt/jails/devuan # mount -F /etc/fstab.devuan `pwd`/sys
root@zefiris:/opt/jails/devuan # mount -F /etc/fstab.devuan `pwd`/proc
root@zefiris:/opt/jails/devuan # mount -F /etc/fstab.devuan `pwd`/tmp
root@zefiris:/opt/jails/devuan # mount -t devfs none dev
root@zefiris:/opt/jails/devuan # chroot . /bin/bash
I have no name!@zefiris:/#
Code:
I have no name!@zefiris:/# dpkg --force-depends -i /var/cache/apt/archives/*.deb
Selecting previously unselected package adduser.
(Reading database ... 0 files and directories currently installed.)
Preparing to unpack .../archives/adduser_3.115_all.deb ...
Unpacking adduser (3.115) ...
Selecting previously unselected package apt-utils.
Preparing to unpack .../apt-utils_1.4.8_amd64.deb ...
Unpacking apt-utils (1.4.8) ...Inconsistent (but usable) state
Now, you're probably going to end up with a few error messages. As I mentioned earlier Linux compatibility isn't perfect:
Code:
Setting up sysvinit-core (2.88dsf-59.9+devuan2) ...
cp: preserving permissions for '/etc/inittab': No data available
dpkg: error processing package sysvinit-core (--install):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
base-files
debianutils
libc-bin
readline-common
sysvinit-core
Code:
I have no name!@zefiris:/# dpkg -l base-files
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-============-============-=================================
iF base-files 9.9+devuan2. all Devuan base system miscellaneous
I have no name!@zefiris:/# # dpkg --configure --pending, this will sort out other left over issues. Now normally you'll only end up with 2 errors and there's little we can do about them:
Code:
Setting up sysvinit-core (2.88dsf-59.9+devuan2) ...
sysvinit: creating /run/initctl
mv: cannot move '/dev/initctl.new' to '/dev/initctl': Operation not supported
dpkg: error processing package sysvinit-core (--configure):
subprocess installed post-installation script returned error exit status 1
Setting up base-files (9.9+devuan2.5) ...
cp: preserving permissions for '/root/.profile': No data available
dpkg: error processing package base-files (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
sysvinit-core
base-files
Code:
I have no name!@zefiris:~# dpkg -l | grep -v ii
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-=============================-==================================-============-========================================================================
iF base-files 9.9+devuan2.5 all Devuan base system miscellaneous files
rF sysvinit-core 2.88dsf-59.9+devuan2 amd64 System-V-like init utilities # dpkg --force-all -i /var/cache/apt/archives/sysvinit-core*. The status you want for both packages is iF. Because once you have that then you'll still run into warnings whenever you install another package (during its configure stage) but your installation won't fail because of it.Speaking of which... do you know why a FreeBSD base system is far superior than a Devuan Linux base system?
Code:
I have no name!@zefiris:~# man dpkg
bash: man: command not found # apt-get install man-db. This will also be a good test to see if your system is in a usable status. Keep in mind that building the database of manualpages is going to take a while. Another con of Linux in my opinion: it creates the database immediately when you install the package instead of leaving it up to the user to decide when (I forgot about this detail and now it messes up the timing for my guide ).3 - Setting up the jail
So now that we have our Devuan environment setup it's time to actually use it within FreeBSD. Edit /etc/jail.conf and add this section:
Code:
devuan {
host.hostname = "devuan.jail";
interface = lo0;
ip4.addr = 127.0.0.5;
path = /opt/jails/devuan;
exec.start = "/etc/init.d/rc 3";
exec.stop = "/etc/init.d/rc 0";
persist;
mount.devfs;
mount.fstab = /etc/fstab.devuan;
allow.mount;
allow.mount.devfs;
}My reasoning is simple: if you really need quick access then you can always rely on chroot for now, just as I did earlier.
When this is done then all you have to do is fire up the jail:
# jail -c devuan, and to get onto the console: # jexec devuan /bin/bash.4 - Summing up
- Set up Linux compatibility. Usually you only have to add:
linux_enable="YES"to /etc/rc.conf.- Although optional it is advisable to install emulators/linux_base-c7 as well.
- Optionally set up a dedicated section for your jail. When using ZFS then creating a new dedicated filesystem is definitely a good idea.
- Install sysutils/debootstrap.
- Bootstrap your system:
# debootstrap --foreign --arch=amd64 stable /path/to/jail http://deb.devuan.org/merged/- Obviously replace amd64 for i386 on 32bit machines!
- Mount linux file systems, for best results add these to a dedicated "jail fstab" like /etc/fstab.devuan.
# mount -t linprocfs none /path/to/jail/proc# mount -t linsysfs none /path/to/jail/sys# mount -t tmpfs none /path/to/jail/tmp
- Enter your new system to finish up:
# chroot /path/to/jail /bin/bash# dpkg --force-depends -i /var/cache/apt/archives/*.deb# dpkg --configure --pending
- Make sure that only base-files and sysvinit-core are partially configured:
# dpkg -l | grep -v ^ii.- Their status should be iF.
- If there is a problem try to forcefully reinstall the package:
# dpkg --force-all -i /path/to/package.
- Check that everything works by installing a package:
# apt-get install man-db. - Set up your jail by adding the following to /etc/jail.conf:
Code:
devuan {
host.hostname = "devuan.jail";
interface = lo0;
ip4.addr = 127.0.0.5;
path = /path/to/jail;
exec.start = "/etc/init.d/rc 3";
exec.stop = "/etc/init.d/rc 0";
persist;
mount.devfs;
mount.fstab = /etc/fstab.devuan;
allow.mount;
allow.mount.devfs;
}- Start your new jail using:
# jail -c devuan. - Access the console using
# jexec devuan /bin/bash. - Enjoy!
Best of both worlds, what's there not to like?
