FreeBSD on ZFS without Boot and Swap.

K

kazham

Hi!
First of all, I am complete newbie in FreeBSD.
I would like to install FreeBSD on ZFS. Modern installer have everything to install FreeBSD on ZFS without any problems, but I have a one "Ideafix", if you know what I mean.
I want to remove Boot and Swap partitions from ZFS pool(RAIDZ1) and move them to separate drives (Boot - USB or SSD, Swap - separate small, but fast SSD or remove Swap completely, because I have 64GB RAM).
Is it possible? I mean isn't it against ZFS/FreeBSD ideology?
If this is possible, can someone point me to some article where I can read about this kind of configuration.
I appreciate any help and have a good day/night to everyone!
THX.
 
You don't need swap unless you need it like compiling big ports parallel in ram.
I have 32GB of ram & 64GB of swap. As swap does not cost.
I don't have a freebsd-boot-partition as i boot using linux-grub to boot ufs which loads the kernel on zfs.
 
kazham said:
Is it possible? I mean isn't it against ZFS/FreeBSD ideology?
Click to expand...
Yes it should be possible, no it isn't against anything.
Swap as others have said is trivial. use gpart to create a swap partition on a device, add an entry to /etc/fstab for it and it will be used next boot. patmaddox points to a very good reference on swap.

"Boot" on the other hand you need to be careful with exactly what do you mean.

There is a boot partition, either UEFI or BIOS that holds boot code and then there is the directory
"/boot" which holds loader and kernels and is typically part of a ZFS Boot Environment.

In theory I think you can have /boot separate, I think this used to be a way to create encrypted devices and read only boot for security. People have also created bootable USB devices which mount root from another device.

I have not done this because I don't have a need to or want to make things more complicated.

As for moving a boot partition, as Alain De Vos says you can work around it.
 
I upgraded my zpools with openzfs from ports.
Linux-grub is not aware of this newer zfs-version.
By moving /boot to a specific ufs partition so i could work around it.
I find this setup very flexible as i only have to edit mountfrom in loader.conf to boot any ufs&zfs installation.
 
mer said:
Yes it should be possible, no it isn't against anything.
Swap as others have said is trivial. use gpart to create a swap partition on a device, add an entry to /etc/fstab for it and it will be used next boot. patmaddox points to a very good reference on swap.

"Boot" on the other hand you need to be careful with exactly what do you mean.

There is a boot partition, either UEFI or BIOS that holds boot code and then there is the directory
"/boot" which holds loader and kernels and is typically part of a ZFS Boot Environment.

In theory I think you can have /boot separate, I think this used to be a way to create encrypted devices and read only boot for security. People have also created bootable USB devices which mount root from another device.

I have not done this because I don't have a need to or want to make things more complicated.

As for moving a boot partition, as Alain De Vos says you can work around it.
Click to expand...
By boot I mean only boot partition. THX.
 
Alain De Vos said:
I upgraded my zpools with openzfs from ports.
Linux-grub is not aware of this newer zfs-version.
By moving /boot to a specific ufs partition so i could work around it.
I find this setup very flexible as i only have to edit mountfrom in loader.conf to boot any ufs&zfs installation.
Click to expand...
I think I'm trying to achieve same result on my PC.
I'm afraid that this kind of configuration cannot be made with "standard" installation.
Can you provide me with some helpful links to this kind of installation?
THX.
 
Anyway, if I install FreeBSD on top of ZFS RAIDZ1 following normal installation, can I later make changes how it is loaded?
Thank you all guys for having patience to answer my stupid questions.
 
I think I should follow normal FreeBSD on ZFS-root installation and then say what I don't like in it (with screenshots).
 
Just commandline & enter some commands out of my mind. I will reflect on it & post them later.
In essence you make a ufs partition and a zfs partition.
You drop into commandline.
Mount / in rw mode
You install wget with pkg
You grab kernel.txz with wget & untar it to the ufs-partition. [This contains the loader]
You grab base.txz with wget & untar kernel.txz & baste.txz it to the zfs-partition.
You edit /boot/loader.conf on the ufs-partion to boot the kernel from the zfs partition "mountfrom...".
You edit on linux /etc/grub/40_conf & tell it to load the "loader" on the ufs partition (which will load the kernel on the zfs partition) & run update-grub.
Done.
 
kazham said:
Swap - separate small, but fast SSD or remove Swap completely, because I have 64GB RAM).
Is it possible? I mean isn't it against ZFS/FreeBSD ideology?
Click to expand...
In case you need swap later you can also prepare a swap file later. I am not sure about any performance issue with respect to swap partition and swap file.
 
chrbr said:
In case you need swap later you can also prepare a swap file later. I am not sure about any performance issue with respect to swap partition and swap file.
Click to expand...
Swapfile have many issues compared to a swap partition, specially on freebsd.
 
jbo said:
I'd like to learn more. Anything you can reference in this regard?
Click to expand...
  1. Swapfile can generate fragments, while isn't a "visible" issue using SSDs, it'll make even slower than ram because the filesystem will have to find where to allocate it (holes). Take in consideration that RAM is way slower than most of the regular used storage devices.
  2. Not to mention, since a swapfile depends on the filesystem where it sits, the filesystem still manages so it will eventually be even slower than a swap partition itself. A swap partition doesn't have this issue.
  3. You can't do any kind of dump to a swapfile (FreeBSD can use it for debug in case of something weird happens) unless this had changed.
You can use a swap space inside zfs, but there's some caveats, if I remember, you can't use debug facilities on swap on zfs too.
Just to mention, despite we have lots of ram nowadays, it's a good practice to have even a tiny swap (I have 16G of ram and 512M of swap). In the worst case scenario, you have somewhere to run.
 
Menelkir said:
Not to mention, since a swapfile depends on the filesystem where it sits, the filesystem still manages so it will eventually be even slower than a swap partition itself. A swap partition doesn't have this issue.
Click to expand...
This is a very important point (at least in my opinion).

Why do things go to swap? Basically because they can't be held in RAM.
What do files use? Buffers. Where are buffers located? RAM.

So getting into a low free memory situation with swapfiles can get you into a death spiral quicker.
Now swapfiles can get you out of a temporary situation, so it's a good trick to have in your back pocket.

I typically always create a swap partition on all my systems, even if it's never hit. Why? If I ever need to setup kernel coredumps or other emergency situations, I'm not scrambling. I just need to tweak a few sysctls or rc variables. Basically #3 from Menelkir
 
mer said:
This is a very important point (at least in my opinion).

Why do things go to swap? Basically because they can't be held in RAM.
What do files use? Buffers. Where are buffers located? RAM.

So getting into a low free memory situation with swapfiles can get you into a death spiral quicker.
Now swapfiles can get you out of a temporary situation, so it's a good trick to have in your back pocket.
Click to expand...
I don't think that's correct. The problem is with swap on ZFS - whether you use a swap file or a zfs volume. I'd be very surprized if it applies to a swap file on UFS.
 
bob2112 said:
I don't think that's correct. The problem is with swap on ZFS - whether you use a swap file or a zfs volume. I'd be very surprized if it applies to a swap file on UFS.
Click to expand...
UFS also uses cache/buffers, heck, even FAT32 uses cache/buffers.
 
  • Like
Reactions: mer
bob2112 said:
I'd be very surprized if it applies to a swap file on UFS.
Click to expand...
As Menelkir points out, UFS also uses cache/buffers. It's just the way filesystems in general work. Physical device as the ultimate backing store, read/write data flows through in memory structures (buffers/cache). Typically reads and writes are held around in that memory structure for some amount of time to satisfy a "next" read because it's quicker going to memory than to physical device.

Now a swapfile on UFS may be better than a swapfile on ZFS in overall memory pressure, but not much difference if it takes 35 secs for your system to crash instead of 28secs, at least in my opinion.

Alain De Vos said:
Swap on zfs is not a good idea. It made my kernel reboot spontaneous. Better a separate swap partition.
Click to expand...
I agree with this.

Now a lot of people running with gobs of memory may never touch swap, even in their most loaded condition. But again, my opinion, you don't lose very much by creating a swap partition given the size and cost per byte of current storage devices. I don't think the old rule of "swap space 2 times RAM" is valid anymore, I would size swap big enough to hold a kernel crash.
 
Hi!
So I made typical ZFS -on-root installation and now I am ready to show what I would like to change in it, if it makes any sense.

1. BIOS boot entry. As you all know BIOS must know which device to use for booting OS. Right now it is set like this:​
IMG_20230204_113912.jpg
It means that if I remove this drive during disk failure(I have RAIDZ1), the system will probably not boot. I think, this can be fixed by setting other two hard drives also as bootable, but I don't like this solution. I prefer to have fixed boot drive here, for example some small 2G USB-pen-drive.​
​
2. Because of the previous issue, installation program made three identical(EFI) partitions on all three drives:​
IMG_20230202_222525.jpg
I would like to eliminate these partitions and keep only freebsd-zfs partitions.​
​
3. Thanks to vermaden I set swap partition size to 0G and it was completely eliminated.​
​
4. I also would like to remove zroot/tmp from the pool, because I was going to make a sort of RAM-drive and put /tmp there. zroot/home must be moved to a separate SSD, which in turn later will be substituted by separate vdev with RAIDZ1.​
IMG_20230202_221201.jpg

So these are my intentions.
What do you think of them?
Thanks in advance for taking the time to clarify these questions.
 
Thanks to Alain De Vos I was able to create tmpfs partition and mounted it to /tmp:
IMG_20230205_163308.jpg

But now I need to delete zroot/tmp. Can I make it with:
Code: 
zfs destroy -r -n zroot/tmp
zfs destroy -r  zroot/tmp
THX.
 
You must log in or register to reply here.
Back
Top