FreeBSD as storageserver using zfs?!

swemattias

swemattias

I got an idea a few weeks back, I want to "upgrade" my storage-server, från Ubuntu Server 19.04, with Kernal 5.x, using 8 2Tb disks in RAID-6 with XFS.
Hardware is an AMD Ryzen 2200G, 16 GB of non-ECC RAM (please don't comment this, I know), Asus TUF board, with two PCI-e cards, one extra NIC and one RAID/SATA-port. All in all I got 10 SATA 6 GB/s ports.

My goal is:
Dual SSDs in RAID-1 for the system.
2 pools with 4 2 TB disks each using RaidZ2.

The things I wonder about is:
- Does FreeBSD play nice with AMD Ryzen?
- Does FreeBSD come with zfs, particularly v0.8.2?

Do you see anything worth mentioning about what I am about to do?
 
swemattias said:
I got an idea a few weeks back, I want to "upgrade" my storage-server, från Ubuntu Server 19.04, with Kernal 5.x, using 8 2Tb disks in RAID-6 with XFS.
Hardware is an AMD Ryzen 2200G, 16 GB of non-ECC RAM (please don't comment this, I know), Asus TUF board, with two PCI-e cards, one extra NIC and one RAID/SATA-port. All in all I got 10 SATA 6 GB/s ports.
Click to expand...
Nice hardware!

swemattias said:
- Does FreeBSD play nice with AMD Ryzen?
Click to expand...
AFAIK - yes. A friend has a Ryzen system and it seems to work fine. I use Threadripper and it works well, except for sound. Sometimes the whole system loses sound and playing audio or video is not possible until restart. I have not been able to track down the cause :(.

swemattias said:
- Does FreeBSD come with zfs, particularly v0.8.2?
Click to expand...
FreeBSD does come with zfs. I don't know how to check the version though. Why do you need 0.8.2 specifically? In the man page of zpool it's written that the pool version is independent from the ZFS software version.

swemattias said:
Do you see anything worth mentioning about what I am about to do?
Click to expand...
No, it seems just fine for storage. It depends on your concrete requirements though.
 
If you are not planning on the box doing anything else but being a file server, have a look at XigmaNAS or FreeNAS, although the people over at FreeNAS will give you a hard time about the non-EEC RAM... They both are based on FreeBSD... Personally, I use XigmaNAS...
 
bjs said:
If you are not planning on the box doing anything else but being a file server, have a look at XigmaNAS or FreeNAS, although the people over at FreeNAS will give you a hard time about the non-EEC RAM... They both are based on FreeBSD... Personally, I use XigmaNAS...
Click to expand...
A friend used FreeNAS but dropped it for FreeBSD. Upgrading FreeNAS was quite a hassle at the time, and their kernel was a bit older than it should have been.
 
I haven't used FreeNAS since it was bought by iXSystems so I don't know much about it... However, it doesn't surprise me... XigmaNAS is using current kernels...
 
FreeBSD has traditionally used OpenZFS from Illumos and so the version numbering from ZFSonLinux is irrelevant. You only need to care about the pool versions to know if they are compatible or not.

That said, FreeBSD is ditching this and moving to ZFSonLinux itself. This is likely to be the default version in FreeBSD 13.0, but I don't know which version it's currently based on. Support was added back in July. https://github.com/zfsonlinux/zfs/pull/8987
 
bjs said:
If you are not planning on the box doing anything else but being a file server, have a look at XigmaNAS or FreeNAS, although the people over at FreeNAS will give you a hard time about the non-EEC RAM... They both are based on FreeBSD... Personally, I use XigmaNAS...
Click to expand...
Tell me about it! When I built the current server I was thinking fo using FreeNAS but the hate I got on the FreeNAS forums told me to never ever thouc h that system with a 10 foot pole ever again.
 
Yeah, it's a shame they have to be that way, there is really no call for it... Much better people and forum support for XigmaNAS...
 
I should add some more info to what the system does today.
First off it is the storage for my plex media library, running as a VM inside proxmox, connected via nfs.
Second, it is used as common storage for lots of Docker containers and it is also the storage for my Proxmox setup.
So at times, it is under heavy load but not that much I presume....haven't given me any problems at all.

I am worried about the safety, I have no motioner solution for the storage atm, no nothing really to check the health. That is the main reason for going zfs.
 
So what do I want to do at installtion?
Create and RAID-1 for the system disks, 2 250 GB SSDs.
Create an NIC bond with the two NICs in the machine, set an static IPv4 address to it.
After that the rest can be done after install I hop.
 
I don't quite understand this FreeNAS thing. What can it do that FreeBSD can't?

So why not just have FreeBSD as a fileserver?
 
xtaz said:
FreeBSD has traditionally used OpenZFS from Illumos and so the version numbering from ZFSonLinux is irrelevant. You only need to care about the pool versions to know if they are compatible or not.

That said, FreeBSD is ditching this and moving to ZFSonLinux itself. This is likely to be the default version in FreeBSD 13.0, but I don't know which version it's currently based on. Support was added back in July. https://github.com/zfsonlinux/zfs/pull/8987
Click to expand...
Ditching openzfs for ZFS on Linux? Hm... Will ZFS be still in the kernel or is fbsd gonna use modules? Big advantage of kernel support is speed. I'm a little worried reading this.
 
Terpentijn said:
Will ZFS be still in the kernel or is fbsd gonna use modules?
Click to expand...
It's using modules now anyway.

Terpentijn said:
Big advantage of kernel support is speed.
Click to expand...
There is no difference between a loadable kernel module or something that's statically linked into the kernel. It's all the same. Interesting detail, on FreeBSD everything is built as a module, even the stuff that's statically linked into the kernel.
 
For some reason I have a hard time accepting ready-made solutions like FreeNAS, Unraid and all of those, I like to build my own solutions so I know what's installed and used and everything.
 
xtaz said:
You only need to care about the pool versions to know if they are compatible or not.
Click to expand...
Pool versions are meaningless now as everything should be 5000 AFAIK (except for maybe de jure ZFS). What you should care about are feature flags and what's enabled there.
 
Terpentijn said:
Ditching openzfs for ZFS on Linux? Hm... Will ZFS be still in the kernel or is fbsd gonna use modules? Big advantage of kernel support is speed. I'm a little worried reading this.
Click to expand...

This was the original announcement about it: https://lists.freebsd.org/pipermail/freebsd-fs/2018-December/027085.html and it's moved on quite a lot since then. There are ports for it in sysutils/openzfs and sysutils/openzfs-kmod.

The reasoning behind it is that ZFSonLinux has moved forward significantly from the Illumos code that FreeBSD use and so it now makes more sense to move to ZOL with a single code base. Will get features etc. a lot quicker. Same reason FreeBSD now uses the KMS graphics drivers from Linux as well for X11.

And yes, ZFS has never been in the kernel anyway, it's always been a kernel module because of the GPL licenced code I believe.
 
xtaz said:
This was the original announcement about it: https://lists.freebsd.org/pipermail/freebsd-fs/2018-December/027085.html and it's moved on quite a lot since then. There are ports for it in sysutils/openzfs and sysutils/openzfs-kmod.

The reasoning behind it is that ZFSonLinux has moved forward significantly from the Illumos code that FreeBSD use and so it now makes more sense to move to ZOL with a single code base. Will get features etc. a lot quicker. Same reason FreeBSD now uses the KMS graphics drivers from Linux as well for X11.

And yes, ZFS has never been in the kernel anyway, it's always been a kernel module because of the GPL licenced code I believe.
Click to expand...
My confusion comes from reading that zfs on freebsd was/ is much faster than ZoL, because it was supported by the kernel. I guess this info was incorrect. The way I read it now is that zfs on Linux makes no difference. It's as fast as the way it works on freebsd. Am I right?
 
So after running my tests in 3 different VMs, Debian 10.1 ZoL, Ubuntu Server 19.10 beta and FreeBSD ZFS, I've made my decision.
I am way more at home in the Linux world than the Unix world, I wish it wasn't so, since I began with Unix, that is a reason why I write this on a Mac OS X machine.
So my choise is Ubuntu Server 19.10 with zfs 0.8.2 that is released next week.
Thanks for all the input and knowledge shared!
 
I don't know of any recent benchmarks that show a major difference except for the BlogBench results on the Phoronix site. However, these benchmarks have to be taken with a grain of salt, as:
  • He's using a pool consisting of 1 SSD and nothing else. This is atypical for any production environment I've ever seen.
  • ZFS is notoriously hard to benchmark as the ARC fills up. You can benchmark several ways, e.g. for cached data vs uncached, but great care must be taken and what you're testing is often closely tied to the hardware it's running on.
  • The BlogBench results are so wildly different that I suspect their surprising results are not related in any way to ZFS/ZoL, but instead probably due to differences in the disk access (e.g. CAM) or methodology. This can be further emphasized by the fact that the iX systems results (which uses the ZoL ported to FreeBSD) show values in line with FreeBSD and not Ubuntu ZoL.
Since the ZoL FreeBSD port is already showing improvements over the existing ZFS code in all but one benchmark, I certainly wouldn't be concerned. There was a lot of FUD spread in another thread on this forum about ZoL coming to FreeBSD, but I've yet to see any complaints that are well founded. I for one am looking forward to it making pools easy to transfer between OSes, as well as its new features (such as native encryption).
 
swemattias said:
I got an idea a few weeks back, I want to "upgrade" my storage-server, från Ubuntu Server 19.04, with Kernal 5.x, using 8 2Tb disks in RAID-6 with XFS.
Hardware is an AMD Ryzen 2200G, 16 GB of non-ECC RAM (please don't comment this, I know), Asus TUF board, with two PCI-e cards, one extra NIC and one RAID/SATA-port. All in all I got 10 SATA 6 GB/s ports.

My goal is:
Dual SSDs in RAID-1 for the system.
2 pools with 4 2 TB disks each using RaidZ2.

The things I wonder about is:
- Does FreeBSD play nice with AMD Ryzen?
- Does FreeBSD come with zfs, particularly v0.8.2?

Do you see anything worth mentioning about what I am about to do?
Click to expand...

Maybe this will help you:
vermaden.wordpress.com

FreeBSD Enterprise 1 PB Storage

Today FreeBSD operating system turns 26 years old. 19 June is an International FreeBSD Day. This is why I got something special today :). How about using FreeBSD as an Enterprise Storage solution o…
vermaden.wordpress.com vermaden.wordpress.com
 
Geezer said:
I don't quite understand this FreeNAS thing. What can it do that FreeBSD can't?

So why not just have FreeBSD as a fileserver?
Click to expand...
It has a web based system management interface and some fancy customizing and tuning.
It can do basically the same stuff, it just lets you do it in a pointy-clicky way.
It's a neat idea, the devil is however in the proverbial details.
 
swemattias said:
So after running my tests in 3 different VMs, Debian 10.1 ZoL, Ubuntu Server 19.10 beta and FreeBSD ZFS, I've made my decision.
I am way more at home in the Linux world than the Unix world, I wish it wasn't so, since I began with Unix, that is a reason why I write this on a Mac OS X machine.
So my choise is Ubuntu Server 19.10 with zfs 0.8.2 that is released next week.
Thanks for all the input and knowledge shared!
Click to expand...
It's most important that your data is safe and accessible. I think that you can do this with any OS and ZFS when used intelligently. Just use whatever you are comfortable with. You could even run ZFS with Windows.
 
roccobaroccoSC said:
It has a web based system management interface and some fancy customizing and tuning.
It can do basically the same stuff, it just lets you do it in a pointy-clicky way.
It's a neat idea, the devil is however in the proverbial details.
Click to expand...


Exactly. There is nothing that demands another 'distribution', merely some configuration of packages, or at the most, some new programs that runs ontop of existing packages.
 
You must log in or register to reply here.
Back
Top