swap size on ZFS with high amount of RAM

0w8States

0w8States

Hi!

I'm doing my first install of FreeBSD and really looking forward to getting started with FreeBSD. I've recently purchase a copy of 'Absolute FreeBSD' and I'm working my way through the chapters. While I've previously used FreeBSD inside VirtualBox, this time I'm doing my first physcial install on a Lenovo P1 Workstation.

Not to continue the age-old question of what the appropriate swap size should be, but there is mention in the install to have swap be equal to the amount of RAM if you plan to do core-dumps in kernel development. The Lenovo P1 I'm working on has 64GB of RAM, so would it be advisable to have a swap of 64GB or slightly higher?

The next question comes when installing with ZFS, as both ZFS mirrored or striped on my 1TB ADATA drives allocates double the swap; is this to be expected? e.g. I enter 64g of swap on my ZFS install and it actually allocates 128g total (64g accross both drives). In this case, and assuming I get into kernel development and require core-dumps, should I set this ZFS install setting to 32g and have it auto-doubled to 64g, or should I have it set to 64g and auto-doubled to 128g.

Thanks in advance! :)
 
A: What is the probability that you will do kernel development? Do you have sufficient programming experience for that? If the answer is that you won't be doing it anytime soon, then it really doesn't matter what you do for this install; you'll be reinstalling sooner or later anyway.

B: Disk space is cheap. You seem to have two 1TB drives, and you seem to be planning to use ZFS for your data, either with mirroring (good) or striping (bad). That means you will have a little less than 1TB or than 2TB of disk space for user files. Now, whether you create 32GB or 64GB per drive as a swap partition makes only a tiny amount of difference to your user space (you probably can't predict your space usage accurate enough to know whether 0.968 or 0.936 TB of disk space is what you need *). So just go for 64GB on each drive; that gives you interesting options in the future. You can always make the partitions smaller later, but not easily bigger.

Footnote *: Your real usable disk space will be considerably less than the examples I gave, after accounting for decimal -> binary conversion, metadata, system overhead, various rounding errors, and so on. Still, the 3% won't make a difference.
 
ralphbsz said:
A: What is the probability that you will do kernel development? Do you have sufficient programming experience for that? If the answer is that you won't be doing it anytime soon, then it really doesn't matter what you do for this install; you'll be reinstalling sooner or later anyway.

B: Disk space is cheap. You seem to have two 1TB drives, and you seem to be planning to use ZFS for your data, either with mirroring (good) or striping (bad). That means you will have a little less than 1TB or than 2TB of disk space for user files. Now, whether you create 32GB or 64GB per drive as a swap partition makes only a tiny amount of difference to your user space (you probably can't predict your space usage accurate enough to know whether 0.968 or 0.936 TB of disk space is what you need *). So just go for 64GB on each drive; that gives you interesting options in the future. You can always make the partitions smaller later, but not easily bigger.

Footnote *: Your real usable disk space will be considerably less than the examples I gave, after accounting for decimal -> binary conversion, metadata, system overhead, various rounding errors, and so on. Still, the 3% won't make a difference.
Click to expand...

Thanks for the response! I'll let it ride at 64GB in this case.

I doubt I'll be doing any kernel development in the near future, I just don't want to block myself in a corner if the book examples get to doing core-dumps. The next book on my list is "The Design and Implementation of the FreeBSD kernel" and I wanted to make sure I set myself up for the future.
 
0w8States said:
on my 1TB ADATA drives
Click to expand...
Is this SSD? 64GB RAM is too much for me and if I have 64GB (or even 32GB) will not use swap file. You can allocate 64GB swap partition for future use and work only with the physical RAM. Do you plan to run applications using large amount of RAM (e.g. big database)?
 
It's a 1TB SSD, I have 2 installed running ZFS mirror. My daily driver is an Arch Linux system and I only use a 2GB swap, but in this case I don't want to block myself in when going through learning examples with the various FreeBSD books and documents.


From what I've read in 'Absolute FreeBSD' by Michael Lucas, it's best to have an equal sized partition to the hosts RAM of type freebsd-swap, to use as a dump partition. I'm fairly new to FreeBSD and I'm learning as I go; correct me if I'm wrong, but I believe this allows you to do a full kernel memory dump down the line.
 
Planning for Swap Space from Oracle Solaris Administration: Devices and File Systems Oracle Solaris 11 Information Library shows next approximated rules swap size for Solaris 11 (I don't know works this rules for FreeBSD 11.X/12.X or not)
Table 19-1 Swap and Dump Volume Sizes for ZFS File Systems
System Type​
Swap Volume Size​
Dump Volume Size​
System with about 4 GB of physical memory​
1 GB​
2 GB​
Mid-range server with about 8 GB of physical memory​
2 GB​
4 GB​
High-end server with about 16 to 128 GB of physical memory​
4 GB​
8-64 GB​
High-end server with more than 128 GB of physical memory​
1/4 of physical memory size​
1/2 of physical memory size​
 
Vadim_Mkk said:
As a rule of thumb, the swap partition should be about double the size of physical memory (RAM)
Click to expand...
That's a good rule but severe overkill when you're dealing with 380 GB (or more) of RAM for example. Up to about 64 GB I'd keep the double size, anything above that is going to depend on the expected workload and may just get 128 GB of swap or even less. That said, even with a freaking humongous amount of RAM I always configure some swap.
 
May be for typical laptop using hibernation RAM to HDD with value less 8 GB - a rule of thumb (IMHO) is right.
For correct determination of swap size for desktop workstation and server - use top () and vmstat () under full typical workload.
 
Vadim_Mkk said:
but other hand
2.6.1. Designing the Partition Layout from Chapter 2. Installing FreeBSD from FreeBSD Handbook
As a rule of thumb, the swap partition should be about double the size of physical memory (RAM)
Click to expand...

This is *OLD*. It was common recommendation until typical bigger ram sizes reached some 10 GB.

Today, I would recommend the upper range of swap size at what the system can practically move out within 5-10 minutes. Because, if the machine is busy for 5 minutes with paging, this time is either enough for whatever corrective measures to work, or otherwise the machine becomes practically useless.
Doing the math, with regular disks at 150 MB/sec, this is about 64GB.

With SSD drives the calculated amount would be larger, but then, under such write load the SSD would survive only few hours, so this is not a feasible use-case either.
 
0w8States said:
I'm doing my first install of FreeBSD and really looking forward to getting started with FreeBSD. I've recently purchase a copy of 'Absolute FreeBSD' and I'm working my way through the chapters. While I've previously used FreeBSD inside VirtualBox, this time I'm doing my first physcial install on a Lenovo P1 Workstation.
Click to expand...
Congrats and welcome to the club!

0w8States said:
Not to continue the age-old question of what the appropriate swap size should be, but there is mention in the install to have swap be equal to the amount of RAM if you plan to do core-dumps in kernel development. The Lenovo P1 I'm working on has 64GB of RAM, so would it be advisable to have a swap of 64GB or slightly higher?
Click to expand...
The efficient swap size depends on your usage. Why do we use swap in the first place? It's a much slower addition to the RAM, which helps in cases where RAM is insufficient.
Swap is meant to be used occasionally - when the RAM is a bottleneck for a short time. For example imagine you use 10 apps, each of which consumes 1 GB of memory. However, your machine has 2 GB RAM and 10 GB of swap. If you always use 1 app at a time for 2 hours, then switch to the next app, all of them can be put in the swap and when needed - put back into RAM.
If however you always have 5 of the 10 apps working all the time, your system will be busy swapping all the time and become very very slow.

In summary, make sure your system has an adequate amount of memory for the load that is running all the time. If you have short term peaks in memory usage, you can buffer that with swap. Depending on the usage, you may need a lot of swap (if you have lots of dormant processes that occasionally wake, like on a web or DB server, or host with virtual machines) or maybe just 1 or 2 gigs (in case you just want a small buffer to prevent running out of memory).

Some more thoughts on swapping: Systems with large memory in many cases don't need any swap whatsoever. On a host with 64 GB and one with 32 GB I don't use any swap.
Also, swapping on an SSD can trash the drive, because an SSDs has a certain limited number of writes per cell until the cell dies, so with very frequent writes the disk can die much more quickly than usual.
As a rule of thumb, system with lots of memory an SSD - no swap. If you need swap, put it on a HDD.
And if you discover that your system's swap is never used (swapinfo shows 0%), this means you probably don't need a swap.

0w8States said:
The next question comes when installing with ZFS, as both ZFS mirrored or striped on my 1TB ADATA drives allocates double the swap; is this to be expected? e.g. I enter 64g of swap on my ZFS install and it actually allocates 128g total (64g accross both drives). In this case, and assuming I get into kernel development and require core-dumps, should I set this ZFS install setting to 32g and have it auto-doubled to 64g, or should I have it set to 64g and auto-doubled to 128g.
Click to expand...
In my opinion mirroring your swap does not help in most cases (maybe it adds somewhat to robustness, but for me the cost is too much).
If I had a mirrored ZFS root partition on 2 or more disks, I would add the swap partitions separately to the fstab. Mirroring the swap means you're wasting disk space by writing the same swap data twice or more with no essential benefit.
You could have for example a 16 GB mirrored 2 way partition, or add 2 swap partitions x16GB = 32 GB.

To answer your question: When you enter 64GB swap in the installer, you can select "Mirror swap" or not. So it's up to you how much would be allocated. I personally never mirror the swap.
 
Understandable confusion due to their similar names. And using a dump volume isn't that common these days, perhaps on other Unix(-like) systems, I've never encountered it on FreeBSD though.
 
I don't know why people always ask the same question regarding swap size. When you have more than necessary you lose nothing. Just lose a bit of space on an SSD and very tiny bit of memory. On embedded systems the question can be relevant.
 
You must log in or register to reply here.
Back
Top