Poudriere on Low Resource Machine

STREBLO

STREBLO

I'm running poudriere on a very low resource machine. It is an old HP Mediasmart EX470 with 1 CPU, 2GB of RAM, and 16GB of swap. It's running ZFS with 4 hard drives mirrored in 2 VDEVs.

I have ran a test build and it seems to run fine however I was looking at seeing what I could do to optimize it.

First of all I had a few questions about some settings I was a bit unsure of. I was wondering what I should set the MAX_MEMORY to. Should this just be the maximum amount of memory in the system? Should it include swap? Should I limit it because the machine has so little memory? I had set MAX_MEMORY to 1 in order to leave enough for the system, should I change this?

What are some other settings I should set considering the resources?

Is it hopeless to run poudriere on this machine? I did have a crush once while building, i'm not sure why it crashed but hopefully it was a one-off.

What should I do to optimize this server for building ports? I don't care if it runs very slow, as long as it runs.
 
It depends on what you're building, I'm able to build full x11/xorg without any problems on VirtualBox VM that has 2GBs of memory. I haven't tried some of the worst memory hogs like www/firefox or java/openjdk-jre but I guess my system should be able to build them. If you get crashes while building anything you really should verify that you don't have hardware problems such as overheating and fluctuating voltages, it's all too common that a system that can run the built software is not able to build the same software because it's been put together from substandard components or the manufacturer has intenced the system only for "light use" and the system doesn't really work when put to a proper stability test by trying to build software with it.
 
So what should MAX_MEMORY be? What do you have yours set to? Mine seems to be running fine but i'm not sure if i'd benegfit from removing r increasing MAX_MEMORY.
 
Shit, crashed again... Might need to try synth or something else.

I'm not sure if it's just running out of memory or something else, because none of the package builds are listed as failed. If I resume the build it appears to work fine, but after awhile it seems to crash. Does anyone have any recommendations?
 
STREBLO said:
Shit, crashed again... Might need to try synth or something else.

I'm not sure if it's just running out of memory or something else, because none of the package builds are listed as failed. If I resume the build it appears to work fine, but after awhile it seems to crash. Does anyone have any recommendations?
Click to expand...
Try Synth. It is easy to configure and it works.
 
Don't be so surprised if it also crashes with Synth the same way. The package builders themselves are really lightweight except for some operations like jail cloning on poudriere if used with UFS. The actual building of the software is what stresses the hardware and exposes problems if there are any.
 
STREBLO said:
What should I do to optimize this server for building ports?
Click to expand...

You're going to have to set USE_TMPFS=no. Running a ZFS-based system and building packages in tmpfs, with only 2Gb of RAM to work with, guarantees memory problems. From the relevant section of /usr/local/etc/poudriere.conf:

Code: 
# How much memory to limit jail processes to for *each builder*
# in GiB (default: none)

A single builder is assigned to a single CPU core to build a single package. Setting MAX_MEMORY=1 means that each builder---and each package being built---will have 1Gb of RAM available to it. You only have 2Gb of RAM, so you've allowed Poudriere to use up to half your RAM to build just one package. If you have multiple CPU cores, Poudriere will try to build multiple packages simultaneously. In such a case, you're "limiting" Poudriere to an amount of RAM greater than the machine actually has. The only option is to turn off building in tmpfs
 
Since you're running on ZFS with just 2GBs of memory you should limit the maximum size of the ARC cache to something sensible to leave enough memory for the actual building. I wouldn't give the ARC cache more than 512MBs, maybe even as low as 384MBs could do. It's very likely that your crashes have been ARC cache related if they are not related to faulty hardware. Put this in your loader.conf(5) and reboot:

Code: 
vfs.zfs.arc_max="384M"
 
ANOKNUSA said:
You're going to have to set USE_TMPFS=no. [...] The only option is to turn off building in tmpfs
Click to expand...
Small warning in advance: To disable tmpfs usage in Poudriere you need to comment/delete the USE_TMPFS line. Counter-intuitively USE_TMPFS=no doesn't work.

EDIT: This has been fixed now in Poudriere and USE_TMPFS=no will work in the future.
 
Last edited:
tobik said:
Small warning in advance: To disable tmpfs usage in Poudriere you need to comment/delete the USE_TMPFS line. Counter-intuitively USE_TMPFS=no doesn't work.
Click to expand...

Just like in src.conf(5) in fact:

Code: 
The values of variables are ignored regardless of their setting; even if
     they would be set to “FALSE” or “NO”.  Just the existence of an option
     will cause it to be honoured by make(1).

Evaluating against non-empty/empty value is much easier in sh(1) scripts than actually looking at the values.
 
kpa said:
Since you're running on ZFS with just 2GBs of memory you should limit the maximum size of the ARC cache to something sensible to leave enough memory for the actual building. I wouldn't give the ARC cache more than 512MBs, maybe even as low as 384MBs could do. It's very likely that your crashes have been ARC cache related if they are not related to faulty hardware. Put this in your loader.conf(5) and reboot:

Code: 
vfs.zfs.arc_max="384M"
Click to expand...
I'm going to limit my arc to 384MB, hopefully that will help.
tobik said:
Small warning in advance: To disable tmpfs usage in Poudriere you need to comment/delete the USE_TMPFS line. Counter-intuitively USE_TMPFS=no doesn't work.
Click to expand...

I have had tmpfs commented out from the get-go.

Thanks everyone for the advice.
 
I think I just need to admit that though machine has too little resources. I mean, it only has a single processor!

Everything seemed to run fine except when it got to a big port, particularly llvm, at which point it would usually crash halfway through the build. After restarting it multiple times, one time It was able to get through the build of llvm, but it took 4 plus hours on just that port!

I was planning on setting up FreeBSD on a desktop and using the little server to build ports that we're not already packaged. I might just have to run poudriere on the box i'll be using the packages for, Is there any reason I shouldn't do this?

My other alternative is running poudriere in a jail on my FreeNAS, I realize this isn't ideal.

Now I just have to think of something to use this server for!
 
Single processor don't mean anything either. Its all about the cores. Single socket servers are pretty powerful these days.
It is amazing to watch the load on each core during build..
With a quad core with HT you get up to 8 build processes simultaneously.
 
Just a datapoint, but I have some very small VMs, one on AWS (single core, 2GB), one on Digital Ocean (single core, 512MB). Both hosts are essentially little islands so I opted to build my small collection of packages on each using poudriere. So far no problems. It's not fast, but I don't care so much about speed - build one day, install the next. :)

Of note, I'm obviously not attempting ZFS on such RAM-limited hosts, I suspect that's a recipe for disaster.
 
STREBLO said:
I was planning on setting up FreeBSD on a desktop and using the little server to build ports that we're not already packaged. I might just have to run poudriere on the box i'll be using the packages for, Is there any reason I shouldn't do this?
Click to expand...

If your only goal is to build ports that (for whatever reason) aren't packaged in the official repo, then using ports-mgmt/synth is a better idea. Synth has a "Fetch prebuilt packages" option that will pull packages available from the official repository, and only build what is not. Poudriere is an excellent tool, but it's primarily designed to build a repository entirely from source and to strenuously enforce the consistency of that repository. While it works fine for home use cases, it's really geared toward professional settings where integrity and reliability are paramount. That means every package gets built from source the first time around, and lots of packages get rebuilt time and time again. The more packages you customize, the more things need to be rebuilt, and the more frequently that rebuilding takes place.

Synth, on the other hand, can build a local repository from a combination existing packages in the official repository and ports that are either customized or don't have existing packages, and in such a configuration will only build things from the ports tree when necessary (that is, it won't rebuild 50 packages because you changed the options in 1). Synth also offers the option to upgrade the packages of the system it's installed on in one process: synth upgrade-system will rebuild the local repository first, then upgrade installed packages from that local repository.
 
You must log in or register to reply here.
Back
Top