Solved Each website in its own jail

quamenzullo

quamenzullo

Hello everyone,

I'm quite new to jails, have read a lot of doc and forum posts and am a bit lost wondering what is the best/easiest/safest set up to serve several websites from the same server. I would like to put each website into its own jail, and use nginx as webserver.

Questions:

1. Is it necessary to use and configure the host's PF to redirect http traffic to the jails? Or setting up a webserver (like www/nginx) on the host is an alternative way of doing the same thing?

2. If using pf is necessary, how is it possible to redirect traffic to several different jails and not only one?

3. Is there a way to put NGINX in only one jail that would be shared with all the other ones? (In order to update only the jail running NGINX).

4. I've read (or misunderstood?) about letting NGINX run on the host and put the php into jails: how is it possible without running another NGINX instance inside the PHP jails?

Many thanks for any hint!


QZ.
 
It depends. If you have multiple public IP addresses available, things may be easier.

1. It is not necessary, but you may want to have your machine protected by firewall anyway. It is also possible to run one webserver on the host as proxy between internet visitors and jailed webservers. In case you have only one public address, this is necessary, because there is no (known to me) way, how to redirect traffic with the pf, you have to inspect HTTP headers to know which server should serve given request.

2. Each jail have at least one public or private IP address and you would redirect/translate packets to those addresses.

3. I don't understand this question, sorry. Maybe you are looking for something like the sysutils/ezjail basejail and/or flavours?

4. You can jail PHP alone, running as daemon, see php-fpm. Then you will connect either to the jails IP address from your webserver or you may probably mount socket of the php-fpm daemon to the webserver jail using mount_nullfs(8). There may be some security implications attached to this solution (inter-jails mounted sockets).
 
Many thanks for your answers!

Some precisions follow because I see some things were not clear.

In question 2. I meant how is it possible to redirect traffic for site number 1 to jail number 1, for site number 2 to jail number 2 etc. But I think you answered this in question 1. already (not possible with pf).

About question 3., I intend to use ezjail, indeed, but I'm not sure the basejail is meant for what I'd liked to do. It allows to share a base system between other jails. If I install nginx in the basejail, does it mean I can use it from the others jails? (nginx does not belong to base system, so I thought things are not planned this way).

Thanks to your answer, I know now that I won't use the solution described in question 4. because my point is security, so I don't want to run into more security issues because of the configuration I choose.

Recently my server has been hacked. I think the weakness was in one website, the attacker could obviously modify the content of the www/ folder plus some log files and maybe config files (not sure).

So I want to be sure that if there's any weakness in one website, then the attacker gets only access to the jail, nothing more.

So, I keep now two ideas of solutions:

A: one nginx running on the host redirecting requests to the matching jails, one website per jail (an nginx instance is running inside each jail as webserver)
B: using pf to redirect traffic to one jail where all websites are stored, and only one nginx instance is running as webserver inside this jail

What is the best for security? If an attacker manages to get an access through a weakness in one website, can he get to the host easier if I use the solution A?
 
quamenzullo said:
In question 2. I meant how is it possible to redirect traffic for site number 1 to jail number 1, for site number 2 to jail number 2 etc. But I think you answered this in question 1. already (not possible with pf).
Click to expand...
The "problem" is that you can only forward port 80 once per IP address. The solution is not to forward traffic based on layer 3 (IP) but on layer 7 (HTTP). To do that you need a proxy. This could be done with nginx or something like net/haproxy.

About question 3., I intend to use ezjail, indeed, but I'm not sure the basejail is meant for what I'd liked to do. It allows to share a base system between other jails. If I install nginx in the basejail, does it mean I can use it from the others jails? (nginx does not belong to base system, so I thought things are not planned this way).
Click to expand...
The base jail contains the base OS, nginx is installed outside of the base. The added benefit of the base jail is that you only need to update once, the other jails all use this same updated base jail. Ports/packages however will need to be updated on each jail individually.

If an attacker manages to get an access through a weakness in one website, can he get to the host easier if I use the solution A?
Click to expand...
No. The attacker can't even modify the jail, only the website. The website was attacked which means anything the www user can do the attacker can too. Unless they elevated their privileges and became root only files that are writable by the www user can be changed. They would need a local exploit to increase their privileges and if you keep your systems nicely updated those chances are minute.
 
Ok, thanks for you enlightening answer!

Last thing: I guess using nginx on the host to redirect traffic to the different webserver jails can be combined to the use of pf to redirect other things (traffic to an ftp jail for instance)?
 
The "trick" with nginx and haproxy is they work on layer 7 and understand the HTTP protocols. Both are able to "forward" connections to different hosts based on the "Host:" header of the HTTP headers. There is no such header within the FTP protocol so there's nothing to differentiate traffic destined for jail1 or jail2.
 
So, in order to redirect ftp traffic to different jails, is it possible to use pf and different ports? (each ftp jail matching one port, that the client must know to be able to connect)
 
Well - yes that's possible to redirect different ports to different Jails, though it might not make much sense. Let alone the problem with choosing passive port ranges you've to redirect as well to the matching jails.

One possible solution for customers without too much hassle on the client (using FTP to connect to specific ports etc.) might be the following setup:

For Port 80:

External IP -> Port 80 redirect to a reverse proxy (e.g. Nginx) -> based on the HOST variable redirect to Jail 1, Jail 2, Jail 3, Jail 4 etc.

For FTP:

External IP -> Port 21 (and passive ports) -> FTP Jail

Inside the FTP jail create separated directories for each user account and "mount" them into the corresponding webserver jail e.g. with mount_nullfs
 
Ok, thanks! So it seems using pf is almost unavoidable to setup several jails (each one "hosting" a different service, like a ftp server, a webserver etc.).

I will look further into this once I have managed to setup a first jail correctly (that's another topic!).
 
To your third question: I've never used sysutils/ezjail, but it's definitly possible. I do that at work to great extend.
This explains the basics: https://www.freebsd.org/doc/handbook/jails-application.html
And you can just create a basejail and mount it completely and only link your nginx config and webroot folder.

Our running jails without deployed websites only contain a few folders and no single binary or any other file.

Edit: It's probably a good idea to also have a writable /var
 
SirDice said:
The "problem" is that you can only forward port 80 once per IP address. The solution is not to forward traffic based on layer 3 (IP) but on layer 7 (HTTP). To do that you need a proxy. This could be done with nginx or something like net/haproxy.
Click to expand...
+1
That is exactly way to do it. It is too bad that FreeBSD port of relayd is obsolete IIRC and can't do layer 7 proxy. I am personally starting to switch some of the less feature websites to OpenBSD httpd which is ported to FreeBSD. Ideally one would like to be able to take the advantage of Jails on the top of ZFS mirror using iocage but at the same time to use simple OpenBSD httpd and relayd for proxy-ing. In my case. My relayd proxy runs on OpenBSD firewall and my websites are in FreeBSD Jails on the top of ZFS mirrors. OpenBSD httpd runs very well on FreeBSD. I use cronjob to snapshot jails and remote replicate them.
 
Thanks for your answers!

So, as a conclusion:

1. and 2. I will take the solution of nginx (or maybe net/haproxy) as http proxy to redirect http and https traffic to the jails. While pf will redirect traffic either to the http proxy or to the ftp jail.

3. Looks possible from krawall's post. I won't try to look further into it yet (won't have a lot of jailed webservers anyway and I'll have to resolve other difficulties).

4. Introduces security problems, so this idea is to avoid.

Now I'm still beginner and need to read and learn a lot to set up all this! Thank you!
 
You must log in or register to reply here.
Back
Top