PHP-FPM and MYSQL in separate jail than nginx

Hi,
after I was able to setup a nginx reverse proxy in jail to serve traffic for other jails I am stuck at another point:
Nginx+Mysql+PHP in one jail works perfect for one website(and probably for more than 1). But the point of jails is that we want to run one site or one application in one jail. As it is known if we install multiple instances of mysql in multiple jails it will eat the server memory fast, so I tried to install PHP and MYSQL in a jail and NGINX with site contents in another jail(several sites, several jails with nginx only, all of them using the single php-mysql jail)
As I read around there are two ways of configuring php and mysql in separate jails: by using mount_nullfs and by using nginx proxy at the network level. I have read that the solution using mount_nullfs might be faster cause it uses sockets for talking. I didn't give a try to this solution because I am using bastille jails and do not know how to mount directories from jail to jail(they provide only command for mounting host to jail). Therefor I tried the solution with nginx as proxy but it gives me a headache and cant have the php to talk to nginx at all. So far I didn't went further to mysql cause I am still stuck with php.
Appreciate if someone cand draw a nginx config example and a www.conf on php side and php.ini if there is necessary. It looks that the root directory which I set on nginx side can't be read by php side.
my nginx reads like this(nginx jail has IP 10.0.0.1 and php-mysql 10.0.0.4):

Code:
 server {
        listen 10.0.0.1:443 ssl http2;
  #      listen [::]:443 ssl http2;
        server_name example.com;
        root usr/local/www/example.com;
        index index.php;
        ssl_certificate /usr/local/etc/letsencrypt/live/example.com/fullchain.pem      
        ssl_certificate_key /usr/local/etc/letsencrypt/live/example.com/privkey.pem
        location / {
            try_files $uri $uri/ /index.php?$args;
        }

        location ~ \.php$ {
            try_files $uri =404;
            fastcgi_split_path_info ^(.+\.php)(/.+)$;
            fastcgi_pass 10.0.0.4:9000;
            fastcgi_index index.php;
            include fastcgi_params;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            fastcgi_param PATH_INFO $fastcgi_path_info;
            
        }
    }
I also set in www.conf on php jail the listen 10.0.0.4:9000 and listen.allowed_clients 10.0.0.1
with the above config I get the error "no input file specified"
If I play around with root path I get 404
 
PHP doesn't use that much memory. MySQL on the other hand does, depending on how it's configured (most people configure it incorrectly).
 
Other methods then (fastcgi_pass)php-fpm will use more memory ...
fastcgi_pass is used in all cases, only when it is used localy, in same jail with nginx it uses unix socket(fastcgi_pass unix:/var/run/php81-fpm.sock) and when used remote, in another jail, it uses network level(fastci_pass xxx.xxx.xxx.xxx:9000). The problem is that if I install 10 websites in 10 jails then I will have 10 php installations and the question in regards of memory is what is worth better, to run one instance of php in remote jail and to go through hell with configurations or to run in every jail a php instance with no configuration headache
 
mysql has many tunables.
Here mine for mariadb,
Code:
###Zfs recordsize 16K
innodb_log_write_ahead_size=16384
key_buffer_size=256M
innodb_log_file_size=256M
tmp_table_size=256M
max_heap_table_size=256M
join_buffer_size=256M
key_buffer_size=256M
sort_buffer_size=256M
aria_pagecache_buffer_size=256M
innodb_buffer_pool_size=1G
 
Hi, I'm do that as OP mention - in separate jails and communication throght sockets, which binds as nullfs:
/etc/fstab
Code:
# Binded folders MySQL Socket
/var/mysql              /usr/jails/db_mysql/var/mysql   nullfs  rw,late 0 0
/var/mysql              /usr/jails/mail_postfix/var/mysql       nullfs  rw,late 0 0
/var/mysql              /usr/jails/voip_asterisk/var/mysql      nullfs  rw,late 0 0
/var/mysql              /usr/jails/www_admin/var/mysql  nullfs  rw,late 0 0
 
Hi, I'm do that as OP mention - in separate jails and communication throght sockets, which binds as nullfs:
/etc/fstab
Code:
# Binded folders MySQL Socket
/var/mysql              /usr/jails/db_mysql/var/mysql   nullfs  rw,late 0 0
/var/mysql              /usr/jails/mail_postfix/var/mysql       nullfs  rw,late 0 0
/var/mysql              /usr/jails/voip_asterisk/var/mysql      nullfs  rw,late 0 0
/var/mysql              /usr/jails/www_admin/var/mysql  nullfs  rw,late 0 0
are you binding folders host2jails or jail2jail? It seems that you have mysql on host. I have it in jail. Not sure how to bind folders from jail to jail(I use Bastille)
 
are you binding folders host2jails or jail2jail? It seems that you have mysql on host. I have it in jail. Not sure how to bind folders from jail to jail(I use Bastille)
I have a host dir
/var/mysql
and it binds to every jails's <<JAIL_ROOT>>/var/mysql

MySQL also in his own jail which name db_mysql, and mysql creates socket at <<db_mysql_ROOT>>/var/mysql which in fact reflects to /var/mysql and exposed to all other jails.
 
I have a host dir
/var/mysql
and it binds to every jails's <<JAIL_ROOT>>/var/mysql

MySQL also in his own jail which name db_mysql, and mysql creates socket at <<db_mysql_ROOT>>/var/mysql which in fact reflects to /var/mysql and exposed to all other jails.
What about php, do you install it in every jail or in a single jail and bind it same as mysql?
 
If you separate PHP to its own jail you're going to need to install all the PHP applications there too. So you're essentially putting the cart before the horse.

Just configure MySQL to bind to the jail's IP address and have the PHP application(s) connect to that. It's not going to matter much speed-wise, as it's all local traffic anyway. Just make sure each application has its own database and configure proper user accounts with privileges to only be able to access the database it needs.
 
If you separate PHP to its own jail you're going to need to install all the PHP applications there too. So you're essentially putting the cart before the horse.

Just configure MySQL to bind to the jail's IP address and have the PHP application(s) connect to that. It's not going to matter much speed-wise, as it's all local traffic anyway. Just make sure each application has it's own database and configure proper user accounts with privileges to only be able to access the database it needs.
that makes sense, thanks again, I'll follow that pattern
 
You can run php-fpm & nginx in different jails using a nullfs mount. But it will unnecessary complate things
Unless I am missing something myself I think you're misunderstanding how a webserver interacts with something like PHP. This usually happens through (Fast)CGI. The webserver is not simply serving the *.php as a file like it would for a *.html.
I don't think that there is any reasonable/practical approach involving nullfs here.
 
I found this linux article,
&
&
 
That setup is still relying on a network overlay/component to do the actual CGI (eg. the communication between the webserver and PHP).

Anyway, let's not hijack the thread any further. As mentioned the reasonable thing to do is running both in the same jail. Hence also my notion of "reasonable/practical approach". Many things are possible, but few are actually reasonable/sensible.
 
Hence also my notion of "reasonable/practical approach". Many things are possible, but few are actually reasonable/sensible.
Indeed.

The first question that comes to my mind here is: Why would you ever want to do that?

Step back, why do you use jails at all? Canonical answer is, to isolate "things". But what exactly do you want to isolate here? Part of this thread looks like the answer was "every single technical component". But then, why do you want to isolate at all? Typical answers are a) to have some self-contained system that can operate independently and b) to confine the impact of successful security breaches.

By isolating along the lines of "technical components", you reach neither of these goals. The components must communicate in order to deliver a "service". Take the example above, having a single jail running mysql and many jails using it. So, all these jails will have to be allowed to communicate with the single mysql instance. And imagine the worst case, someone succeeds "hacking" your mysql through "jail A". Then, they can access all the data of jails "B", "C" and "D" as well.

Don't do that. Isolate services that form a logical unit. Put everything that's needed to deliver ONE specific service in one jail. And if that means you need 5 instances of mysql, 5 instances of nginx, 5 instances of php, whatever ... then just do it. That way, they can really be operated independent from each other, and security breaches will stay confined to one single jail.
 
Indeed.

The first question that comes to my mind here is: Why would you ever want to do that?

Step back, why do you use jails at all? Canonical answer is, to isolate "things". But what exactly do you want to isolate here? Part of this thread looks like the answer was "every single technical component". But then, why do you want to isolate at all? Typical answers are a) to have some self-contained system that can operate independently and b) to confine the impact of successful security breaches.

By isolating along the lines of "technical components", you reach neither of these goals. The components must communicate in order to deliver a "service". Take the example above, having a single jail running mysql and many jails using it. So, all these jails will have to be allowed to communicate with the single mysql instance. And imagine the worst case, someone succeeds "hacking" your mysql through "jail A". Then, they can access all the data of jails "B", "C" and "D" as well.

Don't do that. Isolate services that form a logical unit. Put everything that's needed to deliver ONE specific service in one jail. And if that means you need 5 instances of mysql, 5 instances of nginx, 5 instances of php, whatever ... then just do it. That way, they can really be operated independent from each other, and security breaches will stay confined to one single jail.
5 mysql instances will eat your all memory. We are talking about a budget vps scenario with 30G RAM. If you are a company with a high budget to spend on servers then other scenario are considered.
 
5 mysql instances will eat your all memory.
Why? Did you try?

We are talking about a budget vps scenario with 30G RAM. If you are a company with a high budget to spend on servers then other scenario are considered.
Then you could just as well not use jails at all. Or, maybe, put everything "internet-facing" into one single jail, which would give at least a little part of the advantage. "Isolating" components that must communicate anyways (so, you have to come up with tricks to allow that) won't give any relevant benefit over not isolating them at all.
 
If I'm not mistaken, sharing (read-only) text/rodata segments (binary itself as well as shared libs) should work across jails as well. So, it's just the actual data (bss and dynamically allocated) that will need extra RAM per instance. Relevance of that of course depends on the implemetation.

But anyways, if you can't properly isolate "vertically" (cutting by services offered, not by technical components), you lose much of what isolation can offer.
 
Back
Top