Weird trouble with Nginx/Apache reverse proxy on Jail.

Wamphyre

Wamphyre

Hello.

I'm experiencing a strange problem in my FreeBSD 11.1 jailed web server.

4 hours ago I needed to replace the hard drive of my home server, because was broken and I reinstalled FreeBSD with exactly the same config before hard drive change, where the jailed server was doing the work so well.

All the base/network hardware are the really same, even the same software installed.

Jailed server:

Apache24
NGINX as reverse proxy
MariaDB
PHP 7.1
Varnish

Now, the problems appeared...

I can load HTML pages with no trouble: https://venenonegro.tk/

But, index.php files doesn't work: https://rootmaster.tk/

Even if I try to load phpmyadmin (for example) it loads with only raw code, no kind of correct rendering.

These two domains have their own NGINX vhost to separate directories. Permissions and directives are correct.

Sounds like some kind of trouble with PHP, but I don't sure, because before the system reinstall, all was working flawlessly.

Back to server config, there's the nginx.conf:

Code: 
user  www;
worker_processes  1;
error_log  /var/log/nginx/error.log;
events {
worker_connections  1024;
}
http {
include       mime.types;
default_type  application/octet-stream;
log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log;
sendfile        on;
keepalive_timeout  65;
# Nginx cache configuration
proxy_cache_path    /var/nginx/cache levels=1:2 keys_zone=my-cache:8m max_size=1000m inactive=600m;
proxy_temp_path     /var/nginx/cache/tmp;
proxy_cache_key     "$scheme$host$request_uri";
gzip  on;
server {
listen       80;
server_name  (my server name);
location /nginx_status {
stub_status on;
access_log off;
}
# redirect server error pages to the static page /50x.html
#
error_page   500 502 503 504  /50x.html;
location = /50x.html {
root   /usr/local/www/nginx-dist;
}
# proxy the PHP scripts to Apache listening on 127.0.0.1:8080
#
location ~ \.php$ {
proxy_pass   http://127.0.0.1:8080;
include /usr/local/etc/nginx/proxy.conf;
}
}
include /usr/local/etc/nginx/vhost/*;
}

Now, the proxy.conf:

Code: 
proxy_buffering         on;
proxy_redirect          off;
proxy_set_header        Host            $host;
proxy_set_header        X-Real-IP       $remote_addr;
proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size    10m;
client_body_buffer_size 128k;
proxy_connect_timeout   90;
proxy_send_timeout      90;
proxy_read_timeout      90;
proxy_buffers           100 8k;
add_header              X-Cache $upstream_cache_status;

That's one of my vhosts:

Code: 
server {
# Replace with your FreeBSD Server IP
listen my_server_ip:80;
# Document Root
root /usr/local/www/apache24/data/;
index index.php index.html index.htm;
# Domain
server_name www.mydomain.com mydomain.com;
# Error and Access log file
error_log  /var/log/nginx/mydomain-error.log;
access_log /var/log/nginx/mydomain-access.log main;
# Reverse Proxy Configuration
location ~ \.php$ {
proxy_pass http://127.0.0.1:8080;
include /usr/local/etc/nginx/proxy.conf;
# Cache configuration
proxy_cache my-cache;
proxy_cache_valid 10s;
proxy_no_cache $cookie_PHPSESSID;
proxy_cache_bypass $cookie_PHPSESSID;
proxy_cache_key "$scheme$host$request_uri";
}
# Disable Cache for the file type html, json
location ~* .(?:manifest|appcache|html?|xml|json)$ {
expires -1;
}
# Enable Cache the file 30 days
location ~* .(jpg|png|gif|jpeg|css|mp3|wav|swf|mov|doc|pdf|xls|ppt|docx|pptx|xlsx)$ {
proxy_cache_valid 200 120m;
expires 30d;
proxy_cache my-cache;
access_log off;
}
}

The following are in the Includes/php.conf of Apache:

Code: 
<IfModule dir_module>
DirectoryIndex index.php index.html
<FilesMatch "\.php$">
SetHandler application/x-httpd-php
</FilesMatch>
<FilesMatch "\.phps$">
SetHandler application/x-httpd-php-source
</FilesMatch>
</IfModule>

What's wrong here?

Thanks for the help :)
 
Why are you running Apache and nginx on the same host/jail? Or is nginx running on the host and Apache inside the jail?
 
SirDice said:
Why are you running Apache and nginx on the same host/jail? Or is nginx running on the host and Apache inside the jail?
Click to expand...

Hello! Apache and Nginx are running in the same Jail because I don't really need to get them working separate. Anyway, 4 hours ago this config was working well.

It's a recommended way to separate Nginx and Apache for jails config?
 
Wamphyre said:
It's a recommended way to separate Nginx and Apache for jails config?
Click to expand...
Not this way, it would make sense if you ran nginx on the host with Apache in a jail. But in this case I'd remove nginx, it's over-complicating the configuration with little to no benefit.
 
PHP, like many other plugins, doesn't not get automatically configured and you have to do that yourself. Ergo, I can't help but think that you didn't restore all your configuration files.

To get PHP working I normally add Includes/php5.conf manually:

Code: 
<IfModule mod_php5.c>
        DirectoryIndex index.php
        AddType application/x-httpd-php .php
        AddType application/x-httpd-php-source .phps
</IfModule>
Ironically enough it does get enabled in httpd.conf but yeah, that's kind of useless without the rest.
 
I finally changed all my jailed server stack for:

Nginx + PHP-FPM + PHP 7.1 + MariaDB + Varnish and I did a script for automated stack install and config :)

This time, all works flawlessly! So it's Solved! (by hard ways, but yes) thank you all!


Code: 
#!/bin/bash
echo AutoFEMP by Wamphyre
echo Version 1.0

echo 'Bienvenido al script de instalación de AutoFEMP';

echo 'Esto instalará NGINX, PHP 7.1, PHP-FPM, MariaDB, Varnish y phpmyadmin en tu máquina';

echo 'Tienes 10 segundos para cancelar la instalación...';

sleep 10;

pkg update && pkg upgrade -y;

pkg install -y htop nano wget curl zip unzip;

pkg install -y nginx mariadb102-client mariadb102-server php71-pdo php71-pdo_mysql php71-xml php71-filter php71-posix php71-bcmath php71-zip php71 php71-mysqli mod_php71 php71-mbstring php71-gd php71-json php71-mcrypt php71-zlib php71-curl php71-session;

cp /usr/local/etc/php-fpm.d/www.conf{,.backup}

mv /usr/local/etc/php.ini-production /usr/local/etc/php.ini.original;

cd /usr/local/etc/;

wget nekromancerecords.tk/php.zip;

unzip php.zip;

rm -rf *.zip;

cd /usr/local/etc/nginx;

mv nginx.conf nginx.conf_original;

wget nekromancerecords.tk/nginx.zip;

unzip nginx.zip;

rm -rf *.zip;

mkdir -p /var/nginx/cache;
mkdir -p /var/nginx/tmp;
mkdir -p /var/log/nginx/;
cd /usr/local/etc/nginx/;
mkdir vhost;
cd vhost;
touch vhost_test.conf;

echo 'server {
        listen       127.0.0.1:8080;
        server_name  www.mydomain.com mydomain.com;
        location / {
        root /usr/local/www/nginx/;
        index  index.php index.html index.htm;
        error_log  /var/log/nginx/mydomain-error.log;
        access_log /var/log/nginx/mydomain-access.log main;
        location ~ \.php$ {
            fastcgi_pass   127.0.0.1:9000;
            fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME  /usr/local/www/nginx$fastcgi_script_name;
            include        fastcgi_params;
        proxy_cache my-cache;
        proxy_cache_valid 10s;
        proxy_no_cache $cookie_PHPSESSID;
        proxy_cache_bypass $cookie_PHPSESSID;
        proxy_cache_key "$scheme$host$request_uri";
}
        location ~* .(?:manifest|appcache|html?|xml|json)$ {
        expires -1;
}
        location ~* .(jpg|png|gif|jpeg|css|mp3|wav|swf|mov|doc|pdf|xls|ppt|docx|pptx|xlsx)$ {
        proxy_cache_valid 200 120m;
        expires 30d;
        proxy_cache my-cache;
        access_log off;
}
}
}' >> vhost_test.conf;

cd;

echo 'Instalando Varnish';

sleep 2

pkg install -y varnish5;

sysrc varnishd_enable=YES
sysrc varnishd_listen=":80"
sysrc varnishd_backend="localhost:8080"
sysrc varnishd_storage="malloc,512M"
sysrc varnishd_admin=":8081"

sysrc mysql_enable="YES"
sysrc mysql_args="--bind-address=127.0.0.1"
sysrc php_fpm_enable="YES"
sysrc nginx_enable="YES"

service mysql-server start
service varnishd start
service php-fpm start
service nginx start
service php-fpm restart

echo 'Instalando phpmyadmin';
sleep 3

cd /usr/local/www/nginx/;
wget https://files.phpmyadmin.net/phpMyAdmin/4.7.7/phpMyAdmin-4.7.7-all-languages.zip;
unzip phpMyAdmin-4.7.7-all-languages.zip;
rm -rf *.zip;
mv phpMyAdmin-4.7.7-all-languages phpmyadmin;
cd;

echo 'Corrigiendo permisos';

sleep 1

chown -R www:www /usr/local/www/nginx/
chown -R www:www /usr/local/www/nginx/*

echo 'Optimizando sistema...';

sleep 2

sysrc pf_enable="YES"
sysrc pflog_enable="YES"
sysrc clear_tmp_enable="YES"
sysrc syslogd_flags="-ss"
sysrc sendmail_enable="NONE"
sysrc dumpdev="NO"
sysrc powerd_enable="YES"
sysrc powerd_flags="-a hiadaptive"
sysrc ntpd_enable="YES"
sysrc ntpdate_enable="YES"

echo 'Securizando sistema...';

sleep 2

echo 'kern.elf64.nxstack=1' >> /etc/sysctl.conf;
echo 'sysctl security.bsd.map_at_zero=0' >> /etc/sysctl.conf;
echo 'security.bsd.see_other_uids=0' >> /etc/sysctl.conf;
echo 'security.bsd.see_other_gids=0' >> /etc/sysctl.conf;
echo 'security.bsd.unprivileged_read_msgbuf=0' >> /etc/sysctl.conf;
echo 'security.bsd.unprivileged_proc_debug=0' >> /etc/sysctl.conf;
echo 'kern.randompid=1000' >> /etc/sysctl.conf;
echo 'security.bsd.stack_guard_page=1' >> /etc/sysctl.conf;
echo 'net.inet.udp.blackhole=1' >> /etc/sysctl.conf;
echo 'net.inet.tcp.blackhole=2' >> /etc/sysctl.conf;

echo 'Revisando y limpiando archivos temporales...';

pkg update && pkg upgrade -y;
pkg clean;
pkg audit -F;

echo 'FINALIZADO';

echo 'Tu servidor FEMP con Varnish y phpmyadmin ya está instalado';

echo 'Ejecuta ahora el comando /usr/local/bin/mysql_secure_installation';

echo 'Reinicia el equipo en cuanto te sea posible';
 
There are a few caveats with your script.
  • For starters /bin/bash doesn't exist on FreeBSD. It's either /bin/sh (recommended) or /usr/local/bin/bash (if you insist on using bash(1)).
  • php71-mysqli depends on MySQL 5.6, which is going to remove your MariaDB. Unless it comes from your own repository?
  • Instead of curl(1) or wget(1), use fetch(1).
  • This is fairly useless: mv /usr/local/etc/php.ini-production /usr/local/etc/php.ini.original;. The php.ini-production is an example and never used. So there's really no need to move it out of the way.
 
Hi!

Yes, you're right about these tips, but, for example I'm really used to use bash for all and lauch scripts with sh but anybody can modify the script :)

About php71-mysqli it is compatible with MariaDB and doesn't remove it by MySQL. I ever use this PHP extension with MariaDB:

Code: 
root@nk02:~ # mysql --version
mysql  Ver 15.1 Distrib 10.2.13-MariaDB, for FreeBSD11.1 (amd64) using readline 5.1

About fetch and wget commands, what's the real difference on the execution?

By the php.ini.original yeap, you're right that's useless, just a "commodity" for me :P

Thanks for the tips and help!
 
Wamphyre said:
About [BGCOLOR=#dee3e7] fetch[/BGCOLOR] and [BGCOLOR=#dee3e7] wget[/BGCOLOR] commands, what's the real difference on the execution?
Click to expand...
Nothing really, options are slightly different but the biggest reason is that fetch(1) is included in the base OS, so you don't need to install anything for it.
 
You must log in or register to reply here.
Back
Top