mbuf_cluster exhaustion

We have dozens of FreeBSD hosts running, most in VM's on Proxmox, all using virtio NIC's.
All are deployed with ansible, so there should be no pet configs lying around.

*One* of the hosts has, for the last 3 or 4 months started to exhaust available mbuf_clusters and becomes unresponsive.
We set `kern.ipc.nmbclusters` to 3000000, which only stalls the problem, used mbuf clusters keeps rising until it hits the
limit.

We've narrowed it down to apache24 (and php-fpm). We don't see the problem if only apache is started without php-fpm,
but then there is very little that apache serves besides it's status page.

We have plenty of other hosts running apache and php-fpm which don't display this behaviour.

What is the best way for me to debug the issue and try to pinpoint the cause?
 
This is a graph showing mbuf_cluster for the last hour.
There are multiple network services running on the host, but if I stop php-fpm, the mbuf_cluster use stops growing.

Screenshot 2024-11-20 at 10.32.09.png
 
Back
Top