RAM allocation troubleshooting

fred974

fred974

Hi Guys,

I haven't got a problem on my server, this post is more about preventing problem from occuring and learning process...
I have a server with 48GB of ram installed and I can only see 2GB of free Ram available and it is puzzling me as I don't think I should be using that much..

I have 21 sysutils/iocage web jails all running their own webserver and version of php but all sharing 1 common databases/mariadb101-server database (also in a jail).

Here are some system info:
top
Code: 
last pid: 53101;  load averages:  2.21,  0.93,  0.60                                                                                                                                                                                                   up 4+17:23:58  10:37:57
287 processes: 1 running, 286 sleeping
CPU:  7.7% user,  0.0% nice,  0.4% system,  0.0% interrupt, 91.8% idle
Mem: 4597M Active, 4386M Inact, 35G Wired, 117M Cache, 2532M Free
ARC: 30G Total, 6723M MFU, 19G MRU, 35M Anon, 670M Header, 4797M Other
Swap: 4096M Total, 308M Used, 3788M Free, 7% Inuse
ps -aux
Give me the following result (too big for forum)
Looking at the ps output and adding all the %MEM, I don't get anywere neare available memory

Am I using the wrong command to analyse this problem? What service is using my resources

Thank you
 
fred974 said:
I have a server with 48GB of ram installed and I can only see 2GB of free Ram available and it is puzzling me as I don't think I should be using that much..
Click to expand...
Your server will eventually use all its memory, unused memory is useless memory. Most of this will be taken up by various caches, like file/directory cache, process cache, etc. There's nothing wrong with your system, it's doing exactly what it's supposed to be doing.

You will also always have some swap usage. It's just the system paging out stuff it doesn't need to have around. Swap usage in and of itself is not a problem. Excessive swapping however is a problem and is an indication the system could use more memory (or you've configured MySQL incorrectly and it's trying to use too much memory).
 
SirDice said:
Excessive swapping however is a problem and is an indication the system could use more memory
Click to expand...
Given that I have 48GB of ram, what value will be classified as "excessive"?

SirDice said:
(or you've configured MySQL incorrectly and it's trying to use too much memory).
Click to expand...
Good point.. I'll run mysqltuner to take a look..

this is the status now:
Code: 
last pid: 78552;  load averages:  0.39,  0.37,  0.44                                                                                                                                                                                                   up 4+21:21:35  14:35:34
326 processes: 1 running, 325 sleeping
CPU:  0.9% user,  0.0% nice,  0.1% system,  0.0% interrupt, 99.0% idle
Mem: 4784M Active, 3611M Inact, 38G Wired, 67M Cache, 303M Free
ARC: 21G Total, 3212M MFU, 13G MRU, 37M Anon, 627M Header, 4722M Other
Swap: 4096M Total, 125M Used, 3971M Free, 3% Inuse
The server is a bit slow but the swap usage hasn't changed
 
fred974 said:
Given that I have 48GB of ram, what value will be classified as "excessive"?
Click to expand...
It's not related to the amount of memory, it's related to how much is swapped in and out. You can tell by running vmstat 1 10 for example. Look at the pi (Page In) and po (Page Out) numbers. This will tell you how much is being paged. Zero or very low numbers are fine, high numbers indicate a lot of paging.
 
Ran vmstat 1 10 but didn't really understand the output..
I read the vmstat(8) but that wasn't any clearer to me..

I have however gain the understanding of what pi and po by reading this article I found on a web search :)

Thank you SirDice
 
fred974 said:
Ran vmstat 1 10 but didn't really understand the output..
I read the vmstat(8) but that wasn't any clearer to me..
Click to expand...

Here is an output from my server, a home NAS, running FreeBSD 10.3-RELEASE. vmstat tells you a lot.

CPU Idle is at the end.
PI and PO is paging.
Disks is the reading/writing. Look at mine. Notice how the busyness comes and goes. That shows files being read every now and then. My machine has two drives and the user files being read are on drive ad1. The FreeBSD OS is on drive ad0.
Active memory and Free should be explanatory.
Look at Procs r b w - run, blocked, runnable/sleep/swapped. A non-zero number in 'b' column indicates a process was blocked for resources. Frequently seeing a non-zero number here would be signs of an overloaded machine I do believe. I know I've tried a couple things on my server that made it very slugglish, and this number would stay around 3 or 4 continuesly, yet CPU and memory numbers were fine. I had made the hard drive 100% busy so some processes were not getting 'drive time' so to say.

CPU cs - context switching - not exactly sure what this means, but I do know in virtual machine environments that number can get much higher, and that can be normal.

CPU us and sy - user processes versus system process and what they are consuming.

Faults in - are CPU interrupts I do believe, which is one mechanism in how a CPU gets 'tasked' to tackle some work. NIC can interrupt a CPU for example.

I think ultimately you need to baseline your system when its new and healthy with lots of free resources, and over time as you add more programs, and those programs do more and more work, you should run vmstat and compare against your baselines.

vmstat -w 1 -c 25
Code: 
 procs      memory      page                    disks     faults         cpu
 r b w     avm    fre   flt  re  pi  po    fr  sr ad0 ad1   in   sy   cs us sy id
 0 0 0   1870M    42M    52   0   0   0   330 193   0   0  781 2574 3296 15  4 81
 0 0 0   1870M    42M     0   0   0   0     0  52   0   0  124  309  364  0  0 100
 0 0 0   1870M    42M     2   0   0   0     0  53   0   0  162  603  544  0  1 99
 0 0 0   1870M    42M     2   0   0   0     0  52   0   0  194  617  678  0  1 99
 0 0 0   1870M    42M     1   0   0   0     0  52   1   0  140  458  490  0  0 99
 0 0 0   1870M    42M     0   0   0   0     0  52   0   0  127  376  452  0  0 100
 0 0 0   1870M    42M     1   0   0   0     0  53   0   0  133  335  402  0  0 100
 2 0 0   1870M    42M     1   0   0   0     1  53   6   0  149  595  520  1  0 99
 0 0 0   1870M    42M     1   0   0   0     0  54   1   0  127  333  406  0  0 100
 0 0 0   1870M    42M    20   0   0   0     0  53   1   0  166  556  560  1  0 99
 0 0 0   1870M    42M     2   0   0   0   813  53   0 354 1281 21848 13083 21  8 71
 1 0 0   1870M    42M     0   0   0   0   955  52   0 423 1627 26085 18219 33  8 59
 2 0 0   1870M    42M     2   0   0   0   813  52   0 345 1586 25215 17533 35 10 55
 1 0 0   1870M    42M     1   0   0   0   801  54   0 336 1618 22602 17227 34  7 59
 1 0 0   1870M    42M     1   0   0   0   540  52   0 238 1446 21066 15697 36  6 58
 1 0 0   1870M    42M     3   0   0   0   306  52   0 135 1306 25519 18078 41 10 49
 1 0 0   1870M    42M     1   0   0   0   774  52   0 341 1543 22747 16148 32 10 57
 1 0 0   1870M    42M     1   0   0   0  1000 104   7 441 1651 22035 16554 29  7 64
 0 0 0   1870M    42M     1   0   0   0  1151  52   0 505 1666 24580 18096 29  9 63
 0 0 0   1870M    42M     2   0   0   1   329  52   2 143  917 13998 10364 18  7 75
 0 0 0   1870M    42M     1   0   0   0     0  52   0   0  188  521  600  0  0 99
 1 0 0   1870M    41M     2   0   0   0     1  52   0   4  190  550  583  0  1 99
 0 0 0   1870M    41M     1   0   0   0     0  52   0   0  212  846  742  1  0 99
 1 0 0   1870M    41M     1   0   0   0    54  52   0  24  232 3044 1518  3  1 96
 1 0 0   1870M    41M     1   0   0   0   441  52   0 194 1393 37710 16755 29 17 54

Show us yours if you want. I'm curious to see.
 
PacketMan, thank you for sharing your knoledge :)
Here is my output for vmstat -w 1 -c 25
Code: 
 r b w     avm    fre   flt  re  pi  po    fr  sr mf0 mf1   in   sy   cs us sy id
 0 0 11  45059M   579M  1556   0   1   0  1620 1308   0   0  674 3449 5408  2  1 97
 0 0 11  45059M   579M     2   0   0   0     0 2004 117 113  681  945 5899  0  0 100
 0 0 11  45059M   579M    84   0   0   0     0 2004   0   0   18 1685 1308  1  0 99
 0 0 11  45059M   579M  6385   0   0   0  6514 2004   0   0   63 2122 1312  0  0 99
 0 0 11  45059M   579M    84   0   0   0     0 2004   0   0   15  684  692  0  0 100
 0 0 11  45059M   579M     0   0   0   0     0 2004   0   0    2  543  668  0  0 100
 0 0 11  45059M   578M   254   0   0   0     0 2004  95  94  599 1247 5568  0  0 100
 0 0 11  45059M   578M    64   0   0   0     0 2004   0   0   75 2238 1830  1  0 99
 0 0 11  45059M   578M  6471   0   0   0  6514 2004   1   1   16 1496 1036  0  0 99
 0 0 11  45059M   578M     0   0   0   0     0 2004   0   0   12  561  671  0  0 100
 0 0 11  45059M   578M    85   0   1   0     1 2004   6   6   85 7389 1970  2  1 97
 0 0 11  45059M   578M     3   0   1   0     1 2004 205 192 1341 6751 11064  2  2 96
 0 0 11  45059M   578M    84   0   0   0     0 2004   5   6   31 1731 1550  0  0 100
 0 0 11  45022M   579M  6471   0   0   0  6807 2004   3   5   65 1618 1223  1  1 99
 0 0 11  44986M   580M   436   0   0   0   693 2004   0   0    9  767  736  0  0 100
 0 0 11  44986M   580M     0   0   0   0     0 4008   0   0   31  657  861  0  0 100
 0 0 11  44986M   580M   264   0   1   1   182 2004 136 134  922 1631 7993  0  0 100
 0 0 11  45043M   578M   574   0   0   0   127 2004   2   0   30 2483 1421  0  0 100
 0 0 11  45043M   578M  6504   0   0   1  6514 2004   1   1   21 1357  916  0  0 100
 0 0 11  44987M   580M   527   0   0   0   945 2004   0   0   18  892  688  0  0 100
 1 0 11  44934M   581M    32   0   0   0   365 2003   0   0   23  924  778  0  0 100
 0 0 11  44934M   581M    85   0   0   0     0 2003 125 125  850 7988 8350  3  1 97
 0 0 11  44934M   581M  6778   0   4   0   581 2003  16  16  145 12038 4023  3  1 96
 1 0 11  44934M   581M   185   0   0   0  6427 2003   2   4  116 1376 1230  1  0 99
 0 0 11  44934M   581M     0   0   0   0     0 2003   0   0  154 1928 1351  1  0 99
 
fred974 said:
I have 21 sysutils/iocage web jails all running their own webserver.....
Click to expand...

Hmmm, maybe my knowledge is not that good. I would have thought for a machine running 21 web servers it would have been a fair bit busier than mine. All of the columns, except flt and fr, generally seem less 'busy' than my little home NAS. Is this server in production or is it a test/lab/dev kinda of machine? If it is indeed in production it must be pretty light duty purpose.

The memory and cpu number seem cool too.
 
PacketMan
Ys it is production but we only just started so not much doing on yet.
We average 24000 visitor a month and use about 630GB bandwith monthly
 
You must log in or register to reply here.
Back
Top