Need to use less power!

J

JayArr

Hi All

My servers have been running fine for years and years but they are getting old and expensive. I'm in the middle of a power audit and the rack is coming up as a major consumer. It's looking like 20-25kWh per day.

Current:
DL320 - PFSense firewall with an extra nic, no HD and just a USB key
DL360 - Desktop with X - this is my computer, single 72GB raid
DL360 - Apache, Finance software, storage/archives, 3 72GB raids
DL360 - PBX with Asterisk and a Digium card to interface to PTSN, single 72GB raid

I think spinning all those disks is a big part of the problem but I'm also running way more CPU power than needed for the applications I have. I set this up this way because these servers were only $50 each when I bought them and electricity was only 7 cents per kWh at the time. Now it's 16 cents per kWh and I have added raids over the years.

First action: replace the firewall with a box from Amazon that also has PF Sense , a SSD and only draws 6 watts.

My next action will be to try to consolidate everything onto one server and use only SSDs. I have a DL380 G5 that isn't doing anything and has 8 bays. My idea is to get 6 120GB SSD drives in three arrays and try to load everything I need onto one server.

FreeBSD 13
xorg so I have a desktop
NFS and SMB for file sharing/archiving
Apache, Postgresql, Pearl and SQL-Ledger (finance software)
Asterisk PBX for the phones.

Does anyone think there will be a problem running all this on a DL380 G5

Comments on whether this setup will use less power than three DL360s spinning 10 HDs?
 
Well, you need to tell us a bit more. For all we know you are running 4 dual-Xeon v2 with dual PSUs in there. What is the spec of a g5 DL380 (320, 360)?

Why do you have so many individual, very small RAID arrays? Mechanical disks take 8-10 watts each. Still, it would normally take a long while to make the money from replacing them with SSDs back, but since you seem to have so little storage that changes things.
 
Well there was a reason why you got them for $50 - those are old servers. Each averaging on ~300W per box is not unreasonable. I think these boxes had dynamic power saving features. Do have a look in BIOS config (RBSU) and look for power management or alike settings. I don't remember from top of my head. I remember it used to mess our serviceguard settings (don't recall why); we did set them all to full power, no energy savings.

JayArr said:
Does anyone think there will be a problem running all this on a DL380 G5
Click to expand...
Impossible to answer as we don't know what you're doing, what load do you expect, what reaction times these all should have. And also if this is commercial business what SLAs do you guarantee.

Most likely you'd be better off with the new entry level board though.
 
24kWh a day, that means the servers draw 1000W (250W each on average). When you say rack, does that mean your servers are rented in a colocated space? Or do they run in your basement? If the latter, a point to ponder might be whether to switch to a custom built box.

I've transitioned from overpowered machines to building something more energy saving, but still with the bang to do what I do almost daily (compiling kernels & ports).
At the top of the bang/buck spectrum I now have an 8 core/16 threads AMD Ryzen 7 5700G (G=graphics builtin), with 64G of RAM and 2 x 2TB SSDs configured as a ZFS mirror. When busy on all 16 threads after "make -j 16" it draws 120W, and 25 to 30 W when there's not much to do. And all that in a form factor of 16x16x8 cm! Less than 3 liters. That's an AsRock DeskMini X300. All components together set me back about EUR 700.

The 5700G has a 65W TDP; you can get to 35W with the GE series (E = energy efficient) but these are pricier.

Someone must have produced a table with MIPS/W somewhere I guess :)
 
Looks like a g5 DL380 is an original Nehalem. Shouldn't take 250 W idle with a couple of disks. Of course we don't know what the actually running servers are since the OP doesn't specify.
 
OK, lots of data to supply:

They are in a rack in my house, as is my electronics repair company. Not in a cloud or off-site.

They all have two CPUs that are Xeons dual cores running around 2.66GHz

The raid arrays came about because each new task was built on a new server with a raid for the OS and a separate raid for storage. I thought that was smart since I could move the storage around if needed and didn't have to touch the data when upgrading the OS. Nicely separated but at the expense of extra power consumption.

The PBX was put onto it's own server because at the time there was no stable Asterisk binaries for FreeBSD, it's a Shmooz linuz with the FreePBX version of Asterisk. It loaded off a single DVD in 30 minutes and was working within the hour. It's main function is to allow me to screen out robocalls and salespeople and to record every call incoming and outgoing to an archive.

I've been bitten by HD failures so I really like the raids, I hope that I can store everything on 240GB on the new server.

The DL380 has dual power supplies but only one CPU, still an Intel 2.33GHz dual core, I haven't looked but likely the same era Xeon.
 
That's assuming that one server has enough CPU/mem/HD power to run all the applications without slowing down.

Yes, despite being home based this is a business, repairing circuit boards for over 25 years. We've got a few employees, a web site, Apache/perl/postgreql finance software, a phone system etc. This isn't something I want to try to run on a white box from Staples. I chose FreeBSD almost 20 years ago to get rid of the Microsoft headache and it's worked great. I only need to address IT about every five years now.

Maybe my question is wrong.
 
Well this week I bought a bunch of SSD drives. These are older but should be a savings over spinning discs. The HDs I have are already a concern since some of them are approaching 50K hours, I've known for a year this setup was end of life.

The drives I bought are HP P/N 570774-002 Samsung Midline series. I don't care if they are slow, I have a bunch of apps but very little load, if they are slower than more modern stuff I probably won't even notice.

I thought I would have to use then in my DL380 since they weren't listed as compatible with the DL360G5. I tried a pair in a spare DL360 and they work! That's even better since the 360 has a smaller power supply.

I bought a stand alone firewall that only draws 6W and a standalone PBX that draws 8W.

Now all I have to do is combine my desktop and finance server onto one 360 with SSD and I should be able to shave 3/4 of my power requirements.
 
JayArr said:
I bought a stand alone firewall that only draws 6W and a standalone PBX that draws 8W.

Now all I have to do is combine my desktop and finance server onto one 360 with SSD and I should be able to shave 3/4 of my power requirements.
Click to expand...

Sounds sound. Consider keeping one of the 360s? as a same-configured backup, cold not hot, for any possible catastrophic failure of your new single server?
 
I don't know why your server has that drive's configuration, it's not my business, and for sure you will have your reasons,
but 3 arrays consisting each of 6 times 120G jumped to my eyes:
That's 2TB on 18 drives.

Besides looking for power supplys with high energy efficiency (gold, even better titanium),
which will pay of in short time, especially in 24/7 use,
and try to match the size of powersupply the usage - efficiency significantly drops when the used power is way lower than the supply's rated value,
I recommend to check if and how to reduce the number of drives.

18 SSD drives at 6W would need 108W, so the drives alone use > 2.5 kWh per day.
Having e.g. two 2TB drives as a mirror also would give you some redundancy,
but significantly reduce the amount of drives, so the amount of energy, even whith "turning rust".
 
There must be a misunderstanding somewhere Profighost, I don't have that many drives at all.

I hope to fit everything on a pair of mirrored RAIDS. each raid will be two 120GB SSDs for a total of 240GB.Both RAIDS will be on one server and all applications other than the firewall and the PBX will be on it.

I Currently run:

PFSense
PostgreSQL
Apache (internal only not outward facing)
Perl
SQL Ledger (Finance software)
SAMBA
NFS
Shmooz/FreePBX/Asterisk

and X and desktop apps like Firefox, OpenOffice etc.

Dedicated boxes for firewall and PBX eliminates two servers and one RAID (the firewall never had disks)

That leaves two servers with three 72 GB RAIDS - I'll condense them into one server with two RAIDS of 120GB SSDs.
 
That's a great idea Smithi, I'll keep a hardware matching unit in the rack that I can move the drives to in case of a failure.
 
I replaced a DL320G5 with a Barracuda/PFSense box and the monitor says I only saved 120W. Still that's $165/yr.

I think it's low because it had a single power supply, a single CPU and no hard drives.

Next up is to get the new Grandstream 6202 running and take down a DL360.
 
To get serious about reducing your computing electric consumption, your really should be thinking about ARM.

I've found that a Raspberry Pi 4 with 8GB of RAM and a USB attached SSD is a surprisingly fast and capable system. When they're back in stock, buy 3 of them. One to run your linux PBX, one to run everything else on FreeBSD, and one to play with and use as a backup. You'll use way less power than with that DL320.

or...

Buy any M1 or M2 powered mac. Mini. Macbook. Whatever. Install VMware Fusion on it. Then run FreeBSD arm64 in a VM. It's very very fast and you can also run windows and linux on it as additional VMs. Excellent performance for very very low power budget.
 
schweikh said:
24kWh a day, that means the servers draw 1000W (250W each on average). When you say rack, does that mean your servers are rented in a colocated space? Or do they run in your basement? If the latter, a point to ponder might be whether to switch to a custom built box.

I've transitioned from overpowered machines to building something more energy saving, but still with the bang to do what I do almost daily (compiling kernels & ports).
At the top of the bang/buck spectrum I now have an 8 core/16 threads AMD Ryzen 7 5700G (G=graphics builtin), with 64G of RAM and 2 x 2TB SSDs configured as a ZFS mirror. When busy on all 16 threads after "make -j 16" it draws 120W, and 25 to 30 W when there's not much to do. And all that in a form factor of 16x16x8 cm! Less than 3 liters. That's an AsRock DeskMini X300. All components together set me back about EUR 700.

The 5700G has a 65W TDP; you can get to 35W with the GE series (E = energy efficient) but these are pricier.

Someone must have produced a table with MIPS/W somewhere I guess :)
Click to expand...
Forgive my newbiness, but would that ASRock DeskMini X300 run as a desktop? Will the graphics support KDE etc? Thanks,
 
tnpimatt said:
Buy any M1 or M2 powered mac. Mini. Macbook. Whatever. Install VMware Fusion on it. Then run FreeBSD arm64 in a VM. It's very very fast and you can also run windows and linux on it as additional VMs. Excellent performance for very very low power budget.
Click to expand...
Looking at the stats here:

https://web.eece.maine.edu/~vweaver/group/green_machines.html

It does appear that the M1 is currently winning in terms of GFLOPS/W. However if Apple's DRM is against policies, then the Pi 5 seems next closest.

That said, if fully utilized, a bog standard haswell machine does surprisingly well, so perhaps the solution is to collapse your infrastructure into a smaller number of generic machines (i.e Jails / VMs).
 
Just curious... do the guys complaining about power consumption own an EV automobile?
?

Unless the business is supporting 24x7 clients, there is no need to leave it powered up and spinning all night long or when away from the home office.
Everything that moves, wears out.

My 24x7 customers get disk replacements every 26,280 hours as a preventative measure.
I can replace the disk on my schedule, or the machine's schedule, which is usually at the most inopportune time... same as car tires.

I power up a big ESXi host (all SSD) when I need a VM for a specific purpose.
None of those VMs are daily drivers.

My daily workload is development on a workstation accessing source code on a server.
Both are shut down at bed time.
 
  • Like
Reactions: mro
kpedersen said:
https://web.eece.maine.edu/~vweaver/group/green_machines.html
Click to expand...
For a second I was proud I'm running a Haswell-EP rack server (for ports building) which is 3rd on that list and thus the best x86 processor line, but then I noticed that that's simply the most modern x86 on the list. A quick check on Intel APP Metrics and ARK's Intel® Core™ i9 processor 14900K tells me it's around 4.85 GFLOPS/W for a modern processor. Not sure how the current Xeon's fare, but in the past they were better than consumer hardware.

Simply put: more modern machines will have much more processing power for the same electrical power. Quite logical of course. Using smaller, more modern machines will definitely impact power consumption and thus electrical bill and is what I would recommend for you current workload. No need to replace them with new big rack servers if you're serving a few websites. Modern rack servers will set you back a lot in purchase price and probably have a comparable or higher power consumption (at load, but also at idle).
To give you an idea: my Lenovo x3650 M5 (comparable to a HP DL380 G9 - far from modern but much more recent than a G5) consumes over 500W at full load. For that consumption one can run several modern (server) motherboards with ECC memory and a single CPU socket.

Edit: the GFLOPS/W number on that website is apparently calculated based of off the average power, while I calculated the 4.85 above from the max turbo TDP. Running the actual benchmark in the same way will likely result in a higher GFLOPS/W. I'm actually impressed by the current x86 crop.
 
Zagzigger said:
Forgive my newbiness, but would that ASRock DeskMini X300 run as a desktop? Will the graphics support KDE etc? Thanks,
Click to expand...
Yes, it's my desktop machine and it easily drives a 4K monitor playing 4K full screen streams from youtube. KDE is just another user interface on top of X11. All of them work easily. But it's not a gaming rig optimized for frames per second.
 
bgavin said:
Unless the business is supporting 24x7 clients, there is no need to leave it powered up and spinning all night long or when away from the home office.
Click to expand...
Lets break this down. ISP to device (cable modem) is up all the time so "always internet". Then a firewall device again up all the time because "streaming TV", so we have at least cable modem, firewall device, maybe switch and wifi AP for the TV. Everything else is up as needed
 
You must log in or register to reply here.
Back
Top