So my 18-year old Areca hardware RAID card finally died. (I was prepared, I got backups).
This time I decided to build my file server with software raid only. (And use SSD-s).
The problem is the speed is not quite what I expected.
I am using RAID 1+0
I have 5x8TB and 10x4TB disks.
4TB disks are in pairs of gstripe and then mirrored with 8TB disks. And then that all in one big gstripe:
(systems disks not included here)
But that all big gstripe gives me about same read speed as single disk:
Should I replace internal (4+4) gstripe with gconcat instead, to get more speed?
Also, does it matter, that gmirror seems to have TRIM support, but gstripe does not? Seems gconcat does not support TRIM either.
Any other tips?
This time I decided to build my file server with software raid only. (And use SSD-s).
The problem is the speed is not quite what I expected.
I am using RAID 1+0
I have 5x8TB and 10x4TB disks.
4TB disks are in pairs of gstripe and then mirrored with 8TB disks. And then that all in one big gstripe:
(systems disks not included here)
Code:
# gmirror status
Name Status Components
mirror/k1 COMPLETE da0 (ACTIVE)
stripe/liit1 (ACTIVE)
mirror/k2 COMPLETE da1 (ACTIVE)
stripe/liit2 (ACTIVE)
mirror/k3 COMPLETE da2 (ACTIVE)
stripe/liit3 (ACTIVE)
mirror/k4 COMPLETE da3 (ACTIVE)
mirror/k5 COMPLETE da4 (ACTIVE)
# gstripe status
Name Status Components
stripe/liit1 UP ada3
da5
stripe/liit2 UP ada5
da6
stripe/liit3 UP ada7
da7
stripe/suur UP mirror/k4
mirror/k5
mirror/k1
mirror/k2
mirror/k3But that all big gstripe gives me about same read speed as single disk:
Code:
# diskinfo -t stripe/suur
stripe/suur
512 # sectorsize
40007816089600 # mediasize in bytes (36T)
78140265800 # mediasize in sectors
4096 # stripesize
0 # stripeoffset
No # TRIM/UNMAP support
Unknown # Rotation rate in RPM
Seek times:
Full stroke: 250 iter in 0.055887 sec = 0.224 msec
Half stroke: 250 iter in 0.042207 sec = 0.169 msec
Quarter stroke: 500 iter in 0.074149 sec = 0.148 msec
Short forward: 400 iter in 0.058676 sec = 0.147 msec
Short backward: 400 iter in 0.067298 sec = 0.168 msec
Seq outer: 2048 iter in 0.143136 sec = 0.070 msec
Seq inner: 2048 iter in 0.143004 sec = 0.070 msec
Transfer rates:
outside: 102400 kbytes in 0.228471 sec = 448197 kbytes/sec
middle: 102400 kbytes in 0.216937 sec = 472026 kbytes/sec
inside: 102400 kbytes in 0.166122 sec = 616414 kbytes/sec
# diskinfo -t da4
da4
512 # sectorsize
8001563222016 # mediasize in bytes (7.3T)
15628053168 # mediasize in sectors
0 # stripesize
0 # stripeoffset
972801 # Cylinders according to firmware.
255 # Heads according to firmware.
63 # Sectors according to firmware.
ATA Samsung SSD 870 # Disk descr.
S5SSNF0R901312J # Disk ident.
Yes # TRIM/UNMAP support
0 # Rotation rate in RPM
Not_Zoned # Zone Mode
Seek times:
Full stroke: 250 iter in 0.012770 sec = 0.051 msec
Half stroke: 250 iter in 0.011183 sec = 0.045 msec
Quarter stroke: 500 iter in 0.021915 sec = 0.044 msec
Short forward: 400 iter in 0.017358 sec = 0.043 msec
Short backward: 400 iter in 0.020292 sec = 0.051 msec
Seq outer: 2048 iter in 0.054629 sec = 0.027 msec
Seq inner: 2048 iter in 0.053259 sec = 0.026 msec
Transfer rates:
outside: 102400 kbytes in 0.215702 sec = 474729 kbytes/sec
middle: 102400 kbytes in 0.207632 sec = 493180 kbytes/sec
inside: 102400 kbytes in 0.207143 sec = 494344 kbytes/sec
# diskinfo -t mirror/k5
mirror/k5
512 # sectorsize
8001563221504 # mediasize in bytes (7.3T)
15628053167 # mediasize in sectors
0 # stripesize
0 # stripeoffset
Yes # TRIM/UNMAP support
Unknown # Rotation rate in RPM
Seek times:
Full stroke: 250 iter in 0.014900 sec = 0.060 msec
Half stroke: 250 iter in 0.012241 sec = 0.049 msec
Quarter stroke: 500 iter in 0.023651 sec = 0.047 msec
Short forward: 400 iter in 0.018956 sec = 0.047 msec
Short backward: 400 iter in 0.022050 sec = 0.055 msec
Seq outer: 2048 iter in 0.061932 sec = 0.030 msec
Seq inner: 2048 iter in 0.061643 sec = 0.030 msec
Transfer rates:
outside: 102400 kbytes in 0.217632 sec = 470519 kbytes/sec
middle: 102400 kbytes in 0.210079 sec = 487436 kbytes/sec
inside: 102400 kbytes in 0.211576 sec = 483987 kbytes/sec
# diskinfo -t ada7
ada7
512 # sectorsize
4096805658624 # mediasize in bytes (3.7T)
8001573552 # mediasize in sectors
0 # stripesize
0 # stripeoffset
7938069 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
4TB SATA 2.5 SSD # Disk descr.
2024102816000108 # Disk ident.
Yes # TRIM/UNMAP support
0 # Rotation rate in RPM
Not_Zoned # Zone Mode
Seek times:
Full stroke: 250 iter in 0.034545 sec = 0.138 msec
Half stroke: 250 iter in 0.065941 sec = 0.264 msec
Quarter stroke: 500 iter in 0.124298 sec = 0.249 msec
Short forward: 400 iter in 0.105638 sec = 0.264 msec
Short backward: 400 iter in 0.099888 sec = 0.250 msec
Seq outer: 2048 iter in 0.078301 sec = 0.038 msec
Seq inner: 2048 iter in 0.046663 sec = 0.023 msec
Transfer rates:
outside: 102400 kbytes in 0.487445 sec = 210075 kbytes/sec
middle: 102400 kbytes in 0.568197 sec = 180219 kbytes/sec
inside: 102400 kbytes in 0.267658 sec = 382578 kbytes/sec
# diskinfo -t ada5
ada5
512 # sectorsize
4000787030016 # mediasize in bytes (3.6T)
7814037168 # mediasize in sectors
0 # stripesize
0 # stripeoffset
7752021 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
Samsung SSD 870 EVO 4TB # Disk descr.
S6BCNF0W302128X # Disk ident.
id1,enc@n3061686369656d30/type@0/slot@6/elmdesc@Slot_05 # Physical path
Yes # TRIM/UNMAP support
0 # Rotation rate in RPM
Not_Zoned # Zone Mode
Seek times:
Full stroke: 250 iter in 0.026466 sec = 0.106 msec
Half stroke: 250 iter in 0.026360 sec = 0.105 msec
Quarter stroke: 500 iter in 0.049068 sec = 0.098 msec
Short forward: 400 iter in 0.040180 sec = 0.100 msec
Short backward: 400 iter in 0.040070 sec = 0.100 msec
Seq outer: 2048 iter in 0.034423 sec = 0.017 msec
Seq inner: 2048 iter in 0.034204 sec = 0.017 msec
Transfer rates:
outside: 102400 kbytes in 0.202335 sec = 506091 kbytes/sec
middle: 102400 kbytes in 0.202415 sec = 505891 kbytes/sec
inside: 102400 kbytes in 0.202378 sec = 505984 kbytes/sec
# diskinfo -t stripe/liit1
stripe/liit1
512 # sectorsize
8001574051840 # mediasize in bytes (7.3T)
15628074320 # mediasize in sectors
4096 # stripesize
0 # stripeoffset
No # TRIM/UNMAP support
Unknown # Rotation rate in RPM
Seek times:
Full stroke: 250 iter in 0.056910 sec = 0.228 msec
Half stroke: 250 iter in 0.334056 sec = 1.336 msec
Quarter stroke: 500 iter in 0.369228 sec = 0.738 msec
Short forward: 400 iter in 0.040594 sec = 0.101 msec
Short backward: 400 iter in 0.040107 sec = 0.100 msec
Seq outer: 2048 iter in 0.323392 sec = 0.158 msec
Seq inner: 2048 iter in 0.049658 sec = 0.024 msec
Transfer rates:
outside: 102400 kbytes in 0.730137 sec = 140248 kbytes/sec
middle: 102400 kbytes in 0.773519 sec = 132382 kbytes/sec
inside: 102400 kbytes in 0.289748 sec = 353411 kbytes/sec
diskinfo -t mirror/k1
mirror/k1
512 # sectorsize
8001563221504 # mediasize in bytes (7.3T)
15628053167 # mediasize in sectors
4096 # stripesize
0 # stripeoffset
Yes # TRIM/UNMAP support
Unknown # Rotation rate in RPM
Seek times:
Full stroke: 250 iter in 0.040802 sec = 0.163 msec
Half stroke: 250 iter in 0.022701 sec = 0.091 msec
Quarter stroke: 500 iter in 0.041262 sec = 0.083 msec
Short forward: 400 iter in 0.335174 sec = 0.838 msec
Short backward: 400 iter in 0.354776 sec = 0.887 msec
Seq outer: 2048 iter in 0.446335 sec = 0.218 msec
Seq inner: 2048 iter in 0.417780 sec = 0.204 msec
Transfer rates:
outside: 102400 kbytes in 0.701626 sec = 145947 kbytes/sec
middle: 102400 kbytes in 0.737018 sec = 138938 kbytes/sec
inside: 102400 kbytes in 0.576959 sec = 177482 kbytes/secShould I replace internal (4+4) gstripe with gconcat instead, to get more speed?
Also, does it matter, that gmirror seems to have TRIM support, but gstripe does not? Seems gconcat does not support TRIM either.
Any other tips?