ZFS 12.0R - New server - How to check for disk performance

Hello.
I've bought a new server for my shop. Dell T130 with just 1 2TB hdd just for hosting bhyve vms. I installed zfs as root dir. Is it possible that zfs slows a mere file move? I moved an ISO from my home dir to cbsd src iso directory and I took longer than I expected. To make things certain how do I quantify it?

Code:
mfi0: <Fury> port 0x3000-0x30ff mem 0x92c00000-0x92c0ffff,0x92b00000-0x92bfffff at device 0.0 on pci1
mfi0: Using MSI
mfi0: Megaraid SAS driver Ver 4.23
mfi0: FW MaxCmds = 240, limiting to 128
mfi0: MaxCmd = 240, Drv MaxCmd = 128, MaxSgl = 70, state = 0xb73c00f0
mfi0: 321 (600803618s/0x0020/info) - Shutdown command received from host
mfi0: 322 (boot + 6s/0x0020/info) - Firmware initialization started (PCI ID 005f/1000/1f44/1028)
mfi0: 323 (boot + 6s/0x0020/info) - Firmware version 4.290.01-8335
mfi0: 324 (boot + 8s/0x0020/info) - Package version 25.5.5.0005
mfi0: 325 (boot + 8s/0x0020/info) - Board Revision A08
mfi0: 326 (boot + 14s/0x0002/info) - Inserted: PD 00(e0xff/s0)
mfi0: 327 (boot + 14s/0x0002/info) - Inserted: PD 00(e0xff/s0) Info: enclPd=ffff, scsiType=0, portMap=00, sasAddr=4433221106000000,0000000000000000
mfi0: 328 (boot + 26s/0x0020/info) - Controller operating temperature within normal range, full operation restored
pcib2: mfi0: 329 (600803646s/0x0020/info) - Time established as 01/14/19 17:54:06; (26 seconds since power on)
mfi0: 330 (600803750s/0x0020/info) - Host driver is loaded and operational
mfid0 on mfi0
mfid0: 1907200MB (3905945600 sectors) RAID volume (no label) is degraded

Code:
$ zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot  1.81T  2.74G  1.81T        -         -     0%     0%  1.00x  ONLINE  -
$ zfs list
NAME                                           USED  AVAIL  REFER  MOUNTPOINT
zroot                                         2.72G  1.75T    88K  /zroot
zroot/ROOT                                    2.71G  1.75T    88K  none
zroot/ROOT/default                            2.71G  1.75T  1.98G  /
zroot/ROOT/default/bcbsd-ubuntusrv1-dsk1.vhd   711M  1.75T   711M  -
zroot/tmp                                      176K  1.75T   120K  /tmp
zroot/usr                                      456K  1.75T    88K  /usr
zroot/usr/home                                 192K  1.75T   128K  /usr/home
zroot/usr/ports                                 88K  1.75T    88K  /usr/ports
zroot/usr/src                                   88K  1.75T    88K  /usr/src
zroot/var                                      840K  1.75T    88K  /var
zroot/var/audit                                 88K  1.75T    88K  /var/audit
zroot/var/crash                                 88K  1.75T    88K  /var/crash
zroot/var/log                                  280K  1.75T   192K  /var/log
zroot/var/mail                                 152K  1.75T    92K  /var/mail
zroot/var/tmp                                  144K  1.75T    88K  /var/tmp
 
Try looking at the setting of vfs.read_max (careful here...):
> sysctl -a | grep vfs.read_max

I think the default is 64. I've used 256 since I went to 32GB of RAM. I no longer even remember what it is or how the proper setting is derived. Memory pages perhaps. :) To change it, (takes effect immediately):
> sysctl vfs.read_max=<some number>

Then add 'vfs.read_max=<that number>' to /etc/sysctl.conf so it survives reboots. You'll have to play around to determine what's best for your machine. Search on it. hth
 
Thanks for your reply, my fear is that I'm using a raid controller under zfs and that's causing the problems.

Try looking at the setting of vfs.read_max (careful here...):
> sysctl -a | grep vfs.read_max

I think the default is 64. I've used 256 since I went to 32GB of RAM. I no longer even remember what it is or how the proper setting is derived. Memory pages perhaps. :) To change it, (takes effect immediately):
> sysctl vfs.read_max=<some number>

Then add 'vfs.read_max=<that number>' to /etc/sysctl.conf so it survives reboots. You'll have to play around to determine what's best for your machine. Search on it. hth
 
First, it is mostly recommended not to use ZFS on top of hardware RAID; use JBOD. Second, you didn't posted any details about your controller model, hard disk model, how did you partitioned your storage and your RAID volume settings. Besides you should not expect great performance from a single hard drive, while copying large files, when reading and writing on the very same drive; seek times kill performance.

Use gstat(8) to monitor I/O utilization.
 
The quintessential tool to measure and compare disk performance is benchmarks/bonnie++.

And I agree, configure the MegaRAID card for JBOD, don't use the RAID functionality if you're going to use ZFS. Some older MegaRAID cards may not have the JBOD option though.
 
I did a reinstall of the server Dell T130, and changed the raid controller perc 330 to HBA.
Here's the dmesg. I kept reading about raid - zfs and wanted to avoid any future problems. No the disk is shown as mfisyspd0, would it work as a regular ada0 drive with smartmontools? Right now I'm installing a basic freebsd + bhybe (ubuntu + ms sqlserver linux) and I'm planning of buying a couple of more hdd or SSD. I want to be able to have everything in place to add/move everything smoothly.

Code:
---<<BOOT>>---
Copyright (c) 1992-2018 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 12.0-RELEASE-p2 GENERIC amd64
FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1)
VT(efifb): resolution 1024x768
CPU: Intel(R) Xeon(R) CPU E3-1220 v6 @ 3.00GHz (3000.10-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x906e9  Family=0x6  Model=0x9e  Stepping=9
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x121<LAHF,ABM,Prefetch>
  Structured Extended Features=0x29c6fbf<FSGSBASE,TSCADJ,SGX,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,NFPUSG,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PROCTRACE>
  Structured Extended Features3=0x9c000000<IBPB,STIBP,L1DFL,SSBD>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics
real memory  = 17179869184 (16384 MB)
avail memory = 16412999680 (15652 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <DELL   PE_SC3  >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-23 on motherboard
Launching APs: 1 3 2
Timecounter "TSC-low" frequency 1500048354 Hz quality 1000
random: entropy device external interface
netmap: loaded module
[ath_hal] loaded
module_register_init: MOD_LOAD (vesa, 0xffffffff810f9770, 0) error 19
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
kbd0 at kbdmux0
nexus0
efirtc0: <EFI Realtime Clock> on motherboard
efirtc0: registered as a time-of-day clock, resolution 1.000000s
cryptosoft0: <software crypto> on motherboard
acpi0: <DELL PE_SC3> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 24000000 Hz quality 950
Event timer "HPET" frequency 24000000 Hz quality 550
Event timer "HPET1" frequency 24000000 Hz quality 440
Event timer "HPET2" frequency 24000000 Hz quality 440
Event timer "HPET3" frequency 24000000 Hz quality 440
Event timer "HPET4" frequency 24000000 Hz quality 440
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: Warning: Couldn't map I/O.
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
AVAGO MegaRAID SAS FreeBSD mrsas driver version: 06.712.04.00-fbsd
mfi0: <Fury> port 0x3000-0x30ff mem 0x92c00000-0x92c0ffff,0x92b00000-0x92bfffff at device 0.0 on pci1
mfi0: Using MSI
mfi0: Megaraid SAS driver Ver 4.23
mfi0: FW MaxCmds = 240, limiting to 128
mfi0: MaxCmd = 240, Drv MaxCmd = 128, MaxSgl = 70, state = 0xb73c00f0
mfi0: 398 (600866786s/0x0020/info) - Shutdown command received from host
mfi0: 399 (boot + 6s/0x0020/info) - Firmware initialization started (PCI ID 005f/1000/1f44/1028)
mfi0: 400 (boot + 6s/0x0020/info) - Firmware version 4.290.01-8335
mfi0: 401 (boot + 8s/0x0020/info) - Package version 25.5.5.0005
mfi0: 402 (boot + 8s/0x0020/info) - Board Revision A08
mfi0: 403 (boot + 14s/0x0002/info) - Inserted: PD 00(e0xff/s0)
pcib2: <ACPI PCI-PCI bridge> at device 1.2 on pci0
pci2: <ACPI PCI bus> on pcib2
xhci0: <Intel Sunrise Point USB 3.0 controller> mem 0x92d00000-0x92d0ffff at device 20.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
pci0: <simple comms> at device 22.0 (no driver attached)
pci0: <simple comms> at device 22.1 (no driver attached)
ahci0: <Intel Sunrise Point AHCI SATA controller> port 0x4040-0x4047,0x4048-0x404b,0x4020-0x403f mem 0x92d14000-0x92d15fff,0x92d17000-0x92d170ff,0x92d16000-0x92d167ff at device 23.0 on pci0
ahci0: AHCI v1.31 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
pcib3: <ACPI PCI-PCI bridge> at device 29.0 on pci0
pci3: <ACPI PCI bus> on pcib3
bge0: <Broadcom NetXtreme Gigabit Ethernet, ASIC rev. 0x5720000> mem 0x92a30000-0x92a3ffff,0x92a40000-0x92a4ffff,0x92a50000-0x92a5ffff at device 0.0 on pci3
bge0: APE FW version: NCSI v1.4.22.0
bge0: CHIP ID 0x05720000; ASIC REV 0x5720; CHIP REV 0x57200; PCI-E
miibus0: <MII bus> on bge0
brgphy0: <BCM5720C 1000BASE-T media interface> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge0: Using defaults for TSO: 65518/35/2048
bge0: Ethernet address: 50:9a:4c:94:33:5a
bge1: <Broadcom NetXtreme Gigabit Ethernet, ASIC rev. 0x5720000> mem 0x92a00000-0x92a0ffff,0x92a10000-0x92a1ffff,0x92a20000-0x92a2ffff at device 0.1 on pci3
bge1: APE FW version: NCSI v1.4.22.0
bge1: CHIP ID 0x05720000; ASIC REV 0x5720; CHIP REV 0x57200; PCI-E
miibus1: <MII bus> on bge1
brgphy1: <BCM5720C 1000BASE-T media interface> PHY 2 on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge1: Using defaults for TSO: 65518/35/2048
bge1: Ethernet address: 50:9a:4c:94:33:5b
pcib4: <ACPI PCI-PCI bridge> at device 29.2 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 0.0 on pci4
pci5: <ACPI PCI bus> on pcib5
pcib6: <PCI-PCI bridge> at device 0.0 on pci5
pci6: <PCI bus> on pcib6
pcib7: <PCI-PCI bridge> at device 0.0 on pci6
pci7: <PCI bus> on pcib7
vgapci0: <VGA-compatible display> mem 0x91000000-0x91ffffff,0x92800000-0x92803fff,0x92000000-0x927fffff at device 0.0 on pci7
vgapci0: Boot video device
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
pci0: <memory> at device 31.2 (no driver attached)
acpi_button0: <Sleep Button> on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
orm0: <ISA Option ROM> at iomem 0xed800-0xf17ff pnpid ORM0000 on isa0
vga0: <Generic ISA VGA> at port 0x3d0-0x3db iomem 0xb8000-0xbffff pnpid PNP0900 on isa0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 200200002100
device_attach: est0 attach returned 6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 1ffe00002100
device_attach: est1 attach returned 6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 200200002100
device_attach: est2 attach returned 6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 1ffd00002100
device_attach: est3 attach returned 6
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
mfisyspd0 on mfi0
mfisyspd0: 1907729MB (3907029168 sectors) SYSPD volume (deviceid: 0)
mfisyspd0:  SYSPD volume attached
mfi0: 404 (boot + 14s/0x0002/info) - Inserted: PD 00(e0xff/s0) Info: enclPd=ffff, scsiType=0, portMap=00, sasAddr=4433221106000000,0ugen0.1: <0x8086 XHCI root HUB> at usbus0
000000000000000
mfi0: 405 (boot + 26s/0x0020/info) - Controller operating temperature within normal range, full operation restored
mfi0: 406 (600866815s/0x0020/info) - Time established as 01/15/19 11:26:55; (26 seconds since power on)
mfi0: 407 (600866857s/0x0020/info) - Host driver is loaded and operational
ses0 at ahciem0 bus 0 scbus6 target 0 lun 0
ses0: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
cd0 at ahcich4 bus 0 scbus4 target 0 lun 0
cd0: <PLDS DVD+-RW DU-8A5LH 6D5M> Removable CD-ROM SCSI device
cd0: Serial Number 2WXYCPLC0086451X0A00
cd0: 150.000MB/s transfers (SATA 1.x, UDMA6, ATAPI 12bytes, PIO 8192bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed
Trying to mount root from zfs:zroot/ROOT/default []...
Root mount waiting for: usbus0
Root mount waiting for: usbus0
uhub0: 26 ports with 26 removable, self powered
ugen0.2: <no manufacturer Gadget USB HUB> at usbus0
uhub1 on uhub0
uhub1: <no manufacturer Gadget USB HUB, class 9/0, rev 2.00/0.00, addr 1> on usbus0
Root mount waiting for: usbus0
uhub1: 6 ports with 6 removable, self powered
Root mount waiting for: usbus0
ugen0.3: <LITEON Technology USB Keyboard> at usbus0
ukbd0 on uhub0
ukbd0: <EP1 Interrupt> on usbus0
kbd1 at ukbd0
lo0: link state changed to UP
bge0: link state changed to DOWN
bge0: link state changed to UP
bridge1: Ethernet address: 02:1c:1d:b0:da:01
bge0: promiscuous mode enabled
bridge1: link state changed to UP
tap1: Ethernet address: 00:bd:be:9b:ff:01
tap1: promiscuous mode enabled
tap2: Ethernet address: 00:bd:52:4e:05:02
tap2: promiscuous mode enabled
tap2: link state changed to UP
nd6_dad_timer: cancel DAD on tap2 because of ND6_IFF_IFDISABLED.
tap2: link state changed to DOWN
tap2: link state changed to UP
tap2: link state changed to DOWN
tap2: link state changed to UP

Code:
root@freebsd:/usr/jails/src # zfs list
NAME                                           USED  AVAIL  REFER  MOUNTPOINT
zroot                                         5.02G  1.74T    88K  /zroot
zroot/ROOT                                    4.99G  1.74T    88K  none
zroot/ROOT/default                            4.99G  1.74T  1.87G  /
zroot/ROOT/default/bcbsd-ubuntusrv1-dsk1.vhd  3.12G  1.74T  3.12G  -
zroot/tmp                                      172K  1.74T   116K  /tmp
zroot/usr                                     24.4M  1.74T    88K  /usr
zroot/usr/home                                24.1M  1.74T  24.1M  /usr/home
zroot/usr/ports                                 88K  1.74T    88K  /usr/ports
zroot/usr/src                                   88K  1.74T    88K  /usr/src
zroot/var                                      800K  1.74T    88K  /var
zroot/var/audit                                 88K  1.74T    88K  /var/audit
zroot/var/crash                                 88K  1.74T    88K  /var/crash
zroot/var/log                                  244K  1.74T   148K  /var/log
zroot/var/mail                                 148K  1.74T    92K  /var/mail
zroot/var/tmp                                  144K  1.74T    88K  /var/tmp

Code:
root@freebsd:/usr/jails/src # zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot  1.80T  5.02G  1.79T        -         -     0%     0%  1.00x  ONLINE  -
 
Code:
root@freebsd:/usr/jails/src # mfiutil show drives
mfi0 Physical Drives:
 0 ( 1863G) JBOD <HUS722T2TALA600 MU03 serial=WMC6N0M1FFPK> SATA S0

What's the benefit from using mrsas from the other? Also how do I switch?
Having the controller changed to HBA is faster. I've just restored a MS SQL Server Backup inside a VM and something that took around 6 minutes before.

Code:
RESTORE DATABASE successfully processed 594994 pages in 24.092 seconds (192.943 MB/sec).
 
What's the benefit from using mrsas from the other?
Improved driver mostly. I've had to run and maintain various LSI cards with the mfi(4) driver and it's certainly not a bad driver. Over the past few years it has proven to be quite stable and performance has always been as expected. But the mrsas(4) driver integrates a bit better with some of FreeBSD's own systems and drivers and that's reason enough for me :)

I'm trying now with mrsas, changing /boot/device.hints
hw.mfi.mrsas_enable="1"
If that doesn't work add it to /boot/loader.conf instead. I believe this to be an error as last time I tried using device.hints(5) it didn't work. Setting it in loader.conf(5) did work. Note that your disk may change name, from mfid0 to da0. But ZFS doesn't care about that. Because of that you should be able to switch back and forth between mfi(4) and mrsas(4).

It would be interesting to see a real performance comparison, on the exact same hardware, between the two drivers. I've never actually done that. I don't expect any major performance differences though.
 
Fortunately it worked from device.hints. I was worried that ZFS wouldn't interpret the change or something but as you mentioned it doesn't care about those things. Let's say I add two SSD I could create a raidz mixing those technologies? But it's not the purpose of this post. I would mark it solved and the resolution of the poor performance what that the PERC330 RAID was enable on the BIOS once changed to HBA everything ran fast.

Improved driver mostly. I've had to run and maintain various LSI cards with the mfi(4) driver and it's certainly not a bad driver. Over the past few years it has proven to be quite stable and performance has always been as expected. But the mrsas(4) driver integrates a bit better with some of FreeBSD's own systems and drivers and that's reason enough for me :)


If that doesn't work add it to /boot/loader.conf instead. I believe this to be an error as last time I tried using device.hints(5) it didn't work. Setting it in loader.conf(5) did work. Note that your disk may change name, from mfid0 to da0. But ZFS doesn't care about that. Because of that you should be able to switch back and forth between mfi(4) and mrsas(4).

It would be interesting to see a real performance comparison, on the exact same hardware, between the two drivers. I've never actually done that. I don't expect any major performance differences though.
 
Let's say I add two SSD I could create a raidz mixing those technologies?
Possible but not recommended. The performance of the whole pool would be dictated by the slowest disks, negating the advantages of the SSDs. Depending on your load they're better used as L2ARC and ZIL. Whether or not you actually benefit from L2ARC or ZIL is going to depend on your storage use. Alternatively you could create two pools, one SSD based and one based on spinning rust.
 
I installed another hdd + 1 SSD zfs mirror with the help from #freeebsd (freenode) I did some tests and the results I got before and I think I should be getting more read/write speeds. All disks are 6Gbps capable, also the SSD.
Both disks and the SSD are detected as 150MB/s transfer capable (sorry I'm not at work and I can't get the dmesg) Those devices should be doing 600MB/s don't they? Should I try enabling ahci_load="YES" in loader.conf ?
 
should be doing 600MB/s shouldn't they?
Well the SATA3 interface shows 600MB/s but even SSD only provide ~540MB/s with overhead.
Code:
dmesg | grep ada
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <SanDisk SSD U110 64GB U221000> ACS-2 ATA SATA 3.x device
ada0: Serial Number 140861403937
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes)
ada0: Command Queueing enabled
ada0: 60103MB (123091920 512 byte sectors)
diskinfo -t /dev/ada0
Transfer rates:
outside: 102400 kbytes in 0.538727 sec = 190078 kbytes/sec
middle: 102400 kbytes in 0.263119 sec = 389178 kbytes/sec
inside: 102400 kbytes in 0.246838 sec = 414847 kbytes/sec
So I am only getting ~400MB/s on this drive.
Max per SATA3 interface I have seen is ~540MB/s.

192 MB/sec sequential read or write is pretty close to the max performance you can expect from a single hard drive.
This is what I have found as well.
 
I booted with a USB pen drive to Debian and did a test:

Code:
dd if=/dev/sda of=/dev/null bs=4096 count=5000000
101 seconds

dd if=/dev/sdc of=/dev/null bs=4096 count=5000000
40 seconds

In linux I get higher read speeds.

Description of the connected devices
Code:
camcontrol devlist
<ATA HUS722T2TALA600 MU03>         at scbus1 target 0 lun 0 (pass0,da0)
<ATA ST2000NM0018-2F3 EA04>        at scbus1 target 1 lun 0 (pass1,da1)
<ATA SanDisk SDSSDA24 30RL>        at scbus1 target 2 lun 0 (pass2,da2)
<PLDS DVD+-RW DU-8A5LH 6D5M>       at scbus6 target 0 lun 0 (cd0,pass3)
<AHCI SGPIO Enclosure 1.00 0001>   at scbus8 target 0 lun 0 (pass4,ses0)

camcontrol identify /dev/da0
pass0: <HUS722T2TALA600 RADEMU03> ACS-3 ATA SATA 3.x device
pass0: 150.000MB/s transfers, Command Queueing Enabled

protocol              ATA/ATAPI-10 SATA 3.x
device model          HUS722T2TALA600
firmware revision     RADEMU03
serial number         WMC6N0M1FFPK
WWN                   50014ee0048da0c8
cylinders             16383
heads                 16
sectors/track         63
sector size           logical 512, physical 512, offset 0
LBA supported         268435455 sectors
LBA48 supported       3907029168 sectors
PIO supported         PIO4
DMA supported         WDMA2 UDMA6
media RPM             7200
Zoned-Device Commands no

Feature                      Support  Enabled   Value           Vendor
read ahead                     yes      yes
write cache                    yes      yes
flush cache                    yes      yes
overlap                        no
Tagged Command Queuing (TCQ)   no       no
Native Command Queuing (NCQ)   yes              32 tags
NCQ Queue Management           no
NCQ Streaming                  no
Receive & Send FPDMA Queued    yes
SMART                          yes      yes
microcode download             yes      yes
security                       no       no
power management               yes      yes
advanced power management      yes      no      0/0x00
automatic acoustic management  no       no
media status notification      no       no
power-up in Standby            yes      no
write-read-verify              no       no
unload                         yes      yes
general purpose logging        yes      yes
free-fall                      no       no
Data Set Management (DSM/TRIM) no
Host Protected Area (HPA)      no

 camcontrol identify /dev/da1
pass1: <ST2000NM0018-2F3130 EA04> ACS-3 ATA SATA 3.x device
pass1: 150.000MB/s transfers, Command Queueing Enabled

protocol              ATA/ATAPI-10 SATA 3.x
device model          ST2000NM0018-2F3130
firmware revision     EA04
serial number         ZDS0LJKY
WWN                   5000c500b2a1dd3c
cylinders             16383
heads                 16
sectors/track         63
sector size           logical 512, physical 512, offset 0
LBA supported         268435455 sectors
LBA48 supported       3907029168 sectors
PIO supported         PIO4
DMA supported         WDMA2 UDMA6
media RPM             7200
Zoned-Device Commands no

Feature                      Support  Enabled   Value           Vendor
read ahead                     yes      yes
write cache                    yes      yes
flush cache                    yes      yes
overlap                        no
Tagged Command Queuing (TCQ)   no       no
Native Command Queuing (NCQ)   yes              32 tags
NCQ Queue Management           no
NCQ Streaming                  no
Receive & Send FPDMA Queued    no
SMART                          yes      yes
microcode download             yes      yes
security                       yes      no
power management               yes      yes
advanced power management      no       no
automatic acoustic management  no       no
media status notification      no       no
power-up in Standby            no       no
write-read-verify              yes      no      0/0x0
unload                         yes      yes
general purpose logging        yes      yes
free-fall                      no       no
Data Set Management (DSM/TRIM) no
Host Protected Area (HPA)      no

 camcontrol identify /dev/da2
pass2: <SanDisk SDSSDA240G Z33130RL> ACS-2 ATA SATA 3.x device
pass2: 150.000MB/s transfers, Command Queueing Enabled

protocol              ATA/ATAPI-9 SATA 3.x
device model          SanDisk SDSSDA240G
firmware revision     Z33130RL
serial number         172757804261
WWN                   5001b448b4046ead
cylinders             16383
heads                 16
sectors/track         63
sector size           logical 512, physical 512, offset 0
LBA supported         268435455 sectors
LBA48 supported       468862128 sectors
PIO supported         PIO4
DMA supported         WDMA2 UDMA6
media RPM             non-rotating
Zoned-Device Commands no

Feature                      Support  Enabled   Value           Vendor
read ahead                     yes      yes
write cache                    yes      yes
flush cache                    yes      yes
overlap                        no
Tagged Command Queuing (TCQ)   no       no
Native Command Queuing (NCQ)   yes              32 tags
NCQ Queue Management           no
NCQ Streaming                  no
Receive & Send FPDMA Queued    no
SMART                          yes      yes
microcode download             yes      yes
security                       yes      no
power management               yes      yes
advanced power management      yes      yes     128/0x80
automatic acoustic management  no       no
media status notification      no       no
power-up in Standby            no       no
write-read-verify              no       no
unload                         no       no
general purpose logging        yes      yes
free-fall                      no       no
Data Set Management (DSM/TRIM) yes
DSM - max 512byte blocks       yes              8
DSM - deterministic read       yes              zeroed
Host Protected Area (HPA)      yes      no      468862128/453002416
HPA - Security                 no

last dmesg

Code:
---<<BOOT>>---
Copyright (c) 1992-2018 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 12.0-RELEASE-p2 GENERIC amd64
FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1)
VT(vga): resolution 640x480
CPU: Intel(R) Xeon(R) CPU E3-1220 v6 @ 3.00GHz (3000.11-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x906e9  Family=0x6  Model=0x9e  Stepping=9
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x121<LAHF,ABM,Prefetch>
  Structured Extended Features=0x29c6fbf<FSGSBASE,TSCADJ,SGX,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,NFPUSG,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PROCTRACE>
  Structured Extended Features3=0x9c000000<IBPB,STIBP,L1DFL,SSBD>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics
real memory  = 17179869184 (16384 MB)
avail memory = 16413990912 (15653 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <DELL   PE_SC3  >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-23 on motherboard
Launching APs: 2 1 3
Timecounter "TSC-low" frequency 1500055916 Hz quality 1000
random: entropy device external interface
netmap: loaded module
[ath_hal] loaded
module_register_init: MOD_LOAD (vesa, 0xffffffff810f9770, 0) error 19
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
kbd1 at kbdmux0
nexus0
vtvga0: <VT VGA driver> on motherboard
cryptosoft0: <software crypto> on motherboard
acpi0: <DELL PE_SC3> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 24000000 Hz quality 950
Event timer "HPET" frequency 24000000 Hz quality 550
Event timer "HPET1" frequency 24000000 Hz quality 440
Event timer "HPET2" frequency 24000000 Hz quality 440
Event timer "HPET3" frequency 24000000 Hz quality 440
Event timer "HPET4" frequency 24000000 Hz quality 440
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: Warning: Couldn't map I/O.
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
AVAGO MegaRAID SAS FreeBSD mrsas driver version: 06.712.04.00-fbsd
mrsas0: <AVAGO Fury SAS Controller> port 0x3000-0x30ff mem 0x92c00000-0x92c0ffff,0x92b00000-0x92bfffff irq 16 at device 0.0 on pci1
mrsas0: Using MSI-X with 4 number of vectors
mrsas0: FW supports <96> MSIX vector,Online CPU 4 Current MSIX <4>
mrsas0: FW supports: UnevenSpanSupport=1

mrsas0: MSI-x interrupts setup success
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 1.2 on pci0
pci2: <ACPI PCI bus> on pcib2
xhci0: <Intel Sunrise Point USB 3.0 controller> mem 0x92d00000-0x92d0ffff irq 16 at device 20.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
usbus0: waiting for BIOS to give up control
xhci_interrupt: host controller halted
xhci_interrupt: host controller halted
xhci_interrupt: host controller halted
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
pci0: <simple comms> at device 22.0 (no driver attached)
pci0: <simple comms> at device 22.1 (no driver attached)
ahci0: <Intel Sunrise Point AHCI SATA controller> port 0x4040-0x4047,0x4048-0x404b,0x4020-0x403f mem 0x92d14000-0x92d15fff,0x92d17000-0x92d170ff,0x92d16000-0x92d167ff irq 16 at device 23.0 on pci0
ahci0: AHCI v1.31 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
pcib3: <ACPI PCI-PCI bridge> irq 16 at device 29.0 on pci0
pci3: <ACPI PCI bus> on pcib3
bge0: <Broadcom NetXtreme Gigabit Ethernet, ASIC rev. 0x5720000> mem 0x92a30000-0x92a3ffff,0x92a40000-0x92a4ffff,0x92a50000-0x92a5ffff irq 16 at device 0.0 on pci3
bge0: APE FW version: NCSI v1.4.22.0
bge0: CHIP ID 0x05720000; ASIC REV 0x5720; CHIP REV 0x57200; PCI-E
miibus0: <MII bus> on bge0
brgphy0: <BCM5720C 1000BASE-T media interface> PHY 1 on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge0: Using defaults for TSO: 65518/35/2048
bge0: Ethernet address: 50:9a:4c:94:33:5a
bge1: <Broadcom NetXtreme Gigabit Ethernet, ASIC rev. 0x5720000> mem 0x92a00000-0x92a0ffff,0x92a10000-0x92a1ffff,0x92a20000-0x92a2ffff irq 17 at device 0.1 on pci3
bge1: APE FW version: NCSI v1.4.22.0
bge1: CHIP ID 0x05720000; ASIC REV 0x5720; CHIP REV 0x57200; PCI-E
miibus1: <MII bus> on bge1
brgphy1: <BCM5720C 1000BASE-T media interface> PHY 2 on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
bge1: Using defaults for TSO: 65518/35/2048
bge1: Ethernet address: 50:9a:4c:94:33:5b
pcib4: <ACPI PCI-PCI bridge> irq 18 at device 29.2 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 0.0 on pci4
pci5: <ACPI PCI bus> on pcib5
pcib6: <PCI-PCI bridge> at device 0.0 on pci5
pci6: <PCI bus> on pcib6
pcib7: <PCI-PCI bridge> at device 0.0 on pci6
pci7: <PCI bus> on pcib7
vgapci0: <VGA-compatible display> mem 0x91000000-0x91ffffff,0x92800000-0x92803fff,0x92000000-0x927fffff irq 18 at device 0.0 on pci7
vgapci0: Boot video device
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
pci0: <memory> at device 31.2 (no driver attached)
acpi_button0: <Sleep Button> on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xcb7ff,0xed800-0xf17ff pnpid ORM0000 on isa0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 201f00002100
device_attach: est0 attach returned 6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 201f00002100
device_attach: est1 attach returned 6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 202400002100
device_attach: est2 attach returned 6
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr 202400002100
device_attach: est3 attach returned 6
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
mrsas0: Disestablish mrsas intr hook
ugen0.1: <0x8086 XHCI root HUB> at usbus0
uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
ses0 at ahciem0 bus 0 scbus8 target 0 lun 0
ses0: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
cd0 at ahcich4 bus 0 scbus6 target 0 lun 0
cd0: <PLDS DVD+-RW DU-8A5LH 6D5M> Removable CD-ROM SCSI device
cd0: Serial Number 2WXYCPLC0086451X0A00
cd0: 150.000MB/s transfers (SATA 1.x, UDMA6, ATAPI 12bytes, PIO 8192bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed
da0 at mrsas0 bus 1 scbus1 target 0 lun 0
da0: <ATA HUS722T2TALA600 MU03> Fixed Direct Access SPC-4 SCSI device
da0: Serial Number WMC6N0M1FFPK
da0: 150.000MB/s transfers
da0: 1907729MB (3907029168 512 byte sectors)
da2 at mrsas0 bus 1 scbus1 target 2 lun 0
da2: <ATA SanDisk SDSSDA24 30RL> Fixed Direct Access SPC-4 SCSI device
da2: Serial Number 172757804261
da2: 150.000MB/s transfers
da2: 228936MB (468862128 512 byte sectors)
da1 at mrsas0 bus 1 scbus1 target 1 lun 0
da1: <ATA ST2000NM0018-2F3 EA04> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number ZDS0LJKY
da1: 150.000MB/s transfers
da1: 1907729MB (3907029168 512 byte sectors)
Trying to mount root from zfs:zroot/ROOT/default []...
uhub0: 26 ports with 26 removable, self powered
ugen0.2: <no manufacturer Gadget USB HUB> at usbus0
uhub1 on uhub0
uhub1: <no manufacturer Gadget USB HUB, class 9/0, rev 2.00/0.00, addr 1> on usbus0
uhub1: 6 ports with 6 removable, self powered
ugen0.3: <LITEON Technology USB Keyboard> at usbus0
ukbd0 on uhub0
ukbd0: <EP1 Interrupt> on usbus0
kbd2 at ukbd0
ugen0.4: <Kingston DataTraveler 2.0> at usbus0
umass0 on uhub0
umass0: <Kingston DataTraveler 2.0, class 0/0, rev 2.00/1.10, addr 3> on usbus0
umass0:  SCSI over Bulk-Only; quirks = 0xc100
umass0:9:0: Attached to scbus9
da3 at umass-sim0 bus 0 scbus9 target 0 lun 0
da3: <Kingston DataTraveler 2.0 PMAP> Removable Direct Access SCSI device
da3: Serial Number 5B75139A229B
da3: 40.000MB/s transfers
da3: 3936MB (8060928 512 byte sectors)
da3: quirks=0x3<NO_SYNC_CACHE,NO_6_BYTE>
lo0: link state changed to UP
bge0: link state changed to DOWN
bge0: link state changed to UP
bridge1: Ethernet address: 02:1c:1d:b0:da:01
bge0: promiscuous mode enabled
bridge1: link state changed to UP
tap1: Ethernet address: 00:bd:09:2f:f7:01
tap1: promiscuous mode enabled
tap2: Ethernet address: 00:bd:e5:2f:f7:02
tap2: promiscuous mode enabled
tap2: link state changed to UP
nd6_dad_timer: cancel DAD on tap2 because of ND6_IFF_IFDISABLED.
tap3: Ethernet address: 00:bd:2e:64:f7:03
tap3: promiscuous mode enabled
tap3: link state changed to UP
nd6_dad_timer: cancel DAD on tap3 because of ND6_IFF_IFDISABLED.
ugen0.4: <Kingston DataTraveler 2.0> at usbus0 (disconnected)
umass0: at uhub0, port 11, addr 3 (disconnected)
da3 at umass-sim0 bus 0 scbus9 target 0 lun 0
da3: <Kingston DataTraveler 2.0 PMAP>  s/n 5B75139A229B detached
(da3:umass-sim0:0:0:0): Periph destroyed
umass0: detached

The disks are handled through mrsas, is there any way to configure/set speeds via the driver?
 
Some tests with diskinfo:

Code:
diskinfo -t /dev/da2
/dev/da2
        512             # sectorsize
        240057409536    # mediasize in bytes (224G)
        468862128       # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        29185           # Cylinders according to firmware.
        255             # Heads according to firmware.
        63              # Sectors according to firmware.
        ATA SanDisk SDSSDA24    # Disk descr.
        172757804261            # Disk ident.
        Yes             # TRIM/UNMAP support
        0               # Rotation rate in RPM
        Not_Zoned       # Zone Mode

Seek times:
        Full stroke:      250 iter in   0.006808 sec =    0.027 msec
        Half stroke:      250 iter in   0.005397 sec =    0.022 msec
        Quarter stroke:   500 iter in   0.011091 sec =    0.022 msec
        Short forward:    400 iter in   0.008828 sec =    0.022 msec
        Short backward:   400 iter in   0.008945 sec =    0.022 msec
        Seq outer:       2048 iter in   0.036558 sec =    0.018 msec
        Seq inner:       2048 iter in   0.036142 sec =    0.018 msec

Transfer rates:
        outside:       102400 kbytes in   0.198618 sec =   515563 kbytes/sec
        middle:        102400 kbytes in   0.197992 sec =   517193 kbytes/sec
        inside:        102400 kbytes in   0.201707 sec =   507667 kbytes/sec

[root@freebsd ~]# diskinfo -t /dev/da0
/dev/da0
        512             # sectorsize
        2000398934016   # mediasize in bytes (1.8T)
        3907029168      # mediasize in sectors
        0               # stripesize
        0               # stripeoffset
        243201          # Cylinders according to firmware.
        255             # Heads according to firmware.
        63              # Sectors according to firmware.
        ATA HUS722T2TALA600     # Disk descr.
        WMC6N0M1FFPK    # Disk ident.
        No              # TRIM/UNMAP support
        7200            # Rotation rate in RPM
        Not_Zoned       # Zone Mode

Seek times:
        Full stroke:      250 iter in   5.803383 sec =   23.214 msec
        Half stroke:      250 iter in   4.354528 sec =   17.418 msec
        Quarter stroke:   500 iter in   6.846779 sec =   13.694 msec
        Short forward:    400 iter in   2.538103 sec =    6.345 msec
        Short backward:   400 iter in   2.174141 sec =    5.435 msec
        Seq outer:       2048 iter in   0.102694 sec =    0.050 msec
        Seq inner:       2048 iter in   0.041046 sec =    0.020 msec

Transfer rates:
        outside:       102400 kbytes in   0.508258 sec =   201472 kbytes/sec
        middle:        102400 kbytes in   0.593155 sec =   172636 kbytes/sec
        inside:        102400 kbytes in   1.149544 sec =    89079 kbytes/sec

According to diskinfo I'm getting 517193 kbytes/sec around 500MB/s but with dd other results?
 
I booted with a USB pen drive to Debian and did a test:

Code:
dd if=/dev/sda of=/dev/null bs=4096 count=5000000
101 seconds

dd if=/dev/sdc of=/dev/null bs=4096 count=5000000
40 seconds

In linux I get higher read speeds.
I would say the numbers are the same. 101 seconds for 19 GB is about 193 MB/sec. 40 seconds for 19 GB is about 488 MB/sec.

You'll have to use NVMe to get single storage device (meaning not a RAID volume) speeds above 500 MB/sec.

You can also try diskinfo -i /dev/da0.
 
I would say the numbers are the same. 101 seconds for 19 GB is about 193 MB/sec. 40 seconds for 19 GB is about 488 MB/sec.

You'll have to use NVMe to get single storage device (meaning not a RAID volume) speeds above 500 MB/sec.

You can also try diskinfo -i /dev/da0.

If I do a dd with the SSD disk I get slower speed than disks however if I increase the blocksize to 1M or more the speed increases substantially.
 
Back
Top