Part 2 Disk subsystem testing
To start this off I wanted to do a simple baseline comparison between installing FOG on a single sata disk using the onboard sata controller, a single sata hdd disk (same) on a raid controller as a JBOD disk, then setup a 4 disk raid 0 on the raid controller. The next steps are to replace the sata hdd with sata sdd drives and repeat the steps as with the hdd.
The the simple disk baseline I’m using the following linux command to create a sequential 1GB file on disk and then to read it back. This process is designed to simulate the single unicast workload. The command used to write the 1GB file is this:
dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
The command to read it back is:
echo 3 | tee /proc/sys/vm/drop_caches && time dd if=/tmp/test1.img of=/dev/null bs=8k
The echo command is intended to disable the read cache so we get a true read back value.
The disks I used are as follows
- (3) Dell Constellation ES 1TB server hard drives [hdd] (what I had in my magic box of extra bits).
- (3) Crucial MX300 275GB SDD
I used 3 because that is what I had of the ssd drives in my no so magic box from amazon.
Test Process:
- Install the test drives into the 790 and installed Centos 7 1611
- No updates were applied, the install image was straight off usb.
- Log in as root to the linux command prompt
- Run the sequential write command 3 times (avg results)
5, Run the sequential read command 3 times (avg results)
- Shutdown and prep for next test.
Test 1: Single Constellation (hdd) attached to on board sata
[root@localhost ~]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 13.9599 s, 76.9 MB/s
[root@localhost ~]#
[root@localhost ~]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 13.9033 s, 77.2 MB/s
[root@localhost ~]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 13.7618 s, 78.0 MB/s
[root@localhost ~]# time dd if=/tmp/test1.img of=/dev/null bs=8k
131072+0 records in
131072+0 records out
1073741824 bytes (1.1 GB) copied, 13.6594 s, 78.6 MB/s
[root@localhost ~]# time dd if=/tmp/test1.img of=/dev/null bs=8k
131072+0 records in
131072+0 records out
1073741824 bytes (1.1 GB) copied, 13.5738 s, 79.1 MB/s
real 0m13.577s
user 0m0.040s
sys 0m0.888s
Average speed write 77MB/s (4.7 GB/m) read 78MB/s
Test 2: Single MX300 (ssd) attached to on board sata
[root@localhost ~]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 2.24173 s, 479 MB/s
[root@localhost ~]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 2.24117 s, 479 MB/s
[root@localhost ~]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 2.24441 s, 478 MB/s
[root@localhost ~]# echo 3 | tee /proc/sys/vm/drop_caches
[root@localhost ~]# time dd if=/tmp/test1.img of=/dev/null bs=8k
131072+0 records in
131072+0 records out
1073741824 bytes (1.1 GB) copied, 2.10576 s, 510 MB/s
real 0m2.109s
user 0m0.018s
sys 0m0.664s
Average speed write 478MB/s and read 510MB/s
Test 3: 3 Constellations (hdd) in software raid-0 configuration to on board sata
[root@localhost ~]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 2.90412 s, 370 MB/s
[root@localhost ~]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 2.78557 s, 385 MB/s
[root@localhost ~]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 2.75433 s, 390 MB/s
[root@localhost ~]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 2.802 s, 383 MB/s
[root@localhost ~]# time dd if=/tmp/test1.img of=/dev/null bs=8k
131072+0 records in
131072+0 records out
1073741824 bytes (1.1 GB) copied, 2.75442 s, 390 MB/s
real 0m2.967s
user 0m0.016s
sys 0m0.461s
Average speed write 380MB/s and read 390MB/s
* since this was a software raid, I feel the runs after the very first one may be tainted because of some buffering in the software raid driver in linux
Test 4: 3 MX300 (ssd) in software raid-0 configuration to on board sata
[root@localhost ~]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 1.4921 s, 720 MB/s
[root@localhost ~]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 1.50214 s, 715 MB/s
[root@localhost ~]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 1.49913 s, 716 MB/s
[root@localhost ~]# echo 3 | tee /proc/sys/vm/drop_caches
[root@localhost ~]# time dd if=/tmp/test1.img of=/dev/null bs=8k
131072+0 records in
131072+0 records out
1073741824 bytes (1.1 GB) copied, 1.33486 s, 804 MB/s
real 0m1.343s
user 0m0.016s
sys 0m0.385s
[root@localhost ~]# echo 3 | tee /proc/sys/vm/drop_caches
[root@localhost ~]# time dd if=/tmp/test1.img of=/dev/null bs=8k
131072+0 records in
131072+0 records out
1073741824 bytes (1.1 GB) copied, 1.31937 s, 814 MB/s
real 0m1.323s
user 0m0.013s
sys 0m0.322s
Average speed write 718MB/s and read 800MB/s
* since this was a software raid, I feel the runs after the very first one may be tainted because of some buffering in the software raid driver in linux
Test 5: Dell PE2950 6i Raid with 6 x WD RE drives (hdd) in Raid-10 configuration. (just a comparison test)
[root@localhost /]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 2.96148 s, 363 MB/s
[root@localhost /]# dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 2.86738 s, 374 MB/s
[root@localhost /]# echo 3 | tee /proc/sys/vm/drop_caches
[root@localhost /]# time dd if=/tmp/test1.img of=/dev/null bs=8k
131072+0 records in
131072+0 records out
1073741824 bytes (1.1 GB) copied, 3.199 s, 336 MB/s
real 0m3.367s
user 0m0.024s
sys 0m0.861s
Average speed write 368MB/s and read 336MB/s
* performance values may be tainted by current workload on the server. The intent of this test was to identify a ball park number with production server vs Dell 790 desktop