正文
开启raid卡缓存 Write back 对IO性能的影响
背景
公司买了一台服务器. 想进行一下升级
但是因为管理员担心数据丢失, 使用了write through + (raid6 + hotspare)
的终极保护模式.
但是发现磁盘的IO比较差, 测试结果也不是非常好.
基于此进行了一下raid卡的设置, 希望能够提高一下性能.
调优背景
计划打开raid卡缓存前后进行一下简要的性能测试：
工具为： fio
还有数据库测试软件： sysbench
对比结果
| IO块大小 |
是否开启RAID缓存 |
顺序写入 |
顺序读取 |
随机写入 |
随机读取 |
| 128KB |
是 |
10100 |
5098 |
910 |
266 |
| 128KB |
否 |
272 |
11500 |
117 |
1103 |
| 16KB |
是 |
33200 |
28300 |
1902 |
363 |
| 16KB |
否 |
591 |
22900 |
137 |
959 |
| 8KB |
是 |
36500 |
33700 |
1863 |
390 |
| 8KB |
否 |
837 |
37800 |
173 |
946 |
| 1KB |
是 |
39500 |
36300 |
837 |
358 |
| 1KB |
否 |
700 |
33900 |
115 |
884 |
数据库部分的差异
| 是否开启超线程 |
事务效率(每秒) |
查询效率(每秒) |
最小响应时间(ms) |
平均响应时间 |
99%的响应时间 |
| 直写 32核心/32线程 |
244.78 |
5011.98 |
14.43 |
40.82 |
97.55 |
| 写回 32核心/32线程 |
497.56 |
10012.29 |
14.19 |
20.09 |
27.17 |
图形展示
部分原始数据
write128k
write: IOPS=10.1k, BW=1258MiB/s (1319MB/s)(4096MiB/3257msec)
read128k
read: IOPS=5098, BW=637MiB/s (668MB/s)(4096MiB/6427msec)
randwrite128k
write: IOPS=910, BW=114MiB/s (119MB/s)(3420MiB/30065msec)
randread128k
read: IOPS=266, BW=33.4MiB/s (34.0MB/s)(1001MiB/30006msec)
write16k
write: IOPS=33.2k, BW=519MiB/s (544MB/s)(4096MiB/7896msec)
read16k
read: IOPS=28.3k, BW=442MiB/s (464MB/s)(4096MiB/9265msec)
randwrite16k
write: IOPS=1902, BW=29.7MiB/s (31.2MB/s)(893MiB/30026msec)
randread16k
read: IOPS=363, BW=5814KiB/s (5953kB/s)(170MiB/30003msec)
write8k
write: IOPS=36.5k, BW=285MiB/s (299MB/s)(4096MiB/14369msec)
read8k
read: IOPS=33.7k, BW=264MiB/s (276MB/s)(4096MiB/15542msec)
randwrite8k
write: IOPS=1863, BW=14.6MiB/s (15.3MB/s)(437MiB/30001msec)
randread8k
read: IOPS=390, BW=3128KiB/s (3203kB/s)(91.7MiB/30006msec)
write1k
write: IOPS=39.5k, BW=38.5MiB/s (40.4MB/s)(1156MiB/30001msec)
read1k
read: IOPS=36.3k, BW=35.5MiB/s (37.2MB/s)(1064MiB/30001msec)
randwrite1k
write: IOPS=837, BW=838KiB/s (858kB/s)(24.5MiB/30013msec)
randread1k
read: IOPS=358, BW=358KiB/s (367kB/s)(10.5MiB/30002msec)
write128k
write: IOPS=272, BW=34.1MiB/s (35.8MB/s)(1023MiB/30002msec)
read128k
read: IOPS=11.5k, BW=1442MiB/s (1512MB/s)(4096MiB/2841msec)
randwrite128k
write: IOPS=117, BW=14.7MiB/s (15.4MB/s)(441MiB/30008msec)
randread128k
read: IOPS=1103, BW=138MiB/s (145MB/s)(4096MiB/29697msec)
write16k
write: IOPS=591, BW=9456KiB/s (9683kB/s)(277MiB/30005msec)
read16k
read: IOPS=22.9k, BW=358MiB/s (375MB/s)(4096MiB/11442msec)
randwrite16k
write: IOPS=137, BW=2202KiB/s (2255kB/s)(64.5MiB/30013msec)
randread16k
read: IOPS=959, BW=14.0MiB/s (15.7MB/s)(450MiB/30011msec)
write8k
write: IOPS=837, BW=6700KiB/s (6861kB/s)(196MiB/30001msec)
read8k
read: IOPS=37.8k, BW=295MiB/s (310MB/s)(4096MiB/13862msec)
randwrite8k
write: IOPS=143, BW=1144KiB/s (1172kB/s)(33.5MiB/30001msec)
randread8k
read: IOPS=946, BW=7569KiB/s (7750kB/s)(222MiB/30001msec)
write1k
write: IOPS=700, BW=700KiB/s (717kB/s)(20.5MiB/30003msec)
read1k
read: IOPS=33.9k, BW=33.1MiB/s (34.8MB/s)(995MiB/30001msec)
randwrite1k
write: IOPS=115, BW=116KiB/s (118kB/s)(3471KiB/30005msec)
randread1k
read: IOPS=884, BW=885KiB/s (906kB/s)(25.9MiB/30005msec)
sysbench的结果
[ 10s ] thds: 10 tps: 493.60 qps: 9952.25 (r/w/o: 6974.03/1975.31/1002.90) lat (ms,99%): 34.33 err/s: 3.70 reconn/s: 0.00
[ 20s ] thds: 10 tps: 451.21 qps: 9067.76 (r/w/o: 6349.91/1804.43/913.42) lat (ms,99%): 26.68 err/s: 2.80 reconn/s: 0.00
[ 30s ] thds: 10 tps: 547.70 qps: 11024.03 (r/w/o: 7723.12/2189.41/1111.50) lat (ms,99%): 25.28 err/s: 3.70 reconn/s: 0.00
SQL statistics:
queries performed:
read: 210532
write: 59728
other: 30291
total: 300551
transactions: 14936 (497.56 per sec.)
queries: 300551 (10012.29 per sec.)
ignored errors: 102 (3.40 per sec.)
reconnects: 0 (0.00 per sec.)
General statistics:
total time: 30.0166s
total number of events: 14936
Latency (ms):
min: 14.19
avg: 20.09
max: 1041.03
99th percentile: 27.17
sum: 300053.39
Threads fairness:
events (avg/stddev): 1493.6000/82.27
execution time (avg/stddev): 30.0053/0.01
[ 10s ] thds: 10 tps: 243.35 qps: 4991.08 (r/w/o: 3504.25/972.02/514.80) lat (ms,99%): 82.96 err/s: 6.10 reconn/s: 0.00
[ 20s ] thds: 10 tps: 288.50 qps: 5892.27 (r/w/o: 4134.45/1152.11/605.71) lat (ms,99%): 68.05 err/s: 6.90 reconn/s: 0.00
[ 30s ] thds: 10 tps: 202.30 qps: 4163.91 (r/w/o: 2923.90/814.10/425.90) lat (ms,99%): 893.56 err/s: 6.50 reconn/s: 0.00
SQL statistics:
queries performed:
read: 105658
write: 29400
other: 15475
total: 150533
transactions: 7352 (244.78 per sec.)
queries: 150533 (5011.98 per sec.)
ignored errors: 195 (6.49 per sec.)
reconnects: 0 (0.00 per sec.)
General statistics:
total time: 30.0331s
total number of events: 7352
Latency (ms):
min: 14.43
avg: 40.82
max: 2938.61
99th percentile: 97.55
sum: 300120.33
Threads fairness:
events (avg/stddev): 735.2000/39.34
execution time (avg/stddev): 30.0120/0.01
