White Papers

Dell Storage for HPC with Intel Enterprise Edition 2.3 for Lustre sofware
level out to 256. This is partially a result of increasing the number of OSTs utilized, as the number of
threads is increased (up to the 24 OSTs in our system).
To maintain the higher throughput for an even greater number of files, increasing the number of OSTs
is likely to help. A review of the storage array performance using the tools provided by the Dell
PowerVault Modular Disk Storage Manager, Performance Monitor, was performed to independently
confirm the throughput values produced by the benchmarking tools.
We also explored the use of currently available LSI SAS drivers on the storage servers. This is in
comparison with the native version that is available with RHEL/CentOS 6.6. Figure 12 below will
illustrate that the two driver versions performed very similarly with slight variations on certain
workloads.
Figure 12: SAS drivers
There are various OS and SAS driver level tuning parameters that can be used to optimize the Lustre
storage servers for specific workloads. For instance, Lustre by default is tuned for and ideally should be
left at 1MB transfers. For this reason, it is generally best practice to align your storage system to 1MB
I/O request size. For instance, your storage servers could have driver parameters configured with
default limits that could result in I/O request size that will cause fragmentation of the 1MB Lustre RPC.
We explored avoiding this fragmentation by setting max_sgl_entries from default of 128 to 256 and
monitored the Lustre brw stats to verify 100% of I/O was in 1MB blocks. We found that in our
configuration with our tests workloads, this tuning did not provide an overwhelming difference in
performance.
The one tuning that we found to have significant advantage to sequential write performance was
disabling the write caching ability of the storage system. By default, when creating and configuring
your virtual disks on the Dell PowerVault MD3460 storage arrays, “Write Caching” is enabled. We used
the Dell PowerVault Modular Disk Storage Manager to disable this Write Caching tuning parameter on
0
2000
4000
6000
8000
10000
12000
1 2 4 8 12 16 24 32 48 64 72 96 120 128 256
Throughput in MB/s
Number of concurrent threads
Iozone Sequential I/O - Different SAS Drivers - Dell Storage for HPC with
Intel EE for Lustre
Write P8 Read P8 Write Native Read Native