White Papers

Dell Storage for HPC with Intel Enterprise Edition 2.3 for Lustre sofware
Table 2: Dell Storage for HPC with Intel EE for Lustre Configuration
Configuration Size
960TB RAW
Lustre Server Version
2.5.37.7
Intel EE for Lustre Version
v2.3
OSS Nodes
2 x PowerEdge R630 Servers
OSS Memory
256GB DDR4 2133MT/s
OSS Processors
2 x Intel Xeon™ E5-2660V3 @ 2.60GHz 10 cores
OSS Server BIOS
0.3.28
OSS Storage Array
2 x PowerVault MD3460, 2 x PowerVault MD3060e
Drives in OSS Storage Arrays
240 3.5" 4 TB 7.2K RPM NL SAS
OSS SAS Controllers
2 x SAS 12Gbps HBA LSI 9300-8e
MDS Nodes
2 x PowerEdge R630 Servers
MDS Memory
256GB DDR4 2133MT/s
MDS Processors
2 x Intel Xeon™ E5-2660V3 @ 2.60GHz 10 cores
MDS Server BIOS
0.3.28
MDS Storage Array
1 x PowerVault MD3420
Drives in MDS Storage Array
24 - 2.5" 300GB NL SAS
MDS SAS Controller
2 x SAS 12Gbps HBA LSI 9300-8e
Data network - InfiniBand
OSS, MDS Servers
Mellanox FDR HCA MT27500
Compute Nodes
Mellanox QDR HCA MT26428
Client QDR IB Switch
Mellanox M3601Q
FDR IB Switch
Mellanox 36 Ports SX6025
IB Switch Connectivity
Clients: QDR cables; Servers: FDR cables
3 uplinks from the QDR switch to the FDR switch
To prevent inflated results due to caching effects, tests were performed with a cold cache established
with the following technique. Before each test started, a remount of the Lustre File System under test
was executed. A sync was performed and the kernel was instructed to drop caches on all the clients
with the following commands:
 sync
 echo 3 > /proc/sys/vm/drop_caches
In addition, to simulate a cold cache on the server, a “sync” was performed on all the active servers
(OSS and MDS) before each test and the kernel was instructed to drop caches with the same commands
used on the client.
In measuring the performance of the Dell Storage for HPC with Intel EE for Lustre solution, all tests
were performed with similar initial conditions. The file system was configured to be fully functional
and the targets tested were emptied of files and directories prior to each test.