White Papers

Dell HPC Lustre Storage solution with Mellanox Infiniband EDR

Also during the preliminary metadata testing, we concluded that the number of files per directory

significantly affects the results, even while keeping constant the total number of files created. For

example when testing with 64 threads creating 3125 files per directory in 5 directories per thread OR

creating 625 files per directory in 25 directories per thread, both result in the creation of 1 million

files, but the measured performance in IOPS is not the same. This is due to overhead of seeks

performed on the OSTs when changing directories. In order to present coherent results, the number of

files per directory was fixed at 3125 while we varied the number of directories per thread to yield total

number of files to be no less than 1 million and steadily increasing as thread counts increase. Table 3

represents the values used in each test. For both the file operation tests as well as the directory

operations tests, we performed the tests with eight iterations, each taking the average value for

recorded results.

Table 3: Parameters used on MDtest

Number of Threads (N)

Number of Files per