White Papers

Dell Storage for HPC with Intel Enterprise Edition 2.3 for Lustre sofware

4.3 Metadata Testing

Metadata testing measures the time to complete certain file or directory operations that return

attributes. MDtest is an MPI-coordinated benchmark that performs Create, Stat, and Remove

operations on files or directories. This study used MDtest version 1.9.3. The MPI stack used for this

study was Intel MPI version 5.0 Update 1. The metric reported by MDtest is the rate of completion in

terms of operations per second (OP/sec). MDtest can be configured to compare metadata performance

for directories and files. For this test, we perform a pass with file operations, then a separate pass

with directory operations.

On a Lustre file system, OSTs are queried for object identifiers in order to allocate or locate extents

associated with the metadata operations. This interaction requires the indirect involvement of OSTs in

most metadata operations. In lab experiments with the test bed, it was found that using OST count of 1

was more efficient for most metadata operations, especially for higher thread counts. The experiments

consisted of conducting tests, with up to 64 clients for the following Lustre topologies (results not

presented here), to select the most efficient configuration for the metadata performance tests for this

version of the solution:

 1 OST, 1MB Stripes

 1 OST, 4MB Stripes

 24 OSTs, 1MB Stripes

 24 OSTs, 4MB Stripes

The most efficient configuration was found to be with 1 OST with 1MB stripes, therefore the results

presented in this section were obtained with this Lustre topology.

Also during the preliminary metadata testing, it was found that the number of files per directory

significantly affects the results, even while keeping constant the total number of files created. For

example when testing with 64 threads creating 3125 files per directory in 5 directories per thread OR

creating 625 files per directory in 25 directories per thread, both result in the creation of 1 million

files, but the measured performance in IOPS is not the same. This is due to overhead of seeks

performed on the OSTs when changing directories. In order to present coherent results, the number of

files per directory was fixed at 3125 while we varied the number of directories per thread to yield total

number of files to be no less than 1 million and steadily increasing as thread counts increase. Table 3

represents the values used in each test. For both the file operation tests as well as the directory

operations tests, we performed the tests with eight iterations, each taking the average value for

recorded results.

Table 3: Parameters used on MDtest

Number of Threads (N)

Number of Files per