White Papers

Dell Storage for HPC with Intel Enterprise Edition 2.3 for Lustre sofware
1. Introduction
In high performance computing (HPC), the efficient delivery of data to and from the compute nodes is
critical and often complicated to execute. Researchers can generate and consume data in HPC systems
at such speed that turns the storage components into a major bottleneck. Managing and monitoring
such complex storage systems add to the burden on storage administrators and researchers. The data
requirements around performance and capacity keep increasing rapidly. Increasing the throughput and
scalability of storage systems supporting the high performance computing system can require a great
deal of planning and configuration.
The Dell Storage for HPC with Intel® Enterprise Edition (EE) for Lustre
®
Solution, referred to as Dell
Storage for HPC with Intel EE for Lustre for the rest of this document, is designed for academic and
industry users who need to deploy a fully-supported, easy-to-use, high-throughput, scale-out and cost-
effective parallel file system storage solution. Dell Storage for HPC with Intel EE for Lustre is a scale-
out storage solution appliance capable of providing a high performance and high availability storage
system. Utilizing an intelligent, extensive and intuitive management interface, the Intel Manager for
Lustre (IML), the solution greatly simplifies deploying, managing and monitoring all of the hardware
and storage system components. It is easy to scale in capacity, performance or both, thereby providing
a convenient path to grow in the future. The storage solution uses Lustre, the leading HPC open source
parallel file system
1
.
The storage solution utilizes the 13th generation of enterprise Dell PowerEdge™ servers and the latest
generation of high-density PowerVault™ storage products. With full hardware and software support
from Dell and Intel, the Dell Storage for HPC with Intel EE for Lustre Solution delivers a superior
combination of performance, reliability, density, ease of use and cost-effectiveness.
The following sections of this paper will describe the Lustre File System and the Dell Storage for HPC
with Intel EE for Lustre Solution, followed by performance analysis,conclusions and Appendix A:
Benchmark Command Reference.
2. The Lustre File System
Lustre is a parallel file system, offering high performance through parallel access to data and
distributed locking. A Lustre installation consists of three key elements: the metadata subsystem, the
object storage subsystem (data) and the compute clients that access and operate on the data.
The metadata subsystem is comprised of the Metadata Target (MDT), the Management Target (MGT)
and a Metadata Server (MDS). The MDT stores all metadata for the file system including file names,
permissions, time stamps, and the location of data objects within the object storage system. The MGT
stores management data such as configuration information and registry. The MDS is a dedicated server
that manages the MDT. This solution stack will also introduce the option to have a Distributed
Namespace or Lustre DNE configuration.
1
A New Generation of Lustre Software Expands HPC into the Commercial Enterprise