User guide

Overview1–2

1.1 Overview of the Lustre file system

Lustre is a design for a networked file system that is coherent, scalable, parallel, and targeted towards high

performance computing (HPC) environments. Lustre separates access to file data from access to file meta-

data. File data is accessed through an object interface, which provides a higher level of access than a basic

block store. Each logical file store is called an Object Storage Target (OST) service. Data is stored on multiple

OST services, which may be served from one or more Object Storage Servers. Scalable access to file data

is provided by the Object Storage Servers, and scalable, independent access to file meta-data is provided

by the meta-data servers (MDS servers). Lustre configuration information is distributed to client nodes

through a configuration management server.

This modular architecture allows Lustre to overcome many of the bottlenecks and deficiencies of existing file

systems. The separation of data from meta-data allows extra capability to be added easily as data or meta-

data loads increase. Because Lustre is designed for HPC environments, high performance is at the centre of

the Lustre architecture.

Lustre networking protocols are implemented using the Lustre Networking Model API

(LNET) message

passing interface. The LNET network layer provides a network-independent transport layer that allows Lustre

to operate in multiple networking environments. Network types are implemented by Lustre Networking

Device layers (LNDs).

Lustre file systems can be accessed in the same way as other POSIX-compliant file systems.

Lustre is being developed and maintained as Open Source software under the GNU General Public License

(GPL), enabling broad support for industry-standard platforms.

Figure 1-1 shows a logical overview of the architecture of Lustre and its main features, which include the

following:

• Separation of data and meta-data

• Scalable meta-data

• Scalable file data

• Efficient locking

• Object architecture

Figure 1-1 Logical overview of the Lustre file system

Lustre Client

MDS Server

Configuration information, network connection details,

& security management

File I/O & file locking

Directory operations,

meta-data & concurrency

Recovery, file status,

file creation

Object Storage Servers

Configuration Management Server