White Papers

ManualsBrandsDell ManualsConverged InfrastructureHigh Performance Computing Solution Resources

Easy Peasy NFS with RHEL6.5

Nirmala Sundararajan, Anthony Fernandez, June 2014

Network File System (NFS) is a central component of data processing. Multiple hardware

components and interconnects as well as performance tuning options can make this process a

complex one. Often times this can result in component mismatch or unsupported

configurations. So were you looking for a solution that is simple yet scalable?

This blog describes a solution that we verified and benchmarked in the Dell HPC Engineering

lab. It’s called NSS5.5 (NFS Storage Solution, version 5.5) and aims to offer more than

simplicity and scalability. Described below are the configuration and performance plus

information on a step-by-step guide on best practices and how to set up such a solution. Read

on for details!

For those of you familiar with our NSS-HA solutions, NSS is the non-HA version .The NSS5.5

solution includes one NFS server direct attached to standard SAS JBOD arrays. In the current

5.5 version, the NFS server is a PowerEdge R620 server with Ivy Bridge based processors (Intel

Xeon E5-2600 v2 series) and the NSS storage comprises of four direct attached PowerVault

MD1200 storage arrays. The NSS5.5 uses Red Hat Enterprise Linux 6.5 as the operating system

on the NFS server, InfiniBand ConnectX-3 as the I/O network between the NFS server and the

compute clients, and Mellanox OFED 2.1.1.0.6.

Figure 1 shows the NSS architecture as well as the high level file system layout.

Summary of content (7 pages)

PAGE 1
Easy Peasy NFS with RHEL6.5 Nirmala Sundararajan, Anthony Fernandez, June 2014 Network File System (NFS) is a central component of data processing. Multiple hardware components and interconnects as well as performance tuning options can make this process a complex one. Often times this can result in component mismatch or unsupported configurations. So were you looking for a solution that is simple yet scalable? This blog describes a solution that we verified and benchmarked in the Dell HPC Engineering lab.
PAGE 2
Figure 1 Architecture Diagram 3TB HDD RAID60, 10+2 span 512K Segment Size LV Stripped across RAID60 volumes XFS Format 3TB HDD RAID60, 10+2 span 512K Segment Size The NFS server uses two PERC H810 RAID controllers, each of which is connected to two storage arrays. Each of the four storage array has 12 3TB 7.2K RPM NL-SAS drives. Each storage array was configured as RAID 60 unit with 10+2 span length, 512KB stripe element size.
PAGE 3
The total I/O was kept constant at 256 GB. That is, for the 1 thread/1 client case, a single client read/wrote one 256 GB file; with 2 clients each client read/wrote a 128 GB file size and so on. 256 GB is 2x larger than the memory on the NFS server and using a large file size helps minimize cache effects and report the true performance from the storage solution. Details of the benchmark tool and methodology used are provided in Table 1. Table 1 Benchmark Information Iozone benchmark. V 3.
PAGE 4
As you can see from the graph a single NFS server can provide peak throughput of 2,671 MiB/s for sequential writes and 2,818 MiB/s for sequential reads. The results of the random reads and writes are shown in Figures 3 and 4. The random tests use a block size of 4K and a file size of 4G for all test cases. Note: Depending on the data file size used, these numbers may vary.
PAGE 5
Figure 4 Random Write Test Result NFS IPoIB random write I/O performance 120000 100000 IOPS 80000 60000 40000 20000 0 1 2 4 8 16 32 48 64 Number of concurrent clients A peak close to 100,000 IOPS was achieved on random writes. With 64 simultaneous clients, the peak measured with random reads is ~10,000 IOPS. From the graph in figure 3 it can be seen that the random read performance is still scaling, and has not reached saturation. Details of the test bed are provided in the Tables 2, 3, 4 and 5.
PAGE 6
1gbE Ethernet card (Daughter card slot) (1) Intel Gigabit Ethernet Quad Port I350-t Network Daughter Card External storage controller (2) PERC H810 RAID Controllers (Slot 2, 3) Systems Management iDRAC7 Enterprise Power Supply (2) Power Supply Units Storage configuration Storage Enclosure (4) Dell PowerVault MD1200 JBOD enclosure Hard Disk Drives (48) 3TB 7200 rpm NL SAS drives NSS5.5 - Software Versions Software Operating system Red Hat Enterprise Linux (RHEL) 6.5 x86_64 Kernel version 2.6.
PAGE 7
NSS5.5 - Client / HPC Compute Cluster Client / HPC Compute Cluster 64 PowerEdge M420 blade servers Clients 32 blades in each of two PowerEdge M1000e chassis Red Hat Enterprise Linux 6.4 x86-64 Chassis configuration Two PowerEdge M1000e chassis, each with 32 blades Two Mellanox M4001F FDR10 I/O modules per chassis Two PowerConnect M6220 I/O switch modules per chassis Each blade server has one Mellanox ConnectX-3 Dual-port InfiniBand FDR10 Mezzanine I/O Card Mellanox OFED 2.0-2.0.