White Papers

Dell HPC NFS Storage Solution High Availability Configurations with Large Capacities
22
NSS-HA client configuration details Table 8.
Client / HPC Compute Cluster
Clients
64 PowerEdge R410 compute nodes
Red Hat Enterprise Linux 6.1 x86-64
InfiniBand
Mellanox ConnectX-2 QDR HCA
Mellanox OFED 1.5.3-3.0.0
InfiniBand
fabric
All clients connected to a single large port count InfiniBand switch
(Mellanox IS5100).
Both R710 NSS-HA servers also connected to the InfiniBand switch.
Ethernet
Onboard 1 GbE Broadcom 5716 network adapter.
bnx2 driver v2.1.6.
Ethernet
fabric
Two sets of 32 compute nodes connected to two PowerConnect
6248 Gigabit Ethernet switches.
Both PowerConnect 6248 switches have four 10GbE links each to a
10GbE PowerConnect 8024 switch.
Both R710 NSS-HA servers connected directly to the PowerConnect
8024 switch.
Flow control was disabled on the PowerConnect 8024 switch and
two PowerConnect 6248 switches.
5.3. Tuning options
The tuning options and design choices in this version of the NSS-HA solution are similar to those in the
previous version
(4)
of the solution. The design of this solution emphasizes data reliability and
availability sometimes at the expense of performance. For custom configuration, some of these options
may not apply. Analysis of NFSv3 and NFSv4 is new to this release and is discussed in detail. For other
options a quick summary is provided here, detailed explanations can be found in the Solution Guide
titled “Dell HPC NFS Storage Solution High Availability Configurations, Version 1.1”.
Appendix A: NSS-HA Recipe provides instructions on how these tuning options should be configured.
Storage array configuration
1) Each storage array is configured with twelve 3.5” 3TB NearLine SAS disks.
2) Virtual disks are created using RAID 6, with 10 data disks and 2 parity disks.
3) Virtual disks are created with a segment size of 512k. This value should be set based on the
expected application I/O profile for the cluster.
4) Cache block size on the RAID controller is set to 32k to maximize performance. This value
should be set based on the expected application I/O profile for the cluster.
5) The read and write caches on the RAID controller are enabled.
6) Write cache mirroring is enabled between the two PowerVault MD3200 RAID controllers to
protect data if there is a controller failure. Cache mirroring between the controllers ensures
that the second controller can complete the writes to disk.