HP Fabric Clustering System for InfiniBand™ Topologies

2
Document overview
This document describes the results of testing various HP Fabric Clustering System for InfiniBand
system and switch configurations with the intent to provide customers with the information needed to
determine which switch topology meets their budget and performance goals.
Document structure
This document is organized as follows:
Section 1 contains the overall purpose and organization of this paper.
Section 2 outlines general InfiniBand concepts and definitions used throughout this document.
Section 3 discusses the variety of configurations and results found.
Section 4 summarizes the conclusions.
General InfiniBand Concepts and Definitions
InfiniBand (IB) is a switched I/O fabric designed for low latency, high bandwidth, and with
capabilities that enable very low CPU consumption.
HP Fabric Clustering System supports InfiniBand on HP-UX as a High Performance Computing
clustering fabric specifically for MPI applications and, in the future, will also be supported as the
clustering interconnect for Oracle RAC.
The HP-UX Fabric Clustering System product consists of:
• AB286A - HP PCI-X 2-port 4X Fabric (HPC) Host Channel Adapter (HCA)
• AB291A – HP 12-port 4X Fabric Copper Switch
• 5m, 7m, and 10m 4x Fabric Copper Cables
• HP-UX software stack
HP also lists a reference 96-port 4X IB Switch, the Topspin TS170.
Finally, for the construction of MPI clusters on HP-UX 11iv2, HP supports MPI 2.0.
Methodology
HP used two classes of switch in analysis of the HP Fabric Clustering System product. One, the
AB291A, is a 12-port 4X switch sold by HP. The other, the Topspin TS170, is a 96-port 4X CLOS
switch listed as a reference product for HP-UX (www.hp.com/go/hphp
).
HP has compared the results between the two switch classes. The AB291A is not a CLOS switch and
is expected to yield differing results from a CLOS switch in similar configurations.
The methodology has been to construct clusters using compute nodes consisting of an HP-UX server
platform, the rx2600 with 2-1.5GHz Itanium processors and 4GB of memory, running the Fabric
Clustering System software stack on HP-UX 11iv2 with HP’s MPI middleware. The compute nodes
each are outfitted with a single 133 MHz PCI-X 4X Fabric Clustering System HCA and are each
interconnected using 4X Fabric Copper Cabling between one HCA port and the switches.
Once the clusters are constructed, we run the Pallas benchmark (PMB) suite (www.pallas.com
) on
each configuration and graph the result. In HP’s view, the Pallas benchmark can give insight into the
projected performance of a high-performance computing workload. We run PMB with both one