Using InfiniBand for a Scalable Compute Infrastructure, 4th Edition

ManualsBrandsHP ManualsComputer AccessoriesHP 4X QDR QLogic InfiniBand Management Module for c-Class BladeSystem

Using InfiniBand for a scalable compute

infrastructure

Technology brief, 4

edition

Introduction ......................................................................................................................................... 2

InfiniBand technology ........................................................................................................................... 3

InfiniBand architecture ....................................................................................................................... 3

InfiniBand Quality of Service functions ................................................................................................ 5

InfiniBand performance ..................................................................................................................... 5

Scale-out clusters built on InfiniBand and HP technology ........................................................................... 7

HPC configuration with HP BladeSystem solutions ................................................................................. 8

HPC-optimized HP Cluster Platforms .................................................................................................. 10

Conclusion ........................................................................................................................................ 11

For more information .......................................................................................................................... 12

Call to action ..................................................................................................................................... 12

Summary of content (12 pages)

PAGE 1
Using InfiniBand for a scalable compute infrastructure Technology brief, 4th edition Introduction ......................................................................................................................................... 2 InfiniBand technology ........................................................................................................................... 3 InfiniBand architecture......................................................................................................
PAGE 2
Introduction Increasingly, IT organizations are deploying scale-out computing using clusters for these applications: • High performance computing (HPC) • Enterprise applications • Financial services • Scale-out databases Scale-out computing uses interconnect technology to combine a large number of stand-alone server nodes into an integrated and centrally managed system (Figure 1).
PAGE 3
InfiniBand technology InfiniBand is an industry-standard, channel-based architecture with an application-centric view to provide an easy-to-use messaging service. InfiniBand uses techniques such as stack bypass using remote direct memory access (RDMA) to let applications directly communicate with each other across the wire. This communication results in high-speed, low-latency connections for scale-out computing.
PAGE 4
Figure 3. InfiniBand link types 1x Link 4x Link (1 send channel 1 receive channel each using differential voltage or fiber optic signaling) 12x Link The host channel adapter (HCA) can divide (multiplex) each link into a set of virtual lanes, similar to highway lanes (Figure 4). Using virtual lanes, an InfiniBand link can share bandwidth between various sources and targets simultaneously. For example, if the HCA divides a 10 Gb/s link into five virtual lanes, each lane has a bandwidth of 2 Gb/s.
PAGE 5
InfiniBand Quality of Service functions All traffic in a clustered environment competes for fabric resources such as physical links and queues. To minimize interference and congestion, InfiniBand implements Quality of Service (QoS) functions on each virtual lane. Flow control Ethernet uses flow control as a time-based function. To avoid input overload, the receiving NIC tells the sending NIC to stop sending for a time. The protocol does not guarantee packet delivery; the NIC must re-send dropped packets.
PAGE 6
Figure 5. InfiniBand cluster topologies B: Fat tree A: Binary tree Root level Switch Switch Switch Leaf level Switch Switch Switch Switch To node I/Fs Switch Switch Switch Switch Switch Switch Switch To node I/Fs InfiniBand provides native support for fat-tree and other mesh topologies, allowing simultaneous connections across multiple links. The fabric scales as you connect more nodes and additional links.
PAGE 7
Scale-out clusters built on InfiniBand and HP technology Scale-out cluster computing has become the common architecture for HPC, and broader markets are adopting it. The trend is toward using space- and power-efficient systems for scale-out solutions. Figure 6 shows the key components for building a scalable HPC solution. Figure 6.
PAGE 8
HPC configuration with HP BladeSystem solutions Figure 7 shows a full-bandwidth, fat-tree configuration of HP BladeSystem c-Class components providing 576 nodes in a cluster. Each c7000 enclosure includes an HP 4x QDR InfiniBand Switch Blade, with 16 downlinks for server blade connection and 16 QSFP uplinks for fabric connectivity. Sixteen 36-port QDR InfiniBand switches provide spine-level fabric connectivity. Figure 7.
PAGE 9
To meet extreme density goals, the half-height HP BL2x220c server blade includes two server nodes. Each node can support two quad-core Intel® Xeon® 5400-series processors and a slot for a mezzanine board. That equals up to 32 nodes (256 cores) per c7000 enclosure. Each c7000 enclosure contains two HP 4x QDR InfiniBand Switch Blades. Figure 8 shows how using the dual-node BL2x220c blade lets you deploy 576 nodes in half as much rack space. Figure 8.
PAGE 10
HPC-optimized HP Cluster Platforms Deploying the HP Unified Cluster Portfolio (UCP) can give you the performance and flexibility of a custom solution with the simplicity and value of a factory-built product. The HP UCP is a modular package of hardware, software, and services. You can configure HP Cluster Platforms with HP ProLiant BL, DL, or SL systems. You have a choice of packaging styles, processor types, and density levels.
PAGE 11
Conclusion You should base your decision to use Ethernet or InfiniBand on performance and cost requirements. We are committed to supporting both InfiniBand and Ethernet infrastructures. We want to help you choose the most cost-effective fabric solution for your environment. InfiniBand is the best choice for HPC clusters requiring scalability from hundreds to thousands of nodes.
PAGE 12
For more information Resource description Web address HP products www.hp.com HPC/IB/cluster products www.hp.com/go/hptc HP InfiniBand products http://h18004.www1.hp.com/products/servers/networking/index -ib.html InfiniBand Trade Organization http://www.infinibandta.org Open Fabrics Alliance http://www.openib.org/ RDMA Consortium http://www.rdmaconsortium.org. Technology brief discussing iWARP RDMA http://h20000.www2.hp.com/bc/docs/support/SupportManual/c 00589475/c00589475.