HP High Performance Clusters LC 3000 Series Cabling Guide

ManualsBrandsHP ManualsServerHP 2064-F LC Cluster Server

HP High Performance Clusters LC 3000 Series

Cabling Guide

March 2004 (First Edition)

Part Number 365686-001

Summary of content (80 pages)

PAGE 1
HP High Performance Clusters LC 3000 Series Cabling Guide March 2004 (First Edition) Part Number 365686-001
PAGE 2
© Copyright 2004 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.
PAGE 3
Contents About This Guide Audience Assumptions..................................................................................................................................v Where to Go for Additional Help..................................................................................................................v Technical Support ...................................................................................................................................
PAGE 4
Contents Chapter 7 24 Node Fast Ethernet Expandable to 36 Nodes Rack Diagram ............................................................................................................................................7-2 Point-to-Point Cabling Guide ....................................................................................................................7-3 Chapter 8 32 Node Myrinet Expandable to 64 Nodes Rack Diagram ..............................................................................
PAGE 5
About This Guide This guide provides cabling information for the HP High Performance Cluster LC 3000. Audience Assumptions This guide is for the person who installs, administers, and troubleshoots servers. HP assumes you are qualified in the servicing of computer equipment and trained in recognizing hazards in products with hazardous energy levels and are familiar with weight and stability precautions for rack installations.
PAGE 6
1 Server and Switch Connections Server Connections Figure 1-1: DL145 rear view Figure 1-2: DL145 rear view with Myrinet adapter Figure 1-3: DL380 rear view HP High Performance Clusters LC 3000 Series Cabling Guide 1-1
PAGE 7
Server and Switch Connections Switch Connections Figure 1-4: ProCurve 2650 rear view Figure 1-5: ProCurve 2848 rear view Figure 1-6: ProCurve 5372 rear view showing all modules 1-2 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 8
Server and Switch Connections Figure 1-7: Myricom 5 slot switch rear view Figure 1-8: Myricom 9 slot switch rear view HP High Performance Clusters LC 3000 Series Cabling Guide 1-3
PAGE 9
Server and Switch Connections Figure 1-9: Myricom 17 slot switch rear view 1-4 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 10
2 16 Node Myrinet Expandable to 32 Nodes HP High Performance Clusters LC 3000 Series Cabling Guide 2-1
PAGE 11
16 Node Myrinet Expandable to 32 Nodes Rack Diagram 2-2 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 12
16 Node Myrinet Expandable to 32 Nodes Point-to-Point Cabling Guide Table 2-1: Cable Label Nomenclature Symbol Description R Rack number S Control/Compute/Storage node number SW Out of Band (OOB) Management Switch/Cluster Interconnect Switch/In Band (IB) Management Switch number M Module letter or Module number P Port number NIC NIC port number IPMI IPMI port PCI Myrinet Adapter (in PCI slot) Table 2-2: 16 Node Myrinet Cabling for DL145 LC Clusters Expandable to 32 Nodes OOB Switch Cablin
PAGE 13
16 Node Myrinet Expandable to 32 Nodes Table 2-2: 16 Node Myrinet Cabling for DL145 LC Clusters Expandable to 32 Nodes continued OOB Switch Cabling Origin (server) Cluster Interconnect Switch Cabling Destination (switch) Origin (server) IB Management Switch Cabling Destination (switch) Origin (server) Destination (switch) R1-S14-IPMI R1-SW3-P14 R1-S14-PCI R1-SW1-M2-P13 R1-S14-NIC2 R1-SW2-P14 R1-S15-IPMI R1-SW3-P15 R1-S15-PCI R1-SW1-M2-P14 R1-S15-NIC2 R1-SW2-P15 R1-S16-IPMI R1-SW3-P16
PAGE 14
3 16 Node GigE Expandable to 22 Nodes HP High Performance Clusters LC 3000 Series Cabling Guide 3-1
PAGE 15
16 Node GigE Expandable to 22 Nodes Rack Diagram 3-2 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 16
Node GigE Expandable to 22 Nodes Point-to-Point Cabling Guide Table 3-1: Cable Label Nomenclature Symbol Description R Rack number S Control/Compute/Storage node number SW Out of Band (OOB) Management Switch/Cluster Interconnect Switch/In Band (IB) Management Switch number M Module letter or Module number P Port number NIC NIC port number IPMI IPMI port Table 3-2: 16 Node GigE Cabling for DL145 LC Clusters Expandable to 22 Nodes OOB Switch Cabling Origin (server) Destination (switch) C
PAGE 17
16 Node GigE Expandable to 22 Nodes Table 3-2: 16 Node GigE Cabling for DL145 LC Clusters Expandable to 22 Nodes continued OOB Switch Cabling Origin (server) Cluster Interconnect Switch Cabling Destination (switch) Origin (server) IB Management Switch Cabling Destination (switch) Origin (server) Destination (switch) R1-S14-IPMI R1-SW2-P14 R1-S14-NIC1 R1-SW1-P14 R1-S14-NIC2 R1-SW1-P36 R1-S15-IPMI R1-SW2-P15 R1-S15-NIC1 R1-SW1-P15 R1-S15-NIC2 R1-SW1-P37 R1-S16-IPMI R1-SW2-P16 R1-S16-NIC
PAGE 18
4 16 Node Fast Ethernet Expandable to 22 Nodes HP High Performance Clusters LC 3000 Series Cabling Guide 4-1
PAGE 19
16 Node Fast Ethernet Expandable to 22 Nodes Rack Diagram 4-2 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 20
16 Node Fast Ethernet Expandable to 22 Nodes Point-to-Point Cabling Guide Table 4-1: Cable Label Nomenclature Symbol Description R Rack number S Control/Compute/Storage node number SW Out of Band (OOB) Management Switch/Cluster Interconnect Switch/In Band (IB) Management Switch number M Module letter or Module number P Port number NIC NIC port number IPMI IPMI port Table 4-2: 16 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 22 Nodes OOB Switch Cabling Origin (server) Desti
PAGE 21
16 Node Fast Ethernet Expandable to 22 Nodes Table 4-2: 16 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 22 Nodes continued OOB Switch Cabling Origin (server) Cluster Interconnect Switch Cabling Destination (switch) Origin (server) IB Management Switch Cabling Destination (switch) Origin (server) Destination (switch) R1-S14-IPMI R1-SW2-P14 R1-S14-NIC1 R1-SW1-P14 R1-S14-NIC2 R1-SW1-P38 R1-S15-IPMI R1-SW2-P15 R1-S15-NIC1 R1-SW1-P15 R1-S15-NIC2 R1-SW1-P39 R1-S16-IPMI R1-
PAGE 22
5 24 Node Myrinet Expandable to 32 Nodes HP High Performance Clusters LC 3000 Series Cabling Guide 5-1
PAGE 23
24 Node Myrinet Expandable to 32 Nodes Rack Diagram 5-2 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 24
Node Myrinet Expandable to 32 Nodes Point-to-Point Cabling Guide Table 5-1: Cable Label Nomenclature Symbol Description R Rack number S Control/Compute/Storage node number SW Out of Band (OOB) Management Switch/Cluster Interconnect Switch/In Band (IB) Management Switch number M Module letter or Module number P Port number NIC NIC port number IPMI IPMI port PCI Myrinet Adapter (in PCI slot) Table 5-2: 24 Node Myrinet Cabling for DL145 LC Clusters Expandable to 32 Nodes OOB Switch Cablin
PAGE 25
24 Node Myrinet Expandable to 32 Nodes Table 5-2: 24 Node Myrinet Cabling for DL145 LC Clusters Expandable to 32 Nodes continued OOB Switch Cabling Origin (server) Cluster Interconnect Switch Cabling Destination (switch) Origin (server) IB Management Switch Cabling Destination (switch) Origin (server) Destination (switch) R1-S14-IPMI R1-SW3-P14 R1-S14-PCI R1-SW1-M2-P13 R1-S14-NIC2 R1-SW2-P14 R1-S15-IPMI R1-SW3-P15 R1-S15-PCI R1-SW1-M2-P14 R1-S15-NIC2 R1-SW2-P15 R1-S16-IPMI R1-SW3-P16
PAGE 26
6 24 Node GigE Expandable to 36 Nodes HP High Performance Clusters LC 3000 Series Cabling Guide 6-1
PAGE 27
24 Node GigE Expandable to 36 Nodes Rack Diagram 6-2 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 28
24 Node GigE Expandable to 36 Nodes Point-to-Point Cabling Guide Table 6-1: Cable Label Nomenclature Symbol Description R Rack number S Control/Compute/Storage node number SW Out of Band (OOB) Management Switch/Cluster Interconnect Switch/In Band (IB) Management Switch number M Module letter or Module number P Port number NIC NIC port number IPMI IPMI port Table 6-2: 24 Node GigE Cabling for DL145 LC Clusters Expandable to 36 Nodes OOB Switch Cabling Origin (server) Destination (switch) C
PAGE 29
24 Node GigE Expandable to 36 Nodes Table 6-2: 24 Node GigE Cabling for DL145 LC Clusters Expandable to 36 Nodes continued OOB Switch Cabling Origin (server) Cluster Interconnect Switch Cabling Destination (switch) Origin (server) IB Management Switch Cabling Destination (switch) Origin (server) Destination (switch) R1-S14-IPMI R1-SW3-P14 R1-S14-NIC1 R1-SW1-P14 R1-S14-NIC2 R1-SW2-P14 R1-S15-IPMI R1-SW3-P15 R1-S15-NIC1 R1-SW1-P15 R1-S15-NIC2 R1-SW2-P15 R1-S16-IPMI R1-SW3-P16 R1-S16-NIC
PAGE 30
7 24 Node Fast Ethernet Expandable to 36 Nodes HP High Performance Clusters LC 3000 Series Cabling Guide 7-1
PAGE 31
24 Node Fast Ethernet Expandable to 36 Nodes Rack Diagram 7-2 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 32
24 Node Fast Ethernet Expandable to 36 Nodes Point-to-Point Cabling Guide Table 7-1: Cable Label Nomenclature Symbol Description R Rack number S Control/Compute/Storage node number SW Out of Band (OOB) Management Switch/Cluster Interconnect Switch/In Band (IB) Management Switch number M Module letter or Module number P Port number NIC NIC port number IPMI IPMI port Table 7-2: 24 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 36 Nodes OOB Switch Cabling Origin (server) Desti
PAGE 33
24 Node Fast Ethernet Expandable to 36 Nodes Table 7-2: 24 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 36 Nodes continued OOB Switch Cabling Origin (server) Cluster Interconnect Switch Cabling Destination (switch) Origin (server) IB Management Switch Cabling Destination (switch) Origin (server) Destination (switch) R1-S14-IPMI R1-SW3-P14 R1-S14-NIC1 R1-SW1-P14 R1-S14-NIC2 R1-SW2-P14 R1-S15-IPMI R1-SW3-P15 R1-S15-NIC1 R1-SW1-P15 R1-S15-NIC2 R1-SW2-P15 R1-S16-IPMI R1-
PAGE 34
8 32 Node Myrinet Expandable to 64 Nodes HP High Performance Clusters LC 3000 Series Cabling Guide 8-1
PAGE 35
32 Node Myrinet Expandable to 64 Nodes Rack Diagram 8-2 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 36
32 Node Myrinet Expandable to 64 Nodes Point-to-Point Cabling Guide Table 8-1: Cable Label Nomenclature Symbol Description R Rack number S Control/Compute/Storage node number SW Out of Band (OOB) Management Switch/Cluster Interconnect Switch/In Band (IB) Management Switch number M Module letter or Module number P Port number NIC NIC port number IPMI IPMI port PCI Myrinet Adapter (in PCI slot) Table 8-2: 32 Node Myrinet Cabling for DL145 LC Clusters Expandable to 64 Nodes OOB Switch Cablin
PAGE 37
32 Node Myrinet Expandable to 64 Nodes Table 8-2: 32 Node Myrinet Cabling for DL145 LC Clusters Expandable to 64 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R1-S14-IPMI R1-SW3-P14 R1-S14-PCI R1-SW1-M2-P13 R1-S14-NIC2 R1-SW2-P14 R1-S15-IPMI R1-SW3-P15 R1-S15-PCI R1-SW1-M2-P14 R1-S15-NIC2 R1-SW2-P15 R1-S16-IPMI R1-SW3-P16 R1
PAGE 38
32 Node Myrinet Expandable to 64 Nodes Table 8-2: 32 Node Myrinet Cabling for DL145 LC Clusters Expandable to 64 Nodes continued OOB Switch Cabling Origin (server) Cluster Interconnect Switch Cabling Destination (switch) Origin (server) IB Management Switch Cabling Destination (switch) Origin (server) Destination (switch) R2-S11-IPMI R2-SW2-P11 R2-S11-PCI R1-SW1-M6-P10 R2-S11-NIC2 R2-SW1-P11 R2-S12-IPMI R2-SW2-P12 R2-S12-PCI R1-SW1-M6-P11 R2-S12-NIC2 R2-SW1-P12 R2-S13-IPMI R2-SW2-P13
PAGE 39
32 Node Myrinet Expandable to 64 Nodes Table 8-5: IB Management Switch Uplink From 8-6 To R1-SW2-P46 R2-SW1-P47 R1-SW2-P45 R2-SW1-P46 R1-SW2-P44 R2-SW1-P45 R1-SW2-P43 R2-SW1-P44 Trunked together HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 40
9 32 Node GigE Expandable to 64 Nodes HP High Performance Clusters LC 3000 Series Cabling Guide 9-1
PAGE 41
32 Node GigE Expandable to 64 Nodes Rack Diagram 9-2 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 42
32 Node GigE Expandable to 64 Nodes Point-to-Point Cabling Guide Table 9-1: Cable Label Nomenclature Symbol Description R Rack number S Control/Compute/Storage node number SW Out of Band (OOB) Management Switch/Cluster Interconnect Switch/In Band (IB) Management Switch number M Module letter or Module number P Port number NIC NIC port number IPMI IPMI port Table 9-2: 32 Node GigE Cabling for DL145 LC Clusters Expandable to 64 Nodes OOB Switch Cabling Origin (server) Destination (switch) C
PAGE 43
32 Node GigE Expandable to 64 Nodes Table 9-2: 32 Node GigE Cabling for DL145 LC Clusters Expandable to 64 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R1-S14-IPMI R1-SW3-P14 R1-S14-NIC1 R1-SW1-M2-P13 R1-S14-NIC2 R1-SW2-P14 R1-S15-IPMI R1-SW3-P15 R1-S15-NIC1 R1-SW1-M2-P14 R1-S15-NIC2 R1-SW2-P15 R1-S16-IPMI R1-SW3-P16 R1-S16
PAGE 44
32 Node GigE Expandable to 64 Nodes Table 9-2: 32 Node GigE Cabling for DL145 LC Clusters Expandable to 64 Nodes continued OOB Switch Cabling Origin (server) Cluster Interconnect Switch Cabling Destination (switch) Origin (server) IB Management Switch Cabling Destination (switch) Origin (server) Destination (switch) R2-S11-IPMI R2-SW2-P11 R2-S11-NIC1 R1-SW1-M6-P10 R2-S11-NIC2 R2-SW1-P11 R2-S12-IPMI R2-SW2-P12 R2-S12-NIC1 R1-SW1-M6-P11 R2-S12-NIC2 R2-SW1-P12 R2-S13-IPMI R2-SW2-P13 R2-S
PAGE 45
32 Node GigE Expandable to 64 Nodes Table 9-5: IB Management Switch Uplink From 9-6 To R1-SW2-P46 R2-SW1-P47 R1-SW2-P45 R2-SW1-P46 R1-SW2-P44 R2-SW1-P45 R1-SW2-P43 R2-SW1-P44 Trunked together HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 46
10 32 Node Fast Ethernet Expandable to 72 Nodes HP High Performance Clusters LC 3000 Series Cabling Guide 10-1
PAGE 47
32 Node Fast Ethernet Expandable to 72 Nodes Rack Diagram 10-2 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 48
32 Node Fast Ethernet Expandable to 72 Nodes Point-to-Point Cabling Guide Table 10-1: Cable Label Nomenclature Symbol Description R Rack number S Control/Compute/Storage node number SW Out of Band (OOB) Management Switch/Cluster Interconnect Switch/In Band (IB) Management Switch number M Module letter or Module number P Port number NIC NIC port number IPMI IPMI port Table 10-2: 32 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 72 Nodes OOB Switch Cabling Origin (server) Des
PAGE 49
32 Node Fast Ethernet Expandable to 72 Nodes Table 10-2: 32 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 72 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R1-S14-IPMI R1-SW3-P14 R1-S14-NIC1 R1-SW1-MB-P14 R1-S14-NIC2 R1-SW2-P14 R1-S15-IPMI R1-SW3-P15 R1-S15-NIC1 R1-SW1-MB-P15 R1-S15-NIC2 R1-SW2-P15 R1-S16-IPMI
PAGE 50
32 Node Fast Ethernet Expandable to 72 Nodes Table 10-2: 32 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 72 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R2-S11-IPMI R2-SW2-P11 R2-S11-NIC1 R1-SW1-MC-P19 R2-S11-NIC2 R2-SW1-P11 R2-S12-IPMI R2-SW2-P12 R2-S12-NIC1 R1-SW1-MC-P20 R2-S12-NIC2 R2-SW1-P12 R2-S13-IPMI
PAGE 51
32 Node Fast Ethernet Expandable to 72 Nodes Table 10-3: OOB Management Switch Uplink From To R1-SW3-P48 R1-SW2-P48 R2-SW2-P48 R2-SW1-P48 Table 10-4: IB Management Switch Uplink From 10-6 To R1-SW2-P50 R2-SW1-P50 R1-SW2-P49 R2-SW1-P49 Trunked together HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 52
11 64 Node Myrinet Expandable to 128 Nodes HP High Performance Clusters LC 3000 Series Cabling Guide 11-1
PAGE 53
64 Node Myrinet Expandable to 128 Nodes Rack Diagram 11-2 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 54
64 Node Myrinet Expandable to 128 Nodes Point-to-Point Cabling Guide Table 11-1: Cable Label Nomenclature Symbol Description R Rack number S Control/Compute/Storage node number SW Out of Band (OOB) Management Switch/Cluster Interconnect Switch/In Band (IB) Management Switch number M Module letter or Module number P Port number NIC NIC port number IPMI IPMI port PCI Myrinet Adapter (in PCI slot) Table 11-2: 64 Node Myrinet Cabling for DL145 LC Clusters Expandable to 128 Nodes OOB Switch Ca
PAGE 55
64 Node Myrinet Expandable to 128 Nodes Table 11-2: 64 Node Myrinet Cabling for DL145 LC Clusters Expandable to 128 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R1-S14-IPMI R1-SW2-P14 R1-S14-PCI R2-SW1-M6-P9 R1-S14-NIC2 R1-SW1-P14 R1-S15-IPMI R1-SW2-P15 R1-S15-PCI R2-SW1-M6-P10 R1-S15-NIC2 R1-SW1-P15 R1-S16-IPMI R1-SW2-P16
PAGE 56
64 Node Myrinet Expandable to 128 Nodes Table 11-2: 64 Node Myrinet Cabling for DL145 LC Clusters Expandable to 128 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R2-S4-IPMI R2-SW3-P4 R2-S4-PCI R2-SW1-M1-P11 R2-S4-NIC2 R2-SW2-P4 R2-S5-IPMI R2-SW3-P5 R2-S5-PCI R2-SW1-M1-P12 R2-S5-NIC2 R2-SW2-P5 R2-S6-IPMI R2-SW3-P6 R2-S6-PCI
PAGE 57
64 Node Myrinet Expandable to 128 Nodes Table 11-2: 64 Node Myrinet Cabling for DL145 LC Clusters Expandable to 128 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R3-S5-IPMI R3-SW2-P5 R3-S5-PCI R2-SW1-M10-P8 R3-S5-NIC2 R3-SW1-P5 R3-S6-IPMI R3-SW2-P6 R3-S6-PCI R2-SW1-M10-P9 R3-S6-NIC2 R3-SW1-P6 R3-S7-IPMI R3-SW2-P7 R3-S7-PCI
PAGE 58
64 Node Myrinet Expandable to 128 Nodes Table 11-2: 64 Node Myrinet Cabling for DL145 LC Clusters Expandable to 128 Nodes continued OOB Switch Cabling Origin (server) Cluster Interconnect Switch Cabling Destination (switch) Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R3-S35-IPMI R3-SW2-P35 R3-S35-PCI R2-SW1-M13-P14 R3-S35-NIC2 R3-SW1-P35 R3-S36-IPMI R3-SW2-P36 R3-S36-PCI R2-SW1-M13-P15 R3-S36-NIC2 R3-SW1-P36 R3-S37-IPMI R3-SW2-P
PAGE 59
64 Node Myrinet Expandable to 128 Nodes Table 11-4: OOB Management Switch Uplink From To R1-SW2-P48 R1-SW1-P48 R2-SW3-P48 R2-SW2-P48 R3-SW2-P48 R3-SW1-P48 R4-SW2-P48 R4-SW1-P48 Table 11-5: IB Management Switch Uplink From 11-8 To R2-SW2-P46 R1-SW1-P47 R2-SW2-P45 R1-SW1-P46 R2-SW2-P44 R1-SW1-P45 R2-SW2-P43 R1-SW1-P44 R2-SW2-P42 R3-SW1-P47 R2-SW2-P41 R3-SW1-P46 R2-SW2-P40 R3-SW1-P45 R2-SW2-P39 R3-SW1-P44 R2-SW2-P38 R4-SW1-P47 R2-SW2-P37 R4-SW1-P46 R2-SW2-P36 R4-SW1-P45 R2-S
PAGE 60
12 64 Node GigE Expandable to 128 Nodes HP High Performance Clusters LC 3000 Series Cabling Guide 12-1
PAGE 61
64 Node GigE Expandable to 128 Nodes Rack Diagram 12-2 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 62
64 Node GigE Expandable to 128 Nodes Point-to-Point Cabling Guide Table 12-1: Cable Label Nomenclature Symbol Description R Rack number S Control/Compute/Storage node number SW Out of Band (OOB) Management Switch/Cluster Interconnect Switch/In Band (IB) Management Switch number M Module letter or Module number P Port number NIC NIC port number IPMI IPMI port Table 12-2: 64 Node GigE Cabling for DL145 LC Clusters Expandable to 128 Nodes OOB Switch Cabling Origin (server) Destination (switch
PAGE 63
64 Node GigE Expandable to 128 Nodes Table 12-2: 64 Node GigE Cabling for DL145 LC Clusters Expandable to 128 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R1-S14-IPMI R1-SW2-P14 R1-S14-NIC1 R2-SW1-M6-P9 R1-S14-NIC2 R1-SW1-P14 R1-S15-IPMI R1-SW2-P15 R1-S15-NIC1 R2-SW1-M6-P10 R1-S15-NIC2 R1-SW1-P15 R1-S16-IPMI R1-SW2-P16 R1-S
PAGE 64
Node GigE Expandable to 128 Nodes Table 12-2: 64 Node GigE Cabling for DL145 LC Clusters Expandable to 128 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R2-S4-IPMI R2-SW3-P4 R2-S4-NIC1 R2-SW1-M1-P11 R2-S4-NIC2 R2-SW2-P4 R2-S5-IPMI R2-SW3-P5 R2-S5-NIC1 R2-SW1-M1-P12 R2-S5-NIC2 R2-SW2-P5 R2-S6-IPMI R2-SW3-P6 R2-S6-NIC1 R2-
PAGE 65
64 Node GigE Expandable to 128 Nodes Table 12-2: 64 Node GigE Cabling for DL145 LC Clusters Expandable to 128 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R3-S5-IPMI R3-SW2-P5 R3-S5-NIC1 R2-SW1-M10-P8 R3-S5-NIC2 R3-SW1-P5 R3-S6-IPMI R3-SW2-P6 R3-S6-NIC1 R2-SW1-M10-P9 R3-S6-NIC2 R3-SW1-P6 R3-S7-IPMI R3-SW2-P7 R3-S7-NIC1 R2-
PAGE 66
64 Node GigE Expandable to 128 Nodes Table 12-2: 64 Node GigE Cabling for DL145 LC Clusters Expandable to 128 Nodes continued OOB Switch Cabling Origin (server) Cluster Interconnect Switch Cabling Destination (switch) Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R3-S35-IPMI R3-SW2-P35 R3-S35-NIC1 R2-SW1-M13-P14 R3-S35-NIC2 R3-SW1-P35 R3-S36-IPMI R3-SW2-P36 R3-S36-NIC1 R2-SW1-M13-P15 R3-S36-NIC2 R3-SW1-P36 R3-S37-IPMI R3-SW2-P37
PAGE 67
64 Node GigE Expandable to 128 Nodes Table 12-4: OOB Management Switch Uplink From To R1-SW2-P48 R1-SW1-P48 R2-SW3-P48 R2-SW2-P48 R3-SW2-P48 R3-SW1-P48 R4-SW2-P48 R4-SW1-P48 Table 12-5: IB Management Switch Uplink From 12-8 To R2-SW2-P46 R1-SW1-P47 R2-SW2-P45 R1-SW1-P46 R2-SW2-P44 R1-SW1-P45 R2-SW2-P43 R1-SW1-P44 R2-SW2-P42 R3-SW1-P47 R2-SW2-P41 R3-SW1-P46 R2-SW2-P40 R3-SW1-P45 R2-SW2-P39 R3-SW1-P44 R2-SW2-P38 R4-SW1-P47 R2-SW2-P37 R4-SW1-P46 R2-SW2-P36 R4-SW1-P45 R2-SW2-
PAGE 68
13 64 Node Fast Ethernet Expandable to 192 Nodes HP High Performance Clusters LC 3000 Series Cabling Guide 13-1
PAGE 69
64 Node Fast Ethernet Expandable to 192 Nodes Rack Diagram 13-2 HP High Performance Clusters LC 3000 Series Cabling Guide
PAGE 70
64 Node Fast Ethernet Expandable to 192 Nodes Point-to-Point Cabling Guide Table 13-1: Cable Label Nomenclature Symbol Description R Rack number S Control/Compute/Storage node number SW Out of Band (OOB) Management Switch/Cluster Interconnect Switch/In Band (IB) Management Switch number M Module letter or Module number P Port number NIC NIC port number IPMI IPMI port Table 13-2: 64 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 192 Nodes OOB Switch Cabling Origin (server) D
PAGE 71
64 Node Fast Ethernet Expandable to 192 Nodes Table 13-2: 64 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 192 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R1-S14-IPMI R1-SW2-P14 R1-S14-NIC1 R2-SW1-MC-P22 R1-S14-NIC2 R1-SW1-P14 R1-S15-IPMI R1-SW2-P15 R1-S15-NIC1 R2-SW1-MC-P23 R1-S15-NIC2 R1-SW1-P15 R1-S16-IP
PAGE 72
64 Node Fast Ethernet Expandable to 192 Nodes Table 13-2: 64 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 192 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R2-S4-IPMI R2-SW3-P4 R2-S4-NIC1 R2-SW1-MB-P4 R2-S4-NIC2 R2-SW2-P4 R2-S5-IPMI R2-SW3-P5 R2-S5-NIC1 R2-SW1-MB-P5 R2-S5-NIC2 R2-SW2-P5 R2-S6-IPMI R2-SW3-P6
PAGE 73
64 Node Fast Ethernet Expandable to 192 Nodes Table 13-2: 64 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 192 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R3-S1-IPMI R3-SW2-P1 R3-S1-NIC1 R2-SW1-ME-P1 R3-S1-NIC2 R3-SW1-P1 R3-S2-IPMI R3-SW2-P2 R3-S2-NIC1 R2-SW1-ME-P2 R3-S2-NIC2 R3-SW1-P2 R3-S3-IPMI R3-SW2-P3
PAGE 74
64 Node Fast Ethernet Expandable to 192 Nodes Table 13-2: 64 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 192 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R3-S31-IPMI R3-SW2-P31 R3-S31-NIC1 R2-SW1-MF-P7 R3-S31-NIC2 R3-SW1-P31 R3-S32-IPMI R3-SW2-P32 R3-S32-NIC1 R2-SW1-MF-P8 R3-S32-NIC2 R3-SW1-P32 R3-S33-IPMI
PAGE 75
64 Node Fast Ethernet Expandable to 192 Nodes Table 13-2: 64 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 192 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R4-S21-IPMI R4-SW2-P21 R4-S21-NIC1 R2-SW1-MG-P13 R4-S21-NIC2 R4-SW1-P21 R4-S22-IPMI R4-SW2-P22 R4-S22-NIC1 R2-SW1-MG-P14 R4-S22-NIC2 R4-SW1-P22 R4-S23-IP
PAGE 76
64 Node Fast Ethernet Expandable to 192 Nodes Table 13-2: 64 Node Fast Ethernet Cabling for DL145 LC Clusters Expandable to 192 Nodes continued OOB Switch Cabling Origin (server) Destination (switch) Cluster Interconnect Switch Cabling Origin (server) Destination (switch) IB Management Switch Cabling Origin (server) Destination (switch) R5-S11-IPMI R5-SW2-P11 R5-S11-NIC1 R2-SW1-MH-P19 R5-S11-NIC2 R5-SW1-P11 R5-S12-IPMI R5-SW2-P12 R5-S12-NIC1 R2-SW1-MH-P20 R5-S12-NIC2 R5-SW1-P12 R5-S13-IP
PAGE 77
64 Node Fast Ethernet Expandable to 192 Nodes Table 13-3: OOB Management Switch Uplink From To R1-SW2-P48 R1-SW1-P48 R2-SW3-P48 R2-SW2-P48 R3-SW2-P48 R3-SW1-P48 R4-SW2-P48 R4-SW1-P48 R5-SW2-P48 R5-SW1-P48 Table 13-4: IB Management Switch Uplink From 13-10 To R2-SW2-P47 R1-SW1-P50 R2-SW2-P46 R1-SW1-P49 R2-SW2-P45 R3-SW1-P50 R2-SW2-P44 R3-SW1-P49 R2-SW2-P43 R4-SW1-P50 R2-SW2-P42 R4-SW1-P49 R2-SW2-P41 R5-SW1-P50 R2-SW2-P40 R5-SW1-P49 Trunked together Trunked together Trunked t
PAGE 78
A Cabling Rules for LC Clusters HP factory-configured LC clusters are built according to these guidelines for numbering, labeling, and populating LC clusters. The guidelines are included in this document as reference information for customers who are customizing or upgrading existing LC clusters. Numbering and Cable Labeling Rules Racks—Racks are numbered 1 to n and from left to right. The racks are labeled “Rack n,” where n is the rack number.
PAGE 79
Cabling Rules for LC Clusters PCI Cards—Labels on cables connected to the PCI cards use the PCI slot markings on the server if available. The designator of the actual slot name is used on the labels, if present, or “PCIn” is used, where n is the slot number, if the slots are numbered. For example, if the hardware marks the slots with “1” or “2,” then the cable labels refer to these slots as “PCI1” and “PCI2,” respectively.
PAGE 80
Cabling Rules for LC Clusters Power Connections—The factory distributes the power connections across the PDUs in the LC cluster to avoid overload conditions. The most common PDUs used in the LC clusters are the 24A and 32A Control Units with four, eight-receptacle Extension Bars. Using these PDUs, the factory will not connect more than five components to an Extension Bar and no more than 14 total components to each PDU. Different population rules may apply to different capacity PDUs.