TruCluster Server Cluster Hardware Configuration Technical Update for the AlphaServer ES45 May 2002 Product Version: TruCluster Server Version 5.1A Operating System and Version: Tru64 UNIX Version 5.1A This technical update describes how to configure an AlphaServer ES45 in a TruCluster Server environment, including configuration restrictions.
© 2002 Compaq Computer Corporation Compaq, the Compaq logo, Alpha, AlphaServer, OpenVMS, StorageWorks, Tru64, and TruCluster are trademarks of Compaq Information Technologies Group, L.P. in the U.S and/or other countries. Microsoft, Windows, and Windows NT are trademarks of Microsoft Corporation in the U.S. and/or other countries. UNIX and The Open Group are trademarks of The Open Group in the U.S. and/or other countries.
Contents About This Technical Update 1 Overview 1.1 1.2 2 General Overview . . .. . .. . .. . . .. . .. . .. . .. . .. . . .. . .. . .. . .. . . .. . .. . .. . .. . AlphaServer ES45 Overview . . .. . .. . .. . .. . . .. . .. . .. . .. . . .. . .. . .. . .. . 1–1 1–1 Hardware Requirements and Restrictions 2.1 2.1.1 2.1.2 2.1.3 2.2 2.2.1 2.2.2 2.2.3 2.2.4 PCI Configuration . .. . .. . .. . . .. . .. . .. . .. . .. . . .. . .. . .. . .. . . .. . .. . .. . .. . PCI Configuration Guidelines . . .. . .. . .
Examples 3–1 3–2 3–3 3–4 Displaying Configuration on an AlphaServer ES45 in a Cluster Using a SCSI Bus for Shared Storage .. . .. . . .. . .. . .. . .. . Displaying Devices on an AlphaServer ES45 in a Cluster Using a SCSI bus for Shared Storage . .. . . .. . .. . .. . .. . . .. . .. . .. . .. . Displaying Configuration on an AlphaServer ES45 in a Cluster Using Fibre Channel for Shared Storage .. . . .. . .. . .. . .. .
About This Technical Update This technical update adds important information to the TruCluster™ Server Cluster Hardware Configuration manual about using the AlphaServer™ ES45 with the TruCluster Server Version 5.1A software product. Audience This technical update is for system administrators who plan to use an AlphaServer ES45 in a TruCluster Server configuration. New and Changed Features The following change has been made to this technical update since the 1 May 2002 edition: • Section 2.2.
information, and provides procedures for removing and replacing field replaceable units (FRU). • AlphaServer ES45 Rackmount Installation Guide — Provides procedures for installing an AlphaServer ES45 rackmount system into a cabinet. • ES45 QuickSpecs — Describes the AlphaServer ES45 system. You can find the latest version of the QuickSpecs at: http://www.compaq.com/products/quickspecs/productbulletin.html Select U.S.
1 Overview This technical update to the TruCluster Server Cluster Hardware Configuration manual provides important information about support for the AlphaServer ES45 with the TruCluster Server Version 5.1A product. 1.1 General Overview The AlphaServer ES45 was recently qualified for use with the TruCluster Server Version 5.1A product.
2 Hardware Requirements and Restrictions This chapter describes the hardware requirements and restrictions for configuring an AlphaServer ES45 system in a TruCluster Server Version 5.1A cluster. For other restrictions not listed here, see the hardware requirements and restrictions chapter of Cluster Hardware Configuration. 2.1 PCI Configuration The AlphaServer ES45 Model 2 supports a 10-slot PCI backplane. It consists of both 3.3V 66-MHz and 5.0V 33-MHz slots over four PCI buses (hoses).
• PCI modules designed for 5.0V only operate in 5.0V slots (slots 3, 6, 9, and 10). • PCI modules designed for 3.3V only operate in 3.3V slots (slots 1, 2, 4, 5, 7, and 8). • Universal PCI modules can operate in either 3.3V or 5.0V slots. • A universal PCI module that operates at 66 MHz will only operate at 33 MHz if it is plugged into a 5.0V slot. • A universal PCI module that operates at 33 MHz will cause the entire 66-MHz bus to operate at 33 MHz if it is plugged into a 3.3V slot. 2.1.
Table 2–2: ES45 PCI Modules Supported in a TruCluster Server V5.1A Configuration for Cluster Operations (cont.) e If you install a universal 66 Mhz PCI host bus adapter in a 5.0V 33 MHz slot (slots 3, 6, 9, or 10), the module will only operate at 33 MHz. f The DEGPA-TA Gigabit Ethernet adapter cannot be installed in PCI bus 1. 2.1.3 Differences in PCI Edge Connectors The differences in PCI slots and module edge connectors prevent you from installing a 5.0V module into a 3.3V slot.
– Fibre Channel: Eight-member systems may be connected to common storage over Fibre Channel in a fabric (switch) configuration. – Parallel SCSI: Only four of the member systems may be connected to any one SCSI bus, but you can have multiple SCSI buses connected to different sets of nodes, and the sets of nodes may overlap. We recommend you use a DS-DWZZH-05 UltraSCSI hub with fair arbitration enabled when connecting four-member systems to a common SCSI bus using RAID array controllers.
• The DSGGA, DSGGB, and DSGGC Fibre Channel switches support both shortwave (GBIC-SW) and longwave (GBIC-LW) Gigabit Interface Converter (GBIC) modules. Seven of the eight DSGGC-AA ports are fixed shortwave optical transceivers. Only one DSGGC-AA port is configured as a removable GBIC. It may be shortwave or longwave. The GBIC-SW module supports 50-micron, multimode fiber cables with the standard subscriber connector (SC) connector in lengths up to 500 meters (1640.4 feet). It also supports 62.
startup error message on the ES45 jumpered for 128 MB states: panic: MC2 adapter has too little memory See Section 3.5 if you intend to add an ES45 system with dual Memory Channel adapters jumpered for 128 MB into an existing cluster with Memory Channel adapters jumpered for 512 MB. • The Memory Channel API is not supported for data transfers larger than 8 KB when loopback mode is enabled in two member clusters configured in virtual hub mode. See the TruCluster Server Version 5.
31-meter (101.7-foot) (BN34R-31) lengths. Customers may provide their own fiber-optic cables to achieve greater separation of systems. • Always examine a Memory Channel link cable for bent or broken pins. Be sure that you do not bend or break any pins when you connect or disconnect a cable. 2.2.
3 Installing an AlphaServer ES45 into a TruCluster Server Configuration 3.1 Preparing the AlphaServer ES45 for a TruCluster Server Configuration The AlphaServer ES45 has PCI slot restrictions that must be adhered to, but otherwise, creating a cluster of ES45 systems, or adding an ES45 system to an existing cluster, is basically the same as any AlphaServer system supported in a TruCluster Server configuration.
up a Memory Channel cluster interconnect in Cluster Hardware Configuration. • To install and configure a local area network as the cluster interconnect, see the Cluster LAN Interconnect manual. ______________________ Note _______________________ You may save time if you install host bus adapters for shared storage, cluster interconnect cards (either Memory Channel or Ethernet cards for a private LAN), and extra Ethernet cards all at the same time. 3.
Figure 3–1: Fibre Channel HSG80 NSPOF Configuration Member System 1 Cluster Interconnects Member System 2 KGPSA KGPSA KGPSA KGPSA Fibre Channel Switch Port 1 Port 1 Fibre Channel Switch HSG 80 Controller A HSG 80 Controller B Port 2 Port 2 RA8000/ESA12000 ZK-1707U-AI 3.3.
Figure 3–2: TruCluster Server Configuration with UltraSCSI Components Networks Member System 2 Member System 1 Cluster Interconnects T KZPBA-CB (ID 6) T KZPBA-CB (ID 7) KZPBA-CB (ID 6) T 1 1 T KZPBA-CB (ID 7) 1 1 T T DS-DWZZH-03 T DS-DWZZH-03 T T T 2 3 Port 1 4 4 T T Port 2 2 3 Port 1 Controller A HSZ80 Port 2 Controller B HSZ80 StorageWorks RAID Array 8000 ZK-1602U-AI Table 3–1 lists the components used to create the cluster shown in Figure 3–2.
3.3.3 TruCluster Server Configuration Using Non-UltraSCSI Devices for Storage If your shared storage consists of disks in BA350, BA356, or UltraSCSI BA356 storage units, or non-UltraSCSI RAID array controllers (HSZ40 and HSZ50) connected to a KZPBA-CB UltraSCSI host bus adapter on a SCSI bus using external termination, see Section 3.4.
Figure 3–3: TruCluster Server Configuration with Non-UltraSCSI Devices Used for Storage Network Member System 1 Member System 2 Cluster Interconnect KZPSA-BB (ID 6) T KZPSA-BB (ID 7) 2 T 2 1 1 3 3 BA356 BA356 3 4 Do not use for data disk. May be used for redundant power supply.
Table 3–2: Hardware Components Used for Configuration Shown in Figure 3–3 Callout Number Description 1 BN21W-0B Y cable 2 H879-AA terminator 3 BN21K, BN21L, or 328215-00X cablea 4 H885-AA trilink connector a The maximum combined length of the BN21K, BN21L, or 328215-00X cables must not exceed 25 meters (82 feet). 3.4 Displaying Adapters with the show Console Commands The Cluster Hardware Configuration manual does not have any examples that include the AlphaServer ES45.
Example 3–1: Displaying Configuration on an AlphaServer ES45 in a Cluster Using a SCSI Bus for Shared Storage (cont.) Cchip Dchip PPchip 0 PPchip 1 TIG Memory Array --------0 1 2 3 Rev Rev Rev Rev Rev 17 17 17 17 2.
Example 3–1: Displaying Configuration on an AlphaServer ES45 in a Cluster Using a SCSI Bus for Shared Storage (cont.) Slot 2 6 Option Hose 1, Bus 0, PCI - 33 MHz DEFPA fwa0.0.0.2.1 00-60-6D-93-A0-1A Yukon PCI Hot-Plug C Slot 1/0 Option Adaptec AIC-7899 1/1 Adaptec AIC-7899 Slot 6 P00>>> Option Hose 2, Bus 0, PCI - 66 MHz 4 pkc0.7.0.1.2 SCSI Bus ID 7 dkc0.0.0.1.2 COMPAQ BD036635C5 dkc100.1.0.1.2 COMPAQ BD036635C5 dkc200.2.0.1.2 COMPAQ BD0186349B dkc300.3.0.1.2 COMPAQ BD0096349A dkc400.4.0.1.
Example 3–2: Displaying Devices on an AlphaServer ES45 in a Cluster Using a SCSI bus for Shared Storage P00>>> show device dka0.0.0.8.0 dka100.1.0.8.0 dkb0.0.0.10.0 dkb1.0.0.10.0 dkb100.1.0.10.0 dkb101.1.0.10.0 . . . dkb205.2.0.10.0 dkb206.2.0.10.0 dkb207.2.0.10.0 dkb3.0.0.10.0 dkb300.3.0.10.0 dkb4.0.0.10.0 dkb5.0.0.10.0 dkb6.0.0.10.0 dkb7.0.0.10.0 dkc0.0.0.1.2 dkc100.1.0.1.2 dkc200.2.0.1.2 dkc300.3.0.1.2 dkc400.4.0.1.2 dkc500.5.0.1.2 dkd0.0.0.101.2 dkd100.1.0.101.2 dkd200.2.0.101.2 dkd300.3.0.101.2 dkd400.
Example 3–3: Displaying Configuration on an AlphaServer ES45 in a Cluster Using Fibre Channel for Shared Storage P00>>> show config Compaq Computer Corporation Compaq AlphaServer ES45 Model 2 Firmware SRM Console: PALcode: T6.1-5 OpenVMS PALcode V1.93-36 Tru64 UNIX PALcode V1.88-28 Serial ROM: V2.18-F RMC ROM: V1.0 RMC Flash ROM: T1.
Example 3–3: Displaying Configuration on an AlphaServer ES45 in a Cluster Using Fibre Channel for Shared Storage (cont.) Option Floppy Hose 0, Bus 1, ISA dva0.0.0.1000.0 Slot 2 6 Option Hose 1, Bus 0, PCI - 33 MHz DEFPA fwa0.0.0.2.1 00-60-6D-93-A0-1A Yukon PCI Hot-Plug C Slot 1/0 Option Adaptec AIC-7899 1/1 Adaptec AIC-7899 Slot 1 2 6 P00>>> Hose 2, Bus 0, PCI - 66 MHz 3 pkb0.7.0.1.2 SCSI Bus ID 7 dkb0.0.0.1.2 COMPAQ BD036635C5 dkb100.1.0.1.2 COMPAQ BD036635C5 dkb200.2.0.1.
Example 3–4 shows the output from the show device console command entered on an AlphaServer ES45 system in a cluster using Fibre Channel for shared storage. Example 3–4: Displaying Devices on an AlphaServer ES45 in a Cluster Using Fibre Channel for Shared Storage P00>>> show device P00>>>show device dgb42.1001.0.2.3 dgb42.1002.0.2.3 dgb42.1003.0.2.3 dgb42.1004.0.2.3 dka0.0.0.8.0 dka100.1.0.8.0 dkb0.0.0.1.2 dkb100.1.0.1.2 dkb200.2.0.1.2 dkb300.3.0.1.2 dkb400.4.0.1.2 dkb500.5.0.1.2 dkc0.0.0.101.2 dkc100.1.0.
To add an ES45 with dual Memory Channel adapters into an existing cluster with Memory Channel adapters jumpered for 512 MB, follow this procedure: 1. Shut down all cluster member systems except one. ____________________ Notes ____________________ Do not shut down a cluster member that has an application with restricted placement policy unless the member that will not be shut down is on the hosting list for the application. See the Cluster Highly Available Applications manual for more information.
jumpers on the other cluster member systems, the ES45 will not boot into the cluster. A system jumpered for 128 MB cannot join the cluster. You would receive a startup error message on the system jumpered for 128 MB: panic: MC2 adapter has too little memory 9. Boot the ES45 system into the cluster. ____________________ Note _____________________ Readjust quorum votes and reset the placement policies as needed.
