Installation Guide, Server Expansion Unit for the HP Integrity rx8640 HP Part Number: AB301-9008A Published: September 2007 Edition: 3
Legal Notices © Copyright 2006 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Microsoft and Windows are U.S.
Table of Contents About this Document..........................................................................................................9 Book Layout............................................................................................................................................9 Intended Audience.................................................................................................................................9 Printing History..................................................
Voltage Check (Additional Procedure)...........................................................................................48 Connecting AC Input Power...........................................................................................................49 Installing The Line Cord Anchor (rack mounted servers)..............................................................51 IO Expander Installation........................................................................................................
List of Figures 1-1 1-2 1-3 1-4 1-5 1-6 2-1 2-2 2-3 2-4 2-5 2-6 2-7 2-8 2-9 2-10 2-11 2-12 2-13 2-14 2-15 2-16 2-17 2-18 2-19 2-20 2-21 2-22 2-23 2-24 2-25 2-26 2-27 2-28 2-29 SEU (Front View)...........................................................................................................................11 SEU (Without Front Bezel)............................................................................................................12 Internal Disks...........................................
List of Tables 1-1 1-2 1-3 1-4 1-5 1-6 1-7 1-8 2-1 2-2 SEU Core IO Boot Path..................................................................................................................13 Cell 2 PCI Slot Boot Paths..............................................................................................................15 Cell 3 PCI Slot Boot Paths..............................................................................................................15 Removable DVD Media Drive Path..........
About this Document This document covers the new HP Server Expansion Unit. This document does not describe system software or partition configuration operations in any detail. For information concerning those topics refer to the HP System Partitions Guide.
Books about HP-UX Published by Prentice Hall The http://www.hp.com/hpbooks/ Web site lists the HP books that Prentice Hall currently publishes, such as HP-UX books including: • HP-UX 11i System Administration Handbook http://www.hp.com/hpbooks/prentice/ptr_0130600814.html • HP-UX Virtual Partitions http://www.hp.com/hpbooks/prentice/ptr_0130352128.html HP Books are available worldwide through bookstores, online booksellers, and office and computer stores.
1 Server Expansion Unit Overview The Server Expansion Unit (SEU) is a member of the HP business-critical computing platform family: a 9U tall, mid-range, mid-volume IO expansion cabinet. It provides additional IO performance with the inclusion of 16 PCI-X slots or a combination of 8 PCI-X slots and 8 PCIe slots, two additional core IO cards, four more hard disk drives, and two removable media drives. The SEU connects to the HP Integrity rx8640 server, providing additional IO slots to the server.
Figure 1-2 SEU (Without Front Bezel) Features include: • • • • • • • • • • • Better availability and up time than previous SEU Depth optimized (shallower, fewer racking issues) Increased performance density Increased PCI-X performance Internal removable media Up to four additional internal disks Dual power cords enable N+1 redundancy on AC input Up to sixteen additional PCI-X slots Combination of eight additional PCI-X slots and eight additional PCIe slots (with PCI-X/PCIe backplane).
contains clock generation circuits, manageability circuits, DC-to-DC converters, power monitor logic, and fan control. Core IO Backplane Module The core IO backplane is housed in a separate sheet metal module holding the backplane and the core IO boards. This module can be removed from the rear of the chassis for easy access to the core IO backplane. The module adapts the core IO sockets in a horizontal mounting position to the system backplane, which is mounted vertically.
Table 1-1 SEU Core IO Boot Path (continued) Core IO card Device Path Description 3 1 Gb LAN 3/0/0/1/0 The SYS LAN connector located on core IO 3. 3 SCSI drive 3/0/0/2/0.6.0 Hard drive located in the bottom left disk bay. 3 SCSI drive 3/0/0/2/1.X.0 Removable media DVD (X = 2) or DDS-4 (X = 3) tape drive located in the upper disk bay. 3 SCSI drive 3/0/0/3/0.6.0 Hard drive located in the bottom right disk bay.
NOTE: Fans driven to a high RPM in dense air cannot maintain expected RPM and will be considered bad by the MP leading to a “False Fan Failure” condition. PCI-X Slot Boot Paths Table 1-3 and Table 1-3 list the mapping of PCI-X slots to ropes and boot paths. Pathing will have to be modified for PCI cards that have different devices and functions.
primary IO system interface. Upstream, the IO chips communicate directly with the cell controller ASIC on the host cell board via a high bandwidth logical connection known as the HSS link. When installed in the SEU chassis within a fully configured system, The ASIC on cell location 0 connects to the cell controller chip on cell board 2 and the ASIC on cell location 1 connects to the cell controller chip on cell board 3 through external link cables.
Table 1-4 Removable DVD Media Drive Path DVD Media Path Slot 0 Media 2/0/0/2/1.2.0 Slot 1 Media 3/0/0/2/1.2.0 Table 1-5 Removable DDS-4 Media Drive Path DDS-4 Media Path Slot 0 Media 2/0/0/2/1.3.0 Slot 1 Media 3/0/0/2/1.3.0 Table 1-6 Hard Disk Drive Path Hard Drive Path Slot 0 Drive 2/0/0/2/0.6.0 Slot 1 Drive 2/0/0/3/0.6.0 Slot 2 Drive 3/0/0/2/0.6.0 Slot 3 Drive 3/0/0/3/0.6.0 System Backplane The SEU system backplane provides inter-connection between the PCI backplane and the core IO.
Figure 1-4 System Backplane (Side View in Chassis) The voltage regulator modules (VRMs) that provide 3.3 V and 3.3 V standby are located on this backplane. The backplane also has the connectors to provide a hardware interface to the SEU chassis. Clocks and Reset The system backplane contains reset and clock circuitry that propagates through the whole system. The system backplane central clocks drive all major chip set clocks.
slots are keyed for 3.3V connectors (accepting both Universal and 3.3V cards). The PCI-X backplane does not provide any 5V slots for the IO cards. The external link for the SEU attaches the cell controller on cell board 2 to SBA 0 and attaches the cell controller on cell board 3 to SBA 1 of the PCI-X board in the SEU cabinet. The PCI-X backplane is physically one board but behaves like two independent partitions. SBA 0 and its associated LBAs and eight PCI-X slots form one IO partition.
Table 1-8 PCI-X/PCIe IO Backplane Slot Types IO Partition 0 1 1 2 Slot1 Maximum Speed MHz Ropes Supported Cards PCI Mode Supported 82 66 001 3.3 V PCI or PCI-X Mode 1 7 133 002/003 3.3 V PCI or PCI-X Mode 1 6 266 004/005 3.3 V PCIe 5 266 006/007 3.3 V PCIe 4 266 014/015 3.3 V PCIe 3 266 012/013 3.3 V PCIe 2 133 010/011 3.3 V PCI or PCI-X Mode 1 1 133 008/009 3.3 V PCI or PCI-X Mode 1 82 66 001 3.3 V PCI or PCI-X Mode 1 7 133 002/003 3.
SEU Description Dimensions The dimensions of the SEU are as follows: • • Width: 17.5 inches, constrained by electronic industries alliance (EIA) standard 19-inch racks. Depth: Defined by cable management constraints to fit into a standard 36-inch deep rack: * 25.5 inches from front rack column to PCI connector surface. * 26.7 inches from front rack column to core IO connector surface. * 30 inches overall package dimension, including 2.7 inches protruding in front of the front rack columns.
The PCI on-line replacement (OLR) fan modules are located in front of the PCI cards. These six fans are housed in plastic carriers. They are configured in two rows of three fans. Four OLR system fan modules, externally attached to the chassis, are 120-mm (4.7-inch) fans. Two fans are mounted on the front surface of the chassis and two are mounted on the rear surface.A cable harness that connects from the rear of the BPSs to the system backplane provides DC power distribution.
2 Installing the Server Expansion Unit The following instructions are included for unpacking a racked SEU. There are also instructions for unpacking a non-racked SEU. Inspect shipping containers when the equipment arrives at the site. Check equipment after the packing has been removed. This chapter discusses how to inspect and receive the SEU. Inspecting the Server Cabinet NOTE: The server will ship in one of three different configurations.
If the equipment has any damage, a damage claim form must be obtained by the customer from the shipping representative. The customer should complete the form and return it to the shipping representative. NOTE: The SEU may come already racked or ready for rack installation. Unpacking a Racked SEU This section contains information about unpacking the cabinet. WARNING! Wear protective glasses while cutting the plastic bands around the shipping container. These bands are under tension.
4. Remove the packing materials. CAUTION: The plastic wrapping material should be cut off rather than pulled off. Pulling the plastic covering off represents an electrostatic discharge (ESD) hazard. Figure 2-1 Removing the Polystraps and Cardboard 5. Remove the four bolts securing the ramps to the pallet and remove the ramps. See Figure 2-2.
Figure 2-2 Removing the Shipping Bolts and Plastic Cover 26 Installing the Server Expansion Unit
6. Remove the six bolts from the base attaching the rack to the pallet. See Figure 2-3. Figure 2-3 Preparing to Roll Off the Pallet WARNING! Be sure that the leveling feet on the rack are raised before you roll the rack down the ramp and any time you roll the rack on the casters. Use caution when rolling the cabinet off the ramp. A single server in the cabinet weighs several hundred pounds. HP strongly recommends that two people roll the cabinet off the pallet.
Unloading with a Lifter Use the following procedure to unload the SEU from the pallet using a lifter. WARNING! Use caution when using a lifter. Because of the weight of the SEU, it must be centered on the lifter forks before raising it off the pallet to avoid injury. Never extend more than one server from the same cabinet while installing or servicing either an SEU or another server product. Failure to follow these instructions could result in the cabinet tipping over.
IMPORTANT: The SEU must be mounted in the same rack as the server to which it will be connected. The unit must also be mounted directly above the server in the same rack cabinet. Any SEU installed into a rack is shipped with equipment slides. With every set of slides comes an installation guide: installation guide, hp J1530B, rack integration kit (lower case intended). Follow the steps in this installation guide to determine where and how to place the SEU into the rack before proceeding with Step 1.
7. Roll the lifter and SEU away from the pallet. Do not raise the SEU any higher than necessary when moving it over to the rack. Figure 2-7 Lifting the SEU into Position for Rack Installation Installing the Cable Management Arm After the SEU is installed in the rack, the cable management arm (CMA) must be installed on the rear of the SEU. Follow the instructions for installing the CMA in the installation guide, hp J1530B, rack integration kit (lower case intended).
System damage can occur through improper removal and re-installation of devices. This task must be performed by trained personnel only. Instructions for removing and re-installing these components can be found in the Removal & Replacement chapter of the Service Guide. CAUTION: Observe all ESD safety precautions before attempting this procedure. Failure to follow ESD safety precautions could result in damage to the server. Any hp SEU installed into a rack is shipped with equipment slides.
Figure 2-8 Embedded Disks Hard Disk Drive Installation The disk drives are located in the front of the chassis. The hard disk drives are hot-plug drives. CAUTION: Observe all ESD safety precautions before attempting this procedure. Failure to follow ESD safety precautions could result in damage to the SEU. 1. 2. 3. 4. Be sure the front locking latch is open, then position the disk drive in the chassis. Slide the disk drive into the chassis; a slow, firm pressure is needed to properly seat the connector.
Table 2-1 covers cards supported in the SEU when connected to the respective server shown in the table. Refer to the Installation Guide for the server to obtain the number and type of PCI/PCI-X/PCIe cards supported in a server.
Table 2-1 HP Integrity rx8640 SEU PCI-X/PCIe IO Cards (continued) Part Number Card Description AB429A1 1-Port 4Gb FC QLogic – AB378A equivalent AB465A1 2-port 1000b-T 2Gb FC Combo AB466A1 Emulex 1050DC Fibre Channel B AB467A1 Emulex 1050D Fibre Channel B AB545A 4-Port 1000b-T Ethernet 1 HP-UX Windows Linux B B Bb VMS B b AD167A Emulex 4Gb/s B B AD168A1 Emulex 4Gb/s DC B B AD278A 8-Port Terminal MUX AD279A 64-Port Terminal MUX J3525A 2-port Serial 337972-B21 SA P600 (Red
PCI IO Card Installation CAUTION: While the installation process for PCI/PCI-X cards and PCIe cards is the same; PCIe cards are physically smaller than PCI-X cards and are not interchangeable with PCI/PCI-X cards. Check Table 1-8 (page 20) to verify the slot types and order. NOTE: The PCI IO card installation process varies depending on which version of the HP-UX operating system you are running on your system. PCI IO card installation procedures should be downloaded from the http://docs.hp.com/ Web site.
NOTE: The Server Expansion Unit implements manual retention latch (MRL) hardware for use in online add or replacement (OLAR) operations. If an MRL is left open while the server is booting, HP-UX can incorrectly cache PCI slot power status causing OLAR operations to fail. To prevent this situation, ensure all the MRLs are closed before booting the server.
HP Integrity rx8640 Server E-Link Cable Installation NOTE: server. This section describes the installation of the E-link cables between the SEU and the INSTALLATION NOTES BEFORE GETTING STARTED • • • • • • • Before attaching the SEU, the server must be shut down. Schedule this with the customer prior to beginning the SEU installation. The SEU is only supported with servers containing a minimum of three cell boards. The cell boards must be loaded in the server in cell board slots 0, 1, and 2.
Type Control-b (^B) to exit the VFP display. • If an nPartition has booted HP-UX, or if it is in the process of launching HP-UX, you must shut down HP-UX on the nPartition. When HP-UX is running on an nPartition, its VFP displays “HP-UX heartbeat” with a blinking asterisk (*) to indicate its interactivity. In this case, proceed with the next step. • • If the nPartition is at its extensible firmware interface (EFI) shell, then HP-UX has already been shut down.
Figure 2-10 Server Top Cover Removal Removing the Server Top Cover 1. 2. 3. 4. 5. Connect to ground with a wrist strap. Loosen the blue retaining screws securing the cover to the chassis. Slide the cover toward the rear of the chassis. Lift the cover up and away from the chassis. Place the cover in a safe location.
Side Cover Removal Figure 2-11 Side Cover Removal 1. 2. 3. Loosen the blue retaining screw securing the side cover to the chassis. (See Figure 2-11.) The screw is located at the bottom of the server chassis. Slide the cover from the chassis toward the rear of the system. Place the cover in a safe location. After removing the side cover, the system backplane is visible.
NOTE: The term REO on the REO Cable Instructions label has the same meaning as the term SBA. 5. 6. 7. Snap each cable under the cable clamps provided on the chassis frame. For cable to cable clamp positioning, refer to Figure 2-14 (page 43). Push the cable on the connector and secure the cable attached to the shroud by tightening the captive screw attached to the cable. Complete the procedure by securing the remaining three E-link cables in place with the captive screw located on each cable.
Figure 2-13 E-Link Cable Bracket Secured to SEU Backplane 8. 42 Reinstall the top and side covers.
Figure 2-14 E-Link Cable Connection (Server to Server Expansion Unit) E-Link Cable Routing for SEU Installed in Same Cabinet as the Server The E-link cable bundle must be routed on the right side of the cabinet using the path shown in Figure 2-15 when the SEU is installed in the same rack cabinet with the server.
Figure 2-15 E-Link Cable Routing (Rear of Cabinet) E-Link Cable Routing for SEU Installed in Adjacent Cabinet to the Server The E-link cable bundle must be routed using the path shown in Figure 2-16 when the SEU is installed in a rack cabinet adjacent to the server.
Figure 2-16 E-Link Cable Routing (Rear of Cabinets) • When the SEU is installed in an adjacent rack to the server, adhere to the following guidelines for installation: — Two SEUs can be installed in a single rack cabinet. — The racks must be either 41U or 33U. A 25U rack is not permitted for SEU installation. — The rack cabinets must be installed with proper anti-tip feet, system interlock rods, and ballast.
Figure 2-17 Cabinet Label Placement Voltage Check This section provides voltage check information for use on the customer site. The emphasis focuses on measuring the voltages at the power cord plug end specified as an IEC-320 C19 type plug. This is the end that plugs directly into the back of the SEU chassis.
NOTE: These procedures must be performed for each power cord that will be plugged directly into the back of the SEU. If the expected results from this procedure are not observed during the voltage check, refer to the next section titled “Voltage Check (Additional Procedure).” Voltage Range Verification of Receptacle This measures the voltage between L1 and L2, L1 to ground, and L2 to ground. Three separate measurements are performed during this procedure.
Safety Ground Verification This measures the voltage level between B0 and A0. The measurement is taken between ground pins of the two power cords. Refer to Figure 2-19 for ground reference points when performing this measurement. Figure 2-19 Safety Ground Reference Check 1. 2. 3. 4. Measure the voltage between B0 and A0. Take the AC voltage down to the lowest scale on the volt meter. Insert one probe into the ground pin for B0. Insert the other probe into the ground pin for A0.
Figure 2-20 Wall Receptacle Pinouts WARNING! There is a risk of shock hazard while testing primary power. Use properly insulated probes. 5. 6. 7. 8. Verify that the voltage between receptacle pins X and Y is between 200–240V AC. Set the site power circuit breaker to OFF. Ensure that power is removed from the server. Route and connect the server power connector to the site power receptacle. • For locking type receptacles, line up the key on the plug with the groove in the receptacle.
Figure 2-21 Power Cord Receptacle Configuration Power cord receptacles are identified using a label at the rear side of the chassis indicating which receptacle is B0 and which receptacle is A0. The SEU follows the power configuration that is selected for the server. So, if grid A0 & B0 are selected for the server, the SEU needs to have both power cords connected. Otherwise, power fault messages are logged. You cannot select one power configuration for the SEU and another power configuration for the server.
Figure 2-22 Power Source versus Power Distribution WARNING! Voltage is present at various locations within the SEU whenever a power source is connected. This voltage is present even when the main power switch is in the OFF position. To completely remove power, both power cords must be removed from the SEU. Failure to observe this warning could result in personal injury or damage to equipment. 1. 2. 3.
Figure 2-23 IO Expander Line Cord Anchor MP Core IO Connections Each SEU has two core IO cards installed and each core IO card has a management processor (MP). This enables two partitions to operate, or enables MP core IO redundancy in a single partition configuration. Each MP core IO board is oriented horizontally and accessed from the back of the server. IMPORTANT: The MP LAN, local and remote console ports are never functional for a core IO card installed in the SEU.
• • • • One Ultra 320 (320Mb/second) 68-pin SCSI port—A connection for external SCSI devices (by way of a VHDCI connector). VHDCI is an industry-standard term that stands for Very High-Density Cable Interconnect. One RJ-45 style 10Base-T/100Base-T/1000Base-T system LAN connector—This LAN uses standby power and is active when AC is present and the front panel power switch is off. One RJ-45 style 10Base-T/100Base-T MP LAN connector—This LAN port is not functional when the core IO is installed in the SEU.
Standby Power and Logging in to the MP Housekeeping power (also known as standby power) is generated as soon as AC power is applied to the server. Because the MP uses standby power, it is possible to login to the server MP even when the power switch is in the OFF position. The power switch is a DC power switch that controls +48V DC. NOTE: 1. Connection to the MP is performed through the core IO card installed in the server.
Configuring LAN Information for the MP To set the server MP LAN IP address: 1. At the server MP Main Menu prompt (MP>), enter cm.From the MP Command Menu prompt (MP:CM>), enter lc (for LAN configuration). The screen lists the default values and asks if you want to modify them. It is a good idea to write down the information, as it may be required for future troubleshooting. NOTE: If the Command Menu is not shown, enter q to return to the MP Main Menu, then enter cm. Enter lc and press the Return key.
11. A screen similar to the following appears allowing verification of the settings: Figure 2-28 The ls Command Screen To return to the server MP main menu, enter ma. To exit the server MP, enter x at the MP main menu. Examining the MP Bus Devices To determine what is seen by the MP in the system: 1. At the server MP prompt, enter cm. The Command Menu is displayed. With the Command Menu, you can view or modify the configuration and look at utilities controlled by the MP.
Figure 2-29 The pwrgrd Command Screen 3. 4. Verify that the power grid configuration is correct by examining the output from the pwrgrd command. The previous power configuration indicates that only grid A has been configured. To change the configuration, select the proper response and enter the appropriate numeric value when Select Option: appears on the screen. If no change is desired, enter q and press the Enter key.
Index B backplane, 17 mass storage, 20 PCI, 12, 18 PCI-X, 14 PCI-X/PCIe, 15 system, 12, 17, 18, 20, 22 BPS (Bulk Power Supply), 54 C cable management arm, 30 cell board, 17, 18, 21, 53 cm (Command Menu) command, 56 commands cm (Command Menu), 56 du (display MP bus topology), 56 lc (LAN configuration), 55 ls (LAN status), 56 pwrgrd (Power Grid), 56 configuring LAN information, 55 connecting AC, 49 core IO, 17, 18, 20 D disk internal, 11, 12, 16, 20, 31, 52 du (display MP bus topology) command, 56 F login
