Installing and Administering HyperFabric HP-UX 11i v1 and HP-UX 11i v2 Edition 13 Manufacturing Part Number: B6257-90060 October 2006 Printed in U.S.A. © Copyright 2006 Hewlett-Packard Company.
Legal Notices The information in this document is subject to change without notice. Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. Hewlett-Packard shall not be held liable for errors contained herein or direct, indirect, special, incidental or consequential damages in connection with the furnishing, performance, or use of this material. Warranty.
Contents 1. Overview Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HyperFabric Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HyperFabric Adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Switches and Switch Modules . . . . .
Contents With the Rail Kit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Installing the HF2 Switch With the Rail kit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Without the Rail Kit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 4. Configuring HyperFabric Configuration Overview. . . . . . . . . . . . . .
Contents Determining Whether an Adapter or a Cable is Faulty. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Determining Whether a Switch is Faulty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HF1 Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HF2 Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents 4
Tables HF1 Speed and Latency w/ TCP/IP Applications 30 HF2 Speed and Latency w/ TCP/IP Applications 30 Supported Configurations for A6386A HF2 Adapter On PCI (4X) 31 HF1 Speed and Latency w/ HMP Applications 41 HF2 Speed and Latency w/ HMP Applications 41 Supported Configurations for A6386A HF2 Adapter On PCI (4X) 42 Important OLAR Terms 56 LED Names (by Adapter) 135 HyperFabric Adapter LED Colors and Meanings 136 HF1 Switch LED Colors and Meanings 140 HF2 Switch LED Colors and Meanings 144 5
Tables 6
Figures TCP/IP Point-To-Point Configurations 33 TCP/IP Basic Switched Configuration 34 TCP/IP High Availability Switched Configuration 35 TCP/IP Hybrid Configuration 36 TCP/IP Mixed HF1 & HF2 Configuration 37 HMP Point-To-Point Configurations 44 HMP Enterprise (Database) Configuration, Single Connection Between Nodes 46 HMP Enterprise (Database) Configuration, Multiple Connections Between Nodes 47 Technical Computing Configuration 49 Large Technical Computing Configuration 50 HyperFabric File Structure 60
Figures 8
Printing History The manual printing date and part number indicate its current edition. The printing date will change when a new edition is printed. Minor changes may be made at reprint without changing the printing date. The manual part number will change when extensive changes are made. Manual updates may be issued between editions to correct errors or document product changes. To ensure that you receive the updated or new editions, you should subscribe to the appropriate product support service.
1 Overview This chapter contains the following sections that give general information about HyperFabric: Chapter 1 • “Overview” on page 15 • “HyperFabric Products” on page 16 13
Overview • 14 “HyperFabric Concepts” on page 19 Chapter 1
Overview Overview Overview HyperFabric is a Hewlett-Packard high-speed, packet-based interconnect for node-to-node communications. HyperFabric provides higher speed, lower network latency and less CPU usage than other industry standard protocols (e.g. Fibre Channel and Gigabit Ethernet). Instead of using a traditional bus based technology, HyperFabric is built around switched fabric architecture, providing the bandwidth necessary for high speed data transfer.
Overview HyperFabric Products HyperFabric Products HyperFabric hardware consists of host-based interface adapter cards, interconnect cables and optional switches. HyperFabric software resides in ASICs and firmware on the adapter cards and includes user space components and HP-UX drivers. Currently both copper and fibre based HyperFabric hardware is available. There is also a hybrid switch that has 8 fibre ports and 4 copper ports to support mixed HF1 and HF2 clusters.
Overview HyperFabric Products The A6092A HyperFabric adapter is supported beginning with the following HyperFabric software versions: • HP-UX 10.20: HyperFabric software version B.10.20.09 • HP-UX 11.0: HyperFabric software version B.11.00.09 • HP-UX 11i: HyperFabric software version B.11.11.00 The A6386A HyperFabric2 adapter is supported beginning with the following HyperFabric software versions: • HP-UX 11.0: HyperFabric software version B.11.00.11 • HP-UX 11i: HyperFabric software version B.11.
Overview HyperFabric Products To determine the version of HyperFabric you have, issue this command: swlist | grep -i hyperfabric Other Product Elements The other elements of the HyperFabric product family are the following: • A4892A HF1 copper cable (in 35-foot and 60-foot lengths).
Overview HyperFabric Concepts HyperFabric Concepts Some basic HyperFabric concepts and terms are briefly described below. The fabric is the physical configuration that consists of all of the HyperFabric adapters, the HyperFabric switches (if any are used) and the HyperFabric cables connecting them. The network software controls data transfer over the fabric. A HyperFabric configuration contains two or more HP 9000 systems and optional HyperFabric switches. Each HP 9000 acts as a node in the configuration.
Overview HyperFabric Concepts 20 Chapter 1
2 Planning the Fabric This chapter contains the following sections offering general guidelines and protocol specific considerations for planning HyperFabric clusters that will run TCP/IP or HMP applications.
Planning the Fabric 22 • “HyperFabric Functionality for TCP/IP and HMP Applications” on page 24 • “TCP / IP” on page 25 • “Hyper Messaging Protocol (HMP)” on page 37 Chapter 2
Planning the Fabric Preliminary Considerations Preliminary Considerations Before beginning to physically assemble a fabric, follow the steps below to be sure all appropriate issues have been considered: Step 1. Read Chapter 1, “Overview,” on page 13 to get a basic understanding of HyperFabric and its components. Step 2. Read this chapter, Planning the Fabric, to gain an understanding of protocol specific configuration guidelines for TCP/IP and HMP applications. Step 3.
Planning the Fabric HyperFabric Functionality for TCP/IP and HMP Applications HyperFabric Functionality for TCP/IP and HMP Applications The following sections in this chapter define HyperFabric features, parameters, and supported configurations for TCP/IP applications and Hyper Messaging Protocol (HMP) applications. There are distinct differences in supported hardware, available features and performance, depending on which protocol is used by applications running on the HyperFabric.
Planning the Fabric TCP / IP TCP / IP TCP/IP is supported on all HF1 (copper) and HF2 (fibre) hardware. Although some of the HyperFabric adapter cards support both HMP and TCP/IP applications, our focus in this section will be on TCP/IP HyperFabric applications. Application Availability All applications that use the TCP/IP stack are supported, including Oracle 9i and HP-MPI.
Planning the Fabric TCP / IP the package to another node in the cluster, allowing services to remain available with minimal interruption. MC/ServiceGuard via EMS, directly monitors cluster nodes, LAN interfaces, and services (the individual processes within an application). MC/ServiceGuard uses a heartbeat LAN to monitor the nodes in a cluster. It is not possible to use HyperFabric as a heartbeat LAN. Instead a separate LAN must be used for the heartbeat.
Planning the Fabric TCP / IP • Load Balancing: Supported When a HP 9000 HyperFabric cluster is running TCP/IP applications, the HyperFabric driver balances the load across all available resources in the cluster including nodes, adapter cards, links, and multiple links between switches. • Switch Management: Not Supported Switch Management is not supported. Switch management will not operate properly if it is enabled on a HyperFabric cluster.
Planning the Fabric TCP / IP • Maximum Cable Lengths: HF1 (copper): The maximum distance between two nodes or between a node and a switch is 60 ft. (2 standard cable lengths are sold and supported: 35 ft. and 60 ft.) TCP/IP supports up to four HF1 switches connected in series with a maximum cable length of 60 ft. between the switches and 60 ft. between switches and nodes. HF2 (fibre): The maximum distance is 200m (4 standard cable lengths are sold and supported: 2m, 16m, 50m and 200m).
Planning the Fabric TCP / IP Speed and Latency: Table 2-1 HF1 Speed and Latency w/ TCP/IP Applications Server Class rp7400 Table 2-2 1.28 + 1.
Planning the Fabric TCP / IP Table 2-3 Supported Configurations for A6386A HF2 Adapter On PCI (4X) Supported HP 9000 Systems 30 HP-UX Version OLAR Support? Maximum Adapters per System rp24xx (A400 and A500) 11.0, 11i v1, 11iv2 No 2 rp34xx Series 11i v1 and 11i v2 No 2 rp44xx Series 11i v1 and 11i v2 Yes 4 rp54xx Series (L Class Servers) 11.0, 11i v1 and 11i v2 Yes (11iv1 and later) 2 rp74x0 (N-Class Series) 11.
Planning the Fabric TCP / IP TCP/IP Supported Configurations Multiple TCP/IP HyperFabric configurations are supported to match the cost, scaling and performance requirements of each installation. In the previous “Configuration Guidelines” section the maximum limits for TCP/IP enabled HyperFabric hardware configurations were outlined. In this section the TCP/IP enabled HyperFabric configurations that HP supports will be detailed.
Planning the Fabric TCP / IP Figure 2-1 32 TCP/IP Point-To-Point Configurations Chapter 2
Planning the Fabric TCP / IP Switched This configuration offers the same benefits as the point to point configurations illustrated in figure 1, but it has the added advantage of greater connectivity (see Figure 2-2).
Planning the Fabric TCP / IP High Availability Switched This configuration has no single point of failure. The HyperFabric driver provides end to end HA. If any HyperFabric resource in the cluster fails, traffic will be transparently rerouted through other available resources. This configuration provides high performance and high availability (see Figure 2-3).
Planning the Fabric TCP / IP Hybrid Servers and workstations can be interconnected in a single heterogeneous HyperFabric cluster. In this configuration the servers are highly available. In addition, the workstations and the servers can be running the same application or different applications (see Figure 2-4).
Planning the Fabric TCP / IP Mixed HF1 / HF2 (Copper & fibre) All currently available HyperFabric products can be interconnected in a single HyperFabric cluster. The HF1 and HF2 products are interoperable enabling user controlled migration from copper based to fibre based technologies (see Figure 2-5).
Planning the Fabric Hyper Messaging Protocol (HMP) Hyper Messaging Protocol (HMP) Hyper Messaging protocol (HMP) is Hewlett-Packard’s patented, high performance cluster interconnect protocol. HMP provides reliable, high speed, low latency, low CPU overhead, datagram service to applications running on HP-UX platforms. HMP was jointly developed with Oracle Corp. The resulting feature set was tuned to enhance the scalability of the Oracle Cache Fusion clustering technology.
Planning the Fabric Hyper Messaging Protocol (HMP) Starting with the December 2000 releases B.11.00.11 and B.11.11.01, the HyperFabric EMS monitor allows the system administrator to separately monitor each HyperFabric adapter on every node in the fabric, in addition to monitoring the entire HyperFabric subsystem. The monitor can inform the user if the resource being monitored is UP or DOWN. The administrator defines the condition to trigger a notification (usually a change in interface status).
Planning the Fabric Hyper Messaging Protocol (HMP) • Switch Management: Not Supported Switch Management is not supported. Switch management will not operate properly if it is enabled on a HyperFabric cluster. • Diagnostics: Supported Diagnostics can be run to obtain information on many of the HyperFabric components via the clic_diag, clic_probe and clic_stat commands, as well as the Support Tools Manager (STM).
Planning the Fabric Hyper Messaging Protocol (HMP) Instead, all of the traffic will be sent over a single connection with no failover capability and without the performance increase that would come from balancing the load over multiple connections. • Maximum Cable Lengths: HF1 (copper): The maximum distance between two nodes or between a node and a switch is 60 ft. (2 standard cable lengths are sold and supported: 35 ft. and 60 ft.
Planning the Fabric Hyper Messaging Protocol (HMP) Table 2-6 Supported Configurations for A6386A HF2 Adapter On PCI (4X) Supported HP 9000 Systems Chapter 2 HP-UX Version OLAR Support? Maximum Adapters per System rp24xx (A400 and A500) 11.0, 11i v1, 11iv2 No 2 rp34xx Series 11i v1 and 11i v2 No 2 rp44xx Series 11i v1 and 11i v2 Yes 4 rp54xx Series (L Class Servers) 11.0, 11i v1 and 11i v2 Yes (11iv1 and later) 8 (maximum 4 per PCI card cage) rp74x0 (N-Class Series) 11.
Planning the Fabric Hyper Messaging Protocol (HMP) HMP Supported Configurations Multiple HMP HyperFabric configurations are supported to match the performance, cost and scaling requirements of each installation. In the previous “Configuration Guidelines” section, the maximum limits for HMP enabled HyperFabric hardware configurations were outlined. In this section, the HMP enabled HyperFabric configurations that HP supports will be detailed.
Planning the Fabric Hyper Messaging Protocol (HMP) Figure 2-6 Chapter 2 HMP Point-To-Point Configurations 43
Planning the Fabric Hyper Messaging Protocol (HMP) Enterprise (Database) The HMP enterprise configuration illustrated in Figure 2-7 is very popular for running Oracle RAC 9i. Superdomes or other large servers make up the Database Tier. Database Tier nodes communicate with each other using HMP. Application Tier nodes communicate with each other and to the Database Tier using TCP/IP. The HMP enterprise configuration is a scalable solution.
Planning the Fabric Hyper Messaging Protocol (HMP) Figure 2-7 Chapter 2 HMP Enterprise (Database) Configuration, Single Connection Between Nodes 45
Planning the Fabric Hyper Messaging Protocol (HMP) Figure 2-8 46 HMP Enterprise (Database) Configuration, Multiple Connections Between Nodes Chapter 2
Planning the Fabric Hyper Messaging Protocol (HMP) Technical Computing (Work Stations) This configuration is typically used to run technical computing applications with HP-MPI. A large number of small nodes are interconnected to achieve high throughput (see Figure 2-9). High availability is not usually a requirement in technical computing environments. HMP provides the high performance, low latency path necessary for these technical computing applications.
Planning the Fabric Hyper Messaging Protocol (HMP) Figure 2-9 48 Technical Computing Configuration Chapter 2
Planning the Fabric Hyper Messaging Protocol (HMP) Figure 2-10 Chapter 2 Large Technical Computing Configuration 49
Planning the Fabric Hyper Messaging Protocol (HMP) 50 Chapter 2
3 Installing HyperFabric This chapter contains the following sections that describe installing HyperFabric: Chapter 3 • “Checking HyperFabric Installation Prerequisites” on page 53 • “Installing HyperFabric Adapters” on page 54 • “Installing the Software” on page 60 51
Installing HyperFabric • 52 “Installing HyperFabric Switches” on page 66 Chapter 3
Installing HyperFabric Checking HyperFabric Installation Prerequisites Checking HyperFabric Installation Prerequisites Before installing HyperFabric, check to make sure the following hardware and software prerequisites have been met: ✓ Check the HP HyperFabric Release Note for any known problems, required patches, or other information needed for installation. ✓ Confirm the /usr/bin, /usr/sbin, and /sbin directories are in your PATH by logging in as root and using the echo $PATH command.
Installing HyperFabric Installing HyperFabric Adapters Installing HyperFabric Adapters This section contains information about installing HyperFabric adapters in HP 9000 systems. Online Addition and Replacement (OLAR) information is provided in the “Online Addition and Replacement—HP-UX 11i Only” section on page 62. CAUTION HyperFabric adapters contain electronic components that can easily be damaged by small amounts of electricity.
Installing HyperFabric Installing HyperFabric Adapters Online Addition and Replacement—HP-UX 11i Only Online Addition and Replacement (OLAR) allows PCI I/O cards, adapters or controllers to be replaced or added to HP 9000 systems, without the need for completely shutting down and rebooting the system, or adversely affecting other system components. This feature is only available on HP 9000 systems that are designed to support OLAR.
Installing HyperFabric Installing HyperFabric Adapters There are two methods to add or replace OLAR-compatible cards: • Using the SAM utility. • Issuing command-line commands, through rad, that refer to the HyperFabric OLAR script (/usr/sbin/olard.d/clicd). HP recommends that SAM be used for OLAR procedures, instead of the rad command. This is primarily because SAM prevents the user from doing things that might have adverse effects. This is not true when the rad command is used.
Installing HyperFabric Installing HyperFabric Adapters IMPORTANT In many cases, other interface cards and slots within the system are dependent on the target card. For example, if the target card is a multiple-port card, suspending or deleting drivers for the target card slot also suspends individual drivers for the multiple hardware paths on that card.
Installing HyperFabric Installing HyperFabric Adapters The card must also operate at the slot’s bus frequency. A PCI card must run at any frequency lower than its maximum capability, but a card that could operate at only 33 MHz would not work on a bus running at 66 MHz. rad provides information about the bus frequency and power available at a slot, as well as other slot-related data.
Installing HyperFabric Installing HyperFabric Adapters • If the driver is static and not configured in the kernel, then the card cannot be added online. The card could be physically inserted online, but no driver would claim it. If there is any question about the driver’s presence, or if it is uncertain that the replacement card is identical to the existing card, ioscan can be used together with rad to investigate.
Installing HyperFabric Installing the Software Installing the Software This section describes the HyperFabric file structure and the steps necessary to load the software. The software must be installed on each instance of the HP-UX operating system in the fabric. File Structure The HyperFabric file structure is shown in Figure 3-1 below. Note that the structure is shown for informational purposes only. The user cannot modify any of the files or move them to a different directory.
Installing HyperFabric Installing the Software The global configuration file, which contains the IP addresses for each adapter and each HyperFabric switch (if any) in the fabric. • /sbin/init.d/clic The system boot startup script for the HyperFabric management process. • /var/adm/clic_ip_drv.trc One of the software’s trace files. This file is created when the clic_diag -D TCP_IP command is run. • /var/adm/clic_ip_drv.trc0 One of the HyperFabric software’s trace files.
Installing HyperFabric Installing the Software • /var/adm/OLDclic_log The log file from the previous time the clic_start command was executed. • /usr/conf/lib/libclic_dlpi_drv.a The kernel library that contains the HyperFabric software. • /usr/conf/lib/libha_drv.a The kernel library that contains the High Availability (HA) software. • /usr/conf/master.d/clic This file is described along with the other master files in the master man page (type man master at the HP-UX prompt).
Installing HyperFabric Installing the Software • /opt/clic/firmware/clic_fw_4x8c The 4X PCI HyperFabric 8-bit CRC firmware. Note that this file must not be modified for any reason. • /opt/clic/firmware/clic_fw_4x32c The 4X HyperFabric PCI 32-bit CRC firmware. Note that this file must not be modified for any reason. • /opt/clic/firmware/clic_fw_hf28c The HyperFabric2 8-bit firmware. Note that this file must not be modified for any reason.
Installing HyperFabric Installing the Software Loading the Software Listed below are the steps you must follow to load the HyperFabric software, using the HP-UX swinstall program. Step 1. Log in as root. Step 2. Insert the software media into the appropriate drive. If the software is being loaded from a CD-ROM, go to step 3. Otherwise, go to step 4. Step 3. Mount the CD-ROM drive by using this command: mount device_name where device_name is the name assigned to the CD-ROM drive. Step 4.
Installing HyperFabric Installing the Software • hp-clic 3384/udp #clic switch management Note that these lines are used by the HyperFabric software—and are not comments—so do not remove them from the file. Step 14. Verify that all installed HyperFabric adapters have a software state of “CLAIMED,” by running the ioscan -nf -C clic command. Note: A check is also done to make sure all of the HyperFabric adapters have been claimed when clic_init is activated or when SAM is used to configure HyperFabric.
Installing HyperFabric Installing HyperFabric Switches Installing HyperFabric Switches This section contains the information you need to install HyperFabric switches. As stated earlier, in this manual the term HyperFabric2 (HF2) switch refers to the functional switch (the A6384A switch chassis with one of the switch modules installed).
Installing HyperFabric Installing HyperFabric Switches ❏ After the HyperFabric switch is mounted in the rack, you attach the various cables to the switch. To avoid damage to any of the cables, follow these guidelines: — If your cables have dust caps over the connectors, keep them in place until you are ready to connect them. This prevents dirt and oils from soiling any important surfaces. — Be careful not to stretch, puncture, or crush the cable.
Installing HyperFabric Installing HyperFabric Switches Steps for Installing the HF1 Switch This section contains information for installing a HF1 switch. As mentioned earlier, the HF1 switch must be mounted using the rail kit shipped with the switch. Figure 3-2 below shows the locations of the ports, LEDs, and power cord inlet on the back of the HF1 switch.
Installing HyperFabric Installing HyperFabric Switches Step 2. Install and secure the rails in the rack, using two screws per rail. The figure below shows the rack with the rails installed. screws rack front screws rack back Step 3. From the front of the rack, install a bracket on the outside of each rail, using two screws per bracket. Be sure to use the upper screw holes on each bracket. Put the screws in the second and third square holes—counting away from yourself—in each rail.
Installing HyperFabric Installing HyperFabric Switches bracket 2 rack front bracket 1 rack back Step 4. From the back of the rack, slide the switch—with the front of the switch facing the front of the rack—into the rack, on the rails. Move it until it is touching brackets 1 and 2.
Installing HyperFabric Installing HyperFabric Switches that you cannot easily slide the switch. If so, try lifting the switch over the rail screws. If you cannot do this, remove the rail screws, slide the switch into position, and put the rail screws back in. The figure below shows the rack with the switch in this position. rack front f ro nt of sw it c h bracket 2 bracket 1 rack back Step 5. From the back of the rack, install a bracket on the outside of each rail, using two screws per bracket.
Installing HyperFabric Installing HyperFabric Switches rack front fr o nt of s wi tch bracket 4 rack back bracket 3 Step 6. Tighten all four screws in brackets 3 and 4.
Installing HyperFabric Installing HyperFabric Switches Step 7. From the front of the rack, push the switch so that it barely touches brackets 3 and 4. The figure below shows how to move the switch to this position.
Installing HyperFabric Installing HyperFabric Switches Step 8. Once the switch is snug against brackets 3 and 4, push brackets 1 and 2 in towards the switch, so that they are snug against the switch. The figure below shows brackets 1 and 2 in this position. bracket 2 bracket 4 f ro nt of sw it c h rack front bracket 1 rack back bracket 3 Step 9. Tighten the four screws in brackets 1 and 2. Step 10.
Installing HyperFabric Installing HyperFabric Switches Step 12. Plug the switch’s power cord into the rack’s power distribution unit (PDU), if it has one. Alternatively, you can plug a power cord that is compatible with your country’s requirements into a power strip or outlet that you want to use for the switch. (In this case, you are responsible for obtaining a compatible power cord.) Step 13. Power on the HF1 switch by plugging the power cord into the AC inlet on the back of the switch.
Installing HyperFabric Installing HyperFabric Switches Installing the HF2 Switch This section contains information for installing an HF2 switch. The front of the HF2 switch has a flange—or “wing”—on each side, with two holes for attaching the switch to the rack. Note that the two figures below do not show the flanges. Figure 3-3 below shows the front of the HF2 switch with an A6388A HF2 8-port fibre switch module installed in the switch’s expansion slot.
Installing HyperFabric Installing HyperFabric Switches Figure 3-4 Front of HF2 Switch (A6389A Switch Module Installed) Label showing Ethernet MAC address Integrated Ethernet management LAN card Status Status Status Power A B Ethernet Port Main Port 7 Port 6 Port 11 Port LED colors and meanings legend Integrated 8-port fibre card Ethernet Port Aux Port 5 Port 4 Port 3 Port 10 Port 2 Port 9 Port 1 Port 0 Port 8 A6389A HF2 4-port copper switch module in expansion slot You can install t
Installing HyperFabric Installing HyperFabric Switches With the Rail Kit HP recommends that you install the HF2 switch using the rail kit that is shipped with the switch. The rail kit includes two adjustable rails, screws, nuts, and washers. To install the HF2 switch, you need eight screws and four nuts. Use the square cage nuts if you are installing the HF2 switch in a square-hole rack. Use the u-type clip nuts if you are installing the HF2 switch in a round-hole rack.
Installing HyperFabric Installing HyperFabric Switches One end of each rail has six screw-holes (End-A), the other end has two screw holes (End-B). Figure 3-6 shows both the ends of the rail. Figure 3-6 The Ends of the Rail Kit Step 3. Orient the rails so that End-A faces the back of the rack and aligns with the front end of the switch with flanges. Step 4. Loosen the wing nuts on each rail and adjust the length of each rail to fit the length of the rack.
Installing HyperFabric Installing HyperFabric Switches HP recommends employing two people to support the weight of the switch because End-A of the rail is not yet secured. NOTE Step 8. Secure the switch and End-A of each rail by aligning the two holes in each flange with the two holes in each rack column, and two of the holes in each rail. Secure the entire assembly with two screws in each flange. Step 9.
Installing HyperFabric Installing HyperFabric Switches Without the Rail Kit As mentioned earlier, HP strongly recommends installing the HF2 switch using the rail kit (described in the previous section, “With the Rail Kit” on page 78). When you install the HF2 switch, you will be putting the front of the switch—the end with the flanges (“wings”)—at the back of the rack. The steps for installing the HF2 switch without using the rail kit are as follows: Step 1. Prepare the rack for switch installation. Step 2.
Installing HyperFabric Installing HyperFabric Switches If you want to install another HF2 switch using the rail kit, go to “With the Rail Kit” on page 78. Otherwise, go to Chapter 4, “Configuring HyperFabric,” on page 83.
4 Configuring HyperFabric This chapter contains the following sections that describe configuring HyperFabric: Chapter 4 • “Configuration Overview” on page 85 • “Information You Need” on page 86 • “Doing the Configuration” on page 91 83
Configuring HyperFabric 84 • “Deconfiguring a HyperFabric Adapter with SAM—HP-UX 11.
Configuring HyperFabric Configuration Overview Configuration Overview You do not need to configure the HyperFabric switch because the HyperFabric management process performs automatic routing and configuring for the switch. So, configuring HyperFabric consists only of creating the HyperFabric /etc/rc.config.d/clic_global_conf global configuration file on each node in the fabric.
Configuring HyperFabric Information You Need Information You Need When you run the clic_init command or use SAM for configuration, you have to provide certain configuration information. So, before you run clic_init or use SAM, you should get the following information: ❏ For each node in the fabric, determine if that node will need to interoperate with other nodes that are using; any HP-UX 10.20 version of HyperFabric, any HP-UX 11.0 HyperFabric versions earlier than B.11.00.
Configuring HyperFabric Information You Need ❏ For each node in the fabric, you need the IP address of the node’s Ethernet LAN interface that is on the same subnet as the switches. (As mentioned earlier, a node might have multiple LAN interfaces.) Note that if you do not have switch management enabled, you do not need this information (clic_init will not prompt you for it). As stated earlier, we recommend that you do not enable switch management.
Configuring HyperFabric Information You Need Configuration Information Example For this example, we have added some “dummy” (that is, not valid) addresses to the components in Figure 4-1, Map for Configuration Information Example, below. The “dummy” addresses are used only to show the flow of the information provided as input to the clic_init command and SAM. Do not try to use these addresses in your configuration.
Configuring HyperFabric Information You Need 4. What is the subnet mask of the first adapter? (255.255.255.0) If you do not specify a value for this, a default mask is chosen. You will most likely just accept the default. However, in this example, we are showing a value for the subnet mask just to illustrate the correlation between the “dummy” information in Figure 4-1 and where that information is specified or generated during clic_init and SAM. 5.
Configuring HyperFabric Information You Need 8. If switch management has been enabled, how many switches will be configured? As stated earlier, we recommend that you do not enable switch management. 9. What is the IP address of the first switch (sw_clic0)? (193.0.0.20) 10. What is the Ethernet hardware address of the first switch? (0060b0d00257) 11. What is the IP address of the second switch (sw_clic1)? (193.0.0.21) 12. What is the Ethernet hardware address of the second switch? (0060b0d00256) 13.
Configuring HyperFabric Doing the Configuration Doing the Configuration As explained in “Configuration Overview” on page 85, you must create the global configuration file (/etc/rc.config.d/clic_global_conf) on each node in the fabric. This consists mostly of specifying HyperFabric adapter-related information. (Note that if you are also going to enable switch management—which we do not recommend doing—you need to specify additional configuration information.
Configuring HyperFabric Doing the Configuration Using the clic_init Command Run the clic_init command to create the global configuration file. IMPORTANT If the global configuration file already exists and you are running clic_init again (to change the file), you have the option of retaining or modifying the existing configuration information, in addition to adding new information pertaining to new hardware.
Configuring HyperFabric Doing the Configuration Examples of clic_init Some examples of using the clic_init command are shown below.
Configuring HyperFabric Doing the Configuration Using SAM—HP-UX 11.0 and HP-UX 11i This section describes how to use SAM to configure HyperFabric. IMPORTANT If the global configuration file already exists, and you are running SAM again (to change the file), you can keep or modify the existing configuration information, in addition to adding new information pertaining to new hardware.
Configuring HyperFabric Doing the Configuration • If you have just created the global configuration file on the local node for the first time (and you are not configuring MC/ServiceGuard), run clic_start or use SAM to start HyperFabric. • If you have just changed an existing configuration file on the node, run clic_shutdown or use SAM to stop HyperFabric, and then run clic_start or use SAM to start HyperFabric. Until you do those two things, your configuration changes will not take effect.
Configuring HyperFabric Deconfiguring a HyperFabric Adapter with SAM—HP-UX 11.0 and 11i 0nly Deconfiguring a HyperFabric Adapter with SAM—HP-UX 11.0 and 11i 0nly To use SAM to deconfigure a HyperFabric adapter on an HP 9000 system running HP-UX 11.0 or 11i, follow these steps: Step 1. Start SAM. Step 2. Select the “Networking and Communications” area. Step 3. Select “HyperFabric.” All HyperFabric adapters installed in the system are listed.
Configuring HyperFabric Configuring the HyperFabric EMS Monitor Configuring the HyperFabric EMS Monitor Starting with the December 2000 releases B.11.00.11 and B.11.11.01, the HyperFabric Event Monitoring Service (EMS) monitor allows system administrators to separately monitor each HyperFabric adapter on every node in the fabric, in addition to monitoring the entire HyperFabric subsystem. The monitor can inform the user if the resource being monitored is UP or DOWN.
Configuring HyperFabric Configuring HyperFabric with MC/ServiceGuard Configuring HyperFabric with MC/ServiceGuard HyperFabric supports the MC/ServiceGuard HA product. NOTE If you plan to configure HyperFabric with MC/ServiceGuard, please read this section. Otherwise, skip this section and go on to the next chapter, Chapter 4, “Managing HyperFabric,” on page 83. MC/ServiceGuard lets you create HA clusters of HP 9000 server systems.
Configuring HyperFabric Configuring HyperFabric with MC/ServiceGuard NOTE Chapter 4 Because the HyperFabric network does not currently support MC/ServiceGuard heartbeat connections, you must use an alternative type of connection for the heartbeat, such as FDDI, Token Ring, 100BaseT, or Ethernet (as shown in Figure 4-2 below).
Configuring HyperFabric Configuring HyperFabric with MC/ServiceGuard Figure 4-2 An MC/ServiceGuard Configuration (with Two HyperFabric Switches) Ethernet Heartbeat LAN 1 Ethernet Heartbeat LAN 0 node B node A HF adapter 1 HF adapter 0 HF adapter 0 HF adapter 1 HF switch 1 node C Ethernet Port node D HF adapter 1 HF adapter 1 HF adapter 0 S S HF adapter 0 S HF switch 0 S Ethernet Port How HyperFabric Handles Adapter Failures HyperFabric adapters are handled differently than other types of
Configuring HyperFabric Configuring HyperFabric with MC/ServiceGuard adapter is now handling the network traffic.) This rearrangement is handled by the HyperFabric software, and MC/ServiceGuard is not notified. However, note that if all of the HyperFabric adapters fail, HyperFabric does notify MC/ServiceGuard. In both cases, though, the events are logged to /var/adm/clic_log and /var/adm/syslog.log.
Configuring HyperFabric Configuring HyperFabric with MC/ServiceGuard Example 1: This example, illustrated by Figure 4-3 below, presents an HA configuration using MC/ServiceGuard with HyperFabric. Both of the HyperFabric adapters are active on node A. The HyperFabric Resource Monitor reports the active status of the HyperFabric resource to the Event Monitoring Service (EMS), which lets MC/ServiceGuard know that the HyperFabric resource is available to Packages A and B.
Configuring HyperFabric Configuring HyperFabric with MC/ServiceGuard Example 2: This example, illustrated by Figure 4-4 below, shows the same node after the failure of one of the HyperFabric adapters. The remaining adapter in node A is now handling all HyperFabric network traffic for the node. Because the HyperFabric resource is still available, MC/ServiceGuard has not been notified; HyperFabric handles the local HyperFabric adapter failover.
Configuring HyperFabric Configuring HyperFabric with MC/ServiceGuard Example 3: This final example, illustrated by Figure 4-5 below, shows a situation in which all of the HyperFabric adapters on node A fail. The HyperFabric Resource Monitor reports to the Event Monitoring Service (EMS). The EMS then notifies the MC/ServiceGuard cmcld daemon that the HyperFabric resource on node A is unavailable.
Configuring HyperFabric Configuring HyperFabric with MC/ServiceGuard You should configure HyperFabric with MC/ServiceGuard before running the clic_start command or using SAM to start HyperFabric.
Configuring HyperFabric Configuring HyperFabric with MC/ServiceGuard Configuring MC/ServiceGuard for HyperFabric Relocatable IP Addresses If you are using HyperFabric version B.11.00.05, B.11.11.00, or later (note no HP-UX 10.20), and you want to use relocatable IP addresses, configure the relocatable IP addresses with the IP[n] command in the package control script. For example, to configure the relocatable address 192.0.0.3 for adapter 0 and 192.0.8.5 for adapter 1, specify this: IP[0]= 192.0.0.
4 Managing HyperFabric This chapter contains the following sections that give information about managing HyperFabric: Chapter 4 • “Starting HyperFabric” on page 85 • “Verifying Communications within the Fabric” on page 87 83
Managing HyperFabric 84 • “Displaying Status and Statistics” on page 91 • “Viewing man Pages” on page 98 • “Stopping HyperFabric” on page 99 Chapter 4
Managing HyperFabric Starting HyperFabric Starting HyperFabric HyperFabric is started in one of these three ways: • As part of the normal local node boot process (HP 9000 system). • By running the HyperFabric clic_start command (described below). • By starting HyperFabric through SAM (described in “Using SAM—HP-UX 11.0 and 11i 0nly” on page 86).
Managing HyperFabric Starting HyperFabric The clic_start -? command can be issued to display the online help for clic_start, or look at the clic_start (1m) man page by issuing the man clic_start command. If HyperFabric is already running, you will receive an informational (FYI) message telling you so. Your reaction to this message depends on the situation: • If you have simply forgotten (or did not know) that HyperFabric was already running, you do not have to do anything.
Managing HyperFabric Verifying Communications within the Fabric Verifying Communications within the Fabric You can verify the communications within the fabric by running the clic_probe command, which is described below. You can also use clic_probe to verify the status of specific adapters. NOTE Beginning with HyperFabric versions B.11.00.11 and B.11.11.01, clic_probe replaces the clic_ping command. For systems running HP-UX 10.
Managing HyperFabric Verifying Communications within the Fabric • -r specifies that VRID switch_hopcount is the routing information for the adapter. To determine the VRID and switch_hopcount to specify, first run the clic_stat -d VRID command (see “The clic_stat Command” on page 91). Note that if you specify this parameter (-r VRID switch_hopcount), you must also specify the -c adapter_ID parameter (described above). • -l specifies that you want to do local loopback testing on a particular adapter.
Managing HyperFabric Verifying Communications within the Fabric NOTE Also see the clic_diag command to: Probe a specific remote node. Dump and format trace data. Set the tracing level for the HyperFabric software and firmware. The clic_diag command is detailed in the Running Diagnostics section of Chapter 6, Troubleshooting. Examples of clic_probe Some examples of using clic_probe are shown below.
Managing HyperFabric Verifying Communications within the Fabric • Example 3 If the local node is bently6, and you want to send five packets to verify communications with the remote node bently7, using the adapter clic0 (which is on bently6), issue this command: clic_probe bently7 -c clic0 -p 5 CLIC_PROBE: 256 byte packets Source adapter id: bently6.corp3.com:clic0 Target adapter id: bently7.corp4.com:clic1 256 bytes: seq_num = 1. Packet Acknowledged. 256 bytes: seq_num = 2. Packet Acknowledged.
Managing HyperFabric Displaying Status and Statistics Displaying Status and Statistics You can get the status of and statistics associated with many of the HyperFabric components by using the clic_stat command, which is described below. The clic_stat Command The following list contains some of the information that the clic_stat command will provide: • The current fabric map, in textual format. • The status of one or more HyperFabric adapters.
Managing HyperFabric Displaying Status and Statistics Note that turning on performance statistics gathering will affect the performance of HyperFabric, by increasing CPU usage and message latency (the time it takes a message to get from one point to another). It might also decrease the throughput of data. It is recommended that you do not specify this parameter unless you are trying to troubleshoot HyperFabric problems. You must be logged in as root to use this parameter.
Managing HyperFabric Displaying Status and Statistics ============================================================================ • Example 2 If the local node is bently7, and you want to display the management daemon (clic_mgmtd) configuration and statistics data, issue this command: clic_stat -d CFG The generated output could look like this: ============================================================================ Date: Sat Aug 5 16:08:12 2000 Node: bently7.corp2.
Managing HyperFabric Displaying Status and Statistics ============================================================================ Date: Sat Aug 5 16:08:30 2000 Node: bently7.corp2.com ---------------------------------------------------------------------------Adapter ID: clic1 Instance Number: 1 Adapter Type: 1X PCI or HSC Firmware File: /opt/clic/firmware/clic_fw_1x32c Major Num: 238 Mgmt process driver handle: 5 Version: 1.0 H/W Path: 8/4/0/0 Primary IP address: 192.0.0.
Managing HyperFabric Displaying Status and Statistics • Example 4 If the local node is bently6, and you want to display the VRIDs, IP addresses, switch hopcounts, and physical routes for each HyperFabric adapter in bently6, issue this command: clic_stat -d VRID The generated output if the nodes are connected in a Point-to-Point configuration could look like this: ============================================================================ Date: Sat Aug 5 16:08:12 2000 Node: bently6.corp4.
Managing HyperFabric Displaying Status and Statistics ============================================================================ Date: Sat Aug 5 16:08:12 2000 Node: bently6.corp4.
Managing HyperFabric Displaying Status and Statistics ---------------------------------------------------------------------------No performance statistics are being gathered - Reset successful ============================================================================ Chapter 4 97
Managing HyperFabric Viewing man Pages Viewing man Pages If you want to be able to view the HyperFabric man pages, you must first add /opt/clic/share/man to your MANPATH environment variable.
Managing HyperFabric Stopping HyperFabric Stopping HyperFabric You can stop HyperFabric only on a local node. Stopping HyperFabric on a node makes all of the HyperFabric adapters in that node unreachable by all other nodes in the fabric. It stops the HyperFabric management process, which stops all interconnect operations on the node. To stop HyperFabric, you use (1) the clic_shutdown command (described below) or (2) SAM (described in “Using SAM—HP-UX 11.0 and 11i 0nly” on page 100).
Managing HyperFabric Stopping HyperFabric Using SAM—HP-UX 11.0 and 11i 0nly To use SAM to stop HyperFabric on a local HP 9000 system running HP-UX 11.0 or 11i, follow these steps: Step 1. Start SAM. Step 2. Select the “Networking and Communications” area. Step 3. Select “HyperFabric.” Step 4. Pull down the “Actions” menu and select Stop HyperFabric. Note that if HyperFabric is not running on the system, Stop HyperFabric is grayed out and you cannot select it.
5 Troubleshooting HyperFabric This chapter contains these sections that describe troubleshooting HyperFabric: Chapter 5 • “Running Diagnostics” on page 103 • “Using Support Tools Manager” on page 108 • “Useful Files” on page 109 101
Troubleshooting HyperFabric 102 • “LED Colors and Their Meanings” on page 111 • “Determining Whether an Adapter or a Cable is Faulty” on page 122 • “Determining Whether a Switch is Faulty” on page 123 • “Replacing a HyperFabric Adapter” on page 125 • “Replacing a HyperFabric Switch” on page 126 Chapter 5
Troubleshooting HyperFabric Running Diagnostics Running Diagnostics Before running HyperFabric diagnostics: 1. Confirm HyperFabric adapters are installed on each node. Use the following command to display a list of HyperFabric adapters installed each node: ioscan -funC clic 2. Check to see if HyperFabric software is installed. Use the following command to display the HyperFabric software version installed on each node: swlist | grep -i hyperfabric 3. Check the patch level.
Troubleshooting HyperFabric Running Diagnostics clic_shutdown clic_init (The answer to the interoperability question must be consistent on all nodes in the fabric.) clic_start Check to make sure all IP addresses have been assigned. Check the subnet. Every HyperFabric adapter in the fabric must be able to communicate with every other HyperFabric adapter. Every HyperFabric adapter in the fabric must be connected, point-to-point or via a switch.
Troubleshooting HyperFabric Running Diagnostics If you are running an Oracle application using HMP and traffic does not appear to be flowing after completing all of the steps listed above, check the ORAHOME/rdbms/log/alert*log files. The display should show Cluster Interconnect IPC version:Oracle using HP-HMP logged in the alert log files. If not, recompile the Oracle application to run using HMP according to the instructions provided in the Oracle Installation and Administration Guide.
Troubleshooting HyperFabric Running Diagnostics • -r specifies that you want to probe a specific remote node identified by remote_node_name. The probe is done on all operational routes to the remote node.
Troubleshooting HyperFabric Running Diagnostics Example of clic_diag An example of the clic_diag command is shown below. If the local node is bently6, and you want to confirm that all of the adapters on bently6 are communicating with the target adapters on bently8, issue this command: clic_diag -r bently8 The generated output could look like this: CLIC_PROBE: 256 byte packets Source adapter id: bently6.corp4.com:clic0 Target adapter id: bently8.corp2.com:clic1 256 bytes: seq_num = 0. Packet Acknowledged.
Troubleshooting HyperFabric Using Support Tools Manager Using Support Tools Manager Use Support Tools Manager (STM) with HyperFabric to gather information about HyperFabric components and to diagnose hardware problems. Two tools are available in STM for HyperFabric: • The Information Tool provides information about the HyperFabric adapter, without resetting the adapter. • The Diagnostics Tool can be used to run tests on the HyperFabric adapter; the tool reports any failures.
Troubleshooting HyperFabric Useful Files Useful Files When you are troubleshooting HyperFabric-related problems, you might find it useful to look at the contents of the following files: • /etc/rc.config.d/clic_global_conf This is the global configuration file. Check it to confirm that the configuration information is correct. • /var/adm/clic_log This is a global log file that contains a history of significant HyperFabric events.
Troubleshooting HyperFabric Useful Files /var/adm/clic_fw.dumpx This is the default file for a memory dump of an HyperFabric adapter, created when the clic_diag -d command is run without specifying an output file. • /etc/rc.config.d/netconf This file contains IP-related configuration information for all of the networking adapters installed in the HP 9000. IMPORTANT: clic_init and SAM modify this file, adding some HyperFabric-related lines that end with the characters #clic.
Troubleshooting HyperFabric LED Colors and Their Meanings LED Colors and Their Meanings Listed below are the possible colors (and the corresponding meaning) of the LEDs on the HyperFabric adapters and switches. Adapter LEDs Table 5-1 below shows the names of the LEDs on each HyperFabric adapter. Note that the LEDs on the A4920A adapter are labeled, but the labels might be hard to see when a cable is connected to the adapter.
Troubleshooting HyperFabric LED Colors and Their Meanings — The adapter is connected to a non-operational adapter in the remote node (in a node-to-node configuration). ✓ If the connection from the adapter to the corresponding switch port (if a switch is used) or the corresponding adapter in the remote node (in a node-to-node configuration) is operational, the “Connected/Traffic,” “Link,” or “Link Connected/Traffic” LED on the adapter shows as solid green.
Troubleshooting HyperFabric LED Colors and Their Meanings Table 5-2 HyperFabric Adapter LED Colors and Meanings LED “Connected/Traffic” (A4919A and A6092A adapters) Color None Meaning Adapter is not operational. “Link” (A4920A and A6386A adapters) “Link Connected/Traffic” (A4921A adapter) “Error” Chapter 5 Notes • HP 9000 is not operational. • HyperFabric is not running on HP 9000. • Adapter is installed in a slot in HP 9000, but cable is attached incorrectly or no cable is attached at all.
Troubleshooting HyperFabric LED Colors and Their Meanings HF1 Switch LEDs The HF1 switch LED colors and meanings are explained below. ❏ For each port on the switch: ✓ If the port is not operational, the LED is off. Some of the possible reasons for this happening are the following: — A cable is not attached correctly to the port or no cable is attached at all. — The switch port is connected to a non-operational adapter in an HP 9000.
Troubleshooting HyperFabric LED Colors and Their Meanings ❏ For the “Ethernet” port and LED on the switch: ✓ If the Ethernet port is not connected to the Ethernet network, the LED is off. ✓ If the connection from the Ethernet port to the Ethernet network is operational, the LED on the port shows as solid green. ✓ If data is flowing between the Ethernet port and the Ethernet network, the LED shows as flashing green. Table 5-3 below summarizes the HF1 switch LED information in a table format.
Troubleshooting HyperFabric LED Colors and Their Meanings Table 5-3 HF1 Switch LED Colors and Meanings LED Port “Power” 116 Color None Meaning Port is not operational. Solid green Connection from port to adapter is operational. Flashing green Data is flowing between the port and the adapter. None Power to switch is off. Solid green Power to switch is on. Notes • A cable is not attached correctly to the port or no cable is attached at all.
Troubleshooting HyperFabric LED Colors and Their Meanings Table 5-3 HF1 Switch LED Colors and Meanings (Continued) LED “Ethernet” Chapter 5 Color Meaning Notes None Ethernet port is disconnected. This can happen if the Ethernet port is not correctly connected to the Ethernet network. Management is disabled when the Ethernet port is not operating, however, this does not prevent the fabric from operating. Solid green Connection from Ethernet port to the Ethernet network is operational.
Troubleshooting HyperFabric LED Colors and Their Meanings HF2 Switch LEDs The HF2 switch LED colors and meanings are explained below. ❏ For each “Status” LED on the switch: ✓ If the card/switch module is not operating, the LED is off. — For the integrated Ethernet management LAN adapter card (in the top slot) and the integrated 8-port fibre card (in the middle slot): the card can be safely removed by qualified Hewlett-Packard personnel only.
Troubleshooting HyperFabric LED Colors and Their Meanings ❏ For each port on the cards/switch module in the switch: ✓ If the port is not operational, the LED is off. Some of the possible reasons for this happening are the following: — A cable is not attached correctly to the port or no cable is attached at all. — The port is connected to a non-operational adapter in an HP 9000. (See “Adapter LEDs” on page 111 or Table 5-2 on page 113 for some tips about a non-operational adapter.) — The cable is bad.
Troubleshooting HyperFabric LED Colors and Their Meanings Table 5-4 HF2 Switch LED Colors and Meanings LED “Status” “Power A” and “Power B” “Ethernet Port Main” and “Ethernet Port Aux” 120 Color Meaning Notes Integrated Ethernet management LAN adapter card (in the top slot) & integrated 8-port fibre card (in the middle slot): the card can be safely removed by qualified Hewlett-Packard personnel only.
Troubleshooting HyperFabric LED Colors and Their Meanings Table 5-4 HF2 Switch LED Colors and Meanings (Continued) LED “Port x” Chapter 5 Color None Meaning Port is not operational. Solid green Connection from port to adapter is operational. Flashing green Data is flowing between the port and the adapter. Notes • A cable is not attached correctly to the port or no cable is attached at all. • The port is connected to a non-operational adapter in an HP 9000.
Troubleshooting HyperFabric Determining Whether an Adapter or a Cable is Faulty Determining Whether an Adapter or a Cable is Faulty If you suspect that either an HyperFabric adapter or a cable attached to it is faulty, follow these steps to determine which component needs replacing: Step 1. Disconnect the cable from the HyperFabric adapter. Step 2.
Troubleshooting HyperFabric Determining Whether a Switch is Faulty Determining Whether a Switch is Faulty If you suspect that a HyperFabric switch is faulty, follow the steps below to determine if the switch needs replacing. HF1 Switch The steps for determining if an HF1 switch is faulty are as follows: Step 1. Disconnect the cable from the switch port you suspect is faulty. Step 2. Attach a copper loopback plug to the relevant port.
Troubleshooting HyperFabric Determining Whether a Switch is Faulty a. If the port is connected to an operational Ethernet network, the port’s LED (“Ethernet Port Main” or “Ethernet Port Aux”) shows as solid or blinking green. b. If the port is disconnected, or connected to an Ethernet network that is not operational, the port’s LED is off. Step 4. Check the ports on the cards/switch module: a. Disconnect the cable from the card/switch module port you suspect is faulty. b.
Troubleshooting HyperFabric Replacing a HyperFabric Adapter Replacing a HyperFabric Adapter If OLAR is supported for the HP 9000 system and the HyperFabric adapter, and you want to do OLR, see “Online Replacement (OLR)” on page 58. The HP HyperFabric Release Note contains information about which HP 9000 systems and HyperFabric adapters OLAR is supported for. If you cannot or do not want to do OLR to replace a HyperFabric adapter, follow these steps: Step 1.
Troubleshooting HyperFabric Replacing a HyperFabric Switch Replacing a HyperFabric Switch If you need to replace a faulty HyperFabric switch, follow these steps: Step 1. If you do not have an HA environment, stop HyperFabric on all nodes that are connected to the faulty switch. (See “Stopping HyperFabric” on page 99.) If you have an HA environment with two switches (where the backup switch will have taken over when the first switch failed), disconnect all cables attached to the faulty switch. Step 2.
Index A adapters deconfiguring, 96 faulty, testing for, 146 installing, 54 LED colors and meanings, 135, 137 OLA of, 57 OLR of, 58 151
replacing, 149 C cables, testing for faulty, 146 CLIC, definition of, 19, 60 clic_diag command, 113, 129 clic_init command, 92 after OLA, 58 clic_ping command, 111 clic_probe command, 111 to remote node, 130 clic_shutdown command, 123 clic_start command, 109 after OLA, 58 clic_stat command, 115 CLuster InterConnect.
current map of, 115, 116 definition of, 19 planning, 23 files global configuration contents of, 85 creating, 91 software description of, 60 structure of, 60 H HA configuration, description of, 19 definition of, 19 MC/ServiceGuard and HyperFabric, 98 High Availability. See HA I installation adapters, 54 prerequisites for, 53 software, 60 steps for, 64 switch, 66 HF2, 76 steps for HF, 68 steps for HF2, no rail kit, 81 IP addresses, relocatable. See relocatable IP addresses.
P planning the fabric, 23 prerequisites for installation, 53 R relocatable IP addresses, 19 configuring, 106 S SAM using for configuration, 94 using for deconfiguring an adapter, 96 using for starting HyperFabric, 110 using for stopping HyperFabric, 124 software file structure , 60 files, description of, 60 installing, 60 steps for, 64 starting HyperFabric after OLA, 58 using clic_start, 109 using SAM , 110 statistics, displaying, 115 status displaying, 115 of one switch, 116 STM tools for HyperFabric, 132
viewing man pages, 122 VRID, 86 155
