HP Serviceguard for Linux Version A.11.
Legal Notices The information in this document is subject to change without notice. Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. Hewlett-Packard shall not be held liable for errors contained herein or direct, indirect, special, incidental or consequential damages in connection with the furnishing, performance, or use of this material. Warranty.
Table of Contents 1 Serviceguard for Linux Version A.11.19 Release Notes.....................................................................9 Announcements....................................................................................................................9 Supported Platforms and Linux Distributions...............................................................9 Product Structure ...........................................................................................................
Support for HPVM and VMware ESX Server..........................................................26 Points to Note.....................................................................................................26 About HPVM Virtual Machines and Cluster Re-formation Time......................27 For More Information.........................................................................................28 Support for DM-MPIO..................................................................................
Installing the Quorum Server........................................................................................48 Setting Permissions on the /root/.rhosts File.................................................................48 Uninstalling Serviceguard for Linux.............................................................................48 Rules.........................................................................................................................49 Rolling Software Upgrades...........
List of Figures 1-1 1-2 1-3 1-4 1-5 1-6 6 Running Cluster Before Rolling Upgrade ..................................................................56 Running Cluster with Packages Moved to Node 2 ....................................................57 Node 1 Upgraded to Red Hat EL 3.............................................................................57 Node 1 Rejoining the Cluster .....................................................................................
List of Tables 1-1 Serviceguard Manager and Serviceguard Versions....................................................
1 Serviceguard for Linux Version A.11.19 Release Notes Announcements This section announces the most important features and limitations of Serviceguard for Linux A.11.19. For more information, see “What’s in this Release” (page 13). NOTE: These Release Notes also include information about features first introduced in A.11.18 patches; see “New Features First Introduced in Serviceguard for Linux A.11.18 Patches” (page 22). Supported Platforms and Linux Distributions Serviceguard A.11.
◦ one license required per host node If Serviceguard for Linux is running in virtual machines, only one license is required, irrespective of the number of virtual machines running on the host that has the license ◦ license certificate has license count equal to number of licenses ordered — one year of 24x7 support services included ◦ registration is required to activate support services upgraded support services are available • T8719AA — A.11.
The CD is the HP Serviceguard for Linux CD™, containing: • Serviceguard for Linux version A.11.19 for: — Red Hat Enterprise Linux 5 server or Advanced Platform — SUSE SLES 10 — Quorum Server software and documentation — Serviceguard Manager software and documentation Complete Serviceguard documentation is provided on the CD. New Licensing Method Serviceguard for Linux A.11.19 is shipped with an instant-on license valid for 90 days. This allows you to start using Serviceguard A.11.
of Managing Serviceguard). In some cases new capabilities provided in modular packages (those created by the method introduced in A.11.18) have been added to legacy packages as well. Use the modular method to create new packages whenever possible. See Chapter 6 of Managing Serviceguard for Linux for more information. .rhosts Deprecated Using the .rhosts file as a means of allowing root access to an unconfigured node is deprecated as of Serviceguard A.11.19. You should use $SGCONF/cmclnodelist instead.
High Availability Consulting Services Because Serviceguard configurations can be complex to configure and maintain, HP recommends that you use its high availability consulting services to ensure a smooth installation and rollout; contact your HP representative for more information. You should also work with your HP representative to ensure that you have the latest firmware revisions for disk drives, disk controllers, LAN controllers, and other hardware.
• • • Serviceguard A.11.19 uses Persistent Reservations (PR). See “About Persistent Reservations” (page 18) You can now configure a maximum of 300 packages in a cluster. The previous limit was 150. There are new Serviceguard demons: — Service Assistant Daemon: cmserviced (replaces cmsrvassistd) — Cluster WBEM Agent Daemon: cmwbemd — Proxy Daemon: cmproxyd See the section “Serviceguard Daemons” in Chapter 3 of the latest edition Managing Serviceguard for Linux for more information about these daemons.
— Utility Daemon: cmlockd Runs on every node on which cmcld is running. It maintains the active and pending cluster resource locks. — Cluster SNMP Subagent (optionally running):cmsnmpd • • • Hostnames for Serviceguard cluster nodes can be up to 39 characters (bytes) long. (The former limit was 31 characters.) A new command, cmdisklock, can be used to repair or replace a lock LUN; see “Replacing a Lock LUN” in Chapter 8 of Managing Serviceguard for Linux, and the cmdisklock manpage, for more information.
Capacity and Package Weight Feature, which you can find on docs.hp.com under High Availability —> Serviceguard —> White Papers. New Support for IPv6 Serviceguard A.11.19 allows you to choose either the IPv4 or the IPv6 address family for any subnet configured into the cluster, including subnets that carry the cluster heartbeat. For SLES 10 only, you can also configure IPv6 connections to the Quorum Server.
New Online Cluster Configuration Capabilities Serviceguard A.11.19 introduces new online cluster-management capabilities. The new capabilities include: • Changing the Quorum Server configuration • Changing the lock LUN configuration • Changing an interface from IPv4 to IPv6, or vice versa For a summary of what can be changed online, with instructions and caveats, see the section “Reconfiguring a Cluster” in Chapter 7 of the latest version of Managing Serviceguard for Linux.
IMPORTANT: The package must be down and disabled before you can place it it in, or take it out, of partial-startup maintenance mode. For more information, see “Maintaining a Package: Partial-Startup Maintenance Mode” in Chapter 7 of Managing Serviceguard for Linux. Other Package Changes NOTE: For information about legacy and modular packages, see Chapter 6 of Managing Serviceguard. Serviceguard A.11.
SCSI Primary Commands version 3 (SPC-3) standard, provide a means to register I/O initiators and specify who can access LUN devices (anyone, all registrants, only one registrant) and how (read-only, write-only). Unlike exclusive activation for volume groups, which does not prevent unauthorized access to the underlying LUNs, PR controls access at the LUN level.
NOTE: • DSAU does not support a local log consolidation server in a cross-subnet cluster. Instead, you can set up a remote log consolidation server on the Quorum service node or cluster. • When AppArmor (SUSE) or SELinux (Red Hat) is enabled, you may need to modify the security configuration to enable DSAU to run. Refer to the Distributed Systems Administration Utilities (DSAU) Release Notes for Linux on docs.hp.com for more information. Native Language Support HP Serviceguard Manager Version B.02.
— Tooltips for cluster and package configuration parameters — Cluster configuration wizard — Advanced cluster configuration options • Ease-of-use improvements: — Preview Administration operations (e.g. simulate an administration operation such as “Halt Node”) — Display Serviceguard patch version — Context-sensitive links to Partition Manager ◦ Partition Manager is a free HP tool to configure and manage nPartitions on HP server systems.
Table 1-1 Serviceguard Manager and Serviceguard Versions Number of Clusters Serviceguard Version Serviceguard Manager Version 1 A.11.19 B.02.00 A.11.18 B.01.01 A.11.18 and later Multiple browser sessions of B.0x.0x More than 1 TIP: To prevent an OutofMemory error reported by Tomcat (Exception in thread "main" java.lang.
Configuration Tasks Cluster and package configuration tasks are affected as follows: • • You must use the -w full option of cmquerycl to discover actual or potential nodes and subnets across routers.
monitored for this package are configured for PARTIAL access, each node on the node_name list must have at least one of these subnets configured. — As in other configurations, a package will not start on a node unless the subnets configured on that node, and specified in the package configuration file as monitored subnets, are up. • You cannot use Serviceguard Manager to configure a cross-subnet cluster or package.
— failure - set if the package halts because of the failure of a resource or service it depends on — user_halt - set if the package is halted by a cmhaltpkg or cmhaltnode command, or by corresponding actions in Serviceguard Manager — automatic_halt - set if the package is failed over automatically because of the failure of a package it depends on, or is failed back to its primary node automatically (failback_policy = automatic) You can add custom code to the package to interrogate this variable, determine w
IMPORTANT: As of Serviceguard A.11.19, this capability requires Quorum Server Version A.04.00. (It is also provided in a patch to Serviceguard A.11.18 with Quorum Server Version A.03.00, but Quorum Server Version A.03.00 does not support an alternate subnet with Serviceguard A.11.19.) See “Quorum Server Upgrade Required if You Are Using an Alternate Address” (page 12). Support for HPVM and VMware ESX Server Serviceguard for Linux supports HP Integrity Virtual Machines (HPVM) version 3.5, 4.0, or 4.
• increases. See “About HPVM Virtual Machines and Cluster Re-formation Time” (page 27). VMware nodes. In the case of VMware nodes, Persistent Reservations (PR) handle the I/O issues. This entails the following restriction: — Serviceguard does not support configurations with more than one VMware virtual-machine node on the same host (“cluster-in-a-box”) because such configurations are not supported by PR.
IMPORTANT: This represents a net addition to the time it takes for the cluster to re-form. For example, if the cluster typically took 40 seconds to re-form before any HPVM nodes were added, it will generally take about 80 seconds when one or more VM nodes are members of the cluster, if all those nodes have the hpvminfo software. If any HPVM node without that software is a member of the cluster, it will take about 110 seconds.
Changes First Introduced in Older Releases The following important changes have either been introduced since the previous edition of the Serviceguard Release Notes was published, or may affect the upgrade from an earlier release. • As of A.11.16, Serviceguard uses Access Control Policies to control access to, and administrative control over, the cluster. See “Access changes as of A.11.16” (page 29). Access changes as of A.11.16 Serviceguard version A.11.16 introduced a new access method. As of A.11.
Considerations when Installing Serviceguard When you install Serviceguard for the first time on a node, the node is not yet part of a cluster, and so there is no Access Control Policy. For instructions on how to proceed, see the subsection “Allowing Root Access to an Unconfigured Node” under “Configuring Root-Level Access” in Chapter 5 of the Managing Serviceguard for Linux manual. What’s Not in this Release • Serviceguard A.11.19 does not support Red Hat 4. The last version that did was A.11.18.
• • • • • • • • Designing High-Availability for Xen Virtual Machines with HP Serviceguard for Linux at http://docs.hp.com -> High Availability -> Serviceguard for Linux -> White Papers HP Serviceguard for Linux Version A.11.19 Deployment Guide at http:// docs.hp.com -> High Availability -> Serviceguard for Linux -> White Papers Clustering Linux Servers with the Concurrent Deployment of HP Serviceguard for Linux and Red Hat Global File Systems for RHEL5 at http://docs.hp.
Release Notes). Versions with the same part number are differentiated by the publication date. Compatibility Information and Installation Requirements Read this entire document and any other Release Notes you may have before you begin an installation. NOTE: All nodes must be on the same distribution, including errata level and architecture version. Using different distribution versions within the same cluster is supported only during rolling upgrade.
• • wbem-http TCP/5988 wbem-https TCP/5989 If you will be using the Quorum Server • hacl-qs 1238/TCP HA Quorum Server If you will be using the appserver utility: • hacl-poll 5315/TCP Ports Needed on SUSE: • ident 113/TCP used by identd • hacl-hb 5300/TCP High Availability (HA) Cluster heartbeat • hacl-hb 5300/UDP High Availability (HA) Cluster heartbeat • hacl-cfg 5302/TCP HA Cluster TCP configuration • hacl-cfg 5302/UDP HA Cluster UDP configuration If you will be using SNMP: • snmp 161/UDP • snmptrap 1
This port is configurable; if port 1775 is already being used by another application, configure and open another free port when you configure the firewall. System Firewalls When using a system firewall with Serviceguard for Linux, you must leave open the ports listed above. Serviceguard also uses some dynamic ports for some cluster services; these also need to be open in the firewall. They are typically in the range 32768-61000 for Red Hat and 1024-29999 for SUSE.
All nodes in the cluster must allow the following communications: • from the remote nodes: — TCP on ports 5302 — and allow only packets with the SYN flag — UDP on port 5302 • to the remote nodes: — TCP and UDP on dynamic ports The remote nodes must allow the following communications: • from the cluster nodes — TCP and UDP on dynamic ports • to the cluster nodes — TCP on ports 5302 — and allow only packets with the SYN flag — UDP on port 5302 Authentication communication must allow the following ports:
5. Finish the registration process. At this point you have a license key (an alphanumeric string), usually contained in a file that HP emails to you. It may also be possible to copy and paste the key directly from your browser window; in this case you will need to save the key in a file that you create. IMPORTANT: In either case, make sure you save the file and make a note of its path. (The name does not matter. In the Example (page 37) we have called it /mystuff/mySGlicense.
Example 1. 2. 3. mv $SGCONF/AutoPass/LicFile.txt $SGCONF/AutoPass/LicFile.txt.sav cp /mystuff/mySGlicense $SGCONF/AutoPass/LicFile.txt cmcheckconf –v –C where is the configuration file used to create the existing cluster. 4. • If the license is valid, keep the new license file $SGCONF/AutoPass/ LicFile.txt You can delete the old license file $SGCONF/AutoPass/LicFile.txt.sav. • If the license is invalid, restore the original license: mv $SGCONF/AutoPass/LicFile.txt.
Installing HP Serviceguard for Linux NOTE: This section provides a “manual” method for installing and configuring Serviceguard. HP also provides an automated method (scripts that take you through the process of installing and configuring Serviceguard for Linux, including Serviceguard Manager and ancillary software). The automated method is suitable for most installations. For more information and instructions see the HP Serviceguard for Linux Version A.11.19 Deployment Guide, which you can find on docs.hp.
— lm_sensors — tog-pegasus NOTE: HP recommends that after installing Linux, you enable ssh, and use it for remote login (instead of rlogin, remsh, or telnet). Use the related command scp (instead of rcp) for remote copy. ssh and scp encrypt passwords before transmitting them, whereas rlogin, rcp, etc., do not. After Linux is correctly configured, install HP Serviceguard using rpm commands: install Serviceguard, and then Serviceguard Manager and other ancillary products if you choose to do so.
SUSE Installation 1. On the Serviceguard for Linux CD, change to the appropriate directory: cd /Serviceguard/ For example cd SLES10/Serviceguard/IA64 2. Install the autopass rpm; for example, on an HP Integrity system: rpm -ihv SGLic-6.0-SG11.19.sles10.ia64.rpm 3. Install the sgcmom and serviceguard rpms; for example, on an HP Integrity system: rpm -ihv sgcmom-B.06.00.00-0.sles10.ia64.rpm rpm -ihv serviceguard-A.11.19.00-0.sles10.ia64.rpm 4.
2. Install the rpm as follows (all on one line): rpm -ihv cmsnmpd-A.02.00-0...rpm For example, on an HP Integrity system: rpm -ihv cmsnmpd-A.02.00-0.sles10.ia64.rpm Installing the WBEM Provider Proceed as follows if you need to install the WBEM Provider (for example, for use with Serviceguard Manager and HP SIM.) For Red Hat: 1.
For example: rpm -ihv tog-pegasus-2.5.3-1.sles10.ia64.rpm 3. Install the WBEM rpm as follows (all on one line): rpm -ihv sgproviders-A.03.00.00-0...rpm For example, rpm -ihv sgproviders-A.03.00.00-0.sles10.ia64.rpm Installing Serviceguard Manager for Linux Before You Start Before you start make sure you have done the following: 1. 2. 3. Installed Red Hat 5 or SLES 10 Installed Serviceguard for Linux A.11.19 Checked that libXp is installed.
NOTE: As of the date of publication of these Release Notes, HP recommends you use JDK version 1.6.0 Update 11. Make sure you write down the path to the JDK, for example /usr/java/ jdk1.6.0_11. You will need it to complete the hpsmh-tomcat installation; see “Installing the Serviceguard Manager Software”. NOTE: These URLs were correct at the time of publication, but may have changed in the meantime. 5. Updated your web browser if necessary.
• • rpm -ihv sgmgrpi-B.02.00.00-1.rhel5.noarch.rpm rpm -ihv hpdsau-2.4-1.rhel5.i386.rpm For Red Hat 5 (x86_64) Navigate to RedHat5/SGManager/x86_64 on the DVD and install the following: • • • • rpm rpm rpm rpm -ihv -ihv -ihv -ihv hpsmh-3.0.0-68.x86_64.rpm hpsmh-tomcat-1.0-11.linux.x86_64.rpm sgmgrpi-B.02.00.00-1.rhel5.noarch.rpm hpdsau-2.4-1.rhel5.x86_64.
Post-Installation Tasks When you have finished installing the Serviceguard Manager software, do the following. 1. Restart SMH to integrate SMH with Serviceguard Manager: /etc/init.d/hpsmhd restart 2. To verify the installation, open a web browser and navigate to http:// [hostname]:2301/ The System Management Home Page appears. Go to the SMH Tools and select Serviceguard Manager.
1. Choose from the list of disks to be configured for shared storage: fdisk -l Output such as the following appears. Choose a disk accessible from all the members of the cluster. The Id should be 8e and the System should be Linux LVM. For example, in this case it could be either /dev/sda1 or /dev/sdb1. This example uses /dev/sda1. Disk /dev/cciss/c0d0: 72.
10. On the other nodes in the cluster, re-read the disk header and scan for the file system: vgscan NOTE: node. If vgscan does not find the volume group, you will need to reboot the 11. Create a /clog mount point on each of the other cluster nodes, allowing the clog package to fail over in case of a node failure: mkdir /clog 12. Back up the configuration on each node: vgcfgbackup /dev/vgclog 13.
Installing the Quorum Server If you choose to use a Quorum Server rather than a lock LUN for tie-breaking, install the Quorum Server software on a system outside the cluster. The Quorum Server software is on the Serviceguard for Linux CD. Updated versions may be available from http://www.hp.com/go/softwaredepot -> High Availability -> Quorum Server. CAUTION: If you are upgrading both the Quorum Server and Serviceguard for Linux, upgrade the Quorum Server before you upgrade Serviceguard.
Rules 1. 2. 3. Serviceguard for Linux must be halted (not running) on the node from which the rpm -e command is issued. The node from which the rpm -e command is issued must not be a member of any cluster. The rpm -e command should be issued from one system at a time. That is, if Serviceguard for Linux is being uninstalled from more than one system, it should be removed from one system at a time.
NOTE: If the target version of Serviceguard does not support the version of the operating system currently on the node, then you must upgrade the operating system before upgrading Serviceguard. • • • • The OS upgrade for the node must be from and to the same Linux distribution (Example: Red Hat EL 4 to Red Hat EL 5; not Red Hat EL 4 to SLES 10) All members of the cluster must be upgraded to the same version of OS and the Serviceguard.
can take up to one second, during which time the old cluster manager will shut down and the new cluster manager will start. CAUTION: From the time when the old cluster manager is shut down until the new cluster manager forms its first cluster, a node failure will cause the entire cluster to fail. HP strongly recommends that you use no Serviceguard commands other than cmviewcl (1m) until the new cluster manager successfully completes its first cluster re-formation.
• • • • • • • • the software. Performing tasks on a node containing an earlier revision of the software will not work or will cause inconsistent results. You cannot modify the cluster or package configuration until the upgrade is complete. You cannot modify the hardware configuration—including the cluster’s network configuration—during rolling upgrade. This means that you must upgrade all nodes to the new release before you can modify the configuration file and copy it to all nodes.
NOTE: Make sure that all your network and storage interfaces are supported by the new OS. 3. Record the storage configuration, such as all LVM information, and if possible, collect a list of hardware disks configured, for example sfdisk -l On SUSE Linux Enterprise Server, you may need to run YAST or YAST2. 4. Back up the following files on media that can be easily recovered by the node after its upgrade or a new OS installation: • Host files: /root/.
5. Verify that all disks and their filesystems are the same prior to this OS upgrade or new installation. Check and compare with the disk layout collected before the upgrade. Use command vgscan to make sure the node with new OS sees all its previously configured LVM disks. 6. Install the new Serviceguard release. Follow the instructions under “Installing HP Serviceguard for Linux” (page 38). The following examples show only the Serviceguard rpm.
The deadman driver sources and associated files are located at $SGROOT/drivers. The pidentd (sg_pidentd for Red Hat 5) driver is located at: /usr/src/pidentd-/driver where version represents the version number for the pidentd that was last installed. Follow the instructions in the README file in the directory of each driver. If you have installed a new OS version, you must run a convert program. This will convert the binary file (cmclconfig) to its new release format.
NOTE: This and the following figures are from a use case in which the starting point of the upgrade was Serviceguard A.11.14.02 and Red Hat AS 2.1, and the rolling upgrade was to Serviceguard A.11.15.02 and Red Hat EL3. This is for illustration only; the upgrade process has not changed since. Figure 1-1 Running Cluster Before Rolling Upgrade Node1 Pkg1 SG A 11.14.02 Red Hat AS 2.1 Node2 Pkg2 SG A.11.14.02 Red Hat AS 2.1 Step 1. Record all data as specified in the preparation section.
Figure 1-2 Running Cluster with Packages Moved to Node 2 Node1 Node2 SG SG A.11.14.02 Pkg1 A.11.14.02 AS 2.1 Pkg2 AS 2.1 Step 2. Upgrade node1 to the new operating system release (in this example, it was Red Hat EL 3), and install the new version of Serviceguard, as shown in Figure 1-3. Figure 1-3 Node 1 Upgraded to Red Hat EL 3 Node1 Node2 Pkg1 Red Hat EL 3 Pkg2 SG A.11.14.02 Red Hat AS 2.1 Step 3.
Figure 1-4 Node 1 Rejoining the Cluster Step 4. Repeat the process on node2. Halt the node as follows: # cmhaltnode -f node2 This causes both packages to move to node1. Then upgrade node2 to the new Linux distribution and the new version of Serviceguard. Figure 1-5 Running Cluster with Packages Moved to Node 1 Node1 Pkg1 Pkg2 Node2 SG SG A.11.15.02 A.11.15.02 Red Hat EL 3 Red Hat EL 3 Step 5. Move pkg2 back to its original node.
# cmrunpkg -n node2 pkg2 # cmmodpkg -e pkg2 The cmmodpkg command re-enables switching of the package, which is disabled by the cmhaltpkg command. The final running cluster is shown in Figure 1-6. Figure 1-6 Running Cluster After Upgrades Node1 Pkg1 Node2 SG Pkg2 A.11.15.02 Red Hat EL 3 SG A.11.15.
Preventing Boot-Time vgscan and Ensuring Serviceguard Volume Groups Are Deactivated See Chapter 5 of Managing HP Serviceguard for Linux. Patches for this Version As of the date of publication, no patches are required for version A.11.19 of HP Serviceguard for Linux. Before installing Serviceguard, you should check the Hewlett-Packard IT Resource Center web page for any new patch requirements: http://itrc.hp.com (Americas and Asia Pacific) http://europe.itrc.hp.
• • • • • • • • • • • • • • • • • • • • QXCR1000588963 (JAGag44319): cmcluster boot script does not print errors of cmrunnode QXCR1000590010 (JAGag45586): cmmodnet: Non-root users are not authorized to perform this command QXCR1000590767 (JAGag46475): cmquerycl -f line output may display incorrect VG disk device files QXCR1000589971 (JAGag45533): cmviewconf does not display package service failfast flag correctly QXCR1000740150: cmcld abort when deleting node online QXCR1000741571: hang on one system, caus
• • • JAGag31375: SGeSAP missing from cluster extensions JAGag32353: With Mozilla or Firefox browser, configuring a SGeRAC package is confusing JAGag35535: Adding a node to a cluster with lock LUN fails Known Problems This section lists problems in Serviceguard for Linux Version A.11.19 known at the time of publication. This list is subject to change without notice.
• • QXCR1000877581 — Unresponsive pull-down menu selection using Firefox 3.0 QXCR1000887237 — Cannot detect that SGeSAP is installed on Red Hat Linux Software Availability in Native Languages The Serviceguard Manager plug-in provides Native Language Support, but the command line interface does not.
