Managing Serviceguard NFS for Linux Manufacturing Part Number : T1442-90014 September 2006
Legal Notices © Copyright 2006 Hewlett-Packard Development Company, L.P. Publication Date: 2006 Confidential computer software. Valid license from HP required for possession, use, or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor’s standard commercial license. The information contained herein is subject to change without notice.
Contents 1. Serviceguard NFS for LINUX Introduction Overview of Serviceguard NFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Limitations of Serviceguard NFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Supported Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Failover to an Idle Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents Package Control Script for pkg03 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . NFS Control Script for pkg03 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring One Adoptive Node to Support Failover of Multiple Packages. . . . . . . . . Cluster Configuration File for Adoptive Node for Two Packages . . . . . . . . . . . . . . . Package Configuration File for pkg01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Serviceguard NFS for LINUX Introduction 1 Serviceguard NFS for LINUX Introduction This manual describes how to install and configure an Serviceguard NFS toolkit on a Linux system. Serviceguard NFS® is a toolkit that allows you to use Serviceguard to set up highly available NFS servers. The information presented in this manual assumes you are familiar with Serviceguard and NFS operations. Refer to your Serviceguard and/or NFS documentation for additional information.
Serviceguard NFS for LINUX Introduction Overview of Serviceguard NFS Overview of Serviceguard NFS An NFS server is a host that “exports” its local directories (makes them available for client hosts to mount using NFS). On the NFS client, these mounted directories look to users like part of the client’s local file system. Serviceguard allows you to create high availability clusters of HP Linux computers (nodes).
Serviceguard NFS for LINUX Introduction Limitations of Serviceguard NFS Limitations of Serviceguard NFS The following limitations apply to Serviceguard NFS: Chapter 1 • File locks are not maintained when an NFS server package moves to an adoptive node. Any applications that use file locking must reclaim their locks after an NFS server package fails over. An application that loses its file lock as a result of an NFS package failover will not be notified.
Serviceguard NFS for LINUX Introduction Supported Configurations Supported Configurations Serviceguard NFS supports the following configurations and are illustrated in the following sections: 8 • Simple failover from an active NFS server node to an idle NFS server node. • Failover from one active NFS server node to another active NFS server node, where the adoptive node supports more than one NFS package after the failover. • A host configured as an adoptive node for more than one NFS package.
Serviceguard NFS for LINUX Introduction Supported Configurations Failover to an Idle Node Figure 1-1 shows a simple failover from an active NFS server node to an idle NFS server node. Figure 1-1 Simple Failover to an Idle NFS Server Node_A is the primary node for NFS server package Pkg_1. When Node_A fails, Node_B adopts Pkg_1. This means that Node_B locally mounts the file systems associated with Pkg_1 and exports them.
Serviceguard NFS for LINUX Introduction Supported Configurations Failover between Active Nodes Figure 1-2 shows a failover from one active NFS server node to another active NFS server node. In Figure 1-2, Node_A is the primary node for Pkg_1, and Node_B is the primary node for Pkg_2. When Node_A fails, Node_B adopts Pkg_1 and becomes the server for both Pkg_1 and Pkg_2.
Serviceguard NFS for LINUX Introduction Supported Configurations Failover with an Adoptive Node for Multiple Packages Figure 1-3 shows a three-node configuration where one node is the adoptive node for packages on both of the other nodes. If either Node_A or Node_C fails, Node_B adopts the NFS server package from that node. When Node_A fails, Node_B becomes the server for Pkg_1. If Node_C fails, Node_B will become the server for Pkg_2.
Serviceguard NFS for LINUX Introduction Supported Configurations Cascading Failover with Multiple Adoptive Nodes Consider a package that is configured up to three adoptive nodes. Figure 1-4 shows this configuration. If Node_A fails, Pkg_1 is adopted by Node_B. However, if Node_B is down, Pkg_1 is adopted by Node_C, and if Node_C is down, Pkg_1 is adopted by Node_D.
Serviceguard NFS for LINUX Introduction Understanding the Serviceguard NFS Files Understanding the Serviceguard NFS Files Serviceguard NFS uses files similar and typical to Serviceguard. These include configuration files, control scripts, monitoring scripts, and templates. As is true for all Serviceguard packages, you configure and view a small number of files. The following is a brief description of the files: • Files that apply to the whole cluster: — cluster configuration file, cluster.
Serviceguard NFS for LINUX Introduction How the Control and Monitor Scripts Work How the Control and Monitor Scripts Work As with all Serviceguard packages, the package control scripts start and stop the NFS package and determine how the package will operate once it becomes available on a particular node. Each control script contains two sets of code that operate depending on whether the script is called with the start parameter or the stop parameter. A template package control script pkg.
Serviceguard NFS for LINUX Introduction How the Control and Monitor Scripts Work Halting the NFS Services When called with the stop parameter, the control script does the following: • Removes the package IP address from the LAN card on the current node. • The package control script invokes the toolkit.sh to run the NFS script and to halt the NFS related process. • The NFS script un-exports all file systems associated with the package so that they can no longer be NFS-mounted by clients.
Serviceguard NFS for LINUX Introduction How the Control and Monitor Scripts Work NOTE To configure NFS for maximal availability, you must do the following: • To have the NFS package start automatically when the cluster starts up, and to start on an adoptive node after a failure, you need to specify AUTO_RUN=YES in the package configuration file. • Also, the default NFS control script does not invoke the NFS monitoring script, nfs.mon.
Installing and Configuring Serviceguard NFS for Linux 2 Installing and Configuring Serviceguard NFS for Linux This chapter explains how to configure Serviceguard NFS. NOTE You must set up your Serviceguard cluster and make sure NFS server is installed before configuring Serviceguard NFS. For instructions on setting up an Serviceguard cluster, see Chapters 5 and 6 of the Managing Serviceguard for Linux user’s guide.
Installing and Configuring Serviceguard NFS for Linux Installing Serviceguard NFS for Linux Installing Serviceguard NFS for Linux The following descibes the Serviceguard NFS for Linux installation process. NOTE The following procedures assume you are working in a RedHat environment. If your environment is SLES, replace all occurrences of /usr/local with /opt. 1.
Installing and Configuring Serviceguard NFS for Linux Installing Serviceguard NFS for Linux The files will be installed in the /usr/local/cmcluster/nfstoolkit and /usr/bin directories. The following files are part of the toolkit: NOTE The following procedures assume your environment is RedHat. If your environment is SLES, replace all occurences of “/usr/local” with “/opt”. • /usr/local/cmcluster/nfstoolkit/README. Description of the toolkit contents. • /usr/local/cmcluster/nfstoolkit/hanfs.conf.
Installing and Configuring Serviceguard NFS for Linux Installing Serviceguard NFS for Linux 5. Issue the following command to copy the Serviceguard NFS template files to the newly created package directory: # cp /usr/local/cmcluster/nfstoolkit/* \ /usr/local/cmcluster/ Copying the Template Files If you will run only one Serviceguard NFS package in your Serviceguard cluster, technically you do not have to copy the template files.
Installing and Configuring Serviceguard NFS for Linux Before Creating an Serviceguard NFS Package Before Creating an Serviceguard NFS Package Before creating a Serviceguard NFS package, perform the following tasks: NOTE The following procedures assume your environment is RedHat. If your environment is SLES, replace all occurences of “/usr/local” with “/opt”. 1. Select the NFS Server package during Red Hat Linux installation and verify that the NFS is properly installed.
Installing and Configuring Serviceguard NFS for Linux Before Creating an Serviceguard NFS Package — From the File Systems window, click the Network File Systems button. — Verify that both NFS file system support and NFS server support items are checked. If both are checked, NFS is included in Linux Kernel. 2. Set up your Serviceguard cluster according to the instructions in the Managing Serviceguard for Linux user’s guide.
Installing and Configuring Serviceguard NFS for Linux Before Creating an Serviceguard NFS Package 3. Configure the disk hardware for high availability. Disks may be protected using disk mirroring or an HP High Availability Disk Array. Data disks associated with Serviceguard NFS must be external disks. All the nodes that support the Serviceguard NFS package must have access to the external disks.
Installing and Configuring Serviceguard NFS for Linux Before Creating an Serviceguard NFS Package 6. Create an entry for the name of the package in the DNS or NIS name resolution files, or in /etc/hosts, so that users will mount the exported file systems from the correct node. This entry maps the package name to the package’s relocatable IP address. 7. Decide whether to place executables locally on each client or on the NFS server.
Installing and Configuring Serviceguard NFS for Linux Configuring a Serviceguard NFS Package Configuring a Serviceguard NFS Package To configure a Serviceguard NFS package, complete the following tasks, described in this section: NOTE • Editing the Package Configuration File (pkg.conf) • Editing the Package Control Scripts (pkg.cntl) • Editing the NFS Configuration File (hanfs.conf) • Editing the NFS Control Script (hanfs.
Installing and Configuring Serviceguard NFS for Linux Configuring a Serviceguard NFS Package NODE_NAME thyme NODE_NAME basil NODE_NAME sage 4. Set the RUN_SCRIPT and HALT_SCRIPT variables to the full path name of the control script. You do not have to specify a timeout for either script. For example: RUN_SCRIPT /usr/local/cmcluster/pkg1/pkg1.cntl RUN_SCRIPT_TIMEOUT NO_TIMEOUT HALT_SCRIPT /usr/local/cmcluster/pkg1/pkg1.cntl HALT_SCRIPT_TIMEOUT NO_TIMEOUT 5.
Installing and Configuring Serviceguard NFS for Linux Configuring a Serviceguard NFS Package Editing the Package Control Scripts (pkg.cntl) The following steps describe the required modifications to the Package Control Scripts. Make one Package Control Script for each package. NOTE The following procedures assume your environment is RedHat. If your environment is SLES, replace all occurences of “/usr/local” with “/opt”. 1.
Installing and Configuring Serviceguard NFS for Linux Configuring a Serviceguard NFS Package The IP address you specify is the relocatable IP address for the package. NFS clients that mount the file systems in the package will use this IP address to identify the server. You should configure a name for this address in the DNS or NIS database, or in the /etc/hosts file.
Installing and Configuring Serviceguard NFS for Linux Configuring a Serviceguard NFS Package 6. Specify that this package uses the high availability NFS server by uncommenting the HA_APP_SERVER variable. Uncomment the following line: HA_APP_SERVER=”pre-IP” 7. If two packages have the same adoptive node, and you want to prevent a shared adoptive node from adopting both packages at once, specify the cmmodpkg command with the package control option (-d) in the customer_defined_run_cmds.
Installing and Configuring Serviceguard NFS for Linux Configuring a Serviceguard NFS Package 8. Use the default values for the rest of the variables in the control script, or change them as needed. For instructions on modifying the default values, see the Managing Serviceguard for Linux manual, or read the comments in the /usr/local/cmcluster/nfstoolkit/pkg.cntl template file. Editing the NFS Configuration File (hanfs.
Installing and Configuring Serviceguard NFS for Linux Configuring a Serviceguard NFS Package happens if the server is an adoptive node for a file system, and the file system is available on the server only after failover of the primary node. 2. If you want to run the NFS monitor script: • Set the NFS_SERVICE_NAME variable to the value of the SERVICE_NAME variable in the package configuration file. Each package must have a unique service name. For example: NFS_SERVICE_NAME[0]=nfs1.
Installing and Configuring Serviceguard NFS for Linux Configuring a Serviceguard NFS Package 2. Use your favorite copy utility (for example, ftp or rcp) to copy the package control, NFS control, and monitor scripts to the same path names on all the nodes in the cluster. For example, to copy the files from host thyme to host basil, issue the following command from host thyme: # rcp /usr/local/cmcluster/cluster.conf/pkg1/* \ basil:/usr/local/cmcluster/cluster.
Sample Configurations 3 Sample Configurations This chapter gives sample cluster configuration files, package configuration files, package control script, and NFS control script for configurations supporting the following failover options: NOTE • Failover between multiple active nodes. The sample configuration has three servers and three Serviceguard NFS packages and supports a three-server mutual takeover. Each server is the primary node for one package and an adoptive node for the other two packages.
Sample Configurations Configuring Multiple Nodes to Support Failover of Multiple Packages Configuring Multiple Nodes to Support Failover of Multiple Packages This configuration has three servers and three Serviceguard NFS packages. Each server is the primary node for one package and an adoptive node for the other two packages. Figure 3-1 illustrates this configuration. Dotted lines indicate which servers are adoptive nodes for the packages. Figure 3-2 illustrates the configuration after host basil fails.
Sample Configurations Configuring Multiple Nodes to Support Failover of Multiple Packages Figure 3-2 shows the three-server mutual takeover configuration after host basil has failed and host sage has adopted pkg02. Dotted lines indicate which servers are adoptive nodes for the packages.
Sample Configurations Configuring Multiple Nodes to Support Failover of Multiple Packages Cluster Configuration File for Three-Server Mutual Takeover This section shows the cluster configuration file (cluster.conf) for this configuration example. The comments are not shown. 36 CLUSTER_NAME MutTakOvr QS_HOST QS_POLLING_INTERVAL qs 300000000 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP NETWORK_INTERFACE thyme eth0 192.100.112.146 eth1 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP basil eth0 192.100.112.
Sample Configurations Configuring Multiple Nodes to Support Failover of Multiple Packages Package Configuration File for pkg01 This section shows the package configuration file (nfs1.conf) for the package pkg01 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Multiple Nodes to Support Failover of Multiple Packages Package Control Script for pkg01 This section shows the package control script (pkg1.cntl) for the package pkg01 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring Multiple Nodes to Support Failover of Multiple Packages Package Configuration File for pkg02 This section shows the package configuration file (nfs2.conf) for the package pkg02 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Multiple Nodes to Support Failover of Multiple Packages Package Control Script for pkg02 This section shows the package control script (pkg2.cntl) for the package pkg02 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring Multiple Nodes to Support Failover of Multiple Packages Package Configuration File for pkg03 This section shows the package configuration file (nfs3.conf) for the package pkg03 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Multiple Nodes to Support Failover of Multiple Packages Package Control Script for pkg03 This section shows the NFS control script (pkg3.cntl) for the package pkg03 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Configuring One Adoptive Node to Support Failover of Multiple Packages This configuration has two packages, each owned by a different server. The adoptive node for both packages is the same host. This sample configuration uses the package control option, which prevents the adoptive node from adopting another package if it has already adopted one. Figure 3-3 illustrates this configuration.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Figure 3-4 shows this sample configuration after host basil has failed. Host sage has adopted pkg02. NOTE Setting the package control option (-d) of the cmmodpkg command, prevents host sage from adopting another package, so host sage is no longer an adoptive node for pkg01. This prevent the adoptive node (sage) from becoming overloaded when multiple packages failover.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Cluster Configuration File for Adoptive Node for Two Packages This section shows the cluster configuration file (cluster.conf) for this configuration example. The comments are not shown. Chapter 3 CLUSTER_NAME PkgCtrl QS_HOST QS_POLLING_INTERVAL qs 300000000 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP NETWORK_INTERFACE thyme eth0 192.100.112.146 eth1 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP basil eth0 192.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Package Configuration File for pkg01 This section shows the package configuration file (pkg1.conf) for the package pkg01 in this sample configuration. The comments are not shown.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Package Control Script for pkg01 This section shows the package control script (pkg1.cntl) for the package pkg01 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages NFS Control Script for pkg01 This section shows the NFS control script (hanfs.sh) for the package pkg03 on this sample configuration on the user-configured part of the script is shown: XFS[0]="-o rw *:/hanfs/nfsu011" NFS_SERVICE_NAME[0]="nfs1.monitor" NFS_SERVICE_CMD[0]="/usr/local/cmcluster/pkg1/nfs.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Package Configuration File for pkg02 This section shows the package configuration file (pkg2.conf) for the package pkg02 in this sample configuration. The comments are not shown.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Package Control Script for pkg02 This section shows the package control script (pkg2.cntl) for the package pkg02 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages NFS Control Script for pkg02 This section shows the NFS control script (hanfs.sh) for the package pkg02 on this sample configuration on the user-configured part of the script is shown: XFS[0]="-o rw *:/hanfs/nfsu021" NFS_SERVICE_NAME[0]="nfs2.monitor" NFS_SERVICE_CMD[0]="/usr/local/cmcluster/pkg2/nfs.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Configuring Multiple Adoptive Nodes for Cascading Failover This configuration has two packages and three servers. One server is the primary node for both packages. The other servers are adoptive nodes for the two packages. Figure 3-5 illustrates this configuration. Dotted lines indicate which servers are adoptive nodes for the packages. Figure 3-6 illustrates the configuration after host thyme fails.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Figure 3-6 shows the cascading failover configuration after host thyme has failed. Host basil is the first adoptive node configured for pkg01, and host sage is the first adoptive node configured for pkg02.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Cluster Configuration File for Three-Server Cascading Failover This section shows the cluster configuration file (cluster.conf) for this configuration example. The comments are not shown. CLUSTER_NAME Cascading QS_HOST QS_POLLING_INTERVAL qs 300000000 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP NETWORK_INTERFACE thyme eth0 192.100.112.146 eth1 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP basil eth0 192.100.112.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Package Configuration File for pkg01 This section shows the package configuration file (pkg1.conf) for the package pkg01 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Package Control Script for pkg01 This section shows the package control script (pkg1.cntl) for the package pkg01 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Package Configuration File for pkg02 This section shows the package configuration file (pkg2.conf) for the package pkg02 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Package Control Script for pkg02 This section shows the package control script (pkg2.cntl) for the package pkg02 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Index A adoptive nodes, 8 configuring, 25 example of cascading failover, 52 example of package control option, 43 for multiple packages, 8, 11, 29, 43 illustration of cascading failover, 12 AUTO_RUN, 26 automounter timeout, 16 B binary configuration files, creating, 31 C cascading failover, 8 example configuration, 52 illustration of, 12 client behavior, 6, 16 cluster configuration file (cluster.
Index L lockd monitoring, 15 locked files, during package failover, 7 logging, NFS monitor script, 16 logical volumes configuration, 23 specifying in nfs.cntl, 27 LV variable, in nfs.cntl script, 27 M MC/ServiceGuard documentation, 22 MD devices activation, 14 deactivation, 15 monitor script (nfs.mon), 15 logging, 16 specified in nfs.cntl, 31 specified in nfs.
Index T timeout, automounter, 16 U unexporting file systems, 15 unmounting file systems, 15 user IDs, 23 V VG variable, in nfs.cntl script, 27 volume groups activating, 14 configuring, 23 deactivating, 15 major and minor numbers, 23 specifying in nfs.cntl, 27 X XFS variable, in nfs.
