Managing ServiceGuard NFS for Linux Manufacturing Part Number : T1442-90008 June 2003
Legal Notices The information in this document is subject to change without notice. Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. Hewlett-Packard shall not be held liable for errors contained herein or direct, indirect, special, incidental or consequential damages in connection with the furnishing, performance, or use of this material. Warranty.
ServiceGuard® is a registered trademark of Hewlett-Packard Company, and is protected by copyright. NFS® is a registered trademark of Sun Microsystems, Inc. NIS™ is a trademark of Sun Microsystems, Inc. UNIX® is a registered trademark of The Open Group. Linux® is a registered trademark of Linus Torvalds. Red Hat® is a registered trademark of Red Hat Software, Inc.
Contents 1. ServiceGuard NFS for LINUX Introduction Overview of ServiceGuard NFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Limitations of ServiceGuard NFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Supported Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Failover to an Idle Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents NFS Control Script for pkg03 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring One Adoptive Node to Support Failover of Multiple Packages. . . . . . . . . Cluster Configuration File for Adoptive Node for Two Packages . . . . . . . . . . . . . . . Package Configuration File for pkg01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Package Control Script for pkg01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ServiceGuard NFS for LINUX Introduction 1 ServiceGuard NFS for LINUX Introduction This manual describes how to install and configure an ServiceGuard NFS toolkit on a Linux system. ServiceGuard NFS® is a tool kit that allows you to use ServiceGuard to set up highly available NFS servers. The information presented in this manual assumes you are familiar with ServiceGuard and NFS operations. Refer to your ServiceGuard and/or NFS documentation for additional information.
ServiceGuard NFS for LINUX Introduction Overview of ServiceGuard NFS Overview of ServiceGuard NFS An NFS server is a host that “exports” its local directories (makes them available for client hosts to mount using NFS). On the NFS client, these mounted directories look to users like part of the client’s local file system. ServiceGuard allows you to create high availability clusters of HP Linux computers (nodes).
ServiceGuard NFS for LINUX Introduction Limitations of ServiceGuard NFS Limitations of ServiceGuard NFS The following limitations apply to ServiceGuard NFS: Chapter 1 • File locks are not maintained when an NFS server package moves to an adoptive node. Any applications that use file locking must reclaim their locks after an NFS server package fails over. An application that loses its file lock as a result of an NFS package failover will not be notified.
ServiceGuard NFS for LINUX Introduction Supported Configurations Supported Configurations ServiceGuard NFS supports the following configurations and are illustrated in the following sections: 10 • Simple failover from an active NFS server node to an idle NFS server node. • Failover from one active NFS server node to another active NFS server node, where the adoptive node supports more than one NFS package after the failover. • A host configured as an adoptive node for more than one NFS package.
ServiceGuard NFS for LINUX Introduction Supported Configurations Failover to an Idle Node Figure 1-1 shows a simple failover from an active NFS server node to an idle NFS server node. Figure 1-1 Simple Failover to an Idle NFS Server Node_A is the primary node for NFS server package Pkg_1. When Node_A fails, Node_B adopts Pkg_1. This means that Node_B locally mounts the file systems associated with Pkg_1 and exports them.
ServiceGuard NFS for LINUX Introduction Supported Configurations Failover between Active Nodes Figure 1-2 shows a failover from one active NFS server node to another active NFS server node. In Figure 1-2, Node_A is the primary node for Pkg_1, and Node_B is the primary node for Pkg_2. When Node_A fails, Node_B adopts Pkg_1 and becomes the server for both Pkg_1 and Pkg_2.
ServiceGuard NFS for LINUX Introduction Supported Configurations Failover with an Adoptive Node for Multiple Packages Figure 1-3 shows a three-node configuration where one node is the adoptive node for packages on both of the other nodes. If either Node_A or Node_C fails, Node_B adopts the NFS server package from that node. When Node_A fails, Node_B becomes the server for Pkg_1. If Node_C fails, Node_B will become the server for Pkg_2.
ServiceGuard NFS for LINUX Introduction Supported Configurations Cascading Failover with Multiple Adoptive Nodes Consider a package that is configured up to three adoptive nodes. Figure 1-4 shows this configuration. If Node_A fails, Pkg_1 is adopted by Node_B. However, if Node_B is down, Pkg_1 is adopted by Node_C, and if Node_C is down, Pkg_1 is adopted by Node_D.
ServiceGuard NFS for LINUX Introduction Understanding the ServiceGuard NFS Files Understanding the ServiceGuard NFS Files ServiceGuard NFS uses files similar and typical to ServiceGuard. These include configuration files, control scripts, monitoring scripts, and templates. As is true for all ServiceGuard packages, you configure and view a small number of files. The following is a brief description of the files: • Files that apply to the whole cluster: — cluster configuration file, cluster.
ServiceGuard NFS for LINUX Introduction How the Control and Monitor Scripts Work How the Control and Monitor Scripts Work As with all ServiceGuard packages, the package control scripts start and stop the NFS package and determine how the package will operate once it becomes available on a particular node. Each control script contains two sets of code that operate depending on whether the script is called with the start parameter or the stop parameter. A template package control script pkg.
ServiceGuard NFS for LINUX Introduction How the Control and Monitor Scripts Work Halting the NFS Services When called with the stop parameter, the control script does the following: • Removes the package IP address from the LAN card on the current node. • The package control script invokes the toolkit.sh to run the NFS script and to halt the NFS related process. • The NFS script un-exports all file systems associated with the package so that they can no longer be NFS-mounted by clients.
ServiceGuard NFS for LINUX Introduction How the Control and Monitor Scripts Work NOTE To configure NFS for maximal availability, you must do the following: • To have the NFS package start automatically when the cluster starts up, and to start on an adoptive node after a failure, you need to specify AUTO_RUN=YES in the package configuration file. • Also, the default NFS control script does not invoke the NFS monitoring script, nfs.mon.
Installing and Configuring ServiceGuard NFS for Linux 2 Installing and Configuring ServiceGuard NFS for Linux This chapter explains how to configure ServiceGuard NFS. NOTE You must set up your ServiceGuard cluster and make sure NFS server is installed before configuring ServiceGuard NFS. For instructions on setting up an ServiceGuard cluster, see Managing ServiceGuard for Linux, Chapters 5 and 6.
Installing and Configuring ServiceGuard NFS for Linux Installing ServiceGuard NFS for Linux Installing ServiceGuard NFS for Linux 1. Check for and remove any previous version of ServiceGuard NFS for Linux: Query the rpm database for the NFS Toolkit: # rpm -qa |grep nfstoolkit If any part of the NFS Toolkit is installed, the grep returns the version number. Erase older versions of the NFS Toolkit, if needed: # rpm -e nfstoolkit 2.
Installing and Configuring ServiceGuard NFS for Linux Installing ServiceGuard NFS for Linux • /usr/local/cmcluster/nfstoolkit/nfs.mon. The NFS monitor script. • /usr/bin/sync_rmtab. Remote mount table synchronization binary code. • /usr/local/cmcluster/nfstoolkit/toolkit.sh. The interface script between the package control script and hanfs.sh. 3.
Installing and Configuring ServiceGuard NFS for Linux Installing ServiceGuard NFS for Linux NOTE 24 pkg.cntl, toolkit.sh , and hanfs.sh should be in the same directory. Do not rename the hanfs.sh,toolkit.sh, and nfs.mon. These files are hard coded in the control scripts.
Installing and Configuring ServiceGuard NFS for Linux Before Creating an ServiceGuard NFS Package Before Creating an ServiceGuard NFS Package Before creating a ServiceGuard NFS package, perform the following tasks: NOTE The following procedures presume the environment is RedHat. If environment is SLES 8/UL, please replace all occurances of “/usr/local” with “/opt” 1. Select the NFS Server package during Red Hat Linux installation and verify that the NFS is properly installed.
Installing and Configuring ServiceGuard NFS for Linux Before Creating an ServiceGuard NFS Package If both are checked, NFS is included in Linux Kernel. 2. Set up your ServiceGuard cluster according to the instructions in the Managing ServiceGuard for Linux manual.
Installing and Configuring ServiceGuard NFS for Linux Before Creating an ServiceGuard NFS Package 3. Configure the disk hardware for high availability. Disks may be protected using disk mirroring or an HP High Availability Disk Array. Data disks associated with ServiceGuard NFS must be external disks. All the nodes that support the ServiceGuard NFS package must have access to the external disks.
Installing and Configuring ServiceGuard NFS for Linux Before Creating an ServiceGuard NFS Package 6. Create an entry for the name of the package in the DNS or NIS name resolution files, or in /etc/hosts, so that users will mount the exported file systems from the correct node. This entry maps the package name to the package’s relocatable IP address. 7. Decide whether to place executables locally on each client or on the NFS server.
Installing and Configuring ServiceGuard NFS for Linux Configuring a ServiceGuard NFS Package Configuring a ServiceGuard NFS Package To configure a ServiceGuard NFS package, complete the following tasks, described in this section: NOTE • Editing the Package Configuration File (pkg.conf) • Editing the Package Control Scripts (pkg.cntl) • Editing the NFS Control Script (hanfs.sh) • Creating the ServiceGuard Binary Configuration File Repeat the configuration process for each NFS package.
Installing and Configuring ServiceGuard NFS for Linux Configuring a ServiceGuard NFS Package NODE_NAME thyme NODE_NAME basil NODE_NAME sage 4. Set the RUN_SCRIPT and HALT_SCRIPT variables to the full path name of the control script. You do not have to specify a timeout for either script. For example: RUN_SCRIPT /usr/local/cmcluster/pkg1/pkg1.cntl RUN_SCRIPT_TIMEOUT NO_TIMEOUT HALT_SCRIPT /usr/local/cmcluster/pkg1/pkg1.cntl HALT_SCRIPT_TIMEOUT NO_TIMEOUT 5.
Installing and Configuring ServiceGuard NFS for Linux Configuring a ServiceGuard NFS Package Editing the Package Control Scripts (pkg.cntl) The following steps describe the required modifications to the Package Control Scripts. Make one Package Control Script for each package. NOTE The following procedures presume the environment is RedHat. If environment is SLES 8/UL, please replace all occurances of “/usr/local” with “/opt” 1.
Installing and Configuring ServiceGuard NFS for Linux Configuring a ServiceGuard NFS Package The IP address you specify is the relocatable IP address for the package. NFS clients that mount the file systems in the package will use this IP address to identify the server. You should configure a name for this address in the DNS or NIS database, or in the /etc/hosts file.
Installing and Configuring ServiceGuard NFS for Linux Configuring a ServiceGuard NFS Package 6. Specify that this package uses the high availability NFS server by uncommenting the HA_APP_SERVER variable. Uncomment the following line: HA_APP_SERVER=”pre-IP” 7. If two packages have the same adoptive node, and you want to prevent a shared adoptive node from adopting both packages at once, specify the cmmodpkg command with the package control option (-d) in the customer_defined_run_cmds.
Installing and Configuring ServiceGuard NFS for Linux Configuring a ServiceGuard NFS Package 8. Use the default values for the rest of the variables in the control script, or change them as needed. For instructions on modifying the default values, see the Managing ServiceGuard for Linux manual, or read the comments in the /usr/local/cmcluster/nfstoolkit/pkg.cntl template file.
Installing and Configuring ServiceGuard NFS for Linux Configuring a ServiceGuard NFS Package Editing the NFS Control Script (hanfs.sh) The following steps describe the required modifications to the NFS Control Script. NOTE The following procedures presume the environment is RedHat. If environment is SLES 8/UL, please replace all occurances of “/usr/local” with “/opt” 1. Create a separate XFS[n] variable for each NFS directory to be exported. Specify the directory name and any export options.
Installing and Configuring ServiceGuard NFS for Linux Configuring a ServiceGuard NFS Package 3. If you do not want to run the NFS monitor script: Comment out the NFS_SERVICE_NAME and NFS_SERVICE_CMD variables. For example: # NFS_SERVICE_NAME[0]=nfs1.monitor By default, the NFS_SERVICE_NAME and NFS_SERVICE_CMD variables are commented out, and the NFS monitor script is not run. Creating the ServiceGuard Binary Configuration File 1.
Sample Configurations 3 Sample Configurations This chapter gives sample cluster configuration files, package configuration files, package control script, and NFS control script for configurations supporting the following failover options: NOTE • Failover between multiple active nodes. The sample configuration has three servers and three ServiceGuard NFS packages and supports a three-server mutual takeover. Each server is the primary node for one package and an adoptive node for the other two packages.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Configuring Mulitple Nodes to Support Failover of Multiple Packages This configuration has three servers and three ServiceGuard NFS packages. Each server is the primary node for one package and an adoptive node for the other two packages. Figure 3-1 illustrates this configuration. Dotted lines indicate which servers are adoptive nodes for the packages. Figure 3-2 illustrates the configuration after host basil fails.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Figure 3-2 shows the three-server mutual takeover configuration after host basil has failed and host sage has adopted pkg02. Dotted lines indicate which servers are adoptive nodes for the packages.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Cluster Configuration File for Three-Server Mutual Takeover This section shows the cluster configuration file (cluster.conf) for this configuration example. The comments are not shown. 38 CLUSTER_NAME MutTakOvr QS_HOST QS_POLLING_INTERVAL qs 300000000 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP NETWORK_INTERFACE thyme eth0 192.100.112.146 eth1 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP basil eth0 192.100.112.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Package Configuration File for pkg01 This section shows the package configuration file (nfs1.conf) for the package pkg01 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Package Control Script for pkg01 This section shows the package control script (pkg1.cntl) for the package pkg01 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Package Configuration File for pkg02 This section shows the package configuration file (nfs2.conf) for the package pkg02 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Package Control Script for pkg02 This section shows the package control script (pkg2.cntl) for the package pkg02 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Package Configuration File for pkg03 This section shows the package configuration file (nfs3.conf) for the package pkg03 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Package Control Script for pkg03 This section shows the NFS control script (pkg3.cntl) for the package pkg03 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Configuring One Adoptive Node to Support Failover of Multiple Packages This configuration has two packages, each owned by a different server. The adoptive node for both packages is the same host. This sample configuration uses the package control option, which prevents the adoptive node from adopting another package if it has already adopted one. Figure 3-3 illustrates this configuration.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Figure 3-4 shows this sample configuration after host basil has failed. Host sage has adopted pkg02. NOTE Setting the package control option (-d) of the cmmodpkg command, prevents host sage from adopting another package, so host sage is no longer an adoptive node for pkg01. This prevent the adoptive node (sage) from becomming overloaded when multiple packages failover.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Cluster Configuration File for Adoptive Node for Two Packages This section shows the cluster configuration file (cluster.conf) for this configuration example. The comments are not shown. Chapter 3 CLUSTER_NAME PkgCtrl QS_HOST QS_POLLING_INTERVAL qs 300000000 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP NETWORK_INTERFACE thyme eth0 192.100.112.146 eth1 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP basil eth0 192.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Package Configuration File for pkg01 This section shows the package configuration file (pkg1.conf) for the package pkg01 in this sample configuration. The comments are not shown.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Package Control Script for pkg01 This section shows the package control script (pkg1.cntl) for the package pkg01 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages NFS Control Script for pkg01 This section shows the NFS control script (hanfs.sh) for the package pkg03 on this sample configuration on the the user-configured part of the script is shown: XFS[0]="-o rw *:/hanfs/nfsu011" NFS_SERVICE_NAME[0]="nfs1.monitor" NFS_SERVICE_CMD[0]="/usr/local/cmcluster/pkg1/nfs.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Package Configuration File for pkg02 This section shows the package configuration file (pkg2.conf) for the package pkg02 in this sample configuration. The comments are not shown.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Package Control Script for pkg02 This section shows the package control script (pkg2.cntl) for the package pkg02 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages NFS Control Script for pkg02 This section shows the NFS control script (hanfs.sh) for the package pkg02 on this sample configuration on the the user-configured part of the script is shown: XFS[0]="-o rw *:/hanfs/nfsu021" NFS_SERVICE_NAME[0]="nfs2.monitor" NFS_SERVICE_CMD[0]="/usr/local/cmcluster/pkg2/nfs.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Configuring Multiple Adoptive Nodes for Cascading Failover This configuration has two packages and three servers. One server is the primary node for both packages. The other servers are adoptive nodes for the two packages. Figure 3-5 illustrates this configuration. Dotted lines indicate which servers are adoptive nodes for the packages. Figure 3-6 illustrates the configuration after host thyme fails.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Figure 3-6 shows the cascading failover configuration after host thyme has failed. Host basil is the first adoptive node configured for pkg01, and host sage is the first adoptive node configured for pkg02.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Cluster Configuration File for Three-Server Cascading Failover This section shows the cluster configuration file (cluster.conf) for this configuration example. The comments are not shown. CLUSTER_NAME Cascading QS_HOST QS_POLLING_INTERVAL qs 300000000 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP NETWORK_INTERFACE thyme eth0 192.100.112.146 eth1 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP basil eth0 192.100.112.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Package Configuration File for pkg01 This section shows the package configuration file (pkg1.conf) for the package pkg01 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Package Control Script for pkg01 This section shows the package control script (pkg1.cntl) for the package pkg01 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Package Configuration File for pkg02 This section shows the package configuration file (pkg2.conf) for the package pkg02 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Package Control Script for pkg02 This section shows the package control script (pkg2.cntl) for the package pkg02 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Index A adoptive nodes, 10 configuring, 27 example of cascading failover, 54 example of package control option, 45 for multiple packages, 10, 13, 30, 45 illustration of cascading failover, 14 AUTO_RUN, 28 automounter timeout, 19 B binary configuration files, creating, 33 C cascading failover, 10 example configuration, 54 illustration of, 14 client behavior, 8, 19 cluster configuration file (cluster.
Index L lockd monitoring, 18 locked files, during package failover, 9 logging, NFS monitor script, 19 logical volumes configuration, 25 specifying in nfs.cntl, 29 LV variable, in nfs.cntl script, 29 M ServiceGuard documentation, 24 MD devices activation, 17 deactivation, 18 monitor script (nfs.mon), 18 logging, 19 specified in nfs.cntl, 32 specified in nfs.
Index T timeout, automounter, 19 U unexporting file systems, 18 unmounting file systems, 18 user IDs, 25 V VG variable, in nfs.cntl script, 29 volume groups activating, 17 configuring, 25 deactivating, 18 major and minor numbers, 25 specifying in nfs.cntl, 29 X XFS variable, in nfs.