Managing MC/ServiceGuard NFS for Linux Manufacturing Part Number: T1442-90002 November 2001
Legal Notices The information in this document is subject to change without notice. Hewlett-Packard makes no warranty of any kind with regard to this manual, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. Hewlett-Packard shall not be held liable for errors contained herein or direct, indirect, special, incidental or consequential damages in connection with the furnishing, performance, or use of this material. Warranty.
NFS® is a registered trademark of Sun Microsystems, Inc. NIS™ is a trademark of Sun Microsystems, Inc. UNIX® is a registered trademark of The Open Group. Linux® is a registered trademark of Linus Torvalds. Red Hat® is a registered trademark of Red Hat Software, Inc.
Contents 1. MC/ServiceGuard NFS for LINUX Introduction Overview of MC/ServiceGuard NFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Limitations of MC/ServiceGuard NFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Supported Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Failover to an Idle Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents NFS Control Script for pkg03 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring One Adoptive Node to Support Failover of Multiple Packages . . . . . . . . Cluster Configuration File for Adoptive Node for Two Packages . . . . . . . . . . . . . . . Package Configuration File for pkg01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Package Control Script for pkg01 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
MC/ServiceGuard NFS for LINUX Introduction 1 MC/ServiceGuard NFS for LINUX Introduction This manual describes how to install and configure an MC/ServiceGuard NFS toolkit on a Linux system. MC/ServiceGuard NFS® is a tool kit that allows you to use MC/ServiceGuard to set up highly available NFS servers. The information presented in this manual assumes you are familiar with MC/ServiceGuard and NFS operations. Refer to your MC/ServiceGuard and/or NFS documentation for additional information.
MC/ServiceGuard NFS for LINUX Introduction Overview of MC/ServiceGuard NFS Overview of MC/ServiceGuard NFS An NFS server is a host that “exports” its local directories (makes them available for client hosts to mount using NFS). On the NFS client, these mounted directories look to users like part of the client’s local file system. MC/ServiceGuard allows you to create high availability clusters of HP Linux computers (nodes).
MC/ServiceGuard NFS for LINUX Introduction Limitations of MC/ServiceGuard NFS Limitations of MC/ServiceGuard NFS The following limitations apply to MC/ServiceGuard NFS: Chapter 1 • File locks are not maintained when an NFS server package moves to an adoptive node. Any applications that use file locking must reclaim their locks after an NFS server package fails over. An application that loses its file lock as a result of an NFS package failover will not be notified.
MC/ServiceGuard NFS for LINUX Introduction Supported Configurations Supported Configurations MC/ServiceGuard NFS supports the following configurations and are illustrated in the following sections: 10 • Simple failover from an active NFS server node to an idle NFS server node. • Failover from one active NFS server node to another active NFS server node, where the adoptive node supports more than one NFS package after the failover.
MC/ServiceGuard NFS for LINUX Introduction Supported Configurations Failover to an Idle Node Figure 1-1 shows a simple failover from an active NFS server node to an idle NFS server node. Figure 1-1 Simple Failover to an Idle NFS Server Node_A is the primary node for NFS server package Pkg_1. When Node_A fails, Node_B adopts Pkg_1. This means that Node_B locally mounts the file systems associated with Pkg_1 and exports them.
MC/ServiceGuard NFS for LINUX Introduction Supported Configurations Failover between Active Nodes Figure 1-2 shows a failover from one active NFS server node to another active NFS server node. In Figure 1-2, Node_A is the primary node for Pkg_1, and Node_B is the primary node for Pkg_2. When Node_A fails, Node_B adopts Pkg_1 and becomes the server for both Pkg_1 and Pkg_2.
MC/ServiceGuard NFS for LINUX Introduction Supported Configurations Failover with an Adoptive Node for Multiple Packages Figure 1-3 shows a three-node configuration where one node is the adoptive node for packages on both of the other nodes. If either Node_A or Node_C fails, Node_B adopts the NFS server package from that node. When Node_A fails, Node_B becomes the server for Pkg_1. If Node_C fails, Node_B will become the server for Pkg_2.
MC/ServiceGuard NFS for LINUX Introduction Supported Configurations Cascading Failover with Multiple Adoptive Nodes A package may be configured with up to three adoptive nodes. Figure 1-4 shows this configuration. If Node_A fails, Pkg_1 is adopted by Node_B. However, if Node_B is down, Pkg_1 is adopted by Node_C, and if Node_C is down, Pkg_1 is adopted by Node_D.
MC/ServiceGuard NFS for LINUX Introduction Understanding the MC/ServiceGuard NFS Files Understanding the MC/ServiceGuard NFS Files MC/ServiceGuard NFS uses files similar and typical to ServiceGuard. These include configuration files, control scripts, monitoring scripts, and templates. As is true for all ServiceGuard packages, you configure and view a small number of files. Two of these, nfs.mon and ha.nfs are new to MC/ServiceGuard NFS.
MC/ServiceGuard NFS for LINUX Introduction How the Control and Monitor Scripts Work How the Control and Monitor Scripts Work As with all ServiceGuard packages, the package control scripts start and stop the NFS package and determine how the package will operate once it becomes available on a particular node. Each control script contains two sets of code that operate depending on whether the script is called with the start parameter or the stop parameter. A template package control script pkg.
MC/ServiceGuard NFS for LINUX Introduction How the Control and Monitor Scripts Work Halting the NFS Services When called with the stop parameter, the control script does the following: • Removes the package IP address from the LAN card on the current node. • The package control script invoke the NFS script to halt the NFS related process. • The NFS script un-exports all file systems associated with the package so that they can no longer be NFS-mounted by clients.
MC/ServiceGuard NFS for LINUX Introduction How the Control and Monitor Scripts Work NOTE To configure NFS for maximal availability, you must do the following: • To have the NFS package start automatically when the cluster starts up, and to start on an adoptive node after a failure, you need to specify AUTO_RUN=YES in the package configuration file. • Also, the default NFS control script does not invoke the NFS monitoring script, nfs.mon.
Installing and Configuring MC/ServiceGuard NFS for Linux 2 Installing and Configuring MC/ServiceGuard NFS for Linux This chapter explains how to configure MC/ServiceGuard NFS. NOTE You must set up your MC/ServiceGuard cluster and make sure NFS server is installed before configuring MC/ServiceGuard NFS. For instructions on setting up an MC/ServiceGuard cluster, see Managing MC/ServiceGuard for Linux, Chapters 5 and 6.
Installing and Configuring MC/ServiceGuard NFS for Linux Installing MC/ServiceGuard NFS for Linux Installing MC/ServiceGuard NFS for Linux 1. Check for and remove any previous version of MC/ServiceGuard NFS for Linux: Query the rpm database for the NFS Toolkit: # rpm -qa |grep nfstoolkit If any part of the NFS Toolkit is installed, the grep returns the version number. Erase older versions of the NFS Toolkit, if needed: # rpm -e nfstoolkit 2.
Installing and Configuring MC/ServiceGuard NFS for Linux Installing MC/ServiceGuard NFS for Linux # cmmakepkg -p pkg.conf # cmmakepkg -s pkg.cntl 4. Create a directory for your package files, for example: # mkdir /usr/local/cmcluster/ 5.
Installing and Configuring MC/ServiceGuard NFS for Linux Before Creating an MC/ServiceGuard NFS Package Before Creating an MC/ServiceGuard NFS Package Before creating a MC/ServiceGuard NFS package, perform the following tasks: 1. Select the NFS Server package during Red Hat Linux installation and verify that the NFS is properly installed.
Installing and Configuring MC/ServiceGuard NFS for Linux Before Creating an MC/ServiceGuard NFS Package 3. Configure the disk hardware for high availability. Disks may be protected using disk mirroring or an HP High Availability Disk Array. Data disks associated with MC/ServiceGuard NFS must be external disks. All the nodes that support the MC/ServiceGuard NFS package must have access to the external disks.
Installing and Configuring MC/ServiceGuard NFS for Linux Before Creating an MC/ServiceGuard NFS Package 6. Create an entry for the name of the package in the DNS or NIS name resolution files, or in /etc/hosts, so that users will mount the exported file systems from the correct node. This entry maps the package name to the package’s relocatable IP address. 7. Decide whether to place executables locally on each client or on the NFS server.
Installing and Configuring MC/ServiceGuard NFS for Linux Configuring a MC/ServiceGuard NFS Package Configuring a MC/ServiceGuard NFS Package To configure a MC/ServiceGuard NFS package, complete the following tasks, described in this section: NOTE • Editing the Package Configuration File (pkg.conf) • Editing the Package Control Scripts (pkg.cntl) • Editing the NFS Control Script (hanfs.
Installing and Configuring MC/ServiceGuard NFS for Linux Configuring a MC/ServiceGuard NFS Package NODE_NAME thyme NODE_NAME basil NODE_NAME sage 4. Set the RUN_SCRIPT and HALT_SCRIPT variables to the full path name of the control script. You do not have to specify a timeout for either script. For example: RUN_SCRIPT /usr/local/cmcluster/pkg1/pkg1.cntl RUN_SCRIPT_TIMEOUT NO_TIMEOUT HALT_SCRIPT /usr/local/cmcluster/pkg1/pkg1.cntl HALT_SCRIPT_TIMEOUT NO_TIMEOUT 5.
Installing and Configuring MC/ServiceGuard NFS for Linux Configuring a MC/ServiceGuard NFS Package Editing the Package Control Scripts (pkg.cntl) The following steps describe the required modifications to the Package Control Scripts. Make one Package Control Script for each package. 1. Specify the configuration file that will be used to define the MD raid devices for each package. For example: RAIDTAB=”/usr/local/cmcluster/conf/raidtab.sg” 2. Specify which MD devices are used by this package.
Installing and Configuring MC/ServiceGuard NFS for Linux Configuring a MC/ServiceGuard NFS Package 6. Specify that this package uses the high availability NFS server by uncommenting the HA_NFS_SERVER variable. For example: HA_NFS_SERVER=”yes” 7. If two packages have the same adoptive node, and you want to prevent a shared adoptive node from adopting both packages at once, specify the cmmodpkg command with the package control option (-d) in the customer_defined_run_cmds.
Installing and Configuring MC/ServiceGuard NFS for Linux Configuring a MC/ServiceGuard NFS Package Editing the NFS Control Script (hanfs.sh) The following steps describe the required modifications to the NFS Control Script. 1. Create a separate XFS[n] variable for each NFS directory to be exported. Specify the directory name and any export options. For example: XFS[0]=”*:/ha_root” XFS[1]=”*:/users/scaf” XFS[2]=”-o ro *:/ha_data” Do not configure these exported directories in the /etc/exports file.
Installing and Configuring MC/ServiceGuard NFS for Linux Configuring a MC/ServiceGuard NFS Package Creating the MC/ServiceGuard Binary Configuration File 1. Use the cmapplyconf command to verify the content of your cluster and package configuration and to copy the binary configuration file to all the nodes in the cluster. In the following example, the cluster configuration file is /usr/local/cmcluster/cluster.conf.
Sample Configurations 3 Sample Configurations This chapter gives sample cluster configuration files, package configuration files, package control script, and NFS control script for configurations supporting the following failover options: • Failover between multiple active nodes. The sample configuration has three servers and three MC/ServiceGuard NFS packages and supports a three-server mutual takeover. Each server is the primary node for one package and an adoptive node for the other two packages.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Configuring Mulitple Nodes to Support Failover of Multiple Packages This configuration has three servers and three MC/ServiceGuard NFS packages. Each server is the primary node for one package and an adoptive node for the other two packages. Figure 3-1 illustrates this configuration. Dotted lines indicate which servers are adoptive nodes for the packages.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Figure 3-2 Chapter 3 Three-Server Mutual Takeover After One Server Fails 33
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Cluster Configuration File for Three-Server Mutual Takeover This section shows the cluster configuration file (cluster.conf) for this configuration example. The comments are not shown. 34 CLUSTER_NAME MutTakOvr QS_HOST QS_POLLING_INTERVAL qs 300000000 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP NETWORK_INTERFACE thyme eth0 192.100.112.146 eth1 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP basil eth0 192.100.112.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Package Configuration File for pkg01 This section shows the package configuration file (nfs1.conf) for the package pkg01 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Package Control Script for pkg01 This section shows the package control script (pkg1.cntl) for the package pkg01 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Package Configuration File for pkg02 This section shows the package configuration file (nfs2.conf) for the package pkg02 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Package Control Script for pkg02 This section shows the package control script (pkg2.cntl) for the package pkg02 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Package Configuration File for pkg03 This section shows the package configuration file (nfs3.conf) for the package pkg03 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Mulitple Nodes to Support Failover of Multiple Packages Package Control Script for pkg03 This section shows the NFS control script (pkg3.cntl) for the package pkg03 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Configuring One Adoptive Node to Support Failover of Multiple Packages This configuration has two packages, each owned by a different server. The adoptive node for both packages is the same host. This sample configuration uses the package control option, which prevents the adoptive node from adopting another package if it has already adopted one. Figure 3-3 illustrates this configuration.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Figure 3-4 shows this sample configuration after host basil has failed. Host sage has adopted pkg02. NOTE Setting the package control option (-d) of the cmmodpkg command, prevents host sage from adopting another package, so host sage is no longer an adoptive node for pkg01. This prevent the adoptive node (sage) from becomming overloaded when multiple packages failover.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Cluster Configuration File for Adoptive Node for Two Packages This section shows the cluster configuration file (cluster.conf) for this configuration example. The comments are not shown. Chapter 3 CLUSTER_NAME PkgCtrl QS_HOST QS_POLLING_INTERVAL qs 300000000 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP NETWORK_INTERFACE thyme eth0 192.100.112.146 eth1 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP basil eth0 192.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Package Configuration File for pkg01 This section shows the package configuration file (pkg1.conf) for the package pkg01 in this sample configuration. The comments are not shown.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Package Control Script for pkg01 This section shows the package control script (pkg1.cntl) for the package pkg01 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages NFS Control Script for pkg01 This section shows the NFS control script (hanfs.sh) for the package pkg03 on this sample configuration on the the user-configured part of the script is shown: XFS[0]="-o rw *:/hanfs/nfsu011" NFS_SERVICE_NAME[0]="nfs1.monitor" NFS_SERVICE_CMD[0]="/usr/local/cmcluster/pkg1/nfs.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Package Configuration File for pkg02 This section shows the package configuration file (pkg2.conf) for the package pkg02 in this sample configuration. The comments are not shown.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages Package Control Script for pkg02 This section shows the package control script (pkg2.cntl) for the package pkg02 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring One Adoptive Node to Support Failover of Multiple Packages NFS Control Script for pkg02 This section shows the NFS control script (hanfs.sh) for the package pkg02 on this sample configuration on the the user-configured part of the script is shown: XFS[0]="-o rw *:/hanfs/nfsu021" NFS_SERVICE_NAME[0]="nfs2.monitor" NFS_SERVICE_CMD[0]="/usr/local/cmcluster/pkg2/nfs.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Configuring Multiple Adoptive Nodes for Cascading Failover This configuration has two packages and three servers. One server is the primary node for both packages. The other servers are adoptive nodes for the two packages. Figure 3-5 illustrates this configuration. Dotted lines indicate which servers are adoptive nodes for the packages. Figure 3-6 illustrates the configuration after host thyme fails.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Figure 3-6 shows the cascading failover configuration after host thyme has failed. Host basil is the first adoptive node configured for pkg01, and host sage is the first adoptive node configured for pkg02.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Cluster Configuration File for Three-Server Cascading Failover This section shows the cluster configuration file (cluster.conf) for this configuration example. The comments are not shown. CLUSTER_NAME Cascading QS_HOST QS_POLLING_INTERVAL qs 300000000 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP NETWORK_INTERFACE thyme eth0 192.100.112.146 eth1 NODE_NAME NETWORK_INTERFACE HEARTBEAT_IP basil eth0 192.100.112.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Package Configuration File for pkg01 This section shows the package configuration file (pkg1.conf) for the package pkg01 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Package Control Script for pkg01 This section shows the package control script (pkg1.cntl) for the package pkg01 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Package Configuration File for pkg02 This section shows the package configuration file (pkg2.conf) for the package pkg02 in this sample configuration. The comments are not shown.
Sample Configurations Configuring Multiple Adoptive Nodes for Cascading Failover Package Control Script for pkg02 This section shows the package control script (pkg2.cntl) for the package pkg02 in this sample configuration. Only the user-configured part of the script is shown; the executable part of the script and most of the comments have been omitted. PATH=/sbin:/usr/bin:/usr/sbin:/etc:/bin:usr/local/ cmcluster/bin RAIDTAB="/usr/local/cmcluster/conf/raidtab.
Index A adoptive nodes, 10 configuring, 25 example of cascading failover, 50 example of package control option, 41 for multiple packages, 10, 13, 28, 41 illustration of cascading failover, 14 AUTO_RUN, 26 automounter timeout, 18 B binary configuration files, creating, 30 C cascading failover, 10 example configuration, 50 illustration of, 14 client behavior, 8, 18 cluster configuration file (cluster.
Index L lockd monitoring, 17 locked files, during package failover, 9 logging, NFS monitor script, 18 logical volumes configuration, 23 specifying in nfs.cntl, 27 LV variable, in nfs.cntl script, 27 M MC/ServiceGuard documentation, 22 MD devices activation, 16 deactivation, 17 monitor script (nfs.mon), 17 logging, 18 specified in nfs.cntl, 29 specified in nfs.
Index T timeout, automounter, 18 U unexporting file systems, 17 unmounting file systems, 17 user IDs, 23 V VG variable, in nfs.cntl script, 27 volume groups activating, 16 configuring, 23 deactivating, 17 major and minor numbers, 23 specifying in nfs.cntl, 27 X XFS variable, in nfs.
