Red Hat Enterprise Linux 5 Cluster Administration Configuring and Managing a Red Hat Cluster Edition 5
Red Hat Enterprise Linux 5 Cluster Administration Configuring and Managing a Red Hat Cluster Edition 5
Legal Notice Co pyright © 20 14 Red Hat Inc.. This do cument is licensed by Red Hat under the Creative Co mmo ns Attributio n-ShareAlike 3.0 Unpo rted License. If yo u distribute this do cument, o r a mo dified versio n o f it, yo u must pro vide attributio n to Red Hat, Inc. and pro vide a link to the o riginal. If the do cument is mo dified, all Red Hat trademarks must be remo ved.
T able of Cont ent s T able of Contents .Int . .roduct . . . . . .ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4. . . . . . . . . . 1. Do c ument Co nventio ns 4 1.1. Typ o g rap hic Co nventio ns 4 1.2. Pull-q uo te Co nventio ns 6 1.3. No tes and Warning s 6 2 . Feed b ac k 7 . .hapt C . . . .er . .1. .. Red . . . . Hat . . . .Clust . . . . er . . .Configurat . . . . . . . . . ion . . .
Red Hat Ent erprise Linux 5 Clust er Administ rat ion 3 .7.2. Mo d ifying a Failo ver Do main 46 3 .8 . Ad d ing Clus ter Res o urc es 47 3 .9 . Ad d ing a Clus ter Servic e to the Clus ter 47 3 .10 . Co nfig uring Clus ter Sto rag e 50 . .hapt C . . . .er . .4. .. Managing . . . . . . . . . Red . . . . Hat . . . .Clust . . . . .er. .Wit . . .h. Conga . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 ........... 4 .1.
T able of Cont ent s D .1. Parent, Child , and Sib ling Relatio ns hip s Amo ng Res o urc es D .2. Sib ling Start O rd ering and Res o urc e Child O rd ering 110 111 D .2.1. Typ ed Child Res o urc e Start and Sto p O rd ering T yp ed Child Res o urc e Starting O rd er 112 113 T yp ed Child Res o urc e Sto p p ing O rd er D .2.2.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Introduction This document provides information about installing, configuring and managing Red Hat Cluster components. Red Hat Cluster components are part of Red Hat Cluster Suite and allow you to connect a group of computers (called nodes or members) to work together as a cluster. This document does not include information about installing, configuring, and managing Linux Virtual Server (LVS) software. Information about that is in a separate document.
Int roduct ion Four typographic conventions are used to call attention to specific words and phrases. These conventions, and the circumstances they apply to, are as follows. Mono-spaced Bold Used to highlight system input, including shell commands, file names and paths. Also used to highlight keys and key combinations.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion To connect to a remote machine using ssh, type ssh username@domain.name at a shell prompt. If the remote machine is example.com and your username on that machine is john, type ssh john@example.com. The mount -o remount file-system command remounts the named file system. For example, to remount the /home file system, the command is mount -o remount /home. To see the version of a currently installed package, use the rpm -q package command.
Int roduct ion Finally, we use three visual styles to draw attention to information that might otherwise be overlooked. Note Notes are tips, shortcuts or alternative approaches to the task at hand. Ignoring a note should have no negative consequences, but you might miss out on a trick that makes your life easier. Important Important boxes detail things that are easily missed: configuration changes that only apply to the current session, or services that need restarting before an update will apply.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Chapter 1. Red Hat Cluster Configuration and Management Overview Red Hat Cluster allows you to connect a group of computers (called nodes or members) to work together as a cluster. It provides a wide variety of ways to configure hardware and software to suit your clustering needs (for example, a cluster for sharing files on a GFS file system or a cluster with high-availability service failover).
Chapt er 1 . Red Hat Clust er Configurat ion and Management O verview Fibre Channel switch — A Fibre Channel switch provides access to Fibre Channel storage. Other options are available for storage according to the type of storage interface; for example, iSCSI or GNBD . A Fibre Channel switch can be configured to perform fencing. Storage — Some type of storage is required for a cluster. The type required depends on the purpose of the cluster. Fig u re 1.1. R ed H at C lu st er H ard ware O verview 1.1.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion 1. Shut down all cluster services on a single cluster node. For instructions on stopping cluster software on a node, refer to Section 6.1, “ Starting and Stopping the Cluster Software” . It may be desirable to manually relocate cluster-managed services and virtual machines off of the host prior to stopping rgmanager. 2. Execute the yum update command to install the new RPMs. For example: yum update -y openais cman rgmanager lvm2-cluster gfs2-utils 3.
Chapt er 1 . Red Hat Clust er Configurat ion and Management O verview The following cluster configuration tools are available with Red Hat Cluster: C o n g a — This is a comprehensive user interface for installing, configuring, and managing Red Hat clusters, computers, and storage attached to clusters and computers. system-config-cluster — This is a user interface for configuring and managing a Red Hat cluster.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion To administer a cluster or storage, an administrator adds (or registers) a cluster or a computer to a lu ci server. When a cluster or a computer is registered with lu ci, the FQD N hostname or IP address of each computer is stored in a lu ci database. You can populate the database of one lu ci instance from another lu ciinstance. That capability provides a means of replicating a lu ci server instance and provides an efficient upgrade and testing path.
Chapt er 1 . Red Hat Clust er Configurat ion and Management O verview Fig u re 1.4 .
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Fig u re 1.5. lu ci st o rag e T ab 1.3. system-config-cluster Clust er Administ rat ion GUI This section provides an overview of the cluster administration graphical user interface (GUI) available with Red Hat Cluster Suite — system-config-cluster. It is for use with the cluster infrastructure and the high-availability service management components.
Chapt er 1 . Red Hat Clust er Configurat ion and Management O verview Note While system-config-cluster provides several convenient tools for configuring and managing a Red Hat Cluster, the newer, more comprehensive tool, C o n g a, provides more convenience and flexibility than system-config-cluster. 1.3.1. Clust er Configurat ion T ool You can access the C lu st er C o n f ig u rat io n T o o l (Figure 1.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion The C lu st er C o n f ig u rat io n T o o l represents cluster configuration components in the configuration file (/etc/cluster/cluster.conf) with a hierarchical graphical display in the left panel. A triangle icon to the left of a component name indicates that the component has one or more subordinate components assigned to it. Clicking the triangle icon expands and collapses the portion of the tree below a component.
Chapt er 1 . Red Hat Clust er Configurat ion and Management O verview Fig u re 1.7. C lu st er St at u s T o o l The nodes and services displayed in the C lu st er St at u s T o o l are determined by the cluster configuration file (/etc/cluster/cluster.conf). You can use the C lu st er St at u s T o o l to enable, disable, restart, or relocate a high-availability service. 1.4 .
Red Hat Ent erprise Linux 5 Clust er Administ rat ion T ab le 1.1. C o mman d Lin e T o o ls C o mman d Lin e Tool U sed Wit h Pu rp o se ccs_tool — Cluster Configuration System Tool Cluster Infrastructure ccs_tool is a program for making online updates to the cluster configuration file. It provides the capability to create and modify cluster infrastructure components (for example, creating a cluster, adding and removing a node). For more information about this tool, refer to the ccs_tool(8) man page.
Chapt er 2 . Before Configuring a Red Hat Clust er Chapter 2. Before Configuring a Red Hat Cluster This chapter describes tasks to perform and considerations to make before installing and configuring a Red Hat Cluster, and consists of the following sections. Important Make sure that your deployment of Red Hat Cluster Suite meets your needs and can be supported. Consult with an authorized Red Hat representative to verify Cluster Suite and GFS configuration prior to deployment.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion configurations in which some nodes have access to the file system and others do not are not supported.This does not require that all nodes actually mount the GFS/GFS2 file system itself.
Chapt er 2 . Before Configuring a Red Hat Clust er Section 2.3.1, “ Enabling IP Ports on Cluster Nodes” Section 2.3.2, “ Enabling IP Ports on Computers That Run lu ci” 2.3.1. Enabling IP Port s on Clust er Nodes To allow Red Hat Cluster nodes to communicate with each other, you must enable the IP ports assigned to certain Red Hat Cluster components. Table 2.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion T ab le 2.2.
Chapt er 2 . Before Configuring a Red Hat Clust er Note D isabling ACPI Soft-Off with the BIOS may not be possible with some computers. Appending acpi=off to the kernel boot command line of the /boot/grub/grub.conf file Important This method completely disables ACPI; some computers do not boot correctly if ACPI is completely disabled. Use this method only if the other methods are not effective for your cluster.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Note You can fence the node with the fence_node command or C o n g a. 2.4 .2. Disabling ACPI Soft -Off wit h t he BIOS The preferred method of disabling ACPI Soft-Off is with chkconfig management (Section 2.4.1, “ D isabling ACPI Soft-Off with chkconfig Management” ). However, if the preferred method is not effective for your cluster, follow the procedure in this section.
Chapt er 2 . Before Configuring a Red Hat Clust er | Suspend Mode [Disabled] | | | HDD Power Down [Disabled] | | | Soft-Off by PWR-BTTN [Instant-Off | | | CPU THRM-Throttling [50.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion # grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You have a /boot partition. This means that # all kernel and initrd paths are relative to /boot/, eg. # root (hd0,0) # kernel /vmlinuz-version ro root=/dev/VolGroup00/LogVol00 # initrd /initrd-version.
Chapt er 2 . Before Configuring a Red Hat Clust er Fig u re 2.1. Web Server C lu st er Service Examp le Clients access the HA service through the IP address 10.10.10.201, enabling interaction with the web server application, httpd-content. The httpd-content application uses the gfs-content-webserver file system. If node B were to fail, the content-webserver HA service would fail over to node D . If node D were not available or also failed, the service would fail over to node A.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion An HA service is a group of cluster resources configured into a coherent entity that provides specialized services to clients. An HA service is represented as a resource tree in the cluster configuration file, /etc/cluster/cluster.conf (in each cluster node).
Chapt er 2 . Before Configuring a Red Hat Clust er Parent, child, and sibling relationships among resources The types of resources and the hierarchy of resources depend on the type of service you are configuring. The types of cluster resources are listed in Appendix C, HA Resource Parameters. Information about parent, child, and sibling relationships among resources is described in Appendix D , HA Resource Behavior. 2.6.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion C MAN memb ersh ip t imeo u t valu e The CMAN membership timeout value (the time a node needs to be unresponsive before CMAN considers that node to be dead, and not a member) should be at least two times that of the qdiskd membership timeout value. The reason is because the quorum daemon must detect failed nodes on its own, and can take much longer to do so than CMAN. The default value for CMAN membership timeout is 10 seconds.
Chapt er 2 . Before Configuring a Red Hat Clust er Note When using SELinux with Red Hat Cluster Suite in a VM environment, you should ensure that the SELinux boolean fenced_can_network_connect is persistently set to on. This allows the fence_xvm fencing agent to work properly, enabling the system to fence virtual machines. For more information about SELinux, refer to Deployment Guide for Red Hat Enterprise Linux 5. 2.9.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion For gnbd: iptables -I INPUT -p tcp -m state --state NEW -m multiport --dports 14567 -j ACCEPT For luci: iptables -I INPUT -p tcp -m state --state NEW -m multiport --dports 8084 -j ACCEPT For DLM: iptables -I INPUT -p tcp -m state --state NEW -m multiport --dports 21064 -j ACCEPT For ccsd: iptables -I INPUT -p udp -m state --state NEW -m multiport --dports 50007 -j ACCEPT iptables -I INPUT -p tcp -m state --state NEW -m multiport --dports 50008 -j ACC
Chapt er 2 . Before Configuring a Red Hat Clust er The non-default location for virtual machine configuration files may be anywhere. The advantage of using an NFS share or a shared GFS or GFS2 file system is that the administrator does not need to keep the configuration files in sync across the cluster members. However, it is also permissible to use a local directory as long as the administrator keeps the contents synchronized somehow clusterwide.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Chapter 3. Configuring Red Hat Cluster With Conga This chapter describes how to configure Red Hat Cluster software using C o n g a, and consists of the following sections: Section 3.1, “ Configuration Tasks” Section 3.2, “ Starting lu ci and ricci” Section 3.3, “ Creating A Cluster” Section 3.4, “ Global Cluster Properties” Section 3.5, “ Configuring Fence D evices” Section 3.6, “ Configuring Cluster Members” Section 3.
Chapt er 3. Configuring Red Hat Clust er Wit h Conga # service ricci start Starting ricci: [ OK ] 3. Select a computer to host lu ci and install the lu ci software on that computer. For example: # yum install luci Note Typically, a computer in a server cage or a data center hosts lu ci; however, a cluster computer can host lu ci. 4. At the computer running lu ci, initialize the lu ci server using the luci_admin init command.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion 1. As administrator of lu ci, select the clu st er tab. 2. Click C reat e a N ew C lu st er. 3. At the C lu st er N ame text box, enter a cluster name. The cluster name cannot exceed 15 characters. Add the node name and password for each cluster node. Enter the node name for each node in the N o d e H o st n ame column; enter the root password for each node in the R o o t Passwo rd column.
Chapt er 3. Configuring Red Hat Clust er Wit h Conga The Po st - Fail D elay parameter is the number of seconds the fence daemon (fenced) waits before fencing a node (a member of the fence domain) after the node has failed. The Po st - Fail D elay default value is 0. Its value may be varied to suit cluster and network performance. The Po st - Jo in D elay parameter is the number of seconds the fence daemon (fenced) waits before fencing a node after the node joins the fence domain.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion 4. Q u o ru m Part it io n tab — This tab provides an interface for configuring these Q u o ru m Part it io n C o n f ig u rat io n parameters: D o n o t u se a Q u o ru m Part it io n , U se a Q u o ru m Part it io n , In t erval, Vo t es, T K O , Min imu m Sco re, D evice, Lab el, and H eu rist ics. The D o n o t u se a Q u o ru m Part it io n parameter is enabled by default. Table 3.1, “ Quorum-D isk Parameters” describes the parameters.
Chapt er 3. Configuring Red Hat Clust er Wit h Conga Paramet er H eu rist ics D escrip t io n Pat h t o Pro g ram — The program used to determine if this heuristic is alive. This can be anything that can be executed by /bin/sh -c . A return value of 0 indicates success; anything else indicates failure. This field is required. In t erval — The frequency (in seconds) at which the heuristic is polled. The default interval for every heuristic is 2 seconds. Sco re — The weight of this heuristic.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion 1. At the detailed menu for the cluster (below the clu st ers menu), click Sh ared Fen ce D evices. Clicking Sh ared Fen ce D evices causes the display of the fence devices for a cluster and causes the display of menu items for fence device configuration: Ad d a Fen ce D evice and C o n f ig u re a Fen ce D evice. Note If this is an initial cluster configuration, no fence devices have been created, and therefore none are displayed. 2.
Chapt er 3. Configuring Red Hat Clust er Wit h Conga 4. Specify the information in the Fencing Type dialog box according to the type of fence device. Refer to Appendix B, Fence Device Parameters for more information about fence device parameters. 5. Click Add this shared fence device. Clicking Add this shared fence device causes a progress page to be displayed temporarily.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Creating a cluster consists of selecting a set of nodes (or members) to be part of the cluster. Once you have completed the initial step of creating a cluster and creating fence devices, you need to configure cluster nodes. To initially configure cluster nodes after creating a new cluster, follow the steps in this section.
Chapt er 3. Configuring Red Hat Clust er Wit h Conga A progress page shows the progress of those actions for each added node. 5. When the process of adding a node is complete, a page is displayed providing a configuration interface for the cluster. 6. At the detailed menu for the cluster (below the clu st ers menu), click N o d es.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion b. On that page, at the C h o o se a t askdrop-down box, choose to either disable the service are start it on another node and click Go. c. Upon confirmation that the service has been disabled or started on another node, click the clu st er tab. Clicking the clu st er tab causes the Choose a cluster to administer page to be displayed. d. At the Choose a cluster to administer page, click the link of the node to be deleted.
Chapt er 3. Configuring Red Hat Clust er Wit h Conga Note Failover domains are not required for operation. By default, failover domains are unrestricted and unordered. In a cluster with several members, using a restricted failover domain can minimize the work to set up the cluster to run a cluster service (such as httpd), which requires you to set up the configuration identically on all members that run the cluster service).
Red Hat Ent erprise Linux 5 Clust er Administ rat ion 5. To restrict failover to members in this failover domain, click the checkbox next to R est rict f ailo ver t o t h is d o main ' s memb ers. With R est rict f ailo ver t o t h is d o main ' s memb ers checked, services assigned to this failover domain fail over only to nodes in this failover domain. 6. To specify that a node does not fail back in this failover domain, click the checkbox next to D o n o t f ail b ack services in t h is d o main .
Chapt er 3. Configuring Red Hat Clust er Wit h Conga 6. Restricted failover — To enable or disable restricted failover for members in this failover domain, click the checkbox next to R est rict f ailo ver t o t h is d o main ' s memb ers. With R est rict f ailo ver t o t h is d o main ' s memb ers checked, services assigned to this failover domain fail over only to nodes in this failover domain.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion 1. At the detailed menu for the cluster (below the clu st ers menu), click Services. Clicking Services causes the display of services in the center of the page and causes the display of menu items for services configuration: Ad d a Service, Ad d a Virt u al Mach in e Service, and C o n f ig u re a Service. 2. To configure any service other than a virtual machine service, Click Ad d a Service.
Chapt er 3. Configuring Red Hat Clust er Wit h Conga 5. Add a resource to the service; click Add a resource to this service. Clicking Add a resource to this service causes the display of two drop-down boxes: Ad d a n ew lo cal reso u rce and U se an exist in g g lo b al reso u rce. Adding a new local resource adds a resource that is available only to this service. The process of adding a local resource is the same as adding a global resource described in Section 3.8, “ Adding Cluster Resources” .
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Configuring a Virtual Machine Service To configure a virtual machine service, after clicking Services you can click Ad d a Virt u al Mach in e Service. Enter the virtual machine resource parameters. For a description of the virtual machine parameters, refer to Table C.23, “ Virtual Machine” . When you have completed adding the virtual machine resource parameters, click Create Virtual Machine Service. 3.10.
Chapt er 3. Configuring Red Hat Clust er Wit h Conga Note Shared storage for use in Red Hat Cluster Suite requires that you be running the cluster logical volume manager daemon (clvmd) or the High Availability Logical Volume Management agents (HA-LVM). If you are not able to use either the clvmd daemon or HA-LVM for operational reasons or because you do not have the correct entitlements, you must not use single-instance LVM on the shared disk as this may result in data corruption.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Chapter 4. Managing Red Hat Cluster With Conga This chapter describes various administrative tasks for managing a Red Hat Cluster and consists of the following sections: Section 4.1, “ Starting, Stopping, and D eleting Clusters” Section 4.2, “ Managing Cluster Nodes” Section 4.3, “ Managing High-Availability Services” Section 4.4, “ Backing Up and Restoring the luci Configuration” Section 4.5, “ D iagnosing and Correcting Problems in a Cluster” 4 .1.
Chapt er 4 . Managing Red Hat Clust er Wit h Conga For R est art t h is clu st er and St o p t h is clu st er/St art t h is clu st er — D isplays a page with the list of nodes for the cluster. For D elet e t h is clu st er — D isplays the Choose a cluster to administer page in the clu st er tab, showing a list of clusters. 4 .2. Managing Clust er Nodes You can perform the following node-management functions through the lu ci server component of C o n g a: Make a node leave or join a cluster.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion 4. Clicking Go causes a progress page to be displayed. When the action is complete, a page is displayed showing the list of nodes for the cluster. 4 .3. Managing High-Availabilit y Services You can perform the following management functions for high-availability services through the lu ci server component of C o n g a: Configure a service. Stop or start a service. Restart a service.
Chapt er 4 . Managing Red Hat Clust er Wit h Conga R est art t h is service and St o p t h is service — These selections are available when the service is running. Select either function and click Go to make the change take effect. Clicking Go causes a progress page to be displayed. When the change is complete, another page is displayed showing a list of services for the cluster. St art t h is service and D elet e t h is service — These selections are available when the service is not running.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Chapter 5. Configuring Red Hat Cluster With system-configcluster This chapter describes how to configure Red Hat Cluster software using system-config-cluster, and consists of the following sections: Section 5.1, “ Configuration Tasks” Section 5.2, “ Starting the C lu st er C o n f ig u rat io n T o o l” Section 5.3, “ Configuring Cluster Properties” Section 5.4, “ Configuring Fence D evices” Section 5.5, “ Adding and D eleting Members” Section 5.
Chapt er 5. Configuring Red Hat Clust er Wit h syst em- config- clust er 8. Propagating the configuration file to the other nodes in the cluster. Refer to Section 5.9, “ Propagating The Configuration File: New Cluster” . 9. Starting the cluster software. Refer to Section 5.10, “ Starting the Cluster Software” . 5.2.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion dialog box provides a text box for cluster name and the following checkboxes: C u st o m C o n f ig u re Mu lt icast and U se a Q u o ru m D isk. In most circumstances you only need to configure the cluster name. Note Choose the cluster name carefully. The only way to change the name of a Red Hat cluster is to create a new cluster configuration with the new name.
Chapt er 5. Configuring Red Hat Clust er Wit h syst em- config- clust er Important Quorum-disk parameters and heuristics depend on the site environment and special requirements needed. To understand the use of quorum-disk parameters and heuristics, refer to the qdisk(5) man page. If you require assistance understanding and using quorum disk, contact an authorized Red Hat support representative.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Fig u re 5.2. C reat in g A N ew C o n f ig u rat io n 4. When you have completed entering the cluster name and other parameters in the New Configuration dialog box, click OK. Clicking OK starts the C lu st er C o n f ig u rat io n T o o l, displaying a graphical representation of the configuration (Figure 5.3, “ The Cluster Configuration Tool” ).
Chapt er 5. Configuring Red Hat Clust er Wit h syst em- config- clust er Fig u re 5.3. T h e C lu st er C o n f ig u rat io n T o o l T ab le 5.1. Q u o ru m- D isk Paramet ers Paramet er D escrip t io n U se a Q u o ru m D isk Enables quorum disk. Enables quorum-disk parameters in the New Configuration dialog box. The frequency of read/write cycles, in seconds. The number of cycles a node must miss in order to be declared dead.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Paramet er D escrip t io n Lab el Specifies the quorum disk label created by the mkqdisk utility. If this field contains an entry, the label overrides the D evice field. If this field is used, the quorum daemon reads /proc/partitions and checks for qdisk signatures on every block device found, comparing the label against the specified label. This is useful in configurations where the quorum device name differs among nodes.
Chapt er 5. Configuring Red Hat Clust er Wit h syst em- config- clust er Note For more information about Po st - Jo in D elay and Po st - Fail D elay, refer to the fenced(8) man page. 6. Save cluster configuration changes by selecting File => Save. 5.4 . Configuring Fence Devices Configuring fence devices for the cluster consists of selecting one or more fence devices and specifying fence-device-dependent parameters (for example, name, IP address, login, and password).
Red Hat Ent erprise Linux 5 Clust er Administ rat ion 5.5. Adding and Delet ing Members The procedure to add a member to a cluster varies depending on whether the cluster is a newlyconfigured cluster or a cluster that is already configured and running. To add a member to a new cluster, refer to Section 5.5.1, “ Adding a Member to a Cluster” . To add a member to an existing cluster, refer to Section 5.5.2, “ Adding a Member to a Running Cluster” . To delete a member from a cluster, refer to Section 5.5.
Chapt er 5. Configuring Red Hat Clust er Wit h syst em- config- clust er 6. Configure fencing for the node: a. Click the node that you added in the previous step. b. At the bottom of the right frame (below Pro p ert ies), click Manage Fencing For This Node. Clicking Manage Fencing For This Node causes the Fence Configuration dialog box to be displayed. c. At the Fence Configuration dialog box, bottom of the right frame (below Pro p ert ies), click Add a New Fence Level.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Section 5.5.1, “ Adding a Member to a Cluster” . 2. Click Send to Cluster to propagate the updated configuration to other running nodes in the cluster. 3. Use the scp command to send the updated /etc/cluster/cluster.conf file from one of the existing cluster nodes to the new node. 4. At the Red Hat Cluster Suite management GUI C lu st er St at u s T o o l tab, disable each service listed under Services. 5.
Chapt er 5. Configuring Red Hat Clust er Wit h syst em- config- clust er d. service rgmanager start 5. Start the Red Hat Cluster Suite management GUI. At the C lu st er C o n f ig u rat io n T o o l tab, verify that the configuration is correct. At the C lu st er St at u s T o o l tab verify that the nodes and services are running as expected. 5.5.3. Delet ing a Member from a Clust er To delete a member from an existing cluster that is currently in operation, follow these steps: 1.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion b. service gfs stop, if you are using Red Hat GFS c. service clvmd stop, if CLVM has been used to create clustered volumes d. service cman stop 5. Start cluster software on all remaining cluster nodes by running the following commands in this order: a. service cman start b. service clvmd start, if CLVM has been used to create clustered volumes c. service gfs start, if you are using Red Hat GFS d. service rgmanager start 6.
Chapt er 5. Configuring Red Hat Clust er Wit h syst em- config- clust er 5.6. Configuring a Failover Domain A failover domain is a named subset of cluster nodes that are eligible to run a cluster service in the event of a node failure. A failover domain can have the following characteristics: Unrestricted — Allows you to specify that a subset of members are preferred, but that a cluster service assigned to this domain can run on any available member.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Section 5.6.3, “ Removing a Member from a Failover D omain” 5.6.1. Adding a Failover Domain To add a failover domain, follow these steps: 1. At the left frame of the C lu st er C o n f ig u rat io n T o o l, click Failo ver D o main s. 2. At the bottom of the right frame (labeled Pro p ert ies), click the Create a Failover Domain button. Clicking the Create a Failover Domain button causes the Add Failover Domain dialog box to be displayed. 3.
Chapt er 5. Configuring Red Hat Clust er Wit h syst em- config- clust er 5. To restrict failover to members in this failover domain, click (check) the R est rict Failo ver T o T h is D o main s Memb ers checkbox. (With R est rict Failo ver T o T h is D o main s Memb ers checked, services assigned to this failover domain fail over only to nodes in this failover domain.) 6. To prioritize the order in which the members in the failover domain assume control of a failed cluster service, follow these steps: a.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion configuration. 5.6.2. Removing a Failover Domain To remove a failover domain, follow these steps: 1. At the left frame of the C lu st er C o n f ig u rat io n T o o l, click the failover domain that you want to delete (listed under Failo ver D o main s). 2. At the bottom of the right frame (labeled Pro p ert ies), click the Delete Failover Domain button.
Chapt er 5. Configuring Red Hat Clust er Wit h syst em- config- clust er the change immediately, choose File => Save to save the changes to the cluster configuration. 5.7. Adding Clust er Resources To specify a resource for a cluster service, follow these steps: 1. On the R eso u rces property of the C lu st er C o n f ig u rat io n T o o l, click the Create a Resource button. Clicking the Create a Resource button causes the Resource Configuration dialog box to be displayed. 2.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Fig u re 5.9 . Ad d in g a C lu st er Service 4. If you want to restrict the members on which this cluster service is able to run, choose a failover domain from the Failo ver D o main drop-down box. (Refer to Section 5.6, “ Configuring a Failover D omain” for instructions on how to configure a failover domain.) 5. Au t o st art T h is Service checkbox — This is checked by default.
Chapt er 5. Configuring Red Hat Clust er Wit h syst em- config- clust er Note Circumstances that require enabling R u n Exclu sive are rare. Enabling R u n Exclu sive can render a service offline if the node it is running on fails and no other nodes are empty. 7. Select a recovery policy to specify how the resource manager should recover from a service failure.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Note To verify the existence of the IP service resource used in a cluster service, you must use the /sbin/ip addr list command on a cluster node. The following output shows the /sbin/ip addr list command executed on a node running a cluster service: 1: lo: mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.
Chapt er 5. Configuring Red Hat Clust er Wit h syst em- config- clust er 1. service cman start 2. service clvmd start, if CLVM has been used to create clustered volumes Note Shared storage for use in Red Hat Cluster Suite requires that you be running the cluster logical volume manager daemon (clvmd) or the High Availability Logical Volume Management agents (HA-LVM).
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Chapter 6. Managing Red Hat Cluster With system-configcluster This chapter describes various administrative tasks for managing a Red Hat Cluster and consists of the following sections: Section 6.1, “ Starting and Stopping the Cluster Software” Section 6.2, “ Managing High-Availability Services” Section 6.4, “ Backing Up and Restoring the Cluster D atabase” Section 6.6, “ D isabling the Cluster Software” Section 6.
Chapt er 6 . Managing Red Hat Clust er Wit h syst em- config- clust er Fig u re 6 .1. C lu st er St at u s T o o l You can use the C lu st er St at u s T o o l to enable, disable, restart, or relocate a high-availability service. The C lu st er St at u s T o o l displays the current cluster status in the Services area and automatically updates the status every 10 seconds. To enable a service, you can select the service in the Services area and click En ab le.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion The following tables describe the members and services status information displayed by the C lu st er St at u s T o o l. T ab le 6 .1. Memb ers St at u s Memb ers St at u s Memb er D escrip t io n The node is part of the cluster. Note: A node can be a member of a cluster; however, the node may be inactive and incapable of running services.
Chapt er 6 . Managing Red Hat Clust er Wit h syst em- config- clust er Important Although the C lu st er C o n f ig u rat io n T o o l provides a Q u o ru m Vo t es parameter in the Properties dialog box of each cluster member, that parameter is intended only for use during initial cluster configuration. Furthermore, it is recommended that you retain the default Q u o ru m Vo t es value of 1.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion If a cluster member becomes inoperable because of misconfiguration, restore the configuration file according to the following steps: 1. At the C lu st er C o n f ig u rat io n T o o l tab of the Red Hat Cluster Suite management GUI, click File => O p en . 2. Clicking File = > O p en causes the system-config-cluster dialog box to be displayed. 3. At the system-config-cluster dialog box, select a backup file (for example, /etc/cluster/cluster.conf.bak.1).
Chapt er 6 . Managing Red Hat Clust er Wit h syst em- config- clust er So, for example, if you have a database that uses an ext3-formatted filesystem, you can disable the database while preserving the filesystem resource for use in the service. In the following example snippet of a cluster.conf file, a service uses a MySQL database and ext3-formatted filesystem resources. PAGE 88Red Hat Ent erprise Linux 5 Clust er Administ rat ion # # # # chkconfig chkconfig chkconfig chkconfig --level --level --level --level 2345 2345 2345 2345 rgmanager off gfs off clvmd off cman off Once the problems with the disabled cluster member have been resolved, use the following commands to allow the member to rejoin the cluster: # # # # chkconfig chkconfig chkconfig chkconfig --level --level --level --level 2345 2345 2345 2345 rgmanager on gfs on clvmd on cman on You can then reboot the membe
Example of Set t ing Up Apache HT T P Server Example of Setting Up Apache HTTP Server This appendix provides an example of setting up a highly available Apache HTTP Server on a Red Hat Cluster. The example describes how to set up a service to fail over an Apache HTTP Server. Variables in the example apply to this example only; they are provided to assist setting up a service that suits your requirements. Note This example uses the C lu st er C o n f ig u rat io n T o o l (system-config-cluster).
Red Hat Ent erprise Linux 5 Clust er Administ rat ion 2. Use the mkfs command to create an ext3 file system on the partition you created in the previous step. Specify the drive letter and the partition number. For example: # mkfs -t ext3 /dev/sde3 3. Mount the file system that contains the document root directory. For example: # mount /dev/sde3 /var/www/html D o not add this mount information to the /etc/fstab file because only the cluster software can mount and unmount file systems used in a service. 4.
Example of Set t ing Up Apache HT T P Server AllowOverride None Options None Order allow,deny Allow from all Additional changes may need to be made to tune the Apache HTTP Server or add module functionality. For information on setting up other options, refer to the Red Hat Enterprise Linux System Administration Guide and the Red Hat Enterprise Linux Reference Guide. 2. The standard Apache HTTP Server start script, /etc/rc.d/init.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Make sure that the Mo n it o r Lin k checkbox is left checked. Click OK. 4. Click the Services property. 5. Create the Apache HTTP Server service. Click Create a Service. Type a N ame for the service in the Add a Service dialog. In the Service Management dialog, select a Failo ver D o main from the drop-down menu or leave it as N o n e. Click the Add a Shared Resource to this service button.
Fence Device Paramet ers Fence Device Parameters This appendix provides tables with parameter descriptions of fence devices. Note The N ame parameter for a fence device specifies an arbitrary name for the device that will be used by Red Hat Cluster Suite. This is not the same as the D NS name for the device. Note Certain fence devices have an optional Passwo rd Scrip t parameter.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Field D escrip t io n SNMP version SNMP community The SNMP version to use (1, 2c, 3); the default value is 1. The SNMP community string; the default value is private . SNMP security level SNMP authentication protocol SNMP privacy protocol SNMP privacy protocol password SNMP privacy protocol script Power wait Port The SNMP security level (noAuthNoPriv, authNoPriv, authPriv). The SNMP authentication protocol (MD 5, SHA).
Fence Device Paramet ers Field D escrip t io n IP address or hostname UD P/TCP port (optional) Login Password Password Script (optional) SNMP version SNMP community SNMP security level SNMP authentication protocol SNMP privacy protocol SNMP privacy protocol password SNMP privacy protocol script Power wait Port The IP address or hostname assigned to the device. fence_cisco_mds The UD P/TCP port to use for connection with the device; the default value is 161. The login name used to access the device.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Field D escrip t io n Module Name (optional) The module name for the D RAC when you have multiple D RAC modules. The script that supplies a password for access to the fence device. Using this supersedes the Passwo rd parameter. Password Script (optional) Use SSH (D RAC5 only) Power wait fence_drac (Red Hat Enterprise Linux 5.4 and later) Indicates that system will use SSH to access the device.
Fence Device Paramet ers Field D escrip t io n Name Hostname Login Password Password Script (optional) Use SSL connections Power wait A name for the server with HP iLO support. The hostname assigned to the device. The login name used to access the device. The password used to authenticate the connection to the device. The script that supplies a password for access to the fence device. Using this supersedes the Passwo rd parameter. fence_ilo Use SSL connections to communicate with the device.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Field D escrip t io n Name A name for the IBM iPD U device connected to the cluster into which the fence daemon logs via the SNMP protocol. The IP address or hostname assigned to the device. The UD P/TCP port to use for connection with the device; the default value is 161. The login name used to access the device. The password used to authenticate the connection to the device. The script that supplies a password for access to the fence device.
Fence Device Paramet ers Field D escrip t io n Password Script (optional) SNMP version SNMP community SNMP security level SNMP authentication protocol SNMP privacy protocol SNMP privacy protocol password SNMP privacy protocol script Power wait Port The script that supplies a password for access to the fence device. Using this supersedes the Passwo rd parameter. The SNMP version to use (1, 2c, 3); the default value is 1. The SNMP community string.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Field D escrip t io n IP Address Login Password Password Script (optional) Port The IP address assigned to the device. The login name used to access the device. The password used to authenticate the connection to the device. The script that supplies a password for access to the fence device. Using this supersedes the Passwo rd parameter. fence_mcdat a The switch outlet number. The fence agent for McD ata FC switches. T ab le B .20.
Fence Device Paramet ers Field D escrip t io n Port The switch outlet number. The fence agent for the WTI Network Power Switch. fence_wti T ab le B .23. SC SI Fen cin g Field D escrip t io n Name Node name A name for the SCSI fence device. Name of the node to be fenced. Refer to the fence_scsi (8) man page for more information. The fence agent for SCSI persistent reservations.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Field D escrip t io n Use SSL connections Power wait Virtual machine name Virtual machine UUID Use SSL connections to communicate with the device. fence_vmwar e_soap The fence agent for VMWare over SOAP API. Number of seconds to wait after issuing a power off or power on command. Name of virtual machine in inventory path format (e.g., /datacenter/vm/D iscovered_virtual_machine/myMachine). The UUID of the virtual machine to fence. T ab le B .26 .
HA Resource Paramet ers HA Resource Parameters This appendix provides descriptions of HA resource parameters. You can configure the parameters with Lu ci, system-config-cluster, or by editing etc/cluster/cluster.conf. Table C.1, “ HA Resource Summary” lists the resources, their corresponding resource agents, and references to other tables containing parameter descriptions. To understand resource agents in more detail you can view them in /usr/share/cluster of any cluster node.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion T ab le C .2. Ap ach e Server Field D escrip t io n Name Server Root The name of the Apache Service. The default value is /etc/httpd . Config File Specifies the Apache configuration file. The default valuer is /etc/httpd/conf . Other command line options for httpd . httpd Options Shutdown Wait Specifies the number of seconds to wait for correct end of service shutdown. (seconds) T ab le C .3.
HA Resource Paramet ers T ab le C .4 . G FS Field D escrip t io n Name Mount point D evice File system type Options File system ID The name of the file system resource. The path to which the file system resource is mounted. The device file associated with the file system resource. Specify GFS or GFS2. Mount options. Note File System ID is used only by NFS services. When creating a new GFS resource, you can leave this field blank.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Field D escrip t io n Fence the node if it is unable to clean up LVM tags Fence the node if it is unable to clean up LVM tags. T ab le C .7. MySQ L Field D escrip t io n Name Config File Specifies a name of the MySQL server resource. Specifies the configuration file. The default value is /etc/my.cnf . Specifies an IP address for MySQL server. If an IP address is not provided, the first IP address from the service is taken.
HA Resource Paramet ers Field D escrip t io n Name Symbolic name for the NFS mount. Note This resource is required only when a cluster service is configured to be an NFS client. Mount point Host Export path NFS version Path to which the file system resource is mounted. NFS server IP address or hostname. NFS Export directory name. NFS protocol: NFS3 — Specifies using NFSv3 protocol. The default setting is NFS3. NFS4 — Specifies using NFSv4 protocol. Options Mount options.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion T ab le C .13. O racle D B Field D escrip t io n Instance name (SID ) of Oracle instance Oracle user name Oracle application home directory List of Oracle listeners (optional, separated by spaces) Path to lock file (optional) Instance name. This is the user name of the Oracle user that the Oracle instance runs as. This is the Oracle (application, not user) home directory. It is configured when you install Oracle.
HA Resource Paramet ers Field D escrip t io n D atabase type Oracle TNS listener name ABAP stack is not installed, only Java stack is installed Application Level Monitoring Automatic Startup Recovery Path to Java SD K File name of the JD BC D river Path to a prestart script Path to a poststart script Path to a prestop script Path to a poststop script J2EE instance bootstrap directory J2EE security store path Specifies one of the following database types: Oracle, D B6, or AD A.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Field D escrip t io n Number of seconds to wait before checking startup status Enable automatic startup recovery Path to a prestart script Path to a poststart script Path to a prestop script Path to a poststop script Specifies the number of seconds to wait before checking the startup status (do not wait for J2EE-Addin). Enable or disable automatic startup recovery. Path to a pre-start script. Path to a post-start script.
HA Resource Paramet ers T ab le C .20. Service Field D escrip t io n Service name Name of service. This defines a collection of resources, known as a resource group or cluster service. If enabled, this service (or resource group) is started automatically after the cluster forms a quorum. If this parameter is disabled, this service is not started automatically after the cluster forms a quorum; the service is put into the disabled state.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion Field D escrip t io n Name Config File Specifies a service name for logging and other purposes. Specifies the absolute path to the configuration file. The default value is /etc/tomcat5/tomcat5.conf . User who runs the Tomcat server. The default value is tomcat. Other command line options for Catalina. Tomcat User Catalina Options Catalina Base Catalina base directory (differs for each service) The default value is /usr/share/tomcat5.
HA Resource Paramet ers Field D escrip t io n VM Migration Mapping Specifies an alternate interface for migration. You can specify this when, for example, the network address used for virtual machine migration on a node differs from the address of the node used for cluster communication. Specifying the following indicates that when you migrate a virtual machine from member to member2, you actually migrate to target2. Similarly, when you migrate from member2 to member, you migrate using target.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion HA Resource Behavior This appendix describes common behavior of HA resources. It is meant to provide ancillary information that may be helpful in configuring HA services. You can configure the parameters with Lu ci, system-config-cluster, or by editing etc/cluster/cluster.conf. For descriptions of HA resource parameters, refer to Appendix C, HA Resource Parameters.
HA Resource Behavior A cluster service is an integrated entity that runs under the control of rgmanager. All resources in a service run on the same node. From the perspective of rgmanager, a cluster service is one entity that can be started, stopped, or relocated. Within a cluster service, however, the hierarchy of the resources determines the order in which each resource is started and stopped.The hierarchical levels consist of parent, child, and sibling. Example D .
Red Hat Ent erprise Linux 5 Clust er Administ rat ion For more information about typed child resource start and stop ordering, refer to Section D .2.1, “ Typed Child Resource Start and Stop Ordering” . For more information about non-typed child resource start and stop ordering, refer to Section D .2.2, “ Non-typed Child Resource Start and Stop Ordering” . D.2.1.
HA Resource Behavior T ype d Child Re so urce St art ing Orde r In Example D .3, “ Ordering Within a Resource Type” , the resources are started in the following order: 1. lvm:1 — This is an LVM resource. All LVM resources are started first. lvm:1 (
Red Hat Ent erprise Linux 5 Clust er Administ rat ion D.2.2. Non-t yped Child Resource St art and St op Ordering Additional considerations are required for non-typed child resources. For a non-typed child resource, starting order and stopping order are not explicitly specified by the Service resource. Instead, starting order and stopping order are determined according to the order of the child resource in /etc/cluster/cluster.conf.
HA Resource Behavior started after nontypedresource:foo. (Non-typed resources are started in the order that they appear in the Service resource.) No n-t ype d Child Re so urce St o pping Orde r In Example D .4, “ Non-typed and Typed Child Resource in a Service” , the child resources are stopped in the following order: 1. nontypedresourcetwo:bar — This is a non-typed resource. Because it is a non-typed resource, it is stopped before the typed resources are stopped.
Red Hat Ent erprise Linux 5 Clust er Administ rat ion
HA Resource Behavior Examp le D .6 . Service foo N o rmal Failu re R eco very Examp le D .7. Service foo Failu re R eco very wit h __independent_subtree At t rib u t e
