Red Hat Cluster Suite Overview Red Hat Cluster Suite for Red Hat Enterprise Linux 5.
Red Hat Cluster Suite Overview Red Hat Cluster Suite Overview provides an overview of Red Hat Cluster Suite for Red Hat Enterprise Linux 5.
Red Hat Cluster Suite Overview: Red Hat Cluster Suite for Red Hat Enterprise Linux Copyright © 2008 Red Hat, Inc. Copyright © 2008 Red Hat, Inc. This material may only be distributed subject to the terms and conditions set forth in the Open Publication License, V1.0 or later with the restrictions noted below (the latest version of the OPL is presently available at http://www.opencontent.org/openpub/).
Red Hat Cluster Suite Overview
Introduction .............................................................................................................. vii 1. Document Conventions ................................................................................ viii 2. Feedback ...................................................................................................... ix 1. Red Hat Cluster Suite Overview .............................................................................. 1 1. Cluster Basics .......................
vi
Introduction This document provides a high-level overview of Red Hat Cluster Suite for Red Hat Enterprise Linux 5 and is is organized as follows: • Chapter 1, Red Hat Cluster Suite Overview • Chapter 2, Red Hat Cluster Suite Component Summary Although the information in this document is an overview, you should have advanced working knowledge of Red Hat Enterprise Linux and understand the concepts of server computing to gain a good comprehension of the information.
Introduction PDF, and RPM versions on the Red Hat Enterprise Linux Documentation CD and online at http://www.redhat.com/docs/. 1. Document Conventions Certain words in this manual are represented in different fonts, styles, and weights. This highlighting indicates that the word is part of a specific category. The categories include the following: Courier font Courier font represents commands, file names and paths, and prompts . When shown as below, it indicates computer output: Desktop Mail about.
Feedback Tip A tip is typically an alternative way of performing a task. Important Important information is necessary, but possibly unexpected, such as a configuration change that will not persist after a reboot. Caution A caution indicates an act that would violate your support agreement, such as recompiling the kernel. Warning A warning indicates potential data loss, as may happen when tuning hardware for maximum performance. 2.
x
Chapter 1. Red Hat Cluster Suite Overview Clustered systems provide reliability, scalability, and availability to critical production services. Using Red Hat Cluster Suite, you can create a cluster to suit your needs for performance, high availability, load balancing, scalability, file sharing, and economy.
Chapter 1. Red Hat Cluster Suite Overview High-availability clusters provide continuous availability of services by eliminating single points of failure and by failing over services from one cluster node to another in case a node becomes inoperative. Typically, services in a high-availability cluster read and write data (via read-write mounted file systems). Therefore, a high-availability cluster must maintain data integrity as one cluster node takes over control of a service from another cluster node.
Red Hat Cluster Suite Introduction Infrastructure components, the High-availability and Service Management components, and storage. • Linux Virtual Server (LVS) — Routing software that provides IP-Load-balancing. LVS runs in a pair of redundant servers that distributes client requests evenly to real servers that are behind the LVS servers.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.1. Red Hat Cluster Suite Introduction Note Figure 1.1, “Red Hat Cluster Suite Introduction” includes GFS, CLVM, and GNBD, which are components that are part of an optional package and not part of Red Hat Cluster Suite. 3. Cluster Infrastructure The Red Hat Cluster Suite cluster infrastructure provides the basic functions for a group of computers (called nodes or members) to work together as a cluster.
Lock Management the node votes plus 1. For quorum via quorum disk, quorum consists of user-specified conditions. Note By default, each node has one quorum vote. Optionally, you can configure each node to have more than one vote. CMAN keeps track of membership by monitoring messages from other cluster nodes. When cluster membership changes, the cluster manager notifies the other infrastructure components, which then take appropriate action.
Chapter 1. Red Hat Cluster Suite Overview distributed across all nodes in the cluster (refer to Figure 1.2, “CMAN/DLM Overview”). GFS and CLVM use locks from the lock manager. GFS uses locks from the lock manager to synchronize access to file system metadata (on shared storage). CLVM uses locks from the lock manager to synchronize updates to LVM volumes and volume groups (also on shared storage). 3.3. Fencing Fencing is the disconnection of a node from the cluster's shared storage.
Fencing Figure 1.3.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.4. Fibre Channel Switch Fencing Example Specifying a fencing method consists of editing a cluster configuration file to assign a fencing-method name, the fencing agent, and the fencing device for each node in the cluster. The way in which a fencing method is specified depends on if a node has either dual power supplies or multiple paths to storage.
Fencing Figure 1.5.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.6. Fencing a Node with Dual Fibre Channel Connections You can configure a node with one fencing method or multiple fencing methods. When you configure a node for one fencing method, that is the only fencing method available for fencing that node.
Cluster Configuration System The Cluster Configuration System (CCS) manages the cluster configuration and provides configuration information to other cluster components in a Red Hat cluster. CCS runs in each cluster node and makes sure that the cluster configuration file in each cluster node is up to date. For example, if a cluster system administrator updates the configuration file in Node A, CCS propagates the update from Node A to the other nodes in the cluster (refer to Figure 1.7, “CCS Overview”).
Chapter 1. Red Hat Cluster Suite Overview Figure 1.8. Accessing Configuration Information The cluster configuration file (/etc/cluster/cluster.conf) is an XML file that describes the following cluster characteristics: • Cluster name — Displays the cluster name, cluster configuration file revision level, and basic fence timing properties used when a node joins a cluster or is fenced from the cluster.
High-availability Service Management High-availability service management provides the ability to create and manage high-availability cluster services in a Red Hat cluster. The key component for high-availability service management in a Red Hat cluster, rgmanager, implements cold failover for off-the-shelf applications. In a Red Hat cluster, an application is configured with other cluster resources to form a high-availability cluster service.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.9. Failover Domains Figure 1.10, “Web Server Cluster Service Example” shows an example of a high-availability cluster service that is a web server named "content-webserver". It is running in cluster node B and is in a failover domain that consists of nodes A, B, and D. In addition, the failover domain is configured with a failover priority to fail over to node D before node A and to restrict failover to nodes only in that failover domain.
Red Hat GFS Figure 1.10. Web Server Cluster Service Example Clients access the cluster service through the IP address 10.10.10.201, enabling interaction with the web server application, httpd-content. The httpd-content application uses the gfs-content-webserver file system. If node B were to fail, the content-webserver cluster service would fail over to node D. If node D were not available or also failed, the service would fail over to node A.
Chapter 1. Red Hat Cluster Suite Overview uses a lock manager to coordinate I/O. When one node changes data on a GFS file system, that change is immediately visible to the other cluster nodes using that file system. Using Red Hat GFS, you can achieve maximum application uptime through the following benefits: • Simplifying your data infrastructure • Install and patch applications once for the entire cluster. • Eliminates the need for redundant copies of application data (duplication).
Superior Performance and Scalability • Section 5.1, “Superior Performance and Scalability” • Section 5.2, “Performance, Scalability, Moderate Price” • Section 5.3, “Economy and Performance” Note The GFS deployment examples reflect basic configurations; your needs might require a combination of configurations shown in the examples. 5.1. Superior Performance and Scalability You can obtain the highest shared-file performance when applications access storage directly. The GFS SAN configuration in Figure 1.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.12, “GFS and GNBD with a SAN”. SAN block storage is presented to network clients as block storage devices by GNBD servers. From the perspective of a client application, storage is accessed as if it were directly attached to the server in which the application is running. Stored data is actually on the SAN. Storage devices and data can be equally shared by network client applications.
Cluster Logical Volume Manager Figure 1.13. GFS and GNBD with Directly Connected Storage 6. Cluster Logical Volume Manager The Cluster Logical Volume Manager (CLVM) provides a cluster-wide version of LVM2. CLVM provides the same capabilities as LVM2 on a single node, but makes the volumes available to all nodes in a Red Hat cluster. The logical volumes created with CLVM make logical volumes available to all nodes in a cluster. The key component in CLVM is clvmd.
Chapter 1. Red Hat Cluster Suite Overview Using CLVM requires minor changes to /etc/lvm/lvm.conf for cluster-wide locking. Figure 1.14. CLVM Overview You can configure CLVM using the same commands as LVM2, using the LVM graphical user interface (refer to Figure 1.15, “LVM Graphical User Interface”), or using the storage configuration function of the Conga cluster configuration graphical user interface (refer to Figure 1.16, “Conga LVM Graphical User Interface”) . Figure 1.
Cluster Logical Volume Manager Figure 1.15.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.16.
Global Network Block Device Figure 1.17. Creating Logical Volumes 7. Global Network Block Device Global Network Block Device (GNBD) provides block-device access to Red Hat GFS over TCP/IP. GNBD is similar in concept to NBD; however, GNBD is GFS-specific and tuned solely for use with GFS. GNBD is useful when the need for more robust technologies — Fibre Channel or single-initiator SCSI — are not necessary or are cost-prohibitive. GNBD consists of two major components: a GNBD client and a GNBD server.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.18. GNBD Overview 8. Linux Virtual Server Linux Virtual Server (LVS) is a set of integrated software components for balancing the IP load across a set of real servers. LVS runs on a pair of equally configured computers: one that is an active LVS router and one that is a backup LVS router. The active LVS router serves two roles: • To balance the load across the real servers. • To check the integrity of the services on each real server.
Linux Virtual Server Figure 1.19. Components of a Running LVS Cluster The pulse daemon runs on both the active and passive LVS routers. On the backup LVS router, pulse sends a heartbeat to the public interface of the active router to make sure the active LVS router is properly functioning. On the active LVS router, pulse starts the lvs daemon and responds to heartbeat queries from the backup LVS router.
Chapter 1. Red Hat Cluster Suite Overview To an outside user accessing a hosted service (such as a website or database application), LVS appears as one server. However, the user is actually accessing real servers behind the LVS routers. Because there is no built-in component in LVS to share the data among real servers, you have have two basic options: • Synchronize the data across the real servers. • Add a third layer to the topology for shared data access.
Two-Tier LVS Topology Figure 1.20. Two-Tier LVS Topology Service requests arriving at an LVS router are addressed to a virtual IP address or VIP. This is a publicly-routable address that the administrator of the site associates with a fully-qualified domain name, such as www.example.com, and which is assigned to one or more virtual servers1.
Chapter 1. Red Hat Cluster Suite Overview • Round-Robin Scheduling — Distributes each request sequentially around a pool of real servers. Using this algorithm, all the real servers are treated as equals without regard to capacity or load. • Weighted Round-Robin Scheduling — Distributes each request sequentially around a pool of real servers but gives more jobs to servers with greater capacity.
Three-Tier LVS Topology The backup LVS router performs the role of a standby system. Periodically, the LVS routers exchange heartbeat messages through the primary external public interface and, in a failover situation, the private interface. Should the backup LVS router fail to receive a heartbeat message within an expected interval, it initiates a failover and assumes the role of the active LVS router.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.21. Three-Tier LVS Topology This topology is suited well for busy FTP servers, where accessible data is stored on a central, highly available server and accessed by each real server via an exported NFS directory or Samba share. This topology is also recommended for websites that access a central, high-availability database for transactions.
Routing Methods a Red Hat cluster, you can configure one high-availability cluster to serve both of these roles simultaneously. 8.3. Routing Methods You can use Network Address Translation (NAT) routing or direct routing with LVS. The following sections briefly describe NAT routing and direct routing with LVS. 8.3.1. NAT Routing Figure 1.22, “LVS Implemented with NAT Routing”, illustrates LVS using NAT routing to move requests between the Internet and a private network. Figure 1.22.
Chapter 1. Red Hat Cluster Suite Overview interface are taken over by the backup LVS router simultaneously. All the real servers on the private network use the floating IP for the NAT router as their default route to communicate with the active LVS router so that their abilities to respond to requests from the Internet is not impaired. In the example, the LVS router's public LVS floating IP address and private NAT floating IP address are aliased to two physical NICs.
Routing Methods Figure 1.23. LVS Implemented with Direct Routing In a typical direct-routing LVS configuration, an LVS router receives incoming server requests through a virtual IP (VIP) and uses a scheduling algorithm to route the request to real servers. Each real server processes requests and sends responses directly to clients, bypassing the LVS routers.
Chapter 1. Red Hat Cluster Suite Overview In typical situations, a client on the Internet sends a request to an IP address. Network routers typically send requests to their destination by relating IP addresses to a machine's MAC address with ARP. ARP requests are broadcast to all connected machines on a network, and the machine with the correct IP/MAC address combination receives the packet.
Cluster Administration Tools 8.4.2. Firewall Marks Firewall marks are an easy and efficient way to a group ports used for a protocol or group of related protocols. For example, if LVS is deployed to run an e-commerce site, firewall marks can be used to bundle HTTP connections on port 80 and secure, HTTPS connections on port 443.
Chapter 1. Red Hat Cluster Suite Overview luci is accessible through a Web browser and provides three major functions that are accessible through the following tabs: • homebase — Provides tools for adding and deleting computers, adding and deleting users, and configuring user privileges. Only a system administrator is allowed to access this tab. • cluster — Provides tools for creating and configuring clusters. Each instance of luci lists clusters that have been set up with that luci.
Conga Figure 1.24. luci homebase Tab Figure 1.25.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.26. luci storage Tab 9.2. Cluster Administration GUI This section provides an overview of the system-config-cluster cluster administration graphical user interface (GUI) available with Red Hat Cluster Suite. The GUI is for use with the cluster infrastructure and the high-availability service management components (refer to Section 3, “Cluster Infrastructure” and Section 4, “High-availability Service Management”).
Cluster Administration GUI 9.2.1. Cluster Configuration Tool You can access the Cluster Configuration Tool (Figure 1.27, “Cluster Configuration Tool”) through the Cluster Configuration tab in the Cluster Administration GUI. Figure 1.27. Cluster Configuration Tool The Cluster Configuration Tool represents cluster configuration components in the configuration file (/etc/cluster/cluster.conf) with a hierarchical graphical display in the left panel.
Chapter 1. Red Hat Cluster Suite Overview • Cluster Nodes — Displays cluster nodes. Nodes are represented by name as subordinate elements under Cluster Nodes. Using configuration buttons at the bottom of the right frame (below Properties), you can add nodes, delete nodes, edit node properties, and configure fencing methods for each node. • Fence Devices — Displays fence devices. Fence devices are represented as subordinate elements under Fence Devices.
Command Line Administration Tools Cluster Management tab in Cluster Administration GUI. Figure 1.28. Cluster Status Tool The nodes and services displayed in the Cluster Status Tool are determined by the cluster configuration file (/etc/cluster/cluster.conf). You can use the Cluster Status Tool to enable, disable, restart, or relocate a high-availability service. 9.3.
Chapter 1. Red Hat Cluster Suite Overview Administration GUI and init scripts supplied by Red Hat. Table 1.1, “Command Line Tools” summarizes the command line tools.
CONTROL/MONITORING Web browser. You can access it locally with this URL: http://localhost:3636. You can access it remotely with either the hostname or the real IP address followed by :3636. If you are accessing the Piranha Configuration Tool remotely, you need an ssh connection to the active LVS router as the root user. Starting the Piranha Configuration Tool causes the Piranha Configuration Tool welcome page to be displayed (refer to Figure 1.29, “The Welcome Panel”).
Chapter 1. Red Hat Cluster Suite Overview Figure 1.30. The CONTROL/MONITORING Panel Auto update Enables the status display to be updated automatically at a user-configurable interval set in the Update frequency in seconds text box (the default value is 10 seconds). It is not recommended that you set the automatic update to an interval less than 10 seconds. Doing so may make it difficult to reconfigure the Auto update interval because the page will update too frequently.
GLOBAL SETTINGS 10.2. GLOBAL SETTINGS The GLOBAL SETTINGS panel is where the LVS administrator defines the networking details for the primary LVS router's public and private network interfaces. Figure 1.31. The GLOBAL SETTINGS Panel The top half of this panel sets up the primary LVS router's public and private network interfaces. Primary server public IP The publicly routable real IP address for the primary LVS node.
Chapter 1. Red Hat Cluster Suite Overview The next three fields are specifically for the NAT router's virtual network interface connected the private network with the real servers. NAT Router IP The private floating IP in this text field. This floating IP should be used as the gateway for the real servers. NAT Router netmask If the NAT router's floating IP needs a particular netmask, select it from drop-down list.
VIRTUAL SERVERS Figure 1.32. The REDUNDANCY Panel Redundant server public IP The public real IP address for the backup LVS router. Redundant server private IP The backup router's private real IP address. The rest of the panel is for configuring the heartbeat channel, which is used by the backup node to monitor the primary node for failure. Heartbeat Interval (seconds) Sets the number of seconds between heartbeats — the interval that the backup node will check the functional status of the primary LVS node.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.33. The VIRTUAL SERVERS Panel Each server displayed in the VIRTUAL SERVERS panel can be configured on subsequent screens or subsections. To add a service, click the ADD button. To remove a service, select it by clicking the radio button next to the virtual server and click the DELETE button. To enable or disable a virtual server in the table click its radio button and click the (DE)ACTIVATE button.
VIRTUAL SERVERS any of the subsections related to this virtual server, complete this page and click on the ACCEPT button. Figure 1.34. The VIRTUAL SERVERS Subsection Name A descriptive name to identify the virtual server. This name is not the hostname for the machine, so make it descriptive and easily identifiable. You can even reference the protocol used by the virtual server, such as HTTP. Application port The port number through which the service application will listen.
Chapter 1. Red Hat Cluster Suite Overview Virtual IP Network Mask The netmask for this virtual server, in the drop-down menu. Firewall Mark For entering a firewall mark integer value when bundling multi-port protocols or creating a multi-port virtual server for separate, but related protocols. Device The name of the network device to which you want the floating IP address defined in the Virtual IP Address field to bind.
VIRTUAL SERVERS drop-down menu. 10.4.2. REAL SERVER Subsection Clicking on the REAL SERVER subsection link at the top of the panel displays the EDIT REAL SERVER subsection. It displays the status of the physical server hosts for a particular virtual service. Figure 1.35. The REAL SERVER Subsection Click the ADD button to add a new server. To delete an existing server, select the radio button beside it and click the DELETE button.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.36. The REAL SERVER Configuration Panel This panel consists of three entry fields: Name A descriptive name for the real server. Tip This name is not the hostname for the machine, so make it descriptive and easily identifiable. Address The real server's IP address. Since the listening port is already specified for the associated virtual server, do not add a port number.
VIRTUAL SERVERS Weight An integer value indicating this host's capacity relative to that of other hosts in the pool. The value can be arbitrary, but treat it as a ratio in relation to other real servers. 10.4.3. EDIT MONITORING SCRIPTS Subsection Click on the MONITORING SCRIPTS link at the top of the page. The EDIT MONITORING SCRIPTS subsection allows the administrator to specify a send/expect string sequence to verify that the service for the virtual server is functional on each real server.
Chapter 1. Red Hat Cluster Suite Overview dynamically changing data, such as HTTPS or SSL. To use this function, you must write a script that returns a textual response, set it to be executable, and type the path to it in the Sending Program field. Note If an external program is entered in the Sending Program field, then the Send field is ignored. Send A string for the nanny daemon to send to each real server in this field. By default the send field is completed for HTTP.
Chapter 2. Red Hat Cluster Suite Component Summary This chapter provides a summary of Red Hat Cluster Suite components and consists of the following sections: • Section 1, “Cluster Components” • Section 2, “Man Pages” • Section 3, “Compatible Hardware” 1. Cluster Components Table 2.1, “Red Hat Cluster Suite Software Subsystem Components” summarizes Red Hat Cluster Suite components. Function Components Description Conga luci Remote Management System Management Station.
Chapter 2. Red Hat Cluster Suite Component Summary Function Components Description created with GFS 6.0 (and earlier) to the XML format configuration format used with this release of Red Hat Cluster Suite. Cluster Manager (CMAN) ccs_test Diagnostic and testing command that is used to retrieve information from configuration files through ccsd. ccsd CCS daemon that runs on all cluster nodes and provides configuration file data to cluster software. cluster.conf This is the cluster configuration file.
Cluster Components Function Components Description membership and services running. Fence clurgmgrd Daemon used to handle user service requests including service start, service disable, service relocate, and service restart. clurmtabd Daemon used to handle Clustered NFS mount tables. fence_apc Fence agent for APC power switch. fence_bladecenter Fence agent for for IBM Bladecenters with Telnet interface.
Chapter 2. Red Hat Cluster Suite Component Summary Function Components Description environments. DLM fence_ack_manual User interface for fence_manual agent. fence_node A program which performs I/O fencing on a single node. fence_xvm I/O Fencing agent for Xen virtual machines. fence_xvmd I/O Fencing agent host for Xen virtual machines. fence_tool A program to join and leave the fence domain. fenced The I/O Fencing daemon. libdlm.so.
Cluster Components Function LVS Components Description gnbd_serv A server daemon that allows a node to export local storage over the network. pulse This is the controlling process which starts all other daemons related to LVS routers. At boot time, the daemon is started by the /etc/rc.d/init.d/pulse script. It then reads the configuration file /etc/sysconfig/ha/lvs.cf. On the active LVS router, pulse starts the LVS daemon.
Chapter 2. Red Hat Cluster Suite Component Summary Function Quorum Disk Components Description nanny The nanny monitoring daemon runs on the active LVS router. Through this daemon, the active LVS router determines the health of each real server and, optionally, monitors its workload. A separate process runs for each service defined on each real server. lvs.cf This is the LVS configuration file. The full path for the file is /etc/sysconfig/ha/lvs.cf.
Man Pages • ccs (7) - Cluster Configuration System • cman_tool (8) - Cluster Management Tool • cluster.
Chapter 2.
Compatible Hardware • pulse (8) - heartbeating daemon for monitoring the health of cluster nodes • lvs.
64
Index C cluster displaying status, 41 cluster administration displaying cluster and service status, 41 cluster component compatible hardware, 63 cluster component man pages, 60 cluster components table, 55 Cluster Configuration Tool accessing, 40 cluster service displaying status, 41 command line tools table, 41 compatible hardware cluster components, 63 Conga overview, 35 Conga overview, 35 F feedback, ix I introduction, vii other Red Hat Enterprise Linux documents, vii L LVS direct routing requirements
66
