HP StorageWorks XP Cluster Extension Software Administrator Guide This guide contains detailed instructions for configuring and troubleshooting HP StorageWorks XP Cluster Extension Software in AIX, Windows, Solaris, and Linux environments. The intended audience has independent knowledge of related software and of the HP StorageWorks XP disk array and its software.
Legal and notice information © Copyright 2003-2010 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Contents 1 XP Cluster Extension features ............................................................. 11 Integration into cluster software ................................................................................................... Enhanced disaster tolerance ....................................................................................................... Automated monitoring and redirecting of XP Continuous Access Software pairs ................................ Rolling disaster protection ...
Configuring XP Cluster Extension ................................................................................................ Starting the XP Cluster Extension configuration tool ................................................................. Defining XP Cluster Extension configuration information using the GUI ....................................... Defining XP Cluster Extension configuration information using the CLI ........................................
Including the XP Cluster Extension resource type ........................................................................... Configuring the XP Cluster Extension resource ............................................................................... XP Cluster Extension resource types ....................................................................................... Resource type definition ......................................................................................................
Rescanning multipath devices ................................................................................................... Configuring the rescan script ............................................................................................. Finding the user-friendly name of a multipath device ............................................................. Configuring the pair/resync monitor ..........................................................................................
Failover error handling ............................................................................................................ HACMP-specific error handling ................................................................................................. Start errors ...................................................................................................................... Failover errors .......................................................................................................
Figures 1 One-to-one (1:1) configuration ................................................................................. 13 2 Consolidated-site configuration ................................................................................. 14 3 HACMP configuration example ................................................................................. 29 4 Service or application example (quorum service control disks not shown) ....................... 67 5 CLX_FILESHARE resource sample ...................
Tables 1 Setting resource properties and values in the GUI ........................................................ 47 2 Service or application properties and values .............................................................. 48 3 XP disk pair states ................................................................................................. 119 4 Cluster software supported objects .......................................................................... 125 5 Document conventions .................
1 XP Cluster Extension features HP StorageWorks XP Cluster Extension Software monitors HP StorageWorks XP Continuous Access Software disk pairs and enables automatic access to remote data copies when clustered applications become unavailable locally. XP Cluster Extension integrates with popular cluster software to ensure that consistent and concurrent data copies on HP disk arrays can be accessed when needed.
Rolling disaster protection A rolling disaster is a catastrophic event that affects the remote site after an outage at the local site. In a rolling disaster, data stored on remote disks can be entirely lost during a recovery attempt. To ensure the survival of critical data during a resynchronization/restore operation, HP StorageWorks XP Business Copy Software pairs can be associated with the local data disks.
Figure 1 One-to-one (1:1) configuration . Each cluster host node needs redundant FC or SCSI I/O paths to the XP disk array. Individual hosts cannot be connected to both the primary (P-VOL) and the secondary (S-VOL) copy of the application disk set. HP recommends a minimum of two cluster host nodes per site. This allows for a preferred local failover in case of a system failure.
Cluster Extension does not support configurations in which the application service's data disk set is spread over two or more XP disk arrays. Figure 2 Consolidated-site configuration .
or remote site, the other site together with the added node would have a majority. In a MNS with File Share Witness configuration, the, file share should be located at the third site. TIP: To upgrade XP firmware while the application service is running, use host load balancing and multipathing software, such as Auto Path, HP MPIO Full Featured Device Specific Module (DSM) for XP family of Disk Arrays (HP MPIO XP DSM), or VERITAS for Sun Solaris.
quorum depends on the defined no-quorum policy. This behavior is in effect until the cluster is fenced. When the cluster is fenced, the resources owned by the fenced nodes fail over to active cluster nodes. STONITH STONITH is an SLE HA cluster fencing method. SLE HA cluster provides STONITH plug-ins for devices such as UPS, PDU, Blade power control devices, and lights out devices.
operation. The interval attribute defines the time interval in which the monitor operation is executed. The timeout attribute determines how long to wait before considering the resource as failed. Define start, stop, and monitor operations for the XP Cluster Extension resource. XP Cluster Extension resource dependency A Group resource in an SLE HA cluster ensures that the member resource agents are started and stopped in the required order.
RHCS cluster setup considerations Quorum In RHCS, the quorum is based on a simple voting majority of the defined nodes in a cluster. To re-form successfully, a majority of all possible votes is required. Each cluster node is assigned a number of votes, and they contribute to the cluster while they are members. If the cluster has a majority of all possible votes, it has quorum (also called quorate); otherwise, it does not have quorum.
A failover domain can have the following characteristics: • Unrestricted: Specifies that the subset of members is preferred, but the cluster service assigned to this domain can run on any available member. • Restricted: The cluster service is allowed to run only on a subset of failover domain members. • Unordered: The member on which the cluster service runs is chosen from the available list of failover domain members with no preference order.
Disk monitoring For the situations in which disk access is lost or read/write protection is in effect due to storage fencing, application monitoring agents, file system agents, or LVM resource agents detect the IO failure. XP Cluster Extension does not monitor the disk access status. Setting up XP RAID Manager In addition to the cluster software it integrates with, XP Cluster Extension depends on HP StorageWorks XP RAID Manager.
Creating XP RAID Manager device groups A device group is the unit in which the failover/failback operation is performed. A device group can contain several volume groups. Configure a single device group for a service group (VCS), resource group (HACMP), cluster group (MSCS), or resource (SLE HA, RHCS). This device group must include all disks being used for the application service. The XP RAID Manager configuration file (horcmX.conf) is used to map device groups to the internal disk array disks.
Starting and stopping the XP RAID Manager instances Start the XP RAID Manager instances for XP Cluster Extension at system boot time to provide the fastest access to disk status information. XP Cluster Extension provides scripts (Linux/UNIX) or a service (Windows) to integrate the XP RAID Manager instance startup into the system startup process. This feature reduces resource group failover times because the XP Cluster Extension resource does not need to start the XP RAID Manager instances.
The output of the pairdisplay command indicates whether the local disk is the secondary (S-VOL) disk and if so, the horctakeover command shows a SWAP-takeover as a result. If pairdisplay shows the local disk as the primary (P-VOL) disk, log in to a system connected to the secondary (S-VOL) disk and invoke the horctakeover command there. If the horctakeover command does not result in a SWAP-takeover, see “Recovery sequence” on page 120 to resolve the issue.
XP Cluster Extension features
2 Configuring XP Cluster Extension for AIX XP Cluster Extension is integrated with the HACMP cluster software using HACMP customization functions. Cluster administrators configure disk array failover as a pre-event of the standard HACMP event get_disk_vg_fs. For information about how to install XP Cluster Extension in an IBM HACMP environment, see the HP StorageWorks XP Cluster Extension Software installation guide. For supported operating system versions, see the HP SPOCK website: http://www.hp.
Configuring the pair/resync monitor port Enter the port that the pair/resync monitor will monitor. 1. Open the /etc/services file. 2. Choose the port that the pair/resync monitor will use, and then add the following line to the services file: clxmonitor nnnnn /tcp where nnnnn is the port number. For example: clxmonitor clxmonitor 22222/udp 22222/tcp # CLX Pair/Resync Monitor # CLX Pair/Resync Monitor Integrating XP Cluster Extension into HACMP 1. Create a new Custom Cluster Event: #smitty hacmp 2.
4. Configure the newly created Custom Cluster Event as a pre-event of get_disk_vg_fs: #smitty hacmp 5. Select the following options: 6. • Extended Configuration • Extended Event Configuration • Change/Show Pre-Defined HACMP Events Select the event get_disk_vg_fs. 7. Define the previously defined custom event get_disk_vg_fs_pre as a pre-event of get_disk_vg_fs. NOTE: With HACMP 5.2 and later, to have the get_disk_vg_fs event called, you must specify Serial Acquisition Order.
9. Specify the resource groups configured to run with XP Cluster Extension. 10. Once the resource groups have been specified, press Enter to complete the configuration process. XP Cluster Extension controls the disk pairs based on XP RAID Manager device groups. The volume group definition of the HACMP resource group is used to determine the corresponding XP RAID Manager device group.
Figure 3 on page 29 and the examples that follow show two possible mappings. Figure 3 HACMP configuration example . Example 1 The application OracleRG corresponds to an HACMP resource group OracleRG, which consists of the volume groups ora1vg and ora2vg. The corresponding XP RAID Manager device group oracle controls all disks which form the volume groups of the HACMP resource group.
COMMON LogDir /var/opt/hpclx/log/ #default (optional) LogLevel error # error|info default: error (optional) APPLICATION OracleRG # package/service group test_application Vgs ora1vg ora2vg # HACMP specific, to map vg to OracleRG ApplicationDir /etc/opt/hpclx XPSerialNumbers 30368 30380 RaidManagerInstances 11 DeviceGroup oracle # raid manager device group FenceLevel data # values: data | never | async ApplicationStartup resyncwait # values: fastfailback | resyncwait AutoRecover no # possible values: yes | no
3. Select a resource group from the list of available groups. Taking a resource group offline Resource groups will usually be taken offline automatically when the cluster is stopped on a particular system. If necessary, a resource group can be brought offline manually: 1. Run SMIT (HACMP section): #smitty hacmp 2. Select the following: 3.
Pair/resync monitor integration The pair/resync monitor is used to detect and react to suspended XP Continuous Access links. It is activated by setting the ResyncMonitor object to YES. Additionally, the automatic disk pair resynchronization feature is activated if the ResyncMonitorAutoRecover object value is YES. When the HACMP resource group is taken offline, disable the monitor for the XP RAID Manager device group used for this resource group.
3. Enter the following values: • Cluster Event Name: release_vg_fs_pre • Cluster Event Description: XP Cluster Extension Pre-Event • Cluster Event Script Filename: /opt/hpclx/bin/clxstopmonhacmp Redefining the Custom Cluster Event as a pre-event of the standard HACMP event 1. Run SMIT (HACMP section): #smitty hacmp 2. Select the following: • Extended Configuration • Extended Event Configuration • Change/Show Pre-Defined HACMP Events 3. Select event release_vg_fs.
4. Define the previously defined Custom Cluster Event release_vg_fs_pre as a pre-event of release_vg_fs. Timing considerations XP Cluster Extension is designed to prefer XP disk array operations over cluster software operations. If XP Cluster Extension invokes disk pair resynchronization operations or gathers information about the remote XP disk array, XP Cluster Extension will wait until the requested status information is reported.
• If running in fence level ASYNC, the default value of the AsyncTakeoverTimeout can cause the resource group online process to fail because its value is set very high. This is because the takeover process for fence level ASYNC can take longer when slow communication links are in place. To prevent takeover commands from being terminated prematurely by the takeover timeout, measure the time to copy the installed XP disk array cache.
Configuring XP Cluster Extension for AIX
3 Configuring XP Cluster Extension for Windows After installing XP Cluster Extension, use the configuration tool to define the XP Cluster Extension setup configuration. After you configure XP Cluster Extension, use Cluster Administrator (Windows Server 2003), Failover Cluster Management (Windows Server 2008/2008 R2), or cluster commands in the CLI to add and configure resources. For information about how to install XP Cluster Extension, see the HP StorageWorks XP Cluster Extension installation guide.
NOTE: The service name clxmonitor is appended with the text “(not configured)” unless the port number is configured in the configuration tool. Defining XP Cluster Extension configuration information using the GUI 1. Open the configuration tool. For instructions, see “Starting the XP Cluster Extension configuration tool” on page 37. 2.
3. 4. Specify the XP RAID Manager instances that define the device groups you want to manage with XP Cluster Extension. For more information about XP Cluster Extension and XP RAID Manager, see “Setting up XP RAID Manager” on page 20. a. Click Add in the XP RAID Manager Instance Configuration section to open the Add XP RAID Manager instances window. b. Select the XP RAID Manager instances to use, and then click OK. Specify the servers that are possible owners for the XP Cluster Extension-managed disks.
5. Click OK to save the information and close the configuration tool. The configuration information is saved to the ClxXPCfg file. NOTE: XP Cluster Extension provides an XP RAID Manager service, which automatically starts XP RAID Manager instances at system boot time. This feature reduces resource group and service and application failover times because the XP Cluster Extension resource does not need to start the XP RAID Manager instances.
3. Specify the XP RAID Manager instances that define the device groups you want to manage with XP Cluster Extension. For more information about XP Cluster Extension and XP RAID Manager, see “Setting up XP RAID Manager” on page 20. • To view the available XP RAID Manager instances, enter CLXXPCONFIG RM. • To add an XP RAID Manager instance, enter CLXXPCONFIG RM /ADDVAL=value, where value is the XP RAID Manager instance you want to add.
Exporting configuration settings using the CLI 1. Open a command window. 2. Enter CLXXPCONFIG EXPORT /FILE=filepath, where filepath specifies the save location and file name. Importing configuration settings using the GUI 1. Open the configuration tool. For instructions, see “Starting the XP Cluster Extension configuration tool” on page 37. 2. Click Import. 3. When prompted, choose the configuration file, and then click Open. 4. Click OK to save and close the configuration tool.
CAUTION: Do not use the following characters in XP Cluster Extension resource names: \ / : * ? " < > |. Using these characters might affect the creation of the resourcename.online file, which is used for the XP Cluster Extension resource health check mechanism. If the file creation fails and the pair/resync monitor is not configured, the cluster will report a failed state for the XP Cluster Extension resource.
Adding an XP Cluster Extension resource using the Failover Cluster Management GUI (Windows Server 2008/2008 R2) Use the procedure in this section to add a resource using the Failover Cluster Management GUI. For instructions on using the CLI, see “Adding an XP Cluster Extension resource using the Microsoft CLI cluster commands” on page 44. 1. Add a service or application in the Failover Cluster Management GUI, as described in your Microsoft documentation. 2.
1. Open Cluster Administrator. 2. Open the resource Properties window and click the General tab. 3. Enter a new name in the Name field. 4. Click OK to save your changes and close the window. Changing an XP Cluster Extension resource name (Windows Server 2008/2008 R2) In this procedure, you use the Failover Cluster Management GUI to change a resource name.
is configured for that resource. If the device group is the last monitored disk pair, and you take the resource offline, the pair/resync monitor will be stopped. • Windows Server 2008 only: If an XP Cluster Extension resource is not configured, the resource icon in the Failover Cluster Management GUI shows the message “not configured” next to the resource status. • The XP Cluster Extension resource must be the first resource for all disk resources in the dependency list of a resource cluster group.
For more information about setting resource properties, see your Microsoft documentation. Table 1 Setting resource properties and values in the GUI Property Format Description Value Thorough Resource Health Check Interval (Windows Server 2008/2008 R2) Integer Used to poll “Alive” state for the resource. Decreasing this value allows faster resource failure detection but also consumes more system resources.
Property Format Description Value Pending timeout (GUI) Integer Used to specify the timeout for status resolution. For more information, see “Timing considerations for Microsoft Cluster Service ” on page 72. Windows Server 2008/2008 R2: 03:00 mm:ss PendingTimeout (CLI) Set this value in the Policies tab of the resource properties window in Failover Cluster Management, or in the Advanced tab of the resource properties window in Cluster Administrator.
Setting XP Cluster Extension-specific resource properties Changes to resource properties take effect when the resource is brought online again.
3. To change the device group details, select a new value in the RM XP device group menu. 4. Click OK to save your changes and close the window. Configuring XP RAID Manager device group advanced properties The Parameters tab of the XP Cluster Extension resource offers basic settings and is used to enter environment data, such as XP RAID Manager instances. The more advanced settings can be accessed through additional buttons in the Parameters tab. 1.
• For the journal fence level, you can update the Journal data currency on S-VOL and ASYNC takeover timeout values. See JournalDataCurrency on page 136 and AsyncTakeoverTimeout on page 130 for more information about these values. 4. Update the settings as needed, and then click OK to close the window. 5. Click OK to save your changes and close the window.
3. Click Failover/Failback to display the Failover/Failback window. 4. Update the ApplicationStartup and AutoRecover values as needed, and then click OK. 5. Click OK to save your changes and close the window. Activating the pair/resync monitor The pair/resync monitor detects and responds to suspended XP Continuous Access links if the ResyncMonitor object is set to YES. If the ResyncMonitorAutoRecover object is set to YES, automatic disk pair resynchronization is also activated.
5. Select the Pair/resync monitor autoRecovery check box to set the ResyncMonitorAutoRecover object is to YES. 6. If you want to change the monitoring interval (ResyncMonitorInterval), enter a value in the Monitor interval box. 7. Click OK to save your changes and close the Pair/Resync Monitor Properties window. 8. Click OK to save your changes and close the Properties window. TIP: You can activate ResyncMonitor from cluster commands in the CLI.
3. Click Rolling Disaster to display the Rolling Disaster Protection window. 4. Add mirror units to each data center: a. Click Add MU # to DC A. b. Select mirror units from the list, and click OK. c. Repeat the previous steps for Data Center B. 5. Update the BCResyncEnabledA, BCResyncEnabledB, BCResyncMuListA, and BCResyncMuListB values as needed, and then click OK. 6. Click OK to save your changes and close the Properties window.
TIP: For information on managing XP Cluster Extension resources from a remote management station through the MMC, see “Setting XP Cluster Extension resource properties using the MMC” on page 62. Using Failover Cluster Management to set resource properties (Windows Server 2008/2008 R2) For Windows Server 2008/2008 R2, use the Failover Cluster Management GUI to set resource properties. 1. Open Failover Cluster Management. 2.
• You can use the tool to configure multiple resources at one time. This saves time because you can switch resources from the tool menu. • The resource configuration tool is recommended for Hyper-V and Server Core environments because the properties you enter are validated. When you configure XP Cluster Extension resource properties from a remote management station or through the CLI, the properties you enter are not validated.
To use the resource configuration tool: 1. Open a command window and enter ClxXpResConfig.exe. 2. Select the resource you want to change in the XP CLX resource menu. 3. Make the necessary parameter changes, and then click OK. Configuring XP RAID Manager instance numbers for XP RAID Manager service To configure XP RAID Manager instance numbers from the Failover Cluster Management Parameters tab or the resource configuration tool: 1. To add an instance: a.
Configuring the XP RAID Manager device group details To configure XP RAID Manager device group details from the Failover Cluster Management Parameters tab or the resource configuration tool: 1. Select a value in the RM XP device group menu. 2. Click OK to save your changes and close the window. Configuring XP RAID Manager device group advanced properties The Parameters tab of the XP Cluster Extension resource offers basic settings and is used to enter environment data, such as XP RAID Manager instances.
• For the ASYNC fence level, you can update the ASYNC takeover timeout value. See AsyncTakeoverTimeout on page 130 for more information about this value. • For the journal fence level, you can update the Journal data currency on S-VOL and ASYNC takeover timeout values. See JournalDataCurrency on page 136 and AsyncTakeoverTimeout on page 130 for more information about these values. 2. Update the settings as needed, and then click OK to close the window. 3.
4. Click OK to save your changes and close the window. Changing failover and failback behavior To configure failover and failback behavior from the Failover Cluster Management Parameters tab or the resource configuration tool: 1. Click Failover/Failback to display the Failover/Failback window. 2. Update the ApplicationStartup and AutoRecover values as needed, and then click OK. 3. Click OK to save your changes and close the Properties window or Resource Configuration tool.
1. Click Pair ResyncMon to open the Pair/Resync Monitor Properties window. 2. Select the Use pair/resync monitor check box to set the ResyncMonitor object to YES. 3. Select the Pair/resync monitor autoRecovery check box to set the ResyncMonitorAutoRecover object is to YES. 4. If you want to change the monitoring interval (ResyncMonitorInterval), enter a value in the Monitor interval box. 5. Click OK to save your changes and close the Pair/Resync Monitor Properties window. 6.
2. Update the PreExecScript, PostExecScript, and PostExecCheck values as needed, and then click OK. When configuring pre/post takeover executable paths, enter the full path to the script. If a script fails, the XP Cluster Extension resource will fail. 3. Click OK to save your changes and close the Properties window or Resource Configuration tool.
NOTE: When you configure XP Cluster Extension resource properties from a remote management station through the MMC, which uses the standard Microsoft Properties tab, the properties you enter are not validated, so you must enter the property values accurately, and verify them against the XP Cluster Extension documentation. When you use this option, you will see the default Microsoft properties page instead of the XP Cluster Extension Parameters tab.
The following example changes the FenceLevel property of the XP Cluster Extension resource clx_fileshare: C:\>cluster resource clx_fileshare /privprop FenceLevel=data The following example changes the XP RAID Manager instance used for the XP Cluster Extension resource clx_fileshare from 10 to 99, and then adds instance 22 to provide redundancy: C:\>cluster resource clx_fileshare /privprop RaidManagerInstances="99 22" The following example changes the name of XP Cluster Extension resource XP Cluster Extens
• For Windows Server 2008/2008 R2, use the Failover Cluster Management GUI, cluster commands in the CLI, or the MMC for remote management. • For Server Core or Hyper-V Server, use cluster commands in the CLI or the MMC. Adding dependencies using Cluster Administrator (Windows Server 2003) 1. Open Cluster Administrator. 2. Select the Resources folder in the console-tree. 3. Double-click the disk resource you want to edit. 4. Click the Dependencies tab, then click Modify. 5.
Adding dependencies using the CLI The cluster commands in this section can be used with Windows Server 2003, Windows Server 2008/2008 R2, Server Core, or Hyper-V Server.
Figure 4 Service or application example (quorum service control disks not shown) . Figure 5 CLX_FILESHARE resource sample .
Figure 6 XP Cluster Extension resource tree for CLX_SHARE . XP Cluster Extension is configured as a single resource to enable read/write access to the physical disk resource used for the CLX_SHARE cluster group. The physical disk resource depends on the XP Cluster Extension resource and can be brought online only when the XP Cluster Extension resource is already online.
9. For Windows Server 2003 only: Set the XP Cluster Extension resource property RestartAction to zero (0), or check the Do not restart check box in the resource's Advanced tab window, and then use the following commands to check if the value has changed. For example: C:\>cluster resource CLX_FILESHARE /prop RestartAction=0 C:\>cluster resource CLX_FILESHARE /prop If you are using the CLI to set resource properties, the equivalent command is cluster res CLX_FILESHARE /prop RestartAction=0. 10.
21. Change the XP Cluster Extension resource to be able to restart on another system: C:\>cluster resource CLX_FILESHARE /prop RestartAction=2 C:\>cluster resource CLX_FILESHARE /prop Managing XP Cluster Extension resources You can manage resources by bringing them online and offline, or by deleting them. Bringing a resource online Resources are usually brought online automatically when the service or application is brought online.
CAUTION: Failure to delete the monitored device group from the list of monitored device groups can cause data corruption if the ResyncMonitorAutoRecover attribute is set to YES. When deleting resources: • For Windows Server 2008/2008 R2, use the GUI or CLI. • For Server Core or Hyper-V Server, use the CLI or the MMC. • For Windows Server 2003, use the GUI or CLI. For more information on deleting resources, see your Microsoft documentation.
Timing considerations for MSCS XP Cluster Extension gives priority to XP disk array operations over cluster software operations. If XP Cluster Extension invokes a disk pair resynchronization operation or gathers information about the remote XP disk array, XP Cluster Extension waits until the requested status information is reported. This ensures the priority of data integrity over cluster software failover processes.
Administration XP Cluster Extension administration includes remote management of resources and monitoring of system resources and logs. Remote management of XP Cluster Extension resources in a cluster (Windows Server 2008/2008 R2) You can use the MMC with Failover Cluster Management to manage clusters and configure XP Cluster Extension resources.
you will see the customized XP Cluster Extension Parameters tab. The customized tab is displayed because the resource extension DLL is registered by default on Windows Server 2003 cluster nodes, which prevents you from configuring the XP Cluster Extension resource from a remote management station. If you need to configure an XP Cluster Extension resource remotely for a Windows Server 2003based cluster, unregister clxmscsex.
4 Configuring XP Cluster Extension for Solaris HP StorageWorks XP Cluster Extension for VCS provides a resource agent to VCS. This allows cluster administrators to configure the XP disk array-specific failover behavior as easily as any other resource in VCS. XP Cluster Extension objects are configured as attributes of a resource in VCS. For information about how to install XP Cluster Extension, see the HP StorageWorks XP Cluster Extension installation guide.
Example of the clx_web resource ClusterExtensionXP clx_web ( XPSerialNumbers = { 35014, 35013 } RaidManagerInstances = { 11 } DeviceGroup = web PreExecScript = "/etc/opt/hpclx/web_pre.sh" PostExecScript = "/etc/opt/hpclx/web_post.sh" DC_A_Hosts = { sunrise, dawn } DC_B_Hosts = { sunset, dusk } ) Figure 7 VERITAS Cluster Service configuration example . Figure 8 on page 77 shows an example resource graph of the CLX_WEB_SERVER service group.
Figure 8 Sample resource graph of the CLX_WEB_SERVER service group . Configuring the XP Cluster Extension agent according to Figure 7 1. Log in to system sunrise as root. 2. Create the XP Cluster Extension resource (for example, clx_web) in the $VCS_CONF/config/ main.cf file, using the previous example. 3. Link the new resource as a child resource to all disk resources in the service group. 4. Edit the attributes in the file $VCS_CONF/config/main.cf to configure your XP Cluster Extension resource.
10. Start the VCS engine on dawn: #hastart 11. Start the VCS engine on sunset and dusk, and switch the web server service group to dawn and later to sunset and dusk. Before you switch the service group to the remote data center, make sure that the XP Continuous Access Software links are configured for bidirectional mirroring and that XP RAID Manager instances include the device group, configured for the web server service group.
Including the XP Cluster Extension resource type See “Importing the XP Cluster Extension resource types configuration file” in the HP StorageWorks XP Cluster Extension Software Installation Guide. Configuring the XP Cluster Extension resource For VCS, you can configure an XP Cluster Extension resource using either the VCS CLI or the VCS Cluster Manager GUI.
BCEnabledA, BCEnabledB BCResyncEnabledA, BCResyncEnabledB } str ApplicationDir = "/etc/opt/hpclx/" str XPSerialNumbers[] str RaidManagerInstances[] str DeviceGroup str DC_A_Hosts[] str DC_B_Hosts[] str FenceLevel = never str DataLoseMirror = no str DataLoseDataCenter = yes str JournalDataCurrency = yes int AsyncTakeoverTimeout = 1800 str ApplicationStartup = fastfailback int ResyncWaitTimeout = 300 str FastFailbackEnabled = yes str AutoRecover = no str ResyncMonitor = no str ResyncMonitorAutoRecover = no st
2. Select the service group, right-click, and choose Add Resource; or, click Add Resource in the Cluster Explorer toolbar. 3. Enter the resource name in the Resource name box. 4. Select ClusterExtensionXP from the Resource Type list.
5. To change attribute values of the new XP Cluster Extension resource, click the button in the Edit column of the value you want to change, and modify the values as desired in the Edit Attribute window. 6. Select the Critical and Enabled boxes in the Add Resource window. 7. Click OK.
Changing XP Cluster Extension attributes XP Cluster Extension resource attributes can be changed after the configuration has been write-enabled. To change attribute values of the XP Cluster Extension resource the resource must be taken offline. Changing an attribute of the XP Cluster Extension resource while the resource is running is not supported.
3. Click the Properties tab in the View Panel. 4. Click Edit for the attribute you want to change. 5. Enter changes to the attribute value. For nonscalar attributes, use the + and x buttons to add or remove elements. Do not change the attribute's scope to local; all XP Cluster Extension attributes are global in scope. 6. Click OK to accept the change. Linking an XP Cluster Extension resource XP Cluster Extension must be the first resource in the resource chain of a VCS service group.
Linking other resources using the VCS Cluster Manager GUI 1. Open the Cluster Explorer 2. Click the Resources View tab on the View Panel. 3. Click the resource icon of the resource that is to be the parent resource. 4. Move the yellow line to the resource that is to be the (child) XP Cluster Extension resource and click to link the child to the parent resource selected in step 3. 5. Click YES in the dialog box to confirm the dependency.
4. Click YES in the dialog box to confirm your selection. Taking an XP Cluster Extension resource offline Resources are usually taken offline automatically when the service group is taken offline. There are two ways to manually take a resource group offline: • Take only the specified resource offline. • Propagate the offline request to all parent resources, which takes all parent resources offline before the specified resource.
Deleting an XP Cluster Extension resource These procedures remove an XP Cluster Extension resource from an existing service group. Deleting a resource using the VCS CLI Syntax hares –delete ClusterExtensionXP_resource CAUTION: Deleting a running XP Cluster Extension resource does not remove the resource_name.online file and does not remove the device group from the list of monitored device groups if the pair/resync monitor is used to monitor the XP Continuous Access Software link.
• If the service group is offline, you can remove the XP Cluster Extension resource from the service group. To remove the resource, see “Deleting an XP Cluster Extension resource” on page 87. Pair/resync monitor integration The pair/resync monitor is used to detect and react to suspended XP Continuous Access Software links. It is activated if the ResyncMonitor attribute value is set to YES.
In some cases, this behavior could lead to failed XP Cluster Extension resources: • XP Cluster Extension uses XP RAID Manager instances to communicate with the remote XP disk array. Depending on the settings of the XP RAID Manager instance timeout parameter and the number of remote instances, the online operation could time out. This can happen if the local XP RAID Manager instance cannot reach the remote XP RAID Manager instance.
system list (which should be a local system). This fails because the state of the local XP disk array has not changed. The service group fails until the service group is brought online on a system connected to the remote XP disk array. The service group online process takes longer and it does not access the VCS configuration file.
XP Cluster Extension resources go offline because the primary volume state changes from P-VOL_PAIR to P-VOL_PSUE and the secondary volume state changes from S-VOL_PAIR to EX_NORMT. The state combination P-VOL_PSUE and EX_NORMT is not designed to be handled automatically because the remote side (remote XP RAID Manager/ disk array), which has no status information available, could have more current data then the primary (P-VOL_PSUE) site.
Configuring XP Cluster Extension for Solaris
5 Configuring XP Cluster Extension for Linux XP Cluster Extension supports integration with the following cluster software for Linux: • Configuring XP Cluster Extension with RHCS, page 95 • Configuring XP Cluster Extension with SLE HA, page 102 NOTE: For a list of XP Cluster Extension versions and the cluster software versions they support, see the HP SPOCK website: http://www.hp.com/storage/spock.
The configuration example in Figure 9 on page 93 assumes the following information about the cluster: • There are four nodes in the cluster: Host1, Host2, Host3, and Host4. • There are two XP disk arrays with serial numbers 30047 and 30053. • The device group clxwebvgs is configured in the XP RAID Manager /etc/horcm101.conf file. • XP Cluster Extension invokes the pre-executable script clxweb_pre_takeover.sh and the postexecutable script clxweb_post_takeover.sh.
• XP Cluster Extension enables read/write access to the disk groups used for the web server service or resource group. Activation of the volume groups depends on a successful return code from XP Cluster Extension. The logical volumes can be mounted only when their volume groups are active and XP Cluster Extension allows read/write access to the disk group. After the file system for the web server's executables and content data is mounted and checked, the NIC is configured with the web server's IP address.
8. Enter /usr/share/cluster/clxxp.sh in the Full path to script file box. 9. Click Submit. Using system-config-cluster to create a shared resource To create an XP cluster extension shared resource using system-config-cluster: 1. Start system-config-cluster. 2. Click the Cluster Configuration tab. 3. Expand the Managed Resources tree. 4. Select the Resources tree. 5. Click Create a Resource to open the Resource Configuration dialog box. 6. Select Script in the Select a Resource Type box. 7.
Creating an RHCS service using the XP Cluster Extension shared resource After you create a shared resource, create an RHCS service using the XP Cluster Extension shared resource. Configuration overview 1. 2. 3. Create a service at the root of the dependency tree using the XP Cluster Extension shared resource created in “Creating an RHCS XP Cluster Extension shared resource” on page 95. This ensures that the XP Cluster Extension resource is the first resource to start in a service.
11. Click Submit. Conga saves the configuration information and updates all of the other cluster nodes. NOTE: To add additional resources to the service, use the Add a child feature. Using system-config-cluster to create a service To create an XP Cluster Extension service using system-config-cluster: 1. Start the Cluster Configuration tool. 2. Click the Cluster Configuration tab. 3. Expand the Managed Resources tree. 4. Select Services. The Service properties page appears. 5.
6. Enter the service name in the Name box, and then click OK. IMPORTANT: The service name must match the name that is defined for the APPLICATION property in the configuration file CLXXP.config. The Service Management dialog box appears. 7. Click Add a Shared Resource to this service. The Resource Configuration dialog box appears. 8. Select CLXXP in the Select a Resource Type menu, and then click OK. 9.
3. In the configuration file (CLXXP.config), enter the appropriate values for: • • • • • • • • • XPSerialNumbers RaidManagerInstances DeviceGroup DC_A_Hosts DC_B_Hosts ResyncMonitor FenceLevel DataLoseMirror DataLoseDataCenter NOTE: For more information about these values, see Chapter 8 on page 123.
4. Start the service on Host2. #clusvcadm –e CLXWEB –m Host2 5. Relocate the service to a remote data center node. a. Verify that the disks CLXWEB uses are in the PAIR state: #export HORCMINST=101 #pairdisplay –fcx –g clxwebvgs b. Move the service CLXWEB to Host3. Verify that service has successfully moved and started on Host3: #clusvcadm -r CLXWEB -m Host3 #clustat -s CLXWEB c. Verify that the disk pairs are now in read/write mode on the remote storage system: #pairdisplay –fcx –g clxwebvgs d.
NOTE: For instructions on stopping or disabling an XP Cluster Extension service using Conga or the Cluster Configuration Tool, see the RHCS documentation. Configuring XP Cluster Extension with SLE HA XP Cluster Extension Software is integrated with SLE HA using a configuration file and a custom resource agent. The executable clxxplxcs is called by XP Cluster Extension before volume group activation. This checks the status of a device group configured for use with an SLE HA agent.
3. In the configuration file (CLXXP.config), enter the appropriate values for: • • • • • • • • • XPSerialNumbers RaidManagerInstances DeviceGroup DC_A_Hosts DC_B_Hosts ResyncMonitor FenceLevel DataLoseMirror DataLoseDataCenter NOTE: For more information about these values, see Chapter 8 on page 123.
4. Select the following options for the XP Cluster Extension resource: Name Value Class ocf Provider heartbeat Type CLXXP 5. Configure the instance attributes for the resource by selecting the app parameter. In the Value box, enter the APPLICATION tag name configured in the XP Cluster Extension configuration file (/etc/opt/hpclx/CLXXP.config). 6. Configure the start, stop, and monitor operations for the XP Cluster Extension resource. 7. Add additional primitive resources to the group.
Creating an XP Cluster Extension resource for Heartbeat This procedure uses the Linux HA Management Client and Heartbeat. For specific instructions on using the GUI, see the Linux HA Management Client documentation. 1. Start the Linux HA Management Client. 2. Add a group resource for XP Cluster Extension with the following settings: Name Value ID Enter a resource group ID. Ordered true Collocated true The Linux HA Management Client prompts you to enter the resource type details. 3.
Testing the configuration The procedure and commands in this section are based the sample configuration in “XP Cluster Extension for Linux: Sample configuration” on page 93. Use this procedure as a guide for configuring your environment. Test the configuration by migrating the resource group to the remote data center nodes: 1. Verify that the disk pairs are in read-only mode on the remote storage system. 2. In the SLE HA GUI, click Management in the left pane.
1. Copy the multipath_rescan.sh script to the /etc/opt/hpclx/conf folder, and rename the file as follows: • RHCS: multipath_rescan_ServiceName.sh • SLE HA: multipath_rescan_ResourceGroupName.sh 2. Open the script file and enter the user-friendly names of all multipath devices that are in the volume groups configured for the RHCS service or SLE HA resource group.
2. Obtain the SCSI ID for a multipath device. Use the scsi_id command for SUSE Linux Enterprise Server, and the hp_scsi_id command for Red Hat Enterprise Linux. SUSE Linux Enterprise Server: [root@node1 ]# scsi_id -guns /block/dm-21 360060e8014424600000142460000039d Red Hat Enterprise Linux: [root@node1]# hp_scsi_id -guns /block/dm-14 360060e8014424600000142460000039d 3. Use the multipath command to obtain the user-friendly name for the multipath device's generated SCSI ID.
2. Choose the port that the pair/resync monitor will use, and then add the following line to the services file: clxmonitor nnnnn /tcp where nnnnn is the port number. For example: clxmonitor clxmonitor 22222/udp 22222/tcp # CLX Pair/Resync Monitor # CLX Pair/Resync Monitor Activating the pair/resync monitor The pair/resync monitor detects and reacts to suspended XP Continuous Access Software links. To activate the pair/resync monitor, set the ResyncMonitor object to YES.
and the XP firmware fully support delta resynchronization; however, the delta between the primary and secondary disks can be large enough for the copy process to exceed the service or resource group startup timeout value. • The ResyncWaitTimeout object can cause the resource to fail if its value is higher than the resource startup timeout value.
6 XP Cluster Extension and CLI XP Cluster Extension allows integration into almost any cluster software for commercial UNIX, Linux, and Windows operating systems. Use the XP Cluster Extension clxrun command to check proper functionality of XP Cluster Extension prior to integration with the cluster software. The CLI also allows integration of XP Continuous Access Software. For information on supported platforms, see the HP SPOCK website: http://www.hp.com/storage/spock.
of data to be transferred, it could take hours to resynchronize. If this is the case, clxrun may take some time to return. Do not stop clxrun; use it to check the status of the associated XP RAID Manager device groups. Even if the XP RAID Manager version and the XP firmware version allow a delta resynchronization, the amount of delta data to be transferred between the primary and the secondary could be long enough for the copy process to take a while.
APPLICATION netscape #the application service DeviceGroup netscapedg #RM dev group for the app service RaidManagerInstances 22 90#RM instance number for dev group XPSerialNumbers 34001 34005 #local and remote XP Serial Numbers DC_A_Hosts eserv1 eserv2 #data center A hostnames DC_B_Hosts eserv3 eserv4 #data center B hostnames CLI commands This section describes the following CLI commands: • clxrun, page 113 • clxchkmon, page 114 clxrun Check disk set Description clxrun can be used to manually prepare the
CAUTION: The forceflag option is implemented as an emergency switch to manually activate your XP disk set. If the forceflag option has been specified, XP Cluster Extension will not check any consistency or concurrency rules before activating the XP disk set. Return codes clxrun exits with one of the following return codes: 0 OK Application service can be started. 1 ERROR_GLOBAL Application service should not start on any system in either site on either disk array.
To update or remove a specific resource, use clxchkmon –n resource_name –g device_group. If –clx is not specified, the command is applied only to non-XP Cluster Extension resources. To update all non-XP Cluster Extension resources, use clxchkmon –t. To update XP Cluster Extension resources, use clxchkmon –clx –t.
Syntax clxchkmon [-clx] [-s host name] [-n resource_name —g device_group] [[-t monitor_interval | -autorecover mode | -remove [-force] | -show | -pid | -stopsrv | -log [error | warning | info | trace]]] [-p port number] where: -s hostname Specifies the name of a host. –n resource_name Specifies the resource (application) name as used in XP Cluster Extension. –g device_group Specifies an XP RAID Manager group name.
11 Invalid argument to pair/resync monitor. 12 Pair/resync monitor received signal (control-c) interrupt. 13 Unknown status for device group. 14 No port number is specified in services file for clxmonitor. 16 Invalid use of the –clx option on a non-XP Cluster Extension resource or XP Cluster Extension resource specified without the –clx option. 100 XP RAID Manager error. Related information For more information, see “Monitoring and resynchronizing device groups” on page 143.
XP Cluster Extension and CLI
7 XP Cluster Extension recovery procedures XP disk pair states Table 3 on page 119 provides basic XP disk pair state information. The XP disk pair state transition process is complex; see the HP StorageWorks XP Continuous Access and HP StorageWorks XP Continuous Access XP Journal user guides for more information.
State Description PDUB The disk is in a failure mode. Either the XP Continuous Access Software link is down, or the disk must be replaced. This is a special state of PSUE. If you have configured several disks into a LUSE configuration, where several LDEVS are combined to create an extended size disk and one or more disks are in an error condition, this state will be shown. PFUL This state is used to indicate that a threshold of the side file area in the XP disk array cache has been reached.
5. Display the pair status information after a P-VOL takeover (local P-VOL PSUS; remote S-VOL PAIR): pairdisplay –g device_group –fc To recover from these states, invoke the following command from the P-VOL side: pairresync –c 15 –g device_group CAUTION: The application must be shut down and the file systems unmounted before a fenced disk in fence level DATA can be set in read/write mode again. After the P-VOL takeover, the file system must be checked before it can be mounted.
XP Cluster Extension recovery procedures
8 User configuration file and XP Cluster Extension objects Objects (also called properties in this document) define the disk array environment and failover/failback behavior. Information comes directly from the cluster software, indirectly from the disk array through XP RAID Manager, and from a configuration file created by users. This file describes the dependencies between application services and XP RAID Manager device groups in one file for all application services in the cluster.
RHCS and SLE HA XP Cluster Extension integration with RHCS and SLE HA uses an XP Cluster Extension resource configuration file. The objects and format in the configuration file are the same as the UCF.cfg file. For more information, see Chapter 5 on page 93. VERITAS Cluster Server Integrating XP Cluster Extension with VERITAS Cluster Server does not require a user configuration file when the standard environment for XP Cluster Extension is used.
Objects are supported according to the requirements or capabilities of the cluster software, as shown in Table 4 on page 125.
System Object CLI HACMP MSCS RHCS, SLE HA VCS DC_B_Hosts • • • • • DeviceGroup • • • • • • FastFailbackEnabled FenceLevel • • Filesystems • • JournalDataCurrency • • • • • • • • • LocalDCLMForNonPAIRDG PostExecCheck • • • • • PostExecScript • • • • • PreExecScript • • • • • RaidManagerInstances • • • • • ResyncMonitor • • • • ResyncMonitorAutoRecover • • • • ResyncMonitorInterval • • • • • • • • ResyncWaitTimeout • • StatusRefr
COMMON Format tag Description Distinguishes between general (common) and application-specific objects. Format String Description (Optional) Defines the path to the XP Cluster Extension log file. Default value Linux/Unix /var/opt/hpclx/log LogDir Windows %ProgramFiles%\Hewlett-Packard\Cluster Extension XP\log LogLevel Format String Description (Optional) Defines the logging level used by XP Cluster Extension.
APPLICATION objects The APPLICATION section defines the failover and failback behavior of XP Cluster Extension for each application service. APPLICATION is a multitag that can appear in the configuration file for each application service using XP Cluster Extension. The APPLICATION object requires the name of the application service as its value. The objects specified after an APPLICATION tag must appear only once per application.
Default values Linux/UNIX online file: /etc/opt/hpclx force flag file: etc/opt/hpclx/conf Windows %HPCLX_PATH% Files resource_name.createsplitbrain resource_name.forceflag resource_name.online If specified in a user configuration file, resource_name is the value of the APPLICATION tag; otherwise, resource_name is the value of the XP Cluster Extension resource name. ApplicationStartup Format String Description (Optional) Specifies where a cluster group should be brought online.
Valid values FASTFAILBACK (default) The cluster group is brought online in the remote data center (if possible) without waiting for resynchronization. The application startup process is stopped locally and XP Cluster Extension reports a data center error. Depending on the cluster software, the application service cannot start on any system in the local data center, and the cluster software transfers the application service back to the remote data center.
CAUTION: Measure or calculate the full XP disk array cache copy time to use the gathered information for the AsyncTakeoverTimeout object. After a takeover command has been invoked, XP Continuous Access Software copies the side file area residing in the XP disk array cache to the site where the takeover command has been issued (the secondary disks). The side file area cannot exceed the installed cache size.
Valid values YES NO (default) BCEnabledB Format String Description (Optional) Enables rolling disaster protection for data center B. Valid values YES NO (default) BCMuListA Format List Description (Optional) Space-separated list defines the MU number of the XP Business Copy Software disk pairs in data center A. BCMuListB Format List Description (Optional) Space-separated list defines the MU number of the XP Business Copy Software disk pairs in data center B.
BCResyncMuListA Format List Description (Optional) Space-separated list defines the MU number of the XP Business Copy Software disk pairs in data center A. BCResyncMuListB Format List Description (Optional) Space-separated list defines the MU number of the XP Business Copy Software disk pairs in data center B. ClusterNotifyCheckTime Format Integer Description Specifies how often XP Cluster Extension will check for VM live migration state changes.
center failure. The DataLoseDataCenter object is used to allow/prohibit automatic application service startup in this particular case. The combination of setting the DataLoseMirror object to YES and the DataLoseDataCenter object to NO are contradictory. Valid values YES (default) NO DataLoseMirror Format String Description (Optional) Specifies whether a resource should be brought online while the disk pair is suspended or deleted.
DeviceGroup (Required) Format String Description XP RAID Manager device group, containing the application service disk set. Files Linux/UNIX /etc/horcmX.conf Windows: \winnt\horcmX.conf %system_root%\horcmX.conf where X is the XP RAID Manager instance number. FastFailbackEnabled (VCS only) Format String Description (Optional) Disables VCS service groups for the data center. This allows the immediate transferring of the service group back to the remote data center.
Description Space-separated list of file systems. JournalDataCurrency Format String Description (Optional) Specifies whether a resource should be brought online while there could still potentially be a large amount of data on P-VOL Journal that cannot be transmitted to the secondary site due to the XP Continuous Access Software link being down. Used only if the FenceLevel object value is ASYNC and the local device is an S-VOL.
Valid values YES NO (default) PostExecCheck Format String Description (Optional) The PostExecCheck object is used to configure XP Cluster Extension to gather XP disk pair status information after the takeover procedure. That information will be passed to the post-executable. In case of a remote data center failure, it could be time consuming to gather that information, especially if your post-executable does not need any XP status information.
ResyncMonitor Format String Description (Optional) Starts the pair/resync monitor to monitor the disk pair status and resynchronize disk pairs if the ResyncMonitorAutoRecover attribute is set to YES. Valid values YES NO (default) ResyncMonitorAutoRecover Format String Description (Optional) Automatically recovers disk pairs states if the disk pairs are monitored by the pair/resync monitor.
Vgs (CLI and HACMP only) Format List Description List of volume groups. XPSerialNumbers (Required) Format List Description A space-separated list of at least two serial numbers must be specified: the serial numbers of the primary and secondary XP disk arrays. XP Cluster Extension checks whether the local disk array is contained in this list. Serial numbers of the disk arrays of the connected cluster nodes (at least two). VCS This object is a string-vector element.
User configuration file and XP Cluster Extension objects
9 Advanced XP Cluster Extension configuration This chapter describes advanced XP Cluster Extension configuration procedures. Implementing rolling disaster protection To implement rolling disaster protection, create XP Business Copy Software disk pairs for the local XP Continuous Access Software disks. • Create the XP Business Copy Software disk pairs using the –m noread option of the paircreate command.
Extension suspends specified XP Business Copy Software disk pairs that are in PAIR state. For information on setting XP Cluster Extension objects, see Chapter 8 on page 123. When using rolling disaster protection, note the following: • If the BCEnabledA and BCEnabledB objects are set to YES, you must configure specific XP Business Copy Software disk pairs using MU numbers. The MU number defines one of the many disk pair relationships you can create with XP Business Copy Software disk pairs.
Figure 10 Disaster-tolerant configuration with rolling disaster protection . Monitoring and resynchronizing device groups The pair/resync monitor can either only monitor or both monitor and resynchronize the state of the XP RAID Manager device group for an application service.
CAUTION: If the application service stops, the cluster software or your customized solution must be able to stop the monitoring or resynchronization utility. Without this ability, the use of the pair/resync monitor is not supported. HP recommends that you disable application service failover during a disk pair recovery (resynchronization). When the pair/resync monitor is enabled, XP Cluster Extension takes immediate action to recover any reported suspended disk pair.
Executing programs before and after an XP Cluster Extension takeover XP Cluster Extension can invoke other programs, such as Perl scripts, before or after an XP Cluster Extension takeover. These programs can be any executable, and must be able to provide return codes to XP Cluster Extension. If the programs add significant execution time to the application service startup, the timeout values for the startup must be adjusted in the cluster software.
8. remote device group state (display) Pre-executable status before failover and post-executable status after failover IMPORTANT: An empty string is returned if parameter #7 is not SSWS, PSUE, or PDUB. 9. current fence level 10. disk array serial numbers (local) 11. reserved 12. reserved 13. disk array firmware version (local) 14. XP RAID Manager version (local) 15. application directory path (ApplicationDir object) 16. log file location (LogDir object) 17. DC_A_Hosts node names 18.
CAUTION: If the pre-execution program returns 1, 2, 3, or 5, a post-executable will not be executed. If a takeover function fails, the post-executable will not be executed. Post-executable return codes Post-executables must give a return code. These return codes determine whether the application is stopped. 0 POST_OK Post-executable OK; continue. 1 POST_ERROR_GLOBAL Post-executable failed; stop application service cluster-wide.
Advanced XP Cluster Extension configuration
10 Troubleshooting To troubleshoot problems with XP Cluster Extension, you must understand XP Continuous Access Software environments. Many issues can be attributed to incompatible disk pair states. See the XP Continuous Access Software and XP RAID Manager documentation before assuming that a problem has been caused by XP Cluster Extension. For more information on XP Continuous Access Software, see the HP StorageWorks XP Continuous Access Software user guide.
Start errors Start errors can occur when the path to the XP RAID Manager binaries has not been set in the PATH environment variable. If a user configuration file is not found in the correct directory location, XP Cluster Extension returns a local error. A start error occurs if the APPLICATION name tag value in the XP Cluster Extension resource configuration file does not match the service name (RHCS) or the App value of XP Cluster Extension resource (SLE HA).
The XP Cluster Extension log file is named clxhacmp.log. Start errors HACMP will go into a loop and wait until the problem is solved and until the file /etc/opt/hpclx/ application_name.LOCK has been removed. This process has been adopted from HACMP, which will also run in an endless loop if there is a failure and until you recover all errors and start the application manually. After all errors have been recovered, you can invoke the command clruncmd to return control back to the cluster software.
clxHACMP: ERROR - Pair state of device group "oracle" might be clxHACMP: ERROR - incorrect. Manual checking and correction within clxHACMP: ERROR - Continuous Access XP is required. clxHACMP: ERROR - Remove file "/etc/opt/hpclx/OracleRG.LOCK" in order clxHACMP: ERROR - to continue with HACMP specific recovery actions. ================================================================= The last message is repeated every 5 minutes.
MSCS-specific error handling XP Cluster Extension related messages are logged by MSCS to the following locations: • %ClusterLog%\cluster.log. • The XP Cluster Extension log file is named clxmscs.log. • The XP Cluster Extension configuration tool log resides in the %ProgramFiles%\HewlettPackard\Cluster Extension XP\log\ directory. Resource start errors MSCS configurations do not require a UCF.cfg file if the default COMMON objects are used (recommended).
Figure 11 Incompatible XP disk pair state . Using the Domain user account (Windows Server 2008/2008 R2 only) When using the Domain user account to manage the cluster, modifying HORCM files might not be possible, and XP Cluster Extension tools might not run as expected. If you experience any of these issues, turn off UAC. To turn off UAC, select Control panel > User Accounts, and click Turn User Account Control on or off. Clear the User Account Control (UAC) to help protect your computer check box.
This is a general VCS engine log file, which gives an overview of all cluster-related activities and whether they were successful or unsuccessful. • VCS 1.3.0 or later: /var/VRTSvcs/log/ClusterExtensionXP_A.log • VCS 1.1.2: /var/VRTSvcs/log/ClusterExtensionXP.log_A This XP Cluster Extension agent log file of VCS shows agent-related error information. The XP Cluster Extension log file is named clxvcs.log.
Figure 12 Incompatible XP disk pair state (VCS Cluster Manager Log Desk window) . Figure 13 on page 156 shows detailed information for the current XP disk pair state, which will be displayed in the VCS Log Desk only if the XP Cluster Extension LogLevel object is set to INFO. Figure 13 Detailed information of the XP disk pair state (VCS Log Desk) . Linux-specific error handling XP Cluster Extension messages are logged by RHCS and SLE HA to the following location: /var/ log/messages.
Failover errors XP Cluster Extension will fail to bring an RHCS service or SLE HA resource group online on the local system if a configuration error occurs. In this case, XP Cluster Extension returns a local error. The RHCS service or SLE HA resource group will go into a failed state after a startup attempt on any system in the same data center if the disk array status indicates that a problem experienced locally would not be solved on another system connected to the same disk array.
3. Restart the node that was shut down. NOTE: The time to detect a storage outage due to failure of all paths to storage depends on the setting for no_path_retry in the multipath software configuration. A value of fail does not queue I/O in the event of a failure in all paths and returns an immediate failure. For information about the recommended value for your environment, see the DM-Multipath documentation.
Pair/resync monitor messages in syslog/errorlog/messages/event log Using the pair/resync monitor will cause a message in the system log file of your operating system (for any non-PAIR state of the device group being monitored). Those messages might indicate the following: • The XP RAID Manager instance is not running or cannot be used to gather device group state information. • The device group is not in the PAIR state.
Example Set the XP Cluster Extension agent's MonitorInterval attribute to 60 seconds (the default value); then set the XP Cluster Extension resource ResyncMonitorInterval attribute to less than 60 seconds.
11 Support and other resources Contacting HP For worldwide technical support information, see the HP support website: http://www.hp.
In the Storage section, click Storage Software, and then select your product. White papers The following white papers are available at www.hp.
Convention Monospace, italic text Monospace, bold text Element • Code variables • Command variables Emphasized monospace text CAUTION: Indicates that failure to follow directions could result in damage to equipment or data. IMPORTANT: Provides clarifying information or specific instructions. NOTE: Provides additional information. TIP: Provides helpful hints and shortcuts.
Support and other resources
Glossary CHA Channel adapter. A device that provides the interface between the array and the external host system. Occasionally, this term is used synonymously with the term channel host interface processor (CHIP). CLI Command-line interface. An interface comprised of various commands which are used to control operating system responses.
LUSE The LUSE feature is available when the HP StorageWorks LUN Manager product is installed, and allows a LUN, normally associated with only a single LDEV, to be associated with 1 to 36 LDEVs. Essentially, LUSE makes it possible for applications to access a single large pool of storage. See also LD, LDEV LVM Logical Volume Manager. MINAP Minimum active paths. MMC Microsoft Management Console. MNS Majority node set. MSCS Microsoft Cluster Service. MU Mirror unit. NIC Network interface card.
split-brain syndrome A state of data corruption that can occur if a cluster is re-formed as subclusters of nodes at each site, and each subcluster assumes authority, starting the same set of applications and modifying the same data. SPOCK Single Point of Connectivity Knowledge website. SPOCK is the primary portal used to obtain detailed information about supported HP StorageWorks product configurations. SPOF Single point of failure. S-VOL Secondary or remote volume.
Glossary
Index A agent configuring, 75 disabling, 87 APPLICATION section description, 128 application service failover, 159 ApplicationDir object description, 128 ApplicationStartup object description, 129 AsyncTakeoverTimeout object description, 130 AutoFailbackType description, 48 automatic recovery, 149 AutoRecover object description, 131 rolling disaster protection, 142 B Basic Resource Health Check Interval description, 47 BCEnabledA object description, 131 BCEnabledB object description, 132 BCMuListA object d
disaster tolerance, 11 disk pairs XP Continuous Access, 11 document conventions, 162 related documentation, 161 documentation HP website, 161 providing feedback, 163 E enabling a service group, 155 error return codes failover, 150 exporting configuration information, 41 F FailoverPeriod description, 48 FailoverThreshold description, 48 fast failback XP Continuous Access Asynchronous Software, 12 FASTFAILBACK value description, 130 FastFailbackEnabled object description, 135 VCS setting, 89 features XP Clu
LooksAlivePollInterval description, 47 M majority node set Microsoft Cluster Service, 15 MergeCheckInterval UCF requirement, 64 Microsoft Cluster Service adding dependencies, 64 administration, 73, 74 changing resource names, 44, 45 configuration example, 66 configuration file, 123 configuring XP RAID Manager advanced properties, 58 configuring XP RAID Manager device group details, 58 configuring XP RAID Manager instances, 57 data center assignments, 59 group names, 42, 44 integration with XP Cluster Exten
resources adding for Microsoft Cluster Service, 42 adding for VCS, 80 adding for Windows Server 2008/2008 R2, 44 adding with the CLI, 44 bringing online, 70 changing attributes for VCS, 83 changing properties for Microsoft Cluster Service, 49 configuring for Microsoft Cluster Service, 45 configuring for VCS, 79 deleting for MSCS, 70 linking for VCS, 84 Microsoft Cluster Service, 45 properties (CLI), 63 properties (UCF), 64 taking offline, 70 Response to resource failure description, 47 RestartAction descrip
user configuration file APPLICATION section, 128 ApplicationDir object, 128 ApplicationStartup object, 129 AsyncTakeoverTimeout object, 130 AutoRecover object, 131 BCEnabledA object, 131 BCMuListA object, 132 BCMuListB object, 132 BCResyncEnabledA object, 132 BCResyncEnabledB object, 132 BCResyncMuListA object, 133 BCResyncMuListB object, 133 COMMON section, 126, 127 DataLoseDataCenter object, 133 DataLoseMirror object, 134 DC_A_Hosts object, 134 DC_B_Hosts object, 134 DeviceGroup object, 135 FASTFAILBACK v
