Designing Disaster Tolerant HA Clusters Using Metrocluster and Continentalclusters, 1st Edition, December 2006 (B7660-90019)
Manuals
Brands
HP Manuals
Software
HP Serviceguard Continentalclusters
411
412
413
414
415
416
417
418
419
420
Table Of Contents
1 Designing a Metropolitan Cluster
Designing a Disaster Tolerant Architecture for use with Metrocluster Products
Single Data Center
Two Data Centers and Third Location with Arbitrator(s)
Figure11 Two Data Centers and Third Location with Arbitrators
Arbitrator Node Configuration Rules
Disk Array Data Replication Configuration Rules
Calculating a Cluster Quorum
Example Failover Scenarios with One Arbitrator
Figure12 Failover Scenario with a Single Arbitrator
Example Failover Scenarios with Two Arbitrators
Figure13 Failover Scenario with Two Arbitrators
Worksheets
Disaster Tolerant Checklist
Figure14 Disaster Tolerant Checklist
disaster tolerant:Metrocluster worksheet;worksheet, Metrocluster
Figure15 Cluster Configuration Worksheet
disaster tolerant:package worksheet;package:worksheet;worksheet, package
Figure16 Package Configuration Worksheet
Figure17 Package Control Script Worksheet
Next Steps
2 Designing a Continental Cluster
Understanding Continental Cluster Concepts
Figure21 Sample ContinentalClusters Configuration
Mutual Recovery Configuration
Figure22 Sample Mutual Recovery Configuration
Application Recovery in a Continental Cluster
Figure23 Continental Cluster After Recovery
Monitoring over a Network
Cluster Events
Interpreting the Significance of Cluster Events
How Notifications Work
Alerts
Alarms
Creating Notifications for Failure Events
Creating Notifications for Events that Indicate a Return of Service
Performing Cluster Recovery
Notes on Packages in a Continental Cluster
Startup and Switching Characteristics
Network Attributes
How Serviceguard commands work in a Continentalclusters
Designing a Disaster Tolerant Architecture for use with ContinentalClusters
Mutual Recovery
Serviceguard Clusters
Data Replication
Physical Data Replication using Special Environment files
Multiple Recovery Pairs in a Continental Cluster
Figure24 Multiple Recovery Pair Configuration in a Continental Cluster
Highly Available Wide Area Networking
Data Center Processes
ContinentalClusters Worksheets
Data Center Worksheet
Recovery Group Worksheet
Cluster Event Worksheet
Preparing the Clusters
Setting up and Testing Data Replication
Configuring a Cluster without Recovery Packages
Configuring a Cluster with Recovery Packages
Building the Continentalclusters Configuration
Preparing Security Files
Creating the Monitor Package
Editing the ContinentalClusters Configuration File
Editing Section 1—Cluster Information
Editing Section 2—Recovery Groups
Figure27 Sample ContinentalClusters Recovery Groups
Figure28 Sample Bi-directional Recovery Groups
Editing Section 3—Monitoring Definitions
Selecting Notification Intervals
Checking and Applying the Continentalclusters Configuration
Figure29 ContinentalClusters Configuration Files
Starting the ContinentalClusters Monitor Package
Validating the Configuration
Documenting the Recovery Procedure
Figure210 Recovery Checklist
Reviewing the Recovery Procedure
Testing the Continental Cluster
Testing Individual Packages
Testing Continentalclusters Operations
Switching to the Recovery Packages in Case of Disaster
Receiving Notification
Verifying that Recovery is Needed
Using the Recovery Command to Switch All Packages
To Start the Failover Process
How the
Forcing a Package to Start
Restoring Disaster Tolerance
Restore Clusters to their Original Roles
Primary Packages Remaining on the Surviving Cluster
Primary Packages Remaining on the Surviving Cluster using
Newly Created Cluster Will Run Primary Packages
Newly Created Cluster Will Function as Recovery Cluster for All Recovery Groups
Maintaining a Continental Cluster
Adding a Node to a Cluster or Removing a Node from a Cluster
Adding a Package to the Continental Cluster
Removing a Package from the Continental Cluster
Changing Monitoring Definitions
Checking the Status of Clusters, Nodes, and Packages
Reviewing Messages and Log Files
Deleting a Continental Cluster Configuration
Renaming a Continental Cluster
Checking Java File Versions
Next Steps
Support for Oracle RAC Instances in a Continentalclusters Environment
Figure211 Oracle RAC Instances in a ContinentalClusters Environment
Figure212 Sample Oracle RAC Instances in a ContinentalClusters Environment After Failover
Oracle RAC;RAC;Continentalclusters
Figure213 ContinentalClusters Configuration Files in a Recovery Pair with RAC Support
Oracle Clusterware;clusterware
Initial Startup of Oracle RAC Instance in a Continentalclusters Environment
Failover of Oracle RAC Instances to the Recovery Site
Failback of Oracle RAC Instances After a Failover
3 Building Disaster Tolerant Serviceguard Solutions Using Metrocluster with Continuous Access XP
Files for Integrating XP Disk Arrays with Serviceguard Clusters
Overview of Continuous Access XP Concepts
PVOLs and SVOLs
Figure31 XP Series Primary and Secondary Volume Definitions
Device Groups and Fence Levels
Fence Level of
Fence Level of
Fence Level of
Continuous Access Link Timeout
Consistency Group
Limitations of Asynchronous Mode
Other Considerations on Asynchronous Mode
Continuous Access Journal Overview
Journal Volume
Pull-Based Replication
Mitigation of Network Problems
Fence Level
Journal Group
Journal Cache, Journal Volumes, and Inflow Control
Continuous Access Journal Pair State
Limitations of XP12000 Continuous Access Journal
One-to-One Volume Copy Operations
One-to-One Journal Group Operations
Journal Group Requirement
Configuring XP12000 Continuous Access Journal
Registering Journal Volumes
Data Replication Connections
Metrocluster package vs. Journal Group
Creating the Cluster
Preparing the Cluster for Data Replication
Creating the RAID Manager Configuration
Figure34 Disaster Tolerant Cluster
Pair Creation of Journal Groups
Creating Continuous Access Journal Pair
Sample Raid Manager Configuration File
Notes on the Raid Manager Configuration
Configuring Automatic Raid Manager Startup
Defining Storage Units
Creating and Exporting LVM Volume Groups using Continuous Access XP
Creating VxVM Disk Groups using Continuous Access XP
Validating VxVM Disk Groups using Metrocluster/Continuous Access Data Replication
Configuring Packages for Disaster Recovery
Completing and Running a Metrocluster Solution with Continuous Access XP
Maintaining a Cluster that uses Metrocluster with Continuous Access XP
Viewing the Progress of Copy Operations
Viewing Side File Size
Viewing the Continuous Access Journal Status
Viewing the Pair and Journal Group Information - Raid Manager using the “
Viewing the Journal Volumes Information - Raid Manager using the “
Figure35 Q-Marker and Q-CNT
Normal Maintenance
Resynchronizing
Using the
Failback
Timing Considerations
Data maintenance with the failure of a Metrocluster Continuous Access XP Failover
Swap Takeover Failure (Asynchronous/Journal mode)
Takeover Timeout (for Continuous Access Journal mode)
PVOL-PAIR with SVOL-PSUS(SSWS) State (for Continuous Access Journal Mode)
XP Continuos Access Device Group Monitor
XP/Continuous Access Device Group Monitor Operation Overview
Configuring the Monitor
Configure the Monitor’s Variables in the Package Environment File.
Configure XP/Continuous Access Device Group Monitor as a Service of the Package
Troubleshooting the XP/Continuous Access Device Group Monitor
Completing and Running a Continental Cluster Solution with Continuous Access XP
Setting up a Primary Package on the Primary Cluster
Setting up a Recovery Package on the Recovery Cluster
Setting up the Continental Cluster Configuration
Switching to the Recovery Cluster in Case of Disaster
Failback Scenarios
Scenario 1
Scenario 2
Failback in Scenarios 1 and 2
Failback when the Primary has SMPL Status
Maintaining the Continuous Access XP Data Replication Environment
Resynchronizing
Using the
Some Further Points
4 Building Disaster Tolerant Serviceguard Solutions Using Metrocluster with Continuous Access EVA
Files for Integrating the EVA with Serviceguard Clusters
Overview of EVA and Continuous Access EVA Concepts
Data Replication
Copy Sets
DR Groups
DR Group Properties
Log Disk
Managed Sets
Failover
Continuous Access EVA Management Software
Preparing a Serviceguard Cluster for Metrocluster Continuous Access EVA
Setting up the Storage Hardware
Figure41 Configuration of Virtual Disks and DR groups
Cluster Configuration
Management Server/SMI-S and DR Groups Configuration
Defining Management Server and SMI-S Information
Creating the Management Server List
Creating the Management Server Mapping File
Setting a Default Management Server
Displaying the List of Management Servers
Adding or Updating Management Server Information
Deleting a Management Server
Defining EVA Storage Cells and DR Groups
Creating the Storage Map File
Copying the Storage Map File
Displaying Information about Storage Devices
Verifying the EVA Configuration
Figure42 EVA Configuration Checklist
Configuring Volume Groups
Identifying Special Device File Name for Vdisk in DR Group using Secure Path V3.0D or V3.0E
Figure43 EVA Command View for the WWN Identifier
Identifying Special Device Files using Secure Path v3.0F
Identifying Special Device Files for PVLinks Configuration
Creating Volume Groups using Source Volumes for Secure Path v3.0D, v3.0E, and v3.0F
Configuring Volume Groups using PVLinks
Importing Volume Groups on Nodes at the Same Site
Importing Volume Groups on Nodes at the Remote Site
Figure44 EVA Command View DR Group Properties
Building a Metrocluster Solution with Continuous Access EVA
Configuring Packages for Automatic Disaster Recovery
Maintaining a Cluster that Uses Metrocluster Continuous Access EVA
Continuous Access EVA Link Suspend and Resume Modes
Normal Maintenance
Failback
Cluster Re-Configuration
Completing and Running a Continental Cluster Solution with Continuous Access EVA
Setting up a Primary Package on the Primary Cluster
Setting up a Recovery Package on the Recovery Cluster
Setting up the Continental Cluster Configuration
Switching to the Recovery Cluster in Case of Disaster
Failover to Recovery Site
Failover Scenarios
Scenario 1
Failback to the Primary Site
Scenario 2
Failback to the Primary Site
Scenario 3
Failback in Scenario 3
Reconfiguring Recovery Group Site Identities in Continentalclusters after a Recovery
5 Building Disaster Tolerant Serviceguard Solutions Using Metrocluster with EMC SRDF
Files for Integrating Serviceguard with EMC SRDF
Overview of EMC and SRDF Concepts
Figure51 EMC R1 and R2 Definitions
Preparing the Cluster for Data Replication
Installing the Necessary Software
Building the Symmetrix CLI Database
Determining Symmetrix Device Names on Each Node
Figure52 Sample syminq Output from a Node on the R1 Side
Figure53 Sample
Figure54 Parsing the Symmetrix Serial Number
Figure55 Sample
Figure56 Sample
Building a Metrocluster Solution with EMC SRDF
Setting up 1 by 1 Configurations
Creating Symmetrix Device Groups
Configuring Gatekeeper Devices
Verifying the EMC Symmetrix Configuration
Creating and Exporting Volume Groups
Importing Volume Groups on Other Nodes
Configuring PV Links
Grouping the Symmetrix Devices at Each Data Center
Figure58 2 X 2 Node and Data Center Configuration with Consistency Groups
Setting up M by N Configurations
Creating Symmetrix Device Groups
Configuring Gatekeeper Devices
Creating the Consistency Groups
Creating Volume Groups
Creating VxVM Disk Groups using Metrocluster with EMC SRDF
Validating VxVM Disk Groups using Metrocluster with EMC SRDF
Additional Examples of M by N Configurations
Figure510 2 by 1 Configuration
Figure511 Bidirectional 2 by 2 Configuration
Configuring Serviceguard Packages for Automatic Disaster Recovery
Maintaining a Cluster that uses Metrocluster with EMC SRDF
Managing Business Continuity Volumes
Protecting against Rolling Disasters
Using the BCV in Resynchronization
R1/R2 Swapping
R1/R2 Swapping using Metrocluster SRDF
R1/R2 Swapping using Manual Procedures
Some Further Points
Metrocluster with SRDF/Asynchronous Data Replication
Overview of SRDF/Asynchronous Concepts
Figure512 SRDF/Asynchronous Basic Functionality
Requirements for using SRDF/Asynchronous in a Metrocluster Environment
Hardware Requirements
Software Requirements
Preparing the Cluster for SRDF/Asynchronous Data Replication
Metrocluster SRDF Topology using SRDF/Asynchronous
Configuring Metrocluster with EMC SRDF using SRDF/Asynchronous
Building a Device Group for SRDF/Asynchronous
Package Configuration using SRDF/Synchronous or SRDF/Asynchronous
First-time installation of Metrocluster with EMC SRDF using SRDF/Synchronous
Pre-existing Installations of Metrocluster SRDF using SRDF/Synchronous
Migration of Existing Applications from SRDF/Synchronous to SRDF/Asynchronous
Package Failover using SRDF/Asynchronous
Protecting against a Rolling Disaster
Limitations and Restrictions
Building a Continental Cluster Solution with EMC SRDF
Setting up a Primary Package on the Primary Cluster
Setting up a Recovery Package on the Recovery Cluster
Setting up the Continental Cluster Configuration
Switching to the Recovery Cluster in Case of Disaster
Failback Scenarios
Scenario 1
Scenario 2
Maintaining the EMC SRDF Data Replication Environment
Normal Startup
Normal Maintenance
6 Designing a Disaster Tolerant Solution Using the Three Data Center Architecture
Overview of Three Data Center Concepts
Figure61 Three Data Center Solution Overview
Designing a Disaster Tolerant Architecture Using Three Data Center with Continuous Access XP
Figure62 Three Data Center Architecture
Overview of HP XP StorageWorks Three Data Center Architecture
XP 3DC Multi-Target Bi-Link Configuration
Figure63 XP Three Data Center Multi-Target Bi-Link Configuration Data Replication
Figure64 3DC Multi-Hop Bi-Link Configuration Data Replication
Three Data Center Multi-Hop Bi-Link Configuration
HP StorageWorks Mirror Unit Descriptors
Figure65 Mirror Unit Descriptors
Figure66 Mirror Unit Desciptor Usage
Configuring an XP Three Data Center Solution
Creating the Serviceguard Clusters
Creating the Continental Cluster
HP StorageWorks RAID Manager Configuration
Creating the RAID Manager Configuration
Multi-Target Raid Manager Configuration
Figure67 Multi-Target Bi-Link (1:2)
Multi-Hop Raid Manager Configuration
Figure68 Multi-Hop Bi-Link (1:1:1)
Alternative to HORCM_DEV
11. Restart the Raid Manager instance so that the new information in the configuration file is read.
12. Repeat steps 2 through 11 on each host runs this particular application package.
Creating Device Group Pairs
LVM Volume Groups Configuration
VxVM Configuration
Package Configuration in a Three Data Center Environment
Timing Considerations
Bandwidth for Continuous Access and Application Recovery Time
Data Maintenance with the Failure of a Metrocluster Continuous Access XP Failover
Swap Takeover Failure (for Continuous Access Sync Pair)
Takeover Timeout (for third data center)
Continuous Access-Journal Device Group PVOL-PAIR with SVOL-PSUS(SSWS) State
Failback Scenarios
Failback from Data Center 3 (DC3)
MULTI-HOP-BI-LINK (DC1 > DC2 > DC3) Data Recovery from DC3 to DC1
MULTI-TARGET-BI-LINK (DC2 > DC1 > DC3) Data Recovery from DC3 to DC1
Additional Reading
A
B Environment File Variables for Metrocluster Continuous Access EVA
C Environment File Variables for Metrocluster with EMC SRDF
D Configuration File Parameters for Continentalclusters
E Continentalclusters Command and Daemon Reference
Glossary
Designing a Disaster T
olerant Solution Us
ing the Three Data Center Architecture
Additional Reading
Chapter 6
412
1
...
...
410
411
412
413
414
...
...
470