Designing Disaster Tolerant High Availability Clusters, 10th Edition, March 2003 (B7660-90013)

Building a Continental Cluster
Testing the Continental Cluster
Chapter 5246
Testing ContinentalClusters Operations
Use the following procedures to exercise typical ContinentalClusters
behaviors.
1. Halt both clusters, then restart both clusters. The monitor packages
on both clusters should start automatically. The
ContinentalClusters packages (primary, data sender, data receiver,
and recovery) should not start automatically. Any other packages
may or may not start automatically, subject to their configuration.
NOTE If an UP status is configured for a cluster, then an appropriate
notification (email, SNMP, etc.) should be received at the configured
time interval from the node running the monitor package on the
other cluster. Due to delays in email or SNMP, the notifications may
arrive later than expected.
In addition to alerts/alarms sent using the mechanisms defined in
the ContinentalClusters configuration file, they are also recorded in
the file /var/adm/cmconcl/eventlog on the system reporting the
event.
2. While the monitor package is running on a monitoring cluster, halt
the monitored cluster (cmhaltcl -f). An appropriate notification
(email, SNMP, etc.) should be received at the configured time
interval from the node running the monitor package. Run
cmrecovercl. The command should fail. Additional notifications
should be received at the configured time intervals. After the alarm
notification is received, run cmrecovercl. Any data receiver
pakcages on the monitoring cluster should halt and the recovery
package(s) should start with package switching enabled. Halt the
recovery packages.
3. Test 2 should be rerun under a variety of conditions (and multiple
conditions) such as the following:
• Rebooting and powering off systems one at a time
• Rebooting and powering off all systems at the same time
Running the monitor package on each node in each cluster
Disconnecting the WAN connection between the clusters