Understanding and Designing Serviceguard Disaster Recovery Architectures
Special Requirements and Recommendations for using VxVM, CVM and
CFS in Extended Clusters
• The Dirty Region Logging (DRL) feature is highly recommended to be used for resynchronization
after a node failure. RAID-5 mirrors are not supported, because it cannot be verified that both
data centers have a complete copy of all the data. A FastResync map (also known as
FastMirrorResync—FMR) is highly recommended to keep track of changes to a volume when
mirror devices become detached (such as after a data center failure). FMR reduces the amount
of data required to be resynchronized after detached mirror devices are reattached. Without
FMR, the volume manager can only perform a full resynchronization (For example, it cannot
perform an incremental synchronization) when recovering from the failure of a mirror copy
or loss of connectivity to a data center. This can have a large impact on performance and
availability of the cluster if the disk groups are large. Both DRL and FMR are implemented as
bitmaps and are stored together in a version 20 DCO volume that is attached to the “data”
volume. Beginning with version 5.0, VxVM and CVM include a new feature, “site-awareness”,
which simplifies configuring the disk volumes and mirroring in an Extended Cluster, because
it allows the storage in each data center to be tagged with a site ID. A license to enable this
feature is included with specific SG SMS A.02.00, A.02.01, A.02.01.01, and A.03.00
product bundles.
• If you are using CVM or CFS, the CVM Mirror Detachment Policy must be set to “Global.”
• VxVM, CVM, and CFS support up to 32 mirror copies, however increasing the number of
mirror copies adversely affects performance on disk writes.
• In an Extended Cluster, the VxVM/CVM hot relocation feature can be counter productive. If
the inter-site Fiber Channel links fail, or a complete site fails, this feature automatically relocates
plexes to the same storage system in the surviving datacenter. This would result in a
configuration in which both plexes of a volume are stored on subdisks of the same datacenter,
compromising the ability to survive a data center failure at a later time. Therefore disable
VxVM’s hot relocation feature. A convenient way to disable hot relocation for all VxVM/CVM
volumes is to prevent the vxrelocd daemon from starting. This can be achieved by
commenting out the entry (nohup vxrelocd root &) that invokes the vxrelocd daemon
in the startup file (/sbin/rc2.d/S95vxvm-recover) on each node. You must verify these
configuration changes after installing VxVM patches or upgrading to a new version of VxVM
to ensure that hot relocation remains disabled. Note that if you preventvxrelocd from starting,
it will disable the hot relocation feature for all VxVM/CVM volumes on the system. The VxVM
Administration Guide provides additional information on how to use the hot relocation feature
in a more granular way.
• Different CVM/CFS revisions have different limitations and requirements:
CVM 3.5 mirroring is supported for EC RAC clusters for distances up to 10 kilometers
for 2, 4, 6, or 8 node clusters, and up to 100 kilometers for 2 node clusters. CVM 3.5
◦
allows only one heartbeat network to be defined for the cluster, therefore you must make
the heartbeat network highly available, using a standby LAN to provide redundancy for
the heartbeat network. The heartbeat subnets must be dedicated networks, to ensure that
other network traffic does not saturate the heartbeat networks.
◦ CVM 4.1/CFS 4.1 mirroring is supported for distances up to 100 kilometers for 2, 4, 6,
or 8 node clusters. CVM/CFS 4.1 are available only with SG SMS A.01.00 or A.01.01.
Standalone CVM 4.1 (without SG SMS) is also supported.
◦ CVM 5.0/CFS 5.0 and CVM 5.0.1/CFS 5.0.1 mirroring is supported for distances up
to 100 kilometers for 2, 4, 6, or 8 node clusters on HP-UX 11i v2 or 11i v3. Beginning
with version 5.0, VxVM and CVM include a new feature, “site-awareness”, which simplifies
configuring the disk volumes and mirroring in an Extended Cluster, because it allows the
storage in each data center to be tagged with a site ID. A license to enable this feature
is included with specific SG SMS A.02.00 product bundles. CVM/CFS 5.0 are available
Special Requirements and Recommendations for using VxVM, CVM and CFS in Extended Clusters 57