HP P6000 Cluster Extension Software Administrator Guide Abstract This guide contains detailed instructions for configuring and troubleshooting HP P6000 Cluster Extension Software in Windows environment. The intended audience has independent knowledge of related software and of the HP P6000/EVA array and its software.
© Copyright 2012 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Contents 1 P6000 Cluster Extension features..................................................................6 Integration into cluster software...................................................................................................6 Graphical user interface.......................................................................................................6 CLI for easy integration.........................................................................................................
Using Failover Cluster Management to set resource properties (Windows Server 2008/2008 R2/2012)................................................................................................................26 Using the resource configuration tool to set resource properties (Server Core and Hyper-V Server)....................................................................................................................26 Configuring management servers........................................................
Pre-execution return codes...................................................................................................61 Post-execution return codes..................................................................................................61 5 Troubleshooting........................................................................................63 Changing the P6000/EVA SMI-S user name and password.........................................................63 Log facility........................
1 P6000 Cluster Extension features HP P6000 Cluster Extension offers protection against application downtime due to a fault, failure, or site disaster by extending a local cluster between data centers over metropolitan distance. P6000 Cluster Extension reinstates critical applications at a remote site within minutes of an adverse event, integrating your open-system clustering software and HP P6000 Continuous Access to automate failover and failback between sites.
Figure 1 Physical replication using HP P6000 Continuous Access P6000 Continuous Access-mirrored disks have a read/write-enabled source (local) disk and a read-only destination (remote) disk. Current cluster software products cannot distinguish between read-only and write-enabled disks, and cannot enable disk access if the disk is not write-enabled during the server boot process.
MSCS MSCS depends on a quorum resource to maintain a persistent log of cluster configuration changes and status, as well as a single point to resolve any possible events that could result in a split-brain situation. P6000 Cluster Extension supports the following quorum configurations in which one site failure will not cause the entire cluster to fail.
NOTE: For complete information on which write modes are supported on each version of controller software, see the HP Enterprise Virtual Array Compatibility Reference.
When setting up a consolidated DR site configuration (see Figure 3 (page 10)), consider the following: • P6000 Cluster Extension uses the unit of DR groups as the failover unit. This means all vdisks or members of a DR group will be failed over if the cluster software moves disk access from one cluster node to another. • P6000 Cluster Extension does not directly support managed sets.
HP P6000 Continuous Access Figure 4 (page 11) shows the supported P6000 Cluster Extension configuration. Figure 4 Supported P6000 Cluster Extension storage system configuration Synchronous replication Using synchronous mode, all write requests from the server are first transferred to the remote storage system. After each I/O has been mirrored in the cache area of the remote array, it is acknowledged to the local storage system. The write request is then acknowledged to the server.
HP Storage Management Initiative Specification To control P6000 Continuous Access-mirrored vdisks from a server, P6000 Cluster Extension uses the HP SMI-S interface. The HP SMI-S client implemented in the P6000 Cluster Extension cluster integration, or in the CLI, communicates with the HP SMI-S server running on the P6000 Command View management server. NOTE: In this context, the term management server refers to the server running HP P6000 Command View.
User configuration file P6000 Cluster Extension provides a user configuration file to customize failover behavior. You can specify all customizable objects of P6000 Cluster Extension in this file. See User configuration file (page 42) for more information. Force flag The force flag forces P6000 Cluster Extension to skip the internal logic and enables write access to the local vdisks regardless of the DR group member state.
links are down. When the intersite links are down, P6000 Cluster Extension will immediately try to use the remote management servers to interface with the remote array. Configuration tool (clxevaconfig.exe) P6000 Cluster Extension uses the configuration tool (clxevaconfig.exe) to store configuration and access information for each configured management server. This tool stores information in the clxevacfg file.
HP P6000 Command View uses HP P6000 Command View is used to create vdisks and configure the HP P6000/EVA storage system. Using HP P6000 Command View, you can: • Create P6000 Continuous Access DR groups. • Use Replication Solutions Manager or P6000 Command View to view a graphical representation of a DR group setup. • Use Replication Solutions Manager or P6000 Command View to manually switch source and destination roles of DR group members.
2 Configuring P6000 Cluster Extension After installing P6000 Cluster Extension, use the configuration tool to define the P6000 Cluster Extension setup configuration. Use Failover Cluster Manager or cluster commands in the CLI to add and configure resources. NOTE: For a list of supported operating system versions, see the HP SPOCK website: http:// www.hp.com/storage/spock. Integrating P6000 Cluster Extension with MSCS To integrate P6000 Cluster Extension with MSCS: 1.
Configuring P6000 Cluster Extension 17
Defining P6000 Cluster Extension configuration information using the GUI To define configuration information using the GUI: 1. Open the configuration tool. For instructions, see “Starting the P6000 Cluster Extension configuration tool” (page 16). 2. Click Add in the P6000 Management Server Configuration section to open the Add P6000 Management Server (SMA) dialog box. 3. Enter the following values in the appropriate fields, and then click OK: 4. • The name and IP address of the management server.
7. Select the servers that are possible owners of the P6000 Cluster Extension-managed disks, and then click OK. NOTE: See the Microsoft Failover Cluster Management documentation for more information about possible owners. 8. Click OK to close the configuration tool and save the information to all of the configured cluster nodes. Your configuration information is now saved in the clxevacfg file in the %HPCLX_EVA_PATH%\bin directory.
4. Click OK to save and close the configuration tool. Exporting configuration settings using the CLI 1. 2. Open a command window. Enter CLXEVACONFIG EXPORT /FILE=filepath, where filepath specifies the save location and file name. Importing configuration settings using the GUI 1. Open the configuration tool. For instructions, see “Starting the P6000 Cluster Extension configuration tool” (page 16). 2. 3. 4. Click Import. When prompted, choose the configuration file, and then click Open.
cluster resource resource_name /create /group:service_or_application_name /type:"Cluster Extension EVA" Example This command adds a P6000 Cluster Extension resource called clx_fileshare to the CLX_SHARE service or application.
Before configuring P6000 Cluster Extension resources, review the P6000 Cluster Extension objects described in User configuration file and EVA Cluster Extension objects (page 42).
When configuring P6000 Cluster Extension resources, note the following: • If the Failover Cluster Management GUI is used to configure a P6000 Cluster Extension resource, configuring the resource using a user configuration file (UCF file) is not required. • P6000 Cluster Extension creates a file called resource_name.online to indicate that the P6000/EVA resource is online on the local node. This file is created in the directory specified by the ApplicationDir resource property.
Table 1 Setting resource properties and values in the GUI Property Format Description Value Thorough Resource Health Check Interval (Windows Server 2008/2008 R2/ 2012) Integer Used to poll “Alive” state for the resource. Decreasing this value allows faster resource failure detection but also consumes more system resources. Windows Server 2008/2008 R2/2012 GUI: 01:00 mm:ss (Default) Used to poll “Alive” state for the resource.
Table 2 Service or application properties and values Property Format GUI: Failback (Prevent Integer failback or Allow failback) CLI: AutoFailbackType GUI: Period String CLI: FailoverPeriod GUI: Maximum failures in Integer the specified period (Windows Server 2008/2008 R2/2012) CLI: FailoverThreshold Description Value Prevents automatic fail back of a service or application to its primary system. Transfer the service or application back manually after the failure has been recovered.
Using Failover Cluster Management to set resource properties (Windows Server 2008/2008 R2/2012) For Windows Server 2008/2008 R2, use the Failover Cluster Management GUI to set resource properties. 1. Open Failover Cluster Management. 2. Double-click the P6000 Cluster Extension resource in the summary pane to open the Properties window. 3. Click the Parameters tab. 4. Make the necessary parameter changes, and then click OK.
3. Make the necessary parameter changes, and then click OK.
Configuring management servers To configure management servers from the cluster software Parameters tab or the resource configuration tool: 1. To add a management server (CV Server): a. Click Add to open the Add SMA Data center Assignment dialog box. b. c. d. 2. 3. 4. Select a management server from the Server Name list. Assign the management server to a data center selected from the Data center list. Click OK to save and close the dialog box.
Configuring cluster node data center assignments To configure cluster node data center assignments from the cluster software Parameters tab or the resource configuration tool: 1. To add a cluster node: a. Click Add to open the Add Server Data center Assignment window. b. c. d. 2. 3. 4. Select a cluster node from the Server Name list. Assign the cluster node to a data center selected from the Data center list. Click OK to save and close the window.
NOTE: When using P6000 Command View 8.0.2 or later, if you created new DR groups, the groups might not show up immediately in the Select DR Group dialog box. To make a new DR group available in the P6000 Cluster Extension GUI, do one of the following: 3. • Wait 30 minutes after creating the DR group. • Restart the HP CIM Object Manager service after creating the DR group. This service must be restarted on all Command View servers configured for the P6000 Cluster Extension resource.
Use this dialog box to configure the DR group behavior. 2. • The values you can configure depend on whether the DR group uses synchronous or enhanced asynchronous mode. • The settings in the Replication link up scenario section affect the failover behavior when the P6000 Continuous Access link is up. • The settings in the Replication link down scenario section affect the failover behavior when the P6000 Continuous Access link is down.
2. Update the PreExecScript, PostExecScript, and PostExecCheck values as needed, and then click OK. For more information, see APPLICATION objects (page 45). 3. Click Apply to finish your modifications and leave the Properties window open, or OK to save your changes and close the window. Setting P6000 Cluster Extension resource properties using the MMC If you are using Server Core or Hyper-V Server, you can manage a cluster remotely by using the MMC to run Failover Cluster Management.
cluster resource clx_fileshare /privprop The following example changes the CSObjectPathA property for the resource clx_fileshare: cluster resource clx_fileshare /privprop CSObjectPathA= "HPEVA_ConsistencySet.InstanceID=""50001FE15002B5A0.6005-08b40001-2b69-0000-6000-86a0-0000""" To retrieve the values for CSObjectPathA and CSObjectPathB, you must create and configure a UCF.cfg file for this DR group and run the clxconf -smis command. The values for both objects are stored in the clxobjpaths.cfg file.
Adding dependencies using Failover Cluster Management (Windows Server 2008/2008 R2/2012) You can add dependencies with the GUI on a local node or by using the MMC to run the Failover Cluster Management application. 1. Open Failover Cluster Management. 2. Select a service or application that has a P6000 Cluster Extension resource. 3. Double-click a disk in the summary pane. 4. Click the Dependencies tab, and then click Insert. 5. Select the P6000 Cluster Extension resource in the Resource menu. 6.
Disaster-tolerant configuration example using a file share The following example assumes that your environment includes: • Four nodes (host1_DCA, host2_DCA, host3_DCB and host4_DCB) • Two P6000/EVA storage systems with WWNs 50001fe15002B5A0 and 50001fe15002B570 • A DR group configured as \Data Replication\DRG_LUN25 • Two management servers SMAM019FK351K and SMAM019FK001A • An MNS arbitrator node in a third site or separate protected area • A pre-executable clxpre.
Figure 5 (page 36) illustrates failover options and shows a second cluster group CLX_IIS. The resource CLX_FILESHARE is part of the service group CLX_SHARE and must be brought online before the physical disk resources Disk_P6000_LUN25. Figure 5 Service or application example Figure 6 Resource tree for service or application CLX_SHARE P6000 Cluster Extension is configured as a single resource to enable read/write access to the physical disk resource used for the CLX_SHARE cluster group.
2. 3. 4. 5. 6. 7. 8. Create the file share service or application with all previously mentioned resources and its dependencies, except the P6000 Cluster Extension resource on host3_DCB. Create a new resource of type P6000 Cluster Extension and add systems host2_DCA, host3_DCB and host4_DCB to its possible owners. Change the restart behavior of the resource so that the resource can be restarted and that the restart affects the service or application. Set the number of restarts to 0.
Taking a resource offline Resources are usually taken offline automatically when the service or application is taken offline. Taking a resource offline causes resources that depend on that resource to go offline. When taking resources offline: • For Windows Server 2008/2008 R2/2012, use the GUI, MMC, or CLI. • For Server Core or Hyper-V Server, use the CLI or the MMC. For more information on using this command, see your Microsoft documentation.
software failover behavior. However, this behavior can cause a failure of P6000 Cluster Extension resources in some cases. • When P6000 Cluster Extension uses HP SMI-S to communicate with the HP P6000/EVA storage system, depending on the settings of the SMI-S request timeout property and the number of retries, the online operation could time out.
customized P6000 Cluster Extension Parameters tab. For more information about the Properties tab, see “Setting P6000 Cluster Extension resource properties using the MMC” (page 32). • When you install P6000 Cluster Extension into a Windows Server 2008/2008 R2/2012 environment, the resource extension DLL is registered by default, which prevents you from configuring a P6000 Cluster Extension resource from a remote management station.
For example: [10/12/09 20:13:02][2136][CLX_LM: CLXVMDISK04-App01][INFO] CLX detected that Live Migration for VM "Virtual Machine VM04" has begun. [10/12/09 20:13:02][2136][CLX_LM: CLXVMDISK04-App01][INFO] CLX started gathering VM "Virtual Machine VM04" specific storage information.
3 User configuration file and P6000 Cluster Extension objects Objects (also called properties in this document) define the disk array environment and failover/failback behavior. You can customize P6000 Cluster Extension objects in the user configuration file or in the cluster software. User configuration file For the CLI implementation of P6000 Cluster Extension, the user configuration file UCF.cfg is used to configure application service-specific information. UCF.
as well. For example, if you change the FailsafeMode object to enable, you might need to change the FailsafeModeOverride object. Objects are supported according to the requirements or capabilities of the cluster software, as indicated in Table 3 (page 43).
COMMON objects The COMMON section is used to set the environment of P6000 Cluster Extension. The COMMON tag can appear in the configuration file only once. The COMMON object does not require any value. Objects of the type COMMON can appear only one time. Those objects must be placed after the COMMON tag in the configuration file. If the default values fit your environment, there is no need to specify them in the file.
Setting the SmisRequestTimeout to a value less than the default will increase the chance that the P6000 Cluster Extension resource DLL will time out and not complete the online of the P6000 Cluster Extension resource.
If specified in a user configuration file, resource_name is the value of the APPLICATION tag; otherwise, resource_name is the value of the P6000 Cluster Extension resource name. ApplicationStartup (Optional) Format String Description Specifies where a cluster group should be brought online.
FASTFAILBACK is not available when the UseNonCurrentDataOkLinkUp property is set to YES. RESYNCWAIT The online local cluster group must wait until the DR group status is no longer merging or copying. P6000 Continuous Access initiates a resynchronization of the local vdisk based on the remote disk. P6000 Cluster Extension monitors the copy process. If no copy progress is made after a monitoring interval expires, the copy process is considered failed and P6000 Cluster Extension returns a global error.
Description Specifies the HPEVA_ConsistencySet object path for the DR group in data center B. This object path is used in sending management commands directed at the P6000 Continuous Access DR group. MSCS-specific behavior This is a private property for the P6000 Cluster Extension resource type within MSCS. P6000 Cluster Extension automatically runs the Connection Test and updates the consistency set object paths as necessary when the OK or Apply button is selected on the Parameters tab.
connect to the management hosts in the local data center first, before using management hosts in the remote data center. The hosts listed for data center B are associated with data center B and do not necessarily need to be physically located in data center B. You can specify the hosts using either the system host name or the IP address. DC_A_StorageCellWWN (Required) Format String Description The WWN of the P6000/EVA storage system located in data center A.
MergeCheckInterval Format Integer Description Specifies the check interval (in seconds) for checking the status of the DR group synchronization (merging). It might take a while to synchronize member vdisks. You can set this value if the ApplicationStartup property is set to RESYNCWAIT. Default value 20 PostExecCheck (Optional) Format String Description The PostExecCheck object is used to configure P6000 Cluster Extension to gather P6000/EVA disk pair status information after the takeover procedure.
Default value 300 seconds UseNonCurrentDataOk (Optional) Format String Description When the replication link is down, this property specifies whether P6000 Cluster Extension can fail over to the destination site when the data at the destination site might not be current. Use this property when FailsafeMode is disabled for the DR group, or FailsafeMode is enabled and FailsafeModeOverride is set to YES.
#DC_A_ManagementHosts smam019fk351k # active sma #DC_B_ManagementHosts clxeva1 # passive sma #CSObjectPathA HPEVA_ConsistencySet.InstanceID="50001FE15002B5A0.600508b4-0001-2b69-0000-7000-245b-0000" #CSObjectPathB HPEVA_ConsistencySet.InstanceID="50001FE15002B570.
4 CLI commands and utilities Using its CLI and command-line utilities, P6000 Cluster Extension can be integrated with almost any cluster software for the Windows operating systems.
Related information • clxevarun expects only the app_name parameter, which uniquely identifies an application service in the APPLICATION section of the user configuration file. • To force a DR group failover, use the –forceflag option. When using clxevarun, the –forceflag option is used instead of app_name.forceflag, which is only used with the non-CLI version of P6000 Cluster Extension. CAUTION: The forceflag option is an emergency switch used to manually activate your vdisk.
Creating and configuring the user configuration file The CLI expects as an argument the name configured as the APPLICATION tag value. The following is an example of a customized user configuration file when using clxevarun: ---- Sample UCF.
link(s) is re-established. The amount of delta data to be transferred between the source and destination vdisk could be large enough for the copy process to take some time. • To prevent the failover timeout trigger from terminating a failover command before it is finished, the time to copy an average amount of daily processed (write) data should be measured and the resource online timeout interval should be adjusted accordingly.
DeviceGroup \Data Replication\DRG031 #DR group DC_A_StorageCellWWN 50001FE15002B5A0 DC_B_StorageCellWWN 50001FE15002B570 DC_A_ManagementHosts smam019fk351k # active sma DC_B_ManagementHosts clxeva1 # passive sma FailsafeMode disable DC_A_Hosts w2k7 w2k6 DC_B_Hosts w2k5 w2k8 ---- Sample UCF.cfg end clxconf will get the consistency set object paths and place them in clxobjpaths.cfg. For example: ---- Sample clxobjpaths.cfg begin APPLICATION CLX030 CSObjectPathA HPEVA_ConsistencySet.
HPEVA_ConsistencySet.InstanceID="50001FE15002B570. 6005-08b40001-6e12-0000-7000-031a-0000" ---- Sample UCF.cfg end Return codes clxconf exits with one of the following return codes: 0 OK: clxconf successfully obtained both consistency set object paths for all of the applications. 3 FAILURE: clxconf failed to obtain both consistency set object paths for one or more of the applications.
Return codes 0 Successful, returns CSObjectPath_A and CSObjectPath_B values. 1 Found CSObjectPath for P6000/EVA array in data center A only. 2 Found CSObjectPath for P6000/EVA array in data center B only. 3 Connected to SMI-S server but cannot find matching CSObjectPaths for DR group. -1 Input error, prints usage. -2 Cannot find user name/password in bin\clxsmiscfg file. -3 Cannot connect to SMI-S server. -4 The size of return package from SMI-S is less than or equal to 0.
Return codes 0 OK CAUTION: The changes made using the P6000 Cluster Extension password management utility (clxpmu) are only saved on the local node on which it is run. To make the changes global, you must manually run clxpmu on each node or copy the file clxsmiscfg to each node in the cluster after you save the changes in clxpmu. HP recommends that you create a backup copy of the clxsmiscfg file before running clxpmu.
specified (there is an empty value for the object), no preprocessing or postprocessing is performed. If the object is not defined, then the argument value is empty. The 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16.
Pre-execution return code Text Description data-center-wide disabling of resources. This is local error behavior. 2 POST_ERROR_DC Post-executable failed; stop application service in this data center. MSCS does not allow cluster-wide or data-center-wide disabling of resources. This is local error behavior. 3 POST_ERROR_LOCAL Post-executable failed; stop application service on this system. 4 POST_ERROR_CONTINUE Post-executable failed; continue without error.
5 Troubleshooting To troubleshoot problems with P6000 Cluster Extension, you must understand HP P6000 Continuous Access environments. See the HP P6000 Continuous Access and HP P6000 SMI-S documentation before assuming that a problem has been caused by P6000 Cluster Extension. Changing the P6000/EVA SMI-S user name and password If the HP P6000/EVA SMI-S user name or password is changed through the P6000 Command View GUI or any other method, you must change the information in P6000 Cluster Extension.
Log files Log files assist you in locating and identifying problems. Timestamps, component IDs, and message IDs are included. • P6000 Cluster Extension-related messages are logged by MSCS to the file %ClusterLog%\ cluster.log. • For each system, P6000 Cluster Extension keeps its own log file in its default log location. • System Log file Log location Windows clxevamscs.log %HPCLX_EVA_PATH%\log CLI clxevarun.log P6000 Cluster Extension messages can also be found in MSCS log file.
P6000 Cluster Extension resources (MSCS) return a data center error and fail the resource if the DR group status indicates that the problem experienced locally would not be solved on another system connected to the same P6000/EVA storage system. This means that all systems specified in the DC_A_Hosts resource property or the DC_B_Hosts resource property would fail to bring the application online.
Physical disks cannot be added to cluster because physical disk number or drive letter not shown in menu when creating physical disk resources If you cannot see the drive letter or disk when trying to add a new resource: • Restart the Failover Cluster Manager GUI cluadmin.msc. • Ensure that the cluster group is online on the current host in which you are configuring the disk.
6 Support and other resources Contacting HP For worldwide technical support information, see the HP support website: http://www.hp.
White papers The following white papers are available at www.hp.
IMPORTANT: NOTE: TIP: Provides clarifying information or specific instructions. Provides additional information. Provides helpful hints and shortcuts. HP product documentation survey Are you the person who installs, maintains, or uses this HP storage product? If so, we would like to know more about your experience using the product documentation. If not, please pass this notice to the person who is responsible for these activities.
Glossary array A synonym of storage array, storage system, and virtual array. A group of disks in one or more disk enclosures combined with controller software that presents disk storage capacity as one or more virtual disks. asynchronous A term used to describe computing models that eliminate timing dependencies between sequential processes. In asynchronous replication, the array controller acknowledges that data has been written at the source before the data is copied at the destination.
Disk failure protection occurs when the storage system sets aside reserved capacity to take over the functionality of a failed or failing physical disk drive. In groups with mixed capacity drives, the reserved capacity is based on the largest disk in the disk group. The system must cover a failure in any drive, so it reserves enough capacity to cover the largest failure that could happen. disk group A named group of disks selected from all the available disks in a disk array.
FC connection A FC path between two storage systems or between a host and its storage. A connection is made up of multiple FC links. fencing The mechanism that prevents data corruption by removing an errant node from a cluster. Fibre Channel A data transfer architecture designed for mass storage devices and other peripheral devices that require high bandwidth. full copy The process of copying all data written on a source virtual disk directly to the destination virtual disk.
NIC Network interface card. A device that handles communication between a device and other devices on a network. normalization The initial full copy that occurs between a source vdisk to the destination vdisk. OCF Open Cluster Framework. P6000 Command View HP P6000 Command View consists of: • The GUI that displays the usage of the storage system. • The software behind the GUI, which controls and monitors the functions.
storage system A pair of HSV controllers and the array of physical disks they control. A storage system may contain vdisks that are sources as well as vdisks that are destinations. Sometimes used in context as follows: • Source storage system – Used in the context of a particular DR group, this is the storage system at which the source vdisk resides. • Destination storage system – Used in the context of a particular DR group, this is the storage system at which the destination vdisk resides.
Index A APPLICATION description, 45 ApplicationDir description, 45 ApplicationStartup description, 46 asynchronous replication mode, 11 AutoFailbackType description, 25 B Basic Resource Health Check Interval description, 24 C CLI configuration, 6 ClusterNotifyCheckTime description, 47 UCF requirement, 33 ClusterNotifyWaitTime description, 47 UCF requirement, 33 clxconf command, 56 clxevarun command, 53 clxgetobjpath command, 58 clxpmu command, 59 clxresconf command, 59 command line interface configuring,
GUI, 6 integration with cluster software, 6 P6000 Cluster Extension, 6 redirection of mirrored disks, 7 files configuration Windows , 14 force flag, 13 user configuration, 13 Force flag, 13 selecting DR groups, 29 takeover actions, 31 Microsoft Management Console, 23 mirrored disks automated redirection, 7 G objects APPLICATION, 45 COMMON section, 44 P6000 Cluster Extension, 42 GUI configuration, 6 H help obtaining, 67 HP technical support, 67 HP P6000/EVA SMI-S dependencies, 13 Hyper-V Live Migration,
resources, 25 adding for Windows Server 2008/2008 R2, 20 adding with the CLI, 20 bringing online, 37 changing properties for Microsoft Cluster Service, 25 changing properties for MSCS, 25 configuring for MSCS, 21 deleting for MSCS, 38 more than one, 21 MSCS, 21 properties (CLI), 32 properties (Hyper-V Server), 25 properties (Server Core), 25 properties (UCF), 33 properties (Windows Server 2008/2008 R2), 25 taking offline, 38 Response to resource failure description, 24 RestartAction description, 24 RestartP