Implementing UPS Configurations with Microsoft Cluster Server Describes how to configure an uninterruptible power supply in a cluster Minimize the risks introduced by UPS failure Includes CMD files for streamlined UPS control Redpaper Hendrik Ernst Martin Zustak Peter Fuchs Silvio Erdenberger Arwed Tschoeke ibm.
International Technical Support Organization Implementing UPS Configurations with Microsoft Cluster Server March 2001
Take Note! Before using this information and the product it supports, be sure to read the general information in Appendix E, “Special notices” on page 87. First Edition (March 2001) This edition applies to Microsoft Windows NT 4.0 Enterprise Edition with Service Pack 5 or 6a and APC PowerChute PLUS 5.2 for Windows NT. Comments may be addressed to: IBM Corporation, International Technical Support Organization Dept. HZ8 Building 662 P.O.
Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .v The team that wrote this redpaper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Comments welcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi Chapter 1. The problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Chapter 2.
4.4 StartUp Parameter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Appendix A. Downloading the additional material. . . . . . . . . . . . . . . . . . . . . 73 A.1 Using the additional material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 A.2 Readme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 A.2.1 Windows NT 4.0. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Preface This redpaper is the product of a collaboration of specialists from American Power Conversion, Inc. (APC), Computer Service GmbH (CSG), and IBM. The intention was to find a solution for implementing uninterruptible power supplies (UPS) in a Microsoft Cluster Server environment. To our knowledge, this is the first document that covers this topic. The intention in writing this redpaper was to develop a solution for using APC UPS units in a two-node Microsoft Cluster Server environment.
Comments welcome Your comments are important to us! We want our redpapers to be as helpful as possible. Please send us your comments about this redpaper or other Redbooks in one of the following ways: • Use the online evaluation form found at ibm.com/redbooks • Send your comments in an Internet note to redbook@us.ibm.
Chapter 1. The problem How can servers be protected against power failures? Usually redundant power supplies and redundant power cords connected to different power lines are used for basic protection. The failure of one of these components does not affect the operation of the server. To protect the system against a complete power loss, an uninterruptible power supply (UPS) is required.
• UPS monitoring: PowerChute PLUS can monitor only one UPS at a time, independent of the type of communication link to this UPS (serial line or network). If you attach more than one UPS to a server, then the server cannot receive signals from all UPS units. Thus, you cannot attach more than one UPS to a server.
Chapter 2. APC hardware and software background In the following chapters, we describe some UPS units that are important for cluster solutions. All described UPS units are by American Power Conversion (APC). You can also get the Smart UPS units as an IBM option. For a correct sizing of the UPS capacity you need some background information on power.
In a switching power environment (power supplies in the server), a cos( ϕ )=0.707 is assumed. So we can calculate: S P = ------------------ = S ⋅ 1, 414 cos ( ϕ ) For better calculating we assume: P = S ⋅ 1, 5 This was a short introduction to power, but now we will describe the UPS units in detail. 2.1 Smart-UPS family The Smart-UPS family is a line-interactive UPS designed to provide clean, reliable AC power.
Table 1. Technical specifications VA/W ratings Maximum 2200 VA or 1600 W Type of external battery packs SU48RMXLBP (8 x 12 V DC and 17 Ah. Nominal system voltage 48 V DC) Dimensions: UPS Height 22.2 cm (5U) x width 48.3 cm x depth 45.1 cm. (Depth is measured from front of bezel to back of chassis, with a half-inch allowance for screws and outlets.) Dimensions: battery pack Height 17.8 cm x width 48.3 cm x depth 45.7cm Maximum input current 12 Amps (for nominal line voltages indicated and load p.f.
The UPS was designed to mount to any standard EIA RS-310 (ANSI C83.9) 19'' equipment rack. More information available at: http://www.apcc.com/products/techspecs/index.cfm?base_sku=SU2200RMXLINET 2.1.2 SU3000RMINET (5U unit) SU3000RMINET is a rack-mounted UPS with expandable run time via one battery pack. Table 3. Technical specifications VA/W ratings Maximum 3000 VA or 2250 W Optional battery pack SU48BP (4 x 12 V DC and 17 Ah. Nominal system voltage 48 V DC). Only one pack can be connected.
Load Internal battery With external battery pack 2200VA 8 22 2500VA 7 18 3000VA 5 13 The UPS is furnished with one input line cord terminated with a CEE7/7 plug and three 1.8 m long output cords appropriate for connection to equipment with 10 Amp IEC320 appliance receptacles.
Figure 3. APC Smart UPS 3000 — rear view In Figure 2 and Figure 3, you can see the APC Smart UPS 3000 that you can get as an IBM option. Technical data is identical with the 5U model, but the option to increase the run time with an additional battery pack is not available. 2.1.3 SU5000RMINET SU5000RMINET is a rack-mount UPS. The extended run time feature isn't available for this model. Table 5. Technical specifications 8 VA/W ratings Maximum 5000 VA or 3750 W Dimensions: UPS Height 22.
The typical SU5000RMINET on-battery run times versus VA load, in minutes are as follows: Table 6. Run time SU5000RMINET Load Internal battery 1000 VA 64 1200 VA 50 1400 VA 39 1600 VA 32 2000VA 23 2200 VA 20 2500 VA 16 3000 VA 11 3500 VA 10 4000 VA 8 4500 VA 7 5000 VA 5 The UPS is furnished with six 1.8 m long output cords appropriate for connection to equipment with 10 Amp IEC320 appliance receptacles. There is also a set of 19'' rack mounting ears in the package.
Figure 4. APC Smart UPS 5000 — front view Figure 5. APC Smart UPS 5000 — rear view 2.2 Symmetra You cannot buy Symmetra as an IBM option. The Symmetra is available only from APC.
2.2.1 Symmetra Masterframe/Miniframe The Symmetra is an uninterruptible power array system, designed for large-scale loads. It provides conditioned, reliable AC power to load equipment, and provides protection from power blackouts, brownouts, swells, sags, surges and interference. The Symmetra Power Array system comprises either a Miniframe or a Masterframe, and a variable set of modules (see 2.2.1.4, “VA ratings” on page 12). Both battery modules and power modules are available.
2.2.1.1 Battery source The battery source is comprised of parallel, hot-swappable, 120 V battery modules. These are housed in the Symmetra frame, and in an optional XR Extension Battery frame. A Symmetra Miniframe provides bays for up to two battery modules, and a Masterframe provides bays for up to four. Both of these frames can be connected to an XR Extension Battery frame. Additional battery modules increase battery run time. 2.2.1.
UPS UPS UPS UPS UPS UPS UPS UPS Intelligent Battery Battery Power 8 kVA 31”H x 24”W x 27”D SYMINI Intelligent Battery Battery Battery Battery Power 16 kVA Battery Battery Battery Battery Extended Battery Battery Battery Battery Battery Battery Battery Battery Battery Battery Battery Battery Battery Extended Battery 45”H x 24”W x 27”D SYMSTR 18”H x 24”W x 27”D SYXR4 46”H x 24”W x 27”D SYXR12 APC part numbers: • • • • • • • • • SY4KEXI SY8KI SY8KEXI SY12KEXI SY16KI SYPM SYBATT SYXR4 SYXR12 Minifra
For recommendations about Symmetra wiring, see the PDF file at the following address: http://sturgeon.apcc.com/techref.nsf/umanuals/094362544B00279C8525675C006F AEE2?OpenDocument 2.3 UPS options There are many options for UPS units available, such as Interface Expander Cards or WEB/SNMP Management Cards, which we describe in the next sections. 2.3.
Figure 7. AP9607 — APC SmartSlot Interface Expander Card 2.3.2 AP9606 Web/SNMP Management Card The AP9606 Management Card provides the hardware and firmware needed to connect your APC UPS to a 10 Mbps Ethernet network and use that network for remote (over the network) management of the Management Card, its UPS, and a Measure-UPS. The Management Card also allows you to use a terminal for local management.
Figure 8. AP9606 — APC Web/SNMP Card 2.3.3 Redundant Switch The Redundant Switch is a high availability UPS accessory designed to provide clean, reliable AC power. It provides a seamless transfer to an alternative AC source when the input is outside the acceptable range. It can withstand zero to twice the nominal input voltage, while preventing any possibly damaging transients or gaps in output voltage from reaching the protected equipment.
The use of a double pole transfer switch by default makes the Redundant Switch fault tolerant. A single point of failure in the electronics does not cause a dropout of the output voltage. The transfer switch must select one input or the other, which are both acceptable, by definition, when a single fault in the Redundant Switch occurs. The Redundant Switch is furnished with brackets and slides for mounting in standard 19" rack systems. Brackets for 23" applications are available as an accessory.
Figure 10. Redundant Switch For the Redundant Switch SU044-1 we have a sample solution in 3.3.1, “Solution with multiple UPS units and Redundant Switch” on page 51. 2.4 APC monitoring and management software 2.4.1 PowerChute PLUS PowerChute PLUS software provides UPS manageability and safe system shutdown for desktops, workstations, and servers protected by APC UPSs. The software enables you to monitor and control any APC UPS that has a serial interface port. Figure 11.
• User notification of impending shutdown • Power event and data logging • Auto-restart upon power return • UPS battery conservation features • Diagnostic and management features, such as scheduled server shutdowns, interactive/scheduled battery testing, and detailed power quality logging • Real-time graphical displays of transient data, such as battery voltage, UPS load, utility line voltage, run time remaining, battery capacity, and battery voltage 2.4.1.
• Run a command file (an external executable file) • Page users • Send e-mail to notify users that the event occurred Figure 12. PowerChute PLUS Event Actions Menu (Go to Main Menu-Configuration-Event Actions) 2.4.1.
• Comm Lost While On Battery Communication with the UPS has been lost while the UPS is on battery. The event may be caused by a loose communication cable or, rarely, by a software conflict, such as an application inadvertently blocking PowerChute PLUS from monitoring the serial port while the UPS is on battery. RECOMMENDED ACTION: Run the command file UPS.CMD to either move or set the Cluster Groups offline. This script is described in Chapter 4, “The command file UPS.CMD” on page 67.
RECOMMENDED ACTION: Immediately notify the cluster administrator. Use the notify function that was implemented in PowerChute PLUS. 2.4.1.5 Symmetra events Apart from the events mentioned above, the Symmetra system features additional events that are specific to that UPS model only: • UPS On Bypass: Failure The Symmetra UPS is on bypass due to a UPS failure. RECOMMENDED ACTION: Immediately notify the cluster administrator. • UPS Module Failed One of the Symmetra power modules has failed.
UPS On Battery event 60 sec on-battery delay Total run time available System Shutdown Starting event 60 sec shutdown delay 600 sec turn-off delay Command File executed (after a 10-second delay) Operating system shutdown starts "S" issued to put the UPS in sleep mode UPS enters sleep mode, powers outlets off Low Battery Condition event 600 sec Low battery warning delay Battery depleted UPS switches off completely Figure 13. PowerChute PLUS event actions and UPS delays 2.4.
Figure 14. PowerChute Network shutdown Web interface More information is available at: http://www.apcc.com/products/management/pc_networkshutdown.cfm 2.4.3 Signaling cables APC UPS will correctly communicate with the PowerChute monitoring software only when the correct APC cable is used. Using a standard RS232 cable will result in loss of communication between the UPS and the software. Table 9 shows which APC cable is required for your configuration.
AP940-0023A Order # AP9823 Simple signaling Grey 2 meters UNIX (except True64, Digital, DEC/OSF, SGI Irix, HP-UX on 800 machines and AS/400) AP940-0095A Plug & play cable smart signaling Grey 2 meters Windows 95/98 AP940-0095B Plug & play cable smart signaling Grey 2 meters Windows 95/98, NT, 2000 AP940-1500 Order # AP9815 Extension cable Grey 5 meters Only to be used in connection with a smart or simple signaling cable AP9825 Extension cable Grey <100 meters Isolated extension cable
IEC 320-C19 IEC 320-C14 Figure 16. Cable D12 IEC 320-C19 CEE 7/7 Figure 17. Cable CEE 7/7 IEC 320-C13 Figure 18.
Chapter 3. UPS configurations for cluster There are two different ways to use UPS units: • Single power line: one UPS for both servers, controllers, and enclosures. This will be used in small cluster configurations or with a large UPS (Symmetra). • Double power line: two UPS units, one UPS for each server, controller and enclosures are connected to both UPS units. This will be used in large data centers with two independent power lines.
3.1.1 Timing of UPS actions 600 s Power loss / Signal UPSOnBattery UPS Tur n Of f Del ay max. Figure 19 shows the timing of UPS actions. Run command file UPS.CMD Move res ources or set offline (approx. 300 sec) Execute SHUTGUI.EXE UPS On Battery Delay (5 s ec) W ait 120 s ec Begin W indows NT shutdown W indows NT shutdown (approx. 90 s ec) W indows NT shutdown finished Res erved (approx.
4. Some commands in UPS.CMD were launched asynchronously. To allow such operations to complete, we added a delay as a SHUTGUI.EXE parameter. The value of 120 seconds is an example. 5. After this SHUTGUI.EXE delay, the operating system begins to shut down. 6. From our experience, a Windows NT 4.0 shutdown requires approximately 90 seconds. This depends on the number of remaining (non-clustered) services and applications. 7. Figure 19 shows a reserve interval of 90 seconds.
• The IBM Netfinity EXP15 storage expansion enclosure (3520) has an electrical input between approximately 0.06 kVA and 0.39 kVA from two power supplies. (Reference: IBM Netfinity EXP15 Storage Expansion Unit Hardware Maintenance Manual.) • The electrical input of an IBM Fibre Channel RAID Unit (3526) is similar to an EXP15. (Reference: IBM Netfinity Fibre Channel Hardware Maintenance Manual.) The requirement of the IBM Fibre Channel Hub (3523) is approximately 0.2 kVA.
administrator intervention is useful to analyze the status of such components before starting production systems. • There is a possibility that a resource operation might be aborted by an operating system shutdown. In case of an aborted operation, you cannot accurately predict the status of all resources at the moment of restart. • With Windows 2000, the power management differs from Windows NT 4.0. At the end of an operating system shutdown, the machine is powered off automatically.
We recommend that you attach each server to one UPS and then connect each shared storage device to both UPS units, as shown in Figure 21. In this way, timing problems are avoided, and the configuration can easily be extended to a solution with two separate power lines (discussed in 3.3, “Solutions with double power lines” on page 51). Communication - Smart-signaling cable UPS Server 1 Phase A Cluster UPS Server 2 Communication - Smart-signaling cable Figure 21.
3.2.1 Control flow in UPS.CMD In the event of a UPS power loss, a delay (UPS On Battery Delay) is set. After this delay, the UPS service starts the UPS.CMD command file with the parameter SingleUPSOnBattery. The main task of the command file UPS.CMD is to call CLUSTER.EXE. This is a Microsoft Cluster Server utility program installed during cluster setup that can be used to administer clusters from the command prompt. The CLUSTER.
The operating system shutdown is started from the command file, not by using the PowerChute PLUS Shut Down Server action. When using this action, the server would be stopped after a maximum delay of 300 seconds. Since the time necessary to take a cluster resource offline varies, 300 seconds might be too short. Therefore, the script utilizes the SHUTGUI.EXE tool from the Microsoft Windows NT Resource Kit CD. 3.2.
PLUS on the node with a black serial cable” on page 35 and 3.2.6, “Installing PowerChute PLUS on the node with a grey serial cable” on page 44. 1. Create a new directory C:\UPS_CMD. 2. Create the command file UPS.CMD in C:\UPS_CMD (you can find the content of the file in Appendix B, “UPS.CMD” on page 77 or download it from http://www.redbooks.ibm.com/). 3. Copy the file SHUTGUI.EXE from the Windows NT Resource Kit CD to C:\UPS_CMD. As result, you should have a directory similar to Figure 24 on page 35.
2. Select the Custom option and click Next. This displays Figure 26. Figure 26. Install PowerChute PLUS — select components 3. Select the boxes to install the components you wish to install. Select PowerChute PLUS Client and PowerChute PLUS UPS Service. The other components are optional. Click Next to display the Select Components window. Figure 27. Install PowerChute PLUS — automatic shutdown components 4. Do not check any boxes of these components for automatic application shutdown. Click Next.
Figure 28. Install PowerChute PLUS — automatically detect UPS 5. The window shown in Figure 28 will be displayed. First, make sure that the smart-signaling cable is connected to the PC interface port of the UPS and to the serial COM port of the server. The smart-signaling cable is a black cable with the part number 940-0024C (a short cable) or 940-1524C (a longer cable). 6. Click the Yes button to automatically detect the UPS.
. Figure 30. Install PowerChute PLUS — remote monitoring 8. In this window select the box to enable the PowerChute PLUS remote monitoring function. For details see the PowerChute PLUS documentation. 9. Finish the installation by clicking Next. If you wish, you can now register your hardware and software with APC. 3.2.
2. Select your server. In our scenario, the second node is the NF8500R with the black serial cable. Click the Attach button, which will display the PowerChute PLUS main window shown in Figure 32. Figure 32. PowerChute PLUS — main window 3. In the PowerChute PLUS main window, select Configuration > UPS Shutdown Parameters. Figure 33 will be displayed. Figure 33. UPS Shutdown Parameters window 4.
– UPS Turn Off Delay: This period of time begins at power loss. After this interval, the UPS turns off its output power (independent of any shutdown completion). If line voltage returns during this period, turning off output power is canceled (again independent of any shutdown operations in progress). Possible delay values are 20, 180, 300, and 600 seconds. We recommend that you set the value as estimated in the time line (Figure 19 on page 28).
Figure 35. Single UPS Run Command File — UPS On Battery 8. In the Event Actions window, select UPS On Battery. a. Select the Run Command File checkbox and click Options. In the pop-up window, insert the following in the Command File field: “C:\UPS_CMD\UPS.CMD” SingleUPSOnBattery >> C:\UPS_CMD\LOG.TXT b. Set the Wait value to 5 seconds. This wait time (UPS On Battery Delay in Figure 19 on page 28) before executing the command file prevents short power failures from shutting down the cluster.
Figure 36. Single UPS Run Command File — Low Battery Condition 9. Now select Low Battery Condition in the Event Actions window. a. Check Run Command File the checkbox and click Options. b. In the pop-up window, insert the following in the Command File field: “C:\UPS_CMD\UPS.CMD” SingleUPSOnBattery >> C:\UPS_CMD\LOG.TXT c. Set the Wait value to 0 seconds. A low battery condition means that the UPS can only supply the systems for a short period and immediate actions must take place.
Figure 37. Single UPS Event Actions 10.In the Event Actions window, select PowerChute PLUS Started. Clear the check in the Run Command File checkbox. We strongly recommend that you do not start the cluster automatically. An administrator has to evaluate the situation and take the required measurements. Besides, it is mandatory to start the systems in a data center in a certain order (PDC, BDC, databases, etc.). Usually the MSCS is the last component to start. Figure 38.
11.In the Event Actions window shown in Figure 38, select Comm Lost While On Battery. a. Select the Run Command File checkbox and click Options. b. In the pop-up window, insert the following in the Command File field: “C:\UPS_CMD\UPS.CMD” SingleUPSOnBattery >> C:\UPS_CMD\LOG.TXT c. Set the Wait value to 0 seconds. If the UPS is on battery and the communication is lost, an immediate action is required because the state and the run time of the UPS are unknown. 3.2.
Figure 40. Install PowerChute PLUS — Select Components to Install 3. Select the boxes of the components you wish to install. Select PowerChute PLUS Client and PowerChute PLUS UPS Service. The other components are optional. Click Next to display the Select Components window. Figure 41. Install PowerChute PLUS — Select Next Components 4. Do not check any boxes of these components for automatic application shutdown. Click Next. 5. The window shown in Figure 42 will be displayed.
Figure 42. Install PowerChute PLUS — automatically detect UPS parameters 7. The UPS will not be found by the installation program because auto detection via the simple-signaling cable is not possible. Thus click the No button. Figure 43 will be displayed. Figure 43. Select UPS Type and COM Port window 8. Select Back-UPS as the UPS type and the COM port that is connected to the UPS with the grey serial cable (part number 940-0020B). Click the Next button and Figure 44 will be displayed.
Figure 44. Disable remote monitoring 9. In this window, select the box to enable the PowerChute PLUS remote monitoring function. For details see the PowerChute PLUS documentation. 10.Finish the installation by clicking the Next button. If you wish, you can now register your hardware and software with APC. 3.2.
2. Select your server. In our scenario, the first node is the NF8500L with the grey serial cable. Click the Attach button, which will display the main window, shown in Figure 46. Figure 46. PowerChute PLUS — main window 3. In the PowerChute PLUS main window, select Configuration > Application Shutdown Parameters. Figure 47 will be displayed. Figure 47. Single UPS Application Shutdown Parameters 4.
Figure 48. Single UPS Run Command File — UPS On Battery 6. In the Event Actions window, select UPS On Battery. a. Select the Run Command File checkbox and click Options. In the pop-up window, insert the following in the Command File field: “C:\UPS_CMD\UPS.CMD” SingleUPSOnBattery >> C:\UPS_CMD\LOG.TXT b. Set the Wait value to 5 seconds. This wait time (UPS On Battery Delay in Figure 19 on page 28) before executing the command file prevents short power failures from shutting down the cluster.
Figure 49. Single UPS Run Command File — Low Battery Condition 7. In the Event Actions window, select Low Battery Condition. a. Select the Run Command File checkbox and click Options. b. In the pop-up window, enter the following in the Command File field: “C:\UPS_CMD\UPS.CMD” SingleUPSOnBattery >> C:\UPS_CMD\LOG.TXT c. Set Wait to 0 seconds. A low battery condition means that the UPS can supply the systems for a short period only and immediate actions must take place.
8. Select PowerChute PLUS Started. Clear the Run Command File checkbox. We strongly recommend that you do not start the cluster automatically. An administrator has to evaluate the situation and take the required measurements. Besides, it is mandatory to start the systems in a data center in a certain order (PDC, BDC, databases, etc.). Usually the MSCS is the last component to start. 3.
Communication Communication Phase A UPS 1 Power Redundant Switch Phase B UPS 2 Power Server 1 Power Communication Cluster Communication Phase A' UPS 3 Power Redundant Switch Phase B' UPS 4 Power Server 2 Power Communication Communication Figure 51. Multiple UPS units with Redundant Switch UPS fault tolerance is provided by two independent phases (A, B) powering two pairs of UPS units.
Communications - smart-signaling cable Phase A UPS 1 Server 1 Cluster Phase B UPS 2 Server 2 Communications - smart-signaling cable Figure 52. Two UPS units without application handling But connecting the power cords in this way prevents correct application handling. Communication is possible only between one server and one UPS. Therefore, if a server receives a UPS signal, software on this server cannot check the power status of the second UPS. For example, phase B fails.
Communications - smart signaling cable Phase A UPS 1 Server 1 Cluster Phase B UPS 2 Server 2 Communications - smart signaling cable Figure 53. Two UPS units with application handling This setup looks very similar to Figure 52 on page 53. The important difference is the one-to-one relation between UPS status and server status. In Figure 52, the status of a server is determined by the status of two UPS units, but the server can communicate with one UPS only.
What should a node do with cluster resources that are owned at the time of a power loss? If the other node is in status Node Up, then this node is able to get the resources from the failing node. Because each node losing power sets its status to Node Paused, we avoid useless resource moves. In any other case, the resources currently owned by the failing node must be brought offline.
UPS On Battery abnormal condition On Battery Delay in PC+ run UPC.CMD UPSOnBattery set local node to PAUSED if remote node "Up" no yes Delay if remote node "Up" no yes move Cluster Groups set Cluster Groups to offline shut down Windows NT 4.0 Figure 54. Double UPS flowchart The flowchart shown in Figure 54 has three branches. The UPS On Battery condition is signaled by the UPS. PowerChute PLUS recognizes this condition and starts the UPS.CMD command file with the parameter UPSOnBattery.
The three failure scenarios in 3.3, “Solutions with double power lines” on page 51 are handled as follows: 1. One power line works properly and the other fails All resources are moved to the surviving node. If the remote node has been set manually to Node Paused by the administrator, the script will run the branch “both power lines are failing”. 2. Both power lines fail at the same time Each node brings the resources offline (exceptions: quorum disk and Cluster Group). 3.
both APC SMART-UPS 5000 for redundancy. The hubs are connected only to one APC SMART-UPS 5000 because they have only one power supply. Each Netfinity is connected with its UPS via a black serial cable (940-0024C). The left node is named NF8500L, the right node NF8500R. 3.3.5 Preparing both nodes To implement our solution, additional files are necessary, as follows: 1. Create a new directory C:\UPS_CMD. 2. For Windows NT, download the file UPS.
Figure 57. Install PowerChute PLUS — choose type of installation 1. Select the Custom option and click Next. This displays Figure 58. Figure 58. Install PowerChute PLUS — select components to install 2. Select the boxes to install the components you wish to install. Select PowerChute PLUS Client and the PowerChute PLUS UPS Service. The other components are optional. Click Next to display the Select Components window shown in Figure 59. Chapter 3.
Figure 59. Install PowerChute PLUS — automatic shutdown components 3. Do not check any boxes of these components for automatic application shutdown. Click Next. The window shown in Figure 60 will be displayed. Figure 60. Install PowerChute PLUS — select UPS parameters 4. First, make sure that the current node’s COM port is connected via a smart-signaling cable to the PC interface port of the corresponding UPS.
Figure 61. Install PowerChute PLUS — automatically detect the UPS 6. Your UPS has been discovered correctly. Otherwise you can select the UPS type and COM port manually from the pull-down menus. Click the Next button and Figure 62 will be displayed. . Figure 62. Install PowerChute PLUS — remote monitoring 7. In this window, check the box to enable the PowerChute PLUS remote monitoring function. For details see the PowerChute PLUS documentation. 8. Finish the installation by clicking Next.
1. Start PowerChute PLUS. You see a window (shown in Figure 63) with all servers in the same IP subnet segment where the PowerChute PLUS software is installed. Figure 63. Double UPS Monitor Server 2. Select the current node and click the Attach button. This will display the PowerChute PLUS main window, shown in Figure 64. Figure 64. PowerChute PLUS — main window 3. In the PowerChute PLUS main window, select Configuration > UPS Shutdown Parameters. Figure 65 will be displayed.
Figure 65. UPS Shutdown Parameters window 4. On the Shutdown Parameters window, enter the following parameters: – UPS Low Battery Signal Time: This condition will occur if the battery is very old and the UPS can supply power for a short period only (thus immediate actions are required). It is the minimum number of minutes of battery run time that the UPS needs to perform the essential tasks of a safe system shutdown. Possible values are 2, 5, 7, and 10 minutes.
Figure 66. Double UPS Application Shutdown Parameters window 6. Disable the application shutdown (because this will be handled by the command file UPS.CMD) and click the OK button. 7. In the PowerChute PLUS main window (Figure 64 on page 62), select Configuration > Event Actions. Figure 67 will be displayed. Figure 67. Double UPS Run Command File — UPS On Battery 8. In the Event Actions window, select UPS On Battery. a. Check the Run Command File checkbox and click Options.
Figure 68. Double UPS Run Command File — Low Battery Condition 9. In the Event Actions window, select Low Battery Condition. a. Check the Run Command File checkbox and click Options. b. In the pop-up window, insert the following in the Command File field: “C:\UPS_CMD\UPS.CMD” UPSOnBattery >> C:\UPS_CMD\LOG.TXT c. Set the Wait value to 0 seconds. A low battery condition means that the UPS can only supply the systems for a short period, and immediate actions must take place.
Clear the check in the Run Command File checkbox. We strongly recommend that you do not start the cluster automatically. An administrator has to evaluate the situation and take the required measurements. Besides, it is mandatory to start the systems in a data center in a certain order (PDC, BDC, databases, etc.). Usually the MSCS is the last component to start. Figure 70. Double UPS Run Command File — Comm Lost While On Battery 11.In the Event Actions window, select Comm Lost While On Battery. a.
Chapter 4. The command file UPS.CMD All actions necessary for shutting down a node in the cluster are performed by the command file UPS.CMD. The command file covers attachment of the cluster to one or two UPS units. Startup procedures may be implemented as well. Downloading UPS.CMD See Appendix A, “Downloading the additional material” on page 73 for insturctions on how to download UPS.CMD. In this chapter, we explain the command file’s structure.
In lines 49 and 50, the environment variable for the computer name is compared with variables defined in line 43 and 44. 049 IF %NodeAName% == %COMPUTERNAME% SET TheOtherNodeName=%NodeBName% 050 IF %NodeBName% == %COMPUTERNAME% SET TheOtherNodeName=%NodeAName% This comparison is used to set the name of the remote node. In line 56, a check for the start parameter is made.
094 CLUSTER.EXE %TheOtherNodeName% NODE %TheOtherNodeName% | FIND "Up" The result of this check is required for the decision whether the Cluster Groups will be moved to the remote node or brought offline. Note that the code looks for English output from CLUSTER.EXE. In a non-English version of Microsoft Windows NT 4.0, this may fail! Thus, if you use another language, you have to verify the code.
/MOVETO initiates moving of a resource to the node specified. Note that we don’t add a /WAIT parameter; thus all commands are issued asynchronously. The command in line 136 sends a network message to the domain (in our scenario, the domain APC) about the steps initiated by the script.
According to recommendations from Microsoft, there should no application resources belonging to the Cluster Group (see http://support.microsoft.com/support/kb/articles/Q168/9/48.ASP). It is not necessary to set the Cluster Group offline. Hence, it is not required to bring the Cluster Group online after restart; the cluster name and IP address will be accessible via network automatically. The group with the quorum disk is a special case because the quorum disk itself cannot be brought offline.
A parameter value SingleUPSOnBattery means a configuration with one UPS. This is explained below. The parameter value UPSOnBattery in the case of two UPS units was discussed in 4.2, “Parameter UPSOnBattery” on page 68. If the condition in line 59 is fulfilled, then there is one UPS. The code for this case begins on line 107: 107 IF %1 == SingleUPSOnBattery GOTO SingleUPSOnBattery As the first step, the node that runs the command file enters the Node Paused status (line 110): 110 CLUSTER .
Appendix A. Downloading the additional material The programs and CMD files shown in Appendix B, “UPS.CMD” on page 77 and Appendix C, “DELAY3.EXE source” on page 81 are also available from the IBM Redbooks Web server. Point your Web browser to: ftp://www.redbooks.ibm.com/redbooks/REDP0402 Case sensative This FTP URL is case sensative — REDP0402 is in uppercase. Alternatively, you can go to the IBM Redbooks Web site at: ibm.com/redbooks Select the Additional materials and open the directory REDP0402.
1. In the PowerChute PLUS main window, select Configuration > Event Actions. 2. In the Event Actions window, select the PowerChute PLUS Started event. 3. Select the RunCommand File checkbox and click Options. Insert the following in the Command File field: "C:\UPS_CMD\UPS.CMD" StartUp >> C:\UPS_CMD\LOG.TXT You must also change the UPS.CMD command file at point 2. Start the cluster Server. If the Cluster is already started at this moment, you get an error message that the service already started.
A hint for Windows 2000 users: The quorum resource is no longer in a separate resource group. It is now in the Cluster Group. Appendix A.
76 Implementing UPS Configurations with Microsoft Cluster Server
Appendix B. UPS.
058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 78 IF %1 == UPSOnBattery GOTO UPSOnBattery IF %1 == SingleUPSOnBattery GOTO SingleUPSOnBattery GOTO ERROR1 REM --------------------------------------------------------------------------REM --------------------------------------------------------------------------:StartU
118 REM Move groups to the other node 119 120 REM !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 121 122 REM (3) Enter one line as shown below 123 REM for each resource group which should be moved to the surviving node 124 REM when one node shuts down! 125 126 cluster . GROUP “Cluster Group” /MOVETO:%TheOtherNodeName% > NUL 127 cluster . GROUP “Quorum” /MOVETO:%TheOtherNodeName% > NUL 128 cluster . GROUP “VFS_a” /MOVETO:%TheOtherNodeName% > NUL 129 cluster .
176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 80 REM --------------------------------------------------------------------------:Shutdown ECHO Initiating shutdown C:\UPS_CMD\SHUTGUI.
Appendix C. DELAY3.
StopSeconds:=Second + DelaySeconds; Temp := 0; if StopSeconds >= 60 then begin StopSeconds := Stopseconds - 60; Temp := 1; end; StopMinute:=Minute + DelayMinute + Temp; Temp := 0; if StopMinute >= 60 then begin StopMinute := StopMinute - 60; Temp := 1; end; StopHour:=Hour + DelayHour + Temp; Temp := 0; if StopHour >= 24 then begin StopHour := StopHour - 24; Temp := 1; end; str(Hour,strHour); str(Minute,strMinute); str(Second,strSeconds); strTime:=strHour+ strMinute+ strSeconds; str(StopHour,strStopHour); st
writeln('* where xxx are the seconds that should be waited. writeln('* The maximum time is 65000 Seconds *'); *'); writeln('*******************************************************************************'); end; end; {donewincrt;} end. Appendix C. DELAY3.
84 Implementing UPS Configurations with Microsoft Cluster Server
Appendix D. Referenced documents • IBM Netfinity 8500R Hardware Maintenance Manual (8681-4RY, 4RG, 5RY, 5RG, 6RY, and 6RG), available from: ftp://ftp.pc.ibm.com/pub/pccbbs/pc_servers/37l5123.pdf • IBM Netfinity EXP15 Storage Expansion Unit Hardware Maintenance Manual (Type 3520), available from ftp://ftp.pc.ibm.com/pub/pccbbs/pc_servers/10l9839.pdf • IBM Netfinity Fibre Channel Hardware Maintenance Manual, available from ftp://ftp.pc.ibm.com/pub/pccbbs/pc_servers/19k2481.
86 Implementing UPS Configurations with Microsoft Cluster Server
Appendix E. Special notices This publication is intended to help customers implement uninterruptible power supplies in a Microsoft Cluster Server environment. The information in this publication is not intended as the specification of any programming interfaces that are provided by Netfinity servers. See the PUBLICATIONS section of the IBM Programming Announcements for more information about what publications are considered to be product documentation.
The following terms are trademarks of other companies: Tivoli, Manage. Anything. Anywhere.,The Power To Manage., Anything. Anywhere.,TME, NetView, Cross-Site, Tivoli Ready, Tivoli Certified, Planet Tivoli, and Tivoli Enterprise are trademarks or registered trademarks of Tivoli Systems Inc., an IBM company, in the United States, other countries, or both. In Denmark, Tivoli is a trademark licensed from Kjøbenhavns Sommer - Tivoli A/S. C-bus is a trademark of Corollary, Inc.
