HP Global Workload Manager 7.0 User Guide Abstract This document presents an overview of the techniques and tools available for using Global Workload Manager servers (or gWLM). It exposes you to the essentials and allows you to quickly get started with gWLM. This document is intended to be used by HP Matrix Operating Environment system administrators, application administrators, and other technical professionals involved with data center operations, administration, and planning.
© Copyright 2004, 2012 Hewlett-Packard Development Company, L.P. Legal Notice Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor’s standard commercial license. The information contained herein is subject to change without notice.
Contents 1 Overview..................................................................................................6 gWLM Overview......................................................................................................................6 Benefits of using gWLM.............................................................................................................6 Comparison of PRM, WLM, and gWLM features..........................................................................
5 Additional configuration and administration tasks.........................................29 Manually adjusting CPU resources............................................................................................29 Manually adjusting memory resources.......................................................................................30 Setting aside space for historical data.......................................................................................
Deleting workloads takes a long time...................................................................................50 Integrity VM prevents discovery of psets and fss groups...........................................................51 Only workloads with managed siblings can be added to SRDs with nested partitions.................51 Configurations with Psets nested in virtual partitions rejected with vPars versions earlier than vPars A.03.05 ..............................................................
1 Overview This chapter provides an overview of gWLM, including benefits, key concepts and terms, and the gWLM management model. gWLM Overview gWLM allows you to centrally define resource-sharing policies that you can use across multiple HP servers. Using these policies can increase system utilization and facilitate controlled sharing of system resources. In addition, gWLM provides both real-time and historical monitoring of the resource allocation.
Concepts and terms for using gWLM Here are some concepts and terms to know when using gWLM: Workload The collection of processes executing within a single compartment. The compartment can be an nPartition (npar), a virtual partition (vPar), a virtual machine provided by HP Integrity Virtual Machines (hpvm), a processor set (pset), or a Fair Share Scheduler (fss) group. gWLM manages a workload by adjusting the system resource allocations for its compartment.
Mode Two modes are available: advisory and managed. Advisory mode allows you to see what CPU resource requests gWLM would make for a workload—without actually affecting resource allocation. Advisory mode is not available for SRDs containing virtual machines, psets, or fss groups due to the nature of these compartments. Use this mode when creating and fine-tuning your policies.
Combining gWLM A.04.00.07 or later and an appropriate version of the iCAP software, gWLM's ability to manage nPars using iCAP is extended across multiple complexes that are members of the same Global iCAP group. • HP-UX Virtual Partitions (vPar) A virtual partition is a software partition of a server or of a single nPartition, where each virtual partition runs its own instance of the HP-UX operating system. A virtual partition cannot span an nPartition boundary.
a. b. c. gWLM monitors the CPU resource consumption of all the workloads in the SRD during the current allocation interval. At the end of the interval, gWLM adjusts the CPU resource allocations for the compartments in accordance with the policies. It also makes the allocation data available for real-time and historical reports. gWLM repeats the previous two substeps.
(The System Insight Manager menu bar and HP Matrix OE visualization menu bar are discussed in the section “Tabs and menus” (page 15).) • gwlm command A command-line interface, described in gwlm(1M) . Other components of the command-line interface are: vseinitconfig(1M), gwlmcmsd(1M), gwlmagent(1M), gwlmreport(1M), gwlmplace(1M), gwlmsend(1M), gwlmsslconfig(1M), gwlmstatus(1M), and gwlmxml(4).
Table 2 Where to find additional information (continued) To... See... Learn about gWLM concepts. Global Workload Manager topic in online help1 or HP Global Workload Manager User Guide (this document) (http:// www.hp.com/go/matrixoe/docs) Learn gWLM terms. “Concepts and terms for using gWLM” (page 7) or Glossary in online help1 Learn gWLM best practices. “Getting the Most Out of gWLM” topic in online help1 Learn about other gWLM features.
2 Configuring gWLM to manage workloads This chapter describes the various aspects of configuring gWLM to effectively manage the resources for your workloads. Policy types You can define several types of policies to instruct gWLM how to manage the resources for your workloads. These types are: Fixed Allocates a fixed (constant) amount of CPU resources to a workload’s compartment. gWLM satisfies these policies before attempting to satisfy any other type of policies.
Table 3 Choosing a policy type If... Use the following type of policy... You want gWLM to allocate a constant amount of CPU resources to a workload. Fixed You have your own metric by which you want gWLM to manage a workload. Custom IT acts as a service provider to business units. OwnBorrow This policy type allows you to set an owned amount of resources, while also giving you control over how workloads borrow and lend resources.
Seeing how gWLM will perform without affecting the system gWLM provides an advisory mode that allows you to see how gWLM will approximately respond to a given SRD configuration—without putting gWLM in charge of your system’s resources. Using this mode, you can safely gain a better understanding of how gWLM works. In addition, you can check that your policies behave as expected—with minimal effect on the system.
2. On each managed node, start the gWLM agent if it is not already running: # /opt/gwlm/bin/gwlmagent Alternatively, you can start the agent through System Insight Manager, as discussed in the HP Matrix Operating Environment Software Installation and Update Guide. 3. Connect to System Insight Manager by pointing your web browser to: http://hostname:280 where hostname represents the hostname of the CMS. 4. Select: Tools→HP Matrix OE visualization... and then click the Shared Resource Domain tab.
To see gWLM in action: NOTE: You must be logged in as root on the systems where you run the mxstart, gwlmcmsd, and gwlmagent commands mentioned below. In System Insight Manager, you must be logged in as root or have authorizations for “All Tools” or “Matrix OE All tools.” 1.
Common uses for gWLM gWLM is a powerful tool that allows you to manage your systems in numerous ways. The following sections explain some of the more common tasks that gWLM can do for you. Fixing the amount of CPU resources a workload gets gWLM allows you to give a workload a fixed amount of CPU resources. This fixed amount is in the form of a set amount of CPU resources given to an npar, a vpar, a virtual machine, a pset, or an fss group.
NOTE: You must be logged in as root on the systems where you run the gwlmagent command mentioned below. 1. 2. Configure your CMS as indicated in the HP Matrix Operating Environment Software Installation and Update Guide, if you have not already done so.
Creating a new policy A policy instructs gWLM how to manage a workload’s resources. You can create a policy when managing a workload or create a policy separately. To create a policy separately: NOTE: In System Insight Manager, you must be logged in as root or have authorizations for “All Tools” or “Matrix OE All tools.” 1. 2.
Changing which policy is associated with a workload To change the policy affecting how gWLM allocates resources to a workload: NOTE: In System Insight Manager, you must be logged in as root or have authorizations for “All Tools” or “Matrix OE All tools.” 1. 2. Ensure in System Insight Manager, the gWLM CMS daemon or service (gwlmcmsd) and all the gWLM agents (gwlmagent) are still running, as explained in the section “Setting up gWLM (initial setup steps)” (page 18).
Stop managing a workload When you stop managing a workload: • gWLM stops managing resources for the workload • The workload’s definition is removed from the SRD, although it remains available for placing in another SRD NOTE: When gWLM stops managing npar-based or vpar-based workloads, it leaves the nPars or vPars with the number of cores they had in the last allocation interval. For this reason, in Step 3 below, you associate fixed policies with the workloads based on these types of compartments.
3. Associate fixed policies with all nPars or vPars that were in the SRD. When gWLM stops managing an SRD, it leaves compartments based on nPars or vPars as they were in the last allocation interval. Associating fixed policies allows you to set the sizes exactly to what you want. (For virtual machines, gWLM sets the entitlements of the running virtual machines to their minimums. psets and fss groups are removed in this situation, with their processes going to the default pset or default fss group.
3 Monitoring workloads and gWLM This chapter describes how to monitor workloads and gWLM. Monitoring workloads There are several methods for monitoring workloads, as described below. High-Level view To see a high-level view of the performance of your SRDs and workloads: 1. From the System Insight Manager menu bar, select: Tools→HP Matrix OE visualization... 2. Click the Shared Resource Domain tab.
Windows systems, the commands are in C:\Program Files\HP\Virtual Server Environment\bin\gwlm\ by default. However, a different path might have been selected at installation. NOTE: You must be logged in as root on HP-UX or into an account that is a member of the Administrators group on Windows to run the commands below. • gwlm monitor The gwlm command, available only on a CMS, has a monitor subcommand that displays policy, workload, and SRD statistics.
NOTE: On systems running Windows, log files are in C:\Program Files\HP\Virtual Server Environment\logs\ by default. However, a different path might have been selected at installation. The name of the current log always ends in .log.0. Once this file grows to a certain size, it is moved to a filename ending in .log.1 and a new .log.0 file is started. If a .log.1 file already exists, it is renamed .log.2. If a .log.2 file already exists, it is overwritten.
4 Security This chapter highlights several security items you should be aware of. General security topics The following items are a few general topics on security: • HP provides the HP-UX Bastille product, available from http://software.hp.com at no charge, for enhancing system security. • You can secure gWLM’s communications as explained in the following section. • System Insight Manager allows you to create user roles with different levels of privileges.
3. Set the following properties as desired: • oracle.net.encryption_types_client • oracle.net.crypto_checksum_types_client For more information on these properties, read their associated comments in the gwlmcms.properties file. 4. 5. 6. Ensure the Oracle listener and port being used by System Insight Manager is configured to accept secure communication for the encryption and checksum types specified in the previous step.
5 Additional configuration and administration tasks This chapter covers various configuration and administration tasks. Manually adjusting CPU resources When an SRD is created, it has a certain number of cores. gWLM manages the SRD using the same number of cores. If the SRD—or a policy used in the SRD—is configured to use Temporary Instant Capacity (TiCAP), gWLM can automatically activate that additional capacity to meet policies.
• Creation or deletion of a pset using psrset on a system where gWLM is managing pset compartments. • Performing online cell operations using parolrad. • Enabling and disabling Hyper-Threading. To make use of these additional resources using the gWLM command-line interface: 1. Undeploy the SRD containing the systems that you want to adjust. 2. Make your adjustments. 3. Re-create and re-deploy the SRD. 4.
For more information, see vseinitconfig(1M).
# /var/opt/gwlm/gwlm.log.0 # /var/opt/vse/logs/gwlminitconfig.log.0 # Valid levels, from most severe to least, are: # SEVERE # WARNING # INFO # CONFIG # FINE # FINER # FINEST # When you set the level, you will see messages only from that level and # the levels that are more severe. So, the SEVERE level produces the fewest # messages, while the FINEST level includes messages from all seven levels. # com.hp.gwlm.util.Log.logLevel = INFO # # Specify the size (in MB) and number of files to use # for logging.
# # Memory = 5000 * 4 * 20 * (60 / 15) ~ 2.5 MB. # # cachesize: # The number of minutes of real-time data to maintain on the remote # node for future CMS access. This value must be at least three # times the 'samples' value specified below. The default value is # 20 minutes. # # samples: # The number of minutes of real-time data used to aggregate into a # historical data point. The default is to aggregate the data into # 5-minute averages. # com.hp.gwlm.node.cachesize = 20 com.hp.gwlm.node.
# # # # # # # # # # # # # $Date: 2008/12/02 20:17:18 $ $Revision: 1.1 $ Contents: This file contains the default configuration values for the Global Workload Manager Agent on a given managed node. The agent on each managed node uses the default values unless you edit that node's gwlmagent.properties file. You must restart gwlmagent for changes made to this file to take effect. # # Set FileHandler /var/opt/gwlm/gwlmagent.log.0 log level.
# that must be available for TiCAP activation. gWLM will stop using # TiCAP when the available balance goes below this threshold. The # same value should be set on all agents managing the SRD. To use # TiCAP, it must be available on the complex and enabled at the policy # level or the SRD level. # com.hp.gwlm.node.ticap.minimumBalance = 30 Communications ports gWLM uses the following ports for communications: Managed nodes: 9617 CMS: 9618 If you need to change these ports, add the following lines: com.hp.
NOTE: HP recommends starting these daemons at boot only when used in a secure operating environment. Setting GWLM_CMS_START to 0 (zero) prevents automatic use at boot of HP Matrix OE visualization and HP Capacity Advisor. The gwlmCtl file is shown below: ######################################################################## # (C) Copyright 2004-2008 Hewlett-Packard Development Company, L.P. # # gWLM Configuration File # # $Revision: 1.
This feature works best when one managed node is lost at a time or all managed nodes are lost. NOTE: If a vpar is borrowing cores from other vPars when it loses contact with its SRD, those borrowed cores might be separated from the SRD. If the vpar might be down for an extended time, check that the SRD has reformed without that vpar and that it has enough cores to meet its commitments. If not, try using vparmodify to reclaim some of the cores.
2. 3. Verify the agent rejoined the SRD by monitoring the Shared Resource Domain View in System Insight Manager or by using the gwlm monitor command. If the problem persists, check the files /var/opt/gwlm/gwlmagent.log.0 and /var/ opt/gwlm/gwlmcmsd.log.0 for additional diagnostic messages. “SRD Communication Issue” and “SRD Reformed with Partial Set of Nodes” events NOTE: SRD.
Nesting partitions gWLM allows you to form SRDs consisting of various compartment types. This ability provides flexibility in dividing your complex. For example, you can divide your complex as shown in Figure 2. The complex has four nPars, two of which are divided into vPars. One npar is hosting virtual machines, and the fourth npar is not divided. gWLM allows you to create an SRD containing the two virtual machine guests, the two vPars from npar 2, the two vPars from npar 3, and npar 4.
Create→Shared Resource Domain • When editing an SRD From the System Insight Manager menu bar, select: Tools→HP Matrix OE visualization... Then, click the Shared Resource Domain tab. From the HP Matrix OE visualization menu bar, select: Modify→Shared Resource Domain Changing the interval on the command line Use the gwlm command and a text editor to change the interval on the command line: 1. Use gwlm export to get a copy of the SRD’s XML definition from the gWLM configuration repository. 2.
Figure 3 Using gWLM with hosts separated by firewalls Firewall Public network vparA vparB vparC vparD Hostname on public network CMS mgmtA mgmtB mgmtC mgmtD Hostname on management LAN Management LAN On the public network, the hosts can be accessed through their hostnames: vparA, vparB, vparC, and vparD. However, they cannot access each other through those names.
2. The CMS must also be in the management LAN. If the primary hostname for the CMS is not the name it uses in the management LAN: a. Edit the gwlmagent.properties file on the CMS to include the property: com.hp.gwlm.security.virtualLocalHostName=hostnameOnLAN On HP-UX, the gwlmagent.properties file is in /etc/opt/gwlm/conf/. On Windows, it is in C:\Program Files\HP\Virtual Server Environment\conf\. (The given Windows path is the default; however, a different path might have been selected at installation.
Incorrectly configured host name or IP address You might see the following message in a log file (gwlmagent.log.0 or gwlmcmsd.log.0): Unable to determine the network address and/or hostname of the current host. This indicates a mis-configured network and/or a host name resolution issue for this host. For troubleshooting information, see the HP Matrix Operating Environment Release Notes and search for this message.
Trying DNS Name: mysystem.mydomain.com Address: 15.11.100.17 Fix any issues by editing /etc/hosts or for additional information, see: • The HP-UX IP Address and Client Management Administrator's Guide, available online at http://www.hp.com/bizsupport. • The BIND 9 Administrator Reference Manual, available from the Internet Systems Consortium at http://www.isc.org/sw/bind/arm93. • The Windows documentation.
6 Support and other resources This chapter contains support information and the available resources for the HP Global Workload Manager (gWLM) servers.
Warranty information HP will replace defective delivery media for a period of 90 days from the date of purchase. This warranty applies to all Insight software products. HP authorized resellers For the name of the nearest HP authorized reseller, see the following sources: • In the United States, see the HP U.S. service locator web site: http://www.hp.com/service_locator • In other locations, see the Contact HP worldwide web site: http://welcome.hp.com/country/us/en/wwcontact.
A Compatibility with agents The gWLM A.7.0.0.* CMS runs on HP-UX 11i v2 (B.11.23), HP-UX 11i v3 (B.11.31), and Microsoft Windows systems. It works with the following versions of the agents: • gWLM A.03.00.00.05: HP-UX 11i v1, HP-UX 11i v2, HP-UX 11i v3 • gWLM A.03.00.01.05: HP-UX 11i v1, HP-UX 11i v2, HP-UX 11i v3 • gWLM A.04.00.07: HP-UX 11i v1, HP-UX 11i v2, HP-UX 11i v3 • gWLM A.04.01.00.*: HP-UX 11i v1, HP-UX 11i v2, HP-UX 11i v3 • gWLM A.6.0.0.
B Global Workload Manager A.7.0.0* Known issues This appendix contains the limitations and known issues for the Global Workload Manager (gWLM) A.7.0.0* release. Limitations The following are limitations for Global Workload Manager. Unpredictable management when removing GiCAP group members If you have removed hosts from a GiCAP group before removing them from an SRD, management of the SRD becomes unpredictable.
Workaround This message indicates that there are no supported resource-sharing mechanisms available between the specified hosts. The message can occur if: • You have specified hosts in different complexes and the complexes are not managed in the same GiCAP group. • You have specified hosts in different nPartitions in a complex when there are no iCAP usage rights to share between the nPartitions.
between cluster members. Thus, it is recommended that you not directly manage a workload associated with a Serviceguard package. Workaround For all hosts to which a workload associated with a Serviceguard package might fail over, you must apply a policy to an enclosing operating system instance (virtual partition or nPartition). You can use a gWLM conditional policy to change the resource allocation depending on which packages are present.
Workaround Remove old historical monitoring and configuration data from the gWLM database by entering the following command: # gwlm history --truncate --truncate= If you prefer not to trim the database, you can delete multiple workloads simultaneously using the gwlm delete command. For more information, see gwlm(1M).
Workaround There is no workaround; this is simply how fss groups are implemented on a system with psets. You can continue with your fss groups inside pset 0 (leaving the other psets unmanaged), manage using psets instead (ignoring fss groups), or remove all the psets (other than pset 0) using the following command: # psrset -d all Custom metrics lost on redeploy Custom policies use metric values that you provide via the gwlmsend command.
Workaround Use Java 1.5.0.12 or later. gWLM commands core dump Attempts to run gwlm commands result in core dumps when /var is full. Workaround Make space available in /var. Multiple time changes on CMS might render gWLM unusable If the time on the CMS is changed to a point in the future, changes to the gWLM configuration are made and time on the CMS is moved back to the present; gWLM will not recognize any new changes as the latest configuration.
this error happens again, consult gwlmagent man page for steps to return to compliance. Workaround There is no workaround. You can ignore this message since this message is not valid on non-partitionable machines where iCAP is not supported. Remove old configuration data from the gWLM database Changes to workloads are stored in the gWLM database as configuration changes. The gWLM database will grow over time when workloads are being modified.
# gwlmcmsd --stop # gwlmcmsd Combining psets and virtual partitions When using psets on virtual partitions, assigning CPUs to virtual partitions by either path or cell specification can result in processes losing their processor set affiliations when CPUs are removed. Workaround Two workarounds are available: • Do not assign CPUs to virtual partitions by either path or cell specification.
Using the System Insight Manager interface: Select the menus Configure → Matrix OE Agents → Start gWLM Agent Missing or unexpected historical data (system clocks differ) You might have no historical data available for graphing, even though you are certain an SRD was deployed for the time period in question. A related issue occurs when you select a time period where you expect high system activity, but the graph shows limited activity.
Application hangs in fss group On HP-UX 11i v2 (B.11.23), an application inside an fss group might hang when running in a single-processor virtual partition, nPartition, or system. Workaround Install patch PHKL_33052. Scripts not placed in correct workloads With compartments based on psets or fss groups, gWLM allows you to place scripts in the compartments using application records with alternate names. This works only if the shell or interpreter being used is listed in the file /etc/shells.
cannot be calculated when there are not enough resources on the system to manage the set of monitored workloads in pset compartments. Workaround You can manage the workloads using compartments based on fss groups (which have a smaller minimum size), or add resources to the partition or SRD, to fulfill the pset minimum size requirements.
Workaround Perform the following steps:. 1. Log out of System Insight Manager 2. Log in to System Insight Manager (again) 3. Generate the historic report (again) Negative current size for NONVM If the CPUs on a VM Host are oversubscribed when you deploy an SRD on that host, gWLM shows current size for NONVM as a negative value. Workaround Two options are available: • Adjust the entitlements of those virtual machines that are on so that the CPUs are not oversubscribed.
Index security, 27 set up, 18 startup behavior, 35 support, 45 tabs and menus, 15 wizard, 15 A advisory mode, 15 advisory mode to managed mode change, 19 automatic restart gWLM managed nodes in SRD, 36 C communication ports, 35 compartment, 7 compatibility with agents, 47 conditional policy, 13 configuration, 18 CPU resources manual adjustment, 29 custom policy, 13 H D K database backup and restore, 31 deploy, 8 known issues, 48 F memory resources manual adjustment, 30 message logs, 25 messages inc
node failed to rejoin SRD, 37 stop managing, 22 startup behavior, 35 support, 45 T tabs and menus, 15 typographic conventions, 46 U unable to create new native thread, 44 undeploy, 8 utilization policy, 13 V virtual partition, 9 W wizard, 15 workload, 7 disk space requirement, 30 monitoring, 24 stop managing, 22 61
