vSphere Resource Management 17 APR 2018 VMware vSphere 6.7 VMware ESXi 6.7 vCenter Server 6.
vSphere Resource Management You can find the most up-to-date technical documentation on the VMware website at: https://docs.vmware.com/ If you have comments about this documentation, submit your feedback to docfeedback@vmware.com VMware, Inc. 3401 Hillview Ave. Palo Alto, CA 94304 www.vmware.com Copyright © 2006–2018 VMware, Inc. All rights reserved. Copyright and trademark information. VMware, Inc.
Contents About vSphere Resource Management 7 1 Getting Started with Resource Management 9 Resource Types 9 Resource Providers 9 Resource Consumers 10 Goals of Resource Management 10 2 Configuring Resource Allocation Settings 11 Resource Allocation Shares 11 Resource Allocation Reservation Resource Allocation Limit 12 13 Resource Allocation Settings Suggestions Edit Settings 13 14 Changing Resource Allocation Settings—Example Admission Control 14 15 3 CPU Virtualization Basics 17 Softwar
vSphere Resource Management 6 Administering Memory Resources 33 Understanding Memory Overhead 33 How ESXi Hosts Allocate Memory 35 Memory Reclamation Using Swap Files 36 37 Sharing Memory Across Virtual Machines Memory Compression 42 42 Measuring and Differentiating Types of Memory Usage Memory Reliability 43 45 About System Swap 45 7 Persistent Memory 47 8 Configuring Virtual Graphics 49 View GPU Statistics 49 Add an NVIDIA GRID vGPU to a Virtual Machine Configuring Host Graphics 50 50
vSphere Resource Management Configuring DRS with Virtual Flash Create a Cluster 75 76 Edit Cluster Settings 76 Set a Custom Automation Level for a Virtual Machine Disable DRS 78 79 Restore a Resource Pool Tree 79 12 Using DRS Clusters to Manage Resources 81 Adding Hosts to a Cluster 81 Adding Virtual Machines to a Cluster 83 Removing Virtual Machines from a Cluster Removing a Host from a Cluster DRS Cluster Validity 83 84 86 Managing Power Resources Using DRS Affinity Rules 91 96 13 Cr
vSphere Resource Management 16 Advanced Attributes 128 Set Advanced Host Attributes 128 Set Advanced Virtual Machine Attributes Latency Sensitivity 131 133 About Reliable Memory 134 Backing Guest vRAM with 1GB Pages 134 17 Fault Definitions 136 Virtual Machine is Pinned 137 Virtual Machine not Compatible with any Host 137 VM/VM DRS Rule Violated when Moving to another Host Host Incompatible with Virtual Machine 137 137 Host Has Virtual Machine That Violates VM/VM DRS Rules Host has Insuffic
About vSphere Resource Management ® vSphere Resource Management describes resource management for VMware ESXi and vCenter Server environments. ® This documentation focuses on the following topics.
vSphere Resource Management For the workflows that significantly differ between the two clients, there are duplicate procedures. The procedures indicate when they are intended exclusively for the vSphere Client or the vSphere Web Client. Note In vSphere 6.7, most of the vSphere Web Client functionality is implemented in the vSphere Client. For an up-to-date list of the unsupported functionality, see Functionality Updates for the vSphere Client. VMware, Inc.
Getting Started with Resource Management 1 To understand resource management, you must be aware of its components, its goals, and how best to implement it in a cluster setting. Resource allocation settings for a virtual machine (shares, reservation, and limit) are discussed, including how to set them and how to view them. Also, admission control, the process whereby resource allocation settings are validated against existing resources is explained.
vSphere Resource Management A cluster is a group of hosts. You can create a cluster using vSphere Client, and add multiple hosts to the cluster. vCenter Server manages these hosts’ resources jointly: the cluster owns all of the CPU and memory of all hosts. You can enable the cluster for joint load balancing or failover. See Chapter 11 Creating a DRS Cluster for more information. A datastore cluster is a group of datastores.
Configuring Resource Allocation Settings 2 When available resource capacity does not meet the demands of the resource consumers (and virtualization overhead), administrators might need to customize the amount of resources that are allocated to virtual machines or to the resource pools in which they reside. Use the resource allocation settings (shares, reservation, and limit) to determine the amount of CPU, memory, and storage resources provided for a virtual machine.
vSphere Resource Management Specifying shares makes sense only with regard to sibling virtual machines or resource pools, that is, virtual machines or resource pools with the same parent in the resource pool hierarchy. Siblings share resources according to their relative share values, bounded by the reservation and limit. When you assign shares to a virtual machine, you always specify the priority for that virtual machine relative to other powered-on virtual machines.
vSphere Resource Management For example, assume you have 2GHz available and specify a reservation of 1GHz for VM1 and 1GHz for VM2. Now each virtual machine is guaranteed to get 1GHz if it needs it. However, if VM1 is using only 500MHz, VM2 can use 1.5GHz. Reservation defaults to 0. You can specify a reservation if you need to guarantee that the minimum required amounts of CPU or memory are always available for the virtual machine.
vSphere Resource Management n If you expect frequent changes to the total available resources, use Shares to allocate resources fairly across virtual machines. If you use Shares, and you upgrade the host, for example, each virtual machine stays at the same priority (keeps the same number of shares) even though each share represents a larger amount of memory, CPU, or storage I/O resources. Edit Settings Use the Edit Settings dialog box to change allocations for memory and CPU resources.
vSphere Resource Management Figure 2‑1. Single Host with Two Virtual Machines host VM-QA VM-Marketing In the following example, assume that VM-QA is memory intensive and accordingly you want to change the resource allocation settings for the two virtual machines to: n Specify that, when system memory is overcommitted, VM-QA can use twice as much CPU and memory resources as the Marketing virtual machine. Set the CPU shares and memory shares for VM-QA to High and for VM-Marketing set them to Normal.
vSphere Resource Management When the vSphere DPM feature is enabled, hosts might be placed in standby mode (that is, powered off) to reduce power consumption. The unreserved resources provided by these hosts are considered available for admission control. If a virtual machine cannot be powered on without these resources, a recommendation to power on sufficient standby hosts is made. VMware, Inc.
CPU Virtualization Basics 3 CPU virtualization emphasizes performance and runs directly on the processor whenever possible. The underlying physical resources are used whenever possible and the virtualization layer runs instructions only as needed to make virtual machines operate as if they were running directly on a physical machine. CPU virtualization is not the same thing as emulation. ESXi does not use emulation to run virtual CPUs. With emulation, all operations are run in software by an emulator.
vSphere Resource Management Hardware-Assisted CPU Virtualization Certain processors provide hardware assistance for CPU virtualization. When using this assistance, the guest can use a separate mode of execution called guest mode. The guest code, whether application code or privileged code, runs in the guest mode. On certain events, the processor exits out of guest mode and enters root mode.
vSphere Resource Management ESXi supports up to 128 virtual processors (CPUs) for each virtual machine. Note Deploy single-threaded applications on uniprocessor virtual machines, instead of on SMP virtual machines that have multiple CPUs, for the best performance and resource use. Single-threaded applications can take advantage only of a single CPU. Deploying such applications in dual-processor virtual machines does not speed up the application.
Administering CPU Resources 4 You can configure virtual machines with one or more virtual processors, each with its own set of registers and control structures. When a virtual machine is scheduled, its virtual processors are scheduled to run on physical processors. The VMkernel Resource Manager schedules the virtual CPUs on physical CPUs, thereby managing the virtual machine’s access to physical CPU resources. ESXi supports virtual machines with up to 128 virtual CPUs.
vSphere Resource Management Specifying CPU Configuration You can specify CPU configuration to improve resource management. However, if you do not customize CPU configuration, the ESXi host uses defaults that work well in most situations. You can specify CPU configuration in the following ways: n Use the attributes and special features available through the vSphere Client. The vSphere Client allows you to connect to the ESXi host or a vCenter Server system.
vSphere Resource Management Hyperthreading Hyperthreading technology allows a single physical processor core to behave like two logical processors. The processor can run two independent applications at the same time. To avoid confusion between logical and physical processors, Intel refers to a physical processor as a socket, and the discussion in this chapter uses that terminology as well.
vSphere Resource Management If there is no work for a logical processor, it is put into a halted state, which frees its execution resources and allows the virtual machine running on the other logical processor on the same core to use the full execution resources of the core. The VMware scheduler properly accounts for this halt time, and charges a virtual machine running with the full resources of a core more than a virtual machine running on a half core.
vSphere Resource Management In this context, the term CPU refers to a logical processor on a hyperthreaded system and refers to a core on a non-hyperthreaded system. The CPU affinity setting for a virtual machine applies to all of the virtual CPUs associated with the virtual machine and to all other threads (also known as worlds) associated with the virtual machine. Such virtual machine threads perform processing required for emulating mouse, keyboard, screen, CD-ROM, and miscellaneous legacy devices.
vSphere Resource Management n Affinity can interfere with the ESXi host’s ability to meet the reservation and shares specified for a virtual machine. n Because CPU admission control does not consider affinity, a virtual machine with manual affinity settings might not always receive its full reservation. Virtual machines that do not have manual affinity settings are not adversely affected by virtual machines with manual affinity settings.
vSphere Resource Management Select a CPU Power Management Policy You set the CPU power management policy for a host using the vSphere Client. Prerequisites Verify that the BIOS settings on the host system allow the operating system to control power management (for example, OS Controlled). Note Some systems have Processor Clocking Control (PCC) technology, which allows ESXi to manage power on the host system even if the host BIOS settings do not specify OS Controlled mode.
vSphere Resource Management 5 Select the parameter and click the Edit button. Note The default values of power management parameters match the Balanced policy. 6 Parameter Description Power.UsePStates Use ACPI P-states to save power when the processor is busy. Power.MaxCpuLoad Use P-states to save power on a CPU only when the CPU is busy for less than the given percentage of real time. Power.MinFreqPct Do not use any P-states slower than the given percentage of full CPU speed. Power.
Memory Virtualization Basics 5 Before you manage memory resources, you should understand how they are being virtualized and used by ESXi. The VMkernel manages all physical RAM on the host. The VMkernel dedicates part of this managed physical RAM for its own use. The rest is available for use by virtual machines. The virtual and physical memory space is divided into blocks called pages. When physical memory is full, the data for virtual pages that are not present in physical memory are stored on disk.
vSphere Resource Management For example, consider a virtual machine with a configured size of 1GB. When the guest operating system boots, it detects that it is running on a dedicated machine with 1GB of physical memory. In some cases, the virtual machine might be allocated the full 1GB. In other cases, it might receive a smaller allocation. Regardless of the actual allocation, the guest operating system continues to behave as though it is running on a dedicated machine with 1GB of physical memory.
vSphere Resource Management To improve memory utilization, the ESXi host transfers memory from idle virtual machines to virtual machines that need more memory. Use the Reservation or Shares parameter to preferentially allocate memory to important virtual machines. This memory remains available to other virtual machines if it is not in use.
vSphere Resource Management The VMM for each virtual machine maintains a mapping from the guest operating system's physical memory pages to the physical memory pages on the underlying machine. (VMware refers to the underlying host physical pages as “machine” pages and the guest operating system’s physical pages as “physical” pages.) Each virtual machine sees a contiguous, zero-based, addressable physical memory space.
vSphere Resource Management The first layer of page tables stores guest virtual-to-physical translations, while the second layer of page tables stores guest physical-to-machine translation. The TLB (translation look-aside buffer) is a cache of translations maintained by the processor's memory management unit (MMU) hardware. A TLB miss is a miss in this cache and the hardware needs to go to memory (possibly many times) to find the required translation.
Administering Memory Resources 6 Using the vSphere Client you can view information about and make changes to memory allocation settings. To administer your memory resources effectively, you must also be familiar with memory overhead, idle memory tax, and how ESXi hosts reclaim memory. When administering memory resources, you can specify memory allocation. If you do not customize memory allocation, the ESXi host uses defaults that work well in most situations.
vSphere Resource Management ESXi virtual machines can incur two kinds of memory overhead. n The additional time to access memory within a virtual machine. n The extra space needed by the ESXi host for its own code and data structures, beyond the memory allocated to each virtual machine. ESXi memory virtualization adds little time overhead to memory accesses.
vSphere Resource Management How ESXi Hosts Allocate Memory A host allocates the memory specified by the Limit parameter to each virtual machine, unless memory is overcommitted. ESXi never allocates more memory to a virtual machine than its specified physical memory size. For example, a 1GB virtual machine might have the default limit (unlimited) or a user-specified limit (for example 2GB). In both cases, the ESXi host never allocates more than 1GB, the physical memory size that was specified for it.
vSphere Resource Management VMX Swap Files Virtual machine executable (VMX) swap files allow the host to greatly reduce the amount of overhead memory reserved for the VMX process. Note VMX swap files are not related to the swap to host swap cache feature or to regular host-level swap files. ESXi reserves memory per virtual machine for a variety of purposes.
vSphere Resource Management Figure 6‑1. Memory Ballooning in the Guest Operating System 1 memory 2 swap space memory 3 memory swap space Note You must configure the guest operating system with sufficient swap space. Some guest operating systems have additional limitations. If necessary, you can limit the amount of memory vmmemctl reclaims by setting the sched.mem.maxmemctl parameter for a specific virtual machine.
vSphere Resource Management Swap File Location By default, the swap file is created in the same location as the virtual machine's configuration file, which may either be on a VMFS datastore, a vSAN datastore or a VVol datastore. On a vSAN datastore or a VVol datastore, the swap file is created as a separate vSAN or VVol object. The ESXi host creates a swap file when a virtual machine is powered on. If this file cannot be created, the virtual machine cannot power on.
vSphere Resource Management 2 Click Configure. 3 Under Virtual Machines, select Swap file location. 4 Click Edit and select Selected Datastore. 5 Select a local datastore from the list and click OK. Host-local swap is now enabled for the standalone host. Swap Space and Memory Overcommitment You must reserve swap space for any unreserved virtual machine memory (the difference between the reservation and the configured memory size) on per-virtual machine swap files.
vSphere Resource Management When you create a large swap file (for example, larger than 100GB), the amount of time it takes for the virtual machine to power on can increase significantly. To avoid this, set a high reservation for large virtual machines. You can also place swap files on less costly storage using host-local swap files.
vSphere Resource Management 7 Click OK. The virtual machine swapfile is stored in the location you selected. Configure a Virtual Machine Swap File Location for a Cluster By default, swap files for a virtual machine are on a datastore in the folder that contains the other virtual machine files. However, you can instead configure the hosts in your cluster to place virtual machine swap files on an alternative datastore of your choice.
vSphere Resource Management Sharing Memory Across Virtual Machines Many ESXi workloads present opportunities for sharing memory across virtual machines (as well as within a single virtual machine). ESXi memory sharing runs as a background activity that scans for sharing opportunities over time. The amount of memory saved varies over time. For a fairly constant workload, the amount generally increases slowly until all sharing opportunities are exploited.
vSphere Resource Management Enable or Disable the Memory Compression Cache Memory compression is enabled by default. You can use Advanced System Settings in the vSphere Client to enable or disable memory compression for a host. Procedure 1 Browse to the host in the vSphere Client. 2 Click Configure. 3 Under System, select Advanced System Settings. 4 Locate Mem.MemZipEnable and click the Edit button. 5 Enter 1 to enable or enter 0 to disable the memory compression cache. 6 Click OK.
vSphere Resource Management Some of these memory metrics measure guest physical memory while other metrics measure machine memory. For instance, two types of memory usage that you can examine using performance metrics are guest physical memory and machine memory. You measure guest physical memory using the Memory Granted metric (for a virtual machine) or Memory Shared (for a host). To measure machine memory, however, use Memory Consumed (for a virtual machine) or Memory Shared Common (for a host).
vSphere Resource Management A similar result is obtained when determining Memory Shared and Memory Shared Common for the host. n Memory Shared for the host is the sum of each virtual machine's Memory Shared. Calculate shared memory by looking at each virtual machine's guest physical memory and counting the number of blocks that have arrows to machine memory blocks that themselves have more than one arrow pointing at them. There are six such blocks in the example, so Memory Shared for the host is 24 KB.
vSphere Resource Management System swap allows the system to reclaim memory from memory consumers that are not virtual machines. When system swap is enabled you have a tradeoff between the impact of reclaiming the memory from another process and the ability to assign the memory to a virtual machine that can use it. The amount of space required for the system swap is 1GB. Memory is reclaimed by taking data out of memory and writing it to background storage.
Persistent Memory 7 Persistent Memory (PMem), also known as Non-Volatile Memory (NVM), is capable of maintaining data even after a power outage. PMem can be used by applications that are sensitive to downtime and require high performance. VMs can be configured to use PMem on a standalone host, or in a cluster. PMem is treated as a local datastore. Persistent memory significantly reduces storage latency.
vSphere Resource Management Name spaces Name spaces for PMem are configured before ESXi starts. Name spaces are similar to disks on the system. ESXi reads name spaces and combines multiple name spaces into one logical volume by writing GPT headers. This is formatted automatically by default, if you have not previously configured it. If it has already been formatted, ESXi attempts to mount the PMem. GPT tables If the data in PMem storage is corrupted it can cause ESXi to fail.
Configuring Virtual Graphics 8 You can edit graphics settings for supported graphics implementations. vSphere supports multiple graphics implementations. n VMware supports 3d graphics solutions from AMD, Intel and NVIDIA. n NVIDIA GRID support. n Allows single NVIDIA vib to support both vSGA and vGPU implementations. n Provides vCenter GPU performance charts for Intel and NVIDIA. n Enables graphics for Horizon View VDI desktops.
vSphere Resource Management Add an NVIDIA GRID vGPU to a Virtual Machine If an ESXi host has an NVIDIA GRID GPU graphics device, you can configure a virtual machine to use the NVIDIA GRID virtual GPU (vGPU) technology. NVIDIA GRID GPU graphics devices are designed to optimize complex graphics operations and enable them to run at high performance without overloading the CPU. Prerequisites n Verify that an NVIDIA GRID GPU graphics device with an appropriate driver is installed on the host.
vSphere Resource Management 3 4 5 In the Edit Host Graphics Settings window, select: Option Description Shared VMware shared virtual graphics Shared Direct Vendor shared passthrough graphics Select a shared passthrough GPU assignment policy. a Spread VMs across GPUs (best performance) b Group VMs on GPU until full (GPU Consolidation) Click OK. What to do next After clicking OK, you must restart Xorg on the host. Configuring Graphics Devices You can edit graphics type for a video card.
Managing Storage I/O Resources 9 vSphere Storage I/O Control allows cluster-wide storage I/O prioritization, which allows better workload consolidation and helps reduce extra costs associated with over provisioning. Storage I/O Control extends the constructs of shares and limits to handle storage I/O resources.
vSphere Resource Management n Storage I/O Control Resource Shares and Limits n Set Storage I/O Control Resource Shares and Limits n Enable Storage I/O Control n Set Storage I/O Control Threshold Value n Storage DRS Integration with Storage Profiles About Virtual Machine Storage Policies Virtual machine storage policies are essential to virtual machine provisioning.
vSphere Resource Management Storage I/O Control Requirements Storage I/O Control has several requirements and limitations. n Datastores that are Storage I/O Control-enabled must be managed by a single vCenter Server system. n Storage I/O Control is supported on Fibre Channel-connected, iSCSI-connected, and NFS-connected storage. Raw Device Mapping (RDM) is not supported. n Storage I/O Control does not support datastores with multiple extents.
vSphere Resource Management 2 Click the VMs tab. The tab displays each virtual machine running on the datastore and the associated shares value, and percentage of datastore shares. Monitor Storage I/O Control Shares Use the datastore Performance tab to monitor how Storage I/O Control handles the I/O workloads of the virtual machines accessing a datastore based on their shares.
vSphere Resource Management 3 Click the Virtual Hardware tab and select a virtual hard disk from the list. Expand Hard disk. 4 Select a VM storage policy from the drop-down menu. If you select a storage policy, do not manually configure Shares and Limit - IOPS. 5 Under Shares, click the drop-down menu and select the relative amount of shares to allocate to the virtual machine (Low, Normal, or High). You can select Custom to enter a user-defined shares value.
vSphere Resource Management If you change the congestion threshold setting, set the value based on the following considerations. n A higher value typically results in higher aggregate throughput and weaker isolation. Throttling will not occur unless the overall average latency is higher than the threshold. n If throughput is more critical than latency, do not set the value too low. For example, for Fibre Channel disks, a value below 20ms could lower peak disk throughput.
vSphere Resource Management Storage DRS Integration with Storage Profiles Storage Policy Based Management (SPBM) allows you to specify the policy for a virtual machine which is enforced by Storage DRS. A datastore cluster can have set of datastores with different capability profiles. If the virtual machines have storage profiles associated with them, Storage DRS can enforce placement based on underlying datastore capabilities.
Managing Resource Pools 10 A resource pool is a logical abstraction for flexible management of resources. Resource pools can be grouped into hierarchies and used to hierarchically partition available CPU and memory resources. Each standalone host and each DRS cluster has an (invisible) root resource pool that groups the resources of that host or cluster. The root resource pool does not appear because the resources of the host (or cluster) and the root resource pool are always the same.
vSphere Resource Management n Add a Virtual Machine to a Resource Pool n Remove a Virtual Machine from a Resource Pool n Remove a Resource Pool n Resource Pool Admission Control Why Use Resource Pools? Resource pools allow you to delegate control over resources of a host (or a cluster), but the benefits are evident when you use resource pools to compartmentalize all resources in a cluster. Create multiple resource pools as direct children of the host or cluster and configure them.
vSphere Resource Management The numbers in the following figure show the effective allocations to the resource pools. VMware, Inc.
vSphere Resource Management Figure 10‑2. Allocating Resources to Resource Pools Host 6GHz, 6GB RP-Marketing 2GHz, 2GB RP-QA 4GHz, 4GB QA 1 VMware, Inc.
vSphere Resource Management Create a Resource Pool You can create a child resource pool of any ESXi host, resource pool, or DRS cluster. Note If a host has been added to a cluster, you cannot create child resource pools of that host. If the cluster is enabled for DRS, you can create child resource pools of the cluster. When you create a child resource pool, you are prompted for resource pool attribute information.
vSphere Resource Management After you create a resource pool, you can add virtual machines to it. A virtual machine’s shares are relative to other virtual machines (or resource pools) with the same parent resource pool. Example: Creating Resource Pools Assume that you have a host that provides 6GHz of CPU and 3GB of memory that must be shared between your marketing and QA departments. You also want to share the resources unevenly, giving one department (QA) a higher priority.
vSphere Resource Management n If the virtual machine’s shares are high, medium, or low, %Shares adjusts to reflect the total number of shares in use in the new resource pool. n If the virtual machine has custom shares assigned, the share value is maintained.
vSphere Resource Management When you remove a virtual machine from a resource pool, the total number of shares associated with the resource pool decreases, so that each remaining share represents more resources. For example, assume you have a pool that is entitled to 6GHz, containing three virtual machines with shares set to Normal. Assuming the virtual machines are CPU-bound, each gets an equal allocation of 2GHz.
vSphere Resource Management Table 10‑1. Reservation Types Reservation Type Description Fixed The system checks whether the selected resource pool has sufficient unreserved resources. If it does, the action can be performed. If it does not, a message appears and the action cannot be performed. Expandable The system considers the resources available in the selected resource pool and its direct parent resource pool.
vSphere Resource Management n No local resources are available for VM-K2, so it borrows resources from the parent resource pool, RP-MOM. RP-MOM has 6GHz minus 1GHz (reserved by the virtual machine) minus 2GHz (reserved by RP-KID), which leaves 3GHz unreserved. With 3GHz available, you can power on the 2GHz virtual machine. Figure 10‑3.
11 Creating a DRS Cluster A cluster is a collection of ESXi hosts and associated virtual machines with shared resources and a shared management interface. Before you can obtain the benefits of cluster-level resource management you must create a cluster and enable DRS. Depending on whether or not Enhanced vMotion Compatibility (EVC) is enabled, DRS behaves differently when you use vSphere Fault Tolerance (vSphere FT) virtual machines in your cluster. Table 11‑1.
vSphere Resource Management If the cluster does not have sufficient resources to power on a single virtual machine, or any of the virtual machines in a group power-on attempt, a message appears. Otherwise, for each virtual machine, DRS generates a recommendation of a host on which to run the virtual machine and takes one of the following actions n Automatically executes the placement recommendation. n Displays the placement recommendation, which the user can then choose to accept or override.
vSphere Resource Management For each DRS cluster that the virtual machines being powered on belong to, there is a single recommendation, which contains all the prerequisites (or no recommendation). All such cluster-specific recommendations are presented together under the Power On Recommendations tab.
vSphere Resource Management Figure 11‑1. Load Balancing VM1 VM2 VM3 VM4 VM5 VM6 VM1 Host 1 VM7 VM7 VM9 Host 3 VM3 Host 1 Host 2 VM8 VM2 VM4 VM5 Host 2 VM8 VM9 VM6 Host 3 When a cluster becomes unbalanced, DRS makes recommendations or migrates virtual machines, depending on the default automation level: n If the cluster or any of the virtual machines involved are manual or partially automated, vCenter Server does not take automatic actions to balance resources.
vSphere Resource Management You can move the threshold slider to use one of five settings, ranging from Conservative to Aggressive. The five migration settings generate recommendations based on their assigned priority level. Each setting you move the slider to the right allows the inclusion of one more lower level of priority.
vSphere Resource Management Ensure that the managed hosts use shared storage. Shared storage is typically on a SAN, but can also be implemented using NAS shared storage. See the vSphere Storage documentation for information about other shared storage. Shared VMFS Volume Requirements A DRS cluster has certain shared VMFS volume requirements. Configure all managed hosts to use shared VMFS volumes. n Place the disks of all virtual machines on VMFS volumes that are accessible by source and destination hosts.
vSphere Resource Management vCenter Server provides features that help ensure that virtual machines migrated with vMotion meet processor compatibility requirements. These features include: n Enhanced vMotion Compatibility (EVC) – You can use EVC to help ensure vMotion compatibility for the hosts in a cluster. EVC ensures that all hosts in a cluster present the same CPU feature set to virtual machines, even if the actual CPUs on the hosts differ.
vSphere Resource Management Create a Cluster A cluster is a group of hosts. When a host is added to a cluster, the host's resources become part of the cluster's resources. The cluster manages the resources of all hosts within it. Clusters enable the vSphere High Availability (HA) and vSphere Distributed Resource Scheduler (DRS) solutions. Prerequisites n Verify that you have sufficient permissions to create a cluster object. n Verify that a data center exists in the inventory.
vSphere Resource Management The following cluster-level resource management capabilities are also available. Load Balancing The distribution and usage of CPU and memory resources for all hosts and virtual machines in the cluster are continuously monitored. DRS compares these metrics to an ideal resource usage given the attributes of the cluster’s resource pools and virtual machines, the current demand, and the imbalance target.
vSphere Resource Management 6 Select the Predictive DRS check box. In addition to real-time metrics, DRS responds to forecasted metrics provided by vRealize Operations server. You must also configure Predictive DRS in a version of vRealize Operations that supports this feature. 7 Select Virtual Machine Automation check box to enable individual virtual machine automation levels. Override for individual virtual machines can be set from the VM Overrides page.
vSphere Resource Management 7 Select one or more virtual machines. 8 Click the Automation Level column and select an automation level from the drop-down menu. 9 Option Description Manual Placement and migration recommendations are displayed, but do not run until you manually apply the recommendation. Fully Automated Placement and migration recommendations run automatically. Partially Automated Initial placement is performed automatically. Migration recommendations are displayed, but do not run.
vSphere Resource Management Prerequisites n vSphere DRS must be turned ON. n You can restore a snapshot only on the same cluster that it was taken. n No other resource pools are present in the cluster. Procedure 1 Browse to the cluster in the vSphere Client. 2 Right-click on the cluster and select Restore Resource Pool Tree. 3 Click Browse, and locate the snapshot file on your local machine. 4 Click Open. 5 Click OK to restore the resource pool tree. VMware, Inc.
Using DRS Clusters to Manage Resources 12 After you create a DRS cluster, you can customize it and use it to manage resources. To customize your DRS cluster and the resources it contains you can configure affinity rules and you can add and remove hosts and virtual machines. When a cluster’s settings and resources have been defined, you should ensure that it is and remains a valid cluster. You can also use a valid DRS cluster to manage power resources and interoperate with vSphere HA.
vSphere Resource Management You can decide whether you want to associate existing virtual machines and resource pools with the cluster’s root resource pool or graft the resource pool hierarchy. Note If a host has no child resource pools or virtual machines, the host’s resources are added to the cluster but no resource pool hierarchy with a top-level resource pool is created. Procedure 1 Browse to the host in the vSphere Client. 2 Right-click the host and select Move To.... 3 Select a cluster.
vSphere Resource Management 7 Select what to do with the host’s virtual machines and resource pools. n Put this host’s virtual machines in the cluster’s root resource pool vCenter Server removes all existing resource pools of the host and the virtual machines in the host’s hierarchy are all attached to the root. Because share allocations are relative to a resource pool, you might have to manually change a virtual machine’s shares after selecting this option, which destroys the resource pool hierarchy.
vSphere Resource Management You can remove a virtual machine from a cluster in two ways. n When you remove a host from a cluster, all of the powered-off virtual machines that you do not migrate to other hosts are removed as well. You can remove a host only if it is in maintenance mode or disconnected. If you remove a host from a DRS cluster, the cluster can become yellow because it is overcommitted.
vSphere Resource Management n Virtual Machines – A host must be in maintenance mode before you can remove it from the cluster and for a host to enter maintenance mode all powered-on virtual machines must be migrated off that host. When you request that a host enter maintenance mode, you are also asked whether you want to migrate all the powered-off virtual machines on that host to other hosts in the cluster.
vSphere Resource Management Procedure 1 Browse to the host in the vSphere Client. 2 Right-click the host and select Maintenance Mode > Enter Maintenance Mode. When the host is in maintenance mode, move it to a different inventory location, either the top-level data center or to a different cluster. 3 Right-click the host and select Move To.... 4 Select a new location for the lost and click OK. When you move the host, its resources are removed from the cluster.
vSphere Resource Management When considering cluster validity scenarios, you should understand these terms. Reservation A fixed, guaranteed allocation for the resource pool input by the user. Reservation Used The sum of the reservation or reservation used (whichever is larger) for each child resource pool, added recursively. Unreserved This nonnegative number differs according to resource pool type. n Nonexpandable resource pools: Reservation minus reservation used.
vSphere Resource Management n RP1 was created with a reservation of 4GHz. Two virtual machines. (VM1 and VM7) of 2GHz each are powered on (Reservation Used: 4GHz). No resources are left for powering on additional virtual machines. VM6 is shown as not powered on. It consumes none of the reservation. n RP2 was created with a reservation of 4GHz. Two virtual machines of 1GHz and 2GHz are powered on (Reservation Used: 3GHz). 1GHz remains unreserved. n RP3 was created with a reservation of 3GHz.
vSphere Resource Management n RP2 was created with a reservation of 5GHz. Two virtual machines of 1GHz and 2GHz are powered on (Reservation Used: 3GHz). 2GHz remains unreserved. RP3 was created with a reservation of 5GHz. Two virtual machines of 3GHz and 2GHz are powered on. Even though this resource pool is of type Expandable, no additional 2GHz virtual machine can be powered on because the parent’s extra resources are already used by RP1.
vSphere Resource Management n One of the 4GHz hosts becomes unavailable, so total resources reduce to 8GHz. n At the same time, VM4 (1GHz) and VM3 (3GHz), which were running on the host that failed, are no longer running. n The cluster is now running virtual machines that require a total of 6GHz. The cluster still has 8GHz available, which is sufficient to meet virtual machine requirements. The resource pool reservations of 12GHz can no longer be met, so the cluster is marked as yellow.
vSphere Resource Management Figure 12‑4.
vSphere Resource Management vSphere DPM can use one of three power management protocols to bring a host out of standby mode: Intelligent Platform Management Interface (IPMI), Hewlett-Packard Integrated Lights-Out (iLO), or WakeOn-LAN (WOL). Each protocol requires its own hardware support and configuration. If a host does not support any of these protocols it cannot be put into standby mode by vSphere DPM. If a host supports multiple protocols, they are used in the following order: IPMI, iLO, WOL.
vSphere Resource Management 6 n IP address of the NIC associated with the BMC, as distinct from the IP address of the host. The IP address should be static or a DHCP address with infinite lease. n MAC address of the NIC associated with the BMC. Click OK. Test Wake-on-LAN for vSphere DPM The use of Wake-on-LAN (WOL) for the vSphere DPM feature is fully supported, if you configure and successfully test it according to the VMware guidelines.
vSphere Resource Management 4 Observe whether or not the host successfully powers back on. 5 For any host that fails to exit standby mode successfully, perform the following steps. a Select the host in the vSphere Client and select the Configure tab. b Under Hardware > Power Management, click Edit to adjust the power management policy. After you do this, vSphere DPM does not consider that host a candidate for being powered off.
vSphere Resource Management The threshold is configured under Power Management in the cluster’s Settings dialog box. Each level you move the vSphere DPM Threshold slider to the right allows the inclusion of one more lower level of priority in the set of recommendations that are executed automatically or appear as recommendations to be manually executed.
vSphere Resource Management The most serious potential error you face when using vSphere DPM is the failure of a host to exit standby mode when its capacity is needed by the DRS cluster. You can monitor for instances when this error occurs by using the preconfigured Exit Standby Error alarm in vCenter Server.
vSphere Resource Management When you add or edit an affinity rule, and the cluster's current state is in violation of the rule, the system continues to operate and tries to correct the violation. For manual and partially automated DRS clusters, migration recommendations based on rule fulfillment and load balancing are presented for approval. You are not required to fulfill the rules, but the corresponding recommendations remain until the rules are fulfilled.
vSphere Resource Management 2 Click the Configure tab. 3 Under Configuration, select VM/Host Groups and click Add. 4 In the Create VM/Host Group dialog box, type a name for the group. 5 Select VM Group from the Type drop down box and click Add. 6 Click the check box next to a virtual machine to add it. Continue this process until all desired virtual machines have been added. 7 Click OK.
vSphere Resource Management 6 From the Type drop-down menu, select either Keep Virtual Machines Together or Separate Virtual Machines. 7 Click Add. 8 Select at least two virtual machines to which the rule will apply and click OK. 9 Click OK. VM-VM Affinity Rule Conflicts You can create and use multiple VM-VM affinity rules, however, this might lead to situations where the rules conflict with one another. If two VM-VM affinity rules are in conflict, you cannot enable both.
vSphere Resource Management Procedure 1 Browse to the cluster in the vSphere Client. 2 Click the Configure tab. 3 Under Configuration, click VM/Host Rules. 4 Click Add. 5 In the Create VM/Host Rule dialog box, type a name for the rule. 6 From the Type drop down menu, select Virtual Machines to Hosts. 7 Select the virtual machine DRS group and the host DRS group to which the rule applies. 8 Select a specification for the rule. 9 n Must run on hosts in group.
vSphere Resource Management When you create a VM-Host affinity rule, its ability to function in relation to other rules is not checked. So it is possible for you to create a rule that conflicts with the other rules you are using. When two VM-Host affinity rules conflict, the older one takes precedence and the newer rule is disabled. DRS only tries to satisfy enabled rules and disabled rules are ignored.
Creating a Datastore Cluster 13 A datastore cluster is a collection of datastores with shared resources and a shared management interface. Datastore clusters are to datastores what clusters are to hosts. When you create a datastore cluster, you can use vSphere Storage DRS to manage storage resources. Note Datastore clusters are referred to as storage pods in the vSphere API. When you add a datastore to a datastore cluster, the datastore's resources become part of the datastore cluster's resources.
vSphere Resource Management n Datastore Cluster Requirements n Adding and Removing Datastores from a Datastore Cluster Initial Placement and Ongoing Balancing Storage DRS provides initial placement and ongoing balancing recommendations to datastores in a Storage DRS-enabled datastore cluster. Initial placement occurs when Storage DRS selects a datastore within a datastore cluster on which to place a virtual machine disk.
vSphere Resource Management Create a Datastore Cluster You can manage datastore cluster resources using Storage DRS. Procedure 1 Browse to data centers in the vSphere Client. 2 Right-click the data center object and select New Datastore Cluster. 3 To complete the New Datastore Cluster wizard, follow the prompts. 4 Click Finish. Enable and Disable Storage DRS Storage DRS allows you to manage the aggregated resources of a datastore cluster.
vSphere Resource Management Set the Automation Level for Datastore Clusters The automation level for a datastore cluster specifies whether or not placement and migration recommendations from Storage DRS are applied automatically. Procedure 1 Browse to the datastore cluster in the vSphere Client. 2 Click the Configure tab and click Services. 3 Select DRS and click Edit. 4 Expand DRS Automation and select an automation level. Manual is the default automation level.
vSphere Resource Management You can also set advanced options to further configure the aggressiveness level of Storage DRS. Space utilization difference This threshold ensures that there is some minimum difference between the space utilization of the source and the destination. For example, if the space used on datastore A is 82% and datastore B is 79%, the difference is 3. If the threshold is 5, Storage DRS will not make migration recommendations from datastore A to datastore B.
vSphere Resource Management 3 (Optional) Configure advanced options. n No recommendations until utilization difference between source and destination is: Use the slider to specify the space utilization difference threshold. Utilization is usage * 100/capacity. This threshold ensures that there is some minimum difference between the space utilization of the source and the destination. For example, if the space used on datastore A is 82% and datastore B is 79%, the difference is 3.
vSphere Resource Management You can add to a datastore cluster any datastore that is mounted on a host in the vSphere Client inventory, with the following exceptions: n All hosts attached to the datastore must be ESXi 5.0 and later. n The datastore cannot be in more than one data center in the same instance of the vSphere Client. When you remove a datastore from a datastore cluster, the datastore remains in the vSphere Client inventory and is not unmounted from the host. VMware, Inc.
Using Datastore Clusters to Manage Storage Resources 14 After you create a datastore cluster, you can customize it and use it to manage storage I/O and space utilization resources.
vSphere Resource Management Prerequisites Storage DRS is enabled on the datastore cluster that contains the datastore that is entering maintenance mode. No CD-ROM image files are stored on the datastore. There are at least two datastores in the datastore cluster. Procedure 1 Browse to the datastore in the vSphere Client. 2 Right-click the datastore and select Maintenance Mode > Enter Maintenance Mode. A list of recommendations appears for datastore maintenance mode migration.
vSphere Resource Management 6 In the Value column, type 1 to enable the option. Type 0 to disable the option. 7 Click OK. The Ignore Affinity Rules for Maintenance Mode option is applied to the datastore cluster. Applying Storage DRS Recommendations Storage DRS collects resource usage information for all datastores in a datastore cluster. Storage DRS uses the information to generate recommendations for virtual machine disk placement on datastores in a datastore cluster.
vSphere Resource Management Procedure 1 In the vSphere Client datastore view, select the datastore cluster and click the Storage DRS tab. 2 Select the Recommendations view and click the Run Storage DRS link in the upper right corner. The recommendations are updated. The Last Updated timestamp displays the time when Storage DRS recommendations were refreshed.
vSphere Resource Management Prerequisites Enable Storage DRS. Procedure 1 Browse to the datastore cluster in the vSphere Client. 2 Click the Configure tab and click Services. 3 Under vSphere DRS click the Schedule DRS button. 4 In the Edit Datastore Cluster dialog box, click SDRS Scheduling. 5 Expand DRS Automation. a Select an automation level. b Set the Migration threshold.
vSphere Resource Management When you create an anti-affinity rule, it applies to the relevant virtual disks in the datastore cluster. Antiaffinity rules are enforced during initial placement and Storage DRS-recommendation migrations, but are not enforced when a migration is initiated by a user. Note Anti-affinity rules do not apply to CD-ROM ISO image files that are stored on a datastore in a datastore cluster, nor do they apply to swapfiles that are stored in user-defined locations.
vSphere Resource Management n Storage DRS migrates the virtual disks using vMotion according to the rule, even if the migration is for a mandatory reason such as putting a datastore in maintenance mode. n If the virtual machine's virtual disk violates the rule, Storage DRS makes migration recommendations to correct the error or reports the violation as a fault if it cannot make a recommendation that will correct the error. No inter-VM anti-affinity rules are defined by default.
vSphere Resource Management Procedure 1 Browse to the datastore cluster in the vSphere Client. 2 Click the Configure tab and click Configuration. 3 Select VM/Host Rules. 4 Click Add. 5 Type a name for the rule. 6 From the Type menu, select VMDK anti-affinity. 7 Click Add. 8 Click Select Virtual Machine. 9 Select a virtual machine and click OK. 10 Select at least two virtual disks to which the rule applies and click OK. 11 Click OK to save the rule.
vSphere Resource Management 5 Use the + button to select virtual machines. 6 Click the Keep VMDKs together drop-down menu and select No. 7 Click OK. Clear Storage DRS Statistics To diagnose problems with Storage DRS, you can clear Storage DRS statistics before you manually run Storage DRS. Important When you enable the option to clear Storage DRS statistics, statistics are cleared every time Storage DRS runs until you disable the option.
vSphere Resource Management n The destination datastore must have sufficient disk space. n The destination datastore must not be in maintenance mode or entering maintenance mode. VMware, Inc.
Using NUMA Systems with ESXi 15 ESXi supports memory access optimization for Intel and AMD Opteron processors in server architectures that support NUMA (non-uniform memory access). After you understand how ESXi NUMA scheduling is performed and how the VMware NUMA algorithms work, you can specify NUMA controls to optimize the performance of your virtual machines.
vSphere Resource Management Challenges for Operating Systems Because a NUMA architecture provides a single system image, it can often run an operating system with no special optimizations. The high latency of remote memory accesses can leave the processors under-utilized, constantly waiting for data to be transferred to the local node, and the NUMA connection can become a bottleneck for applications with high-memory bandwidth demands. Furthermore, performance on such a system can be highly variable.
vSphere Resource Management Some virtual machines are not managed by the ESXi NUMA scheduler. For example, if you manually set the processor or memory affinity for a virtual machine, the NUMA scheduler might not be able to manage this virtual machine. Virtual machines that are not managed by the NUMA scheduler still run correctly. However, they don't benefit from ESXi NUMA optimizations.
vSphere Resource Management Dynamic Load Balancing and Page Migration ESXi combines the traditional initial placement approach with a dynamic rebalancing algorithm. Periodically (every two seconds by default), the system examines the loads of the various nodes and determines if it should rebalance the load by moving a virtual machine from one node to another.
vSphere Resource Management Transparent page sharing for ESXi systems has also been optimized for use on NUMA systems. On NUMA systems, pages are shared per-node, so each NUMA node has its own local copy of heavily shared pages. When virtual machines use shared pages, they don't need to access remote memory. Note This default behavior is the same in all previous versions of ESX and ESXi.
vSphere Resource Management Virtual NUMA Controls For virtual machines with disproportionately large memory consumption, you can use advanced options to override the default virtual CPU settings. You can add these advanced options to the virtual machine configuration file. Table 15‑1. Advanced Options for Virtual NUMA Controls Option Description Default Value cpuid.coresPerSocket Determines the number of virtual cores per virtual CPU socket.
vSphere Resource Management You can specify the following options. NUMA Node Affinity When you set this option, NUMA can schedule a virtual machine only on the nodes specified in the affinity. CPU Affinity When you set this option, a virtual machine uses only the processors specified in the affinity. Memory Affinity When you set this option, the server allocates memory only on the specified nodes.
vSphere Resource Management Associate Memory Allocations with Specific NUMA Nodes Using Memory Affinity You can specify that all future memory allocations on a virtual machine use pages associated with specific NUMA nodes (also known as manual memory affinity). Note Specify nodes to be used for future memory allocations only if you have also specified CPU affinity. If you make manual changes only to the memory affinity settings, automatic NUMA rebalancing does not work properly.
vSphere Resource Management Associate Virtual Machines with Specified NUMA Nodes When you associate a NUMA node with a virtual machine to specify NUMA node affinity, you constrain the set of NUMA nodes on which NUMA can schedule a virtual machine's virtual CPU and memory. Note When you constrain NUMA node affinities, you might interfere with the ability of the ESXi NUMA scheduler to rebalance virtual machines across NUMA nodes for fairness.
Advanced Attributes 16 You can set advanced attributes for hosts or individual virtual machines to help you customize resource management. In most cases, adjusting the basic resource allocation settings (reservation, limit, shares) or accepting default settings results in appropriate resource allocation. However, you can use advanced attributes to customize resource management for a host or a specific virtual machine.
vSphere Resource Management Advanced Memory Attributes You can use the advanced memory attributes to customize memory resource usage. Table 16‑1. Advanced Memory Attributes Attribute Description Default Mem.ShareForceSalting Mem.ShareForceSalting 0: Inter-virtual machine Transparent Page Sharing (TPS) behavior is still retained. The value of VMX option sched.mem.pshare.salt is ignored even if present. 2 Mem.ShareForceSalting 1: By default the salt value is taken from sched.mem.pshare.salt.
vSphere Resource Management Table 16‑1. Advanced Memory Attributes (Continued) Attribute Description Default Mem.MemZipMaxPct Specifies the maximum size of the compression cache in terms of the maximum percentage of each virtual machine's memory that can be stored as compressed memory. 10 LPage.LPageDefragEnable Enables large page defragmentation. 0 = disable. 1 LPage.LPageDefragRateVM Maximum number of large page defragmentation attempts per second per virtual machine.
vSphere Resource Management Table 16‑2. Advanced NUMA Attributes (Continued) Attribute Description Default Numa.AutoMemAffinity Automatically set memory affinity for virtual machines that have CPU affinity set. 1 Numa.PageMigEnable Automatically migrate pages between NUMA nodes to improve memory locality. Page migration rates set manually are still in effect. 1 Set Advanced Virtual Machine Attributes You can set advanced attributes for a virtual machine.
vSphere Resource Management Table 16‑3. Advanced Virtual Machine Attributes (Continued) Attribute Description Default sched.mem.pshare.salt A salt value is a configurable VMX option for each virtual machine. If this option is not present in the virtual machine's VMX file, then the value of vc.uuid vmx user configurable option is taken as the default value. Since the vc.
vSphere Resource Management Table 16‑4. Advanced NUMA Attributes (Continued) Attribute Description Default numa.vcpu.min Minimum number of virtual CPUs in a virtual machine that are required in order to generate a virtual NUMA topology. 9 numa.vcpu.maxPerMachineNode Maximum number of virtual CPUs that belong to the same virtual machine that can be scheduled on a NUMA node at the same time. Use this attribute to ensure maximum bandwidth, by forcing different NUMA clients on different NUMA nodes.
vSphere Resource Management Procedure 1 Browse to the virtual machine in the vSphere Client. a To find a virtual machine, select a data center, folder, cluster, resource pool, or host. b Click the VMs tab. 2 Right-click the virtual machine and click Edit Settings. 3 Click VM Options and click Advanced. 4 Select a setting from the Latency Sensitivity drop-down menu. 5 Click OK. About Reliable Memory ESXi supports reliable memory.
vSphere Resource Management 1GB page vRAM backing is opportunistic and 1GB pages are allocated on a best effort basis. This includes cases where host CPUs do not have 1GB capabilities. To maximize the chances of having guest vRAM backed with 1GB pages, we recommended to start VMs requiring 1GB pages on a freshly booted host because over time the host RAM is fragmented. A VM with 1GB pages enabled can be migrated to a different host.
Fault Definitions 17 DRS faults indicate the reasons that prevent the generation of DRS actions (or the recommendation of those actions in manual mode). The DRS faults are defined within this section. Note In this chapter, "Memory" can refer to physical RAM or Persistent Memory.
vSphere Resource Management Virtual Machine is Pinned This fault occurs when DRS cannot move a virtual machine because DRS is disabled on it. That is, the virtual machine is "pinned" on its registered host. Virtual Machine not Compatible with any Host This fault occurs when DRS cannot find a host that can run the virtual machine.
vSphere Resource Management Host has Insufficient Capacity for Virtual Machine This fault occurs when the host does not have enough CPU or memory capacity for running the virtual machine. Host in Incorrect State This fault occurs when the host is entering maintenance or standby state when needed for DRS action to occur. To address this fault, cancel the request for the host to enter standby or maintenance mode.
vSphere Resource Management Insufficient Resources to Satisfy Configured Failover Level for HA This fault occurs when the HA configuration of CPU or memory resources reserved for failover is violated or is insufficient for the DRS operation. This fault is reported when: n The host is requested to enter maintenance or standby mode. n The virtual machine violates failover when it attempts to power on.
DRS Troubleshooting Information 18 ® This information describes vSphere Distributed Resource Scheduler (DRS) problems for particular categories: cluster, host, and virtual machine problems. Note In this chapter, "Memory" can refer to physical RAM or Persistent Memory. This chapter includes the following topics: n Cluster Problems n Host Problems n Virtual Machine Problems Cluster Problems Cluster problems can prevent DRS from performing optimally or from reporting faults.
vSphere Resource Management n Virtual machines are not compatible with the hosts to which DRS would move them. That is, at least one of the hosts in the cluster is incompatible for the virtual machines that would be migrated. For example, if host A's CPU is not vMotion-compatible with host B's CPU, then host A becomes incompatible for powered-on virtual machines running on host B.
vSphere Resource Management Cluster Is Red Because Failover Capacity Is Violated A DRS cluster becomes red when it is invalid. It may become red because failover capacity is violated. Problem The cluster attempts to fail over virtual machines if there is host failure, but is not guaranteed to have enough resources available to fail over all virtual machines covered by the failover requirements.
vSphere Resource Management Hosts Are Powered-off When Total Cluster Load Is High Hosts are powered off when total cluster load is high. Problem DRS determined that virtual machines can be run on fewer hosts without degrading the host or virtual machine performance. DRS is also constrained from moving the virtual machines running on the highly utilized hosts to the hosts scheduled for power-off. Cause The total cluster load is too high. Solution Reduce the cluster load.
vSphere Resource Management n Power management. Solution Address the issues that are causing DRS to avoid performing vMotion migrations. Host Problems Host problems might cause DRS to not perform as expected. DRS Recommends Host Be Powered on to Increase Capacity When Total Cluster Load Is Low The host must be powered on to help provide more capacity for the cluster or help hosts that are overloaded.
vSphere Resource Management n No virtual machines on highly used hosts are moved to that host. n DPM is disabled on the host because of a user setting or host previously failing to exit standby. Solution Address that issue that prevents DRS from powering on the host. Total Cluster Load Is Low The total cluster load is low. Problem When the total cluster load is low, DRS does not power off the host.
vSphere Resource Management Solution There is no solution. If appropriate, disable vSphere HA before you attempt to put the host into maintenance mode or standby mode. DRS Does Not Move Any Virtual Machines onto a Host DRS does not move any virtual machines onto a host. Problem DRS does not recommend migration of virtual machine to a host that has been added to a DRS-enabled cluster.
vSphere Resource Management Cause This may be because of problems with vMotion, DRS, or host compatibility. The following are the possible reasons: n vMotion is not configured or enabled on this host. n DRS is disabled for the virtual machines on this host. n Virtual machines on this host are not compatible with any other hosts. n No other hosts have sufficient resources for any virtual machines on this host.
vSphere Resource Management Cluster is Overloaded The cluster on which the virtual machine is running might have insufficient resources. Also, the virtual machine's share value is such that other virtual machines are granted proportionally more of the resources. To determine the demand is larger than the capacity, check the cluster statistics. Host is Overloaded To determine if the host's resources are oversubscribed, check the host statistics.
vSphere Resource Management Cause VM/VM DRS rules specify that selected virtual machines should be placed on the same host (affinity) or that virtual machines be placed on different hosts (anti-affinity). VM/Host DRS rules specify that selected virtual machines should be placed on specified hosts (affinity) or that selected virtual machines should not be placed on specified hosts (anti-affinity).
vSphere Resource Management Problem When you power on a virtual machine, DRS does not migrate it as expected when there are not enough resources on the host where the virtual machine is registered. Cause The following are possible reasons why DRS does not move the virtual machine. n DRS is disabled on the virtual machine. n The virtual machine has a device mounted. n The virtual machine is not compatible with any other hosts.
