Intel® Cloud Builders Guide Intel® Xeon® Processor-based Servers Dell* OpenManage* Power Center Intel® Cloud Builders Guide to Cloud Design and Deployment on Intel® Platforms Dell* OpenManage* Power Center AUDIENCE AND PURPOSE Intel® Xeon® Processor E5 Family This reference architecture outlines the usage of energy management and thermal technologies as part of planning, provisioning, and optimizing strategies in enterprise and cloud data centers to reduce energy cost and address constrained power situa
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Table of Contents Executive Summary......................................................................................................................................................................................................................................................... 4 Introduction........................................................................................................................................................
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Dell OpenManage Power Center.......................................................................................................................................................................................................................... 38 References...............................................................................................................................................................................................
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Executive Summary The evolution of cloud computing has resulted in highly efficient and carefully optimized data centers with increased server density and capacity that makes considerations on energy consumption and utilization extremely critical along with several other factors that were not as significant in smaller data centers of the past.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center a real-time basis. Enterprise computing is very demanding and is increasing rack density to new levels. Power and thermal monitoring and control are gaining more importance with each new system that is focused on delivering the best performance per watt per workload. Companies are consistently focusing on lowering TCO while still meeting customer demands for increased capability to run more workloads in the data center.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center The consequences of not managing power can be related to, but are not limited to these examples: • Rack density suffers from poor server placement, and no real monitoring is available. • Unpredictable power bills due to no central monitoring capability. The energy provider supplies a bill, but there is no breakdown from the overall power. • Brownout or blackout issues cause loss of workloads.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center the total cost of ownership by better managing power and cooling operational costs. Administrators can improve business continuity by dynamically capping power to avoid overcooling systems, reducing downtime and allowing critical operations to continue even during power or thermal events.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Test-Bed Blueprint Design Considerations Software Architecture Intel has worked with Dell to implement a test bed that features Dell PowerEdge R-Series servers, designed for highperformance and efficiency in enterprise computing.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center control. The device must be configured to use at least one of the cipher suites 0-3 and enable “IPMI over LAN” setting. • The WS-MAN user, through which Power Center communicates with the chassis, must be a local user with administrative control and enable the “Web Server” service.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Physical Architecture Figure 3 shows the test bed deployment architecture. Dell OpenManage Power Center is installed on a virtual machine, and the local DNS/DHCP and Active Directory services are provided by the lab subnet on which the systems reside. The two physical Dell PowerEdge R720 servers are used for case testing with Dell OpenManage Power Center.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center c. By default, this root user has administrative rights on the server. Server Setup & Configuration Out of the box setup is quite simple, adding power cords for each PSU and one Ethernet cable for Network Interface Card (NIC) 1, and another Ethernet cable for the iDRAC7 port. In this testing scenario, we chose to use the same subnet for both the data and manageability ports.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center 3. During your license key import (which is matched to your system tag) the iDRAC will setup the license and convert the iDRAC usage to Enterprise Mode as shown in the below figure. The iDRAC7 is now setup with an Enterprise license to monitor and manage power for the next steps in the Power Center installation.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center The following steps assume the reader has a basic understanding of how to install and configure Microsoft Windows Server 2008 R2 x64 Enterprise Edition, and all further steps are based on this installation. 1. Install Dell OpenManage Power Center (version 1.0) and follow the default instructions for installation. 2. After installation, you can connect to the Power Center console via two different methods: a.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Global Configuration On the Initial Steps section, click into the Global Configuration link and verify the following settings: 1. Power Center Monitoring – all devices measuring power and temperature in one minute intervals 2. Default Units – match the local temperature and currency for your region 3. Protocol Type Device Timeout – IPMI and SNMP set to 3 seconds, and WS-MAN set to 60 seconds 4.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center 4. You can add multiple ranges if you wish, but for this demonstration only one subnet will be scanned as shown below. Be sure to select the IP range checkbox, and select Next to move to the next step. 5. In the window shown below, you can select the protocol to be used to scan the IP range selected. For this test bed solution we will use IPMI to communicate with the iDRAC6/iDRAC7.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center 6. You have now added an IPMI Protocol Scanning Profile to Power Center (as shown below) which can be re-used for future IP subnet scans on the same, or other subnets. 7. You are now back at the Device Discovery Summary Page, if your data is setup correctly you can click Search to commence your subnet scan for iDRAC devices. 8. During the scanning process, you will notice the systems that are discovered will be counted on the screen.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center This shows the details of your recent searches, including the most recent time of scan, how long your scan was running, and the number of devices discovered. If you add more devices to the subnet, you can rerun the search.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Creating and Managing Groups Creating a group of servers gives an opportunity for the data center administrator to put some human logic around the servers and devices found in the data center.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Now you have successfully created the first physical group in Power Center, this is the highest level physical entity to monitor and manage power and temperature for all other systems below this level.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Adding Servers to the Rack In order to add servers to the rack, select the rack from the Physical Group Tab – note that there are no devices listed. Below the Physical Group Rack tab is a physical layout of the rack.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center 1. Click + Batch Add to add the iDRAC7 devices found previously in our Device Scan. 2. Be sure to Select both devices, you can also determine which slot (U) the systems reside in the rack. For our test bed, we’ll allow the Auto setting to be selected. Then click Apply. Now you will notice the servers are put into the rack layout, and depending on rack size, you may have to scroll down to see the details.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Energy Management Use Cases Use Case One: Monitoring Power, Temperature, and Events of a Device or Group The most basic use case of monitoring gives us the basis of data to evaluate the systems in the data center and make conscious decisions based on that data. Data center administrators are expected to have information on rack density, thermal footprint, and overall power consumption on a measured basis across the data center.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center In our test bed, we will use 6,000W for the critical wattage consumption – this is the limit as derived from our circuit. To calculate the wattage, we use Ohm’s Law: Watts = Volts x Amps. Power Threshold Determination • In the test bed scenario we have 200V x 30A = 6,000 Watts which is our critical threshold. • The warning threshold is based on exceeding 90 percent utilization of the circuit, and 5,500W is approximately 92 percent utilization.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center On the DASHBOARD tab you can see some higher level data points with an easy to read display of average power consumption (W) and the average inlet temperature for the entire rack. In the screen shot below, you can see the rack level power, and average temperature as measured by both of the servers in the rack. Note the power and temperature thresholds shown as described in the previous segment.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Accuracy of the Dell PowerEdge R720 in the Data Center For accuracy comparison, here are a few pictures of the two Dell PowerEdge R720 servers along with temperature measurements, using an Ultra U12-41380 Infrared Thermometer and power measurements from the Yokagawa WT-210 Power Meter Application.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Power Details You can drill down further into the POWER DETAILS tab to visualize the average power of the rack over time. In the display you will notice several power measurements showcased, some of this data is extracted from Intel Node Manager statistics which are recorded in the firmware of the systems. • Average Power - The average power value from the previous time point to the current time point.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Temperature Details Clicking into the TEMPERATURE DETAILS tab allows you to trend the inlet thermal temperature over time. This data is very helpful in recognizing if there is a cooling failure in the data center. It can also be used to determine if you have too much (or too little) cooling in certain segments of your data center. This measurement can help you to balance your cooling across the room to alleviate over-spending on cooling.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Use Case Two: Creating Power Policies to Increase Server Density As part of the Dell PowerEdge Servers, Intel Node Manager gives the capability to set up to 16 discrete policies per server. These policies can be related to static or dynamic power limits, temperature based triggers, and also used to mitigate potential power outage situations.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center The screen shows that you can name the policy, and that the power cap can be between 136W and 489W – these numbers are derived each time the Dell PowerEdge Server reboots and tests the power limits of the platform. These numbers can change if you replace components, or upgrade memory, hard drives, or other components within the server chassis.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Once you click Apply, this static power cap is installed into the server’s firmware and will remain until you change or remove the policy. If you check the POWER DETAILS tab for this system, you will notice the workload has settled in at 300W.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Multi Server Policy To establish a multi-server policy, you must simply select a group of systems, whether it is an entire data center, room, aisle, rack, or chassis. The data provided when selecting any of those higher level groups will contain the information of all the systems beneath it. There are static and dynamic capping policies for group capping.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center After clicking Next, the Power Allocation screen shows the ability to prioritize different servers to High-Medium-Low if you have different priorities for the workloads on your systems. In the test-bed scenario, we will keep these systems at Medium (which is default). Note how the two systems are operating at different levels, and will both comprise the data that is used to meet the 80 percent limit on the rack.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Once you click Apply, this dynamic power cap is installed into the server’s firmware and will remain until you change or remove the policy. Check the POWER DETAILS tab for this system, and see how the systems are limited to 80 percent of their maximum power usage.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Let’s review the power scaling in building a rack full of these Dell PowerEdge R720xd Servers. Each server has twin 750W Power Supplies, and if we were using the nameplate value of these systems to deploy into a rack, we would simply take the rack power capability of 6kW and divide by 750W to get the number of servers we could put into the rack. That equates to only 8 servers.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Use Case Three: Surviving Power and Thermal Demands in the Data Center Occasionally there are issues that occur in the data center that are beyond IT manager's control, and even customers who have fully redundant power and cooling still have minor hiccups in service from time to time.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center 6. 7. 36 You will be greeted with a warning about implementing this policy as it will reduce system performance. Click Continue. Your rack is now in Emergency Power Reduction Mode as shown in the top bar of the screen.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center And your system power has been drastically reduced. 8. To turn off Emergency Power Reduction mode, click the red bar and you will see the devices or entities that are under the policy control. Then check the Remove box and click Apply. 9. After you turn off Emergency Power Reduction mode, your systems will return to normal power operation.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Things to Consider Architectural Considerations 1. Scalability A single installation of Dell OpenManage Power Center can support up to 2,000 nodes. For larger implementations, multiple instantiations would be required. 2. Management Traffic Separation Many customers feel a need to separate their management data from their production data. The Dell iDRAC allows you to put your management data onto a separate network port.
Intel® Cloud Builders Guide: Dell* OpenManage* Power Center Disclaimers ∆ Intel processor numbers are not a measure of performance. Processor numbers differentiate features within each processor family, not across different processor families. See www.intel.com/ products/processor_number for details. Intel Node Manager: 40% increase in density per published proof of concept http://communities.intel.
