PowerEdge M1000e Technical Guide The M1000e chassis provides flexibility, power and thermal efficiency with scalability for future needs.
Dell This document is for informational purposes only. Dell reserves the right to make changes without further notice to any products herein. The content provided is as is and without express or implied warranties of any kind. Dell, PowerEdge, PowerConnect, RapidRails, VersaRails, FlexAddress, and OpenManage are trademarks of Dell, Inc. Avocent and MergePoint Unity are registered trademarks of Avocent Corporation or its subsidiaries.
Dell Table of Contents 1 2 3 4 5 6 7 8 9 Product Comparison ........................................................................................... 6 Key Technologies............................................................................................... 8 2.1 Overview .................................................................................................. 8 2.2 Detailed Information ....................................................................................
Dell 10 11 12 Storage ........................................................................................................ Video ........................................................................................................... Rack Information ............................................................................................. 12.1 Overview ................................................................................................ 12.2 Rails .....................................
Dell Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. Installed iKVM Blank ............................................................................. Power Supply, CMC, and I/O Module Blanks ..
Dell 1 Product Comparison The Dell™ PowerEdge™ M1000e modular server enclosure offers significant enhancements over its predecessor, the 1955, as can be seen in the following table: Table 1. Comparison of PowerEdge 1855/1955 Chassis and M1000e Chassis Feature 1855/1955 Chassis M1000e Chassis Blade Compatibility PowerEdge 1855/1955 PowerEdge M600/M605/M610/M610x/M710/M710HD/M910/M915 Form Factor 7U 10U Max No.
Dell Greater density means: • • • • • Smaller footprint More processing performance More RAM capacity Lower power consumption per unit Easier manageability Dell’s blade server platform offers superior feature density over comparable rack servers, as can be seen from Table 2 (darker blue shading indicates increased density). Rack vs. Blade Server Rack-Level Specification Comparison1 Table 2.
Dell 2 Key Technologies 2.1 Overview The Dell™ PowerEdge™ M1000e is designed to help customers be more efficient with time, power and cooling, investment, and system performance. It is a breakthrough Dell-engineered and patentpending design that maximizes flexibility, power and thermal efficiency, system-wide availability, performance, and manageability. The chassis integrates the latest in management, I/O, power and cooling technologies in a modular, easy-to-use package.
Dell The chassis features: • • • • • • • A high-speed passive midplane that connects the server modules in the front and to power, I/O, and management infrastructure in the rear of the enclosure Comprehensive I/O options to support dual links of 40 Gigabits per second (with 4x QDR InfiniBand) with future support for higher-bandwidth I/O devices when those technologies become available; this provides high‐speed server module connectivity to the network and to storage, now and well into the future Thorough
Dell 3 3.1 System Information Overview The Dell™ PowerEdge™ M1000e modular server enclosure is a breakthrough in enterprise server architecture. The enclosure and its components spring from a revolutionary, ground-up design incorporating the latest advances in power, cooling, I/O, and management technologies. These technologies are packed into a highly available rack-dense package that integrates into standard Dell and third-party 2000 mm depth racks. 3.2 Product Features Summary Table 3.
Dell 4 4.1 Mechanical Chassis Description The Dell™ PowerEdge™ server modules are accessible from the front of the M1000e enclosure. At the bottom of the enclosure is a flip-out multiple-angle LCD screen for local systems management configuration, system information, and status. The front of the enclosure also contains two USB connections for USB keyboard and mouse, a video connection, and a system power button.
Dell Figure 2. M1000e Front View The M1000e enclosure supports half‐height or full-height server modules, as well as other form factors, including double-width modules. Figure 3.
Dell Server modules can be freely located within each 2 x 2 half-height quadrant. The mechanical design of the M1000e has support structures for half-height server modules above or below double-width server modules, and for half-height server modules side-by-side with full-height server modules. Figure 4. Example Server Module Configurations Fresh air plenums are at the top and bottom of the chassis. The bottom fresh air plenum provides non‐preheated air to the M1000e power supplies.
Dell The enclosure ships by default with a single CMC, with the option of adding a second CMC to provide a fully redundant, active‐standby fault-tolerant solution for management access and control. Interleaved in the center of the chassis are fans and I/O modules. This arrangement optimizes the balance of airflow through the system, allowing lower pressure build-up in the system and resulting in lower airflow requirements for the fans.
Dell For information on cabling for the M1000e modular system, see the Routing and Managing Cables section in the Rack Installation Guide on Support.Dell.com/Manuals. RapidRailsTM Static Rails for square-hole racks support toolless installation in 19-inch EIA-310-E compliant square-hole, 4-post racks including all generations of Dell racks except for the 4200 and 2400 series. Minimum rail depth is 703 mm. Square-hole rack adjustment range is 712–755 mm.
Dell 4.9 Fans ` Figure 7. Back View of Chassis Showing Fans The PowerEdge M1000e chassis comes standard with nine hot-swappable, redundant fan modules that are distributed evenly across the enclosure. The speed of each fan is individually managed by the CMC.
Dell Note: The blank blade, hard drive, and server I/O fillers for every blank slot are required for cooling and airflow reasons. Figure 8. Figure 9.
Dell Figure 10.
Dell Figure 11. Installed CMC, I/O Module, and Power Supply Blanks Figure 12. Figure 13.
Dell 4.
Dell 4.11 Control Panel/LCD The control panel contains the local user interface. Functions include chassis-level diagnostic LEDs, LCD panel, and power button. This device is hot-pluggable and is always powered, even in chassis standby mode. Figure 15. Figure 16.
Dell The LCD panel can be used as a diagnostic source and as a place to configure parameters of certain chassis components as well as the server’s iDRAC network configuration. Figure 17 shows some of the capabilities of the LCD control panel. Figure 17. LCD Panel Capabilities The primary function of the LCD panel is to provide real-time information on the health and status of the modules in the enclosure.
Dell 4.12 Security The M1000e offers many security features, including the ability to: • • • Assign one administrator per blade or one administrator per multiple blades Grant permissions to some blades but not to others Customize administrative access for CMC, iDRAC, and I/O Most of the security capabilities are driven by the CMC, which provides a mechanism for centralized configuration of the M1000e enclosure’s security settings and user access. It is secured by a usermodifiable password.
Dell 5 Power, Thermal, Acoustic Built on Dell™ Energy Smart technology, the PowerEdge™ M1000e is one of the most power-efficient blade solutions on the market. The M1000e enclosure takes advantage of Energy Smart thermal design efficiencies, such as ultra-efficient power supplies and dynamic power-efficient fans with optimized airflow design to efficiently cool the chassis and enable better performance in a lower power envelope.
Dell Figure 19. M1000e Power Supply Back View The Dell power supplies use output ORing FETs to isolate the power supply from the 12V system bus. If a single power supply fails its output ORing FET, the power supply will turn off removing itself from the bus like an electrical switch that turns off when the power supply fails. When Dynamic Power Supply Engagement (DPSE) is enabled, the PSU units move between on and off states depending on actual power draw conditions to achieve high power efficiency.
Dell Figure 20. Power Architecture These supplies are hot-swappable, so at any time you can move the power supply from one slot to another as long as there is enough reserve capacity. 5.1.1 2360 Watt Power Supplies The 2360W (230V) power supply offers current sharing between power supplies, allowing a total system redundant power of approximately 6700W in a 3+3 power supply configuration.
Dell 5.2 Power Redundancy Options The six power supplies allow for two different redundancy options: Grid Redundancy and Power Supply Redundancy. For information on the Grid Redundancy option, see the Power Management chapter in the Dell Chassis Management Controller Firmware Version x.x User Guide on Support.Dell.com/Manuals. With power supply redundancy, the six power supplies are seen as one bank. They should be populated in the following order: 1,2,3,4,5,6. 5.2.
Dell Table 8. Power Supply Redundancy Maximum Power Available (Watts DC) Configuration 2700W PSU 1350W PSU 2360W PSU 1+1 2534 1230 2222 2+1 5068 2460 4444 3+1 7602 3690 6666 4+1 9922 4920 8888 5+1 9922 6150 9922 Table 9. Configuration 5.2.
Dell The following power-supply combinations are not allowed in the M1000e chassis: • • 2360W + 1350W 2700W + 1350W 5.2.4 Chassis Behavior When Powering Up Power Supplies For information on M1000e chassis behavior when powering up the power supplies, see the Power Management chapter in the Dell Chassis Management Controller Firmware Version x.x User’s Guide on Support.Dell.com/Manuals. 5.2.
Dell 5.2.5.4 Behavior of 1350W + 2360W/2700W Power Supplies Mixing 1350W (110VAC) and 2360W/2700W (220VAC) power supplies is not allowed. • • If you insert an 110VAC power supply into a chassis running 220VAC, the system will not allow the 110VAC supply to provide capacity to the system. The AC good light will illuminate, but the DC good light will not. If you insert a 220VAC power supply into a chassis running 110VAC, the system will not allow the 220VAC supply to provide capacity to the system.
Dell The following information is also applicable for half-height blade servers: Throttling (performance reduction to conserve power): o 220–240V power: 2 PSUs—Throttling will occur if heavily configured blades are worked very hard at the same time. 3 PSUs: 130W processors—Some throttling will occur if heavily configured blades are worked hard at the same time. 95W or 80W processors—Throttling may happen if all servers are run under a very heavy load. 4 or 5 PSUs—No throttling will occur.
Dell Table 12.
Dell Dell recommends that all customers running PowerEdge M910 blade servers should install the modular iDRAC firmware 3.02 upgrade, which contains the following enhancements: Improved power budgeting to lower the amount of power budget that the M910 requests from the CMC, enabling more servers to run on fewer power supplies. Functionality to enable iDRAC to retry power-ups for blade servers automatically in the event that some servers do not power up initially due to the pre-boot power inventory.
Dell If power consumption demands exceed available power, the enclosure throttles back the power supplied to blades as prioritized in the CMC. The blades will not shut down; rather they will slow down if necessary; Dell designed the system this way on purpose, in response to customer feedback that they did not want the blades to shut themselves down under any condition.
Dell as a chassis view; or the ability make use of that knowledge (available through SMBIOS structures) to make use of that in DPM/DRS or other tools/applications. For more information, see the Power Management chapter in the Dell Chassis Management Controller Firmware Version x.x User’s Guide on Support.Dell.com/Manuals. 5.
Dell Figure 22. Server Cooling Air Profile The server modules are cooled with traditional front‐to‐back cooling. As shown in the figure, the front of the system is dominated by inlet area for the individual server modules. The air passes through the server modules, through venting holes in the midplane, and is then drawn into the fans which exhaust the air from the chassis.
Dell Figure 24. I/O Cooling Air Profile The I/O modules use a bypass duct to draw ambient air from the front of the system to the I/O module inlet, as seen in the figure. This duct is located above the server modules. This cool air is then drawn down through the I/O modules in a top to bottom flow path and into the plenum between the midplane and fans, from where it is exhausted from the system. Figure 25.
Dell This hardware design is coupled with a thermal cooling algorithm that incorporates the following: • • • Server module level thermal monitoring by the iDRAC I/O module thermal health monitors Fan control and monitoring by the CMC The iDRAC on each server module calculates the amount of airflow required on an individual server module level and sends a request to the CMC. This request is based on temperature conditions on the server module, as well as passive requirements due to hardware configuration.
Dell 5.10 Acoustics The M1000e is engineered for sound quality in accordance with the Dell Enterprise acoustical specification. Compared to previous generations of products, the fans have more levels of control and finer tuning of the fan behavior. Firmware is optimized to choose the lowest fan speeds and therefore the lowest acoustical output for any configuration (components installed), operating condition (applications being run), and ambient temperature.
Dell 6 Processors and Memory See the Dell PowerEdge Mxxx Technical Guide on Dell.com for each of the compatible blade servers offered for more details on processors and memory offered.
Dell 7 Midplane Though hidden from view in an actively running system, the midplane is the focal point for all connectivity within the Dell™ PowerEdge™ M1000e modular system. The midplane is a large printed circuit board providing power distribution, fabric connectivity, and system management infrastructure. Additionally it allows airflow paths for the front-to-back cooling system through ventilation holes. Figure 26.
Dell Figure 27. M1000e Midplane Front View The system is designed for receptacles on all midplane connectors and pins on all pluggable components, so any potential for bent pins is limited to the pluggable field replaceable unit, not to the system. This contributes to the high reliability and uptime of the M1000e modular system. The midplane is physically attached to the enclosure front structural element. It is aligned by guide‐ pins and edges in all three axes.
Dell Figure 28. M1000e Midplane Back View All M1000e midplane routing is fully isolated, supporting all chassis power, fabric, system management, and fault-tolerance requirements.
Dell 8 Embedded NICs/LAN on Motherboard (LOM) For information on embedded NICs/LOMs, see the Dell PowerEdge Mxxx Technical Guide on Dell.com for each of the compatible blade servers.
Dell 9 Input/Output (I/O) 9.1 Overview The Dell™ PowerEdge™ M-series provides complete, snap-in Flex I/O scalability down to the switch interconnects. Flex I/O technology is the foundation of the M1000e I/O subsystem. Customers may mix and match I/O modules from a wide variety of options including Cisco®, Dell™ PowerConnect™, Fibre Channel, and InfiniBand options. The I/O modules may be installed singly or in redundant pairs.
Dell The M1000e system management hardware and software includes Fabric Consistency Checking, preventing the accidental activation of any misconfigured fabric device on a server module. The system will automatically detect this misconfiguration and alert the user of the error. No damage occurs to the system, and the user will have the ability to reconfigure the faulted module. M1000e I/O is fully scalable to current and future generations of server modules and I/O Modules.
Dell Figure 30. Fabric Specifications Data Rate Per Lane (Gbps) Data Rate Per Link (Gbps) Lanes Per Link Per Industry Specification Fabric Encoding Symbol Rate Per Lane (Gbps) PCIe Gen1 8B/10B 2.5 2 8 (4 lane) 1, 2, 4, 8, 12, 16, 32 PCIe Gen2 8B/10B 5 4 16 (4 lane) 1, 2, 4, 8, 12, 16, 32 SATA 3Gbps 8B/10B 3 2.4 2.4 1 SATA 6Gbps 8B/10B 6 4.8 4.8 1 SAS 3Gbps 8B/10B 3 2.4 2.4 1-Any SAS 6Gbps 8B/10B 6 4.8 4.8 1-Any FC 4Gbps 8B/10B 4.25 3.4 3.
Dell I/O modules are used as pairs, with two modules servicing each server module fabric providing a fully redundant solution. I/O modules may be passthroughs or switches. Passthrough modules provide direct 1:1 connectivity from each LOM/mezzanine card port on each server module to the external network. Switches provide an efficient way to consolidate links from the LOM or Mezzanine cards on the server modules to uplinks into the customer’s network. Figure 32.
Dell • • Mellanox® DDR (20Gb) InfiniBand switch Mellanox QDR (40Gb) InfiniBand switch For an up-to-date list of supported I/O modules, see the Ethernet I/O Cards page on Dell.com for supported I/O hardware. 9.4 I/O Module Installation For detailed information on installing the I/O modules in your system, see the I/O Connectivity section in the About Your System chapter in the Dell PowerEdge Modular Systems Hardware Owner’s Manual on Support.Dell.com/Manuals. 9.
Dell Figure 33. FlexAddress Addresses FlexAddress replaces the factory-assigned World Wide Name/Media Access Control (WWN/MAC) IDs on a blade with WWN/MAC IDs from the FlexAddress SD card associated with that slot. This userconfigurable feature enables a choice of iSCSI MAC, Ethernet MAC, and/or WWN persistence, and thus allows blades to be swapped without affecting SAN Zoning, iSCSI zoning, or any MAC-dependent functions.
Dell The CMC manages the following functions specific to FlexAddress: • Provides user interface for enabling or disabling the FlexAddress feature—on a per-blade-slot basis, a per-fabric basis, or both • Identifies and reports device information for each of the supported fabric types—LOMs, Ethernet, and Fibre Channel mezzanine cards • Validates all the components of the FlexAddress feature—SD card validation, System BIOS, I/O controller firmware, CMC firmware, and, iDRAC firmware versions • Reports FlexAddre
Dell Wake on LAN (WOL) is enabled after a power down and power up of the enclosure. FlexAddress on Ethernet devices is programmed by the module server BIOS. In order for the blade BIOS to program the address, it needs to be operational which requires the blade to be powered up. Once the powerdown and power-up sequence completes, the FlexAddress feature is available for Wake-On-LAN (WOL) function.
Dell 10 Storage The Dell™ PowerEdge™ M1000e was designed primarily to support external storage over the network, which is the primary requirement for customers seeking maximum density. Figure 35. Examples of Major Storage Platforms Supported The blade servers also support at least two internal hard drives which can be put into RAID if desired. For details, see the Dell PowerEdge Mxxx Technical Guide for each of the compatible blade servers offered on Dell.com.
Dell 11 Video The iKVM supports a video display resolution range from 640 x 480 at 60 Hz up to 1280 x 1024 x 65,000 colors (noninterlaced) at 75 Hz. Table 14.
Dell 12 Rack Information 12.1 Overview The Dell™ RapidRails™ static rail system for the PowerEdge™ M1000e provides toolless support for racks with square mounting holes including all generations of Dell racks except for the 4200 and 2400 series. Also available are the VersaRails™ static rails, which offer tooled mounting support for racks with square or unthreaded round mounting holes.
Dell Figure 37. M1000e VersaRails Static Rails The VersaRails static rails are not intended to be mounted in threaded-hole racks since the rails cannot be fully tightened and secured against the rack mounting flange. Neither the VersaRails nor the RapidRails kits support mounting in 2-post racks. Table 15.
Dell Figure 38. M1000e Strain Relief Bar and Cable Enumerator Clip (12 Per Kit) 12.4 Rack View The M1000e is installed on the rails by simply resting the back of the system on the rail ledges, pushing the system forward until it fully seats, and tightening the thumbscrews on the chassis front panel. Figure 39.
Dell Cable Enumerator Clips Strain Relief Bar Figure 40. M1000e Strain Relief Bar and Cable Enumerator Clips More information can be found in the Rack Installation Guide on Support.Dell.com/Manuals.
Dell 13 Virtualization The Dell™ PowerEdge™ M1000e and the blade servers which fit in it have been designed for optimal use with all major virtualization software platforms. Figure 41.
Dell 14 Systems Management 14.1 Overview The Dell™ PowerEdge™ M1000e server solution offers a holistic management solution designed to fit into any customer data center.
Dell Management connections transfer health and control traffic throughout the chassis. The system management fabric is architected for 100BaseT Ethernet over differential pairs routed to each module. There are two 100BaseT interfaces between CMCs, one switched and one unswitched. All system management Ethernet is routed for 100 Mbps signaling. Every module has a management network link to each CMC, with redundancy provided at the module level.
Dell o o o Updates are staged on the Lifecycle Controller and applied immediately or during a scheduled maintenance window. Lifecycle controller 1.3 or greater supports updates with BIOS, diagnostics, driver pack, USC, RAID controller firmware, iDRAC firmware, and NIC firmware. CMC 3.0 or greater will enable these updates by one-to-many using the CMC GUI/CLI. More information on the iDRAC and Lifecycle Controller can be found on Support.Dell.com. 14.
Dell Updating o Reporting of firmware versions o One-to-many iDRAC firmware update o Consolidated CMC and iKVM firmware update o One-to-many update of drivers and firmware through a remote file share (with Repository Manager) o 2 x 10/100/1000Mb Ethernet ports + 1 serial port Real-time power and thermal monitoring and management o Consolidated chassis and blade power reporting o Power budget management and allocation o Real-time system AC power consumption with reset-able peak and minimum value
Dell o o o o Two factor authentication with SmartCard Single sign on using OS credentials (with AD) Private Key Authentication (PK Auth) Secure Web (SSL) and CLI (Telnet/SSH) interfaces Support for industry-standard interfaces o SMASH CLP o WSMAN o CIM XML o SNMP PowerEdge M1000e Technical Guide 64
Dell The Integrated Dell Remote Access Controller (iDRAC) on each server module is connected to the CMC through dedicated, fully redundant 100 Mbps Ethernet connections wired through the midplane to a dedicated 24‐port Ethernet switch on the CMC, and exposed to the outside world through the CMC’s external Management Ethernet interface (10/100/1000M). This connection is distinct from the three redundant data Fabrics A, B and C.
Dell 14.4 Integrated Keyboard and Mouse Controller (iKVM) Figure 44. M1000e iKVM The M1000e modular enclosure supports one optional Integrated KVM (iKVM) module. This module occupies a single slot accessible through the back of the chassis. The iKVM redirects local server module video, keyboard, and mouse electrical interfaces to either the iKVM local ports or the M1000e front panel ports. The iKVM allows connection to a VGA monitor, USB keyboard, and USB mouse without use of a dongle.
Dell Figure 45. Front Keyboard/Video Ports Dell modular servers also include vKVM as a standard feature, routing the operator’s keyboard output, mouse output and video between the target server module and a console located on the system management IP network. With up to two simultaneous vKVM sessions per blade, remote management now satisfies virtually any usage model.
Dell 15 Peripherals Common peripherals for the Dell™ PowerEdge™ M1000e include the following: • • • • An external USB DVD-ROM Drive, used for local installation of OS or other software. A Dell 1U rack console which enables customers to mount a system administrator’s control station directly into a Dell rack without sacrificing rack space needed for servers and other peripherals.
Dell Appendix A. Regulatory Certifications See the external Product Safety, EMC, and Environmental Datasheets at http://www.dell.com/regulatory_compliance_datasheets.
Dell Appendix B. Status Messages A 1. LCD Status Messages For a detailed list of LCD status messages, see the LCD Status Messages section in the About Your System chapter in the Dell PowerEdge Modular Systems Hardware Owner’s Manual on on Support.Dell.com/Manuals. A 2. System Status Messages For a detailed list of system status message, see the System Messages section in the About Your System chapter in the Dell PowerEdge Modular Systems Hardware Owner’s Manual on on Support.Dell.com/Manuals.
Dell Appendix C. Additional Information Videos highlighting the major Dell™ PowerEdge™ M1000e features are available on http://www.dell.com/html/us/products/pedge/poweredge_mseries_blade_videos/poweredge.html. The Dell PowerEdge M1000e Configuration Guide, Dell PowerEdge Modular Systems Hardware Owner’s Manual, and CMC Administrator Reference Guide or CMC User’s Guide on Support.Dell.com/Manuals each contain a wealth of additional information about the PowerEdge M1000e’s capabilities.
