PowerEdge M1000e Technical Guide The M1000e chassis provides flexibility, power and thermal efficiency with scalability for future needs.
Dell This document is for informational purposes only. Dell reserves the right to make changes without further notice to any products herein. The content provided is as is and without express or implied warranties of any kind. Dell, PowerEdge, PowerConnect, RapidRails, VersaRails, FlexAddress, and OpenManage are trademarks of Dell, Inc. Avocent is a registered trademark of Avocent Corporation or its subsidiaries.
Dell Table of Contents 1 2 Product Comparison ........................................................................................... 5 New Technologies .............................................................................................. 7 2.1 Overview .................................................................................................. 7 2.2 Detailed Information .................................................................................... 7 3 System Information .........
Dell 12.1 Overview ................................................................................................ 12.2 Rails ...................................................................................................... 12.3 Cable Management Arm (CMA) ....................................................................... 12.4 Rack View ............................................................................................... 13 Virtualization .................................................
Dell Figure 23. Figure 24. Figure 25. Figure 26. Figure 27. Figure 28. Figure 29. Figure 30. Figure 31. Figure 32. Figure 33. Figure 34. Figure 35. Figure 36. Figure 37. Figure 38. Figure 39. Figure 40. Figure 41. Figure 42. Figure 43. Figure 44. Figure 45. Figure 46. Figure 47. Figure 48. Figure 49. Figure 50. Figure 51. Figure 52. Figure 53. Figure 54. Figure 55. Figure 56. Power Architecture .............................................................................. PMBus Communication Channels .....
Dell 1 Product Comparison The Dell™ PowerEdge™ M1000e offers significant enhancements over its predecessor, the 1955, as can be seen in the following table: Table 1. Comparison of PowerEdge 1855/1955 Chassis and M1000e Chassis Feature 1855/1955 Chassis M1000e Chassis Blade Compatibility PowerEdge 1855/1955 PowerEdge M600/M605 11G and beyond Form Factor 7U 10U No.
Dell Greater density means: • • • • • Smaller Footprint More Processing Performance More RAM capacity Lower Power Consumption per unit Easier Manageability Dell’s blade server platform offers superior feature density over comparable rack servers, as can be seen from 0. (Darker blue shading indicates increased memory density.) Table 2. R410 Rack vs.
Dell 2 New Technologies 2.1 Overview The PowerEdge M1000e is designed to help customers be more efficient with time, power and cooling, investment, and system performance. It is a breakthrough Dell engineered and patentpending design that maximizes flexibility, power and thermal efficiency, system-wide availability, performance, and manageability. The chassis integrates the latest in management, I/O, power and cooling technologies in a modular, easy-to-use package.
Dell • • • • • • • A high-speed passive midplane that connects the server modules in the front and power, I/O, and management infrastructure in the rear of the enclosure. Comprehensive I/O options to support dual links of 40 Gigabits per second today (with 4x QDR InfiniBand®) with future support of even higher bandwidth I/O devices when those technologies become available. This provides high‐speed server module connectivity to the network and storage now and well into the future.
Dell 3 System Information 3.1 Overview The Dell PowerEdge M1000e Modular Server Enclosure is a breakthrough in enterprise server architecture. The enclosure and its components spring from a revolutionary, ground-up design incorporating the latest advances in power, cooling, I/O, and management technologies. These technologies are packed into a highly available rack dense package that integrates into standard Dell and third-party 2000mm depth racks. 3.2 Product Features Summary Table 3.
Dell 4 Mechanical 4.1 Chassis Description The Dell M1000e supports up to sixteen half-height or 8 full-height server modules. The chassis guide and retention features are designed such that alternative module form factors are possible. The chassis architecture is flexible enough that server, storage, or other types of front-loading modules are possible. 4.2 Dimensions and Weight Table 4. 4.3 Dimensions Dimension Measurement Width, not including rack ears 447.5 mm Height 440.
Dell Figure 3. Possible Server Module Sizes, Front Panel View Server Modules can be freely located within each 2 x 2 half-height quadrant. The mechanical design of the M1000e has support structures for half-height server modules above or below double-width server modules, and for half-height server modules side-by-side with full-height server modules. Figure 4. Example Server Module Configurations Server modules are accessible from the front of the M1000e enclosure.
Dell Fresh air plenums are at both top and bottom of the chassis. The bottom fresh air plenum provides non‐preheated air to the M1000e power supplies. The top fresh air plenum provides non‐preheated air to the CMC, iKVM and I/O modules. 4.4 Back Panel Features The rear of the M1000e Enclosure contains system management, cooling, power and I/O components. At the top of the enclosure are slots for two Chassis Management Cards and one integrated KVM switch.
Dell 4.
Dell Figure 6. Rack Cabling RapidRailsTM Static Rails for Square Hole Racks supports toolless installation in 19‖ EIA-310-E compliant square hole 4-post racks including all generations of Dell racks except for the 4200 & 2400 series. Minimum rail depth is 703 mm. Square-hole rack adjustment range is 712–755 mm. The rail system includes a strain relief bar and cable enumerators for managing and securing cables.
Dell Figure 7. RapidRails Rack Kit Contents Figure 8. VersaRails Rack Kit Contents See Section 12 for more details.
Dell 4.7 Rack Support The M1000e chassis offers the following options for rack support: • • RapidRails™ static rails for toolless mounting in 4-post racks with square holes VersaRails™ static rails for tooled mounting in 4-post racks with square or unthreaded round holes See Section 12 for more details. 4.8 Rack View Figure 9.
Dell 4.9 Fans ` Figure 10. Rear View Showing Fans The PowerEdge M1000e chassis comes standard with 9 hot-swappable, redundant fan modules that are distributed evenly across the enclosure. The speed of each fan is individually managed by the CMC.
Dell Note: The blank blade, hard drive, and server I/O fillers for every blank slot are required for cooling/airflow reasons. Figure 11. Blades, Blanks, and 1 Open Slot Needing to be Filled Figure 12.
Dell Figure 13.
Dell Figure 14. Installed CMC, I/O Module, and Power Supply Blanks Figure 15.
Dell Figure 16. Power Supply, CMC, and I/O Module Blanks 4.
Dell Figure 17. Simplified Cabling 4.11 Control Panel/LCD The control panel contains the local user interface. Functions include chassis level diagnostic LEDs, LCD panel, and power button. This device is hot-pluggable and is always powered, even in chassis standby mode.
Dell Figure 18. M1000e LCD Panel Recessed Position Figure 19. M1000e LCD Panel During Usage The M1000e chassis LCD shows extensive information about the status of each hardware module, network information for the CMC and each iDRAC, and status messages with detailed explanations in plain language.
Dell information. The LCD panel can be retracted into the chassis body, or extended and angled once deployed for full visibility no matter where the M1000e is mounted in the rack. The LCD panel can be used as a diagnostic source and as a place to configure parameters of certain chassis components as well as the server’s iDRAC network configuration. Figure 20 shows some of the capabilities of the LCD control panel. Figure 20.
Dell Most of the security capabilities are driven by the CMC, which provides a mechanism for centralized configuration of the M1000e enclosure’s security settings and user access. It is secured by a usermodifiable password.
Dell 5 Power, Thermal, Acoustic Built on Dell Energy Smart technology, the M1000e is one of the most power-efficient blade solutions on the market. The M1000e enclosure takes advantage of Energy Smart thermal design efficiencies, such as ultra-efficient power supplies and dynamic power-efficient fans with optimized airflow design to efficiently cool the chassis and enable better performance in a lower power envelope.
Dell Figure 22. M1000e Power Supply Rear View The Dell power supplies utilize output Oring FETs to isolate the power supply from the 12V system bus. If a single power supply fails its output Oring FET for that power supply will turn off removing itself from the bus. Think of it as an electrical switch that turns off when the power supply fails. 5.1.1 Supported Voltages Dell currently offers a power supply rated at 2360W 230V.
Dell • • • 1+1 • 4+1 2+1 • 5+1 3+1 When Dynamic Power Supply Engagement (DPSE) is enabled, the PSU units move between On and Off states depending upon actual power draw conditions to achieve high power efficiency by driving fewer supplies to maximum versus all with partial and less-efficient loading. In the N+N power supply configuration, the system will provide protection against AC grid loss or power supply failures.
Dell 5.1.3 Power Management Power is no longer just about power delivery, it is also about power management. The M1000e System offers many advanced power management features. Most of these features operate transparently to the user, while others require only a one time selection of desired operating modes.
Dell Figure 24. PMBus Communication Channels All VMware® products include consuming the "current power consumption" and "current power cap/limit" retrieval via Dell specific IPMI commands using iDRAC. They are using this to report the total power consumed by the server and also using this as part of their calculations to determine/approximate the VM-level power. The Power Management chapter in the Dell Chassis Management Controller (CMC) User Guide provides extensive information on power management. 5.
Dell of the fan as compared to the electrical power required to run the fan. The M1000e fan operates at extreme efficiencies which correlates directly into savings in the customer’s required power‐to‐cool. The high‐efficiency design philosophy also extends into the layout of the subsystems within the M1000e. The Server Modules, I/O Modules, and Power Supplies are incorporated into the system with independent airflow paths.
Dell Figure 27. I/O Cooling Air Profile The I/O Modules use a bypass duct to draw ambient air from the front of the system to the I/O Module inlet, as seen in the figure. This duct is located above the server modules. This cool air is then drawn down through the I/O Modules in a top to bottom flow path and into the plenum between the midplane and fans, from where it is exhausted from the system. Figure 28.
Dell This hardware design is coupled with a thermal cooling algorithm that incorporates the following: • • • Server module level thermal monitoring by the iDRAC I/O module thermal health monitors Fan control and monitoring by the CMC The iDRAC on each server module calculates the amount of airflow required on an individual server module level and sends a request to the CMC. This request is based on temperature conditions on the server module, as well as passive requirements due to hardware configuration.
Dell therefore the lowest acoustical output for any configuration (components installed), operating condition (applications being run), and ambient temperature. Because acoustical output is dependent and indeed minimized for each combination of these variables, no single acoustical level (sound pressure level or sound power level) represents the M1000e, and instead boundaries on sound power level are provided below: • • • • Lowest Fan Speed: Upper Limit A-weighted Sound Power Level, LwA-UL, is 7.
Dell 6 Processors and Memory With the addition of the PowerEdge M910 server to the PowerEdge portfolio, the M1000e is now scalable to 256 cores & 4TB of RAM: 4 sockets x 8 cores x 8 blades = 256 Cores; 32 DIMM sockets x 16GB DIMMs x 8 Blades = 4096GB or 4TB RAM. See the Technical Guide for each of the compatible blade servers offered for more details on processors and memory offered.
Dell 7 Midplane Though hidden from view in an actively running system, the midplane is the focal point for all connectivity within the M1000e Modular System. The midplane is a large printed circuit board providing power distribution, fabric connectivity, and system management infrastructure. Additionally it allows airflow paths for the front-to-back cooling system through ventilation holes. Figure 29.
Dell Figure 30. M1000e Midplane Front View The system is designed for receptacles on all midplane connectors and pins on all pluggable components, so any potential for bent pins is limited to the pluggable field replaceable unit, not to the system. This contributes to the high reliability and uptime of the M1000e modular system. The midplane is physically attached to the enclosure front structural element. It is aligned by guide‐pins and edges in all 3 axes.
Dell Figure 31. M1000e Midplane Rear View All M1000e midplane routing is fully isolated, supporting all chassis power, fabric, system management, and fault-tolerance requirements.
Dell 8 Embedded NICs/LAN on Motherboard (LOM) See the Technical Guide for each of the compatible blade servers.
Dell 9 I/O 9.1 Overview Dell M-series provides complete, snap-in FlexI/O scalability down to the switch interconnects. Flex I/O technology is the foundation of the M1000e I/O subsystem. Customers may mix and match I/O modules, including Cisco®, Dell™ PowerConnect™, Fibre Channel, and InfiniBand options. The I/O modules may be installed singly or in redundant pairs. See I/O Connectivity in the Hardware Owner’s Manual for detailed information. Figure 32.
Dell • • • • Cisco® Virtual Blade Switch capability Ethernet Port Aggregator Virtualization of Ethernet ports for integration into any Ethernet fabric Fibre Channel products from Brocade and Emulex offering powerful connectivity to Dell/EMC SAN fabrics • High-availability clustering inside a single enclosure or between two enclosures Each server module connects to traditional network topologies while providing sufficient bandwidth for multi‐generational product lifecycle upgrades.
Dell Figure 33. High Speed I/O Architecture Fabric B and C are identical, fully customizable fabrics, routed as two sets of four lanes from mezzanine cards on the server modules to the I/O Modules in the rear of the chassis. Supported bandwidth ranges from 1 to 10 Gbps per lane depending on the fabric type used.
Dell Table 6. Fabric Specifications Data Rate Per Lane (Gbps) Data Rate Per Link (Gbps) Lanes Per Link Per Industry Specification Fabric Encoding Symbol Rate Per Lane (Gbps) PCIe Gen1 8B/10B 2.5 2 8 (4 lane) 1,2,4,8,12,16,32 PCIe Gen2 8B/10B 5 4 16 (4 lane) 1,2,4,8,12,16,32 SATA 3Gbps 8B/10B 3 2.4 2.4 1 SATA 6Gbps 8B/10B 6 4.8 4.8 1 SAS 3Gbps 8B/10B 3 2.4 2.4 1-Any SAS 6Gbps 8B/10B 6 4.8 4.8 1-Any FC 4Gbps 8B/10B 4.25 3.4 3.4 1 FC 8bps 8B/10B 8.5 6.8 6.
Dell I/O Modules are used as pairs, with two modules servicing each server module fabric providing a fully redundant solution. I/O Modules may be passthroughs or switches. Passthrough modules provide direct 1:1 connectivity from each LOM/mezzanine card port on each server module to the external network. Switches provide an efficient way to consolidate links from the LOM or Mezzanine cards on the server modules to uplinks into the customer’s network. Figure 35.
Dell • • • 4Gb Fibre Channel Pass-Through Mellanox® DDR (20Gb) InfiniBand Switch Mellanox QDR (40Gb) InfiniBand Switch See the Ethernet I/O Cards page on Dell.com for supported I/O hardware. 9.4 I/O Module Installation For detailed information on installing the I/O modules in your system, see the I/O Modules section in the Hardware Owner’s Manual for your specific PowerEdge server. 9.
Dell Figure 36. FlexAddress Addresses FlexAddress replaces the factory-assigned World Wide Name/Media Access Control (WWN/MAC) IDs on a blade with WWN/MAC IDs from the FlexAddress SD card associated with that slot. This userconfigurable feature enables a choice of iSCSI MAC, Ethernet MAC, and/or WWN persistence, and thus allows blades to be swapped without affecting SAN Zoning, iSCSI zoning, or any MAC-dependent functions.
Dell The CMC manages the following functions specific to FlexAddress: • • • • • • Provides user interface for enabling or disabling the FlexAddress feature—on a per-blade-slot basis, a per-fabric basis, or both Identifies and reports device information for each of the supported fabric types—LOMs, Ethernet, and Fibre Channel mezzanine cards Validates all the components of the FlexAddress feature—SD card validation, System BIOS, IO controller firmware, CMC firmware, and, iDRAC firmware versions Reports Flex
Dell Wake on LAN (WOL) is enabled after a power down and power up of the enclosure. FlexAddress on Ethernet devices is programmed by the module server BIOS. In order for the blade BIOS to program the address, it needs to be operational which requires the blade to be powered up. Once the powerdown and power-up sequence completes, the FlexAddress feature is available for Wake-On-LAN (WOL) function.
Dell Figure 40. CMC FlexAddress Summary Screen FlexAddress MAC/WWNs are now deployed. CMC GUI shows users a summary of Server Assigned (hardware based) and chassis assigned (FlexAddress) MAC/WWNs for the entire chassis or per slot (see Figure Figure 41 below). Green Checkmarks denote which identifier is currently being used.
Dell Figure 41. CMC FlexAddress Server Detail Screen For more information, review the FlexAddress chapter of the CMC User’s Guide.
Dell 10 Storage The M1000e was designed primarily to support external storage over the network, which is the primary requirement for customers seeking maximum density. Figure 42. Examples of Major Storage Platforms Supported The blade servers also support at least two internal hard drives which can be put into RAID if so desired. For details, see the Technical Guide for each of the compatible blade servers offered.
Dell 11 Video The iKVM supports a video display resolution range from 640x480 at 60Hz up to 1280 x 1024 x 65,000 colors (noninterlaced) at 75Hz.
Dell 12 Rack Information 12.1 Overview The RapidRails™ static rail system for the M1000e provides tool-less support for racks with square mounting holes including all generations of Dell racks except for the 4200 & 2400 series. Also available are the VersaRails™ static rails, which offer tooled mounting support for racks with square or unthreaded round mounting holes. Both versions include a strain relief bar and cable enumerator clips to help manage and secure the cables exiting the back of the system.
Dell Figure 44. M1000e VersaRails Static Rails The VersaRails static rails are not intended to be mounted in threaded hole racks since the rails cannot be fully tightened and secured against the rack mounting flange. Neither the VersaRails nor the RapidRails kits support mounting in 2-post racks.
Dell Figure 45. M1000e Strain Relief Bar and Cable Enumerator Clip (12 Per Kit) 12.4 Rack View The M1000e is installed on the rails by simply resting the back of the system on the rail ledges, pushing the system forward until it fully seats, and tightening the thumbscrews on the chassis front panel. Figure 46. M1000e Mounted in the Rack The strain relief bar and cable enumerator clips can be used to help manage and secure the power cords and I/O cables exiting the back of the system as indicated below.
Dell Cable Enumerator Clips Strain Relief Bar Figure 47. M1000e Strain Relief Bar and Cable Enumerator Clips More information can be found in the Rack Installation Guide.
Dell 13 Virtualization The M1000e and the blade servers which fit in it have been designed for optimal use with all major virtualization software platforms. Figure 48.
Dell Figure 49.
Dell 14 Systems Management 14.1 Overview The M1000e server solution offers a holistic management solution designed to fit into any customer data center.
Dell Figure 50. System Management Architecture Diagram Management connections transfer health and control traffic throughout the chassis. The system management fabric is architected for 100BaseT Ethernet over differential pairs routed to each module. There are two 100BaseT interfaces between CMCs, one switched and one unswitched. All system management Ethernet is routed for 100 Mbps signaling. Every module has a management network link to each CMC, with redundancy provided at the module level.
Dell • • Integration with Microsoft® Active Directory® Security, Local and Active Directory Traditional IPMI-based BMC features like hardware monitoring and power control are supported.
Dell Monitoring and alerting for chassis environmental conditions or component health thresholds.
Dell o SNMP The Integrated Dell Remote Access Controller (iDRAC) on each server module is connected to the CMC via dedicated, fully redundant 100 Mbps Ethernet connections wired through the midplane to a dedicated 24‐port Ethernet switch on the CMC, and exposed to the outside world through the CMC’s external Management Ethernet interface (10/100/1000M). This connection is distinct from the three redundant data Fabrics A, B and C.
Dell Figure 52.
Dell 14.4 Integrated Keyboard and Mouse Controller (iKVM) Figure 53. M1000e iKVM The modular enclosure supports one optional Integrated KVM (iKVM) module. This module occupies a single slot accessible through the rear of the chassis. The iKVM redirects local server module video, keyboard, and mouse electrical interfaces to either the iKVM local ports or the M1000e front panel ports. The iKVM allows connection to a VGA monitor, USB keyboard, and USB mouse without use of a dongle.
Dell Figure 54. Rear iKVM interface Panel The front of the enclosure includes two USB connections for a keyboard and mouse, along with a video connection port, both of which require the Avocent iKVM switch to be activated for them to be enabled. These ports are designed for connecting a local front ―crash cart‖ console to be connected to access the blade servers while standing in front of the enclosure. Figure 55.
Dell emulation of USB DVD‐R/W, USB CD‐R/W, USB Flash Drive, USB ISO image and USB Floppy over an IP interface. Connection to vKVM and vMedia is through the CMC, with encryption available on a per stream basis. It is possible to connect the following Dell\Avocent KVMIP switches to the iKVM card in the M1000e blade enclosure using a CAT5 cable.
Dell 15 Peripherals Common peripherals for the M1000e include: • • • • An external USB DVD-ROM Drive is often used for local installation of OS or other software. A Dell 1U rack console which enables customers to mount a system administrator’s control station directly into a Dell rack without sacrificing rack space needed for servers and other peripherals.
Dell 16 Packaging Options Figure 56. Enclosure After Unpacking The M1000e comes from the factory on a pallet with components installed. The components must be taken out before it is installed in a rack. There are cardboard slings under the chassis to enable two people to lift it.
Dell Appendix A. Regulatory Certifications Please see the external Product Safety, EMC, and Environmental Datasheets on dell.com at: http://www.dell.com/regulatory_compliance_datasheets.
Dell Appendix B. Status Messages C.1 LCD Status Messages See the M1000e Configuration Guide and CMC Administrator Reference Guide for details. C.2 System Status Messages See the CMC Administrator Reference Guide for details.
Dell Appendix C. Additional Information Videos highlighting the major M1000e features are available on http://www.dell.com/html/us/products/pedge/poweredge_mseries_blade_videos/poweredge.html The PowerEdge M1000e Configuration Guide, Hardware Owner’s Manual and CMC Administrator Reference Guide each contain a wealth of additional information about the PowerEdge M1000e’s capabilities. The blade training material on dtt.us.dell.comdtt.us.dell.com is also a useful reference.
