Quadport Capable Hardware for the M1000e Modular Chassis A Dell Technical White Paper By Doug Wallingford, Bob Stevens, Michael Hobbs and Greg Casey
Quadport Capable Hardware for the M1000e Modular Chassis THIS WHITE PAPER IS FOR INFORMATIONAL PURPOSES ONLY, AND MAY CONTAIN TYPOGRAPHICAL ERRORS AND TECHNICAL INACCURACIES. THE CONTENT IS PROVIDED AS IS, WITHOUT EXPRESS OR IMPLIED WARRANTIES OF ANY KIND. © 2009 Dell Inc. All rights reserved. Reproduction of this material in any manner whatsoever without the express written permission of Dell Inc. is strictly forbidden. For more information, contact Dell.
Quadport Capable Hardware for the M1000e Modular Chassis Table of Contents Introduction: ................................................................................................................................................................. 2 The M1000e Chassis, Blades and IO Modules .......................................................................................................... 2 IO Fabric Banks ...........................................................................................
Quadport Capable Hardware for the M1000e Modular Chassis Introduction: The Dell M1000e Modular chassis brings a rich assortment of IO capabilities into a relatively small space. This document introduces QuadPort Mezzanine Cards and PowerConnect M6348 switch that further scales Network density. The M1000e Chassis, Blades and IO Modules The M1000e chassis supports up to 16 Half Height server blades, 8 full height server blades or a logical mix of the two. It also supports up to 6 IO Modules.
Quadport Capable Hardware for the M1000e Modular Chassis Half Height Blade Chassis Midplane IO Modules Fabric A Bank 1 LOM Fabric A Bank 2 Fabric B Bank 1 Fabric B Mezzanine Fabric B Bank 2 Fabric C Bank 1 Fabric C Mezzanine Fabric C Bank 2 FIGURE 1 IO ADAPTER TO IOM RELATIONSHIP Page 3
Quadport Capable Hardware for the M1000e Modular Chassis FIGURE 2 IOM FABRIC AND BANKS The Blades The M1000e chassis supports half and full height blades. The following chart shows the manifest of IO ports by blade type (Assuming Dual Port Mezzanine cards. Certain combinations of Mezzanine cards installed in certain blades do not fully scale their IO ports).
Quadport Capable Hardware for the M1000e Modular Chassis Half Height Blades The M600, M605 and M610 are half height form factor server blades. They feature a Broadcom 57XX dual port embedded Ethernet LOM’s driving fabric “A” and two mezzanine card slots supporting fabrics “B” and “C”. Figure 4 depicts the layout of optional mezzanine cards. FIGURE 4 Half HEIGHT BLADE 1. 2. 3. 4. 5.
Quadport Capable Hardware for the M1000e Modular Chassis Full Height Blades M710, M805 and M905 are Full height blades. The number of potential IO ports was doubled along with the form factor. There are 2 dual port Broadcom 5709 Gigabit Ethernet Lan On Motherboard controllers (LOMS) and 4 Mezzanine card slots on each full height blade. There are 2 mezzanine cards supporting Fabric “B” and 2 supporting Fabric “C”. Figure 5 depicts the layout of optional mezzanine cards. FIGURE 5 FULL HEIGHT BLADE 1. 2. 3.
Quadport Capable Hardware for the M1000e Modular Chassis Dual port and Quadport IOM’s The IO Modules are split into 2 types, Pass-through modules and Switches. We will focus on the PowerConnect M6348 Ethernet switch IOM. The M6348 is a 48 port switch with 32 internal ports meaning there are double the incoming or internal Ethernet ports compared to a dual port switch where standard dual port IOM’s have 16 internal ports.
Quadport Capable Hardware for the M1000e Modular Chassis Figure 7 is a logical drawing of a half height blade with a quadport Mezzanine card mapped to Bank 1 and Bank 2 switch IOM FIGURE 7 Quad port to IOM relationship IO Adapters All Dell blades feature embedded Ethernet Controllers with at least 2 discrete ports. These discrete ports are directly mapped to IOM’s installed in Fabric “A”, banks one and two. IOM slots “A1” and “A2” can only accept Ethernet IO Modules.
Quadport Capable Hardware for the M1000e Modular Chassis Technote: Some blades cannot support full IO capability in Fabric "C". Refer to the IO limitation chart in this document for details Mezzanine Card Enumeration Mezzanine cards are identified and ordered by: MEZZn_FAB_x. Where n is 1 to the maximum number of mezz card slots in the blade, and X is the fabric position of the mezz card, either “B” or “C”.
Quadport Capable Hardware for the M1000e Modular Chassis IO to IOM Port Ordering MAC and WWN Order For all IO adapters, port one maps to IOM bank one and port two maps to IOM bank two. Port one of the LOM or Mezzanine adapters will always be the lowest Physical Identifier and will scale with the port count. The Physical Identifier is either a MAC address for Ethernet or a WWN (World Wide Name) for FibreChannel.
Quadport Capable Hardware for the M1000e Modular Chassis Half Height Blade IO to IOM mapping Mapping for quadport components in half height blades uses the following formulas: (“N” being the slot the blade is installed into; 1-8.
Quadport Capable Hardware for the M1000e Modular Chassis FIGURE 10 HALF HEIGHT BLADE IO TO IOM MAPPING Page 12
Quadport Capable Hardware for the M1000e Modular Chassis Full Height Blade IO to IOM mapping Mapping for quadport capable components in full height blades uses the following formulas: (“N” being the slot the blade is installed into; 1-8.
Quadport Capable Hardware for the M1000e Modular Chassis Figure 11 shows the physical mapping of the ports for a full height blade installed in chassis slot 1 and two Quadport Mezzanine cards installed in Mezz2-Fab_B and Mezz4-Fab_B. The ports illustrated on the IOM are incoming ports, not external uplink ports.
Quadport Capable Hardware for the M1000e Modular Chassis IO to IOM Mapping Charts: Half Height Blades These charts list and illustrate the IO to IOM mapping for LOMs and Quad port mezzanine cards.
Quadport Capable Hardware for the M1000e Modular Chassis Blade 2 Device IOM A1 LOM 1 2 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 IOM A2 2 Fabric B1 Port 1 2 Fabric B1 Port 2 2 Fabric B1 Port 3 18 Fabric B1 Port 4 18 Fabric C1 Port 1 2 Fabric C1 Port 2 2 Fabric C1 Port 3 18 Fabric C1 Port 4 18 Blade 3 Device IOM A1 LOM 1 3 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 Fabric B1 Port 1 3 3 Fabric B1 Port 2 Fabric B1 Port 3 3 19 Fabric B1 Port 4 Fabric C1 Port 1 19 3 Fabric C1 Port 2 Fabr
Quadport Capable Hardware for the M1000e Modular Chassis Blade 4 Device IOM A1 LOM 1 4 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 IOM A2 4 Fabric B1 Port 1 4 Fabric B1 Port 2 4 Fabric B1 Port 3 20 Fabric B1 Port 4 20 Fabric C1 Port 1 4 Fabric C1 Port 2 4 Fabric C1 Port 3 20 Fabric C1 Port 4 20 Blade 5 Device IOM A1 LOM 1 5 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 Fabric B1 Port 1 5 5 Fabric B1 Port 2 Fabric B1 Port 3 5 21 Fabric B1 Port 4 Fabric C1 Port 1 21 5 Fabric C1 Port 2 Fabr
Quadport Capable Hardware for the M1000e Modular Chassis Blade 6 Device IOM A1 LOM 1 6 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 IOM A2 6 Fabric B1 Port 1 6 Fabric B1 Port 2 6 Fabric B1 Port 3 22 Fabric B1 Port 4 22 Fabric C1 Port 1 6 Fabric C1 Port 2 6 Fabric C1 Port 3 22 Fabric C1 Port 4 22 Blade 7 Device IOM A1 LOM 1 7 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 Fabric B1 Port 1 7 7 Fabric B1 Port 2 Fabric B1 Port 3 7 23 Fabric B1 Port 4 Fabric C1 Port 1 23 7 Fabric C1 Port 2 Fabr
Quadport Capable Hardware for the M1000e Modular Chassis Blade 8 Device IOM A1 LOM 1 8 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 IOM A2 8 Fabric B1 Port 1 8 Fabric B1 Port 2 8 Fabric B1 Port 3 24 Fabric B1 Port 4 24 Fabric C1 Port 1 8 Fabric C1 Port 2 8 Fabric C1 Port 3 24 Fabric C1 Port 4 24 Blade 9 Device IOM A1 LOM 1 9 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 Fabric B1 Port 1 9 9 Fabric B1 Port 2 Fabric B1 Port 3 9 25 Fabric B1 Port 4 Fabric C1 Port 1 25 9 Fabric C1 Port 2 Fabr
Quadport Capable Hardware for the M1000e Modular Chassis Blade 10 Device IOM A1 LOM 1 10 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 IOM A2 10 Fabric B1 Port 1 10 Fabric B1 Port 2 10 Fabric B1 Port 3 26 Fabric B1 Port 4 26 Fabric C1 Port 1 10 Fabric C1 Port 2 10 Fabric C1 Port 3 26 Fabric C1 Port 4 26 Blade 11 Device IOM A1 LOM 1 11 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 Fabric B1 Port 1 11 11 Fabric B1 Port 2 Fabric B1 Port 3 11 27 Fabric B1 Port 4 Fabric C1 Port 1 27 11 Fabric C
Quadport Capable Hardware for the M1000e Modular Chassis Blade 12 Device IOM A1 LOM 1 12 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 IOM A2 12 Fabric B1 Port 1 12 Fabric B1 Port 2 12 Fabric B1 Port 3 28 Fabric B1 Port 4 28 Fabric C1 Port 1 12 Fabric C1 Port 2 12 Fabric C1 Port 3 28 Fabric C1 Port 4 28 Blade 13 Device IOM A1 LOM 1 13 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 Fabric B1 Port 1 13 13 Fabric B1 Port 2 Fabric B1 Port 3 13 29 Fabric B1 Port 4 Fabric C1 Port 1 29 13 Fabric C
Quadport Capable Hardware for the M1000e Modular Chassis Blade 14 Device IOM A1 LOM 1 14 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 IOM A2 14 Fabric B1 Port 1 14 Fabric B1 Port 2 14 Fabric B1 Port 3 30 Fabric B1 Port 4 30 Fabric C1 Port 1 14 Fabric C1 Port 2 14 Fabric C1 Port 3 30 Fabric C1 Port 4 30 Blade 15 Device IOM A1 LOM 1 15 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 Fabric B1 Port 1 15 15 Fabric B1 Port 2 Fabric B1 Port 3 15 31 Fabric B1 Port 4 Fabric C1 Port 1 31 15 Fabric C
Quadport Capable Hardware for the M1000e Modular Chassis Blade 16 Device IOM A1 LOM 1 16 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 Fabric B1 Port 1 16 16 Fabric B1 Port 2 Fabric B1 Port 3 16 32 Fabric B1 Port 4 Fabric C1 Port 1 32 16 Fabric C1 Port 2 Fabric C1 Port 3 Fabric C1 Port 4 IOM A2 16 32 32 Page 23
Quadport Capable Hardware for the M1000e Modular Chassis IO to IOM Mapping Charts: Full Height Blades Blade 1 Device IOM A1 LOM 1 1 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 LOM 3 1 9 LOM 4 Fabric B1 Port 1 9 1 Fabric B1 Port 2 Fabric B1 Port 3 1 17 Fabric B1 Port 4 17 Fabric C1 Port 1 1 Fabric C1 Port 2 1 Fabric C1 Port 3 17 Fabric C1 Port 4 Fabric B2 Port 1 17 9 Fabric B2 Port 2 Fabric B2 Port 3 9 25 Fabric B2 Port 4 Fabric C2 Port 1 25 9 Fabric C2 Port 2 Fabric C2 Port 3 Fabric C2 P
Quadport Capable Hardware for the M1000e Modular Chassis Blade 2 Device IOM A1 LOM 1 2 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 LOM 3 2 10 LOM 4 Fabric B1 Port 1 10 2 Fabric B1 Port 2 Fabric B1 Port 3 2 18 Fabric B1 Port 4 18 Fabric C1 Port 1 2 Fabric C1 Port 2 2 Fabric C1 Port 3 18 Fabric C1 Port 4 Fabric B2 Port 1 18 10 Fabric B2 Port 2 Fabric B2 Port 3 10 26 Fabric B2 Port 4 Fabric C2 Port 1 26 10 Fabric C2 Port 2 Fabric C2 Port 3 Fabric C2 Port 4 IOM A2 10 26 26 Page 25
Quadport Capable Hardware for the M1000e Modular Chassis Blade 3 Device IOM A1 LOM 1 3 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 LOM 3 3 11 LOM 4 Fabric B1 Port 1 11 3 Fabric B1 Port 2 Fabric B1 Port 3 3 19 Fabric B1 Port 4 19 Fabric C1 Port 1 3 Fabric C1 Port 2 3 Fabric C1 Port 3 19 Fabric C1 Port 4 Fabric B2 Port 1 19 11 Fabric B2 Port 2 Fabric B2 Port 3 11 27 Fabric B2 Port 4 Fabric C2 Port 1 27 11 Fabric C2 Port 2 Fabric C2 Port 3 Fabric C2 Port 4 IOM A2 11 27 27 Page 26
Quadport Capable Hardware for the M1000e Modular Chassis Blade 4 Device IOM A1 LOM 1 4 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 LOM 3 4 12 LOM 4 Fabric B1 Port 1 12 4 Fabric B1 Port 2 Fabric B1 Port 3 4 20 Fabric B1 Port 4 20 Fabric C1 Port 1 4 Fabric C1 Port 2 4 Fabric C1 Port 3 20 Fabric C1 Port 4 Fabric B2 Port 1 20 12 Fabric B2 Port 2 Fabric B2 Port 3 12 28 Fabric B2 Port 4 Fabric C2 Port 1 28 12 Fabric C2 Port 2 Fabric C2 Port 3 Fabric C2 Port 4 IOM A2 12 28 28 Page 27
Quadport Capable Hardware for the M1000e Modular Chassis Blade 5 Device IOM A1 LOM 1 5 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 LOM 3 5 13 LOM 4 Fabric B1 Port 1 13 5 Fabric B1 Port 2 Fabric B1 Port 3 5 21 Fabric B1 Port 4 21 Fabric C1 Port 1 5 Fabric C1 Port 2 5 Fabric C1 Port 3 21 Fabric C1 Port 4 Fabric B2 Port 1 21 13 Fabric B2 Port 2 Fabric B2 Port 3 13 29 Fabric B2 Port 4 Fabric C2 Port 1 29 13 Fabric C2 Port 2 Fabric C2 Port 3 Fabric C2 Port 4 IOM A2 13 29 29 Page 28
Quadport Capable Hardware for the M1000e Modular Chassis Blade 6 Device IOM A1 LOM 1 6 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 LOM 3 6 14 LOM 4 Fabric B1 Port 1 14 6 Fabric B1 Port 2 Fabric B1 Port 3 6 22 Fabric B1 Port 4 22 Fabric C1 Port 1 6 Fabric C1 Port 2 6 Fabric C1 Port 3 22 Fabric C1 Port 4 Fabric B2 Port 1 22 14 Fabric B2 Port 2 Fabric B2 Port 3 14 30 Fabric B2 Port 4 Fabric C2 Port 1 30 14 Fabric C2 Port 2 Fabric C2 Port 3 Fabric C2 Port 4 IOM A2 14 30 30 Page 29
Quadport Capable Hardware for the M1000e Modular Chassis Blade 7 Device IOM A1 LOM 1 7 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 LOM 3 7 15 LOM 4 Fabric B1 Port 1 15 7 Fabric B1 Port 2 Fabric B1 Port 3 7 23 Fabric B1 Port 4 23 Fabric C1 Port 1 7 Fabric C1 Port 2 7 Fabric C1 Port 3 23 Fabric C1 Port 4 Fabric B2 Port 1 23 15 Fabric B2 Port 2 Fabric B2 Port 3 15 31 Fabric B2 Port 4 Fabric C2 Port 1 31 15 Fabric C2 Port 2 Fabric C2 Port 3 Fabric C2 Port 4 IOM A2 15 31 31 Page 30
Quadport Capable Hardware for the M1000e Modular Chassis Blade 8 Device IOM A1 LOM 1 8 IOM B1 IOM C1 IOM C2 IOM B2 LOM 2 LOM 3 8 16 LOM 4 Fabric B1 Port 1 16 8 Fabric B1 Port 2 Fabric B1 Port 3 8 24 Fabric B1 Port 4 24 Fabric C1 Port 1 8 Fabric C1 Port 2 8 Fabric C1 Port 3 24 Fabric C1 Port 4 Fabric B2 Port 1 24 16 Fabric B2 Port 2 Fabric B2 Port 3 16 32 Fabric B2 Port 4 Fabric C2 Port 1 32 16 Fabric C2 Port 2 Fabric C2 Port 3 Fabric C2 Port 4 IOM A2 16 32 32 Page 31
Quadport Capable Hardware for the M1000e Modular Chassis Port Count Limitations by Blade Model Due to a blade chipset limitation not all blades support all mezzanine cards installed in fabric C slots on the system board. The following Quad port Mezzanine cards are affected: • • Intel Quadport Gigabit ET Mezzanine card Broadcom NetExtreme II 5709 Quad Port Ethernet Mezzanine Card The following chart lists the limitations.
