M-Series Blade I/O Guide I/O Connectivity Options for the Dell PowerEdge M1000e Blade Enclosure April 17, 2013 Send feedback to: BladeInterconnects@dell.
Contents Quick Reference Guides - Ethernet switching - Fibre Channel switching - Cisco and Infiniband switching Infiniband Blades 3 4 5 Converged Ethernet Blades - 10/40GbE Switch – MXL - 10GbE Plug & Play – PowerEdge M I/O - 10GbE Basic – M8024-k - Ethernet / FC Switch – M8428-k - 10Gb Pass-Through - Cisco Nexus Blade – B22DELL FEX 9 11 13 15 17 19 1Gb Ethernet Blades - 1GbE High-density – M6348 - 1GbE Basic – M6220 - 1GbE Pass-Through - Cisco Catalyst Blades 23 25 27 28 Fibre Channel Blades - 16Gb s
Blade Interconnects M-Series Blade I/O Guide Transform your Dell M1000e blade server enclosure. Ethernet Switching Models Overview MXL I/O aggregator M8024-k M8428-k M6348 M6220 10Gb pass-through 10/40GbE Switch 10GbE Plug and Play 10GbE Basic Ethernet / FC Switch 1GbE High-density 1GbE Basic Direct connection High performance blade provides maximum throughput, flexibility, and iSCSI / FCoE convergence.
Blade Interconnects M-Series Blade I/O Guide Transform your Dell M1000e blade server enclosure. Fibre channel switching Models Overview - Brocade M6505 Brocade M5424 High-performance 16Gb Switch Advanced 8Gb Switch Transform SAN connectivity with maximum Connect directly to the Fibre Channel throughput and advanced management SAN, by-passing any external switches and features for virtualized environments. reducing cables, optics, and management.
Blade Interconnects M-Series Blade I/O Guide Transform your Dell M1000e blade server enclosure. Cisco Models 1 product / 3 versions B22DELL FEX 10GbE Nexus Extender Overview Acts as remote line card of the parent Nexus switch fabric. (Models 5548P, 5548UP, or 5596P only) 3130X 3130G 3032 Catalyst Blade Switches Basic interconnects for Dell M1000e customers who want to maintain a Cisco fabric from server to core.
M-Series I/O Modules 6 Converged Ethernet Fibre Channel MXL PowerEdge M I/O Aggregator M8024-k M8428-k 10 Gb Pass-Through Cisco B22DELL FEX Brocade M6505 Brocade M5424 FC SAN Module Pass Through FC8/4 1Gb Ethernet InfiniBand M6348 M6220 1Gb Pass-Through Cisco Catalyst Blade Mellanox M4001F Mellanox M4001T Dell M-Series Blade I/O Guide
Ethernet Blade I/O Modules Product Portfolio MXL External Ports: (2) 10/40GbE QSFP+ (2) Optional Modules 40Gb QSFP+, SFP+, or Base-T 1Gb Performance / Bandwidth 10Gb Cisco Nexus B22DELL (FEX) External Ports: (8) SFP+ 10GbE M8428-k External Ports: (8) SFP+ 10GbE (4) 8Gb FC SAN M8024-k I/O Aggregator External Ports: (4) SFP+ 1/10GbE (1) Optional Module External Ports: (2) QSFP+ ports in 4x10GbE mode (2) Optional Modules SFP+, CX4, or Base-T QSFP+, SFP+, or Base-T FCF Integration w/ ToR FCoE Tr
Converged Ethernet 10/40Gb Switch MXL 8 Dell M-Series Blade I/O Guide 10Gb Plug & Play M-IOA 10Gb Basic M8024-k Ethernet & FC Switch M8428-k 10Gb Cisco PassB22DELL FEX Through
MXL – 10/40GbE blade Converged • Industry leading 56 port design: – 32x 10Gb internal server ports – Up to 6 external 40Gb ports – Up to 24 external 10Gb ports (6 QSFP+ ports with breakout cables) • Two FlexIO bays enable choice (Modules can be different) – 2-port 40GbE QSFP+ module (can convert to 8-port 10GbE SFP+ using breakout cables) – 4-port 10GbE SFP+ module Two FlexIO Modules – 4-port 10GBASE-T module (If running Base-T module then second IO slot must be of different type due to power constrain
MXL - 10/40GbE blade Converged FlexIO modules do not have to be the same Adapters Optical Transceivers SFP+ 10Gb: SR, LR SFP 1GbE: SX, LX 4 port SFP+ Module 11G -Broadcom 57712-k -Brocade BR1741M-k -Intel X520-x/k -QLogic QME8242-k SFP to RJ45 converter 1000Base-T (only capable of 1Gbps) SFP+ Direct Attach Cable (Twinax) (0.
PowerEdge M I/O Aggregator Converged Plug & Play • Easy Deployment – Simplified layer 2 connectivity (no spanning tree) – Faster Deployment: All VLANs on all ports with the option to set VLANs – No touch DCB and no touch FCoE › DCB and FCoE settings detected from top of rack switch through DCBx protocol • Simple GUI Integrated into Chassis Management Controller (CMC) (Note: CMC GUI will not function if the IOA is stacked. IOA must be managed through CLI when stacked.
PowerEdge M I/O Aggregator FlexIO modules do not have to be the same Adapters Optical Transceivers SFP+ 10Gb: SR, LR SFP 1GbE: SX, LX 4 port SFP+ Module 11G -Broadcom 57712-k -Brocade BR1741M-k -Intel X520-x/k -QLogic QME8242-k SFP to RJ45 converter 1000Base-T (only capable of 1Gbps) SFP+ Direct Attach Cable (Twinax) (0.
M8024-k • Fully modular full wire-speed 10GbE managed Layer 2/3 Ethernet switching • Converged • Supports DCB (protocols PFC and DCBx) • FCoE Transit Switch via FIP Snooping Bridge (not supported in Simple Switch Mode) • Stack up to 6 devices using SFP+ fixed ports or SFP+ 2-port 10GBASE-T One FlexIO Module module (not supported in Simple Switch Mode) • 24 port design features: – 16 internal 10Gb server ports – 4 integrated external SFP+ ports (multi-speed 1/10Gb) – Up to 4 additional external ports vi
M8024-k Converged Cables CX4 cables for 10GbE Uplink Uplinks Adapters 11G -Broadcom 57712-k -Brocade BR1741M-k -Intel X520-x/k -QLogic QME8242-k Uplinks Cables RJ45 / Cat6a 12G -Broadcom 57810S-k -Brocade BR1741M-k -Intel X520-x/k -Qlogic QME8262-k Uplinks 10GbE CX4 Copper Module (10Gb only) The M8024-k switch supports connectivity to 10Gb-KR adapters, all of which are notated with “-k.
M8428-k Converged Converged Ethernet & Fibre Channel Switch • Dell 10GbE Converged Network Switch – DCB compliant design accommodates both NIC and Fibre Channel Over Ethernet I/O • Single wide blade I/O module supporting all 10GbE capable M1000e fabric bays • Robust I/O bandwidth solution with 28 active fixed ports – 16 internal server ports – 8 external 10GbE SFP+ uplinks (10Gb speed only) › Brocade Short-wave optical transceivers / fiber › Brocade Long-wave optical transceivers / fiber › Brocade D
M8428-k Converged Adapters 11G -Broadcom 57712-k -Brocade BR1741M-k -Intel X520-x/k -QLogic QME8242-k 8 ports 10Gb Ethernet (DCB) Brocade Optical Transceivers Short Wave, Multi-Mode SFP+ Optics Long Wave, Multi-Mode SFP+ Optics 12G -Broadcom 57810S-k -Brocade BR1741M-k -Intel X520-x/k -Qlogic QME8262-k Supports connectivity to 10Gb-KR adapters, all of which are notated with “-k.” It does not provide connectivity to legacy 10Gb-XAUI NICs/CNAs 1Gb Ethernet mezzanine cards and LOMs are not supported.
Converged 10Gb Ethernet Pass Through -k • 16 ports correspond to 16 server blades – Only supports –k mezz cards • 16 external 10GbE SFP+ ports – Supports 10Gb connections ONLY • Supports DCB/CEE and FCoE – Connect to top-of-rack FCoE switches and Converged Network Adapters (CNA’s) in individual blades • Transparent connection between blade servers and external LAN 17 Dell M-Series Blade I/O Guide
10Gb Ethernet Pass Through -k Converged Adapters 11G -Broadcom 57712-k -Brocade BR1741M-k -Intel X520-x/k -QLogic QME8242-k 12G -Broadcom 57810S-k -Brocade BR1741M-k -Intel X520-x/k -Qlogic QME8262-k 10Gb Optical Transceivers SR & LR Supports connectivity to 10Gb-KR adapters, all of which are notated with “-k.” It does not provide connectivity to legacy 10Gb-XAUI NICs/CNAs 1Gb Ethernet mezzanine cards and LOMs are not supported.
Cisco Nexus Blade Converged B22DELL Fabric Extender (FEX) • Cisco 10GbE offering for the Dell M1000e Blade System – The 16 internal 10Gb or 1Gb ports and 8 external 10Gb ports enables customers to connect via 10GbE to a Cisco Nexus 5500 series Top of Rack switch • The B22DELL FEX is only supported with these specific Cisco Nexus models: – Cisco Nexus 5548P, 5548UP, 5596P – Cisco Nexus 6001, 6004 It cannot connect to Cisco Nexus 5010, 5020, 2000 or 7000 series switches.
Cisco Nexus Blade Converged B22DELL Fabric Extender (FEX) Adapters 11G -Broadcom 57712-k -Brocade BR1741M-k -Intel X520-x/k -QLogic QME8242-k This is not a usable port. There is no management serial port on the B22DELL (external nor internal). The B22DELL is managed from the Cisco Nexus top of rack switch.
Comparison of Converged Blade options Dell MXL Switch Dell PowerEdge M I/O Aggregator Cisco Nexus B22DELL FEX Dell M8024-k Dell M8428-k 10/40GbE Switch 10GbE Plug & Play 10GbE Extender 10GbE Basic Ethernet / FC 32 (10GbE) 32 (10GbE) 16 (10GbE) 16 (10GbE) 16 (10GbE) External 40G Ports (QSFP+) 2 Fixed – 6 Total (Note: QSFP+ ports run in breakout mode 4x10GbE only) None None None External 10G Ports Up to 24 (16 per LAG) Up to 8 8 8 Model Overview Server Ports Supported Flex I/O Expa
1Gb Ethernet 1/10Gb High-density M6348 22 Dell M-Series Blade I/O Guide 1/10Gb Basic M6220 1Gb Cisco PassCatalyst Through 3130X/G & 3032
M6348 1/10GbE High-density 1GbE copper with 10GbE uplinks • Managed Layer 2/3 Gigabit Ethernet switch for M1000e blade enclosure • Industry leading port availability • 32 internal (server) GbE ports; offering support of up to two ports per blade mezz card or Select Network Adapter (i.e.
1/10GbE M6348 Adapters Works with all 1Gb Mezzanine cards and LOMs. Optimal use is with quad-port 1Gb adapters. Cables CAT 5 Functions with all 10Gb Mezzanine cards and Select Network Adapters with the exception of the: Qlogic 8242-k, 8262-k, and Brocade BR1741M-k. Dual port Mezzanine cards or LOMs/ Select Network Adapters will function and are fully supported with this IO module.
M6220 1/10GbE Basic 1GbE copper with FlexIO & 10GbE uplinks • Gigabit Ethernet Layer 2/3 Switch • Optional 10Gb uplinks & resilient stacking • IPv6 support 4 x fixed 10/100/1000Mb (RJ-45) • 24 port switch – 16 internal ports corresponding to 16 blade servers (1Gbps) – 4 external fixed RJ-45 connections (10/100/1000Mbps) – 2 FlexIO bays for: 4 external 10Gbps uplink ports – or – 2 external 10Gbps uplink ports and 2 external stacking ports • Same software image features as PowerConnect 6224/6248 switches
1/10GbE M6220 Adapters Uplinks Works with all 1Gb Mezzanine cards and LOMs. Functions with all 10Gb Mezzanine cards and Select Network Adapters with the exception of the Qlogic 8242-k, 8262-k, and Brocade BR1741M-k. Quad port GbE Mezzanine cards or LOMs will function and are fully supported with this IO module. In such configurations, only half of the card’s ports will be used since the switch only has one internal port per adapter.
Gb Ethernet Pass-Through 1GbE Adapters Cables Works with all 1Gb Mezzanine cards and LOMs. CAT 5 Functions with all 10Gb Mezzanine cards and Select Network Adapters with the exception of the Qlogic 8242-k, 8262-k, and Brocade BR1741M-k. 1GbE Pass Through Module Quad port GbE Mezzanine cards or LOMs will function and are fully supported with this IO module. In such configurations, only half of the card’s ports will be used since the switch only has one internal port per adapter.
Cisco Catalyst Blade Switches 1/10GbE Cisco Catalyst 3130X – 1/10Gb Switch • Two 10GbE uplinks (X2 – CX4, SR, LRM optics) • Four fixed 1GbE uplinks - 4xRJ45 • Virtual Blade Switch interconnect enabled Cisco Catalyst 3130G – GbE Switch • Up to eight GbE uplinks – fixed 4xRJ45 + up to four optional 1GbE SFPs (copper or optical) • Virtual Blade Switch interconnect enabled X Cisco Catalyst 3032 -- Entry Level GbE Switch • Up to eight GbE uplinks - 4xRJ45 & up to 4 SFPs (Cisco copper or optical products only
Cisco Catalyst Blade Switches Adapters 1/10GbE Stacking Ports (supported on 3130G & 3130X models ONLY) 2x 64Gb StackWise Ports (0.5m, 1m, 3m cables purchased separately for factory-installed blade switch) Works with all 1Gb Mezzanine cards and LOMs. Functions with all 10Gb Mezzanine cards and Select Network Adapters with the exception of the Qlogic 8242-k, 8262-k, and Brocade BR1741M-k. Quad port GbE Mezzanine cards or LOMs will function and are fully supported with this IO module.
Fibre Channel See also the M8428-k in the Converged Ethernet section 30 Dell M-Series Blade I/O Guide 16Gb M6505 8Gb M5424 8Gb SAN Module 8Gb PassThrough
M-Series Fibre Channel Comparison 31 8/4Gbps M5424 M6505 FC SAN Module 8Gbps FC SAN Switch 16Gbps FC SAN Switch Model Choices 12-port 12-port, 24-port 24-port (Ent Perf Pk) 12-port, 24-port 24-port (Ent Perf Pk) Scalable Ports Upgrade +12-ports +12-ports (for 12-port SKU) +12-ports (for 12-port SKU) Factory pre-installed SFP+ Transceivers 2 of 8 2 of 8 - 4 of 8 - 8 of 8 2 of 8 - 4 of 8 - 8 of 8 Connect to Brocade FC SAN NPIV Brocade Switch (default) Access Gateway (selectable) Access
Brocade M6505 Fibre Channel 16Gb switch • 16 Gb Fibre Channel ports* Secondary Management port 32 Dell M-Series Blade I/O Guide 24 Fibre Channel ports o Up to 16 internal 16/8Gb server ports* o Up to 8 external 16/8/4Gb SAN ports** *16Gb capacity functions with newer M1000e chassis using 1.1 mid-plane. Switch will function on older M1000e chassis at 8Gb speed only.
Brocade M6505 Fibre Channel 16Gb switch Adapters 11G - Qlogic QME2572 FC8 - Emulex LPe1205-M FC8 12G - Qlogic QME2662 FC16 - Emulex LPm16002 FC16 - Qlogic QME2572 FC8 - Emulex LPe1205-M FC8 *16Gbps speeds on internal ports require the enhanced midplane (1.1). The switch will autonegotiate to 8Gbps on internal ports when using the original midplane (1.0). Does not support 4Gb Mezzanine cards.
Brocade M5424 8Gb switch • 8/4 Gbps Fibre Channel SAN solution • Provides up to 24 8/4Gb FC ports – Up to 16 internal 8/4Gb server ports – Up to 8 external 8/4Gb SAN ports* *For connection to storage devices and/or other FC switches only • One management console port • Configurable as Brocade full fabric switch or Access Gateway Mode (NPIV) for multi-vendor interoperability • Auto-negotiates between 4Gbps and 8Gbps based on linked mezzanine cards and top-of-rack switches • Supports future FOS features an
Brocade M5424 Fibre Channel 8Gb switch Adapters Cables Brocade Transceivers Brocade SWL or LWL 8Gb SFP+ Optics Brocade SWL, LWL or ELWL 4Gb SFP+ Optics 11G - Qlogic QME2572 - Emulex LPe1205 Note: Requires SFP LC connector 12G - Qlogic QME2572 - Emulex LPe1205-M - Qlogic QME2662 (at FC8 speeds) - Emulex LPm16002 (at FC8 speeds) FC4 Mezzanine cards are also supported with this switch at 4Gbps.
Dell 8/4Gbps FC SAN Module Fibre Channel • Base model provides 12 active ports with two external SAN 8Gb SWL optical transceivers • Scalable to 24 active ports using 12-port pay-as-you-grow option kit (includes two additional 8Gb SWL SFP+ transceivers) • Add additional 8Gb SWL SFP+ transceivers for up to 8 external SAN ports • Ideal scalability for data centers deploying increasingly more blade enclosures while requiring FC connectivity • Device is in Access Gateway Mode (NPIV) for multi-vendor interopera
Dell 8/4Gbps FC SAN Module Fibre Channel SimpleConnect for SAN Best solution for modular SAN connectivity • Based on industry-standard NPIV (N-port ID Virtualization) • Combines pass-through simplicity for connecting each server to any SAN fabric with beneficial I/O and cable aggregation • Helps solve interoperability issues with heterogeneous fabrics, i.e. mixed Brocade, Cisco, etc.
Dell 8/4Gbps FC SAN Module Fibre Channel Adapters Mezzanine cards Cables Brocade Transceivers SWL 8Gb SFP+ Optics LWL 8Gb SFP+ Optics 11G - Qlogic QME2572, - Emulex LPe1205 12G - Qlogic QME2572 - Emulex LPe1205-M FC4 Mezzanine cards are also supported with this switch at 4Gbps.
Dell 8/4Gbps FC Pass-Through • 16 ports correspond to 16 server blades • 8, 4, or 2 Gbps connections • Transparent connection between SAN and server blades • As an alternative to this FC8 Pass-Through, the Dell 8/4Gbps FC SAN Module (NPIV aggregator) which provides the simplicity of a pass-through with the aggregation/redundancy benefits of a switch 39 Dell M-Series Blade I/O Guide Fibre Channel
Dell 8/4Gbps FC Pass-Through Fibre Channel Adapters 11G - Qlogic QME2572, - Emulex LPe1205 12G - Qlogic QME25722 - Emulex LPe1205-M Cables *FC4 Mezzanine cards will function with this passthrough. Doing so will cause the pass-through to run at 4Gbps rather than the full-capability 8Gbps.
InfiniBand 56Gb M4001F FDR 41 Dell M-Series Blade I/O Guide 40Gb M4001T FDR10
Mellanox Blades Infiniband • For high performance computing (HPC) & low latency applications • Available in redundant switch configuration • Full non-blocking throughput Models M4001F M4001T Speed FDR FDR10 56Gbps 40Gbps Data rate Total ports 42 Dell M-Series Blade I/O Guide 32 (16 internal and 16 external)
Mellanox M4001F & M4001T Infiniband Adapters Cables Cables QSFP Active Optical QSFP Active Optical QSFP Passive Copper QSFP Passive Copper or Combine the with Mellanox ConnectX3 InfiniBand mezz cards for end to end FDR or FDR10. or QDR ConnectX3 and QDR ConnectX2 are fully supported with these switches. They will connect at QDR speeds. More details in Adapter Portfolio section Designed for I/O bays Not a Management Port. Debug port only B2 B1 C2 C1 B1/B2 Not a Management Port.
Fabric Topologies Find more topologies and guides here: • EqualLogic Compatibility Matrix - http://en.community.dell.com/techcenter/storage/w/wiki/2661.equallogic-compatibility-matrix.aspx • EqualLogic Configuration Guide - http://en.community.dell.com/techcenter/storage/w/wiki/2639.equallogic-configuration-guide.aspx • Rapid EqualLogic Configuration Portal - http://en.community.dell.com/techcenter/storage/w/wiki/3615.rapid-equallogic-configuration-portal-by-sis.
FCoE transit Direct traffic to the Top-of-Rack via FIP Snooping Bridge Topology Diagram Topology / Configuration Topology Fabric Inside Chassis: FCoE Blade models: MXL, IOA, M8024-k Top-of-Rack switch: Nexus 5K Configuration • All FCoE traffic moves from the adapters, to the IOM, then to the Top-of-Rack switch • FC is broken out at the Top-ofRack switch and moves to the SAN or directly to the storage array 45 Dell M-Series Blade I/O Guide SAN A LAN LAG SAN B LAG FCoE LAN FC Inside Chassis FCoE fr
Fibre Channel Breakout at Edge of Chassis Topology Diagram Topology / Configuration Topology Fabric Inside Chassis: FCoE Blade model: M8428-k Configuration FCoE inside chassis (from adapter to blade switch) and Native FC outside the chassis LAN LAN SAN A SAN B LAG LAG LAN FC Inside Chassis FCoE from Adapter to Blade Switch 46 Dell M-Series Blade I/O Guide
iSCSI and LAN Converged Storage Traffic Topology Diagram Topology / Configuration LAN Topology Fabric Inside Chassis: Converged iSCSI Blade models: MXL or IOA Top-of-Rack switch: S4810, S4820T Storage: iSCSI External Array VLT LAG LAG Configuration Converged iSCSI traffic (LAN and iSCSI) up to the Top-of-Rack switch iSCSI & LAN LAN iSCSI 47 Dell M-Series Blade I/O Guide Inside Chassis iSCSI and LAN from Adapter to Blade switch
Storage Blade with Optional External Array Topology Diagram Topology / Configuration Configuration • Converged iSCSI to the blades and up to the Top-of-Rack switch • MXLs are stacked, so that array to array traffic stays inside the chassis LAN VLT Stacked iSCSI & LAN LAN iSCSI Dual Port Adapter 48 Dell M-Series Blade I/O Guide LAG LAG M1000e chassis Topology Fabric Inside Chassis: Converged iSCSI Blade model: MXL Top-of-Rack switch: S4810, S4820T Storage: PS4410 storage blade Optional Storage: Equ
Cross Chassis Stacking Topology / Configuration Topology Diagram VLT Topology Blade models: MXL, M8024-k, M6348, M6248, IOA (using CLI) Configuration Blade switches are stacked vertically so that there are two independent stacking rings. Switches on the left of the chassis form a ring and switches on the right side of the chassis form a ring. Independent stack rings allow each ring to be upgraded independently. Note that IOA is limited to a two unit stack.
Benefits of Stacking Topology Diagram Benefits of Stacking • Single point of management for each stack • Increase of East/West traffic so less traffic goes to Top of Rack – Save on Top of Rack ports – Reduced Cables – Less Congestion at Top of Rack • Use blade switches as the aggregation layer eliminating the need for Top of Rack switches Stack and Connect Directly to Core VLT Core Core Topology / Configuration Topology Stacked blade switches connected directly to the Network Core switches Stacked IOM
Automation & Management 51 Dell M-Series Blade I/O Guide
Enhanced management of the M1000e Simplifying blade server and I/O connectivity The M1000e blade enclosure helps reduce the cost and complexity of managing computing resources with innovative management features. The Chassis Management Controller (CMC) is an integrated hardware module with embedded system management. The simplified software interface, pictured below, gives administrators greater control of the chassis components and automates tasks to improve monitoring and management.
FlexAddress Plus Intelligent Network Addressing • The CMC offers simple interface for enabling FlexAddress by chassis, by slot, or by fabric, assigning WWN/MAC values in place of factory-assigned WWN/MAC • User-configurable enablement of iSCSI MAC, Ethernet MAC, and/or WWN Persistence which allows blades to be swapped without affecting SAN Zoning, iSCSI zoning, or any MAC-dependent functions • FlexAddress Plus SD card provisioned with unique pool of 3136 MACs/WWNs Original hardwareassigned MACs 53 Dell M
SimpleConnect for LAN Easy deployment feature What is SimpleConnect? • Feature included on all PowerConnect blade switches (M8024-k/M6348/M6220); “SimpleConnect” (locked) models also available (M8024S/M6348S/M6220S) • Aggregate traffic from multiple downlinks to one or more uplinks by mapping internal (server) NIC ports to external (top-of-rack) switch ports • Based on port aggregation industry standards Benefits of Simple Switch Mode? • Ease of deployment/management for in-chassis blade switches • Ease of
Fabrics and Port Mapping 55 Dell M-Series Blade I/O Guide
PowerEdge M1000e Chassis Fabrics and Capabilities Fabric A1 Reserved for 1/10GbE LOMs or Select Network Adapters Fabric A2 A1 A2 Fabric B1 1/10/40 GbE, 4/8/16Gb FC, 20/40/56Gb IB Fabric C1 1/10/40 GbE, 4/8/16Gb FC, 20/40/56Gb IB Reserved for 1/10GbE LOMs or Select Network Adapters Fabric B2 B1 B2 C1 C2 1/10/40 GbE, 4/8/16Gb FC, 20/40/56Gb IB Fabric C2 1/10/40 GbE, 4/8/16Gb FC, 20/40/56Gb IB Colors chosen to facilitate whiteboard discussions. The capabilities of the enhanced midplane (1.
M-Series Blade I/O Fabrics C1 Quarter Height C2 OR B1 B2 Quarter Height Blades One dual port LOM •IOM with 32 internal ports (M6348 or Dell Force10 MXL) is needed to connect all LOM ports on all blades •2 x 32 port IOMs needed to connect the 2 LOM ports on each blade One fabric B OR fabric C mezzanine card C1 C2 Half Height B1 B2 Half Height Blades One Select Network Adapter or LOM One fabric B mezzanine card One fabric C mezzanine card C1 C2 B1 Full Height B2 C1 C2 B1 B2 57 Dell Inc.
M1000e Midplane Mapping and Capabilities Right side of chassis M1000e Mid-plane Fabric A LOM 2 lanes 2 lanes Fabric B 4 lanes 4 lanes Mezzanine Fabric C 4 lanes 4 lanes Front of Chassis Mezzanine Fabric A2 Ethernet I/O Module Fabric B2 I/O Module Fabric C2 I/O Module Fabric C1 I/O Module Half-Height Blade Slot (16 of 16) Fabric A LOM 2 lanes 2 lanes Fabric B 4 lanes 4 lanes Mezzanine Fabric C Mezzanine 4 lanes 4 lanes 8.
I/O Fabric Architecture for Half-Height Blades Right side of chassis CPU 1 Fabric A LOM 8 lane PCIe Mezzanine 8 lane PCIe Fabric A2 Ethernet I/O Module Fabric B Fabric B2 I/O Module Fabric C Mezzanine Front of Chassis CPU 2 4-8 lane PCIe M1000e Mid-plane Fabric C2 I/O Module Fabric C1 I/O Module Half-Height Blade Server (16 of 16) CPU 1 CPU 2 4-8 lane PCIe Fabric A LOM 8 lane PCIe Fabric B 8 lane PCIe Fabric B1 I/O Module Mezzanine Fabric A1 Fabric C Mezzanine 8.
Port Mapping of Half Height blades with Dual Port Adapters to IOMs with 16 or 32 Internal Ports IOM ports mapped to half height blade slots Slot1 Slot2 Slot3 Slot4 Slot5 Slot6 Slot7 Slot8 IOM1 P1 IOM1 P2 IOM1 P3 IOM1 P4 IOM1 P5 IOM1 P6 IOM1 P7 IOM1 P8 IOM2 P1 IOM2 P2 IOM2 P3 IOM2 P4 IOM2 P5 IOM2 P6 IOM2 P7 IOM2 P8 Slot9 Slot10 Slot11 Slot12 Slot13 Slot14 Slot15 Slot16 IOM1 P9 IOM1 P10 IOM1 P11 IOM1 P12 IOM1 P13 IOM1 P14 IOM1 P15 IOM1 P16 IOM2 P9 IOM2 P10 IOM2 P11 I
Port Mapping of Half Height blades with Quad Port Adapters to IOMs with 32 Internal Ports IOM ports mapped to half height blade slots Slot1 Slot2 Slot3 Slot4 Slot5 Slot6 Slot7 Slot8 IOM1 P1 IOM1 P2 IOM1 P3 IOM1 P4 IOM1 P5 IOM1 P6 IOM1 P7 IOM1 P8 IOM2 P1 IOM2 P2 IOM2 P3 IOM2 P4 IOM2 P5 IOM2 P6 IOM2 P7 IOM2 P8 IOM1 P17 IOM1 P18 IOM1 P19 IOM1 P20 IOM1 P21 IOM1 P22 IOM1 P23 IOM1 P24 IOM2 P17 IOM2 P18 IOM2 P19 IOM2 P20 IOM2 P21 IOM2 P22 IOM2 P23 IOM2 P24 Slot9 Slot10 Slo
I/O Fabric Architecture for Full-Height Blades Right side of chassis CPU 1 Fabric A LOM 8 lane PCIe Mezzanine 8 lane PCIe Fabric A2 Ethernet I/O Module Fabric B Fabric B2 I/O Module Fabric C Mezzanine Front of Chassis CPU 2 4-8 lane PCIe M1000e Mid-plane Fabric C2 I/O Module Fabric C1 I/O Module CPU 3 CPU 4 4-8 lane PCIe Fabric A LOM 8 lane PCIe Fabric B 8 lane PCIe Fabric B1 I/O Module Mezzanine Fabric A1 Fabric C Mezzanine 8.
Port Mapping of Full Height blades with Dual Port Adapters to IOMs with 16 or 32 Internal Ports IOM ports mapped to full height blade slots Slot1 Slot2 Slot3 Slot4 Slot5 Slot6 Slot7 Slot8 IOM1 P1 IOM1 P2 IOM1 P3 IOM1 P4 IOM1 P5 IOM1 P6 IOM1 P7 IOM1 P8 IOM2 P1 IOM2 P2 IOM2 P3 IOM2 P4 IOM2 P5 IOM2 P6 IOM2 P7 IOM2 P8 IOM1 P9 IOM1 P10 IOM1 P11 IOM1 P12 IOM1 P13 IOM1 P14 IOM1 P15 IOM1 P16 IOM2 P9 IOM2 P10 IOM2 P11 IOM2 P12 IOM2 P13 IOM2 P14 IOM2 P15 IOM2 P16 • All six IOMs
Port Mapping of Full Height blades with Quad Port Adapters to IOMs with 32 Internal Ports IOM ports mapped to full height blade slots Slot1 Slot2 Slot3 Slot4 Slot5 Slot6 Slot7 Slot8 IOM1 P1 IOM1 P2 IOM1 P3 IOM1 P4 IOM1 P5 IOM1 P6 IOM1 P7 IOM1 P8 IOM2 P1 IOM2 P2 IOM2 P3 IOM2 P4 IOM2 P5 IOM2 P6 IOM2 P7 IOM2 P8 IOM1 P17 IOM1 P18 IOM1 P19 IOM1 P20 IOM1 P21 IOM1 P22 IOM1 P23 IOM1 P24 IOM2 P17 IOM2 P18 IOM2 P19 IOM2 P20 IOM2 P21 IOM2 P22 IOM2 P23 IOM2 P24 IOM1 P9 IOM1 P10
I/O Fabric Architecture with Quarter Height Blades Right side of chassis Quarter-Height (Slot a) 8 lane PCIe CPU 2 8 lane PCIe Fabric A LOM Ethernet I/O Module Fabric B Mezzanine Fabric B2 I/O Module Quarter-Height (Slot b) Front of Chassis Fabric A2 CPU 1 8 lane PCIe Fabric A LOM CPU 2 8 lane PCIe Fabric C Fabric C2 I/O Module Mezzanine Quarter-Height (Slot c) CPU 1 8 lane PCIe Fabric A LOM CPU 2 8 lane PCIe Fabric B Fabric C1 I/O Module Mezzanine Fabric B1 I/O Module Quarter-
Port Mapping of Quarter Height blades to two IOMs with 32 Internal Ports on Fabric A: Full LOM Port Redundancy IOM ports mapped to quarter height blade slots Slot1a Slot2a Slot3a Slot4a Slot5a Slot6a Slot7a Slot8a A1 P1 A1 P2 A1 P3 A1 P4 A1 P5 A1 P6 A1 P7 A1 P8 A2 P17 A2 P18 A2 P19 A2 P20 A2 P21 A2 P22 A2 P23 A2 P24 Slot1b Slot2b Slot3b Slot4b Slot5b Slot6b Slot7b • On fabric A, two IOMs with 32 internal ports provide connectivity to two ports of the LOM on each quarter height blade.
Port Mapping of Quarter Height blades to two IOMs with 16 Internal Ports on Fabric A: No LOM Port Redundancy IOM ports mapped to quarter height blade slots Slot1a Slot2a Slot3a Slot4a Slot5a Slot6a Slot7a Slot8a A1 P1 A1 P2 A1 P3 A1 P4 A1 P5 A1 P6 A1 P7 A1 P8 Slot1b Slot2b Slot3b Slot4b Slot5b Slot6b Slot7b Slot8b • On fabric A, two IOMs with 16 internal ports provide connectivity to one port of the LOM on each quarter height blade.
Port Mapping of Quarter Height blades to four IOMs on Fabric B&C: Full Mezz Card Port Redundancy IOM ports mapped to quarter height blade slots Slot1a Slot2a Slot3a Slot4a Slot5a Slot6a Slot7a Slot8a C1 P1 C1 P2 C1 P3 C1 P4 C1 P5 C1 P6 C1 P7 C1 P8 C2 P1 C2 P2 C2 P3 C2 P4 C2 P5 C2 P6 C2 P7 C2 P8 Slot1b Slot2b Slot3b Slot4b Slot5b Slot6b Slot7b Slot8b B1 P1 B1 P2 B1 P3 B1 P4 B1 P5 B1 P6 B1 P7 B1 P8 B2 P1 B2 P2 B2 P3 B2 P4 B2 P5 B2 P6 B2 P7 B2 P8 Slot1c Slot2c
Dell PowerEdge M1000e I/O Interoperability guide 69 Dell M-Series Blade I/O Guide
PowerEdge M1000e 1Gb Ethernet I/O Interoperability 1Gb Ethernet I/O Modules 1GbE M6348 M6220 Cisco 3032 Cisco 3130G Cisco 3130X Broadcom 5708 Mezz Broadcom 5709 LOM/Mezz Broadcom 5709 4-port NDC/Mezz Intel ET 4-port Mezz 1Gb Intel I350 4-port Mezz LOM Broadcom 5719 4-port Mezz Broadcom 5720 4-port Mezz Adapters Pass-Through 70 Dell M-Series Blade I/O Guide
PowerEdge M1000e 10Gb Ethernet I/O Interoperability 10Gb Ethernet I/O Modules MXL PowerEdge M8024-k M I/O M8024 M8428-k (original model) Aggregator Adapters 10Gb Pass-Through 10Gb 10Gb PassThrough II Pass-Through -k B22DELL Broadcom 57710 Mezz Not Compatible Not Compatible Not Compatible Not Compatible Not Compatible Not Compatible Broadcom 57711 Mezz Not Compatible Not Compatible Not Compatible Not Compatible Not Compatible Not Compatible Emulex OCm10102-f-m Mez
PowerEdge M1000e InfiniBand I/O Interoperability Mezzanine Cards I/O Modules M2401G Mellanox DDR M3601Q Mellanox QDR M4001Q Mellanox QDR M4001T Mellanox FDR10 M4001F Mellanox FDR Mellanox DDR Connect-X DDR DDR Not Supported Not Supported Not Supported Mellanox QDR Connect-X2 DDR QDR QDR QDR QDR Mellanox QDR Connect-X3 Not Supported QDR QDR QDR* QDR Mellanox FDR10 Connect-X3 Not Supported QDR QDR FDR10 FDR10 Mellanox FDR Connect-X3 Not Support
PowerEdge Blade Servers and InfiniBand Adapters 73 Mellanox DDR Connect-X Mellanox QDR Connect-X2 Mellanox QDR Connect-X3 Mellanox FDR10 Connect-X3 Mellanox FDR Connect-X3 M420 Not Supported Not Supported Not Supported Not Supported M520 Not Supported Not Supported Not Supported M620 Not Supported Not Supported M820 Not Supported Not Supported Not Supported Not Supported M910 Not Supported M915 Blade Servers Mezzanine Cards Not Suppor
PowerEdge M1000e Fibre Channel I/O Interoperability Mezzanine Cards I/O Modules M4424 Brocade FC4 FC8 Passthrough Dell 8/4Gbps FC SAN Module Brocade FC8 M6505 Brocade FC16 Emulex FC4 FC4 FC4 FC4 FC4 FC4 Not compatible QLogic FC4 FC4 FC4 FC4 FC4 FC4 Not compatible Emulex LPe1205-M FC8 (for 11G and 12G servers) FC4 FC4 FC8 FC8 FC8 FC8 QLogic QME2572 FC8 (for 11G and 12G servers) FC4 FC4 FC8 FC8 FC8 FC8 Emulex LPm16002 FC16 Not com
PowerEdge Blade Servers and Fibre Channel Adapters 75 QLogic QME2662 FC8 Emulex LPm16002 FC16 M420 Not Supported Not Supported M520 Not Supported Not Supported M620 M820 M910 M915 Blade Servers Mezzanine Cards for 12G Servers Dell M-Series Blade I/O Guide Emulex LPe1205-M FC8 QLogic QME2572 FC16
Server Adapter Portfolio Includes: Server Adapter products, features, compatibility and software support matrix 76 Dell M-Series Blade I/O Guide
12G+ M1000e Server Adapter Portfolio: Ethernet and InfiniBand 10Gb Ethernet 10Gb Converged Ethernet 10Gb Converged Ethernet Intel X520-k 2P NDC Intel X520-k 2P Mezz Broadcom 57810S-k 2P NDC Broadcom 57810S-k 2P LOM Broadcom 57810S-k 2P Mezz QLogic QMD8262-k KR NDC QLogic QME8262-k KR Mezz Brocade BR1741M-k Mezz 1Gb Ethernet Broadcom 5720 4P LOM Broadcom 5719 4P Mezz Intel I350 4P Mezz QDR/FDR InfiniBand Mellanox CX3 FDR Mezz Mellanox CX3 FDR10 Mezz 77 Global Marketing
Select Network Adapters for blade servers Broadcom QLogic QMD8262-k 57810S-k NDC Intel X520-k NDC NDC Features (Default choice) Ports x Link Speed 2x10Gb 2x10Gb 2x10Gb Supported Speed 1Gb, 10Gb 1Gb,10Gb 10Gb Chipset 57810S X520/82599 P3+ Interface KR KR KR ISCSI HBA Yes No Yes ISCSI Boot Yes Yes Yes FCoE Yes Yes Yes FCoE Boot Switch Independent Partitioning DCB Yes Yes Yes No Yes Yes Yes Yes SR-IOV Yes 1 No WOL 3 Yes Yes Yes Yes PXE Yes Yes Yes EEE No N
LOMs for Blade Servers Features Ports x Link Speed Supported Speed Chipset Interface ISCSI HBA ISCSI Boot FCoE FCoE Boot Switch Independent Partitioning DCB SR-IOV WOL PXE EEE Multi-queue1 (per port) Supported Servers 79 Dell M-Series Blade I/O Guide Broadcom 57810S-k 2p 10Gb LOM 2x10Gb 1Gb, 10Gb 57810S KR Yes Yes Yes Yes Broadcom 5720 4p 1Gb LOM 4x1Gb 1Gb 5720 Serdes No Yes No No Yes2 No Yes Yes Yes Yes No 128 TX, 128 RX M420 No No Yes Yes Yes 8 TX, 8 RX M520 1: No.
Blade Mezzanine Card: 1Gb Ports x Link speed Supported Speed Intel I350 4p 1Gb Mezz 4x1Gb 1Gb Broadcom 5719 4p 1Gb Mezz 4x1Gb 1Gb Chipset I350 5719 Interface ISCSI HBA iSCSI Boot FCoE FCoE boot Serdes No Yes No No Serdes No Yes No No Switch Independent Partitioning No No DCB SR-IOV WOL PXE EEE No No Yes Yes Yes No No Yes Yes Yes Multi-queue1 (per port) 8 TX, 8 RX 8 TX, 8 RX Supported Servers M420, M520, M620, M820 M420, M520, M620, M820 Great for Preference for Intel Ethernet solutions
Blade Mezzanine Card: 10Gb 81 Features Broadcom 57810S-k DP 10Gb Intel X520 10Gb DP –x/k QLogic QME8262-k Brocade BR1741Mk KR Ports x Link Speed 2x10Gb 2x10Gb 2x10Gb 2x10Gb Supported Speed 1Gb,10Gb 1Gb,10Gb 10Gb 1Gb, 10Gb Chipset 57810S X520 P3+ Catapult I Interface Type KR XAUI/KR KR KR ISCSI HBA Yes No Yes No iSCSI Boot Yes Yes Yes No FCoE Yes Yes Yes Yes FCoE boot Yes Yes Yes Yes Switch Independent Partitioning Yes3 No Yes No DCB Yes Yes Yes Yes SR
Blade Mezzanine Card: FC8Gb and FC16Gb 82 Features QLogic QLE2572 FC8 Emulex 1205-M FC8 Qlogic QME2662 FC16 Emulex LPm16002 FC16 Ports x Link speed 2x8Gb 2x8Gb 2x16Gb 2x16Gb Supported Speed 4Gb, 8Gb 4Gb, 8Gb 8Gb, 16Gb 8Gb, 16Gb Chipset 2500 LightPulse 2600 LightPulse FC Boot Yes Yes Yes Yes Supported Servers M420, M520, M620, M820, M420, M520, M620, M820 M620, M820, M910, M915 M620, M820, M910, M915 Dell M-Series Blade I/O Guide Global Marketing
Blade Mezzanine: InfiniBand 83 Features Mellanox CX3 FDR10 Mellanox CX3 FDR Ports x Link 2 x 40Gb 2 x 56Gb Chipset CX-3 CX-3 Supported Protocols IB IB Supported servers M420, M520, M620, M820 M620 Great for Real time market data distribution HFT, co-located investment banks, algorithmic trading, low latency applications Dell M-Series Blade I/O Guide Global Marketing
10Gb Products with Convergence and Virtualization features Form Factor Device Name Convergence Virtualization FCOE Virtual Queues Yes( 4 partitions per 10Gb port) Yes 128 TX, 128 RX Software FCoE, boot No Yes 64 TX, 64 RX All All Yes( 4 partitions per 10Gb port) No 64 TX, 64 RX Broadcom 57810S-k (2x10Gb) All Yes Yes( 4 partitions per 10Gb port) Yes 128 TX, 128 RX Broadcom 57810S-k (2x10Gb) All Yes Yes( 4 partitions per 10Gb port) Yes 128 TX, 128 RX Software iSCSI, boot Software
12G Server IO Features Vendors FCoE Chipset/ Speed Devices (offload, Boot) ISCSI (Boot, offload, SW) NPAR SR-IOV DCB w/ iSCSI Life Cycle controller 2.0 + Real time monitoring No No No Yes Software iSCSI, iSCSI Boot 5719/5720 (W2K8, Hyper-V, RH5.7, RH5.8, RH6 .1, RH6.2, SLES10 SP4 and SLES11 SP2, ESX/i 4.1/5.0) 1G No 10Gb (W2K8, Hyper-V, RH6 .1/6.2, SLES11 SP2, ESX/i 4.1/5.0) Broadcom Yes2 57810S All (W2K8, Hyper-V, RHE5.7, RH5.8, RH6 .1, RH6.2,, SLES10 SP4 and SLES11 SP2, ESX/i 4.1/5.
Select Network Adapters – 11G vs.
Mezzanine Adpaters - 11G vs.
Fibre Channel Adapters - 11G vs.
12G Systems Management Network Device Support Matrix Form Factor Vendor/Chipsets Speed LC configuration and update Monitoring Support Blade NDC Broadcom 57810S-k NDC Intel X520-kNDC QLogic QMD8262-k NDC 10GbE 10GbE 10GbE Yes Yes Yes Yes Yes Yes Blade LOM Broadcom 57810S-k LOM Broadcom 5720 LOM 10GbE 1GbE Yes Yes Yes Yes Blade Mezz Broadcom 57810S-k Broadcom 5719 Serdes Intel I350 Serdes Intel X520 x/k QLogic QME8262-k Brocade BR1741M-k 10GbE 1GbE 1GbE 10GbE 10GbE 10GbE Yes Yes Yes Yes Yes
Deployment and Technical Guides 90 Dell M-Series Blade I/O Guide
Deployment & Technical Guides Detailed guides to help you get connected Product Focus Document Title Link M6220 Stacking PowerConnect M6220 Blade Switch http://del.ly/m6220stacking M6220 and Cisco MSTP Interoperability of the Dell 6200 & M6220 Series Switches http://del.ly/m6200mstp M6220 and Cisco VLAN Interoperability of the Dell M6220 http://del.ly/m6220vlan M6220, M6348 Sizing and Best Practices for Deploying VMware with Dell EqualLogic Storage http://del.
Interactive 3D Blade Server and Networking Demos! • Get a closer look at the 12th Generation PowerEdge Server portfolio and explore the innovative technologies inside the servers with the new Dell Interactive Rack, Tower and Blade 3D demo tool. Using the tool, you can turn, spin, and pull out components of our servers via their laptop, tablet, phone or touchscreen display. Simply go online or download the new Interactive tool and you are ready to begin. – Blade Servers and Networking: www.bladeserverdemo.
Feedback We encourage readers of this publication to provide feedback on the quality and usefulness of this information by sending an email to BladeInterconnects@Dell.com.
