Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024 and 8024F A Dell Deployment Guide Authors Network Enabled Solutions Team, Kevin Locklear Contributor Kili Land
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F This document is for informational purposes only and may contain typographical errors and technical inaccuracies. The content is provided as is, without express or implied warranties of any kind. © 2011 - 2012 Dell Inc. All rights reserved. Dell and its affiliates cannot be responsible for errors or omissions in typography or photography. Dell, the Dell logo, PowerConnect and PowerEdge are trademarks of Dell Inc.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Contents Introduction ............................................................................................................. 6 Basic Terminology ...................................................................................................... 6 FIP snooping .......................................................................................................... 6 FIP snooping bridge (FSB) .................
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Cisco Nexus 5548UP CLI example ............................................................................... 52 Appendix B - Network Switch Versions ............................................................................ 56 References ............................................................................................................. 56 About Dell ........................................................
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F 5
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Introduction The PowerConnect™ M8024-k, 8024 and 8024F switches are now DCB/DCBx capable with a downloadable update. Starting with firmware 4.2, the latest PowerConnect™ 10 Gigabit switches can now be used as an FCoE Transit Switch (FIP Snooping Bridge, T11, BB-5).
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F PFC Priority Flow Control (PFC), or Per-Priority Pause is defined in the IEEE 802.1Qbb standard. PFC is flow control based on priority settings and adds additional information to the standard pause frame. The additional fields added to the pause frame allow devices to pause traffic on a specific priority instead of pausing all traffic.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Configuration scenarios The following sections will present very basic examples of deploying the 10G switches for FIP Snooping and will provide step-by-step explanations of the CLI commands as a guide. The GUI does not currently support configurations for FIP Snooping. Consult the table of contents above for a list of examples covered in this document. Figure 3.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Important notes prior to deployment No Simple Mode Each of the following scenarios in this document assume that the PowerConnect™ 8024 model switch being used is in normal Switch Mode (not Simple Mode) and is using firmware version 4.2.x.x or later. NOTE: If Simple Mode is enabled it will need to be disabled prior to implementing the deployment covered in this document.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Scenario 1: Deploying the Dell PowerConnect 8024 Series FSB in a Cisco 5000 Series Switch (NPIV) environment This first example is a basic, single connection between devices example using the Dell PowerConnect M8024-k. This configuration is being shown for the purposes of simplification and potentially easing into the progression of a more in-depth setup.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F The flowchart in Figure 8 is a general overview of how the deployment will occur. This includes the basic planning that will need to take place in order for most of the steps in the rest of the document to fall into place. Figure 6.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Figure 9 is a graphical representation of how many of the configuration pieces are considered parallel settings. Most of the configuration will depend heavily on configurations being completed in more than just one place. Figure 7.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Configuring the Dell PowerConnect M8024-k,8024, and 8024F for FIP Snooping The Dell PowerConnect 8024 model switches will monitor FIP packets and will establish the proper filtering, and priorities for the FCoE traffic that is passed through the configured links. To see an example of the full configuration see Appendix-A - M8024-K Example. Command-Line Interface Method Figure 8.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Step by Step explanation of CLI example o Configure – this brings the prompt into the configuration interface o no mode simple – puts switch into normal mode o vlan database - moves down into the VLAN database interface vlan 20 – add VLAN 20 to used for untagged traffic or as the native VLAN vlan 1000 - add VLAN 1000 to the VLAN database, this will be the FCoE VLAN exit – exit the current level of the inter
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F o interface te1/0/20 – this moves into the interface te1/0/20 configuration switchport general pvid 20 – establishes the native VLAN as 20, you must remove VLAN 1 in order for this to function correctly switchport general allowed vlan add 20 - adds VLAN 20 the trunk as an untagged VLAN switchport general allowed vlan add 1000 tagged - – this sets up a trunk with a tagged VLAN of 1000 (the FCoE
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Figure 9.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Configuring the Cisco 5000 series switch with firmware ver 5.x for a single connection from the Dell PowerConnect M8024-k or 8024()(F) The CLI commands below are necessary for an un-configured Cisco 5020. The CLI will show additional lines that are either default or can’t be changed and are not added for this example. The CLI will also show the lines in a different order after they have been entered.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F The following CLI commands are necessary for an un-configured Cisco Nexus 5548UP or Cisco Nexus 5596. The full CLI will show additional lines that are either default or can’t be changed and are not added for this example. The CLI will also show the lines in a different order after they have been entered. Appendix-A Cisco Nexus 5548UP CLI example will have a copy of the full configuration.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Step by Step explanation of the CLI example Configure – this brings you into the configuration interface. o feature FCOE – enables the feature for FCOE as long as the licensing and FC modules are installed. o feature NPIV – enables the FC ports to accept multiple logins (Necessary for Compellent). o feature LACP – enables the switch to be able to use port-channel groups in a LACP mode.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F o Zone name blade1 vsan 2 – this will set the name for your zone (blade1 can be any chosen name); vsan 2 will match the vsan you have created. o zoneset name set1 vsan 2 – move into the zoneset interface (in this case the name is set1 but could be any name and the VSAN number is based on the FCoE VLAN being used. o Member interface fc2/1 - this adds the fc2/1 interface as a member of the zone.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Figure 13. M1000e Chassis Management Controller -> Server Overview -> Properties -> WWN/MAC information for Blade Server 3’s “B” fabric CNA port 2 (B2).
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Basic Validation for the Cisco Nexus 5000 series switch configuration With connections made via fiber optic cable or direct connect cable (twin-ax) the basic connectivity on the port between the two switches can be verified. These cables or SFP+ transceivers must only be Cisco-branded products for the Cisco Nexus to link properly. The following command is used to give a port status overview: SHOW INTERFACE BRIEF .
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Check the spanning tree configuration. Blocking ports should be understood and when they show in this entry it should be the number expected. Otherwise it could be that an unintentional cable loop has been created that will need to be resolved. Type : SHOW SPANNING -TREE SUMMARY Figure 15.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F With all cables in place between the switches and the FC/FCoE SAN, use the following commands to validate your configuration. First check general status of the ports, and links by using: SHOW INTERFACE STATUS Figure 18.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F With the SHOW SPANNING -TREE BLOCKEDPORTS command the status of ports that may be impacted by spanning tree can be checked. This is also a good point to make sure that the spanning-tree behaviors are as expected, such as Priority, which switch is root, etc. Figure 19.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F The following command will show a set of vital information on the DCBX configuration. Type: SHOW LLDP DCBX INTERFACE ALL . The important things to note with this command are that the configuration source selected is “True” and that the configuration source port is configured as expected to be the uplink to the top-of-rack FCF switch.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F The following command will show a set of vital information on the DCBX configuration. Type: SHOW LLDP DCBX INTERFACE TE 1/0/20 DETAIL . This command will show more specific detail about the configuration that has been negotiated between the devices. In this example port te1/0/20 is used as it is one of the uplink ports into the top-of-rack FCF switch.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Finally, after these validation steps, go into the disk management interface of the server. If the SAN is configured appropriately the server will have an available LUN to use for storage.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Problem Potential Problem Area Potential Fix The VFC entries may not be present on the Cisco Nexus 5000 series switch Ensure that VFC's are created that will be applicable to the connection.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Scenario 2: Configuring Multiple Uplinks into LAG for Cisco Nexus 5000 Series Switch (NPIV) Environment Figure 23.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Configuring the Cisco Nexus 5000 series switch with firmware ver 5.x for a multiple link LAG (link aggregation) connection at the Top-of-Rack. The typical scenario in a business environment consists of more than one connection or uplink. The following illustrations and examples describe a two link LAG from an M8024-k to the Cisco Nexus 5020. Command-Line Interface Method Figure 24.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Step by Step explanation of CLI example (only covering the differences from the single port configuration) o interface vfc1 – selects virtual fc interface vfc1 (vfc1 is an example and can be a different number based on choice at configuration time) o interface fc2/2 – selects the fc2/2 interface (this is an additional FC port for the 2 connections o member interface fc2/2 - this adds the fc2/1 interface as
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Configuring the M8024-k,8024, and 8024F for FIP Snooping This is an example of the necessary CLI commands for 8024 model switches configured with two links in a LAG or port-channel uplinked to the Cisco 5020. This provides more bandwidth and fail-over capability which would be more typical in the larger scale infrastructure typically used. Command-Line Interface Method Figure 25.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Step by Step explanation of Dell PowerConnect M8024-k CLI example with multiple port (LAG) uplink (this only covers the differences from the single port configuration).
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Scenario 3: Configuring Multiple Uplinks into LAG for Cisco Nexus 5000 Series Switch (NPV mode) Environment Figure 26.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Configuring the Cisco Nexus 5000 series switch with firmware ver 5.0(3)N2(2a) in NPV mode for a multiple link LAG (link aggregation) connection from the Dell PowerConnect M8024-k or 8024()(F) The typical scenario in a business environment consists of more than one connection or uplink. The following pictures and examples describe a two link LAG from an M8024-k to the Cisco 5020.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Step by Step explanation of CLI example for the Cisco Nexus 5000 Series using NPV (only covering the differences from the single port configuration) o feature NPV– enables the NPV feature which turns off zoning. The FC side of the Nexus 5000 series switch effectively turns into just a FC gateway (just passing FC out to another switch).
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Further notes on ETS / CoS behavior settings Careful consideration should be taken when establishing the strict priority queues and managing bandwidth reservations. It FCoE and iSCSI are configured on the same switch and these settings are used to configured both CoS queues it is possible to choke the bandwidth being allowed for regular LAN traffic.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Updating firmware NOTE:. Be sure to check the Release Notes and any special instructions that may have come with new firmware updates. It is important to follow instructions found in those documents if they deviate from this white paper. Steps for upgrading the firmware on a stack of switches are similar to upgrading the firmware on a single switch.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Perform another show version command to confirm that the new firmware has been downloaded to the switch. Notice that the firmware was copied into the inactive image for each member. console#show version Images currently available on Flash unit image1 image2 current-active next-active ----- ------------ ------------ ----------------- ----------------1 4.1.0.9 4.2.0.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F To validate, login to the switch and perform a show version command. For the example given the following is displayed. console#show version Images currently available on Flash unit image1 image2 current-active next-active ----- ------------ ------------ ----------------- ----------------1 4.1.0.9 4.2.0.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F 3. Enter the IP address of the TFTP server into the Server Address field, and enter the name of the Firmware file into the Source File Name field. 4. If not in the root directory of the TFTP server, enter the path of the firmware file. 5. Click Apply. The dialog box below will appear after a short period stating that the transfer is complete. 6. Click Close. 7. Select System > File Management > Active Images.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Also notice that the switch still has the same Current-Active image as before. 8. Under the Next-Active column, select the new image (i.e. image2) for the switch or stack. 9. Click Apply. The Current-Active column should now show the opposite values as what is in the Next-Active column. A reload is required to active the firmware. 10. Select System > General > Reset. Choose “All” in the Switch ID menu. 11.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Notice that the new firmware (i.e. 4.2.0.4) is now the Current-Active image for each member. While downgrading to a previous firmware is supported, all features and functions that were not part of the previous firmware will be lost, including those features and functions that were introduced in the current firmware in use. Firmware version 4.2 or later must be active for the DCB or stacking features to work correctly.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Appendix A – Full CLI examples M8024-k CLI example show running-config !Current Configuration: !System Description "PowerConnect M8024-k, 4.2.1.3, VxWorks 6.6" !System Software Version 4.2.1.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F ! interface Te1/0/2 cos-queue min-bandwidth 0 0 0 50 0 0 0 cos-queue strict 3 shutdown spanning-tree port-priority 0 mtu 2500 switchport mode general switchport general pvid 20 switchport general allowed vlan add 20 switchport general allowed vlan add 1000 switchport general allowed vlan remove 1 lldp dcbx port-role auto-down exit ! interface Te1/0/3 cos-queue min-bandwidth 0 0 0 50 0 0 0 cos-queue strict 3 mtu 2500 s
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F exit ! interface Te1/0/6 cos-queue min-bandwidth 0 0 0 50 0 0 0 cos-queue strict 3 shutdown spanning-tree port-priority 0 mtu 2500 switchport mode general switchport general pvid 20 switchport general allowed vlan add 20 switchport general allowed vlan add 1000 switchport general allowed vlan remove 1 lldp dcbx port-role auto-down exit ! interface Te1/0/7 cos-queue min-bandwidth 0 0 0 50 0 0 0 cos-queue strict 3 spann
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F lldp dcbx port-role auto-down exit ! interface Te1/0/10 cos-queue min-bandwidth 0 0 0 50 0 0 0 cos-queue strict 3 shutdown spanning-tree port-priority 0 mtu 2500 switchport mode general switchport general pvid 20 switchport general allowed vlan add 20 switchport general allowed vlan add 1000 tagged switchport general allowed vlan remove 1 lldp dcbx port-role auto-down exit ! interface Te1/0/11 cos-queue min-bandwidth
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F switchport general allowed vlan add 1000 switchport general allowed vlan remove 1 lldp dcbx port-role auto-down exit ! interface Te1/0/14 cos-queue min-bandwidth 0 0 0 50 0 0 0 cos-queue strict 3 shutdown spanning-tree port-priority 0 mtu 2500 switchport mode general switchport general pvid 20 switchport general allowed vlan add 20 switchport general allowed vlan add 1000 switchport general allowed vlan remove 1 lldp
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F mtu 2500 switchport mode general switchport general pvid 20 switchport general allowed vlan add 20 switchport general allowed vlan add 1000 switchport general allowed vlan remove 1 exit ! interface Te1/0/18 cos-queue min-bandwidth 0 0 0 50 0 0 0 cos-queue strict 3 shutdown spanning-tree port-priority 0 mtu 2500 switchport mode general switchport general pvid 20 switchport general allowed vlan add 20 switchport general
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F switchport general allowed vlan add 1000 tagged switchport general allowed vlan remove 1 mtu 2500 fip-snooping port-mode fcf exit ! interface port-channel 3 mtu 2500 exit ! interface port-channel 4 mtu 2500 exit Skipped port-channel 5 through 126 for space interface port-channel 127 mtu 2500 exit ! interface port-channel 128 mtu 2500 exit enable password f611e082a05f5562f1d0d2bbcef2b5bf encrypted exit console# 51
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Cisco Nexus 5548UP CLI example Technet5548-1# show running-config !Command: show running-config !Time: Tue Feb 14 22:47:07 2012 version 5.
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F vlan 1000 fcoe vsan 2 name FCoE_VLAN vsan database vsan 2 interface port-channel2 switchport mode trunk switchport trunk native vlan 20 switchport trunk allowed vlan 1000 interface vfc3 bind mac-address 14:fe:b5:8e:5b:f2 no shutdown interface vfc4 bind mac-address 14:fe:b5:8e:5b:f0 no shutdown interface vfc5 bind mac-address 14:fe:b5:8e:5c:09 no shutdown interface vfc6 bind mac-address 14:fe:b5:8e:5c:0b no shutdown in
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F interface fc1/32 switchport trunk mode on no shutdown interface Ethernet1/1 interface Ethernet1/2 interface Ethernet1/3 interface Ethernet1/4 interface Ethernet1/5 interface Ethernet1/6 interface Ethernet1/7 interface Ethernet1/8 interface Ethernet1/9 interface Ethernet1/10 interface Ethernet1/11 interface Ethernet1/12 interface Ethernet1/13 interface Ethernet1/14 interface Ethernet1/15 interface Ethernet1/16 interfac
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F interface Ethernet1/25 interface Ethernet1/26 switchport mode trunk switchport trunk native vlan 20 switchport trunk allowed vlan 1000 interface Ethernet1/27 switchport mode trunk switchport trunk native vlan 20 switchport trunk allowed vlan 1000 interface Ethernet1/28 switchport mode trunk switchport trunk native vlan 20 switchport trunk allowed vlan 1000 interface Ethernet1/29 switchport mode trunk switchport trunk
Deploying FCoE (FIP Snooping) on Dell PowerConnect 10G Switches: M8024-k, 8024, and 8024F Appendix B - Network Switch Versions Version information for the network switches used in creating this document are as follows: Table 1. Switch Firmware Versions Network switch Dell PowerConnect™ M8024k Dell PowerConnect™ 8024/8024F Cisco 5020 Software version 4.2.0.1, 4.2.0.2, 4.2.0.3, 4.2.0.4, 4.2.1.3 4.2.0.1, 4.2.0.2, 4.2.0.3, 4.2.0.4, 4.2.1.3 5.0(3)N2(2a), 5.