Dell PowerEdge Configuration Guide for the M I/O Aggregator Publication Date: March 2013
Notes, Cautions, and Warnings NOTE: A NOTE indicates important information that helps you make better use of your computer. CAUTION: A CAUTION indicates either potential damage to hardware or loss of data and tells you how to avoid the problem. WARNING: A WARNING indicates a potential for property damage, personal injury, or death. Information in this publication is subject to change without notice. © 2012 Dell Force10. All rights reserved.
| iii
iv | www.dell.com | support.dell.
1 About this Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
www.dell.com | support.dell.com External Serial Port with a USB Connector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23 Boot Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Configure a Host Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28 Access the System Remotely . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Data Center Bridging Exchange Protocol (DCBX) . . . . . . . . . . . . . . . . . . . . . . . . . .55 Data Center Bridging in a Traffic Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55 Data Center Bridging: Auto-DCB-Enable Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56 When DCB is Disabled (Default) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56 When DCB is Enabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
www.dell.com | support.dell.com 8 FIP Snooping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Fibre Channel over Ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .93 Ensuring Robustness in a Converged Ethernet Network . . . . . . . . . . . . . . . . . . . . . . . .93 FIP Snooping on Ethernet Bridges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Default VLAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .126 Port-Based VLANs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .127 VLANs and Port Tagging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .127 Configuring VLAN Membership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .128 Displaying VLAN Membership . . . . . . . . . .
www.dell.com | support.dell.com 12 Link Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 How the LACP is Implemented on an Aggregator . . . . . . . . . . . . . . . . . . . . . . . . . . . . .155 Uplink LAG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 Server-Facing LAGs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16 Simple Network Management Protocol (SNMP) . . . . . . . . . . . . . . . . . . . . . . . . . 191 Implementation Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .191 Configuring the Simple Network Management Protocol . . . . . . . . . . . . . . . . . . . . . . . .191 Important Point to Remember . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .191 Setting up SNMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
www.dell.com | support.dell.com Stack-Link Flapping Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .218 Master Switch Recovers from Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .218 Stack Unit in Card-Problem State Due to Incorrect FTOS Version . . . . . . . . . .219 Stack Unit in Card-Problem State Due to Configuration Mismatch . . . . . . . . . .219 Upgrading a Switch Stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Resolution: Disable broadcast storm control globally on the Aggregator. . . . . .245 Steps to Take: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .245 Flooded packets on all VLANs are received on a server . . . . . . . . . . . . . . . . . . . .246 Symptom: All packets flooded on all VLANs on an Aggregator are received on a server, even if the server is configured as a member of only a subset of VLANs.
www.dell.com | support.dell.com RFC and I-D Compliance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .272 General Internet Protocols. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .272 General IPv4 Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .273 Network Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .274 MIB Location . . . . . . .
1 About this Guide Objectives This guide describes the supported protocols and software features, and provides configuration instructions and examples, for the Dell Force10 M I/O Aggregator running FTOS version 8.3.17.4. The M I/O Aggregator is installed in a Dell PowerEdge M1000e Enclosure. For information about how to install and perform the initial switch configuration, refer to the Getting Started Guides on the Dell Support website at http://support.dell.com/manuals.
www.dell.com | support.dell.com Conventions This document uses the following conventions to describe command syntax: Convention Description keyword Keywords are in bold and must be entered in the CLI as listed. parameter Parameters are in italics and require a number or word to be entered in the CLI. {X} Keywords and parameters within braces must be entered in the CLI. [X] Keywords and parameters within brackets are optional.
2 Before You Start By following the instructions in the Dell PowerEdge M I/O Aggregator Getting Started Guide that is shipped with the product, you install the Aggregator in a Dell PowerEdge M1000e Enclosure. The Aggregator installs with zero-touch configuration. After you power it on, an Aggregator boots up with default settings and auto-configures with software features enabled. This chapter describes the default settings and software features that are automatically configured at startup.
www.dell.com | support.dell.com Other Auto-Configured Settings After the Aggregator powers on, it auto-configures and is operational with software features enabled, including: • • • • • • • • • • • 4 | Before You Start Ports: Ports are administratively up and auto-configured to operate as hybrid ports to transmit tagged and untagged VLAN traffic. Ports 1 to 32 are internal server-facing ports, which can operate in either 1GbE or 10GbE mode.
DCB Support DCB enhancements for data center networks are supported to eliminate packet loss and provision links with required bandwidth. The Aggregator provides zero-touch configuration for DCB. The Aggregator auto-configures DCBX port roles as follows: • • Server-facing ports are configured as auto-downstream interfaces. Uplink ports are configured as auto-upstream interfaces. In operation, DCBX auto-configures uplink ports to match the DCB configuration in the ToR switches to which they connect.
www.dell.com | support.dell.com Link Aggregation All uplink ports are configured in a single LAG (LAG 128). Server-facing ports are auto-configured as part of link aggregation groups if the corresponding server is configured for LACP-based NIC teaming. Static LAGs are not supported. Tip: The recommended LACP timeout is long-timeout mode. Link Tracking By default, all server-facing ports are tracked by the operational status of the uplink LAG.
The untagged VLAN of a server-facing LAG is configured based on the untagged VLAN to which the lowest numbered server-facing port in the LAG belongs. Tip: Dell Force10 recommends that you configure the same VLAN membership on all LAG member ports. Stacking Mode Stacking is supported only on the 40GbE ports on the base module. A single stack is limited to two Aggregators in the same chassis. Up to three stacks are supported in an M1000e chassis. To configure a switch stack, you must use the CLI.
www.dell.com | support.dell.com In case of an FTOS upgrade, you can check to see that an Aggregator is running the latest FTOS version by entering the show version command. To download an FTOS version, go to http://support.dell.com. 8 Refer to the appropriate chapter for detailed information on how to reconfigure specific software settings.
3 Configuration Fundamentals The Dell Force10 operating software (FTOS) command line interface (CLI) is a text-based interface through which you can configure interfaces and protocols. The CLI is structured in modes for security and management purposes. Different sets of commands are available in each mode, and you can limit user access to modes using privilege levels. In FTOS, after you enable a command, it is entered into the running configuration file.
www.dell.com | support.dell.com CLI Modes Different sets of commands are available in each mode. A command found in one mode cannot be executed from another mode (with the exception of EXEC mode commands preceded by the command do; for more information, refer to do Command and EXEC Privilege Mode commands). The FTOS CLI is divided into three major mode levels: • • • EXEC mode is the default mode and has a privilege level of 1, which is the most restricted level.
Navigating CLI Modes The FTOS prompt changes to indicate the CLI mode. Table 3-1 lists the CLI mode, its prompt, and information about how to access and exit this CLI mode. You must move linearly through the command modes, with the exception of the end command, which takes you directly to EXEC Privilege mode and the exit command moves you up one command mode level.
www.dell.com | support.dell.com Figure 3-3 shows how to change the command mode from CONFIGURATION mode to INTERFACE configuration mode. Figure 3-3. Changing CLI Modes FTOS(conf)# interface tengigabitethernet 1/2 FTOS(conf-if-te-1/2)# New command prompt do Command Enter an EXEC mode or EXEC privilege mode command from any CONFIGURATION mode (such as CONFIGURATION, INTERFACE, etc.) without returning to EXEC mode by preceding the EXEC mode command with the command do.
Figure 3-5. Undoing a command with the no Command FTOS(conf)# interface managementethernet 0/0 FTOS(conf-if-ma-0/0)# ip address 192.168.5.6/16 Assign an IP address FTOS(conf-if-ma-0/0)# FTOS(conf-if-ma-0/0)# FTOS(conf-if-ma-0/0)#show config ! interface ManagementEthernet 0/0 ip address 192.168.5.
www.dell.com | support.dell.com Figure 3-7. Keyword? Command Example FTOS(conf)#cl? clock FTOS(conf)#cl • partial keyword plus “?” for matching keywords A keyword followed by [space]? lists all of the keywords that can follow the specified keyword. Figure 3-8.
Table 3-2. Short-Cut Keys and their Actions (continued) Key Combination Action CNTL-R Re-enters the previous command. CNTL-U Deletes the line. CNTL-W Deletes the previous word. CNTL-X Deletes the line. CNTL-Z Ends continuous scrolling of command outputs. Esc B Moves the cursor back one word. Esc F Moves the cursor forward one word. Esc D Deletes all characters from the cursor to the end of the word. Command History FTOS maintains a history of previously-entered commands for each mode.
www.dell.com | support.dell.com Figure 3-9. Filtering Command Outputs with the grep Command FTOS(conf)#do show stack-unit stack unit 0 stack-port all 0 Pause Tx pkts, 0 Pause 0 Pause Tx pkts, 0 Pause 0 Pause Tx pkts, 0 Pause 0 Pause Tx pkts, 0 Pause 0 Pause Tx pkts, 0 Pause 0 Pause Tx pkts, 0 Pause all stack-ports all pfc details | grep 0 Rx Rx Rx Rx Rx Rx pkts pkts pkts pkts pkts pkts Note: FTOS accepts a space or no space before and after the pipe.
Multiple Users in Configuration Mode FTOS notifies all users in the event that there are multiple users logged into CONFIGURATION mode. A warning message indicates the username, type of connection (console or vty), and in the case of a vty connection, the IP address of the terminal on which the connection was established.
18 | Configuration Fundamentals www.dell.com | support.dell.
4 Getting Started This chapter contains the following major sections: • • • • • • • • • • • • Front Panel Port Numbering Console access Boot Process Configure a Host Name Configure a Host Name Access the System Remotely Configure the Enable Password Configuration File Management File System Management View the Command History Upgrading and Downgrading FTOS When the boot process is complete, the console monitor displays the Dell Force10 operating software (FTOS) banner and EXEC mode prompt (Figure 4-3).
www.dell.com | support.dell.com Front Panel Figure 4-1 shows the Aggregator front panel: Figure 4-1. Front Panel of the M I/O Aggregator Flex IO Module in Top Expansion Slot Flex IO Module in Bottom Expansion Slot USB Storage Port 40GbE QSFP+ Ports on Base Module USB Console Port Port Numbering When installed in a PowerEdge M1000e Enclosure, Aggregator ports are numbered 1 to 56 and consist of internal server-facing ports, uplink ports, and stacking ports (Figure 4-2).
Uplink Ports Ports 33 to 56 are external ports used for uplinks and numbered from the bottom to the top of the switch as follows: • • • The two base module ports operate by default in standalone 4x10GbE mode and are numbered 33 to 36 and 37 to 40. Ports on the 2-Port 40-GbE QSFP+ module operate only in 4x10GbE mode: • In the bottom expansion slot, ports are numbered 41 to 44 and 45 to 48. • In the top expansion slot, ports are numbered 49 to 52 and 53 to 56.
www.dell.com | support.dell.com Figure 4-2.
Step Task (continued) Note: Terminal settings on the console port cannot be changed in the software and are set as follows: • • • • • 9600 baud rate No parity 8 data bits 1 stop bit No flow control External Serial Port with a USB Connector Table 4-1 lists the pin assignments. Table 4-1.
www.dell.com | support.dell.com Figure 4-3. Completed Boot Process syncing disks... done unmounting file systems... unmounting /f10/flash (/dev/ld0e)... unmounting /usr (mfs:31)... unmounting /lib (mfs:23)... unmounting /f10 (mfs:20)... unmounting /tmp (mfs:15)... unmounting /kern (kernfs)... unmounting / (/dev/md0a)... done rebooting... þ NetLogic XLP Stage 1 Loader Built by build at tools-sjc-01 on Fri Mar 16 Navasota IOM Boot Selector Label 4.0.0.
Figure 4-4. Completed Boot Process (Contd.) ########################################## # # # u-boot: for Navasota board # # # ########################################## Initialized CPLD on CS3 Detected [XLP308 (Lite) Rev A0] CPLD reg 06 val 0xf7 This is a NAVASOTA ... Initializing I2C0: speed = 30 KHz, prescaler = 0x0377 -- done. Initializing I2C1: speed = 100 KHz, prescaler = 0x0109 -- done.
www.dell.com | support.dell.com Figure 4-5. 26 Completed Boot Process (Contd.) FTOS>00:00:30: %STKUNIT0-M:CP %IFMGR-5-IFM_ISCSI_ENABLE: iSCSI has been enabled causing flow control to be enabled on all interfaces.
Figure 4-6. Completed Boot Process (Contd.
www.dell.com | support.dell.com Figure 4-7. Completed Boot Process (Contd.
Access the Aggregator Remotely Configuring the Aggregator for Telnet access is a three-step process: 1. Configure an IP address for the management port. Refer to Configure the Management Port IP Address. 2. Configure a management route with a default gateway. Refer to Configure a Management Route. 3. Configure a username and password. Refer to Configure a Username and Password. Configure the Management Port IP Address Assign IP addresses to the management ports in order to access the system remotely.
www.dell.com | support.dell.com Configure a Management Route Define a path from the system to the network from which you are accessing the system remotely. Management routes are separate from IP routes and are only used to manage the system through the management port. To configure a management route, follow this step: Step 1 Task Command Syntax Command Mode Configure a management route to the network from which you are accessing the system.
Dell Force10 recommends using the enable secret password. To configure an enable password: Task Command Syntax Command Mode Create a password to access EXEC Privilege mode. enable [password | secret] [level level] [encryption-type] password CONFIGURATION level is the privilege level, is 15 by default, and is not required. encryption-type specifies how you are inputting the password, is 0 by default, and is not required. • • • 0 is for inputting the password in clear text.
www.dell.com | support.dell.com • • To copy a local file to a remote system, combine the file-origin syntax for a local file location with the file-destination syntax for a remote file location (Table 4-2). To copy a remote file to a Dell Force10 system, combine the file-origin syntax for a remote file location with the file-destination syntax for a local file location (Table 4-2). Table 4-2.
Figure 4-10 shows an example of using the copy command to import a file to the Dell Force10 system from an FTP server. Figure 4-10. Copying a file from a Remote System Remote Location Local Location FTOS#copy ftp://myusername:mypassword@10.10.10.10//FTOS/FTOS-EF-8.2.1.0.bin flash:// Destination file name [FTOS-EF-8.2.1.0.bin.
www.dell.com | support.dell.com Task Command Syntax Command Mode copy running-config scp:// username:password@{hostip | hostname}/filepath/ filename an SCP server EXEC Privilege Note: When copying to a server, you can only use a hostname if a DNS server is configured. View Files You can only view file information and content on local file systems.
To view the contents of a file, follow this step: Step 1 Task Command Syntax Command Mode contents of a file in the internal flash show file flash://filename EXEC Privilege contents of a file in the usb flash show file usbflash://filename EXEC Privilege running-configuration show running-config EXEC Privilege startup-configuration show startup-config EXEC Privilege View the: View Configuration Files Configuration files have three commented lines at the beginning of the file (Figure 4-12), t
www.dell.com | support.dell.com File System Management The Dell Force10 system can use the internal Flash, USB Flash, or remote devices to store files. The system stores files on the internal Flash by default, but you can configure it to store files elsewhere. To view file system information: Task Command Syntax Command Mode View information about each file system.
Figure 4-14. Alternative Storage Location FTOS#cd usbflash: FTOS#copy running-config test ! 3998 bytes successfully copied No File System Specified FTOS#dir Directory of usbflash: 1 drwx 2 drwx 3 -rwx 4 -rwx 4096 Jan 01 1980 00:00:00 +00:00 . 2048 May 02 2012 07:05:06 +00:00 ..
38 | Getting Started www.dell.com | support.dell.
5 Aggregator Management This chapter explains the different protocols or services used to manage an Aggregator including: • • • • • • • • Logging Disabling System Logging File Transfer Services Terminal Lines Telnet to Another Network Device Recovering from a Forgotten Password Recovering from a Forgotten Enable Password Recovering from a Failed Start Logging FTOS tracks changes in the system using event and error messages.
www.dell.com | support.dell.com Disabling System Logging By default, logging is enabled and log messages are sent to the logging buffer, all terminal lines, console, and syslog servers. To enable and disable system logging: Task Command Syntax Command Mode Disable all logging except on the console. no logging on CONFIGURATION Disable logging to the logging buffer. no logging buffer CONFIGURATION Disable logging to terminal lines. no logging monitor CONFIGURATION Disable console logging.
Changing System Logging Settings You can change the default settings of the system logging by changing the severity level and the storage location. The default is to log all messages up to debug level, that is, all system messages. By changing the severity level in the logging commands, you control the number of system messages logged.
www.dell.com | support.dell.com Displaying the Logging Buffer and Logging Configuration To display the current contents of the logging buffer and the logging settings for the system, enter the show logging command in EXEC privilege mode (Figure 5-1). Figure 5-1.
Configuring a UNIX Logging Facility Level You can save system log messages with a UNIX system logging facility. To configure a UNIX logging facility level, use the following command in CONFIGURATION mode: Command Syntax Command Mode Purpose logging facility [facility-type] CONFIGURATION Specify one of the following parameters.
www.dell.com | support.dell.com Enabling Time Stamps on Syslog Messages By default, syslog messages do not include a time/date stamp stating when the error or message was created. To have FTOS include a timestamp with the syslog message, use the following command syntax in CONFIGURATION mode: Command Syntax Command Mode Purpose service timestamps [log | debug] [datetime [localtime] [msec] [show-timezone] | uptime] CONFIGURATION Add timestamp to syslog messages.
Enabling the FTP Server To enable the system as an FTP server, use the following command in CONFIGURATION mode: Command Syntax Command Mode Purpose ftp-server enable CONFIGURATION Enable FTP on the system. To view the FTP configuration, enter the show running-config ftp command in EXEC privilege mode (Figure 5-3). Figure 5-3.
www.dell.com | support.dell.com Terminal Lines You can access the system remotely and restrict access to the system by creating user profiles. The terminal lines on the system provide different means of accessing the system. The virtual terminal lines (VTY) connect you through Telnet to the system. Telnet to Another Network Device To telnet to another device (Figure 5-4): Task Command Syntax Command Mode Telnet to the stack-unit.
Recovering from a Forgotten Password If you configure authentication for the console and you exit out of EXEC mode or your console session times out, you are prompted to re-enter the password. If you forget your password, follow these steps: Step Task Command Syntax Command Mode 1 Log onto the system using the console. 2 Power-cycle the Aggregator by using the CMC interface or removing it from the M1000e and re-inserting it in the chassis.
www.dell.com | support.dell.com Step Task Command Syntax Command Mode 4 Set the system parameters to ignore the enable password when the system reloads. ignore enable-password BOOT USER 5 Reload the system. reload BOOT USER 6 Configure a new enable password. copy flash://startup-config.bak running-config EXEC Privilege 7 Configure a new enable password. enable {secret | password) CONFIGURATION 8 Save the running-config to the startup-config.
Figure 5-5. Recovering from a Failed Start: Example U-Boot 2010.03-rc1(Dell Force10) Built by build at tools-sjc-01 on Thu May 31 23:53:38 2012 IOM Boot Label 4.0.1.0 DRAM: 2 GB Initialized CPLD on CS3 Detected [XLP308 (Lite+) Rev A0] Initializing I2C0: speed = 30 KHz, prescaler = 0x0377 -- done. Initializing I2C1: speed = 100 KHz, prescaler = 0x0109 -- done.
50 | Aggregator Management www.dell.com | support.dell.
6 Data Center Bridging (DCB) On an Aggregator, data center bridging (DCB) features are auto-configured in standalone mode. You can display information on DCB operation by using show commands. Note: DCB features are not supported on an Aggregator in stacking mode.
www.dell.com | support.dell.com Data center bridging satisfies the needs of the following types of data center traffic in a unified fabric: • • • LAN traffic consists of a large number of flows that are generally insensitive to latency requirements, while certain applications, such as streaming video, are more sensitive to latency. Ethernet functions as a best-effort network that may drop packets in case of network congestion.
Figure 6-1. Priority-Based Flow Control PFC is implemented as follows in the Dell Force10 operating software (FTOS): • • • • • • PFC is supported on specified 802.1p priority traffic (dot1p 0 to 7) and is configured per interface. However, only two lossless queues are supported on an interface: one for FCoE converged traffic and one for SCSI storage traffic. You must configure the same lossless queues on all ports.
www.dell.com | support.dell.com Although you can configure strict-priority queue scheduling for a priority group, ETS introduces flexibility that allows the bandwidth allocated to each priority group to be dynamically managed according to the amount of LAN, storage, and server traffic in a flow. Unused bandwidth in a priority-group is dynamically allocated to other priority groups for which traffic is available to be scheduled. Traffic is queued according to its 802.
Data Center Bridging Exchange Protocol (DCBX) The data center bridging exchange (DCBX) protocol is enabled by default on any switch on which PFC or ETS are enabled. DCBX allows a switch to automatically discover DCB-enabled peers and exchange configuration information. PFC and ETS use DCBX to exchange and negotiate parameters with peer devices.
www.dell.com | support.dell.com Data Center Bridging: Auto-DCB-Enable Mode On an Aggregator in standalone mode, the default mode of operation for data center bridging on Ethernet ports is auto-DCB-enable mode.
Figure 6-4.
www.dell.com | support.dell.com Enabling DCB on Next Reload To configure the Aggregator so that all interfaces come up with DCB enabled and flow control disabled, use the dcb enable on-next-reload command. You must save the configuration change and reload the switch for DCB to be enabled on all interfaces. Internal PFC buffers are automatically configured. Task Command Command Mode Globally enable DCB on all interfaces after next switch reload.
QoS dot1p Traffic Classification and Queue Assignment DCB supports PFC, ETS, and DCBX to handle converged Ethernet traffic that is assigned to an egress queue according to the following quality of service (QoS) methods: • • Important: of two Honor dot1p: dot1p priorities in ingress traffic are used at the port or global switch level. Layer 2 class maps: dot1p priorities are used to classify traffic in a class map and apply a service policy to an ingress port to map traffic to egress queues.
www.dell.com | support.dell.com How Priority-Based Flow Control is Implemented Priority-based flow control provides a flow control mechanism based on the 802.1p priorities in converged Ethernet traffic received on an interface and is enabled by default. As an enhancement to the existing Ethernet pause mechanism, PFC stops traffic transmission for specified priorities (CoS values) without impacting other priority classes. Different traffic types are assigned to different priority classes.
How Enhanced Transmission Selection is Implemented Enhanced transmission selection (ETS) provides a way to optimize bandwidth allocation to outbound 802.1p classes of converged Ethernet traffic. Different traffic types have different service needs. Using ETS, groups within an 802.1p priority class are auto-configured to provide different treatment for traffic with different bandwidth, latency, and best-effort needs.
www.dell.com | support.dell.com • • When an ETS output policy is applied to an interface, ETS-configured scheduling and bandwidth allocation take precedence over any auto-configured settings in the QoS output policies. ETS is enabled by default with the default ETS configuration applied (all dot1p priorities in the same group with equal bandwidth allocation).
DCB Policies in a Switch Stack A DCB input policy with PFC and ETS configuration is applied to all stacked ports in a switch stack or on a stacked switch. DCBX Operation The data center bridging exchange protocol (DCBX) is used by DCB devices to exchange configuration information with directly connected peers using the link layer discovery protocol (LLDP) protocol.
www.dell.com | support.dell.com DCBX Port Roles The following DCBX port roles are auto-configured on an Aggregator to propagate DCB configurations learned from peer DCBX devices internally to other switch ports: • • Auto-upstream: The port advertises its own configuration to DCBX peers and receives its configuration from DCBX peers (ToR or FCF device). The port also propagates its configuration to other ports on the switch.
Note: On a DCBX port, application priority TLV advertisements are handled as follows: - The application priority TLV is transmitted only if the priorities in the advertisement match the configured PFC priorities on the port. - On auto-upstream and auto-downstream ports: - If a configuration source is elected, the ports send an application priority TLV based on the application priority TLV received on the configuration-source port.
www.dell.com | support.dell.com Configuration Source Election When an auto-upstream or auto-downstream port receives a DCB configuration from a peer, the port first checks to see if there is an active configuration source on the switch. • • If a configuration source already exists, the received peer configuration is checked against the local port configuration. If the received configuration is compatible, the DCBX marks the port as DCBX-enabled.
Auto-Detection of the DCBX Version The Aggregator operates in auto-detection mode so that a DCBX port automatically detects the DCBX version on a peer port. Legacy CIN and CEE versions are supported in addition to the standard IEEE version 2.5 DCBX. A DCBX port detects a peer version after receiving a valid frame for that version.
www.dell.com | support.dell.com Figure 6-7. DCBX Sample Topology DCBX Prerequisites and Restrictions • • 68 | DCBX requires LLDP in both send (TX) and receive (RX) mode to be enabled on a port interface. If multiple DCBX peer ports are detected on a local DCBX interface, LLDP is shut down. The CIN version of DCBX supports only PFC, ETS, and FCOE; it does not support iSCSI, backward congestion management (BCN), logical link down (LLD), and network interface virtualization (NIV).
DCBX Error Messages An error in DCBX operation is displayed using the following syslog messages: LLDP_MULTIPLE_PEER_DETECTED: DCBX is operationally disabled after detecting more than one DCBX peer on the port interface. LLDP_PEER_AGE_OUT: DCBX is disabled as a result of LLDP timing out on a DCBX peer interface. DSM_DCBX_PEER_VERSION_CONFLICT: A local port expected to receive the IEEE, CIN, or CEE version in a DCBX TLV from a remote peer but received a different, conflicting DCBX version.
www.dell.com | support.dell.com Verifying DCB Configuration Use the show commands in Table 6-2 to display DCB configurations and statistics. Table 6-2. Displaying DCB Configurations Command Output show dcb [stack-unit unit-number] (Figure 6-8) Displays data center bridging status, number of PFC-enabled ports, and number of PFC-enabled queues. On the master switch in a stack, you can specify a stack-unit number. Valid values: 0 to 5.
Figure 6-10.
www.dell.com | support.dell.com Table 6-3. 72 show interface pfc summary Command Description Field | Description Remote is enabled, Priority list Remote Willing Status is enabled Operational status (enabled or disabled) of peer device for DCBX exchange of PFC configuration with a list of the configured PFC priorities. Willing status of peer device for DCBX exchange (Willing bit received in PFC TLV): enabled or disabled.
Figure 6-11.
www.dell.com | support.dell.com Figure 6-12.
Table 6-4. show interface ets detail Command Description Field Description Admin mode ETS mode: on or off. When on, the scheduling and bandwidth allocation configured in an ETS output policy or received in a DCBX TLV from a peer can take effect on an interface. Admin Parameters ETS configuration on local port, including priority groups, assigned dot1p priorities, and bandwidth allocation.
www.dell.com | support.dell.com Figure 6-13.
Figure 6-15.
www.dell.com | support.dell.com Table 6-5. 78 show interface dcbx detail Command Description Field | Description Local DCBX Configured mode DCBX version configured on the port: CEE, CIN, IEEE v2.5, or Auto (port auto-configures to use the DCBX version received from a peer). Peer Operating version DCBX version that the peer uses to exchange DCB parameters.
Example: PFC and ETS Operation This section contains examples of DCB input and output policies applied on an interface. In the example shown in Figure 6-16 for an Aggregator: • • • Incoming SAN traffic is configured for priority-based flow control. Outbound LAN, IPC, and SAN traffic is mapped into three ETS priority groups and configured for enhanced traffic selection (bandwidth allocation and scheduling). One lossless queue is used. Figure 6-16.
www.dell.com | support.dell.com QoS Traffic Classification: On the Aggregator, the service-class dynamic dot1p command has been used in Global Configuration mode to map ingress dot1p frames to the queues shown in Table 6-6. For more information, refer to QoS dot1p Traffic Classification and Queue Assignment. Table 6-6.
Table 6-8. Example: priority group-bandwidth Assignment Priority Group Bandwidth Assignment IPC 5% SAN 50% LAN 45% Hierarchical Scheduling in ETS Output Policies On an Aggregator, ETS supports up to three levels of hierarchical scheduling. For example, ETS output policies with the following configurations can be applied: • • • Priority group 1 assigns traffic to one priority queue with 20% of the link bandwidth and strict-priority scheduling.
82 | Data Center Bridging (DCB) www.dell.com | support.dell.
Skippy812 7 Dynamic Host Configuration Protocol (DHCP) The Aggregator is auto-configured to operate as a DHCP client. The DHCP server, DHCP relay agent, and secure DHCP features are not supported. DHCP Overview Dynamic host configuration protocol (DHCP) is an application layer protocol that dynamically assigns IP addresses and other configuration parameters to network end-stations (hosts) based on configuration policies determined by network administrators.
www.dell.com | support.dell.com DHCP Packet Format and Options DHCP uses the user datagram protocol (UDP) as its transport protocol. The server listens on port 67 and transmits to port 68; the client listens on port 68 and transmits to port 67. The configuration parameters are carried as options in the DHCP packet in type, length, value (TLV) format; many options are specified in RFC 2132.
Assigning an IP Address Using DHCP When a client joins a network: 1. The client initially broadcasts a DHCPDISCOVER message on the subnet to discover available DHCP servers. This message includes the parameters that the client requires and might include suggested values for those parameters. 2. Servers unicast or broadcast a DHCPOFFER message in response to the DHCPDISCOVER that offers to the client values for the requested parameters.
www.dell.com | support.dell.com FTOS Behavior: DHCP is implemented in FTOS based on RFC 2131 and 3046. DHCP Client An Aggregator is auto-configured to operate as a DHCP client. The DHCP client functionality is enabled only on the default VLAN and the management interface. A DHCP client is a network device that requests an IP address and configuration parameters from a DHCP server.
Releasing and Renewing DHCP-based IP Addresses On an Aggregator configured as a DHCP client, you can release a dynamically-assigned IP address without removing the DHCP client operation on the interface. You can later manually acquire a new IP address from the DHCP server as follows: Task Command Syntax Command Mode Release a dynamically-acquired IP address while retaining the DHCP client configuration on the interface.
www.dell.com | support.dell.com Figure 7-4. show ip dhcp lease FTOS# show ip dhcp Interface ========= Ma 0/0 Vl 1 Lease-IP Def-Router ======== ========= 0.0.0.0/0 0.0.0.0 10.1.1.254/24 0.0.0.0 ServerId ======== 0.0.0.0 10.1.1.
Figure 7-5.
www.dell.com | support.dell.com Figure 7-6 shows an example of the packet- and event-level debug messages displayed for the packet transmissions and state transitions on a DHCP client interface when you release and renew a DHCP client. Figure 7-6.
DHCP Client on a Management Interface The following conditions apply on a management interface that operates as a DHCP client: • • • • • • • The management default route is added with the gateway as the router IP address received in the DHCP ACK packet. This is required to send and receive traffic to and from other subnets on the external network. This route is added irrespective both when the DHCP client and server are in the same or different subnets.
www.dell.com | support.dell.com DHCP Client on a VLAN The following conditions apply on a VLAN that operates as a DHCP client: • • The default VLAN 1 with all ports auto-configured as members is the only L3 interface on the Aggregator. When the default management VLAN has a DHCP-assigned address and you reconfigure the default VLAN ID number, the Aggregator: • Sends a DHCP release to the DHCP server to release the IP address. • Sends a DHCP request to obtain a new IP address.
8 FIP Snooping FIP snooping is auto-configured on an Aggregator in standalone mode. You can display information on FIP snooping operation and statistics by entering show commands.
www.dell.com | support.dell.com To ensure similar Fibre Channel robustness and security with FCoE in an Ethernet cloud network, the Fibre Channel over Ethernet initialization protocol (FIP) establishes virtual point-to-point links between FCoE end-devices (server ENodes and target storage devices) and FCoE forwarders (FCFs) over transit FCoE-enabled bridges.
Figure 8-1. FIP discovery and login between an ENode and an FCF FIP Snooping on Ethernet Bridges In a converged Ethernet network, intermediate Ethernet bridges can snoop on FIP packets during the login process on an FCF. Then, using ACLs, a transit bridge can permit only authorized FCoE traffic to be transmitted between an FCoE end-device and an FCF. An Ethernet bridge that provides these functions is called a FIP snooping bridge (FSB).
www.dell.com | support.dell.com Dynamic ACL generation on an Aggregator operating as a FIP snooping bridge functions as follows: • • • • Global ACLs are applied on server-facing ENode ports. Port-based ACLs are applied on ports directly connected to an FCF and on server-facing ENode ports. Port-based ACLs take precedence over global ACLs. FCoE-generated ACLs take precedence over user-configured ACLs. A user-configured ACL entry cannot deny FCoE and FIP snooping frames.
The following sections describe how to configure the FIP snooping feature on a switch that functions as a FIP snooping bridge so that it can perform the following functions: • • • • • Perform FIP snooping (allowing and parsing FIP frames) globally on all VLANs or on a per-VLAN basis. Set the FCoE MAC address prefix (FC-MAP) value used by an FCF to assign a MAC address to an FCoE end-device (server ENode or storage device) after a server successfully logs in.
www.dell.com | support.dell.com FIP Snooping on VLANs FIP snooping is enabled globally on an Aggregator on all VLANs: • • • • FIP frames are allowed to pass through the switch on the enabled VLANs and are processed to generate FIP snooping ACLs. FCoE traffic is allowed on VLANs only after a successful virtual-link initialization (fabric login FLOGI) between an ENode and an FCF. All other FCoE traffic is dropped.
FIP Snooping Prerequisites On an Aggregator, FIP snooping requires the following conditions: • • A FIP snooping bridge requires DCBX and PFC to be enabled on the switch for lossless Ethernet connections (refer to Data Center Bridging (DCB)). Dell recommends that you also enable ETS; ETS is recommended but not required. DCBX and PFC mode are auto-configured on Aggregator ports and FIP snooping is operational on the port.
www.dell.com | support.dell.com Displaying FIP Snooping Information Use the show commands in Table 8-1 to display information on FIP snooping. Table 8-1.
Figure 8-3. show fip-snooping sessions Command Example FTOS#show fip-snooping sessions Enode MAC Enode Intf aa:bb:cc:00:00:00 Te 0/42 aa:bb:cc:00:00:00 Te 0/42 aa:bb:cc:00:00:00 Te 0/42 aa:bb:cc:00:00:00 Te 0/42 aa:bb:cc:00:00:00 Te 0/42 FCoE MAC 0e:fc:00:01:00:01 0e:fc:00:01:00:02 0e:fc:00:01:00:03 0e:fc:00:01:00:04 0e:fc:00:01:00:05 Table 8-2.
www.dell.com | support.dell.com Figure 8-5. show fip-snooping enode Command Example FTOS# show fip-snooping enode Enode MAC Enode Interface ----------------------d4:ae:52:1b:e3:cd Te 0/11 Table 8-3. FCF MAC ------54:7f:ee:37:34:40 Description ENode MAC MAC address of the ENode. ENode Interface Slot/ port number of the interface connected to the ENode. FCF MAC MAC address of the FCF. VLAN VLAN ID number used by the session. FC-ID Fibre Channel session ID assigned by the FCF.
Figure 8-7.
www.dell.com | support.dell.com Figure 8-8.
Table 8-5. show fip-snooping statistics Command Descriptions Field Description Number of Vlan Requests Number of FIP-snooped VLAN request frames received on the interface. Number of VLAN Notifications Number of FIP-snooped VLAN notification frames received on the interface. Number of Multicast Discovery Solicits Number of FIP-snooped multicast discovery solicit frames received on the interface.
www.dell.com | support.dell.com Figure 8-9. show fip-snooping system Command Example FTOS# show fip-snooping system Global Mode FCOE VLAN List (Operational) FCFs Enodes Sessions : : : : : Enabled 1, 100 1 2 17 Note: NPIV sessions are included in the number of FIP-snooped sessions displayed. Figure 8-10.
FIP Snooping Example Figure 8-11 shows an Aggregator used as a FIP snooping bridge for FCoE traffic between an ENode (server blade) and an FCF (ToR switch). The ToR switch operates as an FCF and FCoE gateway. Figure 8-11. Example: FIP Snooping on an Aggregator In Figure 8-11, DCBX and PFC are enabled on the Aggregator (FIP snooping bridge) and on the FCF ToR switch. On the FIP snooping bridge, DCBX is configured as follows: • • A server-facing port is configured for DCBX in an auto-downstream role.
www.dell.com | support.dell.com Debugging FIP Snooping To enable debug messages for FIP snooping events, enter the debug fip-snooping command. Task Command Command Mode Enable FIP snooping debugging on for all or a specified event type, where: all enables all debugging options. acl enables debugging only for ACL-specific events. error enables debugging only for error conditions. ifm enables debugging only for IFM events. info enables debugging only for information events.
9 Internet Group Management Protocol (IGMP) On an Aggregator, IGMP snooping is auto-configured. You can display information on IGMP by using show commands. Multicast is based on identifying many hosts by a single destination IP address. Hosts represented by the same IP address are a multicast group. The internet group management protocol (IGMP) is a Layer 3 multicast protocol that hosts use to join or leave a multicast group.
www.dell.com | support.dell.com Figure 9-1. IGMP Version 2 Packet Format Preamble IHL Version (4) TOS (0xc0) Total Length Start Frame Delimiter Destination MAC Flags Frag Offset Source MAC TTL (1) Protocol (2) Padding IP Packet Ethernet Type Header Checksum Src IP Addr Dest IP Addr FCS Options (Router Alert) Type Padding Max.
IGMP Version 3 Conceptually, IGMP version 3 behaves the same as version 2. However, there are differences: • Version 3 adds the ability to filter by multicast source, which helps the multicast routing protocols avoid forwarding traffic to subnets where there are no interested receivers. To enable filtering, routers must keep track of more state information, that is, the list of sources that must be filtered.
www.dell.com | support.dell.com Joining and Filtering Groups and Sources Figure 9-4 shows how multicast routers maintain the group and source information from unsolicited reports. 1. The first unsolicited report from the host indicates that it wants to receive traffic for group 224.1.1.1. 2. The host’s second report indicates that it is only interested in traffic from group 224.1.1.1, source 10.11.1.1.
Leaving and Staying in Groups Figure 9-5 shows how multicast routers track and refresh state changes in response to group-and-specific and general queries. 1. Host 1 sends a message indicating it is leaving group 224.1.1.1 and that the included filter for 10.11.1.1 and 10.11.1.2 are no longer necessary. 2. The querier, before making any state changes, sends a group-and-source query to see if any other host is interested in these two sources; queries for state-changes are retransmitted multiple times.
www.dell.com | support.dell.com How IGMP Snooping is Implemented on an Aggregator • • • • • • • • • • IGMP snooping is enabled by default on the switch. FTOS supports version 1, version 2, and version 3 hosts. FTOS IGMP snooping is based on the IP multicast address (not on the Layer 2 multicast MAC address). IGMP snooping entries are stored in the Layer 3 flow table instead of in the Layer 2 forwarding information base (FIB). FTOS IGMP snooping is based on draft-ietf-magma-snoop-10.
Figure 9-6. show ip igmp groups Command Example FTOS# show ip igmp groups Total Number of Groups: 2 IGMP Connected Group Membership Group Address Interface 226.0.0.1 Vlan 1500 226.0.0.1 Vlan 1600 FTOS#show ip igmp groups detail Interface Group Uptime Expires Router mode Last reporter Last reporter mode Last report received Group source list Source address 1.1.1.2 Member Ports: Po 1 Vlan 1500 226.0.0.1 00:00:21 Never INCLUDE 1.1.1.
www.dell.com | support.dell.com Figure 9-7.
10 Interfaces This chapter describes the auto-configuration of 1 Gigabit and 10 Gigabit Ethernet interfaces (physical and logical) on an Aggregator.
www.dell.com | support.dell.com Interface Auto-Configuration An Aggregator auto-configures interfaces as follows: • • • • All interfaces operate as layer 2 interfaces at 10GbE in standalone mode. FlexIO module interfaces support only uplink connections. Only the 40GbE ports on the base module can be used for stacking. • By default, the two fixed 40GbE ports on the base module operate in 4x10GbE mode with breakout cables and support up to eight 10GbE uplinks.
Viewing Interface Information You can use show commands to view interface status and auto-configured parameters. The show interfaces command in EXEC mode lists all configurable interfaces on the chassis and has options to display the interface status, IP and MAC addresses, and multiple counters for the amount and type of traffic passing through the interface. If a port channel interface is configured, the show interfaces command can list the interfaces configured in the port channel.
www.dell.com | support.dell.com Use the show ip interfaces brief command in EXEC Privilege mode to view which interfaces are enabled for Layer 3 data transmission. In Figure 10-2, the TenGigabitEthernet interface 1/5 is in Layer 3 mode because an IP address has been assigned to it and the interface’s status is operationally up. Figure 10-2.
Disabling and Re-enabling a Physical Interface By default, all port interfaces on an Aggregator are operationally enabled (no shutdown) to send and receive Layer 2 traffic. You can reconfigure a physical interface to shut it down by entering the shutdown command. To re-enable the interface, enter the no shutdown command.
www.dell.com | support.dell.com Layer 2 Mode On an Aggregator, physical interfaces, port channels, and VLANs auto-configure to operate in Layer 2 mode. Figure 10-4 shows the basic configuration found in a Layer 2 interface. Note: Layer 3 (network) mode is not supported on Aggregator physical interfaces, port channels, and VLANs. Only management interfaces operate in Layer 3 mode. Figure 10-4.
The Aggregator supports the management ethernet interface as well as the standard interface on any front-end port. You can use either method to connect to the system. Configuring a Management Interface On the Aggregator, the dedicated management interface provides management access to the system.You can configure this interface with FTOS, but the configuration options on this interface are limited. You cannot configure gateway addresses and IP addresses if it appears in the main routing table of FTOS.
www.dell.com | support.dell.com To display the configuration for a given port, use the show interface command from EXEC Privilege mode (Figure 10-5). To display the routing table for a given port, use the show ip route command from EXEC Privilege mode. Figure 10-5.
Configuring a Static Route for a Management Interface When an IP address used by a protocol and a static management route exists for the same prefix, the protocol route takes precedence over the static management route.
www.dell.com | support.dell.com VLAN Membership A virtual LAN (VLANs) is a logical broadcast domain or logical grouping of interfaces in a LAN in which all data received is kept locally and broadcast to all members of the group. In Layer 2 mode, VLANs move traffic at wire speed and can span multiple devices. FTOS supports up to 4093 port-based VLANs and one default VLAN, as specified in IEEE 802.1Q.
Port-Based VLANs Port-based VLANs are a broadcast domain defined by different ports or interfaces. In FTOS, a port-based VLAN can contain interfaces from different stack units within the chassis. FTOS supports 4094 port-based VLANs. Port-based VLANs offer increased security for traffic, conserve bandwidth, and allow switch segmentation. Interfaces in different VLANs do not communicate with each other, adding some security to the traffic on those interfaces.
www.dell.com | support.dell.com Configuring VLAN Membership By default, all Aggregator ports are member of all (4094) VLANs, including the default untagged VLAN 1. You can use the CLI or CMC interface to reconfigure VLANs only on server-facing interfaces (1 to 32) so that an interface has membership only in specified VLANs. To assign an Aggregator interface in Layer 2 mode to a specified group of VLANs, use the vlan tagged and vlan untagged commands.
Displaying VLAN Membership To view the configured VLANs, enter the show vlan command in EXEC privilege mode: Figure 10-8.
www.dell.com | support.dell.com Adding an Interface to a Tagged VLAN Figure 10-9 shows an example of how to add a tagged interface (port channel 1) to a VLAN (VLAN 4). Figure 10-9. Adding an Interface to Another VLAN FTOS(conf-if-te-1/7)# vlan tagged 2 FTOS(conf-if-te-1/7)# exit FTOS(conf)# exit FTOS# show vlan id 2 Enter the vlan tagged command to add interface Te 1/7 to VLAN 4. Enter the show vlan command to verify that interface Te 1/7 is a tagged member of VLAN 2.
Adding an Interface to an Untagged VLAN To move an untagged interfaces from the default VLAN to another VLAN, use the vlan untagged command as shown in Figure 10-10. Figure 10-10. Moving an Untagged Interface to Another VLAN FTOS(conf)# interface tengigabit 0/16 Enter the vlan untagged command to FTOS(conf-if-te-0/16)# vlan untagged 4 as an untagged member of VLAN 4.
www.dell.com | support.dell.com Port Channel Definition and Standards Link aggregation is defined by IEEE 802.3ad as a method of grouping multiple physical interfaces into a single logical interface—a link aggregation group (LAG) or port channel. A LAG is “a group of links that appear to a MAC client as if they were a single link” according to IEEE 802.3ad. In FTOS, a LAG is referred to as a port channel interface.
Port channels can contain a mix of 100, 1000, or 10000 Mbps Ethernet interfaces and TenGigabit Ethernet interfaces. The interface speed (100, 1000, or 10000 Mbps) used by the port channel is determined by the first port channel member that is physically up. FTOS disables the interfaces that do not match the interface speed set by the first channel member. That first interface may be the first interface that is physically brought up or was physically operating when interfaces were added to the port channel.
www.dell.com | support.dell.com Displaying Port Channel Information To view the port channel’s status and channel members in a tabular format, use the show interfaces port-channel brief command in EXEC Privilege mode (Figure 10-11). Figure 10-11.
Figure 10-12. show interface port-channel Command Example FTOS#show interface port-channel Port-channel 1 is a dynamically-created port channel based on the Port-channel 1 is up, line protocol is up NIC teaming configuration in connected servers learned via LACP.
www.dell.com | support.dell.com Interface Range An interface range is a set of interfaces to which other commands may be applied, and may be created if there is at least one valid interface within the range. Bulk configuration excludes from configuring any non-existing interfaces from an interface range. A default VLAN may be configured only if the interface range being configured consists of only VLAN ports.
Create a Multiple-Range Figure 10-14. Creating a Multiple-Range Prompt FTOS(conf)#interface range tengigabitethernet 3/0 , tengigabitethernet 2/1 - 47 , vlan 1000 FTOS(conf-if-range-te-2/1-47)# Exclude a Smaller Port Range If the interface range has multiple port ranges, the smaller port range is excluded from the prompt. Figure 10-15.
www.dell.com | support.dell.com Command Syntax Command Mode Purpose monitor interface interface EXEC Privilege View interface statistics. Enter the type of interface and slot/ port information: • For a 1GbE interface, enter the keyword GigabitEthernet followed by the slot/port numbers; for example, interface tengigabitethernet 0/12. • For a 10GbE interface, enter the keyword TenGigabitEthernet followed by the slot/port numbers; for example, interface tengigabitethernet 0/44.
Maintenance Using TDR The time domain reflectometer (TDR) is supported on all Dell Force10 switch/routers. TDR is an assistance tool to resolve link issues that helps detect obvious open or short conditions within any of the four copper pairs. TDR sends a signal onto the physical cable and examines the reflection of the signal that returns. By examining the reflection, TDR is able to indicate whether there is a cable fault (when the cable is broken, becomes unterminated, or if a transceiver is unplugged).
www.dell.com | support.dell.com Flow Control Using Ethernet Pause Frames An Aggregator auto-configures to operate in auto-DCB-enable mode (see Data Center Bridging: Auto-DCB-Enable Mode).
MTU Size The Aggregator auto-configures interfaces to use a maximum MTU size of 12,000 bytes. If a packet includes a Layer 2 header, the difference in bytes between the link MTU and IP MTU must be enough to include the Layer 2 header. For example, for VLAN packets, if the IP MTU is 1400, the link MTU must be no less than 1422: 1400-byte IP MTU + 22-byte VLAN Tag = 1422-byte link MTU The MTU range is 592-12000, with a default of 1554.
www.dell.com | support.dell.com Auto-Negotiation on Ethernet Interfaces Setting Speed and Duplex Mode of Ethernet Interfaces By default, auto-negotiation of speed and duplex mode is enabled on 1GbE and 10GbE Ethernet interfaces on an Aggregator. The local interface and the directly connected remote interface must have the same setting. Auto-negotiation is the easiest way to accomplish these settings, as long as the remote interface is capable of auto-negotiation.
Figure 10-19.
www.dell.com | support.dell.
Figure 10-21 shows the auto-negotiation options. Figure 10-21.
www.dell.com | support.dell.com In EXEC mode, the show interfaces switchport command displays only interfaces in Layer 2 mode and their relevant configuration information. The show interfaces switchport command (Figure 10-23) displays the interface, whether the interface supports IEEE 802.1Q tagging or not, and the VLANs to which the interface belongs. Figure 10-23. show interfaces switchport Command Example FTOS#show interfaces switchport Name: TenGigabitEthernet 13/0 802.
Clearing Interface Counters The counters in the show interfaces command are reset by the clear counters command. This command does not clear the counters captured by any SNMP program. To clear the counters, use the following command in EXEC Privilege mode: Command Syntax Command Mode Purpose clear counters [interface] EXEC Privilege Clear the counters used in the show interface commands for all VRRP groups, VLANs, and physical interfaces or selected ones.
148 | Interfaces www.dell.com | support.dell.
11 iSCSI Optimization An Aggregator auto-configures to enable internet small computer system interface (iSCSI) optimization with default iSCSI parameter settings (Default iSCSI Optimization Values) and is auto-provisioned to support: • • Detection and Autoconfiguration for Dell EqualLogic Arrays Detection and Port Configuration for Dell Compellent Arrays You can use show commands to display information on iSCSI configuration and sessions (Displaying iSCSI Optimization Information).
www.dell.com | support.dell.com Figure 11-1. iSCSI Optimization Example Monitoring iSCSI Traffic Flows The switch snoops iSCSI session-establishment and termination packets by installing classifier rules that trap iSCSI protocol packets to the CPU for examination. Devices that initiate iSCSI sessions usually use well-known TCP ports 3260 or 860 to contact targets. When you enable iSCSI optimization, by default the switch identifies IP packets to or from these ports as iSCSI traffic.
If no iSCSI traffic is detected for a session during a user-configurable aging period, the session data is cleared. Detection and Autoconfiguration for Dell EqualLogic Arrays The iSCSI optimization feature includes auto-provisioning support with the ability to detect directly connected Dell EqualLogic storage arrays and automatically reconfigure the switch to enhance storage traffic flows. The Aggregator uses the link layer discovery protocol (LLDP) to discover Dell EqualLogic devices on the network.
www.dell.com | support.dell.com The Aggregator auto-configures as follows: • • Jumbo frame size is set to 12000 for all interfaces on all ports and port-channels, if it is not already enabled. Unicast storm control is disabled on the interface identified by LLDP. iSCSI Optimization: Operation When the Aggregator auto-configures with iSCSI enabled, the following actions occur: • • • Link-level flow control is globally enabled and PFC is disabled. iSCSI session snooping is enabled.
Displaying iSCSI Optimization Information Use the show commands in Table 11-2 to display information on iSCSI optimization. Table 11-2. Displaying iSCSI Optimization Information Command Output show iscsi (Figure 11-2) Displays the currently configured iSCSI settings. show iscsi sessions (Figure 11-3) Displays information on active iSCSI sessions on the switch. show iscsi sessions detailed [session isid] (Figure 11-4) Displays detailed information on active iSCSI sessions on the switch.
www.dell.com | support.dell.com Figure 11-4. 154 show iscsi sessions detailed Command Example FTOS# show isci sessions detailed Session 0 : ----------------------------------------------------------------------------Target:iqn.2010-11.com.ixia:ixload:iscsi-TG1 Initiator:iqn.2010-11.com.ixia.ixload:initiator-iscsi-2c Up Time:00:00:01:28(DD:HH:MM:SS) Time for aging out:00:00:09:34(DD:HH:MM:SS) ISID:806978696102 Initiator Initiator Target Target Connection IP Address TCP Port IP Address TCPPort ID 10.10.0.
12 Link Aggregation The Aggregator auto-configures with link aggregation groups (LAGs) as follows: • • All uplink ports are automatically configured in a single port channel (LAG 128). Server-facing LAGs are automatically configured if server NICs are configured for LACP-based NIC teaming (Network Interface Controller (NIC) Teaming). No manual configuration is required to configure Aggregator ports in the uplink or a server-facing LAG. Note: Static LAGs are not supported on the Aggregator.
www.dell.com | support.dell.com Server-Facing LAGs Server-facing ports are configured as individual ports by default. If a server NIC is configured for LACP-based NIC teaming, server-facing ports are automatically configured as part of dynamic LAGs. The LAG range 1 to127 is reserved for server-facing LAGs. Once the Aggregator receives LACPDU from server-facing ports, the information embedded in the LACPDU (remote-system ID and port key) is used to form a server-facing LAG.
LACP Example Figure 12-1 shows an example of how LACP operates in an Aggregator stack by auto-configuring the uplink LAG 128 for the connection to a ToR switch and a server-facing LAG for the connection to an installed server that is configured for LACP-based NIC teaming. Figure 12-1.
www.dell.com | support.dell.com Verifying LACP Operation and LAG Configuration 158 | To verify the operational status and configuration of a dynamically created LAG, and LACP operation on a LAG on an Aggregator, enter the show interfaces port-channel port-channel-number and show lacp port-channel-number commands. The show outputs in this section for uplink LAG 128 and server-facing LAG 1 refer to the example shown in Figure 12-1. Figure 12-2.
Figure 12-3. show lacp 128 Command Example FTOS# show lacp 128 Port-channel 128 admin up, oper up, mode lacp Actor System ID: Priority 32768, Address 0001.e8e1.e1c3 Partner System ID: Priority 32768, Address 0001.e88b.
www.dell.com | support.dell.com Figure 12-4.
Figure 12-5.
162 | Link Aggregation www.dell.com | support.dell.
13 Layer 2 The Aggregator supports CLI commands to manage the MAC address table: • • Clearing MAC Address Entries Displaying the MAC Address Table The Aggregator auto-configures with support for Network Interface Controller (NIC) Teaming. Note: On an Aggregator, all ports are configured by default as members of all (4094) VLANs, including the default VLAN. All VLANs operate in Layer 2 mode.
www.dell.com | support.dell.com Task Command Syntax Command Mode Clear a MAC address table of dynamic entries.
Figure 13-1. Redundant NICs with NIC Teaming X Port 0/1 MAC: A:B:C:D A:B IP: 1.1.1.1 k Active Lin Port 0/5 fnC0025mp MAC Address Station Move When you use NIC teaming, consider that the server MAC address is originally learned on Port 0/1 of the switch (Figure 13-2). If the NIC fails, the same MAC address is learned on Port 0/5 of the switch. The MAC address is disassociated with the one port and re-associated with another in the ARP table; in other words, the ARP entry is “moved”.
166 | Layer 2 www.dell.com | support.dell.
14 Link Layer Discovery Protocol (LLDP) An Aggregator auto-configures to support the link layer discovery protocol (LLDP) for the auto-discovery of network devices. You can use CLI commands to display acquired LLDP information, clear LLDP counters, and debug LACP operation. Overview LLDP—defined by IEEE 802.1AB—is a protocol that enables a local area network (LAN) device to advertise its configuration and receive configuration information from adjacent LLDP-enabled LAN infrastructure devices.
www.dell.com | support.dell.com There are five types of TLVs (Table 14-1). All types are mandatory in the construction of an LLDPDU except Optional TLVs. You can configure the inclusion of individual Optional TLVs. Table 14-1. Type Type, Length, Value (TLV) Types TLV Description 0 End of LLDPDU Marks the end of an LLDPDU. 1 Chassis ID The Chassis ID TLV is a mandatory TLV that identifies the chassis containing the IEEE 802 LAN station associated with the transmitting LLDP agent.
Organizationally Specific TLVs Organizationally specific TLVs can be defined by a professional organization or a vendor. They have two mandatory fields (Figure 14-3) in addition to the basic TLV fields (Figure 14-1): • Organizationally Unique Identifier (OUI)—a unique number assigned by the IEEE to an organization or vendor. OUI Sub-type—These sub-types indicate the kind of information in the following data field. The sub-types are determined by the owner of the OUI. • Figure 14-3.
www.dell.com | support.dell.com Table 14-2. Optional TLV Types Type TLV Description IEEE 802.3 Organizationally Specific TLVs 127 MAC/PHY Configuration/Status Indicates the capability and current setting of the duplex status and bit rate, and whether the current settings are the result of auto-negotiation. This TLV is not available in the FTOS implementation of LLDP, but is available and mandatory (non-configurable) in the LLDP-MED implementation.
TIA Organizationally Specific TLVs The Dell Force10 system is an LLDP-MED Network Connectivity Device (Device Type 4). Network connectivity devices are responsible for: • • transmitting an LLDP-MED capabilities TLV to endpoint devices storing the information that endpoint devices advertise Table 14-3 list the five types of TIA-1057 Organizationally Specific TLVs. Table 14-3.
www.dell.com | support.dell.com LLDP-MED Capabilities TLV The LLDP-MED Capabilities TLV communicates the types of TLVs that the endpoint device and the network connectivity device support. LLDP-MED network connectivity devices must transmit the Network Policies TLV. • • The value of the LLDP-MED Capabilities field in the TLV is a 2 octet bitmap (Figure 14-4), each bit represents an LLDP-MED capability (Table 14-4). The possible values of the LLDP-MED Device Type is listed in Table 14-5.
LLDP-MED Network Policies TLV A network policy in the context of LLDP-MED is a device’s virtual local area network (VLAN) configuration and associated Layer 2 and Layer 3 configurations, specifically: • • • • VLAN ID VLAN tagged or untagged status Layer 2 priority DSCP value The application type is represented by an integer (the Type integer in Table 14-6), which indicates a device function for which a unique network policy is defined.
www.dell.com | support.dell.com Figure 14-5.
LLDP Operation On an Aggregator, LLDP operates as follows: • • • • • • LLDP is enabled by default. LLDPDUs are transmitted and received by default. LLDPDUs are transmitted periodically. The default interval is 30 seconds. LLDPDU information received from a neighbor expires after the default Time to Live (TTL) value: 120 seconds. FTOS supports up to eight neighbors per interface. FTOS supports a maximum of 8000 total neighbors per system.
www.dell.com | support.dell.com Viewing Information Advertised by Adjacent LLDP Agents 176 | To display brief information about adjacent devices, use the show lldp neighbors command (Figure 14-9). To display all of the information that neighbors are advertising, use the show lldp neighbors detail command (Figure 14-10). Figure 14-9.
Figure 14-10.
www.dell.com | support.dell.com Clearing LLDP Counters 178 | You can clear LLDP statistics that are maintained on an Aggregator for LLDP counters for frames transmitted to and received from neighboring devices on all or a specified physical interface. To clear LLDP counters, enter the clear lldp counters command.
Debugging LLDP The debug lldp command allows you to view the TLVs that your system is sending and receiving. • • Use the debug lldp brief command to view a readable version of the TLVs. Use the debug lldp detail command to view a readable version of the TLVs plus a hexadecimal version of the entire LLDPDU. Figure 14-11.
www.dell.com | support.dell.com Relevant Management Objects FTOS supports all IEEE 802.1AB MIB objects. • • • • Table 14-7. Table 14-7 lists the objects associated with received and transmitted TLVs. Table 14-8 lists the objects associated with the LLDP configuration on the local agent. Table 14-9 lists the objects associated with IEEE 802.1AB Organizationally Specific TLVs. Table 14-10 lists the objects associated with received and transmitted LLDP-MED TLVs.
Table 14-8.
www.dell.com | support.dell.com Table 14-9. LLDP 802.1 Organizationally Specific TLV MIB Objects TLV Type TLV Name TLV Variable System LLDP MIB Object 127 Port-VLAN ID PVID Local lldpXdot1LocPortVlanId Remote lldpXdot1RemPortVlanId 127 Port and Protocol VLAN ID port and protocol VLAN supported Local port and protocol VLAN enabled PPVID 127 VLAN Name VID VLAN name length VLAN name Table 14-10.
Table 14-10.
www.dell.com | support.dell.com Table 14-10.
15 Port Monitoring The Aggregator supports user-configured port monitoring. See Configuring Port Monitoring for the configuration commands to use. Port monitoring copies all incoming or outgoing packets on one port and forwards (mirrors) them to another port. The source port is the monitored port (MD) and the destination port is the monitoring port (MG).
www.dell.com | support.dell.com Port Monitoring The Aggregator supports multiple source-destination statements in a monitor session, but there may only be one destination port in a monitoring session (Message 2). Message 2 One Destination Port in a Monitoring Session Error Message % Error: Only one MG port is allowed in a session. The number of source ports supported in a port pipe is equal to the number of physical ports in the port pipe.
Figure 15-2 shows ports 0/25 and 0/26 that belong to port pipe 1 with a maximum of four destination ports. Figure 15-2.
www.dell.com | support.dell.com Configuring Port Monitoring To configure port monitoring, use the following example: Step Task Command Syntax Command Mode 1 Verify that the intended monitoring port has no configuration other than no shutdown (Figure 15-3). show interface EXEC Privilege 2 Create a monitoring session using the command monitor session from CONFIGURATION mode (Figure 15-3).
In Figure 15-4, the host and server are exchanging traffic which passes through the uplink interface 1/1. Port 1/1 is the monitored port and port 1/42 is the destination port, which is configured to only mirror traffic received on tengigabitethernet 1/1 (host-originated traffic). Figure 15-4.
190 | Port Monitoring www.dell.com | support.dell.
16 Simple Network Management Protocol (SNMP) Network management stations use the Simple Network Management Protocol (SNMP) to retrieve or alter management data from network elements. A datum of management information is called a managed object; the value of a managed object can be static or variable. Network elements store managed objects in a database called a Management Information Base (MIB).
www.dell.com | support.dell.com Setting up SNMP FTOS supports SNMP version 1 and version 2 which are community-based security models. The primary difference between the two versions is that version 2 supports two additional protocol operations (informs operation and snmpgetbulk query) and one additional object (counter64 object). Creating a Community For SNMPv1 and SNMPv2, you must create a community to enable the community-based security in FTOS.
Dell Force10 supports RFC 4001, Textual Conventions for Internet Work Addresses that defines values representing a type of internet address. These values display for ipAddressTable objects using the snmpwalk command. In the following figure, the value “4” displays in the OID before the IP address for IPv4. >snmpwalk -v 2c -c public 10.11.195.63 1.3.6.1.2.1.4.34 IP-MIB::ip.34.1.3.1.4.1.1.1.1 = INTEGER: 1107787778 IP-MIB::ip.34.1.3.1.4.2.1.1.1 = INTEGER: 1107787779 IP-MIB::ip.34.1.3.2.16.254.128.0.0.0.0.0.
www.dell.com | support.dell.com Displaying the Ports in a VLAN Using SNMP FTOS identifies VLAN interfaces using an interface index number that is displayed in the output of the command show interface vlan, as shown in Figure 16-5. Figure 16-5.
Figure 16-6 shows the output for an Aggregator. All hex pairs are 00, indicating that no ports are assigned to VLAN 10. In Figure 16-7, Port 0/2 is added to VLAN 10 as untagged. And the first hex pair changes from 00 to 04. Figure 16-7.
www.dell.com | support.dell.com Table 16-1. MIB Objects for Fetching Dynamic MAC Entries in the Forwarding Database MIB Object OID dot1dTpFdbTable .1.3.6.1.2.1.17.4.3 Description MIB List the learned unicast MAC addresses on the default VLAN. Q-BRIDGE MIB dot1qTpFdbTable .1.3.6.1.2.1.17.7.1.2. List the learned unicast MAC addresses on 2 non-default VLANs. dot3aCurAggFdb Table .1.3.6.1.4.1.6027.3.2. List the learned MAC addresses of aggregated 1.1.5 links (LAG).
Use dot3aCurAggFdbTable to fetch the learned MAC address of a port-channel. The instance number is the decimal conversion of the MAC address concatenated with the port-channel number. Figure 16-10.
www.dell.com | support.dell.com Starting from the least significant bit (LSB): • • • • • • the first 14 bits represent the card type the next 4 bits represent the interface type the next 7 bits represent the port number the next 5 bits represent the slot number the next 1 bit is 0 for a physical interface and 1 for a logical interface the next 1 bit is unused For example, the index 44634369 is 10101010010001000100000001 in binary.
Monitor Port-channels To check the status of a Layer 2 port-channel, use f10LinkAggMib (.1.3.6.1.4.1.6027.3.2). Below, Po 1 is a switchport and Po 2 is in Layer 3 mode. [senthilnathan@lithium ~]$ snmpwalk -v 2c -c public 10.11.1.1 .1.3.6.1.4.1.6027.3.2.1.1 SNMPv2-SMI::enterprises.6027.3.2.1.1.1.1.1.1 = INTEGER: 1 SNMPv2-SMI::enterprises.6027.3.2.1.1.1.1.1.2 = INTEGER: 2 SNMPv2-SMI::enterprises.6027.3.2.1.1.1.1.2.1 = Hex-STRING: 00 01 E8 13 A5 C7 SNMPv2-SMI::enterprises.6027.3.2.1.1.1.1.2.
www.dell.com | support.dell.com Entity MIBS The Entity MIB provides a mechanism for presenting hierarchies of physical entities using SNMP tables. The Entity MIB contains the following groups, which describe the physical elements and logical elements of a managed system The following tables are implemented for the Aggregator. • Physical Entity: A physical entity or physical component represents an identifiable physical resource within a managed system.
The status of the MIBS is as follows: $ snmpwalk -c public -v 2c 10.16.130.148 1.3.6.1.2.1.47.1.1.1.1.2 SNMPv2-SMI::mib-2.47.1.1.1.1.2.1 = "" SNMPv2-SMI::mib-2.47.1.1.1.1.2.2 = STRING: "PowerConnect I/O-Aggregator" SNMPv2-SMI::mib-2.47.1.1.1.1.2.3 = STRING: "Module 0" SNMPv2-SMI::mib-2.47.1.1.1.1.2.4 = STRING: "Unit: 0 Port 1 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.5 = STRING: "Unit: 0 Port 2 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.6 = STRING: "Unit: 0 Port 3 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.
www.dell.com | support.dell.com SNMPv2-SMI::mib-2.47.1.1.1.1.2.81 = STRING: "Unit: 1 Port 14 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.82 = STRING: "Unit: 1 Port 15 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.83 = STRING: "Unit: 1 Port 16 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.84 = STRING: "Unit: 1 Port 17 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.85 = STRING: "Unit: 1 Port 18 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.86 = STRING: "Unit: 1 Port 19 10G Level" SNMPv2-SMI::mib-2.47.1.1.1.1.2.
17 Stacking Overview An Aggregator auto-configures to operate in standalone mode. To use an Aggregator in a stack, you must manually configure it using the CLI to operate in stacking mode. Stacking is supported only on the 40GbE ports on the base module. Stacking is limited to two Aggregators in the same chassis in a single stack. Up to three stacks of Aggregators are supported in an M1000e chassis. To configure a stack, you must use the CLI.
www.dell.com | support.dell.com Figure 17-15. A Two-Aggregator Stack Stack Management Roles The stack elects the management units for the stack management: • • Stack master: primary management unit Standby: secondary management unit The master holds the control plane and the other units maintain a local copy of the forwarding databases.
Stack Master Election The stack elects a master and standby unit at bootup time based on MAC address. The unit with the higher MAC value becomes master. To view which switch is the stack master, use the show system command. Figure 17-16 shows sample output from an established stack. A change in the stack master occurs when: • • • You power down the stack master or bring the master switch offline. A failover of the master switch occurs. You disconnect the master switch from the stack.
www.dell.com | support.dell.com MAC Addressing All port interfaces in the stack use the MAC address of the management interface on the master switch. The MAC address of the chassis in which the master Aggregator is installed is used as the stack MAC address. The stack continues to use the master’s chassis MAC address even after a failover. The MAC address is not refreshed until the stack is reloaded and a different unit becomes the stack master.
Stacking Port Numbers By default, each Aggregator in Standalone mode is numbered stack-unit 0. Stack-unit numbers are assigned to member switches when the stack comes up. Figure 17-17 shows the numbers of the 40GbE stacking ports on an Aggregator. Figure 17-17. Stack Groups on an Aggregator Stack Unit 0 / Port 37 Stack Unit 0 / Port 33 Configuring a Switch Stack To configure and bring up a switch stack, follow these steps: 1.
www.dell.com | support.dell.com Stacking Prerequisites Before you cable and configure a stack of Aggregators, review the following prerequisites: • • • • • All Aggregators in the stack must be powered up with the initial or startup configuration before you attach the cables. All stacked Aggregators must run the same FTOS version. The minimum FTOS version required is 8.3.17.0. To check the FTOS version that a switch is running, use the show version command.
Cabling Procedure The following cabling procedure uses the stacking topology in Figure 17-15. To connect the cabling: 1. Connect a 40GbE base port on the first Aggregator to a 40GbE base port on another Aggregator in the same chassis. 2. Connect a 40GbE base port on the second Aggregator to a 40GbE port on the first Aggregator. The resulting ring topology allows the entire stack to function as a single switch with resilient fail-over capabilities.
www.dell.com | support.dell.com Note: If the stacked switches all reboot at approximately the same time, the switch with the highest MAC address is automatically elected as the master switch. The switch with the next highest MAC address is elected as standby. As each switch joins the stack, it is assigned the lowest available stack-unit number from 0 to 5. The default configuration of each stacked switch is stored in the running configuration of the stack.
Resetting a Unit on a Stack Use the following reset commands to reload any of the member units or the standby in a stack. If you try to reset the stack master, an error message is displayed: Reset of master unit is not allowed. Task Command Syntax Command Mode Reload a stack unit from the master switch reset stack-unit unit-number EXEC Privilege Reset a stack-unit when the unit is in a problem state.
www.dell.com | support.dell.com Verifying a Stack Configuration Using LEDs Table 17-2 lists the status of a stacked switch according to the color of the System Status light emitting diodes (LEDs) on its front panel. Table 17-2. System Status LED on a Stacked Switch Color Meaning Blue The switch is operating as the stack master or as a standalone unit. Off The switch is a member or standby unit. Amber The switch is booting or a failure condition has occurred.
Figure 17-19.
www.dell.com | support.dell.com Figure 17-21. show system stack-unit stack-group configured Command Example FTOS# show system stack-unit 1 stack-group configured Configured stack groups in stack-unit 1 -----------------------------------------0 1 Figure 17-22. show system stack-unit stack-group Command Example FTOS# show system stack-unit 1 stack-group Stack group Ports -----------------------------------0 1/33 1 1/37 4 1/49 5 1/53 Figure 17-23.
Troubleshooting a Switch Stack Troubleshooting Commands To perform troubleshooting operations on a switch stack, use the commands in Table 17-4 on the master switch. Table 17-4. Troubleshooting Stack Commands Command Output show system stack-ports (Figure 17-25) Displays the status of stacked ports on stack units.
www.dell.com | support.dell.com Figure 17-26.
Figure 17-27.
www.dell.com | support.dell.com Master Switch Fails Problem: The master switch fails due to a hardware fault, software crash, or power loss. Resolution: A failover procedure begins: 1. Keep-alive messages from the Aggregator master switch time out after 60 seconds and the switch is removed from the stack. 2. The standby switch takes the master role. Data traffic on the new master switch is uninterrupted. Protocol traffic is managed by the control plane. 3. A member switch is elected as the new standby.
Stack Unit in Card-Problem State Due to Incorrect FTOS Version Problem: A stack unit enters a Card-Problem state because the switch has a different FTOS version than the master unit (Figure 17-29). The switch does not come online as a stack unit. Resolution: To restore a stack unit with an incorrect FTOS version as a member unit, disconnect the stacking cables on the switch and install the correct FTOS version. Then add the switch to the stack as described in Adding a Stack Unit.
www.dell.com | support.dell.com Upgrading a Switch Stack To upgrade all switches in a stack with the same FTOS version, follow these steps: Step Task Command Syntax Command Mode 1 Copy the new FTOS image to a network server. 2 Download the FTOS image by accessing an interactive CLI that requests the server IP address and image filename, and prompts you to upgrade all member stack units. Specify the system partition on the master switch into which you want to copy the FTOS image.
Upgrading a Single Stack Unit Upgrading a single stacked switch is necessary when the unit was disabled due to an incorrect FTOS version. This procedure upgrades the image in the boot partition of the member unit from the corresponding partition in the master unit.
222 | Stacking www.dell.com | support.dell.
18 Broadcast Storm Control On the Aggregator, the broadcast storm control feature is enabled by default on all ports, and disabled on a port when an iSCSI storage device is detected. Broadcast storm control is re-enabled as soon as the connection with an iSCSI device ends. Broadcast traffic on Layer 2 and Layer 3 interfaces is limited or suppressed during a broadcast storm. You can view the status of a broadcast-storm control operation by using the show io-aggregator broadcast storm-control status command.
224 | Broadcast Storm Control www.dell.com | support.dell.
19 System Time and Date The Aggregator auto-configures the hardware and software clocks with the current time and date. If necessary, you can manually set and maintain the system time and date using the CLI commands described in this chapter.
www.dell.com | support.dell.com Setting the Time for the Software Clock You can change the order of the month and day parameters to enter the time and date as time day month year. You cannot delete the software clock. The software clock runs only when the software is up. The clock restarts, based on the hardware clock, when the switch reboots.
Setting the Time Zone Universal time coordinated (UTC) is the time standard based on the International Atomic Time standard, commonly known as Greenwich Mean time. When determining system time, you must include the differentiator between the UTC and your local timezone. For example, San Jose, CA is the Pacific Timezone with a UTC offset of -8.
www.dell.com | support.dell.com Setting Daylight Savings Time FTOS supports setting the system to daylight savings time once or on a recurring basis every year. Setting Daylight Saving Time Once Set a date (and time zone) on which to convert the switch to daylight savings time on a one-time basis.
Setting Recurring Daylight Saving Time Set a date (and time zone) on which to convert the switch to daylight savings time on a specific day every year. If you have already set daylight savings for a one-time setting, you can set that date and time as the recurring setting using the clock summer-time time-zone recurring command.
www.dell.com | support.dell.com Command Syntax Command Mode Purpose • • • • • • • • • end-week: If you entered a start-week, enter one of the following as the week that daylight savings ends: week-number: enter a number from 1 to 4 as the number of the week to end daylight savings time. first: enter the keyword first to end daylight savings time in the first week of the month. last: enter the keyword last to end daylight savings time in the last week of the month.
20 Uplink Failure Detection (UFD) Uplink failure detection (UFD) is enabled by default on the Aggregator. Note: Uplink failure detection is also referred to as link tracking. UFD provides detection of the loss of upstream connectivity and, if used with NIC teaming, automatic recovery from a failed link. An Aggregator auto-configures with UFD enabled. You can use CLI commands to display the status and configuration of uplink-state groups, and enable UFD debugging.
www.dell.com | support.dell.com Figure 20-1. Uplink Failure Detection How Uplink Failure Detection Works UFD creates an association between upstream and downstream interfaces. The association of uplink and downlink interfaces is called an uplink-state group. An interface in an uplink-state group can be a physical interface or a port-channel (LAG) aggregation of physical interfaces. An enabled uplink-state group tracks the state of all assigned upstream interfaces.
Figure 20-2. Uplink Failure Detection Example If only one of the upstream interfaces in an uplink-state group goes down, a specified number of downstream ports associated with the upstream interface are put into a link-down state. This number is user-configurable and is calculated by the ratio of upstream port bandwidth to downstream port bandwidth in the same uplink-state group.
www.dell.com | support.dell.com Important Points to Remember UFD operates as follows on an Aggregator: • When the uplink LAG comes up again, server interfaces are brought up (Oper UP) and the error-disabled state is cleared. • • 234 Uplink-state group 1 is created by default with the uplink LAG 128 as the upstream interface and all server ports as downstream interfaces. When the uplink LAG goes down, all server interfaces are brought down and placed in UFD error-disabled state.
Disabling Uplink Failure Detection In some network topologies, you may prefer that downstream links on an Aggregator remain operationally up and continue transmitting traffic without being automatically disabled when connectivity to an upstream switch/router goes down. When UFD is enabled and the upstream connectivity on LAG 128 fails, downstream links are also disabled.
www.dell.com | support.dell.com 236 Displaying Uplink Failure Detection To display information on the Uplink Failure Detection feature, enter any of the following show commands: | Show Command Syntax Description show uplink-state-group [group-id] [detail] Command Mode: EXEC Displays status information on a specified uplink-state group or all groups. Valid group-id values are 1 to 16.
Figure 20-3.
www.dell.com | support.dell.com Figure 20-4.
Debugging UFD on an Interface To enable debug messages for events related to a specified uplink-state group or all groups, enter the debug uplink-state-group command. Task Command Command Mode Enable UFD debugging on all or a specified uplink-state group, where: group-id specifies an uplink-state group. Valid group-id values are 1 to 16. debug uplink-state-group [group-id] EXEC PRIVILEGE To turn off debugging event messages, enter the no debug uplink-state-group [group-id] command.
www.dell.com | support.dell.
21 Upgrade Procedures To view the requirements for upgrading the Dell Force10 operating software (FTOS) on an Aggregator, refer to the FTOS Release Notes for the M I/O Aggregator. Follow the procedures in the FTOS Release Notes for the software version you wish to upgrade to. Direct any questions or concerns about FTOS upgrades to the Dell Force10 Technical Support Center. You can reach technical support: • • • On the Web: www.force10networks.com/support/ By email: support@force10networks.
242 | Upgrade Procedures www.dell.com | support.dell.
22 Debugging and Diagnostics The chapter contains the following sections: • • • • • • • • • • • Debugging Aggregator Operation Software show Commands Offline Diagnostics Trace Logs Show Hardware Commands Environmental Monitoring Buffer Tuning Troubleshooting Packet Loss Application Core Dumps Mini Core Dumps TCP Dumps Debugging and Diagnostics | 243
www.dell.com | support.dell.com Debugging Aggregator Operation This section describes common troubleshooting procedures to use for error conditions that may arise during Aggregator operation. All interfaces on the Aggregator are operationally down Symptom: All Aggregator interfaces are down. Resolution: Ensure that port channel 128 is up and that the Aggregator-facing port channel on the top-of-rack switch is correctly configured. Steps to Take: 1.
Broadcast, unknown multicast, and DLF packets are switched at a very low rate Symptom: Broadcast, unknown multicast, and DLF packets are switched at a very low rate. By default, broadcast storm control is enabled on an Aggregator and rate limits the transmission of broadcast, unknown multicast, and DLF packets to 1Gbps. This default behavior is designed to avoid unnecessarily flooding these packets on all (4094) VLANs on all Aggregator interfaces (default configuration).
www.dell.com | support.dell.com Flooded packets on all VLANs are received on a server Symptom: All packets flooded on all VLANs on an Aggregator are received on a server, even if the server is configured as a member of only a subset of VLANs. This behavior happens because all Aggregator ports are, by default, members of all (4094) VLANs. Resolution: Configure a port that is connected to the server with restricted VLAN membership. Steps to Take: 1.
Auto-configured VLANs do not exist on a stacked Aggregator Symptom: When an Aggregator is configured and used in a stack, traffic does not flow and the VLAN auto-configuration on all ports is lost. This behavior happens because an Aggregator in stacking mode does not support auto-configured VLANs. Only VLANs that were previously manually configured are retained on the master stack unit. Resolution: You must manually configure VLAN membership on each stack-unit port. Steps to Take: 1.
www.dell.com | support.dell.com Software show Commands Use the show version and show system stack-unit 0 commands as a part of troubleshooting an Aggregator’s software configuration in a standalone or stacking scenario. Table 22-1. Software show Commands Command Description show version Display the current version of FTOS software running on an Aggregator. show system stack-unit 0 Display software configuration on an Aggregator in stacking mode. Figure 22-1.
Figure 22-2. show system stack-unit 0 Command Example FTOS#show system stack-unit 0 -- Unit 0 -Unit Type : Management Unit Status : online Next Boot : online Required Type : I/O-Aggregator - 34-port GE/TE (XL) Current Type : I/O-Aggregator - 34-port GE/TE (XL) Master priority : 0 Hardware Rev : Num Ports : 56 Up Time : 17 hr, 8 min FTOS Version : 8-3-17-15 Jumbo Capable : yes POE Capable : no Boot Flash : A: 4.0.1.0 [booted] B: 4.0.1.0bt Boot Selector : 4.0.0.
www.dell.com | support.dell.com Offline Diagnostics The offline diagnostics test suite is useful for isolating faults and debugging hardware.The diagnostics tests are grouped into three levels: • • • Level 0—Level 0 diagnostics check for the presence of various components and perform essential path verifications. In addition, they verify the identification registers of the components on the board. Level 1—A smaller set of diagnostic tests.
Figure 22-3. Taking a Stack Unit Offline FTOS#offline stack-unit 2 Warning - Diagnostic execution will cause stack-unit to reboot after completion of diags. Proceed with Offline-Diags [confirm yes/no]:y 5w6d12h: %STKUNIT0-M:CP %CHMGR-2-STACKUNIT_DOWN: Stack unit 2 down - stack unit offline 5w6d12h: %STKUNIT0-M:CP %IFMGR-1-DEL_PORT: Removed port: Tengig 2/1-48 FTOS#5w6d12h: %STKUNIT1-S:CP %IFMGR-1-DEL_PORT: Removed port: Tengig 2/1-48 2.
www.dell.com | support.dell.com Figure 22-5. Command Example FTOS#dir flash://TRACE_LOG_DIR Directory of flash:/TRACE_LOG_DIR 1 2 3 drwx drwx -rwx 4096 4096 100583 Jan 17 2011 15:02:16 +00:00 . Jan 01 1980 00:00:00 +00:00 .. Feb 11 2011 20:41:36 +00:00 failure_trace0_RPM0_CP flash: 2143281152 bytes total (2069291008 bytes free) Show Hardware Commands The show hardware command tree consists of EXEC Privilege commands used with the Aggregator.
Table 22-2. show hardware Commands Command Description show hardware stack-unit {0-5} stack-port {33-56} View the input and output statistics for a stack-port interface. show hardware stack-unit {0-5 unit {0-0} counters View the counters in the field processors of the stack unit. show hardware stack-unit {0-5} unit {0-0} details View the details of the FP devices and Hi gig ports on the stack-unit.
www.dell.com | support.dell.com Figure 22-6. show interfaces transceiver Command Example FTOS#show int ten 0/49 transceiver SFP is present SFP 49 Serial Base ID fields SFP 49 Id = 0x03 SFP 49 Ext Id = 0x04 SFP 49 Connector = 0x07 SFP 49 Transceiver Code = 0x00 0x00 0x00 0x01 0x20 0x40 0x0c 0x01 SFP 49 Encoding = 0x01 SFP 49 BR Nominal = 0x0c SFP 49 Length(9um) Km = 0x00 SFP 49 Length(9um) 100m = 0x00 SFP 49 Length(50um) 10m = 0x37 SFP 49 Length(62.
Recognize an Over-Temperature Condition An over-temperature condition occurs for one of two reasons: • • The card genuinely is too hot. A sensor has malfunctioned. Inspect cards adjacent to the one reporting condition to discover the cause. • • If directly adjacent cards are not a normal temperature, suspect a genuine overheating condition. If directly adjacent cards are a normal temperature, suspect a faulty sensor. When the system detects a genuine over-temperature condition, it powers off the card.
www.dell.com | support.dell.com Figure 22-8.
The simple network management protocol (SNMP) traps and OIDs in Table 22-3 provide information about environmental monitoring hardware and hardware components. Table 22-3. SNMP Traps and OIDs OID String OID Name Description chSysPortXfpRecvPower OID to display the receiving power of the connected optics. chSysPortXfpTxPower OID to display the transmitting power of the connected optics. chSysPortXfpRecvTemp OID to display the Temperature of the connected optics. Receiving power .1.3.6.1.4.1.6027.
www.dell.com | support.dell.com All ports support eight queues, four for data traffic and four for control traffic. All eight queues are tunable. Physical memory is organized into cells of 128 bytes. The cells are organized into two buffer pools—a dedicated buffer and a dynamic buffer. • • Dedicated buffer is reserved memory that cannot be used by other interfaces on the same ASIC or by other queues on the same interface.
Deciding to Tune Buffers Dell Force10 recommends exercising caution when configuring any non-default buffer settings, as tuning can significantly affect system performance. The default values work for most cases. As a guideline, consider tuning buffers if traffic is very bursty (and coming from several interfaces). In this case: • • • Reduce the dedicated buffer on all queues/interfaces. Increase the dynamic buffer on all interfaces.
www.dell.com | support.dell.com You cannot allocate more than the available memory for the dedicated buffers. If the system determines that the sum of the configured dedicated buffers allocated to the queues is more than the total available memory, the configuration is rejected, returning a syslog message similar to the following. Table 22-4.
Figure 22-11. Displaying Buffer Profile Allocations FTOS#show running-config interface tengigabitethernet 2/0 ! interface TenGigabitEthernet 2/0 no ip address mtu 9252 switchport no shutdown buffer-policy myfsbufferprofile FTOS#show buffer-profile detail int tengig 0/10 Interface Tengig 0/10 Buffer-profile fsqueue-fp Dynamic buffer 1256.00 (Kilobytes) Queue# Dedicated Buffer Buffer Packets (Kilobytes) 0 3.00 256 1 3.00 256 2 3.00 256 3 3.00 256 4 3.00 256 5 3.00 256 6 3.00 256 7 3.
www.dell.com | support.dell.com Using a Pre-Defined Buffer Profile FTOS provides two pre-defined buffer profiles, one for single-queue (for example, non-QoS) applications, and one for four-queue (for example, QoS) applications. Task Command Mode Apply one of two pre-defined buffer profiles for all port pipes in the system. buffer-profile global [1Q|4Q] CONFIGURATION You must reload the system for the global buffer profile to take effect (Message 3).
Sample Buffer Profile Configuration The two general types of network environments are sustained data transfers and voice/data. Dell Force10 recommends a single-queue approach for data transfers (Figure 22-12). Figure 22-12.
www.dell.com | support.dell.com Displaying Drop Counters The show hardware stack-unit 0–11 drops [unit 0 [port 0–63]] command assists in identifying which stack unit, port pipe, and port is experiencing internal drops (Figure 22-13) and (Figure 22-14). Figure 22-13.
Figure 22-14.
www.dell.com | support.dell.com Figure 22-15.
Displaying Stack Port Statistics The show hardware stack-unit stack-port command displays input and output statistics for a stack-port interface (Figure 22-17). Figure 22-17.
www.dell.com | support.dell.com Application Core Dumps Application core dumps are disabled by default. A core dump file can be very large. Due to memory requirements, the file can only be sent directly to an FTP server. It is not stored on the local flash. To enable full application core dumps, use the following command: Task Command Syntax Command Mode Enable RPM core dumps and specify the shutdown mode.
Figure 22-19.
www.dell.com | support.dell.com TCP Dumps TCP dump captures CPU bound control plane traffic to improve troubleshooting and system manageability. When enabled, a TCP dump captures all the packets on the local CPU, as specified in the CLI. You can save the traffic capture files to flash, FTP, SCP, or TFTP. The files saved on the flash are located in the flash://TCP_DUMP_DIR/Tcpdump_/ directory, and labeled tcpdump_*.pcap. There can be up to 20 Tcpdump_ directories.
23 Standards Compliance This chapter contains the following sections: • • • IEEE Compliance RFC and I-D Compliance MIB Location Note: Unless noted, when a standard cited here is listed as supported by Dell Force10 operating software (FTOS), FTOS also supports predecessor standards. One way to search for predecessor standards is to use the http://tools.ietf.org/ website.
www.dell.com | support.dell.com 272 RFC and I-D Compliance The following standards are supported by FTOS on an Aggregator and are grouped by related protocol. The columns showing support by platform indicate which version of FTOS first supports the standard.
General IPv4 Protocols RFC# Full Name 791 Internet Protocol 792 Internet Control Message Protocol 826 An Ethernet Address Resolution Protocol 1027 Using ARP to Implement Transparent Subnet Gateways 1042 A Standard for the Transmission of IP Datagrams over IEEE 802 Networks 1519 Classless Inter-Domain Routing (CIDR): an Address Assignment and Aggregation Strategy 1812 Requirements for IP Version 4 Routers 2131 Dynamic Host Configuration Protocol 3021 Using 31-Bit Prefixes on IPv4 Point-to
www.dell.com | support.dell.
Network Management (continued) RFC# Full Name IEEE 802.1AB The LLDP Management Information Base extension module for IEEE 802.3 organizationally defined discovery information. (LLDP DOT1 MIB and LLDP DOT3 MIB) ruzin-mstp-mib-02 (Traps) Definitions of Managed Objects for Bridges with Multiple Spanning Tree Protocol sFlow.org sFlow Version 5 sFlow.
www.dell.com | support.dell.com MIB Location 276 | Force10 MIBs are under the Force10 MIBs subhead on the Documentation page of iSupport: https://www.force10networks.com/csportal20/KnowledgeBase/Documentation.aspx You also can obtain a list of selected MIBs and their OIDs at the following URL: https://www.force10networks.com/csportal20/MIBs/MIB_OIDs.aspx Some pages of iSupport require a login. To request an iSupport account, go to: https://www.force10networks.com/CSPortal20/Support/AccountRequest.
