53-1002920-02 9 September 2013 Fabric OS Administrator’s Guide Supporting Fabric OS 7.2.
Copyright © 2013 Brocade Communications Systems, Inc. All Rights Reserved. ADX, AnyIO, Brocade, Brocade Assurance, the B-wing symbol, DCX, Fabric OS, ICX, MLX, MyBrocade, OpenScript, VCS, VDX, and Vyatta are registered trademarks, and HyperEdge, The Effortless Network, and The On-Demand Data Center are trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries. Other brands, products, or service names mentioned may be trademarks of their respective owners.
Contents (High Level) Section I Standard Features Chapter 1 Understanding Fibre Channel Services . . . . . . . . . . . . . . . . . . . . . . . . . 45 Chapter 2 Performing Basic Configuration Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Chapter 3 Performing Advanced Configuration Tasks . . . . . . . . . . . . . . . . . . . . . . 83 Chapter 4 Routing Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 25 Managing Long-Distance Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . .587 Chapter 26 Using FC-FC Routing to Connect Fabrics . . . . . . . . . . . . . . . . . . . . . . .593 Appendix A Port Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .641 Appendix B FIPS Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .645 Appendix C Hexadecimal Conversion . . . . . . . . . . . . . . . . . .
Contents About This Document Supported hardware and software . . . . . . . . . . . . . . . . . . . . . . . . . . 35 What’s new in this document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Document conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Notice to the reader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Additional information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 2 Performing Basic Configuration Tasks Fabric OS overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Fabric OS command line interface. . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Console sessions using the serial port. . . . . . . . . . . . . . . . . . . . 58 Telnet or SSH sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Getting help on a command . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 3 Performing Advanced Configuration Tasks Port identifiers (PIDs) and PID binding overview . . . . . . . . . . . . . . . 83 Core PID addressing mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 Fixed addressing mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 10-bit addressing mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 256-area addressing mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 WWN-based PID assignment . . .
Duplicate PWWN handling during device login . . . . . . . . . . . . . . . .110 Setting 0, First login precedence . . . . . . . . . . . . . . . . . . . . . . .110 Setting 1, Second login precedence. . . . . . . . . . . . . . . . . . . . .110 Setting 2, Mixed precedence . . . . . . . . . . . . . . . . . . . . . . . . . .110 Setting the behavior for handling duplicate PWWNs. . . . . . . .111 Enabling forward error correction . . . . . . . . . . . . . . . . . . . . . . . . . .111 FEC Limitations . . . . . . . .
Chapter 5 Buffer-to-Buffer Credits and Credit Recovery Buffer credit management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .135 Buffer-to-buffer flow control . . . . . . . . . . . . . . . . . . . . . . . . . . .135 Optimal buffer credit allocation . . . . . . . . . . . . . . . . . . . . . . . .136 Fibre Channel gigabit values reference definition. . . . . . . . . .137 Buffer credit allocation based on full-size frames. . . . . . . . . .
The boot PROM password . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .163 Setting the boot PROM password for a switch with a recovery string . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .163 Setting the boot PROM password for a Backbone with a recovery string . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .164 Setting the boot PROM password for a switch without a recovery string . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Telnet protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .226 Blocking Telnet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .227 Unblocking Telnet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .228 Listener applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .228 Ports and applications used by switches . . . . . . . . . . . . . . . . . . . .229 Port configuration . . . . . . . . . . .
IP Filter policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .253 Creating an IP Filter policy. . . . . . . . . . . . . . . . . . . . . . . . . . . . .254 Cloning an IP Filter policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . .254 Displaying an IP Filter policy . . . . . . . . . . . . . . . . . . . . . . . . . . .254 Saving an IP Filter policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .255 Activating an IP Filter policy. . . . . . . . . . .
Chapter 10 Installing and Maintaining Firmware Firmware download process overview . . . . . . . . . . . . . . . . . . . . . . .289 Upgrading and downgrading firmware . . . . . . . . . . . . . . . . . . .291 Considerations for FICON CUP environments . . . . . . . . . . . . .291 HA sync state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .291 Preparing for a firmware download . . . . . . . . . . . . . . . . . . . . . . . . .292 Obtaining and decompressing firmware . . . . . . .
Limitations and restrictions of Virtual Fabrics . . . . . . . . . . . . . . . .322 Restrictions on XISLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .323 Restrictions on moving ports . . . . . . . . . . . . . . . . . . . . . . . . . .324 Enabling Virtual Fabrics mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . .324 Disabling Virtual Fabrics mode . . . . . . . . . . . . . . . . . . . . . . . . . . . .325 Configuring logical switches to use basic configuration values. . .
Zone creation and maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . .350 Displaying existing zones . . . . . . . . . . . . . . . . . . . . . . . . . . . . .350 Creating a zone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .350 Adding devices (members) to a zone . . . . . . . . . . . . . . . . . . . .351 Removing devices (members) from a zone . . . . . . . . . . . . . . .352 Replacing zone members . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Traffic Isolation Zoning over FC routers . . . . . . . . . . . . . . . . . . . . . .386 TI zones within an edge fabric . . . . . . . . . . . . . . . . . . . . . . . . .388 TI zones within a backbone fabric . . . . . . . . . . . . . . . . . . . . . .389 Limitations of TI zones over FC routers . . . . . . . . . . . . . . . . . .390 Fabric-Level Traffic Isolation in a backbone fabric . . . . . . . . . . . . .390 Fabric-Level TI zones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
QoS zone-based traffic prioritization . . . . . . . . . . . . . . . . . . . . . . . . 419 QoS zones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419 QoS on E_Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .421 QoS over FC routers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .421 Virtual Fabrics considerations for QoS zone-based traffic prioritization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 16 In-flight Encryption and Compression In-flight encryption and compression overview. . . . . . . . . . . . . . . .445 Supported ports for in-flight encryption and compression . . .446 In-flight encryption and compression restrictions . . . . . . . . . .446 How in-flight encryption and compression are enabled . . . . .448 Authentication and key generation for encryption and compression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Example test scenarios and output . . . . . . . . . . . . . . . . . . . . . . . . .469 Confirming SFP and link status with an HBA . . . . . . . . . . . . . .470 Starting and stopping D_Port testing . . . . . . . . . . . . . . . . . . . .470 Chapter 18 NPIV NPIV overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .473 Upgrade considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474 Fixed addressing mode . . . . . . . . . . . . . . . . . . . . . .
Admin Domain management for physical fabric administrators . .494 Setting the default zoning mode for Admin Domains . . . . . . .495 Creating an Admin Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . .495 User assignments to Admin Domains . . . . . . . . . . . . . . . . . . .496 Removing an Admin Domain from a user account . . . . . . . . .498 Activating an Admin Domain . . . . . . . . . . . . . . . . . . . . . . . . . . .498 Deactivating an Admin Domain . . . . . . . . . . . . . . . . . . . . .
Temporary licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .530 Restrictions on upgrading temporary slot-based licenses . . .531 Date change restriction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .531 Configupload and download considerations . . . . . . . . . . . . . .531 Expired licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .531 Universal temporary licenses . . . . . . . . . . . . . . . . . . . . . . . . . .
End-to-end performance monitoring . . . . . . . . . . . . . . . . . . . . . . . .553 Maximum number of EE monitors . . . . . . . . . . . . . . . . . . . . . .553 Supported port configurations for EE monitors . . . . . . . . . . . .554 Adding EE monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .554 Setting a mask for an EE monitor . . . . . . . . . . . . . . . . . . . . . . .555 Deleting EE monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Enabling trunking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574 Disabling trunking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574 Displaying trunking information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574 Trunk Area and Admin Domains. . . . . . . . . . . . . . . . . . . . . . . . . . . . 576 Example of Trunk Area assignment on port domain,index . . . 576 ISL trunking over long-distance fabrics . . . . . . . . . . . . . .
Backbone fabric IDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .605 Assigning backbone fabric IDs . . . . . . . . . . . . . . . . . . . . . . . . .606 FCIP tunnel configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .606 Inter-fabric link configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .607 Configuring an IFL for both edge and backbone connections 607 Configuring EX_Ports on an ICL . . . . . . . . . . . . . . . . . . . . . . . .
FIPS mode configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .647 LDAP in FIPS mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .648 LDAP certificates for FIPS mode . . . . . . . . . . . . . . . . . . . . . . . .650 Preparing a switch for FIPS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .651 Overview of steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .652 Enabling FIPS mode. . . . . . . . . . . . . . . . . . . . .
Fabric OS Administrator’s Guide 53-1002920-02
Figures Figure 1 Well-known addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Figure 2 Identifying the blades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Figure 3 Blade swap with Virtual Fabrics during the swap. . . . . . . . . . . . . . . . . . . . . . . . 101 Figure 4 Blade swap with Virtual Fabrics after the swap . . . . . . . . . . . . . . . . . . . . . . . . .
Figure 37 Dedicated path is the only shortest path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383 Figure 38 Dedicated path is not the shortest path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 Figure 39 Enhanced TI zones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 Figure 40 Illegal ETIZ configuration: two paths from one port to two devices on the same remote domain . . . . . . . . . . . . . . . . . . . .
Figure 78 Setting end-to-end monitors on a port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 554 Figure 79 Mask positions for end-to-end monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556 Figure 80 Fabric mode Top Talker monitors on FC router do not monitor any flows . . . . 564 Figure 81 Fabric mode Top Talker monitors on FC router monitor flows over the E_Port 564 Figure 82 Port group configuration for the Brocade 5100 . . . . . . . . . . . . . . . . . .
Fabric OS Administrator’s Guide 53-1002920-02
Tables Table 1 Daemons that are automatically restarted. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Table 2 Terminal port parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Table 3 Help topic contents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Table 4 fabricShow fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Table 35 Port information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 Table 36 Valid methods for specifying policy members . . . . . . . . . . . . . . . . . . . . . . . . . . 232 Table 37 FCS policy states. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 Table 38 FCS switch operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Table 74 Number of ports supported for in-flight encryption and compression at various port speeds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447 Table 75 Supported platforms for D_Port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460 Table 76 D_Port configuration mode and nature of test. . . . . . . . . . . . . . . . . . . . . . . . . . 462 Table 77 Limitation on number of D_Ports for simultaneous tests . . . . . . . . . . .
Fabric OS Administrator’s Guide 53-1002920-02
About This Document In this chapter • Supported hardware and software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • What’s new in this document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Document conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Notice to the reader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Additional information . . . . . . . . . . . . . . .
- Brocade 6520 switch Brocade 6547 embedded switch Brocade 7800 extension switch Brocade VA-40FC Brocade Encryption Switch • Brocade DCX Backbone family: - Brocade DCX - Brocade DCX-4S • Brocade DCX 8510 Backbone family: - Brocade DCX 8510-4 - Brocade DCX 8510-8 What’s new in this document Information that was modified: • Renamed and moved the section about the two Ethernet ports on the CP blade to “Management Ethernet port bonding” on page 65.
italic text Provides emphasis Identifies variables Identifies paths and Internet addresses Identifies document titles code text Identifies CLI output Identifies command syntax examples For readability, command names in the narrative portions of this guide are presented in mixed lettercase: for example, switchShow. In actual examples, command lettercase is often all lowercase. Otherwise, this manual specifically notes those cases in which a command is case sensitive.
CAUTION A Caution statement alerts you to situations that can be potentially hazardous to you or cause damage to hardware, firmware, software, or data. DANGER A Danger statement indicates conditions or situations that can be potentially lethal or extremely hazardous to you. Safety labels are also attached directly to products to warn of these conditions or situations. Key terms For definitions specific to Brocade and Fibre Channel, see the Brocade Glossary.
For practical discussions about SAN design, implementation, and maintenance, you can obtain Building SANs with Brocade Fabric Switches through: http://www.amazon.com For additional Brocade documentation, visit the Brocade SAN Info Center and click the Resource Library location: http://www.brocade.com Release notes are available on the My Brocade website and are also bundled with the Fabric OS firmware. Other industry resources For additional resource information, visit the Technical Committee T11 website.
Getting technical help Contact your switch support supplier for hardware, firmware, and software support, including product repairs and part ordering. To expedite your call, have the following information available: 1.
Document feedback Quality is our first concern at Brocade and we have made every effort to ensure the accuracy and completeness of this document. However, if you find an error or an omission, or you think that a topic needs further development, we want to hear from you. Forward your feedback to: documentation@brocade.com Provide the title and version number of the document and as much detail as possible about your comment, including the topic heading and page number and your suggestions for improvement.
Fabric OS Administrator’s Guide 53-1002920-02
Section Standard Features I This section describes standard Fabric OS features, and includes the following chapters: • Chapter 1, “Understanding Fibre Channel Services” • Chapter 2, “Performing Basic Configuration Tasks” • Chapter 3, “Performing Advanced Configuration Tasks” • Chapter 4, “Routing Traffic” • Chapter 5, “Buffer-to-Buffer Credits and Credit Recovery” • Chapter 6, “Managing User Accounts” • Chapter 7, “Configuring Protocols” • Chapter 8, “Configuring Security Policies” • Chapter 9, “Maintai
Fabric OS Administrator’s Guide 53-1002920-02
Chapter Understanding Fibre Channel Services 1 In this chapter • Fibre Channel services overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 • Management server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 • Platform services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 • Management server database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1 Management server Management server — The management server provides a single point for managing the fabric. This is the only service that users can configure. See “Management server” below for more details Alias server — The alias server keeps a group of nodes registered as one name to handle multicast groups. Broadcast server — The broadcast server is optional. When frames are transmitted to this address, they are broadcast to all operational N_ and NL_Ports.
Management server database 1 Platform services and Virtual Fabrics Each logical switch has a separate platform database. All platform registrations done to a logical switch are valid only in that particular logical switch’s Virtual Fabric. Activating the platform services on a switch activates the platform services on all logical switches in a Virtual Fabric. Similarly, deactivating the platform services deactivates the platform service on all logical switches in a Virtual Fabric.
1 Management server database If the list is empty (the default), the management server is accessible to all systems connected in-band to the fabric. For more access security, you can specify WWNs in the ACL so that access to the management server is restricted to only those WWNs listed. NOTE The management server is logical switch-capable. All management server features are supported within a logical switch.
Management server database 1 Example of adding a member to the management server ACL switch:admin> msconfigure 0 Done 1 Display the access list 2 Add member based on its Port/Node WWN 3 Delete member based on its Port/Node WWN select : (0..3) [1] 2 Port/Node WWN (in hex): [00:00:00:00:00:00:00:00] 20:00:00:20:37:65:ce:aa *WWN is successfully added to the MS ACL. 0 Done 1 Display the access list 2 Add member based on its Port/Node WWN 3 Delete member based on its Port/Node WWN select : (0..
1 Management server database 5. At the “select” prompt, enter 1 to display the access list so you can verify that the WWN you entered was deleted from the ACL. 6. After verifying that the WWN was deleted correctly, enter 0 at the “select” prompt to end the session. 7. At the “Update the FLASH?” prompt, enter y. 8. Press Enter to update the nonvolatile memory and end the session.
Topology discovery 1 Number of Associated Node Names: 1 Associated Node Names: 10:00:00:60:69:20:15:75 Clearing the management server database Use the following procedure to clear the management server database: NOTE The command msPlClearDB is allowed only in AD0 and AD255. 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the msplClearDb command. 3. Enter y to confirm the deletion. The management server platform database is cleared.
1 Topology discovery *MS Topology Discovery enabled locally. *MS Topology Discovery Enable Operation Complete!! Disabling topology discovery Use the following procedure to disable topology discovery: 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the appropriate following command based on how you want to disable discovery: • For the local switch, enter the mstdDisable command. • For the entire fabric, enter the mstdDisable all command.
Device login 1 Device login A device can be storage, a host, or a switch. When new devices are introduced into the fabric, they must be powered on and, if a host or storage device, connected to a switch. Switch-to-switch logins (using the E_Port) are handled differently than storage and host logins. E_Ports exchange different frames than the ones listed below with the Fabric Controller to access the fabric. Once storage and host devices are powered on and connected, the following logins occur: 1.
1 Device login Fabric login process A device performs a fabric login (FLOGI) to determine if a fabric is present. If a fabric is detected then it exchanges service parameters with the fabric controller. A successful FLOGI sends back the 24-bit address for the device in the fabric. The device must issue and successfully complete a FLOGI command before communicating with other devices in the fabric.
High availability of daemon processes 1 Duplicate Port World Wide Name According to Fibre Channel standards, the Port World Wide Name (PWWN) of a device cannot overlap with that of another device, thus having duplicate PWWNs within the same fabric is an illegal configuration. If a PWWN conflict occurs with two devices attached to the same domain, Fabric OS handles device login in such a way that only one device may be logged in to the fabric at a time.
1 High availability of daemon processes Table 1 56 Daemons that are automatically restarted (Continued) Daemon Description traced Trace daemon provides trace entry date and time translation to Trace Device at startup and when date/time changed by command. Maintains the trace dump trigger parameters in a Trace Device. Performs the trace Background Dump, trace automatic FTP, and FTP “aliveness check” if auto-FTP is enabled. trafd Traffic daemon implements Bottleneck detection.
Chapter 2 Performing Basic Configuration Tasks In this chapter • Fabric OS overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Fabric OS command line interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Password modification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • The switch Ethernet interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 Fabric OS command line interface Although many different software and hardware configurations are tested and supported by Brocade Communications Systems, Inc., documenting all possible configurations and scenarios is beyond the scope of this document. In some cases, earlier releases are highlighted to present considerations for interoperating with them. The hardware reference manuals for Brocade products describe how to power up devices and set their IP addresses.
Fabric OS command line interface 2 • In a Windows environment enter the following parameters: TABLE 2 Terminal port parameters Parameter Value Bits per second 9600 Databits 8 Parity None Stop bits 1 Flow control None • In a UNIX environment, enter the following string at the prompt: tip /dev/ttyb -9600 If ttyb is already in use, use ttya instead and enter the following string at the prompt: tip /dev/ttya -9600 Telnet or SSH sessions You can connect to the Fabric OS through a Telnet or SSH
2 Fabric OS command line interface Connecting to Fabric OS using Telnet Use the following procedure to connect to the Fabric OS using Telnet: 1. Connect through a serial port to the switch that is appropriate for your fabric: • If Virtual Fabrics is enabled, log in using an admin account assigned the chassis-role permission. • If Virtual Fabrics is not enabled, log in using an account assigned to the admin role. 2.
Fabric OS command line interface 2 The commands in the following table provide help files for the indicated specific topics.
2 Fabric OS command line interface Example cliHistory command output from admin login switch:admin> clihistory CLI history Date & Time Thu Sep 27 10:14:41 2012 Thu Sep 27 10:14:48 2012 Message admin, 10.70.12.101, clihistory admin, 10.70.12.101, clihistory --show cliHistory --show Using the “--show” argument displays the same results as entering “cliHistory” without any arguments.
Password modification 2 Notes: • SSH login CLI logs are not recorded in the command line history. • The CLI command log will be collected as part of any “supportsave” operation. The command long record of such an operation will be the equivalent of running “cliHistory --showall”. • For CLI commands that require a password (Examples: firmwaredownload, configupload/download, supportsave, and so on), only the command (no arguments) is stored (see below for an illustration).
2 The switch Ethernet interface Changing the default account passwords at login Use the following procedure to change the default account passwords: 1. Connect to the switch and log in using the default administrative account. 2. At each of the “Enter new password” prompts, either enter a new password or skip the prompt. To skip a single prompt, press Enter. To skip all of the remaining prompts, press Ctrl-C. Example output of changing passwords login: admin Password: Please change your passwords now.
The switch Ethernet interface 2 NOTE When you change the Ethernet interface settings, open connections such as SSH or Telnet may be dropped. Reconnect using the new Ethernet IP address information or change the Ethernet settings using a console session through the serial port to maintain your session during the change. You must connect through the serial port to set the Ethernet IP address if the Ethernet network interface is not configured already.
2 The switch Ethernet interface The CP8 blade enables eth0 by default. If an error is encountered on eth0, it is treated the same as for any other port, unless the error causes the eth0 port to go down. If eth0 goes down, the eth3 interface becomes active and will remain active even if eth0 comes back up. Use one of the following actions to restore eth0 as the active interface. • Unplug the network cable, wait 5 seconds, and then plug it back in. • Perform an HA failover routine.
The switch Ethernet interface 2 Host Name: ecp1 Gateway IP Address: 10.1.2.3 IPFC address for virtual fabric ID 123: 11.1.2.3/24 IPFC address for virtual fabric ID 45: 13.1.2.4/20 Slot 7 eth0: 11.1.2.4/24 Gateway: 11.1.2.1 Backplane IP address of CP0 : 10.0.0.5 Backplane IP address of CP1 : 10.0.0.
2 The switch Ethernet interface Setting the static addresses for the Ethernet network interface Use the following procedure to set the Ethernet network interface static addresses: 1. Connect to the switch and log in using an account assigned to the admin role. 2. Perform the appropriate action based on whether you have a switch or Backbone: • If you are setting the IP address for a switch, enter the ipAddrSet command.
The switch Ethernet interface 2 DHCP activation Some Brocade switches have DHCP enabled by default. Fabric OS support for DHCP functionality is only provided for Brocade fixed-port switches. These are listed in the Preface. NOTE The Brocade DCX and Brocade DCX-4S Backbones do not support DHCP.
2 The switch Ethernet interface Example of enabling DHCP for IPv4 interactively: switch:admin> ipaddrset Ethernet IP Address [10.1.2.3]: Ethernet Subnetmask [255.255.255.0]: Fibre Channel IP Address [220.220.220.2]: Fibre Channel Subnetmask [255.255.0.0]: Gateway IP Address [10.1.2.1]: DHCP [Off]:on Example of enabling DHCP for IPv4 using a single command: switch:admin> ipaddrset –ipv4 -add -dhcp ON switch:admin> ipaddrshow SWITCH Ethernet IP Address: 10.20.134.219 Ethernet Subnetmask: 255.255.240.
The switch Ethernet interface 2 SWITCH Ethernet IP Address: 10.20.134.219 Ethernet Subnetmask: 255.255.240.0 Gateway IP Address: 10.20.128.1 DHCP: Off IPv6 autoconfiguration IPv6 can assign multiple IP addresses to each network interface. Each interface is configured with a link local address in almost all cases, but this address is only accessible from other hosts on the same network.
2 Date and time settings Date and time settings Switches maintain the current date and time inside a battery-backed real-time clock (RTC) circuit that receives the date and time from the fabric’s principal switch. Date and time are used for logging events. Switch operation does not depend on the date and time; a switch with an incorrect date and time value functions properly. However, because the date and time are used for logging, error detection, and troubleshooting, you must set them correctly.
Date and time settings 2 When you set the time zone for a switch, you can perform the following tasks: • Display all of the time zones supported in the firmware. • Set the time zone based on a country and city combination or based on a time zone ID, such as PST. The time zone setting has the following characteristics: • Users can view the time zone settings. However, only those with administrative permissions can set the time zones. • The setting automatically adjusts for Daylight Savings Time.
2 Date and time settings You are prompted to select a general location. Please identify a location so that time zone rules can be set correctly. 3. Enter the appropriate number or press Ctrl-D to quit. 4. Select a country location at the prompt. 5. Enter the appropriate number at the prompt to specify the time zone region of Ctrl-D to quit. Network time protocol You can synchronize the local time of the principal and primary FCS switch to a maximum of eight external Network Time Protocol (NTP) servers.
Domain IDs 2 Example of setting the NTP server switch:admin> tsclockserver LOCL switch:admin> tsclockserver "10.1.2.3" Example of displaying the NTP server switch:admin> tsclockserver 10.1.2.3 Example of setting up more than one NTP server using a DNS name switch:admin> tsclockserver "10.1.2.4;10.1.2.5;ntp.localdomain.net" Updating Clock Server configuration...done.
2 Switch names ------------------------------------------------------------------------2: fffc02 10:00:00:60:69:e0:01:46 10.3.220.1 0.0.0.0 "ras001" 3: fffc03 10:00:00:60:69:e0:01:47 10.3.220.2 0.0.0.0 "ras002" 5: fffc05 10:00:00:05:1e:34:01:bd 10.3.220.5 0.0.0.0 "ras005" fec0:60:69bc:63:205:1eff:fe34:1bd 6: fffc06 10:00:00:05:1e:34:02:3e 10.3.220.6 0.0.0.0 >"ras006" 7: fffc07 10:00:00:05:1e:34:02:0c 10.3.220.7 0.0.0.
Chassis names 2 The following considerations apply to switch naming: • Switch names can be from 1 through 30 characters long. • All switch names must begin with a letter, and can contain letters, numbers, or the underscore character. • Switch names must be unique across logical switches. • Changing the switch name causes a domain address format RSCN to be issued and may be disruptive to the fabric. Customizing the switch name 1.
2 Switch activation and deactivation The following considerations apply to fabric naming: • Each name must be unique for each logical switch within a chassis; duplicate fabric names are not allowed. • A fabric name can be from 1 through 128 alphanumeric characters. • All switches in a logical fabric must be running Fabric OS v7.2.0. Switches running earlier versions of the firmware can co-exist in the fabric, but do not show the fabric name details.
Switch activation and deactivation TABLE 5 2 Ports affected when you enable or disable a switch in VF or non-VF mode Operation Virtual Fabrics enabled Virtual Fabrics not enabled Enable switch Enables all ports on logical switch Enables all ports on physical chassis Enable chassis Enables all ports on physical chassis Not allowed Disable switch Disables all ports on logical switch Disables all ports on physical chassis Disable chassis Disables all ports on physical chassis Not allowed Dis
2 Switch and Backbone shutdown switch:FID128:admin> chassisdisable This command can cause disruption to multiple logical switches. Are you sure you want to disable all chassis ports now? (yes, y, no, n): [no]y switch:FID128:admin> All Fibre Channel ports on all logical switches are taken offline. If the logical switches are in fabrics, the fabrics are reconfigured. NOTE After a chassisDisable, if you want to do an haFailover, you should wait at least 30 seconds.
Basic connections 2 Broadcast message from root (ttyS0) Wed Jan 25 16:12:09 2006... The system is going down for system halt NOW !! INIT: Switching to runlevel: 0 INIT: Sending processes the TERM signal Unmounting all filesystems. The system is halted flushing ide devices: hda Power down. 5. Power off the switch. Powering off a Brocade Backbone Use the following procedure to power off a Brocade Backbone device: 1. From the active CP in a dual-CP platform, enter the sysShutdown command.
2 Basic connections Device connection To minimize port logins, power off all devices before connecting them to the switch. When powering the devices back on, wait for each device to complete the fabric login before powering on the next one. For devices that cannot be powered off, first use the portDisable command to disable the port on the switch, connect the device, and then use the portEnable command to enable the port.
Chapter Performing Advanced Configuration Tasks 3 In this chapter • Port identifiers (PIDs) and PID binding overview. . . . . . . . . . . . . . . . . . . . . . 83 • Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 • Blade terminology and compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 • Enabling and disabling blades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3 Port identifiers (PIDs) and PID binding overview Core PID addressing mode Core PID is the default PID format for Brocade platforms. It uses the entire 24-bit address space of the domain, area ID, and AL_PA to determine an object’s address within the fabric.
Port identifiers (PIDs) and PID binding overview 3 • Shared area limitations are removed on 48-port and 64-port blades. • Any port on a 48-port or 64-port blade can support up to 256 NPIV devices (in fixed addressing mode, only 128 NPIV devices are supported in non-VF mode and 64 NPIV devices in VF mode on a 48-port blade). • Any port on a 48-port blade can support loop devices. • Any port on a 48-port or 64-port blade can support hard port zoning.
3 Port identifiers (PIDs) and PID binding overview WWN-based PID assignment WWN-based PID assignment is disabled by default. When the feature is enabled, bindings are created dynamically; as new devices log in, they automatically enter the WWN-based PID database. The bindings exist until you explicitly unbind the mappings through the CLI or change to a different addressing mode.
Port identifiers (PIDs) and PID binding overview 3 Use the following procedure to enable automatic PID assignment. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the configure command. 3. At the Fabric parameters prompt, type y. 4. At the WWN Based persistent PID prompt, type y. 5. Press Enter to bypass the remaining prompts without changing them. Example of activating PID assignments switch: admin> configure Configure...
3 Ports Ports Ports provide either a physical or virtual network connection point for a device. Brocade devices support a wide variety of ports. Port Types The following is a list of port types that may be part of a Brocade device: • D_Port — A diagnostic port lets an administrator isolate the inter-switch link (ISL) to diagnose link level faults. This port runs only specific diagnostics tests and does not carry any fabric traffic.
Ports 3 The different blades that can be inserted into a chassis are described as follows: • Control processor (CP) blades contain communication ports for system management, and are used for low-level, platform-wide tasks. • Core blades are used for intra-chassis switching as well as interconnecting two Backbones. • Port blades are used for host, storage, and interswitch connections.
3 Ports Port identification by port area ID The relationship between the port number and area ID depends upon the PID format used in the fabric. When Core PID format is in effect, the area ID for port 0 is 0, for port 1 is 1, and so forth. For 32-port blades (FC8-32, FC8-32E, FC16-32), the numbering is contiguous up to port 15; from port 16, the numbering is still contiguous, but starts with 128.
Ports 3 The setting is retained and applied any time an 8 Gbps device logs in. Upgrades from prior releases which supported only Modes 0 and 1 will not change the existing setting, but switches reset to factory defaults with Fabric OS v6.3.1 or later will be configured to Mode 0 by default. The default setting on new units may vary by vendor.
3 Ports 5. Enter the portSwapShow command to verify that the port area IDs have been swapped. A table shows the physical port numbers and the logical area IDs for any swapped ports. 6. Enter the portSwapDisable command to disable the port swap feature. Port activation and deactivation By default, all licensed ports are enabled. You can disable and re-enable them as necessary.
Ports 3 Fabric OS 7.1.0 and later provides F_Port decommissioning and recommissioning using Brocade Network Advisor 12.1.0 and later. Refer to the Brocade Network Advisor User Manual for details. NOTE All members of a trunk group must have an equal link cost value in order for any of the members to be decommissioned. If any member of a trunk group does not have an equal cost, requests to decommission a trunk member will fail and an error reminding the caller of this requirement is produced.
3 Ports switch:admin> ifmodeset eth3 Exercise care when using this command. Forcing the link to an operating mode not supported by the network equipment to which it is attached may result in an inability to communicate with the system through its ethernet interface. It is recommended that you only use this command from the serial console port. Are you sure you really want to do this? (yes, y, no, n): [no] y Proceed with caution.
Blade terminology and compatibility 3 Setting port speed for a port octet You can use the portCfgOctetSpeedCombo command to configure the speed for a port octet. Be aware that in a Virtual Fabrics environment, this command configures the speed of a port octet chassis-wide and not only on the logical switch. Use the following procedure to set the port speed for a port octet. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the portCfgOctetSpeedCombo command.
3 TABLE 7 Blade terminology and compatibility Port blade terminology, numbering, and platform support Supported on: Blade Blade ID DCX family (slotshow) DCX 8510 family Ports Definition FC8-161 21 Yes No 16 8-Gbps port blade supporting 1, 2, 4, and 8 Gbps port speeds. Ports are numbered from 0 through 15 from bottom to top. FC8-321 55 Yes No 32 8-Gbps port blade supporting 1, 2, 4, and 8 Gbps port speeds.
Blade terminology and compatibility TABLE 7 3 Port blade terminology, numbering, and platform support (Continued) Supported on: Blade Blade ID DCX family (slotshow) DCX 8510 family Ports Definition FCOE10-24 74 Yes No 24 10-GbE DCB ports An application blade that provides Converged Enhanced Ethernet to bridge a Fibre Channel and Ethernet SAN. Ports are numbered from 0 through 11 from bottom to top on the left set of ports and 12 through 23 from bottom to top on the right set of ports.
3 Enabling and disabling blades Port and application blade compatibility Table 7 on page 96 identifies which port and application blades are supported for each Brocade Backbone. NOTE During power up of a Brocade DCX or DCX-4S Backbone, if an FCOE10-24 is detected first before any other AP blade, all other AP and FC8-64 blades are faulted. If a non-FCOE10-24 blade is detected first, then any subsequently-detected FCOE10-24 blades are faulted. Blades are powered up starting with slot 1.
Blade swapping 3 Enabling blades Use the following procedure to enable a blade. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the bladeEnable command with the slot number of the port blade you want to enable. ecp:admin> bladeenable 3 Slot 3 is being enabled FC8-48, FC8-48E, FC8-64, and FC16-48 port blade enabling exceptions Because the area IDs are shared with different port IDs, the FC8-48, FC8-48E, FC8-64, and FC16-48 blades support only F_Ports and E_Ports.
3 Blade swapping • Undetermined board types cannot be swapped. For example, a blade swap will fail if the blade type cannot be identified. • Blade swapping is not supported when swapping to a different model of blade or a different port count. For example, you cannot swap an FC8-32 blade with an FC8-48 port blade. How blades are swapped The bladeSwap command performs the following operations: 1.
Blade swapping 3 The preparation process also includes any special handling of ports associated with logical switches. For example, Figure 3 shows the source blade has ports in a logical switch or logical fabric, and the corresponding destination ports must be included in the associated logical switch or logical fabric of the source ports. FIGURE 3 Blade swap with Virtual Fabrics during the swap 4.
3 Disabling switches FIGURE 4 Blade swap with Virtual Fabrics after the swap Swapping blades Use the following procedure to swap blades. 1. Connect to the Backbone and log in using an account with admin permissions. 2. Enter the bladeSwap command. If no errors are encountered, the blade swap will complete successfully. If errors are encountered, the command is interrupted and the ports are set back to their original configurations. 3.
Power management 3 Power management All blades are powered on by default when the switch chassis is powered on. Blades cannot be powered off when POST or AP initialization is in progress. To manage power and ensure that more critical components are the least affected by power changes, you can specify the order in which the components are powered off by using the powerOffListSet command.
3 Equipment status Equipment status You can check the status of switch operation, High Availability features, and fabric connectivity. Checking switch operation Use the following procedure to check switch operation. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the switchShow command. This command displays a switch summary and a port summary. 3. Check that the switch and ports are online. 4.
Equipment status 3 Verifying fabric connectivity Use the following procedure to verify fabric connectivity. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the fabricShow command. This command displays a summary of all the switches in the fabric. The output of the fabricShow command is discussed in “Domain IDs” on page 75. Verifying device connectivity Use the following procedure to verify device connectivity. 1.
3 Equipment status If the switch is running Fabric Watch, you can use the following procedure to view the switch status policy threshold values. If the switch is running MAPS, refer to the Monitoring and Alerting Policy Suite Administrator’s Guide. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the switchStatusPolicyShow command. Whenever there is a switch change, an error message is logged and an SNMP connUnitStatusChange trap is sent.
Audit log configuration 3 Example output from a switch The following example displays what is typically seen from a Brocade switch, but the quantity and types vary by platform. switch:admin> switchstatuspolicyshow To change the overall switch status policy parameters The current overall switch status policy parameters: Down Marginal ----------------------------------PowerSupplies 2 1 Temperatures 2 1 Fans 2 1 Flash 0 1 MarginalPorts 25.00%[12] 10.00%[5] FaultyPorts 25.00%[12] 10.00%[5] MissingSFPs 0.
3 Audit log configuration Auditable events are generated by the switch and streamed to an external host through a configured system message log daemon (syslog). You specify a filter on the output to select the event classes that are sent through the system message log. The filtered events are streamed chronologically and sent to the system message log on an external host in the specified audit message format.
Audit log configuration 3 Verifying host syslog prior to configuring the audit log Audit logging assumes that your syslog is operational and running. Before configuring an audit log, you must perform the following steps to ensure that the host syslog is operational. 1. Set up an external host machine with a system message log daemon running to receive the audit events that will be generated. 2.
3 Duplicate PWWN handling during device login Oct 10 09:00:04 10.3.220.7 raslogd: AUDIT, 2008/10/10-08:28:16 (GMT), [SEC-3021], INFO, SECURITY, admin/NONE/10.3.220.13/None/CLI, None/ras007/FID 128, , Event: login, Status: failed, Info: Failed login attempt via REMOTE, IP Addr: 10.3.220.13. Duplicate PWWN handling during device login If a device attempts to log in with the same port WWN (PWWN) as another device on the switch, you can configure whether the new login or the existing login takes precedence.
Enabling forward error correction TABLE 11 3 Duplicate PWWN behavior: Port type determines which login takes precedence Input port First port login is NPIV port First port login is F_Port FLOGI received New login forces an explicit logout of original FDISC on the previous NPIV port. New login is rejected and the new port is persistently disabled. FDISC received New FDISC forces an explicit logout of original FDISC on the previous NPIV port. New FDISC is rejected.
3 Enabling forward error correction • FEC enables automatically when negotiation with a switch detects FEC capability. • FEC persists after driver reloads and system reboots. • FEC functions with features such as QoS, trunking, and BB_Credit recovery. FEC Limitations The following limitations apply to FEC: • FEC is configurable only on 16 Gbps-capable switches (Brocade 6505, 6510, 6520, M6505, 6547, and the Brocade DCX 8510 Backbone family).
Enabling forward error correction 3 Disabling forward error correction To disable the FEC feature on a port range, enter the portCfgFec --disable command. switch:admin> portcfgfec --disable 0-8 Enabling or disabling FEC for long-distance ports To enable or disable FEC for long-distance ports, use portCfgLongDistance with the -fecEnable or -fecDisable parameter as required.
3 114 Enabling forward error correction Fabric OS Administrator’s Guide 53-1002920-02
Chapter 4 Routing Traffic In this chapter • Routing overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Inter-switch links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Gateway links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Routing policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4 Routing overview Paths and route selection Paths are possible ways to get from one switch to another. Each inter-switch link (ISL) has a metric cost based on bandwidth. The cumulative cost is based on the sum of all costs of all traversed ISLs. Route selection is the path that is chosen. Paths that are selected from the routing database are chosen based on the minimal cost.
Routing overview 4 FSPF makes minimal use of the ISL bandwidth, leaving virtually all of it available for traffic. In a stable fabric, a switch transmits 64 bytes every 20 seconds in each direction. FSPF frames have the highest priority in the fabric. This guarantees that a control frame is not delayed by user data and that FSPF routing decisions occur very quickly during convergence. FSPF guarantees a routing loop-free topology at all times.
4 Inter-switch links Inter-switch links An inter-switch link (ISL) is a link between two switches, E_Port-to-E_Port. The ports of the two switches automatically come online as E_Ports once the login process finishes successfully. For more information on the login process, refer to Chapter 1, “Understanding Fibre Channel Services”. You can expand your fabric by connecting new switches to existing switches. Figure 6 shows a new switch being added into an existing fabric.
Inter-switch links 4 Buffer credits In order to prevent the dropping of frames in the fabric, a device can never send frames without the receiving device being able to receive them, so an end-to-end flow control is used on the switch. Flow control in Fibre Channel uses buffer-to-buffer credits, which are distributed by the switch. When all buffer-to-buffer credits are utilized, a device waits for a VC_RDY or an R_RDY primitive from the destination switch before resuming I/O.
4 Gateway links FIGURE 7 Virtual channels on a QoS-enabled ISL Gateway links A gateway merges SANs into a single fabric by establishing point-to-point E_Port connectivity between two Fibre Channel switches that are separated by a network with a protocol such as IP or SONET. Except for link initialization, gateways are transparent to switches; the gateway simply provides E_Port connectivity from one switch to another. Figure 8 shows two separate SANs, A-1 and A-2, merged together using a gateway.
Gateway links FIGURE 8 4 Gateway link merging SANs By default, switch ports initialize links using the Exchange Link Parameters (ELP) mode 1. However, gateways expect initialization with ELP mode 2, also referred to as ISL R_RDY mode. Therefore, to enable two switches to link through a gateway, the ports on both switches must be set for ELP mode 2.
4 Routing policies Example of enabling a gateway link on slot 2, port 3 ecp:admin> portcfgislmode 2/3, 1 Committing configuration...done. ISL R_RDY Mode is enabled for port 3. Please make sure the PID formats are consistent across the entire fabric. Routing policies By default, all routing protocols place their routes into a routing table.
Routing policies 4 Port-based routing The choice of routing path is based only on the incoming port and the destination domain. To optimize port-based routing, Dynamic Load Sharing (DLS) can be enabled to balance the load across the available output ports within a domain. NOTE For FC routers only: When an FC router is in port-based routing mode, the backbone traffic is load-balanced based on SID and DID.
4 Routing policies Dynamic Path Selection DPS assigns communication paths between end devices in a fabric to egress ports in ratios proportional to the potential bandwidth of the ISL, ICL, or trunk group. When there are multiple paths to a destination, the input traffic is distributed across the different paths in proportion to the bandwidth available on each of the paths. This improves utilization of the available paths, thus reducing possible congestion on the paths.
Route selection 4 Setting the routing policy Use the following procedure to set the routing policy. 1. Connect to the VF switch and log in as admin. 2. Enter the setcontext [FID | switchname] command for the correct Fabric ID or switch name. • The fabricID parameter is the FID of the logical switch you just created. • The switchname parameter is the name assigned to the logical switch. • You can only use one parameter at a time. switch:admin> setcontext 20 3.
4 Frame order delivery • An EX_Port goes offline • A device goes offline Setting DLS Use the following procedure to set DLS. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the dlsShow command to view the current DLS setting. One of the following messages appears: • “DLS is set” indicates that DLS is turned on. • “DLS is not set” indicates that DLS is turned off. • ”DLS is set with Lossless enabled.” DLS is enabled with the Lossless feature.
Frame order delivery 4 If even one switch in the fabric delivers out-of-order exchanges, then exchanges are delivered to the target out of order, regardless of the policy configured on other switches in the fabric. NOTE Some devices do not tolerate out-of-order exchanges; in such cases, use the port-based routing policy. In a stable fabric, frames are always delivered in order, even when the traffic between switches is shared among multiple paths.
4 Frame order delivery Use the following procedure to view frames. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter frameLog --show.
Lossless Dynamic Load Sharing on ports 4 Lossless Dynamic Load Sharing on ports Lossless Dynamic Load Sharing (DLS) allows you to rebalance port paths without causing input/output (I/O) failures. For devices where in-order delivery (IOD) of frames is required, you can set IOD separately.
4 Lossless Dynamic Load Sharing on ports Lossless DLS does the following whenever paths need to be rebalanced: 1. Pauses ingress traffic by not returning credits. Frames that are already in transit are not dropped. 2. Changes the existing path to a more optimal path. 3. If IOD is enabled, waits for sufficient time for frames already received to be transmitted. This is needed to maintain IOD. 4. Resumes traffic. Table 12 shows the effect of frames when you have a specific routing policy turned on with IOD.
Lossless Dynamic Load Sharing on ports 4 Configuring Lossless Dynamic Load Sharing You configure Lossless DLS switch- or chassis-wide by using the dlsSet command to specify that no frames are dropped while rebalancing or rerouting traffic. Use the following procedure to configure Lossless Dynamic Load Sharing. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the appropriate dlsSet command to enable or disable Lossless Dynamic Load Sharing.
4 Frame Redirection Frame Redirection Frame Redirection provides a means to redirect traffic flow between a host and a target that use virtualization and encryption applications, such as the Brocade SAS blade and Brocade Data Migration Manager (DMM), so that those applications can perform without having to reconfigure the host and target. You can use this feature if the hosts and targets are not directly attached. Frame Redirection depends on the wide distribution of the Defined Zone Database.
Frame Redirection 4 3. Enter the cfgSave command to save the frame redirect zones to the defined configuration. The following example creates a redirect zone, given a host (10:10:10:10:10:10:10:10), target (20:20:20:20:20:20:20:20), virtual initiator (30:30:30:30:30:30:30:30), and virtual target (40:40:40:40:40:40:40:40): switch:admin>zone --rdcreate 10:10:10:10:10:10:10:10 20:20:20:20:20:20:20:20 \ 30:30:30:30:30:30:30:30 40:40:40:40:40:40:40:40 restartable noFCR Deleting a frame redirect zone Use the f
4 134 Frame Redirection Fabric OS Administrator’s Guide 53-1002920-02
Chapter Buffer-to-Buffer Credits and Credit Recovery 5 In this chapter • Buffer credit management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 • Buffer credit recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 • Credit loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5 Buffer credit management Buffer-to-buffer flow control is flow control between adjacent ports in the I/O path, for example, transmission control over individual network links. A separate, independent pool of credits is used to manage buffer-to-buffer flow control. A sending port uses its available credit supply and waits to have the credits replenished by the port on the opposite end of the link.
Buffer credit management 5 Considerations for calculating buffer credits Considerations follow for calculating how many ports can be configured for long distance on all Fabric OS v7.x-capable switch modules: • Each port is part of a port group that includes a pool of buffer credits that can be used. This port group is not the same as the port groups used for ISL Trunking. • Each user port reserves eight buffer credits when online or offline.
5 Buffer credit management Table 14 describes Fibre Channel data frames.
Buffer credit management 5 3. Use one of the following formulas to calculate the reserved buffers for distance: • If QoS is enabled: (Reserved Buffer for Distance Y) = (X * LinkSpeed / 2) + 6 + 14 • If QoS is not enabled: (Reserved Buffer for Distance Y) = (X * LinkSpeed / 2) + 6 The formulas use the following parameters: X = The distance determined in step 1 (in km). LinkSpeed = The speed of the link determined in step 2.
5 Buffer credit management The following values are used in the example: • 484 — The total number of unreserved buffer credits • 206 — Buffer credits needed for 50 km at 8 Gbps • 8 — The number of reserved buffer credits already allocated to that port The resulting number is rounded down to the next whole number because fractions of a port are not allowed. If you have a distance of 50 km at 1 Gbps, then 484 / (31 – 8) = 21 ports.
Buffer credit management 5 If buffer credit recovery is enabled, Fabric OS supports a BB_SC_N range of 1 to 15; therefore, it is impossible for the desired_distance value to be more than the number of buffer credits available in the pool as determined by the previous calculations The distance for buffer credit recovery is well within the range of all possible connections.
5 Buffer credit management Calculating the number of buffers required given the distance, speed, and frame size If you know the distance, speed, and frame size for a given port, you can use the portBufferCalc command to calculate the number of buffers required. If you omit the distance, speed, or frame size, the command uses the currently configured values for the port.
Buffer credit management 5 The average frame size in bytes is shown in parentheses with the average buffer usage for packet transmission and reception.
5 Buffer credit management TABLE 15 Total FC ports, ports per port group, and unreserved buffer credits per port group (Continued) Switch/blade model Total FC ports (per switch/blade) User port group size Unreserved buffer credits per port group VA-40FC 40 40 1692 Brocade Encryption Switch 32 16 1392 FC8-16 16 16 1292/508 FC8-32 32 16 1292/508 FC8-32E 32 16 5456 FC8-48 48 24 1228/716 FC8-48E 48 24 5008 FC8-64 *** Extended Fabrics is not supported on this blade *** FC16-32
Buffer credit management TABLE 16 5 Configurable distances for Extended Fabrics (Continued) Maximum distances (km) that can be configured (assuming a 2112-byte frame size) Switch/blade model 2 Gbps 4 Gbps 8 Gbps 10 Gbps 16 Gbps 6547 7714 3857 1928 1542 964 7800 410 205 102 N/A N/A VA-40FC 1694 847 423 N/A N/A Brocade Encryption Switch 1392 696 348 N/A N/A FC8-16 1294 647 323 N/A N/A FC8-32 1294 647 323 N/A N/A FC8-32E 5190 2595 1297 1038 648 FC8-48 1230
5 Buffer credit recovery When a port is configured with the –buffers option A firmware downgrade is blocked when a port is configured as a long-distance port by means of the –buffers option. The following warning message is displayed: Downgrade to selected version is not allowed because few ports are configured with Longdistance -buffers option. Please remove the configuration using portcfglongdistance / L0 CLI or change the configuration with -distance option on the console.
Buffer credit recovery 5 Buffer credit recovery is enabled automatically across any long-distance connection for which the E_Port, F_Port, or EX_Port buffer credit recovery mechanism is supported. For 16-Gbps FC devices and blades (Brocade 6505, 6510, 6520, M6505, 6547, CR16-4, CR16-8, FC8-32E, FC8-48E, FC16-32, FC16-48), you can use the portCfgCreditRecovery command to disable or enable buffer credit recovery on a port.
5 Buffer credit recovery Buffer credit recovery over an EX_Port Buffer credit recovery is supported on a Fibre Channel router (FCR) EX_Port that connects over an inter-fabric link (IFL) to an edge fabric E_Port when the following conditions are met: • The FCR and the switch at the other end of the IFL must both run Fabric OS v7.1 or later. • The FCR and the switch at either end of the IFL must both support 16 Gbps or 8 Gbps.
Credit loss 5 Credit loss Fabric OS v7.1 and later supports back-end credit loss detection, back-end ports and core blades, and the Brocade 5300 and 6520 switches, although the support is slightly different on each device. Refer to the following details on these switches, and the Fabric OS Troubleshooting and Diagnostics Guide for more general information.
5 Credit loss The following credit loss recovery methods are supported for Brocade 6520 back-end ports: • For all the credit loss methods described previously, a link reset will automatically be performed, assuming that this option was enabled. Refer to “Enabling back-end credit loss detection and recovery” for details on enabling this feature. • A manual link reset option using the creditRecovMode command is also available.
Chapter 6 Managing User Accounts In this chapter • User accounts overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Local database user accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Local user account database distribution . . . . . . . . . . . . . . . . . . . . . . . . . . • Password policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • The boot PROM password . . . . . . . . . . . .
6 User accounts overview Fabric OS provides four options for authenticating users: remote RADIUS service, remote LDAP service, remote TACACS+ service, and the local-switch user database. All options allow users to be managed centrally by means of the following methods: • Remote RADIUS service: Users are managed in a remote RADIUS server. All switches in the fabric can be configured to authenticate against the centralized remote database. • Remote LDAP service: Users are managed in a remote LDAP server.
User accounts overview 6 Admin Domain considerations Legacy users with no Admin Domain specified and whose current role is admin will have access to AD0 through AD255 (physical fabric admin); otherwise, they will have access to AD0 only. If some Admin Domains have been defined for the user and all of them are inactive, the user will not be allowed to log in to any switch in the fabric. If no home domain is specified for a user, the system provides a default home domain.
6 User accounts overview Management channel The management channel is the communication established between the management workstation and the switch. Table 19 shows the number of simultaneous login sessions allowed for each role when authenticated locally. The roles are displayed in alphabetic order, which does not reflect their importance. When LDAP, RADIUS, or TACACS+ are used for authentication, the total number of sessions on a switch may not exceed 32.
Local database user accounts 6 The assigned permissions can be no higher than the admin role permission assigned to the class. The admin role permission for the Security class is Observe/Modify. Therefore, the Observe permission is valid. The roleConfig --show command is available to view the permissions assigned to a user-defined role. You can also use the classConfig --showroles command to see that the role was indeed added with Observe permission for the security commands.
6 Local database user accounts Default accounts Table 20 lists the predefined accounts offered by Fabric OS that are available in the local-switch user database. The password for all default accounts should be changed during the initial installation and configuration of each switch. TABLE 20 Default local user accounts Account name Role Admin Domain Logical Fabric Description admin Admin AD0–255 home: 0 LF1–128 home: 128 Most commands have Observe/Modify permission.
Local database user accounts 6 3. In response to the prompt, enter a password for the account. The password is not displayed when you enter it on the command line. Deleting an account This procedure can be performed on local user accounts. 1. Connect to the switch and log in using an account with admin permissions, or an account associated with a user-defined role with permissions for the UserManagement class of commands. 2. Enter the userConfig --delete command. You cannot delete the default accounts.
6 Local user account database distribution Changing the password for a different account 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the passwd command specifying the name of the account for which the password is being changed. 3. Enter the requested information at the prompts. Local user account database distribution Fabric OS allows you to distribute the user database and passwords to other switches in the fabric.
Password policies 6 Rejecting distributed user databases on the local switch 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the fddCfg --localreject PWD command. Password policies The password policies described in this section apply to the local-switch user database only. Configured password policies (and all user account attribute and password state information) are synchronized across CPs and remain unchanged after an HA failover.
6 Password policies • MinLength Specifies the minimum length of the password. The minimum can be from 8 through 40 characters. New passwords must be between the minimum length specified and 40 characters. The default value is 8. The maximum value must be greater than or equal to the MinLength value. • Repeat Specifies the length of repeated character sequences that will be disallowed.
Password policies 6 Password expiration policy The password expiration policy forces the expiration of a password after a configurable period of time. The expiration policy can be enforced across all user accounts or on specified users only. A warning that password expiration is approaching is displayed when the user logs in. When a password expires, the user must change the password to complete the authentication process and open a user session.
6 Password policies A failed login attempt counter is maintained for each user on each switch instance. The counters for all user accounts are reset to zero when the account lockout policy is enabled. The counter for an individual account is reset to zero when the account is unlocked after a lockout duration period expires, or when the account user logs in successfully. The admin account can also have the lockout policy enabled on it.
The boot PROM password 6 Denial of service implications The account lockout mechanism may be used to create a denial of service condition when a user repeatedly attempts to log in to an account by using an incorrect password. Selected privileged accounts are exempted from the account lockout policy to prevent users from being locked out from a denial of service attack. However, these privileged accounts may then become the target of password-guessing attacks.
6 The boot PROM password 4. Enter 2. • If no password was previously set, the following message is displayed: Recovery password is NOT set. Please set it now. • If a password was previously set, the following messages is displayed: Send the following string to Customer Support for password recovery: afHTpyLsDo1Pz0Pk5GzhIw== Enter the supplied recovery password. Recovery Password: 5. Enter the recovery password (string). The recovery string must be from 8 through 40 alphanumeric characters in length.
The boot PROM password 6 5. Enter 2. Take the following appropriate action based on whether you find the password was previously set: • If no password was previously set, the following message is displayed: Recovery password is NOT set. Please set it now. • If a password was previously set, the following messages are displayed: Send the following string to Customer Support for password recovery: afHTpyLsDo1Pz0Pk5GzhIw== Enter the supplied recovery password. Recovery Password: 6.
6 The boot PROM password 1. Create a serial connection to the switch as described in “Connecting to Fabric OS through the serial port” on page 58. 2. Reboot the switch by entering the reboot command. 3. Press Esc within four seconds after the message “Press escape within 4 seconds...” is displayed. The following options are available: Option Description 1 2 3 Continues the system boot process. Lets you set the recovery string and the boot PROM password. Provides access to boot parameters.
Remote authentication 6 The following options are available: Option Description 1 2 3 Continues the system boot process. Lets you set the recovery string and the boot PROM password. Provides access to boot parameters. Start system. Recovery password. Enter command shell. 6. Enter 3. 7. Enter the passwd command at the shell prompt. The passwd command applies only to the boot PROM password when it is entered from the boot interface. 8.
6 Remote authentication Client/server model When configured to use one of the supported remote authentication services, the switch acts as a Network Access Server (NAS) and RADIUS, LDAP, or TACACS+ client. The switch sends all authentication, authorization, and accounting (AAA) service requests to the authentication server. The authentication server receives the request, validates the request, and sends its response back to the switch.
6 Remote authentication Consider the effects of the use of a remote authentication service on other Fabric OS features. For example, when a remote authentication service is enabled, all account passwords must be managed on the authentication server. The Fabric OS mechanisms for changing switch passwords remain functional; however, such changes affect only the involved switches locally. They do not propagate to the authentication server, nor do they affect any account on the authentication server.
6 Remote authentication TABLE 22 Authentication configuration options (Continued) aaaConfig options Description Equivalent setting in Fabric OS v5.1.0 and earlier --radius --authspec “radius;local” Authenticates management connections against any RADIUS databases first. If RADIUS fails for any reason, authenticates against the local user database. not not supported supported --authspec “radius;local” --backup Authenticates management connections against any RADIUS databases.
Remote authentication 6 Setting the switch authentication mode 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the aaaConfig --authspec command. Fabric OS user accounts RADIUS, LDAP, and TACACS+ servers allow you to set up user accounts by their true network-wide identities rather than by the account names created on a Fabric OS switch. With each account name, assign the appropriate switch access permissions.
6 Remote authentication TABLE 23 Syntax for VSA-based account roles (Continued) Item Value Description Vendor type 1 1 octet, Brocade-Auth-Role; valid attributes for the Brocade-Auth-Role are: Admin BasicSwitchAdmin FabricAdmin Operator SecurityAdmin SwitchAdmin User ZoneAdmin 2 Optional: Specifies the Admin Domain or Virtual Fabric member list. For more information on Admin Domains or Virtual Fabrics, refer to “RADIUS configuration with Admin Domains or Virtual Fabrics” on page 173.
Remote authentication FIGURE 10 6 Windows 2000 VSA configuration Linux FreeRADIUS server For the configuration on a Linux FreeRADIUS server, define the values outlined in Table 24 in a vendor dictionary file called dictionary.brocade. TABLE 24 Entries in dictionary.
6 Remote authentication The values for these attribute types use the syntax key=val[;key=val], where key is a text description of attributes, val is the attribute value for the given key, the equal sign (=) is the separator between key and value, and the semicolon (;) is an optional separator for multiple key-value pairs. Multiple key-value pairs can appear for one Vendor-Type code. Key-value pairs with the same key name may be concatenated across multiple Vendor-Type codes.
Remote authentication 6 In the next example, on a Linux FreeRADIUS Server, the user has the “zoneAdmin” permissions, with VFlist 2, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15 17, 19, 22, 23, 24, 25, 29, 31 and HomeLF 1.
6 Remote authentication ATTRIBUTE ATTRIBUTE ATTRIBUTE ATTRIBUTE ATTRIBUTE ATTRIBUTE ATTRIBUTE Brocade-Auth-Role Brocade-AVPairs1 Brocade-AVPairs2 Brocade-AVPairs3 Brocade-AVPairs4 Brocade-Passwd-ExpiryDate Brocade-Passwd-WarnPeriod 1 2 3 4 5 6 7 string string string string string string string Brocade Brocade Brocade Brocade Brocade Brocade Brocade This information defines the Brocade vendor ID as 1588, Brocade attribute 1 as Brocade-Auth-Role, Brocade attribute 6 as Brocade-Passwd-ExpiryDate, and Br
Remote authentication 6 The Brocade Backbones send their RADIUS requests using the IP address of the active CP. When adding clients, add both the active and standby CP IP addresses so that, in the event of a failover, users can still log in to the switch. 1. Open the $PREFIX/etc/raddb/client.config file in a text editor and add the switches that are to be configured as RADIUS clients. For example, to configure the switch at IP address 10.32.170.59 as a client: client 10.32.170.
6 Remote authentication 3. Configuring a user IAS is the Microsoft implementation of a RADIUS server and proxy. IAS uses the Windows native user database to verify user login credentials; it does not list specific users, but instead lists user groups. Each user group should be associated with a specific switch role. For example, you should configure a user group for root, admin, factory, switchAdmin, and user, and then add any users whose logins you want to associate to the appropriate group. 4.
Remote authentication 6 RSA RADIUS server Traditional password-based authentication methods are based on one-factor authentication, where you confirm your identity using a memorized password. Two-factor authentication increases the security by using a second factor to corroborate identification. The first factor is either a PIN or password and the second factor is the RSA SecurID token. RSA SecurID with an RSA RADIUS server is used for user authentication.
6 Remote authentication ####################################################################### # brocade.dct -- Brocade Dictionary # # (See readme.dct for more details on the format of this file) ####################################################################### # # Use the Radius specification attributes in lieu of the Brocade one: # @radius.
Remote authentication 6 e. Add the Brocade profile. f. In RSA Authentication Manager, edit the user records that will be authenticated using RSA SecurID. LDAP configuration and Microsoft Active Directory LDAP provides user authentication and authorization using the Microsoft Active Directory service or using OpenLDAP in conjunction with LDAP on the switch. This section discusses authentication and authorization using Microsoft Active Directory.
6 Remote authentication Configuring Microsoft Active Directory LDAP service The following is an overview of the process used to set up LDAP. 1. If your Windows Active Directory server for LDAP needs to be verified by the LDAP client (that is, the Brocade switch), then you must install a Certificate Authority (CA) certificate on the Windows Active Directory server for LDAP. Follow Microsoft instructions for generating and installing CA certificates on a Windows server. 2.
Remote authentication 6 Assigning the group (role) to the user To assign the user to a group in Active Directory, refer to www.microsoft.com or Microsoft documentation. If you have a user-defined group, use the ldapCfg -–maprole ldap_role_name switch_role command to map LDAP server permissions to one of the default roles available on a switch. Alternatively, update the memberOf field with the login permissions (root, admin, switchAdmin, user, and so on) that the user must use to log in to the switch.
6 Remote authentication Adding attributes to the Active Directory schema To create a group in Active Directory, refer to www.microsoft.com or Microsoft documentation. You must: • Add a new attribute brcdAdVfData as Unicode String. • Add brcdAdVfData to the person’s properties. LDAP configuration and OpenLDAP Fabric OS provides user authentication and authorization by means of OpenLDAP or the Microsoft Active Directory service in conjunction with LDAP on the switch.
Remote authentication 6 Enabling group membership Group membership in OpenLDAP is specified by an overlay called memberOf. Overlays are helpful in customizing the back-end behavior without requiring changes to the back-end code. The memberOf overlay updates the memberOf attribute whenever changes occur to the membership attribute of entries of the groupOfNames objectClass. To include this overlay, add “overlay memberof” to the slapd.conf file, as shown in the following example.
6 Remote authentication dn: cn=Manager,dc=mybrocade,dc=com objectClass: organizationalRole cn: Manager description: Directory Manager 2. Enter the ldapadd command to add the contents of the .ldif file to the Directory, where test.ldif is the file you created in step 1. > ldapadd -D cn=Manager,dc=mybrocade,dc=com -x -w secret -f test.ldif Assigning a user to a group Before you can assign a user to a group, the memberOf overlay must be added to the slapd.conf file.
Remote authentication 6 #changetype: modify #delete: memberof 2. Enter the following ldapmodify command, where test.ldif is the name of the file you edited in step 1. > ldapmodify -D cn=Sachin,dc=mybrocade,dc=com –x -w secret -f test.ldif Example to add a group member 1. Create or edit a .ldif file with an entry similar to the following. ##########Adding an attr value dn: cn=admin,ou=groups,dc=mybrocade,dc=com changetype: modify add: member member: cn=test1,cn=Users,dc=mybrocade,dc=com 2.
6 Remote authentication 1. In a schema file, assign the brcdAdVfData attribute to a user class. The following sample schema file defines a new objectClass named “user” with optional attributes “brcdAdVfData” and “description”. #New attr brcdAdVfData attributetype ( 1.3.6.1.4.1.8412.100 NAME ( 'brcdAdVfData' ) DESC 'Brocade specific data for LDAP authentication' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} ) objectclass ( 1.3.6.1.4.1.8412.
Remote authentication 6 objectClass: organizationalRole cn: Users description: User # User entries dn: cn=Sachin,cn=Users,dc=mybrocade,dc=com objectClass: user objectClass: person objectClass: uidObject cn: Sachin sn: Mishra description: First user brcdAdVfData: HomeLF=30;LFRoleList=admin:1-128;ChassisRole=admin userPassword: pass uid: mishras@mybrocade.com The following command adds the user to the LDAP directory. > ldapadd -D cn=Sachin,dc=mybrocade,dc=com -x -w secret -f test4.
6 Remote authentication Retry, the number of attempts to authenticate with a TACAS+ server, is also allowed. The default value is 5 attempts. If authentication is rejected or times out, Fabric OS will try again. The retry value can also be customized for each user. Refer to “Remote authentication configuration on the switch” on page 192 for details about configuring the Brocade switch for authenticating users with a TACACS+ server.
Remote authentication 6 brcd-role = securityAdmin; } } Configuring Admin Domain lists If your network uses Admin Domains, you should create Admin Domain lists for each user to identify the Admin Domains to which the user has access. Assign the following key-value pairs to the brcd-AV--Pair1 and, optionally, brcd-AV-Pair2 attributes to grant the account access to the Admin Domains: • HomeAD is the designated home Admin Domain for the account. The valid range of values is from 0 through 255.
6 Remote authentication Configuring the password expiration date FabricOS allows you to configure a password expiration date for each user account and to configure a warning period for notifying the user that the account password is about to expire. To configure these values, set the following attributes: • brcd-passwd-expiryDate sets the password expiration date in mm/dd/yyyy format. • brcd-passwd-warnPeriod sets the warning period as a number of days.
Remote authentication 6 Adding an authentication server to the switch configuration 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the aaaConfig --add command. At least one authentication server must be configured before you can enable the RADIUS, LDAP, or TACACS+ service. If no RADIUS, LDAP, or TACACS+ configuration exists, turning on the authentication mode triggers an error message.
6 Remote authentication Displaying the current authentication configuration 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the aaaConfig --show command. If a configuration exists, its parameters are displayed. If the RADIUS, LDAP, or TACACS+ service is not configured, only the parameter heading line is displayed. Parameters include: Position Server Port Secret Timeouts Authentication The order in which servers are contacted to provide service.
Chapter 7 Configuring Protocols In this chapter • Security protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Secure Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Secure Shell protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Secure Sockets Layer protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7 Secure Copy TABLE 26 Secure protocol support (Continued) Protocol Description SSH Secure Shell (SSH) is a network protocol that allows data to be exchanged over a secure channel between two computers. Encryption provides confidentiality and integrity of data. SSH uses public-key cryptography to authenticate the remote computer and allow the remote computer to authenticate the user, if necessary. SSL Fabric OS uses Secure Socket Layer (SSL) to support HTTPS.
Secure Shell protocol 7 Setting up SCP for configuration uploads and downloads Use the following procedure to configure SCP for configuration uploads and downloads. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter one of the following commands: • If Virtual Fabrics is enabled, enter the configurechassis command. • If Virtual Fabrics is not enabled, enter the configure command. 3. Enter y or yes at the cfgload attributes prompt. 4.
7 Secure Shell protocol SSH public key authentication OpenSSH public key authentication provides password-less logins, known as SSH authentication, that uses public and private key pairs for incoming and outgoing authentication. This feature allows only one allowed-user to be configured to utilize outgoing OpenSSH public key authentication.Any admin user can perform incoming Open SSH public key authentication.
Secure Shell protocol 7 Enter login name:auser Password: Public key is imported successfully. 4. Test the setup by logging in to the switch from a remote device, or by running a command remotely using SSH. Configuring outgoing SSH authentication After the allowed-user is configured, the remaining setup steps must be completed by the allowed-user. Use the following procedure to configure outgoing SSH authentication: 1. Log in to the switch as the default admin. 2.
7 Secure Sockets Layer protocol Deleting public keys on the switch Use the following procedure to delete public keys from the switch. 1. Connect to the switch and log in using an account with admin permissions. 2. Use the sshUtil delpubkeys command to delete public keys. You will be prompted to enter the name of the user whose the public keys you want to delete. Enter all to delete public keys for all users. For more information on IP filter policies, refer to Chapter 8, “Configuring Security Policies”.
Secure Sockets Layer protocol 7 You should upgrade to the Java 1.6.0 plug-in on your management workstation. To find the Java version that is currently running, open the Java console and look at the first line of the window. For more details on levels of browser and Java support, refer to the Web Tools Administrator’s Guide. SSL configuration overview You configure SSL access for a switch by obtaining, installing, and activating digital certificates.
7 Secure Sockets Layer protocol 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the secCertUtil genkey command to generate a public/private key pair. The system reports that this process will disable secure protocols, delete any existing CSR, and delete any existing certificates. 3. Respond to the prompts to continue and select the key size.
Secure Sockets Layer protocol 7 Obtaining certificates Once you have generated a CSR, you will need to follow the instructions on the website of the certificate issuing authority that you want to use; and then obtain the certificate. Fabric OS and HTTPS support the following types of files from the Certificate Authority(CA): • .cer (binary) • .crt (binary) • .pem (text) Typically, the CA provides the certificate files listed in Table 29.
7 Secure Sockets Layer protocol 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the secCertUtil import command. 3. Select a protocol, enter the IP address of the host on which the switch certificate is saved, and enter your login name and password. Example of installing a switch certificate in interactive mode switch:admin> seccertutil import -config swcert -enable https Select protocol [ftp or scp]: ftp Enter IP address: 192.10.11.
Secure Sockets Layer protocol 7 Checking and installing root certificates on Internet Explorer Use the following procedure to check and install a root security certificate on a switch using IE: 1. Select Tools > Internet Options. 2. Click the Content tab. 3. Click Certificates. 4. Click the Intermediate or Trusted Root tab and scroll the list to see if the root certificate is listed.
7 Simple Network Management Protocol 2. Open a Command Prompt window and change the directory to the Java plugin bin directory. 3. Enter the keyTool command and respond to the prompts. Example of installing a root certificate C:\Program Files\Java\j2re1.6.0\bin> keytool -import -alias RootCert -file RootCert.crt -keystore ..\lib\security\RootCerts Enter keystore password: changeit Owner: CN=Brocade, OU=Software, O=Brocade Communications, L=San Jose, ST=California, C=US Issuer: CN=Brocade, OU=Software, O=B
7 Simple Network Management Protocol Management Information Base (MIB) SNMP Agents in the managed devices store the data about these devices in a database called Management Information Base (MIB). The MIB is a hierarchical database, which is structured on the standard specified in the RFC 2578 (Structure of Management Information Version 2 (SMIv2)). The MIB is a database of objects that can be used by a network management system to manage and monitor devices on the network.
7 Simple Network Management Protocol Understanding MIBs The management information base (MIB) is a database of monitored and managed information on a device, in this case a Brocade switch. The MIB structure can be represented by a tree hierarchy. The root splits into three main branches: International Organization for Standardization (ISO), Consultative Committee for International Telegraph and Telephone (CCITT), and joint ISO/CCITT.
Simple Network Management Protocol 7 Once loaded, the MAX-ACCESS provides access levels between the agent and management station. The access levels are as follows: • not accessible You cannot read or write to this variable. • read create Specifies a tabular object that can be read, modified, or created as a new row in a table. • read only - Public You can only monitor information. • read-write - Private You can read or modify this variable.
7 Simple Network Management Protocol There is some overlap in the functionality of these MIBs. If you enable both SW-MIB and FA-MIB traps, you could receive duplicate messages for the switch events that trigger the trap. You can also use these additional MIBs and their associated traps: HA-MIB; FICON-MIB; and SWEXTTRAP. In Fabric OS v6.4.0, you can use the snmpConfig--set mibCapability command to enable or disable all the MIBs.
Simple Network Management Protocol 7 The high availability trap (HA-TRAP) can be configured to send traps using the snmpConfig command. For more information on this command, refer to the Fabric OS Command Reference. SW traps There are fourteen specific traps defined in Brocade SW-TRAP. • swfault (no longer supported) • swSensorScn (no longer supported) • swFCPortScn This trap is generated by a port state change. • swEventTrap This trap is generated by any switch event reported to the system error log.
7 Simple Network Management Protocol • swDeviceStatusTrap This trap is sent whenever a device logs in or logs out. The Brocade trap (SW-TRAP) can be configured to send traps using the snmpConfig command. FICON traps • linkRNIDDeviceRegistration A device registered with the switch. • linkRNIDDeviceDeRegistration A device de-registered with the switch. • linkLIRRListenerAdded A listener for link failure incident is added. • linkLIRRListenerRemoved A listener for link failure incident is removed.
Simple Network Management Protocol 7 Brocade MIB files The Brocade MIB files are as follows: • • • • • • • • • • • • • • bd.mib bcCustomOperation.mib BRCD_REG.mib BRCD_TC.mib brcdfcip.mib CPQ_HOST.mib CPQ_RACK.mib FA.mib faext.mib FICON.mib fod.mib HA.mib IbmBladeCenter.mib SW.mib Standard MIBs Distribution of standard MIBs has been stopped from Fabric OS v6.4.0. Download the following MIBs from the http://www.oidview.
7 Simple Network Management Protocol • • • • • • • • • • • • • • RFC1213-MIB RFC-1215 RMON-MIB RSTP-MIB SNMP-COMMUNITY-MIB SNMP-FRAMEWORK-MIB SNMPv2-CONF SNMPv2-MIB SNMPv2-PARTY-MIB SNMPv2-SMI-MIB SNMPv2-TC SNMP-VIEW-BASED-ACM-MIB SNMP-USER-BASED-SM-MIB SNMP-TARGET-MIB MIB loading order Many MIBs use definitions that are defined in other MIBs. These definitions are listed in the IMPORTS section near the top of the MIB.
Simple Network Management Protocol TABLE 30 Brocade SNMP MIB dependencies MIB Name Dependencies FIBRE-CHANNEL-FE-MIB SNMPv2-SMI SNMPv2-TC SNMP-FRAMEWORK-MIB SNMPv2-CONF FCIP-MGMT-MIB SNMPv2-SMI SNMPv2-TC INET-ADDRESS-MIB FC-MGMT-MIB IF-MIB SNMPv2-CONF SNMP-FRAMEWORK-MIB ENTITY-MIB SNMPv2-SMI SNMPv2-TC SNMPv2-CONF SNMP-FRAMEWORK-MIB SW.mib SNMPv2-TC SNMPv2-SMI Brocade-TC Brocade-REG-MIB FCMGMT-MIB bd.mib SNMPv2-TC SNMPv2-SMI Brocade-TC Brocade-REG-MIB SW-MIB brcdfcip.
7 Simple Network Management Protocol Access Gateway and Brocade MIBs Table 31 shows the MIBs supported by Brocade Access Gateway. TABLE 31 . Access Gateway MIB support MIB name Description MIB-2 Supported in v5.2.1 and later releases. Entity-MIB Supported. HA-MIB Supported. SW-MIB Disabled in Access Gateway because the conventions are specific to fabric switches. In Fabric OS v6.4.0, swConnUnitPortExtensionTable is supported in Access Gateway mode. In Fabric OS v7.0.
Simple Network Management Protocol 7 Support for IPv6 addressing IPv6 addressing is supported in Fabric OS v5.3.0 and later releases. Support for Virtual Fabric Virtual Fabric is supported in Fabric OS v6.2.0 and later releases. When an SNMPv3 request arrives with a particular user name, it executes in the home Virtual Fabric. From the SNMP manager, all SNMPv3 requests must have a home Virtual Fabric that is specified in the contextName field.
7 Simple Network Management Protocol Configuring SNMP using CLI For information about Fabric OS commands for configuring SNMP, refer to the Fabric OS Command Reference. Configuring SNMP security level The following example sets the SNMP security level to 1 (authentication only). This setting allows all SNMPv1 users to perform GET and SET operations on MIBs, but creates an exception for SNMPv3 users that do not have authentication and privacy privileges (noAuthnoPriv).
Simple Network Management Protocol 7 2. Create the SNMPv3 user. switch:root> snmpconfig --set snmpv3 SNMP Informs Enabled (true, t, false, f): [false] t SNMPv3 user configuration(snmp user not configured in FOS user database will have physical AD and admin role as the default): User (rw): [snmpadmin1] Auth Protocol [MD5(1)/SHA(2)/noAuth(3)]: (1..3) [3] Priv Protocol [DES(1)/noPriv(2)/AES128(3)]): (1..
7 Simple Network Management Protocol User 2 User 3 User 4 User 5 User 6 Priv Protocol: noPriv Engine ID: 80:00:05:23:01:0a:23:34:21 (rw): snmpadmin2 Auth Protocol: MD5 Priv Protocol: DES Engine ID: 80:00:05:23:01:0a:23:34:1b (rw): snmpadmin3 Auth Protocol: noAuth Priv Protocol: noPriv Engine ID: 00:00:00:00:00:00:00:00:00 (ro): snmpuser1 Auth Protocol: noAuth Priv Protocol: noPriv Engine ID: 00:00:00:00:00:00:00:00:00 (ro): snmpuser2 Auth Protocol: noAuth Priv Protocol: noPriv Engine ID: 00:00:00:00:
Simple Network Management Protocol 7 Verify Priv Passwd: User (ro): [snmpuser1] Auth Protocol [MD5(1)/SHA(2)/noAuth(3)]: (1..3) [3] Priv Protocol [DES(1)/noPriv(2)/AES128(3)]): (1..3) [2] User (ro): [snmpuser2] Auth Protocol [MD5(1)/SHA(2)/noAuth(3)]: (1..3) [3] Priv Protocol [DES(1)/noPriv(2)/AES128(3)]): (1..3) [2] User (ro): [snmpuser3] Auth Protocol [MD5(1)/SHA(2)/noAuth(3)]: (1..3) [3] Priv Protocol [DES(1)/noPriv(2)/AES128(3)]): (1..
7 Simple Network Management Protocol Trap Port: 162 Trap User: snmpadmin2 Trap recipient Severity level: 2 Trap Entry 3: HCL0389U.corp.brocade.
Simple Network Management Protocol 7 To send the sw-fc-port-scn trap to the configured recipients: switch:root> snmpTraps --send -trap_name sw-fc-port-scn Number of traps sent : 1 To send the sw-fc-port-scn trap to the recipient 10.35.52.33: switch:root> snmpTraps --send -trap_name sw-fc-port-scn -ip_address 10.35.52.
7 Simple Network Management Protocol DesiredSeverity:None swFabricWatchTrap: NO DesiredSeverity:None swTrackChangesTrap: NO swIPv6ChangeTrap: NO swPmgrEventTrap: NO swFabricReconfigTrap: NO swFabricSegmentTrap: NO swExtTrap: NO [...] To enable the SW-MIB MIB only without changing the current trap configuration: switch:admin> snmpconfig --enable mibCapability -mib_name SW-MIB Operation succeeded switch:admin> snmpconfig --show mibCapability [...
Simple Network Management Protocol 7 SW-MIB: YES FA-MIB: YES FICON-MIB: YES HA-MIB: YES FCIP-MIB: YES ISCSI-MIB: YES IF-MIB: YES BD-MIB: YES SW-TRAP: YES swFault: YES swSensorScn: YES swFCPortScn: YES swEventTrap: YES DesiredSeverity:None swFabricWatchTrap: YES DesiredSeverity:None swTrackChangesTrap: YES swIPv6ChangeTrap: YES swPmgrEventTrap: YES swFabricReconfigTrap: YES swFabricSegmentTrap: YES swExtTrap: YES FA-TRAP: YES connUnitStatusChange: YES connUnitDeletedTrap: YES connUnitEventTrap: YES connUni
7 Telnet protocol sysDescr = Fibre Channel Switch sysLocation = End User Premise sysContact = Field Support authTraps = 0 (OFF) ***** Are you sure? (yes, y, no, n): [no] y 3. Set the security level. switch:admin> snmpconfig --set secLevel Select SNMP GET Security Level (0 = No security, 1 = Authentication only, 2 No Access): (0..3) [0] 2 Select SNMP SET Security Level (0 = No security, 1 = Authentication only, 2 No Access): (2..
Telnet protocol 7 Blocking Telnet If you create a new policy using commands with just one rule, all the missing rules have an implicit deny and you lose all IP access to the switch, including Telnet, SSH, and management ports. Use the following procedure to block Telnet access. 1. Connect to the switch and log in using an account with admin permissions. 2. Clone the default policy by typing the ipFilter --clone command. switch:admin> ipfilter --clone BlockTelnet -from default_ipv4 3.
7 Listener applications 1 2 3 4 5 6 7 8 any any any any any any any any tcp tcp tcp tcp udp udp tcp udp 22 23 80 443 161 123 600 - 1023 600 - 1023 permit permit permit permit permit permit permit permit Unblocking Telnet Use the following procedure to unblock Telnet access. 1. Connect to the switch through a serial port or SSH and log in as admin. 2. Enter the ipfilter --delete command. Refer to “Deleting a rule from an IP Filter policy” on page 259 for more information on deleting IP filter rules.
Ports and applications used by switches 7 Ports and applications used by switches If you are using the FC-FC Routing Service, be aware that the secModeEnable command is not supported. Table 34 lists the defaults for accessing hosts, devices, switches, and zones. TABLE 34 Access defaults Access default Hosts Any host can access the fabric by SNMP. Any host can Telnet to any switch in the fabric. Any host can establish an HTTP connection to any switch in the fabric.
7 230 Ports and applications used by switches Fabric OS Administrator’s Guide 53-1002920-02
Chapter 8 Configuring Security Policies In this chapter • ACL policies overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • ACL policy management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • FCS policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Device Connection Control policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • SCC Policies . . . . . .
8 ACL policy management Policies with the same state are grouped together in a Policy Set. Each switch has the following two sets: • Active policy set, which contains ACL policies being enforced by the switch. • Defined policy set, which contains a copy of all ACL policies on the switch. When a policy is activated, the defined policy either replaces the policy with the same name in the active set or becomes a new active policy.
ACL policy management 8 Displaying ACL policies You can view the active and defined policy sets at any time. Additionally, in a defined policy set, policies created in the same login session also appear but these policies are automatically deleted if the you log out without saving them. 1. Connect to the switch and log in using an account with admin permissions, or an account with O permission for the Security RBAC class of commands. 2. Type the secPolicyShow command.
8 ACL policy management Example of deleting an ACL policy switch:admin> secpolicydelete "DCC_POLICY_010" About to delete policy Finance_Policy. Are you sure (yes, y, no, n):[no] y Finance_Policy has been deleted. Adding a member to an existing ACL policy As soon as a policy has been activated, the aspect of the fabric managed by that policy is enforced. 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the Security RBAC class of commands.
FCS policies 8 Example of aborting unsaved changes switch:admin> secpolicyabort Unsaved data has been aborted. All changes since the last time the secPolicySave or secPolicyActivate commands were entered are aborted. FCS policies Fabric configuration server (FCS) policy in base Fabric OS may be performed on a local switch basis and may be performed on any switch in the fabric. The FCS policy is not present by default, but must be created.
8 FCS policies Table 38 shows the commands for switch operations for Primary FCS enforcement.
FCS policies 8 Example of creating an FCS policy The following example creates an FCS policy that allows a switch with domain ID 2 to become a primary FCS and domain ID 4 to become a backup FCS: switch:admin> secpolicycreate "FCS_POLICY", "2;4" FCS_POLICY has been created 3. To save or activate the new policy, enter either the secPolicySave or the secPolicyActivate command. Once the policy has been activated you can distribute the policy. NOTE FCS policy must be consistent across the fabric.
8 Device Connection Control policies FCS policy distribution The FCS policy can be automatically distributed using the fddCfg --fabwideset command or it can be manually distributed to the switches using the distribute -p command. Each switch that receives the FCS policy must be configured to receive the policy. To configure the switch to accept distribution of the FCS policy, refer to “Database distribution settings” on page 261. Database distributions may be initiated from only the Primary FCS switch.
Device Connection Control policies 8 Each device port can be bound to one or more switch ports; the same device ports and switch ports may be listed in multiple DCC policies. After a switch port is specified in a DCC policy, it permits connections only from designated device ports. Device ports that are not specified in any DCC policies are allowed to connect only to switch ports that are not specified in any DCC policies.
8 Device Connection Control policies The following methods of specifying an allowed connection are possible: • deviceportWWN;switchWWN (port or area number) • deviceportWWN;domainID (port or area number) • deviceportWWN;switchname (port or area number) 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the Security RBAC class of commands. 2. Enter the secPolicyCreate “DCC_POLICY_nnn” command.
Device Connection Control policies 8 Example of deleting stale DCC policies switch:admin> secpolicydelete ALL_STALE_DCC_POLICY About to clear all STALE DCC policies ARE YOU SURE (yes, y, no, n): [no] y DCC policy behavior with Fabric-Assigned PWWNs A DCC policy check is always performed for the physical port WWN of a device when the HBA has established that the device is attempting a normal FLOGI and has both a fabric-assigned port WWN (FA-PWWN) and a physical port WWN.
8 SCC Policies TABLE 42 DCC policy behavior when created manually with PWWN Configuration WWN seen on DCC policy list Behavior when DCC policy activates Behavior on portDisable and portEnable • • FA-PWWN has logged into the switch. DCC policy creation manually with physical PWWN of device. DCC policy activation. PWWN Traffic will not be disrupted. Ports will come up without security issues. DCC policy creation. manually with physical PWWN FA-PWWN has logged into the switch.
Authentication policy for fabric elements 8 Creating an SCC policy 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the Security RBAC class of commands. 2. Enter the secPolicyCreate “SCC_POLICY” command. 3. Save or activate the new policy by entering either the secPolicySave or the secPolicyActivate command. If neither of these commands is entered, the changes are lost when the session is logged out.
8 Authentication policy for fabric elements When configured, the secret key pair is used for authentication. Authentication occurs whenever there is a state change for the switch or port. The state change can be due to a switch reboot, a switch or port disable and enable, or the activation of a policy.
Authentication policy for fabric elements 8 ATTENTION A secret key pair has to be installed prior to changing the policy. For more information on setting up secret key pairs, refer to “Setting a secret key pair” on page 250. If you must disable authentication on a port that has in-flight encryption or compression configured, you must first disable in-flight encryption or compression on the port, and then disable authentication. Refer to Chapter 16, “In-flight Encryption and Compression,” for details.
8 Authentication policy for fabric elements either DH-CHAP secrets or PKI certificates depending on the protocol selected. Otherwise, ISLs will be segmented during next E-port bring-up. ARE YOU SURE (yes, y, no, n): [no] y Auth Policy is set to ACTIVE NOTE This authentication-policy change will not affect online EX_Ports. Re-authenticating E_Ports Use the authUtil --authinit command to re-initiate the authentication on selected ports.
Authentication policy for fabric elements 8 By default the devicepolicy is in the OFF state, which means the switch clears the security bit in the FLOGI (fabric login). The authUtil command provides an option to change the device policy mode to select PASSIVE policy, which means the switch responds to authentication from any device and does not initiate authentication to devices. When the policy is set to ON, the switch expects a FLOGI with the FC-SP bit set.
8 Authentication policy for fabric elements • Configupload and download will not be supported for the following AUTH attributes: auth type, hash type, group type. NOTE For information about how to use authentication with Access Gateway, refer to the Access Gateway Administrator’s Guide. Authentication protocols Use the authUtil command to perform the following tasks: • Display the current authentication parameters. • Select the authentication protocol used between switches.
Authentication policy for fabric elements 8 NOTE If you set the authentication protocol to DH-CHAP or FCAP, have not configured shared secrets or certificates, and authentication is checked (for example, you enable the switch), then switch authentication will fail. If the E_Port is to carry in-flight encrypted traffic, the authentication protocol must be set to DH-CHAP. You must also use the -g option to set the DH group value to group 4 or all groups.
8 Authentication policy for fabric elements Note about Access Gateway switches Because Domain ID and name are not supported for Access Gateway, secAuthSecret --show output for Access Gateway appears as follows: WWN DId Name ----------------------------------------------10:00:8C:7C:FF:03:9E:00 -1 Unknown 10:00:8C:7C:FF:03:9E:01 -1 Unknown 10:00:8C:7C:FF:0D:AF:01 -1 Unknown When setting and removing the secret for a switch or device on Access Gateway, only the WWN can be used. Setting a secret key pair 1.
Authentication policy for fabric elements 8 Re-enter peer secret: Enter local secret: Re-enter local secret: Enter WWN, Domain, or switch name (Leave blank when done): Are you done? (yes, y, no, n): [no] y Saving data to key store… Done. 3. Disable and enable the ports on a peer switch using the portDisable and portEnable commands. FCAP configuration overview Beginning with Fabric OS release 7.0.
8 Authentication policy for fabric elements 1. Log in to the switch using an account with admin permissions, or an account associated with the chassis role and having OM permissions for the PKI RBAC class of commands. 2. Enter the secCertUtil generate -fcapall -keysize command on the local switch. switch:admin> seccertutil generate -fcapall -keysize 1024 WARNING!!! About to create FCAP: ARE YOU SURE (yes, y, no, n): [no] y Installing Private Key and Csr... Switch key pair and CSR generated... 3.
IP Filter policy 8 1. Log in to the switch using an account with admin permissions, or an account associated with the chassis role and having OM permissions for the PKI RBAC class of commands. 2. Enter the secCertUtil import –fcapswcert command. switch:admin> seccertutil import -fcapswcert Select protocol [ftp or scp]: scp Enter IP address: 10.1.2.3 Enter remote directory: /myHome/jdoe/OPENSSL Enter certificate name (must have ".crt" or ".cer" ".pem" or ".psk" suffix):01.
8 IP Filter policy Fabric OS supports multiple IP Filter policies to be defined at the same time. Each IP Filter policy is identified by a name and has an associated type. Two IP Filter policy types, IPv4 and IPv6, exist to provide separate packet filtering for IPv4 and IPv6. It is not allowed to specify an IPv6 address in the IPv4 filter, or specify an IPv4 address in the IPv6 filter. There can be up to six different IP Filter policies defined for both types.
IP Filter policy 8 1. Log in to the switch using an account with admin permissions, or an account associated with the chassis role and having the O permission for the IPfilter RBAC class of commands. 2. Enter the ipFilter –-show command. Saving an IP Filter policy You can save one or all IP Filter policies persistently in the defined configuration. Only the CLI session that owns the updated temporary buffer may run this command. Modification to an active policy cannot be saved without being applied.
8 IP Filter policy • Destination Port: The destination port number or name, such as: Telnet, SSH, HTTP, HTTPS. • Protocol: The protocol type. Supported types are TCP or UDP. • Action: The filtering action taken by this rule, either Permit or Deny. A traffic type and destination IP can also be specified Source address For an IPv4 filter policy, the source address has to be a 32-bit IPv4 address in dot decimal notation. The group prefix has to be a CIDR block prefix representation. For example, 208.130.32.
IP Filter policy TABLE 45 8 Supported services (Continued) Service name Port number time 27 name 42 whois 43 domain 53 bootps 67 bootpc 68 tftp 69 http 80 kerberos 88 hostnames 101 sftp 115 ntp 123 snmp 161 snmp trap 162 https 443 ssmtp 465 exec 512 login 513 shell 514 uucp 540 biff 512 who 513 syslog 514 route 520 timed 525 kerberos4 750 Protocol TCP and UDP protocols are valid protocol selections. Fabric OS v6.2.
8 IP Filter policy Traffic type and destination IP The traffic type and destination IP elements allow an IP policy rule to specify filter enforcement for IP forwarding. The INPUT traffic type is the default and restricts rules to manage traffic on IP management interfaces, The FORWARD traffic type allows management of bidirectional traffic between the external management interface and the inband management interface. In this case, the destination IP element should also be specified.
IP Filter policy 8 first rule. If a match is found for the source address, destination port, and protocol, the corresponding action for this rule is taken, and the subsequent rules in this policy are ignored. If there is no match, then it is compared to the next rule in the policy. This process continues until the incoming packet is compared to all rules in the active policy. If none of the rules in the policy matches the incoming packet, the two implicit rules are matched to the incoming packet.
8 Policy database distribution IP Filter policy distribution The IP Filter policy is manually distributed by command. The distribution includes both active and defined IP Filter policies. All policies are combined as a single entity to be distributed and cannot be selectively distributed. However, you may choose the time at which to implement the policy for optimization purposes. If a distribution includes an active IP Filter policy, the receiving switches activate the same IP Filter policy automatically.
Policy database distribution 8 Table 48 on page 261 explains how the local database distribution settings and the fabric-wide consistency policy affect the local database when the switch is the target of a distribution command. TABLE 48 Interaction between fabric-wide consistency policy and distribution settings Distribution setting Fabric-wide consistency policy Absent (default) Tolerant Strict Reject Database is protected, it cannot be overwritten. May not match other databases in the fabric.
8 Policy database distribution Displaying the database distribution settings 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the FabricDistribution RBAC class of commands. 2. Enter the fddCfg --showall command.
Policy database distribution 8 Distributing the local ACL policies 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the FabricDistribution RBAC class of commands. 2. Enter the distribute -p command. Fabric-wide enforcement The fabric-wide consistency policy enforcement setting determines the distribution behavior when changes to a policy are activated.
8 Policy database distribution SCC DCC PWD FCS AUTH IPFILTER - accept accept accept accept accept accept Fabric Wide Consistency Policy:- "" Setting the fabric-wide consistency policy 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the FabricDistribution RBAC class of commands. 2. Enter the fddCfg --fabwideset command. Example shows how to set a strict SCC and tolerant DCC fabric-wide consistency policy.
Policy database distribution 8 Matching fabric-wide consistency policies This section describes the interaction between the databases with active SCC and DCC policies and combinations of fabric-wide consistency policy settings when fabrics are merged. For example: Fabric A with SCC:S;DCC (strict SCC and tolerant DCC) joins Fabric B with SCC:S;DCC (strict SCC and tolerant DCC), the fabrics can merge as long as the SCC policies match, including the order SCC:S;DCC and if both are set to strict.
8 Management interface security TABLE 52 Examples of strict fabric merges Fabric-wide consistency policy setting Strict/Tolerant Strict/Absent Expected behavior Fabric A Fabric B SCC:S;DCC:S SCC;DCC:S SCC;DCC:S SCC:S;DCC SCC:S;DCC SCC:S Ports connecting switches are disabled. SCC:S;DCC:S SCC:S DCC:S Strict/Strict SCC:S DCC:S Table 53 has a matrix of merging fabrics with tolerant and absent policies.
Management interface security 8 • Automated Key Management—Automates the process, as well as manages the periodic exchange and generation of new keys. Using the ipSecConfig command, you must configure multiple security policies for traffic flows on the Ethernet management interfaces based on IPv4 or IPv6 addresses, a range of IPv4 or IPv6 addresses, the type of application, port numbers, and protocols used (UDP/TCP/ICMP).
8 Management interface security Gateway-to-gateway tunnel In this scenario, neither endpoint of the IP connection implements IPsec, but the network nodes between them protect traffic for part of the way. Protection is transparent to the endpoints, and depends on ordinary routing to send packets through the tunnel endpoints for processing.
Management interface security 8 IPsec protocols IPsec ensures confidentiality, integrity, and authentication using the following protocols: • • Authentication Header (AH) Encapsulating Security Payload (ESP) IPsec protocols protect IP datagram integrity using hash message authentication codes (HMAC). Using hash algorithms with the contents of the IP datagram and a secret key, the IPsec protocols generate this HMAC and add it to the protocol header.
8 Management interface security In AH and ESP, hmac_md5 and hmac_sha1 are used as authentication algorithms. Only in ESP, 3des_cbc, blowfish_cbc, aes256_cbc and null_enc are used as encryption algorithms. Use Table 54 when configuring the authentication algorithm. TABLE 54 Algorithms and associated authentication policies Algorithm Encryption Level Policy Description hmac_md5 128-bit AH, ESP hmac_sha1 160-bit AH, ESP A stronger MAC because it is a keyed hash inside a keyed hash.
Management interface security 8 IKE policies When IKE is used as the key management protocol, IKE policy defines the parameters used in IKE negotiations needed to establish IKE SA and parameters used in negotiations to establish IPsec SAs. These include the authentication and encryption algorithms, and the primary authentication method, such as preshared keys, or a certificate-based method, such as RSA signatures.
8 Management interface security Static Security Associations Manual Key Entry (MKE) provides the ability to manually add, delete and flush SA entries in the SADB. Manual SA entries may not have an associated IPsec policy in the local policy database. Manual SA entries are persistent across system reboots. Creating the tunnel Each side of the tunnel must be configured in order for the tunnel to come up.
Management interface security 8 8. Create an IPsec transform on each switch using the ipSecConfig --add command. Example of creating an IPsec transform This example creates an IPsec transform TRANSFORM01 to use the transport mode to protect traffic identified for IPsec protection and use IKE01 as key management policy. switch:admin> ipsecconfig --add policy ips transform –t TRANSFORM01 -mode transport -sa-proposal IPSEC-AH -action protect –ike IKE01 9.
8 Management interface security Example of an end-to-end transport tunnel mode This example illustrates securing traffic between two systems using AH protection with MD5 and configure IKE with pre-shared keys. The two systems are a switch, BROCADE300 (IPv4 address 10.33.74.13), and an external host (10.33.69.132). 1. On the system console, log in to the switch as Admin. 2. Enable IPsec. a.
Management interface security 8 11. Perform the equivalent steps on the remote peer to complete the IPsec configuration. Refer to your server administration guide for instructions. 12. Generate IP traffic and verify that it is protected using defined policies. a. Initiate Telnet or SSH or ping session from BRCD300 to Remote Host. b. Verify that the IP traffic is encapsulated. c. Monitor IPsec SAs created using IKE for the above traffic flow.
8 276 Management interface security Fabric OS Administrator’s Guide 53-1002920-02
Chapter 9 Maintaining the Switch Configuration File In this chapter • Configuration settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Configuration file backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Configuration file restoration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Configurations across a fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9 Configuration settings If your user account has chassis account permissions, you can use any of the following options when uploading or downloading a configuration file: -fid To upload the specified FID configuration. -all To upload all of the system configuration, including the chassis section and all switch sections for all logical switches. NOTE: Use this parameter when obtaining a complete capture of the switch configuration in a switch that has Virtual Fabrics mode disabled.
Configuration file backup 9 The switch section of the configuration file contains information for all of the following: • • • • • • • • • • • • • • Boot parameters Configuration Bottleneck configuration Flow Vision configuration FCoE software configuration Zoning Defined security policies Active security policies iSCSI CryptoDev FICU saved files VS_SW_CONF MAPS configuration Banner Configuration file backup Brocade recommends keeping a backup configuration file.
9 Configuration file restoration 4. Store a soft copy of the switch configuration information in a safe place for future reference. Example of configUpload on a switch without Admin Domains switch:admin> configupload Protocol (scp, ftp, sftp, local) [ftp]: sftp Server Name or IP Address [host]: 10.1.2.3 User Name [user]: UserFoo Path/Filename [/config.txt]: switchConfig.txt Section (all|chassis|FID# [all]): chassis username@10.1.2.
Configuration file restoration 9 Restrictions This section lists restrictions for some of the options of the configDownload command. -chassis The number of switches defined in the downloaded configuration file must match the number of switches currently defined on the switch. -fid FID The FID must be defined in both the downloaded configuration file and the current system. NOTE Brocade recommends you disable a switch before downloading a configuration file.
9 Configuration file restoration TABLE 55 CLI commands to display or modify switch configuration information (Continued) Command Displays fcrXlateConfig Translate (xlate) domain's domain ID for both EX_Port-attached fabric and backbone fabric. fosConfig Fabric OS features. ipAddrShow IP address. isnscCfg Configuration state of the iSNS client operation. licenseShow License keys installed with more detail than the license information from the configShow command.
Configuration file restoration 9 4. Enter the configDownload command. The command becomes interactive and you are prompted for the required information. 5. At the “Do you want to continue [y/n]” prompt, enter y. Wait for the configuration to be restored. 6. If you disabled the switch, enter the switchEnable command when the process is finished. NOTE Always perform a reboot after you download a configuration file. On dual-CP platforms, you must reboot both CPs simultaneously.
9 Configurations across a fabric may cause this switch to fail. A switch reboot is required for the changes to take effect. Please make sure all the switches are disabled by using "chassisdisable" command. Downloading configuration to an online switch may result in some configuration not being downloaded to that switch. configDownload operation may take several minutes to complete for large files.
Configuration management for Virtual Fabrics 9 Configuration management for Virtual Fabrics You can use the configUpload -vf or configDownload -vf command to restore configurations to a logical switch. The -vf option only restores the Virtual Fabrics configuration information on to a switch of the same model and same release. For example, a Virtual Fabrics configuration file for Fabric OS 7.2.x cannot be used on a Fabric OS 7.1.x switch and vice versa.
9 Configuration management for Virtual Fabrics CAUTION You must issue the configDownload command on the switch after restoring the Virtual Fabrics configuration to fully restore your switch or chassis configuration. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the configDownload -vf command. 3. Respond to the prompts. Wait for the configuration file to download on to the switch. You may need to reconnect to the switch. 4. Enter the configDownload command. 5.
Brocade configuration form 9 All of the attributes of the Virtual Fabrics configuration file will be downloaded to the system and take effect. This includes, but is not limited to, logical switch definitions, whether Virtual Fabrics is enabled or disabled, and the F_Port trunking ports, except the LISL ports. The LISL ports on the system are not affected by the Virtual Fabrics configuration file download. You can restore Virtual Fabrics configurations only to a switch of the same model and same release.
9 288 Brocade configuration form Fabric OS Administrator’s Guide 53-1002920-02
Chapter 10 Installing and Maintaining Firmware In this chapter • Firmware download process overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Preparing for a firmware download . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Firmware download on switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Firmware download on a Backbone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Firmware download from a USB device . . . . . . . . . . .
10 Firmware download process overview You can download Fabric OS to a Backbone, which is a chassis; and to a nonchassis-based system, also referred to as a fixed-port switch. The difference in the download process is that Backbones have two CPs and fixed-port switches have one CP. Use the firmwareDownload command to download the firmware from either an FTP or SSH server by using FTP, SFTP, or SCP to the switch. Or you can use a Brocade-branded USB device.
Firmware download process overview 10 Upgrading and downgrading firmware Upgrading means installing a newer version of firmware. Downgrading means installing an older version of firmware. In most cases, you will be upgrading firmware; that is, installing a newer firmware version than the one you are currently running. However, some circumstances may require installing an older version; that is, downgrading the firmware.
10 Preparing for a firmware download TABLE 57 Backbone HA sync states Active CP Fabric OS version Standby CP Fabric OS version HA sync state Remedy v6.4.0 v6.4.0 inSync Run firmwareDownload -s on the standby CP to upgrade it to v7.0.0 v6.4.0 v7.1.0 Not inSync N/A v7.0.0 v6.4.0 inSync Run firmwareDownload -s on the standby CP to upgrade it to v7.0.0 v7.0.0 v7.0.0 inSync N/A v7.0.0 v7.1.0 inSync N/A v7.0.0 v7.2.0 Not inSync N/A v7.1.0 v6.4.0 Not inSync N/A v7.1.0 v7.0.
Preparing for a firmware download 10 5. Connect to the switch and log in using an account with admin permissions. Enter the supportSave command to retrieve all current core files prior to executing the firmware download. This information helps to troubleshoot the firmware download process if a problem is encountered. 6. Optional: Enter the errClear command to erase all existing messages in addition to internal messages.
10 Firmware download on switches Firmware download on switches Brocade fixed-port switches maintain primary and secondary partitions for firmware. The firmwareDownload command defaults to an autocommit option that automatically copies the firmware from one partition to the other. NOTE This section only applies when upgrading from Fabric OS v7.1.x to v7.2.0, downgrading from v7.2.0 to v7.1.x, or going from v7.2.x to v7.2.x If you are upgrading from Fabric OS v7.0.x to v7.2.0 or downgrading from v7.2.
Firmware download on switches 10 Upgrading firmware for Brocade fixed-port switches 1. Take the following appropriate action based on what service you are using: • If you are using FTP, SFTP, or SCP, verify that the FTP or SSH server is running on the host server and that you have a valid user ID and password on that server. • If your platform supports a USB memory device, verify that it is connected and running. 2. Obtain the firmware file from the Brocade website at http://www.brocade.
10 Firmware download on a Backbone Firmware download on a Backbone ATTENTION To successfully download firmware, you must have an active Ethernet connection on each CP. You can download firmware to a Backbone without disrupting the overall fabric if the two CP blades are installed and fully synchronized. Use the haShow command to verify that the CPs are synchronized prior to beginning the firmware download process.
Firmware download on a Backbone 10 Upgrading firmware on Backbones (including blades) There is only one chassis management IP address for the Brocade Backbones. NOTE By default, the firmwareDownload command automatically upgrades both the active and the standby CPs and all co-CPs on the CP blades in the Brocade Backbones. It automatically upgrades all AP blades in the Brocade Backbones using autoleveling. 1. Verify that the Ethernet interfaces located on CP0 and CP1 are plugged into your network. 2.
10 Firmware download on a Backbone If an AP blade is present: At the point of the failover, an autoleveling process is activated. Autoleveling is triggered when the active CP detects a blade that contains a different version of the firmware, regardless of which version is older. Autoleveling downloads firmware to the AP blade, swaps partitions, reboots the blade, and copies the new firmware from the primary partition to the secondary partition.
Firmware download from a USB device 10 Slot 7 (CP1, active): Firmware has been downloaded to the secondary partition of the switch. [5]: Mon Jul 22 04:37:24 2013 Slot 7 (CP1, standby): The firmware commit operation has started. This may take up to 10 minutes. [6]: Mon Jul 22 04:41:59 2013 Slot 7 (CP1, standby): The commit operation has completed successfully. [7]: Mon Jul 22 04:41:59 2013 Slot 7 (CP1, standby): Firmwaredownload command has completed successfully.
10 FIPS support Downloading from the USB device using the relative path 1. Log in to the switch using an account assigned to the admin role. 2. Enter the firmwareDownload -U command. ecp:admin>firmwaredownload –U v7.2.0 Downloading from the USB device using the absolute path 1. Log in to the switch using an account assigned to the admin role. 2. Enter the firmwareDownload command with the -U operand. ecp:admin>firmwaredownload –U /usb/usbstorage/brocade/firmware/v7.2.
FIPS support 10 NOTE If FIPS mode is enabled, all logins should be handled through SSH or direct serial method, and the transfer protocol should be SCP. Updating the firmware key 1. Log in to the switch as admin. 2. Enter the firmwareKeyUpdate command and respond to the prompts. The firmwareDownload command The public key file must be packaged, installed, and run on your switch before you download a signed firmware.
10 Testing and restoring firmware on switches Power-on firmware checksum test FIPS requires the checksums of the executables and libraries on the filesystem to be validated before Fabric OS modules are launched. This is to make sure these files have not been changed after they are installed. When firmware RPM packages are installed during firmware download, the MD5 checksums of the firmware files are stored in the RPM database on the filesystem.
Testing and restoring firmware on switches 10 User Name: userfoo File Name: /home/userfoo/v7.2.0 Password: Do Auto-Commit after Reboot [Y]: n Reboot system after download [N]: y Firmware is being downloaded to the switch. This step may take up to 30 minutes. Checking system settings for firmwaredownload... The switch performs a reboot and comes up with the new firmware to be tested. Your current switch session automatically disconnects.
10 Testing and restoring firmware on Backbones Testing and restoring firmware on Backbones This procedure enables you to perform a firmware download on each CP and verify that the procedure was successful before committing to the new firmware. The old firmware is saved in the secondary partition of each CP until you enter the firmwareCommit command.
Testing and restoring firmware on Backbones 10 8. Verify the failover. a. Connect to the Backbone on the active CP, which is the former standby CP. b. Enter the haShow command to verify that the HA synchronization is complete. It takes a minute or two for the standby CP, which is the old active CP, to reboot and synchronize with the active CP.
10 Validating a firmware download ATTENTION Stop! If you have completed step 11, then you have committed the firmware on both CPs and you have completed the firmware download procedure. 12. Restore the firmware on the standby CP. In the current Backbone session for the standby CP, enter the firmwareRestore command. The standby CP reboots and the current Backbone session ends. Both partitions have the same Fabric OS after several minutes. 13. Perform haFailover on the active CP. a.
Validating a firmware download TABLE 58 10 Commands used for validating a firmware download Command Description firmwareShow Displays the current firmware level on the switch. For Brocade Backbones, this command displays the firmware loaded on both partitions (primary and secondary) for both CPs and AP blades. Brocade recommends that you maintain the same firmware level on both partitions of each CP within the Brocade Backbone. The firmwareShow command displays the firmware version on each CP.
10 308 Validating a firmware download Fabric OS Administrator’s Guide 53-1002920-02
Chapter 11 Managing Virtual Fabrics In this chapter • Virtual Fabrics overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Logical switch overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Logical fabric overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Management model for logical switches . . . . . . . . . . . . . . . . . . . . . . . . . . .
11 Logical switch overview This chapter describes the logical switch and logical fabric features. For information about device sharing with Virtual Fabrics, refer to “FC-FC routing and Virtual Fabrics” on page 636. For information about supported switches and port types, refer to “Supported platforms for Virtual Fabrics” on page 320. Virtual Fabrics and Admin Domains are mutually exclusive and are not supported at the same time on a switch.
Logical switch overview 11 After you enable Virtual Fabrics, you can create up to seven additional logical switches, depending on the switch model. Figure 22 shows a Virtual Fabrics-enabled switch before and after it is divided into logical switches. Before you create logical switches, the chassis appears as a single switch (default logical switch). After you create logical switches, the chassis appears as multiple independent logical switches.
11 Logical switch overview Physical chassis Logical switch 1 (Default logical switch) (FID = 128) Logical switch 2 (FID = 1) Logical switch 3 (FID = 15) Logical switch 4 (FID = 8) Logical switch 5 (FID = 20) FIGURE 23 Fabric IDs assigned to logical switches Port assignment in logical switches Initially, all ports belong to the default logical switch. When you create additional logical switches, they are empty and you must assign ports to those logical switches.
Logical switch overview 11 A given port is always in one (and only one) logical switch. The following scenarios refer to the chassis after port assignment in Figure 24: • If you assign P2 to logical switch 2, you cannot assign P2 to any other logical switch. • If you want to remove a port from a logical switch, you cannot delete it from the logical switch, but must move it to a different logical switch.
11 Management model for logical switches Physical chassis Logical switch 1 P1 (Default logical switch) Fabric ID 128 Logical switch 2 Fabric ID 1 H1 P2 P3 D1 P4 Logical switch 3 Fabric ID 15 Logical switch 4 Fabric ID 8 P5 P6 D2 ISL Switch FIGURE 25 Logical switches connected to devices and non-Virtual Fabrics switch Figure 26 shows a logical representation of the physical chassis and devices in Figure 25. As shown in Figure 26, the devices are isolated into separate fabrics.
Logical fabric overview 11 All user operations are classified into one of the following: • Chassis management operations These are operations that span logical switch boundaries, such as: - Logical switch configuration (creating, deleting, or modifying logical switches) Account management (determining which accounts can access which logical switches) Field-replaceable unit (FRU) management (slot commands, such as slotShow) Firmware management (firmware upgrade, HA failover) • Logical switch operations
11 Logical fabric overview Physical chassis 2 Physical chassis 1 P1 Logical switch 1 (Default logical switch) Fabric ID 128 P1 P2 P2 P3 P3 P4 P5 Logical switch 2 Fabric ID 1 Logical switch 3 Fabric ID 15 Logical switch 4 Fabric ID 8 P5 P6 P6 P8 Logical switch 5 (Default logical switch) Fabric ID 128 Logical switch 6 Fabric ID 1 P4 P7 Logical switch 7 Fabric ID 15 Logical switch 8 Fabric ID 8 P9 Switch FIGURE 27 Logical switches connected to other logical switches through physical I
Logical fabric overview 11 When you divide a chassis into logical switches, you can designate one of the switches to be a base switch. A base switch is a special logical switch that is used for interconnecting the physical chassis. A base switch has the following properties: • ISLs connected through the base switch can be used for communication among the other logical switches. • Base switches do not support direct device connectivity.
11 Logical fabric overview FIGURE 30 Logical ISLs connecting logical switches To use the XISL, the logical switches must be configured to allow XISL use. By default, they are configured to do so; you can change this setting, however, using the procedure described in “Configuring a logical switch to use XISLs” on page 333. NOTE It is a good practice to configure at least two XISLs, for redundancy. You can also connect logical switches using a combination of ISLs and XISLs, as shown in Figure 31.
Account management and Virtual Fabrics 11 ATTENTION If you disable a base switch, all of the logical ISLs are broken and the logical switches cannot communicate with each other unless they are connected by a physical ISL. Base fabric Base switch ports on different chassis can be connected together to form a fabric, called a base fabric. Similar to other logical switches, the base switches must have the same FID to be connected.
11 Supported platforms for Virtual Fabrics When you are logged in to a logical switch, the system prompt changes to display the FID of that switch. The following are example prompts for when you are logged in to the default logical switch (FID = 128) and a user-defined logical switch (FID = 15): switch:FID128:admin> switch:FID15:admin> Refer to Chapter 6, “Managing User Accounts,” for information about creating user accounts and assigning FIDs to user accounts.
Supported platforms for Virtual Fabrics 11 Supported port configurations in Brocade Backbones Some of the ports in the Brocade DCX and DCX 8510 Backbone families are not supported on all types of logical switches. Table 59 lists the blades and ports that are supported on each type of logical switch.
11 Limitations and restrictions of Virtual Fabrics Virtual Fabrics interaction with other Fabric OS features Table 60 lists some Fabric OS features and considerations that apply when using Virtual Fabrics. TABLE 60 Virtual Fabrics interaction with Fabric OS features Fabric OS feature Virtual Fabrics interaction Access Gateway Virtual Fabrics is not supported on a switch if AG mode is enabled.
Limitations and restrictions of Virtual Fabrics 11 The maximum number of logical switches per chassis varies depending on the switch model. Table 61 lists the supported platforms and the maximum number of logical switches (including the default logical switch) supported on each.
11 Enabling Virtual Fabrics mode Restrictions on moving ports The following are restrictions on moving ports among logical switches: • FC ports cannot be moved if any one of the following features is enabled: - Long distance - QoS - F_Port buffers - F_Port trunking • Before moving VE_Ports, you must remove the VE_Port tunnel configuration. • VE_Ports on the FX8-24 blade can be moved to any logical switch independent of the location of the physical GE port.
Disabling Virtual Fabrics mode 11 switch:admin> fosconfig --enable vf WARNING: This is a disruptive operation that requires a reboot to take effect. All EX ports will be disabled upon reboot. Would you like to continue [Y/N] y VF has been enabled. Your system is being rebooted. Disabling Virtual Fabrics mode When you disable VF mode, the following occurs: • The CPs are rebooted. • If F_Port trunking is enabled on ports in the default switch, the F_Port trunking information is deleted.
11 Configuring logical switches to use basic configuration values Configuring logical switches to use basic configuration values All switches in the fabric are configured to use the same basic configuration values. When you create logical switches, the logical switches might have different configuration values than the default logical switch. Use the following procedure to ensure that newly created logical switches have the same basic configuration values as the default logical switch.
Creating a logical switch or base switch 11 In the command syntax, fabricID is the fabric ID that is to be associated with the logical switch. Specify the -base option if the logical switch is to be a base switch. Specify the -force option to execute the command without any user prompts or confirmation. 3. Set the context to the new logical switch. setcontext fabricID (or switchname) The fabricID parameter is the FID of the logical switch you just created.
11 Executing a command in a different logical switch context Executing a command in a different logical switch context This procedure describes how to execute a command for a logical switch while you are in the context of a different logical switch. You can also execute a command for all the logical switches in a chassis. The command is not executed on those logical switches for which you do not have permission. Use the following procedure to execute a command in a different logical switch context: 1.
Deleting a logical switch 11 Switch ID Worldwide Name Enet IP Addr FC IP Addr Name ------------------------------------------------------------------------14: fffc0e 10:00:00:05:1e:82:3c:2b 10.32.79.105 0.0.0.0 >"switch_4" (output truncated) Deleting a logical switch The following rules apply to deleting a logical switch: • You must remove all ports from the logical switch before deleting it. • You cannot delete the default logical switch.
11 Displaying logical switch configuration • If you are deploying ICLs in the base switch, all ports associated with those ICLs must be assigned to the base switch. If you are deploying ICLs to connect to default switches (that is, XISL use is not allowed), the ICL ports should be assigned (or left) in the default logical switch. Use the following procedure to add or move ports on a logical switch: 1. Connect to the physical chassis and log in using an account with the chassis-role permission. 2.
Changing the fabric ID of a logical switch 11 Changing the fabric ID of a logical switch The following procedure describes how you can change the fabric ID of an existing logical switch. The fabric ID indicates in which fabric the logical switch participates. By changing the fabric ID, you are moving the logical switch from one fabric to another. Changing the fabric ID requires permission for chassis management operations. You cannot change the FID of your own logical switch context.
11 Changing a logical switch to a base switch 4. Enter the lsCfg command to change the logical switch to a base switch: lscfg --change fabricID -base The fabricID parameter is the fabric ID of the logical switch with the attributes you want to change. 5. Enable the switch. switchenable Example of changing the logical switch with FID 7 to a base switch sw0:FID128:admin> setcontext 7 switch_25:FID7:admin> switchshow switchName: switch_25 switchType: 66.
Setting up IP addresses for a logical switch 11 Setting up IP addresses for a logical switch Each physical chassis has one common IP address that is shared by all of the logical switches in the chassis. You can also set up individual IPv4 addresses for each logical switch. IPv4 addresses assigned to individual Virtual Fabrics are assigned to IP over Fibre Channel (IPFC) network interfaces.
11 Changing the context to a different logical fabric setcontext fabricID (or switchname) The fabricID parameter is the FID of the logical switch you want to switch to and manage. The switchname parameter is the name assigned to the logical switch. You can only use one parameter at a time. 3. Use the switchShow command and check the value of the Allow XISL Use parameter. 4. Enter the configure command: configure 5. Enter y after the Fabric Parameters prompt: Fabric parameters (yes, y, no, n): [no] y 6.
Creating a logical fabric using XISLs FIGURE 32 11 Example of logical fabrics in multiple chassis and XISLs Use the following procedure to create a logical fabric using XISLs: 1. Set up the base switches in each chassis: a. Connect to the physical chassis and log in using an account with the chassis-role permission. b. Enable the Virtual Fabrics feature, if it is not already enabled. See “Enabling Virtual Fabrics mode” on page 324 for instructions.
11 Creating a logical fabric using XISLs For the example shown in Figure 32, you would create a logical switch with FID 1 and a logical switch with FID 15. c. Assign ports to the logical switch, as described in “Adding and moving ports on a logical switch” on page 329. d. Physically connect devices and ISLs to these ports on the logical switch. e. (Optional) Configure the logical switch to use XISLs, if it is not already XISL-capable.
Chapter 12 Administering Advanced Zoning In this chapter • Zone types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Zoning overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Broadcast zones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Zone aliases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12 Zoning overview • QoS zones Assign high or low priority to designated traffic flows. QoS zones are regular zones with additional QoS attributes specified by adding a QoS prefix to the zone name. Refer to “QoS” on page 415 for more information. • Traffic Isolation zones (TI zones) Isolate traffic to a specific, dedicated path through the fabric. Refer to Chapter 13, “Traffic Isolation Zoning,” for more information.
Zoning overview 12 JBOD Loop 2 Server2 Blue zone Fibre Channel Fabric RAID Hub Server1 Loop 1 Red zone FIGURE 33 Server3 Green zone Zoning example Approaches to zoning Table 62 lists the various approaches you can take when implementing zoning in a fabric. TABLE 62 Approaches to fabric-based zoning Zoning approach Description Recommended approach Single HBA Zoning by single HBA most closely re-creates the original SCSI bus.
12 Zoning overview TABLE 62 Approaches to fabric-based zoning (Continued) Zoning approach Description Operating system Zoning by operating system has issues similar to zoning by application. In a large site, this type of zone can become very large and complex. When zone changes are made, they typically involve applications rather than a particular server type.
Zoning overview 12 Zoning schemes You can establish a zone by identifying zone objects using one or more of the following zoning schemes: • Domain,index (D,I) All members are specified by domain ID, port number, or domain, index number pairs or aliases. • World Wide Name (WWN) All members are specified only by World Wide Names (WWNs) or aliases of WWNs. They can be node or port versions of the WWN.
12 Zoning overview Zoning enforcement Zoning enforcement describes a set of predefined rules that the switch uses to determine where to send incoming data. Fabric OS uses hardware-enforced zoning. Hardware-enforced zoning means that each frame is checked by hardware (the ASIC) before it is delivered to a zone member and is discarded if there is a zone mismatch.
Broadcast zones TABLE 63 12 Considerations for zoning architecture (Continued) Item Description Effect of changes in a production fabric Zone changes in a production fabric can result in a disruption of I/O under conditions when an RSCN is issued because of the zone change and the HBA is unable to process the RSCN fast enough. Although RSCNs are a normal part of a functioning SAN, the pause in I/O might not be acceptable.
12 Broadcast zones If there are no broadcast zones or if a broadcast zone is defined but not enabled, broadcast frames are not forwarded to any F_Ports. If a broadcast zone is enabled, broadcast frames are delivered only to those logged-in Nx_Ports that are members of the broadcast zone and are also in the same zone (regular zone) as the sender of the broadcast packet. Devices that are not members of the broadcast zone can send broadcast packets, even though they cannot receive them.
Broadcast zones 12 "3,1" "1,1" "4,1" "2,1" AD1 AD2 broadcast "2,1; 3,1; 4,1" broadcast "1,1; 3,1; 5,1" "5,1" "1,1" "3,1; 4,1" broadcast "1,1; 3,1; 4,1" FIGURE 34 Broadcast zones and Admin Domains The dotted box represents the consolidated broadcast zone, which contains all of the devices that can receive broadcast packets. The actual delivery of broadcast packets is also controlled by the Admin Domain and zone enforcement logic.
12 Zone aliases High availability considerations with broadcast zones If a switch has broadcast zone-capable firmware on the active CP (Fabric OS v5.3.x or later) and broadcast zone-incapable firmware on the standby CP (Fabric OS version earlier than v5.3.0), then you cannot create a broadcast zone because the zoning behavior would not be the same across an HA failover. If the switch failed over, then the broadcast zone would lose its special significance and would be treated as a regular zone.
Zone aliases 12 Zone configuration naming is flexible. One configuration should be named PROD_fabricname, where fabricname is the name that the fabric has been assigned. The purpose of the PROD configuration is to easily identify the configuration that can be implemented and provide the most generic services. If other configurations are used for specialized purposes, names such as “BACKUP_A,” “RECOVERY_2,” and “TEST_18jun02” can be used.
12 Zone aliases 3. Enter the cfgSave command to save the change to the defined configuration. The cfgSave command ends and commits the current zoning transaction buffer to nonvolatile memory. If a transaction is open on a different switch in the fabric when this command is run, the transaction on the other switch is automatically aborted. A message displays on the other switches to indicate that the transaction was aborted.
Zone aliases 12 Deleting an alias Use the following procedure to delete an alias. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the aliDelete command, using the following syntax. alidelete "aliasname" 3. Enter the cfgSave command to save the change to the defined configuration. The cfgSave command ends and commits the current zoning transaction buffer to nonvolatile memory.
12 Zone creation and maintenance Zone creation and maintenance Fabric OS allows you to create zones to better manage devices. NOTE Broadcast Zone: To create a broadcast zone, use the reserved name “broadcast”. Do not give a regular zone the name of “broadcast”. Refer to “Broadcast zones” on page 343 for additional information about this special type of zone. NOTE Virtual Fabrics considerations: Zone definitions should not include logical port numbers. Zoning is not enforced on logical ports.
Zone creation and maintenance 12 NOTE The zoneCreate command supports partial pattern matching (“wildcards”) of zone member aliases. This allows you to add multiple aliases that match the “aliasname_pattern” in the command line. To create a broadcast zone, use the reserved name “broadcast”. 3. Enter the cfgSave command to save the change to the defined configuration. The cfgSave command ends and commits the current zoning transaction buffer to nonvolatile memory.
12 Zone creation and maintenance The cfgSave command ends and commits the current zoning transaction buffer to nonvolatile memory. If a transaction is open on a different switch in the fabric when this command is run, the transaction on the other switch is automatically aborted. A message displays on the other switches to indicate that the transaction was aborted. 4. Enter the cfgShow command to view the changes.
Zone creation and maintenance 12 zone: matt zeus; bond; jake; jeff; jones; 3,2; 30:06:00:07:1e:a2:10:20 zone: sloth bawn; bolt; bond; brain; 10:00:00:00:01:1e:20:20 alias: bawn 3,5; 4,8 alias: bolt 10:00:00:02:1f:02:00:01 alias: bond 10:00:05:1e:a9:20:00:01; 3,5 alias: brain 11,4; 22,1; 33,6 alias: jake 4,7; 8,9; 14,11 alias: jeff 30:00:00:05:1e:a1:cd:02; 40:00:00:05:1e:a1:cd:04 alias: jones 7,3; 4,5 alias: zeus 4,7; 6,8; 9,2 Effective configuration: No Effective configuration: (No Access) switch:admin> s
12 Zone creation and maintenance NOTE The zoneObjectReplace command does not support partial pattern matching (“wildcards”) of zone member aliases. 3. Enter the cfgSave command to save the change to the defined configuration. The cfgSave command ends and commits the current zoning transaction buffer to nonvolatile memory. If a transaction is open on a different switch in the fabric when this command is run, the transaction on the other switch is automatically aborted.
Zone creation and maintenance 12 Deleting a zone Use the following procedure to delete a zone. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the zoneDelete command, using the following syntax: zonedelete "zonename" 3. Enter the cfgSave command to save the change to the defined configuration. The cfgSave command ends and commits the current zoning transaction buffer to nonvolatile memory.
12 Zone creation and maintenance Viewing a zone in the defined configuration Use the following procedure to view a zone in the configuration. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the zoneShow command, using the following syntax: zoneshow[--sort] ["pattern"] [, mode] If no parameters are specified, the entire zone database (both the defined and effective configuration) is displayed.
Zone creation and maintenance 12 • A minus sign (–) before any entity indicates that this entity has been deleted. If zone members are added as well as deleted in a zone configuration, then a plus sign and a minus sign (+-) will be displayed before the member and a * sign will be displayed before the zone name. • A plus sign (+) before any member of an alias or zone name or any other entity indicates this member has been added, and a minus sign (–) indicates the particular member has been deleted.
12 Zone creation and maintenance Validating a zone Use the following procedure to validate a zone. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the cfgShow command to view the zone configuration objects you want to validate.
Zone creation and maintenance 12 Example validating the zone members beginning with gre, regardless of the case switch:admin> zone --validate -i gre* Defined configuration: zone: GREEN 44, 4; 21:00:00:20:37:0c:71:02; 8,9 zone: green 2,2*; 2,3*; 21:00:00:20:37:0c:76:8c* Effective configuration: zone: green 2,2* 2,3* 21:00:00:20:37:0c:76:8c* -----------------------------------~ - Invalid configuration * - Member does not exist # - Invalid usage of broadcast zone Inconsistencies between the defined and effe
12 Default zoning mode zone: zone2 1,1; 1,2 switch: admin> zoneadd zone1, 10:00:00:00:00:00:00:03 switch: admin> cfgsave WARNING!!! The changes you are attempting to save will render the Effective configuration and the Defined configuration inconsistent. The inconsistency will result in different Effective Zoning configurations for switches in the fabric if a zone merge or HA failover happens. To avoid inconsistency it is recommended to commit the configurations using the 'cfgenable' command.
Default zoning mode 12 Setting the default zoning mode NOTE You should not change the default zone mode from “No Access” to “All Access” if there is no effective zone configuration and more than 120 devices are connected to the fabric. Use the following procedure to set the default zoning mode. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the cfgActvShow command to view the current zone configuration. 3.
12 Zone database size Zone database size The maximum size of a zone database is the upper limit for the defined configuration, and it is determined by the amount of flash memory available for storing the defined configuration. Use the cfgSize command to display the zone database size. The supported maximum zone database size is 2 MB for systems running only Brocade DCX, DCX-4S, and DCX 8510 platforms. The presence of any other platform reduces the maximum zone database size to 1 MB.
Zone configurations 12 If you create or make changes to a zone configuration, you must enable the configuration for the changes to take effect. Creating a zone configuration Use the following procedure to create a zone configuration. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the cfgCreate command, using the following syntax: cfgcreate "cfgname", "member[; member...]" 3. Enter the cfgSave command to save the change to the defined configuration.
12 Zone configurations Do you still want to proceed with saving the Defined zoning configuration only? (yes, y, no, n): [no] y Removing members from a zone configuration Use the following procedure to remove members from a zone configuration. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the cfgRemove command, using the following syntax: cfgremove "cfgname", "member[; member...]" 3. Enter the cfgSave command to save the change to the defined configuration.
Zone configurations 12 Disabling a zone configuration When you disable the current zone configuration, the fabric returns to non-zoning mode. All devices can then access each other or not, depending on the default zone access mode setting. NOTE If the default zoning mode is set to All Access and more than 120 devices are connected to the fabric, you cannot disable the zone configuration because this would enable All Access mode and cause a large number of requests to the switch.
12 Zone configurations Abandoning zone configuration changes To abandon zone configuration changes, enter the cfgTransAbort command. When this command is executed, all changes since the last save operation (performed with the cfgSave, cfgEnable, or cfgDisable command) are cleared.
Zone configurations 12 Viewing selected zone configuration information Use the following procedure to view the selected zone configuration information. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the cfgShow command and specify a pattern.
12 Zone object maintenance Run cfgSave to commit the transaction or cfgTransAbort to cancel the transaction. Do you really want to clear all configurations? (yes, y, no, n): [no] 3. Enter one of the following commands, depending on whether an effective zone configuration exists: • If no effective zone configuration exists, use the cfgSave command.
Zone object maintenance 12 Deleting a zone object The following procedure removes all references to a zone object and then deletes the zone object. The zone object can be a zone member, a zone alias, or a zone. Use the following procedure to delete a zone object. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the cfgShow command to view the zone configuration objects you want to delete.
12 Zone configuration management Renaming a zone object Use the following procedure to rename a zone object. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter cfgShow to view the zone configuration objects you want to rename.
Security and zoning 12 Security and zoning Zones provide controlled access to fabric segments and establish barriers between operating environments. They isolate systems with different uses, protecting individual systems in a heterogeneous environment; for example, when zoning is in secure mode, no merge operations occur. Brocade Advanced Zoning is configured on the primary fabric configuration server (FCS). The primary FCS switch makes zoning changes and other security-related changes.
12 Zone merging • Merging and segmentation The fabric is checked for segmentation during power-up, when a switch is disabled or enabled, or when a new switch is added. The zone configuration database is stored in nonvolatile memory by the cfgSave command. All switches in the fabric have a copy of this database. When a change is made to the defined configuration, the switch where the changes were made must close its transaction for the changes to be propagated throughout the fabric.
Zone merging 12 A merge is not possible if any of the following conditions exist: - Configuration mismatch: Zoning is enabled in both fabrics and the zone configurations that are enabled are different in each fabric. - Type mismatch: The name of a zone object in one fabric is used for a different type of zone object in the other fabric. - Content mismatch: The definition of a zone object in one fabric is different from the definition of the zone object with the same name in the other fabric.
12 Zone merging TABLE 64 Zone merging scenarios: Defined and effective configurations (Continued) Description Switch A Switch B Expected results Switch A and Switch B have the same defined configuration. Neither have an effective configuration. defined: cfg1 zone1: ali1; ali2 effective: none defined: cfg1 zone1: ali1; ali2 effective: none No change (clean merge). Switch A and Switch B have the same defined and effective configuration.
Zone merging TABLE 66 12 Zone merging scenarios: Different names Description Switch A Switch B Expected results Same content, different effective cfg name. defined: cfg1 zone1: ali1; ali2 effective: cfg1 zone1: ali1; ali2 defined:cfg2 zone1: ali1; ali2 effective: cfg2 zone1: ali1; ali2 Fabric segments due to: Zone Conflict cfg mismatch Same content, different zone name.
12 Concurrent zone transactions TABLE 68 Zone merging scenarios: Default access mode Description Switch A Switch B Expected results Different default zone access mode settings. defzone: allaccess defzone: noaccess Clean merge — noaccess takes precedence and defzone configuration from Switch B propagates to fabric. defzone: noaccess Same default zone access mode settings. defzone: allaccess defzone: allaccess Clean merge — defzone configuration is allaccess in the fabric.
Concurrent zone transactions 12 Example of how users are warned if there is already a pending zoning transaction in the fabric u30:FID128:admin> zonecreate z2, "2,3" WARNING!! Multiple open transactions are pending in this fabric. Only one transaction can be saved. Please abort all unwanted transactions using the cfgtransabort command. Use the cfgtransshow --opentrans command to display a list of domains with open transactions If no other transaction is open in this fabric, no message is shown.
12 Concurrent zone transactions Current transaction token is 0x3109 It is abortable Transactions Detect: Capable Current Open Transactions Domain List: -----------1 2 3 4 378 Fabric OS Administrator’s Guide 53-1002920-02
Chapter 13 Traffic Isolation Zoning In this chapter • Traffic Isolation Zoning overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • TI zone failover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Enhanced TI zones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Traffic Isolation Zoning over FC routers . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13 TI zone failover Traffic isolation is implemented using a special zone, called a Traffic Isolation zone (TI zone). A TI zone indicates the set of N_Ports and E_Ports to be used for a specific traffic flow. When a TI zone is activated, the fabric attempts to isolate all inter-switch traffic entering from a member of the zone to only those E_Ports that have been included in the zone. The fabric also attempts to exclude traffic not in the TI zone from using E_Ports within that TI zone.
TI zone failover 13 ATTENTION If failover is disabled, use care when planning your TI zones so that non-TI zone devices are not isolated. If this feature is not used correctly, it can cause major fabric disruptions that are difficult to resolve. See “Additional considerations when disabling failover” on page 381 for additional information about using this feature. Table 70 compares the behavior of traffic when failover is enabled and disabled.
13 TI zone failover • Ensure that there are non-dedicated paths through the fabric for all devices that are not in a TI zone. • If you create a TI zone with just E_Ports, failover must be enabled. If failover is disabled, the specified ISLs will not be able to route any traffic. • If the path between devices in a TI zone is broken, no inter-switch RSCNs are generated.
TI zone failover 13 FSPF routing rules and traffic isolation All traffic must use the lowest cost path. FSPF routing rules take precedence over the TI zones, as described in the following situations. If the dedicated ISL is not the lowest cost path ISL, then the following rules apply: • If failover is enabled, the traffic path for the TI zone is broken, and TI zone traffic uses the lowest cost path instead. • If failover is disabled, the TI zone traffic is blocked.
13 Enhanced TI zones Domain 1 8 Domain 3 1 9 9 14 12 3 15 7 16 6 = Dedicated Path = Ports in the TI zone 5 Domain 4 Domain 2 FIGURE 38 Dedicated path is not the shortest path NOTE For information about setting or displaying the FSPF cost of a path, see the linkCost and topologyShow commands in the Fabric OS Command Reference. Enhanced TI zones In Fabric OS v6.4.0 and later, ports can be in multiple TI zones at the same time.
Enhanced TI zones 13 Illegal configurations with enhanced TI zones When you create TI zones, ensure that all traffic from a port to all destinations on a remote domain have the same path. Do not create separate paths from a local port to two or more ports on the same remote domain. If the TI zones are configured with failover disabled, some traffic will be dropped.
13 Traffic Isolation Zoning over FC routers In this example traffic from the Target to Domain 2 is routed correctly. Only one TI zone describes a path to Domain 2. However, both TI zones describe different, valid paths from the Target to Domain 1. Only one path will be able to get to (1,1). Traffic from port (3,8) cannot be routed to Domain 1 over both (3,6) and (3,7), so one port will be chosen. If (3,7) is chosen, frames destined for (1,1) will be dropped at Domain 1.
Traffic Isolation Zoning over FC routers Edge fabric 1 Backbone fabric 13 Edge fabric 2 = Dedicated path set up by TI zone in edge fabric 1 = Dedicated path set up by TI zone in edge fabric 2 = Dedicated path set up by TI zone in backbone fabric FIGURE 42 Traffic Isolation Zoning over FCR In addition to setting up TI zones, you must also ensure that the devices are in an LSAN zone so that they can communicate with each other.
13 Traffic Isolation Zoning over FC routers TI zones within an edge fabric A TI zone within an edge fabric is used to route traffic between a real device and a proxy device through a particular EX_Port. For example, in Figure 43, you can set up a TI zone to ensure that traffic between Host 1 and the proxy target is routed through EX_Port 9.
Traffic Isolation Zoning over FC routers 13 TI zones within a backbone fabric A TI zone within a backbone fabric is used to route traffic within the backbone fabric through a particular ISL. For example, in Figure 44, a TI zone is set up in the backbone fabric to ensure that traffic between EX_Ports “1,1” and “2,1” is routed through VE_Ports “1,4” and “2,7”.
13 Fabric-Level Traffic Isolation in a backbone fabric Limitations of TI zones over FC routers Be aware of the following when configuring TI zones over FC routers: • A TI zone defined within the backbone fabric does not guarantee that edge fabric traffic will arrive at a particular EX_Port. You must set up a TI zone in the edge fabric to guarantee this. • TI zones within the backbone fabric cannot contain more than one destination router port (DRP) per each fabric.
Fabric-Level Traffic Isolation in a backbone fabric FIGURE 45 13 Fabric-level traffic isolation In the figure, there are two links between each edge fabric and the backbone fabric, and there are five links between the two FC routers in the backbone. Fabric ID 1 and Fabric ID 4 communicate only with each other. Two backbone ISLs are dedicated to traffic between FID1 and FID4. These dedicated ISL are indicted in red and blue.
13 Fabric-Level Traffic Isolation in a backbone fabric There are two options for defining the Fabric-Level Traffic Isolation paths within TI zones. • Create a separate TI zone for each path • Combine all of the paths in a single TI zone The option you select affects the failover behavior of the TI zones. Failover behavior for Fabric-Level TI zones Fabric-Level Traffic Isolation requires the TI zones in the backbone to have failover enabled.
Fabric-Level Traffic Isolation in a backbone fabric Port List: 13 20,5; 20,3; 30,7; 30,9 Configured Status: Activated / Failover-Enabled Enabled Status: Deactivated Note that although the configured status is “Activated”, the enabled status is “Deactivated”. 3. Activate TI zones. switch:admin> cfgactvshow Effective configuration: cfg: … switch:admin> cfgenable You are about to enable a new zoning configuration.
13 General rules for TI zones 2. Display defined TI zone. switch:admin> zone --show Defined TI zone configuration: TI Zone Name: TI_Zone_ALL Port List: 20,3; 20,4 20,5; 20,6; 30,7; 30,8; 30,9; 30,10 Configured Status: Activated / Failover-Enabled Enabled Status: Deactivated Note that although the configured status is “Activated”, the enabled status is “Deactivated”. 3. Activate the TI zone.
General rules for TI zones 13 • TI zones reside only in the defined configuration and not in the effective configuration. When you make any changes to TI zones, including creating or modifying them, you must enable the effective configuration for the changes to take effect, even if the effective configuration is unchanged. • A TI zone only provides traffic isolation and is not a “regular” zone. • Routing rules imposed by TI zones with failover disabled override regular zone definitions.
13 Supported configurations for Traffic Isolation Zoning Example RASlog message when --showTItrunkerrors is added to zone command switch:admin> zone --showTItrunkerrors TI Zone Name: brackets E-Port Trunks Trunk members in TI zone: 16 18 Trunk members not in TI zone: 17 F-Port Trunks Trunk members in TI zone: 4 5 Trunk members not in TI zone: 6 TI Zone Name: loop E-Port Trunks Trunk members in TI zone: 0 Trunk members not in TI zone: 1 TI Zone Name: operand E-Port Trunks Trunk members in TI zone: 8 Trunk
Supported configurations for Traffic Isolation Zoning 13 • If the fabric contains a switch running an earlier version of Fabric OS, you cannot create an enhanced TI zone. You cannot merge a downlevel switch into a fabric containing enhanced TI zones, and you cannot merge a switch with enhanced TI zones defined into a fabric containing switches that do not support ETIZ. • Overlapping TI zones must have the same failover type. That is, both must be either failover enabled or failover disabled.
13 Limitations and restrictions of Traffic Isolation Zoning Limitations and restrictions of Traffic Isolation Zoning The following limitations and restrictions apply to Traffic Isolation Zoning: • For switches running Fabric OS 6.1.0 or later, a maximum of 255 TI zones can be created in one fabric. For switches running Fabric OS 6.0.x, no more than 239 TI zones should be created. A fabric merge resulting in greater than the maximum allowed TI zones results in merge failure and the fabrics are segmented.
Virtual Fabrics considerations for Traffic Isolation Zoning 13 • Use care if defining TI zones with ports that are shared across Admin Domains because of the limitation that a given port can appear in only one TI zone. Best practice: Do not use ports that are shared across Admin Domains in a TI zone. Virtual Fabrics considerations for Traffic Isolation Zoning This section describes how TI zones work with Virtual Fabrics.
13 Virtual Fabrics considerations for Traffic Isolation Zoning Domain 8 Host Domain 3 2 4 Domain 5 Domain 9 11 17 7 6 10 16 8 5 8 Target 9 1 3 = Dedicated Path = Ports in the TI zones FIGURE 48 Creating a TI zone in a logical fabric You must also create and activate a TI zone in the base fabric to reserve the XISLs for the dedicated path.
Traffic Isolation Zoning over FC routers with Virtual Fabrics 13 Traffic Isolation Zoning over FC routers with Virtual Fabrics This section describes how you can set up TI zones over FC routers in logical fabrics. Figure 50 shows two physical chassis configured into logical switches. The initiator in FID 1 communicates with the target in FID 3 over the EX_Ports in the base switches.
13 Creating a TI zone Creating a TI zone You create and modify TI zones using the zone command. Other zoning commands, such as zoneCreate, aliCreate, and cfgCreate, cannot be used to manage TI zones. When you create a TI zone, you can set the state of the zone to activated or deactivated. By default the zone state is set to activated; however, this does not mean that the zone is activated.
Creating a TI zone 13 Example TI zone creation The following examples create a TI zone named “bluezone”, which contains E_Ports 1,1 and 2,4 and N_Ports 1,8 and 2,6.
13 Creating a TI zone Creating a TI zone in a base fabric 1. Connect to the switch and log in using an account with admin permissions. 2. Create a “dummy” zone configuration in the base fabric. For example: zone --create "z1", "1,1" cfgcreate "base_config", z1 3. Enter the zone --create command to create the TI zone in the base fabric: zone --create -t objtype -o f name -p "portlist" The disable failover option is not supported in base fabrics. 4.
Modifying TI zones 13 Modifying TI zones Using the zone --add command, you can add ports to an existing TI zone, change the failover option, or both.You can also activate or deactivate the TI zone. Using the zone --remove command, you can remove ports from existing TI zones. If you remove the last member of a TI zone, the TI zone is deleted. After you modify the TI zone, you must enable the current effective configuration to enforce the changes.
13 Changing the state of a TI zone Example of modifying a TI zone To add port members to the existing TI zone bluezone: switch:admin> zone --add bluezone -p "3,4; 3,6" To add port members to the existing TI zone in a backbone fabric: switch:admin> zone --add backbonezone -p "3,4; 3,6; 10:00:00:04:1f:03:16:f2;" To disable failover on the existing TI zone bluezone: switch:admin> zone --add -o n bluezone To enable failover and add ports to TI zone greenzone: switch:admin> zone --add -o f greenzone -p "3,
Deleting a TI zone 13 Deleting a TI zone Use the zone --delete command to delete a TI zone from the defined configuration. This command deletes the entire zone; to only remove port members from a TI zone, use the zone --remove command, as described in “Modifying TI zones” on page 405. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the zone --delete command.
13 Troubleshooting TI zone routing problems Example displaying information about all TI zones in the defined configuration in ascending order switch:admin> zone --show -ascending Defined TI zone configuration: TI Zone Name: Port List: bluezone: 8,3; 8,5; 9,2; 9,3; Configured Status: Deactivated / Failover-Disabled Enabled Status: Activated / Failover-Enabled TI Zone Name: Port List: greenzone: 2,2; 3,3; 4,11; 5,3; Configured Status: Activated / Failover-Enabled Enabled Status: Activated / Failover-Ena
Setting up TI zones over FCR (sample procedure) 13 • “WARNING” indicates that there is not currently a problem, given the current set of online devices and reachable domains, but given the activated TI zone configuration, parallel exclusive paths between a shared device and a remote domain have been detected, which might cause a problem for devices that join the fabric later. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the zone --showTIerrors command.
13 Setting up TI zones over FCR (sample procedure) NOTE In the following procedure the three TI zones in the edge and backbone fabrics are all given the same name, TI_Zone1. It is not required that the TI zones have the same name, but this is done to avoid confusion. If several dedicated paths are set up across the FC router, the TI zones for each path can have the same name. 1.
Setting up TI zones over FCR (sample procedure) 13 3. Log in to the edge fabric 2 and set up the TI zone. a. Enter the fabricShow command to display the switches in the fabric. From the output, you can determine the front and translate domains. E2switch:admin> fabricshow Switch ID Worldwide Name Enet IP Addr FC IP Addr Name ------------------------------------------------------------------------1: fffc01 50:00:51:e3:95:36:7e:09 0.0.0.0 0.0.0.0 "fcr_fd_1" 4: fffc04 50:00:51:e3:95:48:9f:a1 0.0.0.0 0.0.0.
13 Setting up TI zones over FCR (sample procedure) b. Enter the following commands to reactivate your current effective configuration and enforce the TI zones. BB_DCX_1:admin> cfgactvshow Effective configuration: cfg: cfg_TI zone: lsan_t_i_TI_Zone1 10:00:00:00:00:00:02:00:00 10:00:00:00:00:00:03:00:00 10:00:00:00:00:00:08:00:00 BB_DCX_1:admin> cfgenable cfg_TI You are about to enable a new zoning configuration. This action will replace the old zoning configuration with the current configuration selected.
Chapter 14 Optimizing Fabric Behavior In this chapter • Adaptive Networking overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Ingress Rate Limiting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • QoS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • CS_CTL-based frame prioritization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14 Ingress Rate Limiting • Ingress Rate Limiting Ingress Rate Limiting restricts the speed of traffic from a particular device to the switch port. Ingress Rate Limiting does not require a license. Refer to “Ingress Rate Limiting” on page 414 for more information about this feature. • Quality of Service (QoS) QoS allows you to categorize the traffic flow between a host and target as having a high, medium, or low priority. QoS does not require a license.
QoS 14 Limiting traffic from a particular device 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the portCfgQos --setratelimit command. portcfgqos --setratelimit [slot/]port ratelimit Example of setting the rate limit on slot 3, port 9 to 4000 Mbps portcfgqos --setratelimit 3/9 4000 Disabling Ingress Rate Limiting 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the portCfgQos --resetratelimit command.
14 CS_CTL-based frame prioritization TABLE 71 Comparison between CS_CTL-based and QoS zone-based prioritization CS_CTL-based frame prioritization QoS zone-based traffic prioritization Must be manually enabled. Automatically enabled. No zones are required. Requires you to create QoS zones. Enabled on F_Ports or FL_Ports. Enabled on E_Ports. Takes precedence over QoS zone-based traffic prioritization. Is overridden by CS_CTL-based frame prioritization.
CS_CTL-based frame prioritization TABLE 72 14 Mapping of CS_CTL values to QoS priority for frame prioritization in CS_CTL default mode CS_CTL value Priority 1–8 Low 9–16 Medium 17–24 High Alternatively, the user can apply CS_CTL auto mode. The CS_CTL auto mode uses only three CS_CTL values, as illustrated in Table 73.
14 CS_CTL-based frame prioritization Disabling CS_CTL-based frame prioritization on ports When you disable CS_CTL-based frame prioritization, QoS zone-based traffic prioritization is restored if it had been previously enabled. 1. Connect to the switch and log in to an account that has admin permissions. 2.
QoS zone-based traffic prioritization 14 QoS zone-based traffic prioritization QoS zone-based traffic prioritization allows you to categorize the traffic flow between a host and a target as having a high, medium, or low priority, depending on the type of zone. For example, you could assign online transaction processing (OLTP) to high priority and backup traffic to low priority. All flows without QoS prioritization are considered medium priority.
14 QoS zones The switch automatically sets the priority for the “host,target” pairs specified in the zones according to the priority level (H, M, or L) in the zone name. For high and low priority traffic, the flow id allows you to have control over the VC assignment and control over balancing the flows throughout the fabric. The id range is as follows: • 1 through 5 for high-priority traffic, which corresponds to VCs 10 through 14.
QoS zones 14 QoS on E_Ports In addition to configuring the hosts and targets in a zone, you must also enable QoS on individual E_Ports that might carry traffic between the host and target pairs. Path selection between the “host,target” pairs is governed by FSPF rules and is not affected by QoS priorities. For example, in Figure 54, QoS should be enabled on the encircled E_Ports. NOTE By default, QoS is enabled on 8-Gbps or higher ports, except for long-distance 8-Gbps ports.
14 QoS zones • Define LSAN zones in each edge fabric. • Enable QoS on the E_Ports in each edge fabric. • Enable QoS on the EX_Ports in the backbone fabric. Refer to “Setting QoS zone-based traffic prioritization over FC routers” on page 426 for detailed instructions. The following are requirements for establishing QoS over FC routers: • QoS over FC routers is supported in Brocade native mode only. It is not supported in interopmode 2 or interopmode 3.
QoS zones Domain 1 14 Domain 3 8 9 H1 S1 1 2 5 6 3 4 8 7 LS3, FID1 Domain 7 Chassis 1 LS4, FID3 Domain 8 LS1, FID1 Domain 5 Domain 2 10 12 14 16 Base switch Domain 10 11 13 LS2, FID3 Domain 6 Chassis 2 Base switch Domain 9 15 17 = High priority = E_Ports with QoS enabled FIGURE 55 Traffic prioritization in a logical fabric Supported configurations for QoS zone-based traffic prioritization The following configuration rules apply to QoS zone-based traffic prioritization: • All
14 Setting QoS zone-based traffic prioritization Limitations and restrictions for QoS zone-based traffic prioritization • Enabling and disabling QoS is potentially disruptive to the I/O on the affected port.
Setting QoS zone-based traffic prioritization 14 The id range is from 1 through 5 for high-priority traffic, which corresponds to VCs 10 through 14. For low-priority traffic, the id range is from 1 through 2, which corresponds to VCs 8 and 9. The id is optional; if it is not specified, the virtual channels are allocated by means of a round-robin scheme. 3. Enter the cfgAdd command to add the QoS zone to the zone configuration, by using the following syntax: cfgadd "cfgname", "QOSzonename" 4.
14 Setting QoS zone-based traffic prioritization over FC routers take effect until it is re-enabled. Until the Effective configuration is re-enabled, merging new switches into the fabric is not recommended and may cause unpredictable results with the potential of mismatched Effective Zoning configurations. Do you want to save Defined zoning configuration only? (yes, y, no, n): [no] y Updating flash ... sw0:admin> cfgenable "cfg1" You are about to enable a new zoning configuration.
Chapter 15 Bottleneck Detection In this chapter • Bottleneck detection overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Supported configurations for bottleneck detection . . . . . . . . . . . . . . . . . . • Enabling bottleneck detection on a switch . . . . . . . . . . . . . . . . . . . . . . . . . • Displaying bottleneck detection configuration details . . . . . . . . . . . . . . . . • Setting bottleneck detection alerts . . . . . . . . . . . . . . . . . . . . . . . . . .
15 Bottleneck detection overview You can use the bottleneck detection feature with other Adaptive Networking features to optimize the performance of your fabric. For example, you can do the following: • If the bottleneck detection feature detects a latency bottleneck, you can use TI zones or QoS SID/DID traffic prioritization to isolate latency device traffic from high priority application traffic.
Supported configurations for bottleneck detection 15 You can use the bottleneckMon command to specify the following alerting parameters: • • • • • Whether alerts are to be sent when a bottleneck condition is detected The size of the time window to look at when determining whether to alert How many affected seconds are needed to generate the alert How long to stay quiet after an alert If an enabled alert is for congestion, for latency, or for both NOTE Changing alerting parameters affects RASlog alertin
15 Supported configurations for bottleneck detection High availability considerations for bottleneck detection The bottleneck detection configuration is maintained across a failover or reboot; however, bottleneck statistics collected are lost. Upgrade and downgrade considerations for bottleneck detection The bottleneck detection configuration is persistent across firmware upgrades and downgrades.
Enabling bottleneck detection on a switch 15 Enabling bottleneck detection on a switch Enabling bottleneck detection permits both latency and congestion detection. Bottleneck detection is enabled on a switch basis. It is recommended that you enable bottleneck detection on every switch in the fabric. If you later add additional switches, including logical switches, to the fabric, be sure to enable bottleneck detection on those switches as well.
15 Displaying bottleneck detection configuration details The following initials in the “Per-port overrides for alert parameters,” section of the output indicate which alerts have been set: • • • • C indicates a congestion alert has been set. L indicates a latency alert has been set. Y indicates both alerts are set. N indicates no alerts are set. The following examples show the status of different bottleneck alerts.
Setting bottleneck detection alerts 15 Port Alerts? LatencyThresh CongestionThresh Time (s) QTime(s) ================================================================================ 1 Y 0.100 0.800 300 300 2 C -0.800 600 600 3 L 0.100 -300 300 4 N ----- NOTE If there are no per-port overrides, “Per-port overrides for alert parameters” section is not displayed. Setting bottleneck detection alerts You can configure Fabric OS to log per-port alerts based on the latency and congestion history of the port.
15 Setting bottleneck detection alerts For this time window, 50 percent of the seconds (6 out of 12 seconds) are affected by congestion. This is below the threshold of 80 percent, so an alert would not be generated for a congestion bottleneck. For the same time window, 25 percent of the seconds (3 out of 12 seconds) are affected by latency. This exceeds the threshold of 10 percent, so an alert would be generated for a latency bottleneck.
Changing bottleneck detection parameters 15 Setting a latency alert only Entering the bottleneckmon --enable -alert=latency command enables a congestion alert. This example enables a latency alert and shows its values. switch:admin> bottleneckmon --enable -alert=latency switch:admin> bottleneckmon --status Bottleneck detection - Enabled ============================== Switch-wide sub-second latency bottleneck criterion: ==================================================== Time threshold - 0.
15 Changing bottleneck detection parameters Use the -alert parameter to enable congestion and latency alerts. Use the -cthresh parameter to specify the severity threshold for congestion that triggers an alert. Use the -lthresh parameter to specify the severity threshold for latency that triggers an alert. Use the -time parameter to specify the time window in seconds over which the percentage of seconds affected by bottleneck conditions is computed and compared with the threshold.
Changing bottleneck detection parameters Switch-wide alerting parameters: ================================ Alerts Latency threshold for alert Congestion threshold for alert Averaging time for alert Quiet time for alert - 15 Yes 0.200 0.700 200 seconds 150 seconds Example 3: Disabling bottleneck detection alerts for a port This example disables bottleneck detection alerts for port 46 only.
15 Changing bottleneck detection parameters Per-port overrides for alert parameters: ======================================== Port Alerts? LatencyThresh CongestionThresh Time (s) QTime (s) ================================================================================= 46 N ----47 L 0.750 -200 150 Example 5: Changing the latency time value for a port This example changes the time value to 250 seconds for port 47 only.
Advanced bottleneck detection settings Switch-wide alerting parameters: ================================ Alerts Latency threshold for alert Congestion threshold for alert Averaging time for alert Quiet time for alert - 15 Yes 0.200 0.700 200 seconds 150 seconds Adjusting the frequency of bottleneck alerts Depending on the circumstances, a problematic switch or port may be triggering alerts more frequently than desired.
15 Excluding a port from bottleneck detection • -lsubsecsevthresh (50) specifies the factor by which throughput must drop in a second for that second to be considered affected by latency. The default value of 50 means that the observed throughput in a second must be no more than 1/50th the capacity of the port for that second to be counted as an affected second. 1/50th of capacity equals 2 percent of capacity, which translates to 98 percent loss of throughput.
Excluding a port from bottleneck detection 15 Example showing how to exclude a single port from bottleneck detection The following example excludes port 7 only from bottleneck detection. Refer to “Disabling bottleneck detection on a switch” on page 442 for more information. NOTE Excluding the master port excludes the entire trunk, even if individual slave ports are not excluded.
15 Displaying bottleneck statistics Per-port overrides for alert parameters: ======================================== Port Alerts? LatencyThresh CongestionThresh Time (s) QTime (s) ================================================================================= 46 N ----47 L 0.750 -250 150 Displaying bottleneck statistics You can use the bottleneckmon --show command to display a history of bottleneck conditions, for up to three hours.
Disabling bottleneck detection on a switch 15 Example of disabling bottleneck detection on a switch switch:admin> bottleneckmon --disable switch:admin> bottleneckmon --status Bottleneck detection - Disabled Fabric OS Administrator’s Guide 53-1002920-02 443
15 444 Disabling bottleneck detection on a switch Fabric OS Administrator’s Guide 53-1002920-02
Chapter 16 In-flight Encryption and Compression In this chapter • In-flight encryption and compression overview . . . . . . . . . . . . . . . . . . . . . . • Configuring in-flight encryption and compression on an EX_Port . . . . . . . • Configuring in-flight encryption and compression on an E_Port . . . . . . . . • Viewing the encryption and compression configuration. . . . . . . . . . . . . . . • Configuring and enabling authentication for in-flight encryption. . . . . . . .
16 In-flight encryption and compression overview En cr yp tio on Compression/Encryption si es pr FIGURE 57 om 16G C n 16G 16G Encryption and compression on 16 Gbps ISLs Supported ports for in-flight encryption and compression The in-flight encryption and compression features are supported only on E_Ports and EX_Ports, and only on the Brocade 6510 and 6520 switches, 16 Gbps embedded switches, and the Brocade DCX 8510 Backbone family.
In-flight encryption and compression overview 16 Bandwidth and port limits for in-flight encryption and compression Fabric OS supports up to 32 Gbps of data encryption and 32 Gbps of data compression per 16 Gbps-capable FC platform. This limits the number of ports that can have these features enabled at any one time. The port speed affects the number of supported ports. The slower the speed, the more ports are supported. In general, at 16 Gbps, the number of supported ports is 2 per ASIC or trunk.
16 In-flight encryption and compression overview The port speed values can be displayed through several commands, including portEncCompShow, portShow, and switchShow. You can change the port speed on any port that has encryption or compression enabled with the portCfgSpeed command. If the capacity is available, the port is configured with the new speed. If there is not enough capacity available, you cannot change the port speed. Refer to “Setting port speeds” on page 94 for more information.
In-flight encryption and compression overview 16 • The Diffie-Hellman Challenge Handshake Authentication Protocol (DH-CHAP) must be configured along with the DH group 4 for port level authentication as a prerequisite for in-flight encryption. Pre-shared secret keys must be configured on the devices on both ends of the ISL to perform authentication. Authentication secrets greater than 32 characters are recommended for stronger encryption keys.
16 Configuring in-flight encryption and compression on an EX_Port In-flight compression on long-distance ports When configuring in-flight compression on long-distance ports, it is recommended to configure the long-distance ports with double the number of buffers. Configure the port to use the long-distance LS mode and specify the number of buffers to allocate to the port.
Configuring in-flight encryption and compression on an E_Port 16 3. If you are enabling encryption on the port, configure port level authentication for the port. Omit this step if you want to enable only compression on the port. Refer to “Configuring and enabling authentication for in-flight encryption” on page 453 for instructions. 4. Enable encryption on the port. Refer to “Enabling in-flight encryption” on page 455 for instructions. 5. Enable compression on the port.
16 Viewing the encryption and compression configuration Refer to “Enabling in-flight compression” on page 456 for instructions. Following successful port initialization, the configured features are enabled and active. You can use the islShow command to check that the E_Port has come online with encryption or compression enabled. Alternatively, you can use the portEncCompShow command to see which ports are active.
Configuring and enabling authentication for in-flight encryption 16 Configuring and enabling authentication for in-flight encryption Authentication and a secret key must be configured and established before configuring in-flight encryption. To enable authentication between an FC router and an edge fabric switch, you must first bring all EX_Ports online without using authentication.
16 Configuring and enabling authentication for in-flight encryption 6. Verify the authentication configuration using the authUtil --show command. The following example sets up authentication in preparation for in-flight encryption. Specifically, it configures DH-CHAP for authentication, sets the DH group to group 4, sets up a secret key, and activates authentication. switch:admin> authutil --set -a dhchap Authentication is set to dhchap. switch:admin> authutil --set -g "4" DH Group was set to 4.
Enabling in-flight encryption 16 AUTH TYPE HASH TYPE GROUP TYPE -------------------------------------dhchap md5 4 Switch Authentication Policy: ACTIVE Device Authentication Policy: OFF For additional information about establishing DH-CHAP secrets, refer to “Secret key pairs for DH-CHAP” on page 249. For additional information about configuring DH-CHAP authentication, refer to “Authentication policy for fabric elements” on page 243.
16 Enabling in-flight compression Enabling in-flight compression Enable in-flight compression to provide better bandwidth use on the ISLs, especially over long distance. Frames are compressed at the egress point of an ISL and then decompressed at the ingress point. Enabling compression is an offline event. Ports must be disabled first, and then re-enabled after. Before performing this procedure, it is recommended that you check for port availability.
Disabling in-flight compression 16 3. Disable encryption on the port using the portCfgEncrypt --disable command. The following example disables encryption on port 15 in slot 9 of an enterprise class platform: switch:admin> portcfgencrypt --disable 9/15 4. Enable the port using the portEnable command. The following example disables encryption on port 0.
16 458 Disabling in-flight compression Fabric OS Administrator’s Guide 53-1002920-02
Chapter 17 Diagnostic Port In this chapter • Diagnostic Port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Supported platforms for D_Port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Licensing requirements for D_Port. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Understanding D_Port. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Supported topologies . . . . . . . . . . . . .
17 Licensing requirements for D_Port TABLE 75 Supported platforms for D_Port Product Fabric OS release and later Brocade DCX 8510-4 Backbone v7.0.0 Brocade DCX 8510-8 Backbone v7.0.0 Brocade 6505 switch v7.0.1 Brocade 6510 switch v7.0.0 Brocade 6520 switch v7.1.0 D_Port functionality is supported on the following HBAs: • Brocade 16-Gbps HBA (Brocade Fabric Adapter 1860) ports operating in HBA mode with a 16-Gbps SFP+ on Brocade 16-Gbps switches running Fabric OS version 7.1 or later.
Understanding D_Port 17 Once the ports are configured and enabled as D_Ports, the following basic test suite is executed in the following order, depending on the SFPs installed: 1. Electrical loopback (with 16 Gbps SFP+ only) 2. Optical loopback (with 16 Gbps SFP+ only) 3. Link traffic (with 10 Gbps SFPs, 16 Gbps SFP+, and QSFP+) 4. Link latency and distance measurement (with 10 Gbps SFPs, 16 Gbps SFP+, and QSFP+) NOTE Electrical and optical loopback tests are not supported for ICLs.
17 Understanding D_Port Table 76 summarizes D_Port test initiation modes and test start behavior. TABLE 76 D_Port configuration mode and nature of test D_Port mode/nature of test Description Mode Static User configures port explicitly. Port remains as D_Port until user removes configuration. Dynamic No user configuration required. D_Port mode is initiated by external request from remote port. Nature of test Automatic Test automatically starts when the port comes online.
Supported topologies 17 Also refer to “Limitations and considerations for D_Port with HBAs” on page 468. Supported topologies The following supported topologies illustrate at a high level how D_Port functionality can be used: • • • • “Topology 1: ISLs” on page 463 “Topology 2: ICLs” on page 463 “Topology 3: Access Gateways” on page 464 “Topology 4: HBA to switch” on page 465 Topology 1: ISLs Figure 59 illustrates ISLs that connect multiple switches through a pair of chassis.
17 Supported topologies Topology 3: Access Gateways Figure 61 illustrates a switch configured as a single Access Gateway connected to a fabric switch. N and F represent, respectively, an N_Port and an F_Port to be configured as D_Ports. The Access Gateway must be either a Brocade 6505 or 6510. FIGURE 61 Single Access Gateway to switch Figure 62 illustrates multiple Access Gateways connected to a switch in a cascaded topology.
Using D_Port without HBAs 17 Topology 4: HBA to switch Figure 64 illustrates connectivity between an HBA and a switch. F represents an F_Port to be configured as a D_Port. This topology supports dynamic D_Port mode on both the switch and the HBA. In dynamic mode, the port does not need to be configured explicitly as a D_Port. It comes up in D_Port mode when it receives a request from the remote port. FIGURE 64 HBA to switch For configuration details, refer to “Using D_Port with HBAs” on page 467.
17 Using D_Port without HBAs 5. Enable Port 2 on Switch B by using the portEnable [slot/]port command. switchB:admin> portenable 2 The basic test suite starts as soon as both ports are enabled. 6. While the test is running, enter the portDportTest - -show [slot/]port command to view test results. The following test is successful.
Using D_Port with HBAs 17 3. Repeat steps 1 and 2 for Port 2 on Switch B. switchB:admin> portdisable 2 switchB:admin> portcfgdport --disable 2 4. Enable Port 1 on Switch A by using the portEnable [slot/]port command. switchA:admin> portenable 1 5. Enable Port 2 on Switch B by using the portEnable [slot/]port command.
17 Using D_Port with HBAs Dynamic mode configuration This procedure enables a dynamic D_Port diagnostic session from the connected switch to an HBA. NOTE D_Port on HBAs is supported only on 16-Gbps SFP transceivers. 1. Disable the switch port by using the portDisable [slot/]port command. 2. Enable the switch port as a D_Port by using the portCfgDport - -enable [slot/]port command. 3. Enable the switch port by using the portEnable [slot/]port command.
Controlling testing 17 • Toggling the port on either side of the link does not restart the test. • Because of SFP electrical wrap (EWRAP) bleed-through, during the beginning of switch electrical loopback testing, the HBA will receive some broken frames, which cause the port statistic error counter to increase. Examples are “CRC err,” “bad EOF,” and “invalid order set.” Similar results occur for the optical loopback test. You should ignore these port statistics on the HBA.
17 Example test scenarios and output Confirming SFP and link status with an HBA The steps in the following example illustrate how the bcu diag - -dportenable command will fail with an SFP installed but without a connection to the switch. 1. Confirm the initial port status.
Example test scenarios and output 17 Remote port: 42 Mode: Automatic Start time: Wed Feb 2 01:41:43 2011 End time: Wed Feb 2 01:43:23 2011 Status: PASSED ================================================================================ Test Start time Result EST(secs) Comments ================================================================================ Electrical loopback 01:42:08 PASSED ----------Optical loopback 01:42:16 PASSED ----------Link traffic test 01:43:15 PASSED ----------===================
17 Example test scenarios and output ============================================= 24 ONLINE E,O PASSED 26 ONLINE E,O FAILED 33 ONLINE E,O PASSED Use the switchShow command to see D_Port information. switch:admin> switchshow switchName: switch_10 switchType: 109.
Chapter 18 NPIV In this chapter • NPIV overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Configuring NPIV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Enabling and disabling NPIV. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Viewing NPIV port configuration information. . . . . . . . . . . . . . . . . . . . . . . .
18 NPIV overview Index Port Address Media Speed State Proto ============================================== 0 0 010000 id N4 Online FC F-Port 1 1 010100 id N4 Online FC F-Port 2 2 010200 id N4 Online FC F-Port 3 3 010300 id N4 Online FC F-Port 20:0c:00:05:1e:05:de:e4 0xa06601 1 N Port + 4 NPIV public 1 N Port + 119 NPIV public 1 N Port + 221 NPIV public On the Brocade DCX and DCX-4S with the FC8-64 blade, the base port is not included in the NPIV device count.
Configuring NPIV TABLE 78 18 Number of supported NPIV devices (Continued) Platform Virtual Fabrics Logical switch type NPIV support DCX-4S Enabled Logical switch Yes, 255 virtual device limit.3 DCX-4S Enabled Base switch No. 1. Maximum limit support takes precedence if user-configured maximum limit is greater. This applies to shared areas on the FC4-48, FC8-48, and FC8-64 port blades. 2.
18 Enabling and disabling NPIV VC Link Init Locked L_Port Locked G_Port Disabled E_Port Locked E_Port ISL R_RDY Mode RSCN Suppressed Persistent Disable LOS TOV enable NPIV capability QOS E_Port Port Auto Disable: Rate Limit EX Port Mirror Port Credit Recovery F_Port Buffers Fault Delay: NPIV PP Limit: CSCTL mode: Frame Shooter Port D-Port mode: D-Port over DWDM Compression: Encryption: FEC: OFF OFF OFF OFF OFF OFF OFF OFF OFF ON AE OFF OFF OFF OFF ON OFF 0(R_A_TOV) 128 OFF OFF OFF ..
Viewing NPIV port configuration information 18 The following example shows whether a port is configured for NPIV: switch:admin> portcfgshow Ports of Slot 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 -----------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+-Speed AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN AN Trunk Port ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON Long Distance .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. VC Link Init .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
18 Viewing NPIV port configuration information portFlags: 0x24b03 PRESENT ACTIVE F_PORT G_PORT NPIV LOGICAL_ONLINE LOGIN NOELP LED ACCEPT portType: 10.0 portState: 1Online portPhys: 6In_Sync portScn: 32F_Port port generation number: 148 portId: 630200 portIfId: 43020005 portWwn: 20:02:00:05:1e:35:37:40 portWwn of device(s) connected: c0:50:76:ff:fb:00:16:fc c0:50:76:ff:fb:00:16:f8 ...
Chapter 19 Fabric-Assigned PWWN In this chapter • Fabric-Assigned PWWN overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • User- and auto-assigned FA-PWWN behavior . . . . . . . . . . . . . . . . . . . . . . . • Configuring an FA-PWWN for an HBA connected to an Access Gateway . . • Configuring an FA-PWWN for an HBA connected to an edge switch . . . . . • Supported switches and configurations for FA-PWWN . . . . . . . . . . . . . . . .
19 User- and auto-assigned FA-PWWN behavior NOTE The server must use a Brocade HBA or adapter to use the FA-PWWN feature. Refer to the release notes for the HBA or adapter versions that support this feature.
Configuring an FA-PWWN for an HBA connected to an Access Gateway 19 Configuring an FA-PWWN for an HBA connected to an Access Gateway To configure an FA-PWWN, assign the FA-PWWN on the Access Gateway switch. The FA-PWWN feature is enabled by default on the HBA. Refer to the Brocade Adapters Administrator’s Guide for a list of supported HBAs. 1. Log in to the edge switch to which the Access Gateway is directly connected. 2. Assign the FA-PWWN.
19 Configuring an FA-PWWN for an HBA connected to an edge switch If you move an HBA to a different port on a switch running Fabric OS v7.0.0 or later, the HBA will disable its port. The port remains disabled even if you then move the HBA to a port on a switch running a version of Fabric OS earlier than 7.0.0. Configuring an FA-PWWN for an HBA connected to an edge switch To configure an FA-PWWN, assign the FA-PWWN on the edge switch. The FA-PWWN feature is enabled by default on the HBA.
Supported switches and configurations for FA-PWWN 19 Supported switches and configurations for FA-PWWN The FA-PWWN feature is supported only on switches running Fabric OS 7.0.0 or later and only on Brocade HBAs and adapters. The HBA can be connected to an edge switch or to an Access Gateway switch. The FA-PWWN feature is supported on the following platforms: • Switch platforms running Fabric OS v7.0.
19 Restrictions of FA-PWWN If you are concerned about security for FA-PWWNs, you should configure device authentication. You can use authentication at the device level to ensure security between the switch and the server. Refer to “Device authentication policy” on page 246 for information about configuring device authentication. You can also use the Device Connection Control (DCC) policy to ensure that only an authorized physical server can connect to a specific switch port.
Chapter Managing Administrative Domains 20 In this chapter • Administrative Domains overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485 • Admin Domain management for physical fabric administrators . . . . . . . . 494 • SAN management with Admin Domains . . . . . . . . . . . . . . . . . . . . . . . . . . .
20 Administrative Domains overview NOTE Do not confuse an Admin Domain number with the domain ID of a switch. They are two different identifiers. The Admin Domain number identifies the Admin Domain and has a range from 0 through 255. The domain ID identifies a switch in the fabric and has a range from 1 through 239. Figure 66 shows a fabric with two Admin Domains: AD1 and AD2.
Administrative Domains overview 20 Admin Domain features Admin Domains allow you to do the following: • Define the scope of an Admin Domain to encompass ports and devices within a switch or a fabric. • Share resources across multiple Admin Domains. For example, you can share array ports and tape drives between multiple departments. In Figure 66 on page 486, one of the storage devices is shared between AD1 and AD2. • Have a separate zone database for each Admin Domain.
20 Administrative Domains overview TABLE 79 AD user types User type Description Physical fabric administrator User account with admin permissions and with access to all Admin Domains (AD0 through AD255). Creates and manages all Admin Domains. Assigns other administrators or users to each Admin Domain. The default admin account is the first physical fabric administrator. Only a physical fabric administrator can create other physical fabric administrators.
Administrative Domains overview 20 If you explicitly add DeviceA to AD0, then DeviceA is both an implicit and an explicit member of AD0. AD0 implicit members DeviceA AD0 explicit members DeviceA AD2 members none If you add DeviceA to AD2, then DeviceA is deleted from the AD0 implicit membership list, but is not deleted from the AD0 explicit membership list.
20 Administrative Domains overview FIGURE 68 Fabric with AD0 and AD255 Home Admin Domains and login You are always logged in to an Admin Domain, and you can view and modify only the devices in that Admin Domain. If you have access to more than one Admin Domain, one of them is designated as your home Admin Domain, the one you are automatically logged in to.
Administrative Domains overview 20 • For user-defined accounts, the home Admin Domain defaults to AD0 but an administrator can set the home Admin Domain to any Admin Domain to which the account is given access. • If you are in any Admin Domain context other than AD0, the Admin Domain number is included in the system prompt displayed during your session.
20 Administrative Domains overview If a device is a member of an Admin Domain, the switch port to which the device is connected becomes an indirect member of that Admin Domain and the domain,index is removed from the AD0 implicit membership list. NOTE If the switch domain ID changes, the domain,index members are invalid (they are not automatically changed). You must then reconfigure the Admin Domain with the current domain,index members.
Administrative Domains overview 20 Figure 69 on page 493 shows an unfiltered view of a fabric with two switches, three devices, and two Admin Domains. The devices are labeled with device WWNs and the switches are labeled with domain IDs and switch WWNs. FIGURE 69 Fabric showing switch and device WWNs Figure 70 shows the filtered view of the fabric as seen from AD3 and AD4. The switch WWNs are converted to the NAA=5 syntax; the device WWNs and domain IDs remain the same.
20 Admin Domain management for physical fabric administrators Admin Domain compatibility, availability, and merging Admin Domains maintain continuity of service for Fabric OS features and operate in mixed-release Fabric OS environments. High availability is supported with some backward compatibility. When an E_Port comes online, the adjacent switches merge their AD databases.
Admin Domain management for physical fabric administrators 20 Setting the default zoning mode for Admin Domains To begin implementing an Admin Domain structure within your SAN, you must first set the default zoning mode to No Access. You must be in AD0 to change the default zoning mode. 1. Log in to the switch with the appropriate RBAC role. 2. Ensure you are in the AD0 context by entering the ad --show command to determine the current Admin Domain.
20 Admin Domain management for physical fabric administrators 4. Switch to the AD255 context, if you are not already in that context: ad --select 255 5. Enter the ad --create command using the -d option to specify device and switch port members and the -s option to specify switch members: ad --create ad_id -d "dev_list" -s "switch_list" 6. Enter the appropriate command based on whether you want to save or activate the Admin Domain definition: • To save the Admin Domain definition, enter ad --save.
Admin Domain management for physical fabric administrators 20 Creating a new user account for managing Admin Domains 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the userConfig --add command using the -r option to set the role, the -a option to provide access to Admin Domains, and the -h option to specify the home Admin Domain.
20 Admin Domain management for physical fabric administrators Removing an Admin Domain from a user account When you remove an Admin Domain from an account, all of the currently active sessions for that account are logged out. 1. Connect to the switch and log in using an account with admin permissions. 2.
Admin Domain management for physical fabric administrators 20 Deactivating an Admin Domain If you deactivate an Admin Domain, the members assigned to the Admin Domain can no longer access their hosts or storage unless those members are part of another Admin Domain. You cannot log in to an Admin Domain that has been deactivated. You must activate an Admin Domain before you can log in to it. 1. Connect to the switch and log in using an account with admin permissions. 2.
20 Admin Domain management for physical fabric administrators 4. Enter the appropriate command based on whether you want to save or activate the Admin Domain definition: • To save the Admin Domain definition, enter ad --save. • To save the Admin Domain definition and directly apply the definition to the fabric, enter ad --apply.
Admin Domain management for physical fabric administrators 20 3. Enter the ad --rename command with the present name and the new name. ad --rename present_name new_name 4. Enter the appropriate command based on whether you want to save or activate the Admin Domain definition: • To save the Admin Domain definition, enter ad --save. • To save the Admin Domain definition and directly apply the definition to the fabric, enter ad --apply. The Admin Domain numbers remain unchanged after the operation.
20 Admin Domain management for physical fabric administrators Deleting all user-defined Admin Domains When you clear the Admin Domain configuration, all user-defined Admin Domains are deleted, the explicit membership list of AD0 is cleared, and all fabric resources (switches, ports, and devices) are returned to the implicit membership list of AD0. You cannot clear the Admin Domain configuration if zone configurations exist in any of the user-defined Admin Domains.
Admin Domain management for physical fabric administrators 20 3. Enter the zone --copy command to copy the zones from all user-defined Admin Domains to AD0. zone --copy source_AD.source_name dest_name In this syntax, source_AD is the name of the user-defined AD from which you are copying the zone, source_name is the name of the zone to be copied, and dest_name is the name to give to the zone after it is copied to AD0. 4. Copy the newly added zones in AD0 to the zone configuration.
20 Admin Domain management for physical fabric administrators FIGURE 71 AD0 and two user-defined Admin Domains, AD1 and AD2 At the conclusion of the procedure, all devices and zones are moved to AD0, and the user-defined Admin Domains are deleted, as shown in Figure 72.
Admin Domain management for physical fabric administrators 20 10:00:00:00:02:00:00:00; 10:00:00:00:03:00:00:00 Effective configuration: cfg: AD1_cfg zone: AD1_BlueZone 10:00:00:00:02:00:00:00 10:00:00:00:03:00:00:00 Zone CFG Info for AD_ID: 2 (AD Name: AD2, State: Active) : Defined configuration: cfg: AD2_cfg AD2_GreenZone zone: AD2_GreenZone 10:00:00:00:04:00:00:00; 10:00:00:00:05:00:00:00 Effective configuration: cfg: AD2_cfg zone: AD2_GreenZone 10:00:00:00:04:00:00:00 10:00:00:00:05:00:00:00 sw0:adm
20 SAN management with Admin Domains Validating an Admin Domain member list You can validate the device and switch member list. You can list non-existing or offline Admin Domain members. You can also identify misconfigurations of the Admin Domain. The Admin Domain validation process is not applicable for AD0, because AD0 implicitly contains all unassigned online switches and their devices. 1. Connect to the switch and log in using an account with admin permissions. 2.
SAN management with Admin Domains 20 CLI commands in an AD context The CLI command input arguments are validated against the AD member list; they do not work with input arguments that specify resources that are not members of the current Admin Domain. All commands present filtered output, showing only the members of the current Admin Domain. For example, switchShow displays details for the list of AD members present in that switch.
20 SAN management with Admin Domains Displaying an Admin Domain configuration You can display the membership information and zone database information of a specified Admin Domain. Notice the following differences in the information displayed based on the Admin Domain: • AD255: If you do not specify the AD name or number, all information about all existing Admin Domains is displayed. • AD0–AD254: The membership of the current Admin Domain is displayed.
SAN management with Admin Domains 20 You cannot switch to another Admin Domain context from within the shell created by ad --select. You must first exit the shell, and then issue the ad --select command again. Example of switching to a different Admin Domain context The following example switches to the AD12 context and back. Note that the prompt changes to display the Admin Domain.
20 SAN management with Admin Domains TABLE 81 Admin Domain interaction with Fabric OS features (Continued) Fabric OS feature Admin Domain interaction FDMI FDMI operations are allowed only in AD0 and AD255. FICON Admin Domains support FICON. However, you must perform additional steps because FICON management requires additional physical control of the ports. You must set up the switch as a physical member of the FICON AD.
SAN management with Admin Domains 20 The AD zone database also has the following characteristics: - Each zone database has its own name space. For example, you can define a zone name of test_z1 in more than one Admin Domain. - There is no zone database linked to the physical fabric (AD255) and no support for zone database updates. In the physical fabric context (AD255), you can only view the complete hierarchical zone database, which is all of the zone databases in AD0 through AD254.
20 SAN management with Admin Domains LSAN zone names in AD0 are never converted for backward-compatibility reasons. The auto-converted LSAN zone names might collide with LSAN zone names in AD0 (in the example, if AD0 contains lsan_for_linux_farm_AD005, this causes a name collision). Fabric OS does not detect or report such name clashes. LSAN zone names greater than 57 characters are not converted or sent to the FCR phantom domain.
Section Licensed Features II This section describes optionally licensed Brocade Fabric OS features and includes the following chapters: • • • • • • Chapter 21, “Administering Licensing” Chapter 22, “Inter-chassis Links” Chapter 23, “Monitoring Fabric Performance” Chapter 24, “Managing Trunking Connections” Chapter 25, “Managing Long-Distance Fabrics” Chapter 26, “Using FC-FC Routing to Connect Fabrics” Fabric OS Administrator’s Guide 53-1002920-02 513
Fabric OS Administrator’s Guide 53-1002920-02
Chapter 21 Administering Licensing In this chapter • Licensing overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Brocade 7800 Upgrade license . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • ICL licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • 8G licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21 Licensing overview Some licenses may display with the text “Obsolete license.” This happens because of changes in licensing requirements of some features that no longer require a license key, yet are still installed on a switch. ATTENTION The Adaptive Networking and Server Application Optimization (SAO) licenses are no longer required to be explicitly installed in Fabric OS 7.2.0 and later.
Licensing overview TABLE 83 21 Available Brocade licenses (Continued) License Description Advanced Extension • • • • Advanced FICON Acceleration • • Enables two advanced extension features: FCIP Trunking and Adaptive Rate Limiting. FCIP Trunking feature allows all of the following: - Multiple (up to 4) IP source and destination address pairs (defined as FCIP Circuits) using multiple (up to 4) 1-GbE or 10-GbE interfaces to provide a high bandwidth FCIP tunnel and failover resiliency.
21 Licensing overview TABLE 83 Available Brocade licenses (Continued) License Description Encryption Performance Upgrade Provides additional encryption bandwidth on encryption platforms. For the Brocade Encryption Switch, two Encryption Performance Upgrade licenses can be installed to enable the full available bandwidth. On a Brocade enterprise platform, a single Performance License can be installed to enable full bandwidth on all FS8-18 blades installed in the chassis.
Licensing overview TABLE 83 21 Available Brocade licenses (Continued) License Description ICL 16-Link Activates all 16 links on ICL ports on a Brocade DCX chassis. Each chassis must have the ICL 16-Link license installed in order to enable the full 16-link ICL connections. Available on the Brocade DCX only.
21 Licensing overview TABLE 84 License requirements and location name by feature (Continued) Feature License Where license should be installed Extended Fabrics Extended Fabrics Local switch and any attached switches. Fabric Watch No license required for baseline monitoring capabilities. Fabric Watch license or Fabric Vision license required for full functionality. See the Fabric Watch Administrator’s Guide. FCIP High Performance Extension over FCIP/FC NOTE: Local and attached switches.
Licensing overview TABLE 84 21 License requirements and location name by feature (Continued) Feature License Where license should be installed Inter-chassis link (ICL) • Local and attached platforms. • • • • ICL 1st POD (Ports on Demand) on the Brocade DCX 8510 Backbone family only. ICL 2nd POD on the Brocade DCX 8510-8 only. ICL 8-link on the Brocade DCX and DCX-4S only. ICL 16-link on the Brocade DCX only.
21 Licensing overview TABLE 84 License requirements and location name by feature (Continued) Feature License Where license should be installed QoS No license required. Adaptive Networking with QoS license is required for switches running Fabric OS versions earlier than 7.2.0. The Brocade 6520 does not require a license regardless of Fabric OS version. N/A for local switches running Fabric OS 7.2.0 or later. License required on local and attached switches running Fabric OS versions earlier than 7.2.
Brocade 7800 Upgrade license TABLE 84 21 License requirements and location name by feature (Continued) Feature License Where license should be installed Web Tools No license required. Local and any switch you will be managing using Web Tools. Zoning No license required. N/A Brocade 7800 Upgrade license The Brocade 7800 has four Fibre Channel (FC) ports and two GbE ports active by default. The number of physical ports active on the Brocade 7800 is fixed.
21 ICL licensing On the Brocade DCX 8510-8, this license enables QSFP ports 0–7; QSFP ports 8–15 are disabled. (QSFP ports 0–7 correspond to core blade port numbers 0–31, and QSFP ports 8–15 correspond to core blade port numbers 32–63, as observed in switchShow output.) This license allows you to purchase half the bandwidth of the Brocade DCX 8510-8 ICL ports initially and upgrade with an additional ICL license to use the full ICL bandwidth later.
8G licensing 21 • When Virtual Fabrics are used, the limit on the number of chassis connected together via ICLs depends only on the physical chassis and not on the logical switches. • If the maximum number of ICL-connected chassis exceeds the allowed limit with or without the EICL license, additional links may either be disabled or segmented. The disabling or segmenting reason code depends on whether the EICL license is installed.
21 Slot-based licensing Slot-based licensing Slot-based licensing is used on the Brocade DCX and DCX 8510 Backbone families to support the FX8-24 blade, and on the Brocade DCX 8510 Backbone family to support the 16-Gbps FC port blades (FC16-24 and FC16-48). License capacity is equal to the number of slots. These licenses allow you to select the slots that the license will enable up to the capacity purchased and to increase the capacity without disrupting slots that already have licensed features running.
10G licensing 21 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions in the license class of RBAC commands. 2. Enter the licenseSlotCfg -add command to add the license to the appropriate slot. Removing a license from a slot Use the following procedure to remove a slot-based license from a blade slot. 1.
21 10G licensing Before removing a 10G license from an entire platform (licenseRemove command) or from a specific blade (licenseSlotCfg --remove command), you must first deconfigure all affected FC ports to no longer operate at 10 Gbps. NOTE An FC port that is operating at 10 Gbps FC speed on a 16-Gbps FC blade or 16-Gbps FC switch does not need an Extended Fabrics license to be used for FC long distance connectivity.
10G licensing 21 8510-8switch:admin> portcfgspeed 4/2 10 8510-8switch:admin> Example of assigning a 10G license on a Brocade 6510 and enabling 10 Gbps operation on a port This example assigns a license to a Brocade 6510 switch and enables 10 Gbps operation on port 2.
21 Temporary licenses 8510-4switch:admin> switchshow … 158 7 30 019e00 -159 7 31 019f00 -7 ge0 -7 ge1 -7 ge2 -7 ge3 -7 ge4 -7 ge5 -7 ge6 -7 ge7 -7 ge8 -7 ge9 -7 xge0 -7 xge1 -- -slot 7 --1G 1G 1G 1G 1G 1G 1G 1G 1G 1G 10G 10G Offline Offline No_Module No_Module No_Module No_Module No_Module No_Module No_Module No_Module No_Module No_Module No_Module No_Module VE VE FCIP FCIP FCIP FCIP FCIP FCIP FCIP FCIP FCIP FCIP FCIP FCIP Disabled Disabled Disabled Disabled Disabled Disabled Disabled Disabled Disable
Temporary licenses 21 Restrictions on upgrading temporary slot-based licenses If the capacity of the permanent license is equal to or greater than the capacity of the temporary license and you use the same slot assignments, then replacing the temporary license with a permanent license is non-disruptive. If either condition changes, however, then the process is disruptive. If the permanent license is for fewer slots than the temporary license, you must do the following: 1. Remove the temporary license.
21 Viewing installed licenses Removing an expired license CAUTION This procedure is disruptive to the switch. Use the following procedure to remove an expired license. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the reboot command for the expiry to take affect. Universal temporary licenses Universal temporary license keys include a duration period.
Activating a license 21 Activating a license The transaction key is case-sensitive; it must be entered exactly as it appears in the paperpack. To lessen the chance of error, copy and paste the transaction key. The quotation marks are optional. Use the following procedure to activate a license. 1. Take the appropriate action based on whether you have a license key: • If you have a license key, go to “Adding a licensed feature”.
21 Removing a licensed feature Some features may require additional configuration, or you may need to disable and re-enable the switch to make them operational; see the feature documentation for details.
Ports on Demand 21 Ports on Demand The Brocade models in the following list can be purchased with the number of licensed ports indicated. As your needs increase, you can activate unlicensed ports up to a device-constrained maximum by purchasing and installing the optional Ports on Demand licensed product. • Brocade 300—Can be purchased with 8 ports and no E_Port, 8 ports with full fabric access, or 16 ports with full fabric access.
21 Ports on Demand TABLE 86 List of available user ports when implementing PODs (Continued) Platform Available user ports, No POD license Available user ports, POD1 or POD2 present Available user ports, Both POD licenses present Brocade 6505 0–11 POD 1: 0–23 N/A Brocade 6510 0-23 0-35 0-47 Brocade 6520 0–47 0–71 0–95 Brocade 6547 0–8 and 29–31 POD1: 0–14 and 29–37 POD2: 0–8, 15–31, and 39–47 0–47 Brocade VA-40FC 0-23 0-31 0-39 Ports on Demand is ready to be unlocked in the switch
Ports on Demand 21 ATTENTION If you enable or disable an active port, you will disrupt any traffic and potentially lose data flowing on that port. If the port is connected to another switch, you will segment the switch from the fabric and all traffic flowing between the disabled port and the fabric will be lost. If you remove a Ports on Demand license, the licensed ports will become disabled after the next platform reboot or the next port deactivation.
21 Ports on Demand For the embedded switch modules, the Dynamic POD feature detects and assigns ports to a POD license only if the server blade is installed with an HBA present. A server blade that does not have a functioning HBA is treated as an inactive link during initial POD port assignment. For the non-server blade switches, the dynamic assignment occurs when an attached Fibre Channel link transitions to the “link active” state.
Ports on Demand 21 switch:admin> licenseport --method dynamic The POD method has been changed to dynamic. Please reboot the switch now for this change to take effect. 3. Enter the reboot command to restart the switch. switch:admin> reboot 4. Enter the licensePort --show command to verify the switch started the Dynamic POD feature.
21 Ports on Demand Reserving a port license You can allocate licenses by reserving and releasing POD assignments to specific ports. Disabled ports are not candidates for automatic license assignment by the Dynamic POD feature. Persistently disable an otherwise viable port to prevent it from coming online, and thereby preserve a license assignment for another port. Reserving a license for a port assigns a POD license to that port whether the port is online or offline.
Ports on Demand 21 Use the following procedure to release a port from a POD set: 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the switchDisable command to take the switch offline. switch:admin> switchdisable 3. Enter the switchShow command to verify the switch state is offline. 4. Enter the licensePort --release command to remove the port from the POD license. switch:admin> licenseport --release 0 5.
21 542 Ports on Demand Fabric OS Administrator’s Guide 53-1002920-02
Chapter 22 Inter-chassis Links In this chapter • Inter-chassis links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • ICLs for the Brocade DCX 8510 Backbone family . . . . . . . . . . . . . . . . . . . . • ICLs for the Brocade DCX Backbone family . . . . . . . . . . . . . . . . . . . . . . . . . • Virtual Fabrics considerations for ICLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Supported topologies for ICL connections. . . . . . . . . . . . . . .
22 ICLs for the Brocade DCX 8510 Backbone family NOTE A Brocade trunking license is not required for trunking on ICL connections. Refer to the specific hardware reference manuals for additional information about LED status meanings and ICL connections, including instructions on how to cable ICLs. License requirements for ICLs ICL ports can be used only with an ICL license. An ICL license must be installed on both platforms forming the ICL connection.
ICLs for the Brocade DCX 8510 Backbone family 22 • For High Availability, you should have at least two ICLs from each core blade. Figure 73 shows two Brocade DCX 8510-8 chassis connected with full redundancy using four ICL connections. Domain 1 DCX 8510-8 FIGURE 73 Domain 2 DCX 8510-8 Minimum configuration for 64 Gbps ICLs • The maximum number of ICLs between two Brocade DCX 8510-4 chassis or between a Brocade DCX 8510-8 and a Brocade DCX 8510-4 is 16.
22 ICLs for the Brocade DCX Backbone family To establish ICL trunking between platforms in the Brocade DCX 8510 Backbone family, the QSFP cables must be in the same trunk group, as illustrated in Figure 73. Refer to the specific hardware reference manuals for information about port numbering and connecting the ICL cables. ICLs for the Brocade DCX Backbone family The Brocade DCX has two ICL connectors at ports ICL0 and ICL1 on each core blade, each aggregating a set of 16 ports.
Virtual Fabrics considerations for ICLs 22 ICL trunking on the Brocade DCX and DCX-4S ICL trunks form automatically but additional licenses may be required for enabling all ICL ports or for larger ICL configurations. For more information about ICL licensing options, refer to Chapter 21, “Administering Licensing”. The ICLs are managed the same as ISL trunks. • On the Brocade DCX, each ICL is managed as two 8-port ISL trunks. • On the Brocade DCX-4S, each ICL is managed as one 8-port ISL trunk.
22 Supported topologies for ICL connections FIGURE 75 ICL triangular topology with Brocade DCX 8510-8 chassis During an ICL break in the triangular topology, the chassis that has the connections of the other two is the main chassis. Any error messages relating to a break in the topology appear in the RASlog of the main chassis.
Supported topologies for ICL connections FIGURE 76 22 Full nine-mesh topology Core-edge topology You can also connect the Brocade DCX 8510 Backbones in a core-edge topology. For example, Figure 77 shows six chassis connected in a core-edge topology (four edges and two cores). Although Figure 77 shows only the Brocade DCX 8510-8, each chassis can be either a Brocade DCX 8510-4 or a DCX 8510-8. You can have up to eight edges with DCX 8510-8 cores or up to four edges with DCX 8510-4 cores.
22 Supported topologies for ICL connections FIGURE 77 550 64 Gbps ICL core-edge topology Fabric OS Administrator’s Guide 53-1002920-02
Chapter 23 Monitoring Fabric Performance In this chapter • Advanced Performance Monitoring overview . . . . . . . . . . . . . . . . . . . . . . . • End-to-end performance monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Frame monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Top Talker monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Trunk monitoring . . . . . . . . . . . . . . .
23 Advanced Performance Monitoring overview • Frame monitors measure the traffic transmitted through a port with specific values in the first 64 bytes of the frame. • Top Talker monitors measure the flows that are major consumers of bandwidth on a switch or port. Restrictions for installing monitors • Advanced Performance Monitoring is not supported on VE_Ports and EX_Ports. If you issue commands for Advanced Performance Monitoring on VE_Ports or EX_Ports, you will receive error messages.
End-to-end performance monitoring 23 • Top Talker (port mode): Any port mode Top Talker monitors on the port are deleted. To keep the port mode Top Talker monitor, the monitor must be manually installed on the port after the move. Access Gateway considerations for Advanced Performance Monitoring EE monitors and frame monitors are supported on switches in Access Gateway mode. Top Talker monitors are not supported on these switches. EE monitors must be installed on F_Ports.
23 End-to-end performance monitoring • The Brocade 300, 5300, 5410, 5424, 5430, 5450, 5460, 5470, 5480, and 7800 models allow up to 768 end-to-end monitors shared by all ports in the same ASIC. Also, these models allow up to 192 end-to-end monitors per port. The number of interswitch links (ISLs) configured on the switch affects the amount of resources available for end-to-end monitors.
End-to-end performance monitoring 23 End-to-end performance monitoring looks at traffic on SID and DID pairs in any direction. That is, even if the SID is for a remote device, the traffic is monitored in both directions (the Tx and Rx counters are reversed).
23 End-to-end performance monitoring The perfSetPortEEMask command sets the mask for all end-to-end monitors of a port. If any end-to-end monitors are programmed on a port when the perfSetPortEEMask command is issued, then a message displays similar to the following example: switch:admin> perfsetporteemask 1/2, "00:ff:ff" Changing EE mask for this port will cause ALL EE monitors on this port to be deleted.
End-to-end performance monitoring 23 Displaying EE monitor counters You can use this procedure display the end-to-end monitors on a specified port. You can display either the cumulative count of the traffic detected by the monitors or a snapshot of the traffic at specified intervals. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the perfMonitorShow command.
23 Frame monitoring Frame monitoring Frame monitoring counts the number of times a frame with a particular pattern is transmitted by a port, and generates alerts when thresholds are crossed. Frame monitoring is achieved by defining a filter, or frame type, for a particular purpose.
Frame monitoring 23 Creating frame types to be monitored In addition to the standard frame types, you can create custom frame types to gather statistics that fit your needs. To define a custom frame type, you must specify a series of offsets, bitmasks, and values. For all transmitted frames, the switch performs the following tasks: • • • • Locates the byte found in the frame at the specified offset. Applies the bitmask to the byte found in the frame. Compares the new value with the given value.
23 Frame monitoring Example of creating a user-defined frame type and applying frame monitors to ports 3, 4, and 5 switch:admin> fmmonitor --create myframemonitor -pat "17,0xFF,0x007;7,0x4F,0x01;" -port 3-5 Deleting frame types Deleting a frame type removes the entire configuration, including configured thresholds and associated actions. It also removes any frame monitors of the specified type from all ports. You can delete only user-defined frame types; you cannot delete the predefined frame types. 1.
Frame monitoring 23 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the fmMonitor --save command to save the set of ports on which the frame type is monitored to the persistent configuration. Example In the following example, the first command adds a standard SCSI frame type monitor to ports 3 through 12, but does not save the port configuration. The second command saves the port configuration persistently.
23 Top Talker monitors 2011-03-21 00:59:55 000005| 48.6k (output truncated) Clearing frame monitor counters 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the fmMonitor --clear command to clear the counters on the ports on which the specified frame type is monitored. Example The following example clears the counters for the ABTS monitor from ports 7 through 10.
Top Talker monitors 23 You can configure Top Talker monitors on F_Ports and, depending on the switch model, on E_Ports. The following platforms support Top Talker monitors on E_Ports: - Brocade 6505 Brocade 6510 Brocade 6520 Brocade M6505 Brocade 6547 Brocade DCX 8510 family • Fabric mode Top Talker monitor In fabric mode, Top Talker monitors are installed on all E_Ports in the fabric and measure the data rate of all the possible flows in the fabric (ingress E_Port traffic only).
23 Top Talker monitors Note the following restrictions: • An E_Port-attached switch must be connected and merged with the backbone FC router before you can enable Top Talker monitors on the FC router. • Fabric mode Top Talker monitors do not support requests for domains (either front port domain or xlate domain). • Fabric mode Top Talker monitors do not monitor flows over EX_Ports.
Top Talker monitors 23 Limitations of Top Talker monitors Be aware of the following when using Top Talker monitors: • • • • • Top Talker monitors cannot detect transient surges in traffic through a given flow. You cannot install a Top Talker monitor on a mirrored port. Top Talker monitors can monitor only 10,000 flows at a time. Top Talker monitors are not supported on VE_Ports, EX_Ports, and VEX_Ports. The maximum number of all port mode Top Talker monitors on an ASIC is 16.
23 Top Talker monitors If EE monitors are present on remote switches, the command succeeds; however, on the remote switches, fabric mode fails and a RASlog message is displayed on those switches. If a new switch joins the fabric, you must run the perfTTmon --add fabricmode command on that switch. The Top Talker monitor configuration information is not automatically propagated to the new switch. Displaying the top n bandwidth-using flows on a port (port mode) 1.
Trunk monitoring 23 Deleting a Top Talker monitor on a port (port mode) 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the perfTTmon --delete command. perfttmon --delete [slotnumber/]port The following example deletes the monitor on port 7: perfttmon --delete 7 The following example deletes the monitor on slot 2, port 4 on a Backbone: perfttmon --delete 2/4 Deleting all fabric mode Top Talker monitors 1.
23 Performance data collection • The total number of frame monitors per port is limited to 16. • The total number of monitors per switch is limited to 512. When there are more than 512 monitors in the system, monitors are saved to flash memory in the following order: • The EE monitors for each port (from 0 to MAX_PORT) • The frame monitors for each port EE monitors get preference saving to flash memory when the total number of monitors in a switch exceeds 512.
Chapter 24 Managing Trunking Connections In this chapter • Trunking overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Supported platforms for trunking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Supported configurations for trunking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Requirements for trunk groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Recommendations for trunk groups. . . . .
24 Trunking overview Types of trunking Trunking can be between two switches, between a switch and an Access Gateway module, or between a switch and a Brocade adapter. The types of trunking are as follows: • ISL trunking, or E_Port trunking, is configured on an inter-switch link (ISL) between two Fabric OS switches and is applicable only to E_Ports. • ICL trunking is configured on an inter-chassis link (ICL) between two Brocade DCX or DCX 8510 Backbones and is applicable only to ports on the core blades.
Supported platforms for trunking 24 License requirements for trunking Trunking of non-ICL ports (E_Ports, EX_Ports, and F_Ports) requires the Trunking license. This license must be installed on each switch that participates in trunking. Trunking of ICL ports (E_Ports and EX_Ports) does not require a Trunking license. ATTENTION After you add the Trunking license, to enable trunking functionality, you must disable and then re-enable each port to be used in trunking, or disable and re-enable the switch.
24 Requirements for trunk groups • If in-flight encryption or compression is enabled, you can have a maximum of only two ports per trunk. • An E_Port or EX_Port trunk can be up to eight ports wide. All the ports must be adjacent to each other, in the clearly marked groups on the front of the switch. Trunks operate best when the cable length of each trunked link is roughly equal to the length of the others in the trunk. For optimal performance, no more than 30 meters difference is recommended.
Configuring trunk groups 24 • Place trunking-capable switches adjacent to each other. This maximizes the number of trunk groups that can form. If you are using a core and edge topology, place trunking-capable switches at the core of the fabric and any switches that are not trunking-capable at the edge of the fabric. • When connecting two switches with two or more ISLs, ensure that all trunking requirements are met to allow a trunk group to form.
24 Enabling trunking 3. Enter the portDisable command for each port to be used in a trunk group. Alternatively, you can enter the switchDisable command to disable all ports on the switch. 4. Enter the portEnable command for each port that you disabled in step 3, or enter the switchEnable command to enable all of the ports on the switch. NOTE F_Port trunking requires additional steps to configure the Trunk Area (TA).
Displaying trunking information 24 • Whether the trunking port connection is the master port connection for the trunk group. • Whether trunks are formed correctly. • Trunking information for a switch that is part of an FC router backbone fabric interlinking several edge fabrics. • Trunking information, including bandwidth and throughput for all the trunk groups in a switch.
24 Trunk Area and Admin Domains Trunk Area and Admin Domains Ports from different Admin Domains (ADs) are not allowed to join the same Trunk Area (TA) group. The portTrunkArea command prevents the different ADs from joining the TA group. When you assign a TA, the ports within the TA group have the same index. The index that was assigned to the ports is no longer part of the switch.
EX_Port trunking TABLE 90 24 Trunking over long distance for the Brocade Backbones and blades (Continued) Long-distance mode Distance Number of 2-Gbps ports Number of 4-Gbps ports LD 500 km 0 0 LS Static See note below NOTE The L0 mode supports up to 5 km at 2 Gbps, up to 2 km at 4 Gbps, and up to 1 km at 8 Gbps. The distance for the LS mode is static. You can specify any distance greater than 10 km.
24 EX_Port trunking Supported configurations and platforms for EX_Port trunking EX_Port trunking is a Fiber Channel Routing (FCR) software feature and requires that you have a Trunking license installed on the FC router and on the edge fabric connected to the other side of the trunked EX_Ports. The Trunking license is not required for EX_Ports on an ICL. EX_Port trunking is supported only with Brocade edge fabrics.
F_Port trunking 19 2 20 2 21 2 22 2 23 2 (fabric id 3 ee1300 id 4 ee1400 id 5 ee1500 id 6 ee1600 id 7 ee1700 id = 2 )(Trunk master) N4 N4 N4 N4 N4 No_Light Online Online Online Online EX_Port EX_Port EX_Port EX_Port 24 (Trunk port, master is Slot 2 Port 7 ) (Trunk port, master is Slot 2 Port 7 ) (Trunk port, master is Slot 2 Port 7 ) 10:00:00:60:69:80:1d:bc "MtOlympus_72" F_Port trunking You can configure F_Port trunking in the following scenarios: • Between F_Ports on a Fabric OS switch and N_Port
24 F_Port trunking FIGURE 83 Switch in Access Gateway mode without F_Port masterless trunking FIGURE 84 Switch in Access Gateway mode with F_Port masterless trunking NOTE You do not need to map the host to the master port manually because the Access Gateway will perform a cold failover to the master port. Refer to “Configuring F_Port trunking for an Access Gateway” on page 580 for instructions on configuring F_Port trunking.
F_Port trunking 24 Use the following procedure on the edge switch connected to the Access Gateway module to configure F_Port trunking. 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the portCfgShow command to ensure that the ports have trunking enabled. If trunking is not enabled, enter the portCfgTrunkPort port 1 command. 3. Enter the portDisable command for each port to be included in the TA. 4. Enter the portTrunkArea --enable command to enable the trunk area.
24 F_Port trunking 2. On the host side, enable trunking as described in the Brocade Adapters Administrator’s Guide. 3. On the switch side, enable the ports by using the portEnable command. switch:admin> portenable 3/40 switch:admin> portenable 3/41 F_Port trunking considerations Table 91 describes the F_Port masterless trunking considerations. TABLE 91 F_Port masterless trunking considerations Category Description AD You cannot create a Trunk Area on ports with different Admin Domains.
F_Port trunking TABLE 91 24 F_Port masterless trunking considerations (Continued) Category Description Default Area Port X is a port that has its Default Area the same as its Trunk Area. The only time you can remove port X from the trunk group is when the entire trunk group has the Trunk Area disabled. Downgrade You can have trunking on, but you must disable the trunk ports before performing a firmware downgrade.
24 F_Port trunking TABLE 92 PWWN format for F_Port and N_Port trunk ports NAA = 2 2f:xx:nn:nn:nn:nn:nn:nn (1) Port WWNs for: switch’s Fx_Ports. The valid range of xx is [0–FF], for a maximum of 256. NAA = 2 25:xx:nn:nn:nn:nn:nn:nn (1) Port WWNs for: switch's FX_Ports The valid range of xx is [0–FF], for a maximum of 256.
Displaying F_Port trunking information 24 Displaying F_Port trunking information Use the following commands on the edge switch to verify the F_Port trunking configuration. • Enter the switchShow command to display the switch and port information. • Enter the portTrunkArea --show enabled command to display the TA-enabled port configuration.
24 Enabling the DCC policy on a trunk area Enabling the DCC policy on a trunk area After you assign a trunk area, the portTrunkArea command checks whether there are any active DCC policies on the port with the index TA, and then issues a warning to add all the device WWNs to the existing DCC policy with index as TA. All DCC policies that refer to an index that no longer exists will not be in effect. 1. Add the WWN of all the devices to the DCC policy against the TA. 2.
Chapter 25 Managing Long-Distance Fabrics In this chapter • Long-distance fabrics overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Extended Fabrics device limitations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Long-distance link modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Configuring an extended ISL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25 Extended Fabrics device limitations • Optimized switch buffering When Extended Fabrics is installed on gateway switches (with E_Port connectivity from one switch to another), the ISLs (E_Ports) are configured with a large pool of buffer credits. The enhanced switch buffers help ensure that data transfer can occur at near-full bandwidth to use the connection over the extended links efficiently. This efficiency ensures the highest possible performance on ISLs.
Configuring an extended ISL 25 • Static Mode (LS) — LS calculates a static number of buffer credits based only on a user-defined desired_distance value. LS mode also assumes that all FC payloads are 2,112 bytes. Specify LS mode to configure a static long-distance link with a fixed buffer allocation greater than 10 km.
25 Configuring an extended ISL The following example configures slot 1, port 2 to support a 100-km link in LS mode and to use the extended link initialization sequence. This example is for an 8-Gbps platform. switch:admin> portcfgfillword 1/2 3 switch:admin> portcfglongdistance 1/2 LS 1 -distance 100 Reserved Buffers = 406 Warning: port may be reserving more credits depending on port speed.
Forward error correction on long-distance links 25 3. Disable buffer credit recovery; buffer credit recovery is not compatible with the IDLE mode. If you do not disable buffer credit recovery, it continues to perform a link reset. switch:admin> portcfgcreditrecovery --disable [slot/]port 4. Configure the port to support long-distance links.
25 592 Forward error correction on long-distance links Fabric OS Administrator’s Guide 53-1002920-02
Chapter 26 Using FC-FC Routing to Connect Fabrics In this chapter • FC-FC routing overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Fibre Channel routing concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Setting up FC-FC routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Backbone fabric IDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
26 FC-FC routing overview A Fibre Channel router (FC router) is a switch running the FC-FC routing service. The FC-FC routing service can be simultaneously used as an FC router and as a SAN extension over wide area networks (WANs) using FCIP. You can set up QoS traffic prioritization over FC routers. Refer to “QoS” on page 415 for information about QoS and instructions for setting traffic prioritization over an FC router.
FC-FC routing overview - 26 ICL ports on the core blades. EX-Port on ICL is supported only in DCX 8510-8 and DCX 8510-4 when all the port blades in the chassis belong to one of these blade types: FC16-32, FC16-48, FC8-32E, FC8-48E). NOTE Device discovery will not occur properly with ICL EX-Port connected edge fabrics if the FC Router has unsupported blades which are not mentioned above. For the Brocade Backbone families, the backbones have a limit of 128 EX_Ports for each chassis.
26 Fibre Channel routing concepts Fibre Channel routing concepts Fibre Channel routing introduces the following concepts: • Fibre Channel router (FC router) A switch running the FC-FC routing service. Refer to “Supported platforms for FC-FC routing” on page 594 for a list of platforms that can be FC routers.
Fibre Channel routing concepts 26 • Logical SANs (LSANs) An LSAN is defined by zones in two or more edge or backbone fabrics that contain the same devices. You can create LSANs that span fabrics. These LSANs enable Fibre Channel zones to cross physical SAN boundaries without merging the fabrics while maintaining the access controls of zones. An LSAN device can be a physical device, meaning that it physically exists in the fabric, or it can be a proxy device.
26 Fibre Channel routing concepts • Fabric ID (FID) Every EX_Port and VEX_Port uses the fabric ID (FID) to identify the fabric at the opposite end of the inter-fabric link. The FID for every edge fabric must be unique from the perspective of each backbone fabric. - If multiple EX_Ports (or multiple VEX_Ports) are attached to the same edge fabric, they must be configured with the same FID.
Fibre Channel routing concepts 26 ISL FC router FC router EX_Port EX_Port Backbone fabric IFL IFL E_Port E_Port Edge SAN 1 Edge SAN 2 = LSAN FIGURE 87 Edge SANs connected through a backbone fabric • Phantom domains A phantom domain is a domain emulated by the Fibre Channel router. The FC router can emulate two types of phantom domains: front phantom domains and translate phantom domains. For detailed information about phantom domains, refer to “Phantom domains” on page 601.
26 Fibre Channel routing concepts Proxy host (imported device) Host Proxy target (imported device) Target Fabric 1 Fabric 2 E_Port IFL E_Port EX_Port IFL FC router FIGURE 88 MetaSAN with imported devices FC-FC routing topologies The FC-FC routing service provides two types of routing: • Edge-to-edge Occurs when devices in one edge fabric communicate with devices in another edge fabric through one or more FC routers.
Fibre Channel routing concepts 26 Phantom domains A phantom domain is a domain created by the Fibre Channel router. The FC router creates two types of phantom domains: front phantom domains and translate phantom domains. A front phantom domain, or front domain, is a domain that is projected from the FC router to the edge fabric. There is one front phantom domain from each FC router to an edge fabric, regardless of the number of EX_Ports connected from that router to the edge fabric.
26 Fibre Channel routing concepts Host 1 Fabric 1 Front domain 1 (FC router 1) Front domain 2 (FC router 2) Xlate domain 1 (Fabric 2) Xlate domain 2 (Fabric 3) Target 1' FIGURE 90 Target 2' Target 3' EX_Port phantom switch topology All EX_Ports or VEX_Ports connected to an edge fabric use the same xlate domain ID for an imported edge fabric; this value persists across switch reboots and fabric reconfigurations.
Setting up FC-FC routing 26 1. Connect to the FC router and log in using an account with admin permissions. 2. Enter the fcrXlateConfig --show command to identify any stale xlate domains. 3. Enter the fcrXlateConfig --del command to delete the stale xlate domains.
26 Setting up FC-FC routing 8. Configure LSAN zones to enable communication between devices in different fabrics. (Refer to “LSAN zone configuration” on page 620.) Refer to Chapter 3, “Performing Advanced Configuration Tasks,” for more details about configuration options for Brocade Backbones. Verifying the setup for FC-FC routing Before configuring a fabric to connect to another fabric, you must perform the following verification checks on the FC router. 1.
Backbone fabric IDs 26 4. Verify that the Fabric-Wide Consistency Policy is not in “strict” mode by issuing the fddCfg --showall command. When it is in strict mode, an ACL cannot support Fibre Channel routing in the fabric.
26 FCIP tunnel configuration Assigning backbone fabric IDs 1. Log in to the switch or backbone. 2. Enter the switchDisable command if EX_Ports are online. 3. Enter the fosConfig --disable fcr command to disable the FC-FC routing service. The default state for the FC router is disabled. 4. Enter the fcrConfigure --bbfid command. At the prompt, enter the fabric ID, or press Enter to keep the current fabric ID, which is displayed in brackets. 5.
Inter-fabric link configuration 26 Inter-fabric link configuration Configuring an inter-fabric link (IFL) involves disabling ports and cabling them to other fabrics, configuring those ports for their intended uses, and then enabling the ports. Before configuring an inter-fabric link, be aware that you cannot configure both IFLs (EX_Ports, VEX_Ports) and ISLs (E_Ports) from a backbone fabric to the same edge fabric.
26 Inter-fabric link configuration The following example configures an EX_Port for connecting to a Brocade Network OS fabric. The -m 5 option indicates Network OS connectivity.
Inter-fabric link configuration Speed Level: Trunk Port Long Distance VC Link Init Locked L_Port Locked G_Port Disabled E_Port ISL R_RDY Mode RSCN Suppressed Persistent Disable NPIV capability EX Port Mirror Port FC Fastwrite 26 AUTO OFF OFF OFF OFF OFF OFF OFF OFF OFF ON ON ON ON 9. Enter either the portCfgEXPort or portShow command to verify that each port is configured correctly.
26 Inter-fabric link configuration portState: 2 Offline portPhys: 2 No_Module portScn: 0 port generation number: 0 portId: 014a00 portIfId: 4372080f portWwn: 20:4a:00:60:69:e2:03:86 portWwn of device(s) connected: Distance: normal portSpeed: N4Gbps LE domain: 0 FC Fastwrite: ON Interrupts: Unknown: Lli: Proc_rqrd: Timed_out: Rx_flushed: Tx_unavail: Free_buffer: Overrun: Suspended: Parity_err: 2_parity_err: CMI_bus_err: 0 0 0 0 0 0 0 0 0 0 0 0 0 Link_failure: Loss_of_sync: Loss_of_sig: Protocol_err: Inva
Inter-fabric link configuration 26 5 95 10:00:00:05:1e:37:00:45 10.32.156.31 "Brocade 5300" 6 95 10:00:00:05:1e:37:00:45 10.32.156.31 "Brocade 5300" 12. Enter the iflshow command to display the FC router details and ensure the fabric is functioning correctly.
26 Inter-fabric link configuration 2. On the FC router, disable all QSFP ports by issuing the portDisable command. switch:admin> portdisable 6/20-23 You can verify that all ports have been disabled by issuing the portShow command for the ports. 3. Configure EX_Ports on the ICL by issuing the portCfgEXPort command. If you configure EX_Port on one of the QSFP ports, the configuration is automatically propagated to the other 3 QSFP ports. The following example configures EX_Port on one of the QSFP ports.
FC router port cost configuration 50 50 3/14 3/15 1182 1183 50:00:51:e4:8f:8f:74:9e 50:00:51:e4:8f:8f:74:9f 26 10:00:00:05:1e:48:f8:00 10:00:00:05:1e:48:f8:00 9. Enter the fcrIclPathBwMonitor command to monitor and report path bandwidth imbalances for each edge fabric. The following example enables the monitoring and reporting of bandwidth balances and imbalances, and displays the ICL path bandwidth state for each fabric.
26 FC router port cost configuration Port cost considerations The router port cost has the following considerations: • Router port sets are defined as follows: - 0–7 and FCIP Tunnel 16–23 - 8–15 and FCIP Tunnel 24–31 • The router port cost does not help distinguish one IFL (or EX_ and VEX_Port link) from another, if all the IFLs are connected to the same port set.
Shortest IFL cost configuration 26 3. Enter the fcrRouterPortCost command to display the router port cost for each EX_Port. switch:admin> fcrrouterportcost Port Cost -----------------------7/3 1000 7/4 1000 7/9 1000 7/10 1000 7/13 1000 10/0 1000 You can also use the fcrRouteShow command to display the router port cost. To display the router port cost for a single EX_Port, enter the fcrRouterPortCost command with a port and slot number.
26 Shortest IFL cost configuration • For any path for which the cumulative ISL link cost of the path is greater than or equal to 10,000, the FC router sets the link cost from the front domain to translate the domain as 10,001. • For any path for which the cumulative ISL link cost of the path is less than 10,000, the link cost from front domain to translate domain will remain at 10,000, which is the shortest IFL path.
Shortest IFL cost configuration FIGURE 91 26 Shortest IFL solution Configuring shortest IFL cost 1. Enter the fcrFabricShow command to view the FC routers on the backbone fabric. switch:admin>fcrfabricshow FC Router WWN: 10:00:00:05:1e:58:bd:69, Dom ID: 10, Info: 10.17.33.59, “DID_10" EX_Port FIDNeighbor Switch Info (enet IP, WWN, name) -----------------------------------------------------------------------34 110.17.33.
26 Shortest IFL cost configuration switch:admin>fcrfabricshow FC Router WWN: 10:00:00:05:1e:58:be:67, Dom ID: 40, Info: 10.17.33.62, "DID_40" EX_Port FIDNeighbor Switch Info (enet IP, WWN, name) -----------------------------------------------------------------------34 110.17.33.68 10:00:00:05:1e:61:28:22 "DID_4_1" 2. Enter the islshow command to identify the connections between the FC routers and the destination edge fabric. switch:admin>islshow 1: 10->1010:00:00:05:1e:58:be:69 QOS 20 DID_20 sp: 8.
EX_Port frame trunking configuration 26 • In the following example, the ISL link cost of path 2 from FC router ID Domain 40 to FC router Domain ID 30 is modified. switch:admin>linkcost 10 5000 Interface10 (E_PORT)Cost 5000 • In the following example, the ISL link cost of path 2 from FC router Domain 30 to FC router Domain 20 is modified. switch:admin>linkcost 10 5000 Interface10 (E_PORT)Cost 5000 The modified cumulative link cost for path 2 is 10,000. 6.
26 LSAN zone configuration If router port cost is used with EX_Port trunking, the master port and slave ports share the router port cost of the master port. For information about setting up E_Port trunking on an edge fabric, refer to Chapter 24, “Managing Trunking Connections”. LSAN zone configuration An LSAN consists of zones in two or more edge or backbone fabrics that contain the same devices. LSANs provide selective device connectivity between fabrics without forcing you to merge those fabrics.
LSAN zone configuration 26 NOTE The "LSAN_" prefix must appear at the beginning of the zone name. LSAN zones may not be combined with QoS zones. Refer to “QoS zones” on page 419 for more information about the naming convention for QoS zones.
26 LSAN zone configuration 3. Enter the zoneCreate command to create the LSAN lsan_zone_fabric75, which includes the host. switch:admin> zonecreate "lsan_zone_fabric75", "10:00:00:00:c9:2b:c9:0c" 4. Enter the zoneAdd command to add Target A to the LSAN. FID75Domain5:admin> zoneadd "lsan_zone_fabric75", "50:05:07:61:00:5b:62:ed" 5. Enter the cfgAdd or cfgCreate and cfgEnable commands to add and enable the LSAN configuration.
LSAN zone configuration 26 This action will replace the old zoning configuration with the current configuration selected. Do you want to enable 'zone_cfg' configuration (yes, y, no, n): [no] y zone config "zone_cfg" is in effect Updating flash ... 11. Log in as admin and connect to the FC router. 12. Enter the following commands to display information about the LSANs: • lsanZoneShow -s shows the LSAN.
26 LSAN zone configuration Setting the maximum LSAN count You can set the maximum number of LSAN zones, or LSAN count, that can be configured on the edge fabrics. By default, the maximum LSAN count is set to 3,000. You can increase the maximum LSAN count to 5,000 without disabling the switch. The maximum number of LSAN devices supported is 10,000 (this includes both physical and proxy devices).
LSAN zone configuration 26 You can specify two types of tags: • Enforce tag – Specifies which LSANs are to be enforced in an FC router. • Speed tag – Specifies which LSANs are to be imported or exported faster than other LSANs. The LSAN tags are persistently saved and support configupload and configdownload. Enforce tag The Enforce tag reduces the resources used in an FC router by limiting the number of LSAN zones that will be enforced in that FC router.
26 LSAN zone configuration lsan_f2_f1 (H1, D1) lsan_f2_f3 (H1, D2) The LSAN in the host fabric does not need the tag. 3. In Edge fabric 1, configure the following LSAN: lsan_super_f1_f2 (H1, D1) 4. In Edge fabric 3, configure the following LSAN: lsan_super_f3_f2 (H1, D2) 5. Choose either the host or target to trigger the fast import process. The “super” tag is needed only in the LSANs of the target fabrics.
LSAN zone configuration 26 • The tag is from 1 through 8 alphanumeric characters. • You can configure only one Speed tag on an FC router, and up to eight Enforce tags on an FC router. The maximum number of tags (Enforce and Speed) on an FC router is eight. • Up to 500 Speed LSAN tags are supported. Configuring an Enforce LSAN tag 1. Log in to the FC router as admin. 2. Enter the following command to disable the FC router: switchdisable 3.
26 LSAN zone configuration 1. Log in to the FC router as admin. 2. Enter the fcrlsan --remove command to remove an existing LSAN tag. If you remove an Enforce LSAN tag, you must disable the switch first.
LSAN zone configuration 26 With LSAN zone binding, each FC router in the backbone fabric stores only the LSAN zone entries of the remote edge fabrics that can access its local edge fabrics. The LSAN zone limit supported in the backbone fabric is not limited by the capability of one FC router. In addition, due to the lower LSAN count, the CPU consumption by the FC router is lower.
26 LSAN zone configuration TABLE 94 LSAN information stored in FC routers, with and without LSAN zone binding Without LSAN zone binding With LSAN zone binding FC router 1 FC router 2 FC router 3 FC router 4 FC router 1 FC router 2 FC router 3 FC router 4 LSAN 1 LSAN 2 LSAN 3 LSAN 4 LSAN 1 LSAN 2 LSAN 3 LSAN 4 LSAN 1 LSAN 2 LSAN 3 LSAN 4 LSAN 1 LSAN 2 LSAN 3 LSAN 4 LSAN 1 LSAN 2 LSAN 2 LSAN 3 LSAN 4 LSAN 4 LSAN zone binding considerations • Without LSAN zone binding, the maximum number
LSAN zone configuration 26 FC router matrix definition Depending on the structure of the backbone fabric, you can specify pairs of FC routers that can access each other.
26 LSAN zone configuration Setting up LSAN zone binding 1. Log in to the FC router as admin. 2. Enter the following command to add a pair of FC routers that can access each other: FCR:Admin> fcrlsanmatrix --add -fcr wwn1 wwn2 The variables wwn1 and wwn2 are the WWNs of the FC routers. 3. Enter the following command to add a pair of edge fabrics that can access each other: FCR:Admin> fcrlsanmatrix --add -lsan fid1 fid2 The variables fid1 and fid2 are the fabric IDs of the edge fabrics. 4.
Proxy PID configuration 26 Proxy PID configuration When an FC router is first configured, the PIDs for the proxy devices are automatically assigned. Proxy PIDs (as well as phantom domain IDs) persist across reboots. The most common situation in which you would set a proxy PID is when you replace a switch. If you replace the switch and want to continue using the old PID assignments, you can configure it to do so; this value remains in the system even if the blade is replaced.
26 Inter-fabric broadcast frames Inter-fabric broadcast frames The FC router can receive and forward broadcast frames between edge fabrics and between the backbone fabric and edge fabrics. Many target devices and HBAs cannot handle broadcast frames. In this case, you can set up broadcast zones to control which devices receive broadcast frames. (Refer to “Broadcast zones” on page 343 for information about setting up broadcast zones.
Resource monitoring 26 The default maximum number of LSAN zones is 3,000. Refer to “Setting the maximum LSAN count” on page 624 for information on changing this limit. • Proxy Device Slots — The physical and proxy devices use the 10,000 device slots. The information shows the maximum pool size for translate phantom node and port WWNs and shows the number of translate node and port WWNs from this pool.
26 FC-FC routing and Virtual Fabrics FC-FC routing and Virtual Fabrics If Virtual Fabrics is not enabled, FC-FC routing behavior is unchanged. If Virtual Fabrics is enabled, then in the FC-FC routing context, a base switch is like a backbone switch and a base fabric is like a backbone fabric. If Virtual Fabrics is enabled, the following rules apply: • EX_Ports and VEX_Ports can be configured only on the base switch. When you enable Virtual Fabrics, the chassis is automatically rebooted.
FC-FC routing and Virtual Fabrics 26 Logical switch configuration for FC routing Figure 94 shows an example of two chassis partitioned into logical switches. This configuration allows the device in Fabric 128 to communicate with the device in Fabric 15 without merging the fabrics. Note the following: • The base switch in Physical chassis 1 serves as an FC router and contains EX_Ports that connect to logical switches in the two edge fabrics, Fabric 128 and Fabric 15.
26 FC-FC routing and Virtual Fabrics Edge fabric Fabric 128 Edge fabric Fabric 15 SW3 SW5 E SW1 SW7 E EX SW2 EX Fabric 1 SW4 Backbone fabric Fabric 8 FIGURE 95 SW6 SW8 Logical representation of EX_Ports in a base switch Backbone-to-edge routing with Virtual Fabrics Backbone-to-edge routing is not supported in the base switch, unless you use a legacy FC router. A legacy FC router is an FC router configured on a Brocade 7500 switch.
26 Upgrade and downgrade considerations for FC-FC routing Physical chassis 2 Physical chassis 1 IFL E Logical switch 1 E (Default logical switch) Fabric ID 128 ISL B E Logical switch 5 F (Default logical switch) Fabric ID 128 Logical switch 2 Fabric ID 1 Allows XISL use Edge fabric FID 20 Logical switch 6 Fabric ID 1 Allows XISL use C F Logical switch 3 Fabric ID 15 E ISL E E E Logical switch 7 Fabric ID 15 IFL IFL EX Logical switch 4 EX (Base switch) E Fabric ID 8 XISL E Logical switc
26 Displaying the range of output ports connected to xlate domains 1. Log in to a switch in the edge fabric. 2. Enter the lsDbShow command on the edge fabric. In the lsDbShow output, ports in the range from 129 through 255 are the output ports on the front domain. The following example shows the range of output ports.
Appendix Port Indexing A This appendix shows how to use the switchShow command to determine the mapping among the port index, slot/port numbers, and the 24-bit port ID (PID) on any Brocade Backbone. Enter the switchShow command without parameters to show the port index mapping for the entire platform. Enter the switchShow -slot command for port mapping information for the ports on the blade in a specific slot. Include the --qsfp option to list also the QSFP number, for slots that contain core blades.
A Port Indexing 740 3 20 5 741 3 21 5 742 3 22 5 743 3 23 5 744 3 24 6 745 3 25 6 746 3 26 6 747 3 27 6 748 3 28 7 10:00:00:05:1e:39:e4:5a 749 3 29 7 10:00:00:05:1e:39:e4:5a 750 3 30 7 10:00:00:05:1e:39:e4:5a 751 3 31 7 10:00:00:05:1e:39:e4:5a ---------------------------------------------trunkmaster -----trunkmaster -----trunkmaster -----trunkmaster -16G No_Module -16G No_Module -16G No_Module -16G No_Module -16G No_Module -16G No_Module -16G No_Module -16G No_Module id 16G Online name (Trunk master) id
Port Indexing A Example of port indexing on an FC8-64 blade on a Brocade DCX-4S Backbone. The Brocade DCX-4S does not need a mapping of ports on port blades because it is a one-to-one mapping. The order is sequential starting at slot 1 port 0 all the way through slot 8 port 255 for the FC8-64 blade. For core blades, the port index mapping for the blade in slot 3 begins with port index 256, and port index mapping for the core blade in slot 6 begins with port index 736.
A Port Indexing Example of port indexing on an FS8-18 blade on a DCX 8510-8 Backbone This example shows the truncated switchShow output for an FS8-18 encryption blade on the Brocade DCX 8510-8 Backbone. The assignment of port index numbers to PIDs will vary depending on blade type, platform type, and slot number.
Appendix B FIPS Support In this appendix • FIPS overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Zeroization functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • FIPS mode configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Preparing a switch for FIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
B Zeroization functions TABLE 95 646 Zeroization behavior (Continued) Keys Zeroization CLI Description FCSP Challenge Handshake Authentication Protocol (CHAP) Secret secAuthSecret –-remove The secAuthSecret -–create command is used to input the keys, and the secAuthSecret -–remove command is used to remove and zeroize the keys. All the DH-CHAP and FCAP authenticated ports are disabled after zeroization.
FIPS mode configuration B Power-on self-tests A power-on self-test (POST) is invoked by powering on the switch in FIPS mode and does not require any operator intervention. If any KATs fail, the switch goes into a FIPS Error state, which reboots the system to start the test again. If the switch continues to fail the FIPS POST, you will need to return your switch to your switch service provider for repair.
B FIPS mode configuration TABLE 96 FIPS mode restrictions (Continued) Features FIPS mode Non-FIPS mode FC-FC routing If FIPS is enabled in the FC router and disabled in the edge switch, the EX_Port is disabled if the edge fabric switch has Diffie-Hellman group 0 or hash group MD5. No restrictions. HTTP/HTTPS access HTTPS only HTTP and HTTPS HTTPS algorithms TLS/AES128 cipher suite TLS AES 128 cipher suite SSL is not supported.
FIPS mode configuration TABLE 97 B FIPS and non-FIPS modes of operation (Continued) FIPS mode non-FIPS mode The switch uses FIPS-compliant ciphers regardless of the Microsoft Active Directory server configuration. If the Microsoft Active Directory server is not configured for FIPS ciphers, authentication will still succeed. The Microsoft Active Directory server certificate is validated if the CA certificate is found on the switch.
B FIPS mode configuration LDAP CONFIGURATIONS =================== Position Server Port Domain Timeout(s) : : : : : 1 GEOFF5.ADLDAP.LOCAL 389 adldap.local 3 Primary AAA Service: LDAP Secondary AAA Service: Switch database 4. Set up LDAP according to the instructions in “LDAP configuration and Microsoft Active Directory” on page 181, and then configure the following additional Microsoft Active Directory settings: a.
Preparing a switch for FIPS B Enter remote directory: /users/aUser/certs Enter certificate name (must have ".crt" or ".cer" ".pem" suffix): LDAPTestCa.cer Enter Login Name: aUser Password: Success: imported certificate [LDAPTestCa.cer]. Exporting an LDAP switch certificate This procedure exports the LDAP CA certificate from the switch to the remote host. 1.
B Preparing a switch for FIPS Refer to Table 97 on page 648 for a complete list of restrictions between FIPS and non-FIPS modes. ATTENTION You need both securityadmin and admin permissions to enable FIPS mode. Overview of steps 1. Remove legacy OpenSSH DSA keys. 2. Optional: Configure the RADIUS server or the LDAP server. 3. Optional: Configure any authentication protocols. 4.
Preparing a switch for FIPS B The RADIUS server must also be configured to use only PEAP-MSCHAPv2. Note that among the Windows RADIUS servers supported, only Windows 2000-, Windows 2003-, and Windows 2008-based RADIUS servers may be used in a FIPS-compliant configuration. • If the switch is set for LDAP, refer to the instructions in “Setting up LDAP for FIPS mode” on page 649. 4. Optional: Set the authentication protocols. a.
B Preparing a switch for FIPS NOTE This command can be entered only from the root account. It must be entered before disabling the root account. 9. Enter the configure command and respond to the following prompts to enable signed firmware: • • • • System services: No cfgload attributes: Yes Enforce secure config Upload/Download: Press Enter to accept the default. Enforce firmware signature validation: Yes Example switch:admin> configure Not all options will be available on an enabled switch.
Preparing a switch for FIPS B Zeroizing for FIPS 1. Log in to the switch using an account with admin or securityadmin permissions, or a user account with OM permissions for the FIPSCfg RBAC class of commands. 2. Enter the fipsCfg --zeroize command. NOTE Passwords of the default accounts (admin and user) should be changed after every zeroization operation to maintain FIPS 140-2 compliance. 3. Power-cycle the switch. Displaying FIPS configuration 1.
B 656 Preparing a switch for FIPS Fabric OS Administrator’s Guide 53-1002920-02
Appendix Hexadecimal Conversion C Hexadecimal overview Hexadecimal, also known as hex, is a numeral system with a base of 16, usually written by means of symbols 0–9 and A–F (or a–f). Its primary purpose is to represent the binary code that computers interpret in a format easier for humans to remember. It acts as a form of shorthand, in which one hexadecimal digit takes the place of four binary bits.
C Hexadecimal Conversion Decimal-to-hexadecimal conversion table TABLE 99 658 Decimal-to-hexadecimal conversion table Decimal 01 02 03 04 05 06 07 08 09 10 Hex 01 02 03 04 05 06 07 08 09 0a Decimal 11 12 13 14 15 16 17 18 19 20 Hex 0b 0c 0d 0e 0f 10 11 12 13 14 Decimal 21 22 23 24 25 26 27 28 29 30 Hex 15 16 17 18 19 1a 1b 1c 1d 1e Decimal 31 32 33 34 35 36 37 38 39 40 Hex 1f 20 21 22 23 24 25 26 27 28 Decimal 41
C Hexadecimal Conversion TABLE 99 Decimal-to-hexadecimal conversion table (Continued) Decimal 171 172 173 174 175 176 177 178 179 180 Hex ab ac ad ae af b0 b1 b2 b3 b4 Decimal 181 182 183 184 185 186 187 188 189 190 Hex b5 b6 b7 b8 b9 ba bb bc bd be Decimal 191 192 193 194 195 196 197 198 199 200 Hex bf c0 c1 c2 c3 c4 c5 c6 c7 c8 Decimal 201 202 203 204 205 206 207 208 209 210 Hex c9 ca cb cc cd ce cf d0 d1 d2 Decima
C 660 Hexadecimal Conversion Fabric OS Administrator’s Guide 53-1002920-02
Index Numerics 10 Gbps operation on an FC port, enabling, 528 10-bit addressing mode, 84 10G license, 527–530 128-bit encryption, in browser, 200 16-link ICL license, 524 1st POD ICL license, 523 256-area addressing mode, 85 2nd POD ICL license, 524 8G license, 525 8-link ICL license, 524 A AAA service requests, 168 aaaConfig command, 169, 171, 189, 193, 194, 652 accepting distributed user databases locally, 158 access API, 229 browser security support, 200 changing account parameters, 157 creating accoun
policy management, 232–235 policy members, 232 removing policy member, 234 resolving conflicting ACL policies, 264 activating ACL policy changes, 233 Admin Domains, 498 IP Filter policy, 255 licenses, 533 ports on demand, 535 TI zones, 406 ad command, 494, 498, 499, 500, 501, 502, 503, 506, 507, 508 AD0, ACL management, 232 AD0, and Admin Domains, 488 AD255, ACL management, 232 AD255, and Admin Domains, 489 Adaptive Networking bottleneck detection, 413 Ingress Rate Limiting, 414 overview, 413–414 Quality of
system-defined, 488 TACACS+ service, 191 TI zone considerations, 398 transaction model, 494 trunk area, 576 user-defined, 488 using, 506 validating members, 506 VF mode and, 325 Virtual Fabrics permissions, 151 zone database, 510 admin lockout policy, disabling, 162 admin lockout policy, enabling, 162 Administrative Domains. See: Admin Domains.
B Backbone assigning fabric IDs, 606, 617 blade compatibility, 98 fabric ID, 605–606 fabric, described, 596 port blades, described, 88 port configurations supported, 321 port restrictions, 321 shutdown, 81 upgrading firmware, 297 Backbone fabric, and TI zones, 389 Backbone firmware, 296–299 download, 296 download process overview, 296 version testing, 304 Backbone-to-edge routing, 600, 605 backing up a configuration, 279 base fabric, 319 base switch about, 316 creating, 326 defined, 317 extended ISLs and, 3
Brocade 7800, upgrade license, 516, 523 Brocade 7800, XISL restriction, 320 Brocade adapters, configuring F_Port trunking for, 581 Brocade adapters, F_Port trunking for, 581 Brocade configuration setup form, 287 Brocade DCX, 519, 543, 547 auto-leveling, 290 ICLs, 546 Brocade DCX 8510, 518, 543 auto-leveling, 290 ICLs, 544 Brocade DCX 8510-4, 519 Brocade DCX 8510-8, 519 Brocade DCX-4S, 547 Brocade FC16-48 port blade enabling exceptions, 99 Brocade FC8-48 port blade enabling exceptions, 99 Brocade FC8-48E por
Microsoft Active Directory, 183 OpenLDAP, 188 RADIUS, 173 TACACS+, 188 chassisShow command, 104 CIDR block notation, 67 class 2 and 3 traffic support, 115 classConfig command, 153 classless inter-domain routing. See: CIDR.
ipAddrShow, 66, 69, 70 ipFilter, 227, 228, 254, 255, 259, 653 ipSecConfig, 267, 269, 271, 272, 274, 654 islShow, 452, 573 keyTool, 206 killTelnet, 59 ldapAdd, 189 ldapCfg, 171, 181, 182, 183, 184, 186 licenseAdd, 528, 529, 533 licenseIdShow, 40 licensePort, 538, 539, 540, 541 licenseRemove, 534 licenseShow, 528, 529, 531, 532, 533, 534, 604 licenseSlotCfg, 528, 529, 531 lsCfg, 325, 326, 329, 330, 331, 332 msCapabilityShow, 47 msConfigure, 48, 49 msPlatShow, 47, 50 msPlClearDb, 51 msplMgmtActivate, 46, 47 ms
switchStatusShow, 104 syslogDIpAdd, 109 sysShutdown, 80, 81 tac_plus, 190 topologyShow, 402 trunkShow, 574 tsClockServer, 74 tsTimeZone, 72, 73 usbStorage, 299 userConfig, 155, 497, 498, 654 version, 604 wwn, 40 wwnAddress, 87 zone, 132, 133, 369, 395, 402, 405, 406, 407, 409 zoneAdd, 351 zoneCreate, 350, 424 zoneDelete, 355 zoneHelp, 338 zoneObjectRename, 370 zoneObjectReplace, 353 zoneRemove, 352 zoneShow, 356 command line interface. See: CLI.
interfabric link, 607 IPv6 automatically, 71 links through a gateway, 121 lossless DLS, 131 NTP, 74 outgoing SSH authentication, 199 remote authentication, 167–170 remote authentication on switch, 192 root certificates, 205 security certificates, 200 Speed LSAN tag, 627 SSL, 200, 201–205 TACACS+ service, 189 zone, rules for, 342 conflicting ACL policies, resolving, 264 congestion bottleneck type, 428 congestions versus over-subscription, 119 connected devices and logical switches, 313 connecting device to a
for NPIV ports, 241 policy behavior with fabric-assigned PWWNs, 241 Virtual Fabric considerations, 239 deactivating Admin Domains, 499 TI zones, 406 decimal to hexadecimal conversion table, 658 decommissioning ports, 92 default account passwords, 63 accounts, listed, 63 Fabric OS roles, 152 IP Filter policy names, 254 IP Policy Rules, 258 logical switch, 310 zone access mode, viewing current, 361 zone mode, 360, 495 zoning mode, setting, 361 default logical switch base switch restriction, 321 XISL restricti
configuration settings, 277 current routing policy, 122 domain IDs, 75 encryption support in browser, 200 existing zones, 350 F_Port trunking information, 585 frame monitors, 561 logical switch configuration, 330 LSAN tags, 628 monitor counters, 557 network interface settings, 66 port license assignments, 538 switch name server contents, 54 TI zones, 407 trunking information, 574 distance vector, in routing, 115 distribute command, 152, 238, 262, 263, 264 Distributed Management Server FCS policy, 47 managem
FIPS mode, 652 FIPS mode, permissions needed, 652 ISL trunking, 574 local switch protection, 262 NPIV, 476 port, 92 remote authentication, 193 switches, 79, 80 Virtual Fabrics mode, 324 zone configurations, 364 Encapsulating Security Payload. See: ESP.
configuring trunking on an Access Gateway, 580 described, 88 DH-CHAP protocol failure, 247 disabling trunking, 585 displaying trunking information, 585 duplicate logins, 111 trunking considerations, 582 trunking for access gateways, 579 trunking for Brocade adapters, 581 trunking requirements on an Access Gateway, 580 F_Port trunking, 579–586 and Virtual Fabrics, 584 configuring for Brocade adapters, 581 considerations, 582 for access gateways, 579 for Brocade adapters, 581 fabric access, 229 adding Top Tal
importing switch certificate, 252 PKI certificates required, 243 specifying as authentication protocol, 248 starting authentication, 253 FC-FC routing and FCIP, 606 and Virtual Fabrics, 636 backbone-to-edge, 600 configurations supported, 595 edge-to-edge, 600 fabric mode Top Talker monitors, 605 license requirements, 594 platforms supported, 594 routing service, 593 setup, 603–605 setup verification, 604 Top Talker monitors and, 563 topologies, 600 See also: FCR and Fibre Channel routing FCIP and FC-FC rout
LDAP certificates, 650 restrictions, 647 fipsCfg command, 647, 653, 655 Firefox root certificate installation and verification, 205 SSL support, 200 firmware, 289–307 Backbone, 296–299 Backbone download process overview, 296 Backbone version testing, 304 downgrading, 291 download process, 289 downloading without a password, 291 finding version, 293 for switches, 294–295 obtaining and decompressing, 293 power-on checksum test for FIPS, 302 signed, 301 switch version testing, 302 upgrading, 291 upgrading for
user, adding, 176 vendor attributes, 175 See also: RADIUS and Linux. FSPF described, 116 number of routes supported, 116 path calculation, 117 traffic isolation routing rules, 383 FSPF-1009 RASLOG message, 398 ftp listener application, 228 G G_Port, described, 88 gateway links, 120 buffer credits, 588 gateway, configuring a link through, 121 generating DSA or RSA key pairs, 198 key and CSR for FCAP, 251 PKI key pairs, 199 H HA. See: High Availability.
in-flight encryption configuring, 455 disabling, 456 license, 445 port decommissioning, 448 restrictions, 446 in-flight encryption and compression, 445 overview, 445 ingress rate limiting, 414–415 disabling, 415 Virtual Fabrics considerations, 414 in-order frame delivery, forcing, 127 installing certificates on switch, 203 LDAP certificates, 650 root certificate to Java plugin, 205 Integrated Routing license, 594 Inter-Chassis Links. See: ICL. inter-fabric link See: IFL.
J Java installing root certificate in plugin, 205 installing root certificate to plugin, 205 support for SSL, 200 supported version, 201 Java plugin, installing root certificate for, 205 joining a switch to a fabric, 264 K key adding public key to switch, 198 deleting private from switch, 200 deleting public from switch, 200 generating for FCAP, 251 generation, 201 key management and IPsec, 271 key pair generation for RSA or DSA, 198 manual key entry and IPsec, 272 PKI key pair generation on switch, 199 pr
removing expired, 532 removing features, 534 requirements for SID/DID prioritization, 416 requirements for trunking, 571 reserving for POD, 540 slot-based, 526–527 temporary, 530–532 time-based, 530–532 universal temporary extending, 532 shelf life, 532 universal temporary, described, 532 viewing installed, 532 licenseAdd command, 528, 529, 533 licensed features, 515 listed, 516 licenseIdShow command, 40 licensePort command, 538, 539, 540, 541 licenseRemove command, 534 licenseShow command, 528, 529, 531, 5
number, 311 number per chassis, 323 port assignment, 312 restoring configuration, 285 Top Talkers and, 329 unique names for, 77 login changing password, 157 command for fabric, 53 fails, 59 process for fabric, 54 with Admin Domains, 490 login sessions, maximum allowed, 154 long distance fabrics, and ISL trunking, 576 long distance ports, compression, 450 lossless core, 130 ICL limitations, 130 traffic flow limitations, 130 lossless DLS, 129–131 configuring, 131 in Virtual Fabrics, 131 lossless dynamic load
fabric, 77 security certificate name, 271 switch, 76 name server contents, displaying, 54 naming ports, 89 NAT, 117 network address translation, see NAT network interface displaying settings, 66 logical (bond0), 65 Network OS connectivity, 593 Network OS connectivity, unsupported configurations, 595 network prefix length. See: CIDR.
PEAP-MSCHAPv2, 177, 648, 652 perfAddEeMonitor command, 554 perfCfgClear command, 568 perfCfgRestore command, 568 perfCfgSave command, 568 perfMonitorClear command, 557 perfMonitorShow command, 556, 557 performance data collection, 568 perfSetPortEEMask command, 555 perfTTmon command, 565, 566, 567 permissions assigned to roles, 153 phantom domains, 601–603 described, 599 physical fabric administrator, 487 physical fabric administrator user account, creating, 497 PID, 83–87 10-bit addressing mode, 84 assigni
deleting rule from an IP Filter policy, 259 device authentication, 246 device authentication and Virtual Fabrics considerations, 247 displaying IP Filter, 254 enforcing IP Filter, 258 FCS restrictions, 235 IP Filter, 253 IP Filter policy distribution, 260 management of ACL, 232–235 members, identifying, 232 modifying FCS, 235 password strength, 159 rules for IP Filter, 255 saving IP Filter, 255 using service names in IP Filter rules, 256 policy database distribution, 260 settings, 261 viewing settings, 262
L_Port, 88 M_Port, 88 mirror, 88 U_Port, 88 VE_Port, 88 VEX_Port, 88 Port World Wide Name. See also: PWWN.
adding to switch, 198 authentication, 198 deleting from switch, 200 generation, 201 public key infrastructure and encryption, 200 public key infrastructure. See also: PKI. PWWN assigned by fabric, 479 configuring FLOGI-time handling of duplicates, 110 duplicates, 55 handling duplicates, 111 See also: Port World Wide Name.
LSAN tags, 627 members from a zone configuration, 364 ports from logical switches, 329 zone configuration members, 364 zone members, 352 renaming Admin Domains, 500 requirements Admin Domains, 487 for F_Port trunking on an Access Gateway, 580 for trunk groups, 572 restoring configuration file, 280 logical switch configuration, 285 monitor configuration, 567 unordered frame delivery, 127 restrictions authentication policies, 247 Backbone ports, 321 compression, 446 encryption, 446 fixed-port switch ports, 32
SCP configuration for uploads and downloads, 197 described, 196 for certificates, 202 protocol, described, 195 secure protocol, 196 SCR, defined, 53 secAuthSecret command, 249, 250, 453 secCertUtil command, 201, 202, 203, 252, 253, 271, 650 secModeEnable command, 229 secPolicyAbort command, 234 secPolicyActivate command, 233, 236, 237, 264 secPolicyAdd command, 234 secPolicyCreate command, 236, 240, 243 secPolicyDelete command, 233, 239, 240 secPolicyFCSMove command, 237 secPolicyRemove command, 234 secPoli
Simple Network Management Protocol. See: SNMP. slapd.
naming, 75 PKI key pair generation, 199 ports used, 229 restoring a configuration, 282 serial number location, 40 setting date and time, 72 setting port speed, 94 setting status policy threshold values, 106 shutdown, 80 switch database distribution setting, 260 unique names for logical, 77 user-defined accounts, 155 viewing status policy threshold values, 105 switch authentication mode, setting, 171 switch authentication policy, 244 See also: AUTH. Switch Connection Control. See: SCC.
within a Backbone fabric, 389 within an edge fabric, 388 time and date, 72 time listener application, 228 Time server, described, 45 time settings, 72 time zone setting, 73 setting interactively, 73 time zone settings, 72–74 time, synchronizing local and external, 74 time-based licenses, 530–532 Top Talker monitors adding on all switches in fabric, 565 adding to aport (port mode), 565 and FC-FC routing, 563 defined, 552 deleting all in fabric, 567 deleting on a port, 567 fabric mode, described, 563 limitati
U U_Port, described, 88 unblocking telnet access, 228 understanding MIBs, 208 understanding SNMP bASICs, 207 universal temporary license defined, 530 described, 532 extending, 532 shelf life, 532 unlocking an account, 162 unordered frame delivery, restoring, 127 upgrading firmware, 291 upgrading temporary slot-based licenses, restrictions, 531 uploading AD configuration file, 512 USB device, 299, 299–300 usbStorage command, 299 user account assigning Admin Domains to, 497 creating a physical fabric administ
base switch about, 316 creating, 326 changing logical switch to base switch, 331 configDownload restrictions, 286 configUpload restrictions, 286 configuration management, 285 considerations for Adv. Perf.
Z zeroization functions for FIPS, 645 zeroizing for FIPS, 655 zone access mode, viewing current, 361 accessing, 229 adding a new switch or fabric, 371 adding members, 351 administering security, 371 alias adding members, 347 deleting, 349 removing members, 348 viewing, 349 Virtual Fabrics considerations, 347 wildcard usage, 351, 352, 354 all access, 360 broadcast, 337, 343 broadcast (reserved name), 350 concepts, 338 concurrent transactions, 376 configuration management, 370 configurations, 341 adding membe
zoneShow command, 356 zoning advanced, 337–376 advanced commands, 338 defined, 338 enforcement, 342 on logical ports, 350 overview, 338 694 Fabric OS Administrator’s Guide 53-1002920-02
