53-1002446-01 15 December 2011 Fabric OS Administrator’s Guide Supporting Fabric OS v7.0.
Copyright © 2011 Brocade Communications Systems, Inc. All Rights Reserved. Brocade, the B-wing symbol, BigIron, DCX, Fabric OS, FastIron, NetIron, SAN Health, ServerIron, and TurboIron are registered trademarks, and Brocade Assurance, Brocade NET Health, Brocade One, CloudPlex, MLX, VCS, VDX, and When the Mission Is Critical, the Network Is Brocade are trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries.
Contents About This Document In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxiii How this document is organized . . . . . . . . . . . . . . . . . . . . . . . . . . xxxiii Supported hardware and software . . . . . . . . . . . . . . . . . . . . . . . . xxxiv What’s new in this document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxv Document conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Device login . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Principal switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 E_Port login . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Fabric login . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Port login process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 RSCNs . . . . . .
Switch and Backbone shutdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Powering off a Brocade switch . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Powering off a Brocade Backbone . . . . . . . . . . . . . . . . . . . . . . . 33 Basic connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Device connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Switch connection . . . . . . . . . . . . . . . . . . . . . . .
Track and control switch changes . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Enabling the track changes feature . . . . . . . . . . . . . . . . . . . . . . 56 Displaying the status of the track changes feature. . . . . . . . . . 56 Viewing the switch status policy threshold values. . . . . . . . . . . 56 Setting the switch status policy threshold values . . . . . . . . . . . 57 Audit log configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
User accounts overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Role-Based Access Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 The management channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 Managing user-defined roles . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 Local database user accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Default accounts . . . . . . . . . . . . . . . . . . . . . . . . . .
Simple Network Management Protocol . . . . . . . . . . . . . . . . . . . . . .125 SNMP and Virtual Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . .126 The security level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .127 The snmpConfig command . . . . . . . . . . . . . . . . . . . . . . . . . . . .127 Telnet protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .127 Blocking Telnet . . . . . . . . . . . . . . . . . . . . . . . . .
IP Filter policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .153 Creating an IP Filter policy. . . . . . . . . . . . . . . . . . . . . . . . . . . . .153 Cloning an IP Filter policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . .154 Displaying an IP Filter policy . . . . . . . . . . . . . . . . . . . . . . . . . . .154 Saving an IP Filter policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .154 Activating an IP Filter policy. . . . . . . . . . .
Chapter 9 Installing and Maintaining Firmware In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .191 Firmware download process overview . . . . . . . . . . . . . . . . . . . . . . .191 Upgrading and downgrading firmware . . . . . . . . . . . . . . . . . . .192 Considerations for FICON CUP environments . . . . . . . . . . . . .193 HA sync state . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .193 Preparing for a firmware download .
Supported platforms for Virtual Fabrics . . . . . . . . . . . . . . . . . . . . .222 Supported port configurations in the fixed-port switches. . . .222 Supported port configurations in the Brocade Backbones. . .222 Virtual Fabrics interaction with other Fabric OS features . . . .223 Limitations and restrictions of Virtual Fabrics . . . . . . . . . . . . . . . .224 Restrictions on XISLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .225 Restrictions on moving ports . . . . . . . . . . . . .
Zone aliases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .246 Creating an alias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 Adding members to an alias . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 Removing members from an alias . . . . . . . . . . . . . . . . . . . . . .248 Deleting an alias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .248 Viewing an alias in the defined configuration . . . . . . .
Traffic Isolation Zoning over FC routers . . . . . . . . . . . . . . . . . . . . . . 276 TI within an edge fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .277 TI within a backbone fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . .278 Limitations of TI zones over FC routers . . . . . . . . . . . . . . . . . .279 General rules for TI zones. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .279 Supported configurations for Traffic Isolation Zoning . . . . . . . . .
Disabling bottleneck detection on a switch . . . . . . . . . . . . . . . . . .308 Chapter 14 In-flight Encryption and Compression In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .309 In-flight encryption and compression overview. . . . . . . . . . . . . . . .309 Encryption and compression restrictions. . . . . . . . . . . . . . . . . 310 How encryption and compression are enabled . . . . . . . . . . . .311 Authentication and key generation. . . . . . . . .
Configuration upload and download considerations for FA-PWWN334 Firmware upgrade and downgrade considerations for FA-PWWN .334 Security considerations for FA-PWWN . . . . . . . . . . . . . . . . . . . . . . .334 Restrictions of FA-PWWN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .335 Access Gateway N_Port failover with FA-PWWN . . . . . . . . . . . . . . .335 Chapter 17 Managing Administrative Domains In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Licensing overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .367 Brocade 7800 Upgrade license . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 ICL licensing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 ICL 1st POD license . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 ICL 2nd POD license . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .375 ICL 8-link license . . . . . . . . . .
Virtual Fabrics considerations for ICLs . . . . . . . . . . . . . . . . . . . . . .396 Supported topologies for ICL connections . . . . . . . . . . . . . . . . . . .397 Mesh topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .397 Core-edge topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .398 Chapter 20 Monitoring Fabric Performance In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chapter 21 Optimizing Fabric Behavior In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 Adaptive Networking overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 Ingress Rate Limiting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .418 Limiting traffic from a particular device . . . . . . . . . . . . . . . . . . 419 Disabling ingress rate limiting. . . . . . . . . . . . . . . . . . . . . . . . . .
Supported platforms for trunking. . . . . . . . . . . . . . . . . . . . . . . . . . .436 Recommendations for trunking groups . . . . . . . . . . . . . . . . . . . . . .437 Configuring trunk groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .437 Enabling trunking on a port or switch . . . . . . . . . . . . . . . . . . . . . . .438 Disabling trunking on a port or switch. . . . . . . . . . . . . . . . . . . . . . .438 Displaying trunking information . . . . . . . . . . . . . . . . . . . .
Chapter 24 Using FC-FC Routing to Connect Fabrics In this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .465 FC-FC routing overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .465 License requirements for FC-FC routing . . . . . . . . . . . . . . . . . .466 Supported platforms for FC-FC routing. . . . . . . . . . . . . . . . . . .466 Supported configurations for FC-FC routing. . . . . . . . . . . . . . .
Appendix A Interoperation of Fabric OS and M-EOS Fabrics Using FC Router In this appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .507 Interoperability overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .507 Release Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .507 Features of Connected SANs . . . . . . . . . . . . . . . . . . . . . . . . . .509 Establishing interoperability . . . . . . . . . . . . . . . . . . . .
xxii Fabric OS Administrator’s Guide 53-1002446-01
Figures Figure 1 Well-known addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Figure 2 Identifying the blades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Figure 3 Blade swap with Virtual Fabrics during the swap. . . . . . . . . . . . . . . . . . . . . . . . . 52 Figure 4 Blade swap with Virtual Fabrics after the swap . . . . . . . . . . . . . . . . . . . . . . . . . .
xxiv Figure 36 Illegal ETIZ configuration: two paths from one port to two devices on the same remote domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 Figure 37 Illegal ETIZ configuration: two paths from one port . . . . . . . . . . . . . . . . . . . . . . 276 Figure 38 Traffic Isolation Zoning over FCR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Figure 39 TI zone in an edge fabric . . . . . . . . . . . . . . . . .
Figure 77 EX_Port phantom switch topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474 Figure 78 Example of setting up Speed LSAN tag. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491 Figure 79 LSAN zone binding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494 Figure 80 EX_Ports in a base switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
xxvi Fabric OS Administrator’s Guide 53-1002446-01
Tables Table 1 Daemons that are automatically restarted. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Table 2 Terminal port parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Table 3 Help topic contents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Table 4 fabricShow fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
xxviii Table 37 Interaction between fabric-wide consistency policy and distribution settings . 161 Table 38 Supported policy databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 Table 39 Fabric-wide consistency policy settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 Table 40 Merging fabrics with matching fabric-wide consistency policies. . . . . . . . . . . . 165 Table 41 Examples of strict fabric merges . . . . . . . . . . . . . .
Table 79 Buffer credits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461 Table 80 Configurable distances for Extended Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 462 Table 81 LSAN information stored in FC routers, with and without LSAN zone binding . 495 Table 82 Fabric OS and M-EOSc interoperability compatibility matrix . . . . . . . . . . . . . . . 508 Table 83 Fabric OS and M-EOSn interoperability compatibility matrix .
xxx Fabric OS Administrator’s Guide 53-1002446-01
About This Document In this chapter • How this document is organized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxiii • Supported hardware and software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxiv • What’s new in this document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxv • Document conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxvi • Notice to the reader . . . . . . . . . . . . . . . .
• Chapter 11, “Administering Advanced Zoning,” provides procedures for use of the Brocade Advanced Zoning feature. • Chapter 12, “Traffic Isolation Zoning,” provides concepts and procedures for use of Traffic Isolation Zones within a fabric. • Chapter 13, “Bottleneck Detection,” describes how you can detect and configure alert thresholds for latency and congestion bottlenecks in the fabric.
The following hardware platforms are supported by this release of Fabric OS: • Fixed-port switches: - Brocade 300 switch - Brocade 5100 switch - Brocade 5300 switch - Brocade 5410 embedded switch - Brocade 5424 embedded switch - Brocade 5450 embedded switch - Brocade 5460 embedded switch - Brocade 5470 embedded switch - Brocade 5480 embedded switch - Brocade 6505 switch - Brocade 6510 switch - Brocade 7800 extension switch - Brocade 8000 FCoE switch - Brocade VA-40FC - Brocade Encryption Switch • Brocade D
Document conventions This section describes text formatting conventions and important notice formats used in this document.
NOTE A note provides a tip, guidance or advice, emphasizes important information, or provides a reference to related information. ATTENTION An Attention statement indicates potential damage to hardware or data. CAUTION A Caution statement alerts you to situations that can be potentially hazardous to you or cause damage to hardware, firmware, software, or data. DANGER A Danger statement indicates conditions or situations that can be potentially lethal or extremely hazardous to you.
Additional information This section lists additional Brocade and industry-specific documentation that you might find helpful. Brocade resources To get up-to-the-minute information, go to http://my.brocade.com and register at no cost for a user ID and password. For practical discussions about SAN design, implementation, and maintenance, you can obtain Building SANs with Brocade Fabric Switches through: http://www.amazon.
• syslog message logs 2. Switch Serial Number The switch serial number and corresponding bar code are provided on the serial number label, as illustrated below.: *FT00X0054E9* FT00X0054E9 The serial number label is located as follows: • Brocade 5424 — On the bottom of the switch module. • Brocade 300, 5100, and 5300 — On the switch ID pull-out tab located on the bottom of the port side of the switch. • Brocade 6510 — On the switch ID pull-out tab located inside the chassis on the port side on the left.
xl Fabric OS Administrator’s Guide 53-1002446-01
Section Standard Features I This section describes standard Fabric OS features, and includes the following chapters: • Chapter 1, “Understanding Fibre Channel Services” • Chapter 2, “Performing Basic Configuration Tasks” • Chapter 3, “Performing Advanced Configuration Tasks” • Chapter 4, “Routing Traffic” • Chapter 5, “Managing User Accounts” • Chapter 6, “Configuring Protocols” • Chapter 7, “Configuring Security Policies” • Chapter 8, “Maintaining the Switch Configuration File” • Chapter 9, “Installing
2 Fabric OS Administrator’s Guide 53-1002446-01
Chapter Understanding Fibre Channel Services 1 In this chapter • Fibre Channel services overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 • Management server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 • Platform services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 • Management server database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1 Management server Management server — The management server provides a single point for managing the fabric. This is the only service that users can configure. Alias server — The alias server keeps a group of nodes registered as one name to handle multicast groups. Broadcast server — The broadcast server is optional. When frames are transmitted to this address, they are broadcast to all operational N_ and NL_Ports.
Platform services 1 Platform services and Virtual Fabrics Each logical switch has a separate Platform Database. All platform registrations done to a logical switch are valid only in that particular logical switch’s Virtual Fabric. Activating the platform services on a switch activates the platform services on all logical switches in a Virtual Fabric. Similarly, deactivating the platform services deactivates the platform service on all logical switches in a Virtual Fabric.
1 Management server database Management server database You can control access to the management server database. An access control list (ACL) of WWN addresses determines which systems have access to the management server database. The ACL typically contains those WWNs of host systems that are running management applications. If the list is empty (the default), the management server is accessible to all systems connected in-band to the fabric.
Management server database 1 6. After verifying that the WWN was added correctly, enter 0 at the prompt to end the session. 7. At the “Update the FLASH?” prompt, enter y. 8. Press Enter to update the nonvolatile memory and end the session. Example of adding a member to the management server ACL switch:admin> msconfigure 0 Done 1 Display the access list 2 Add member based on its Port/Node WWN 3 Delete member based on its Port/Node WWN select : (0..
1 Management server database 4. At the “Port/Node WWN” prompt, enter the WWN of the member to be deleted from the ACL. 5. At the “select” prompt, enter 1 to display the access list so you can verify that the WWN you entered was deleted from the ACL. 6. After verifying that the WWN was deleted correctly, enter 0 at the “select” prompt to end the session. 7. At the “Update the FLASH?” prompt, enter y. 8. Press Enter to update the nonvolatile memory and end the session.
Topology discovery 1 Number of Associated Node Names: 1 Associated Node Names: 10:00:00:60:69:20:15:75 Clearing the management server database NOTE The command msPlClearDB is allowed only in AD0 and AD255. 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the msplClearDb command. 3. Enter y to confirm the deletion. The management server platform database is cleared.
1 Device login Disabling topology discovery Topology discovery is disabled by default. NOTE Disabling discovery of management server topology might erase all node ID entries. If Admin Domains are enabled, you must be in the AD0 or AD255 context. Refer to Chapter 17, “Managing Administrative Domains,” for additional information. 1. Connect to the switch and log in using an account with admin permissions. 2.
Device login 1 Principal switch In a fabric with multiple switches, and one inter-switch link (ISL) exists between any two switches, a principal switch is automatically elected. The principal switch provides the following capabilities: • Maintains time for the entire fabric. Subordinate switches synchronize their time with the principal switch. Changes to the clock server value on the principal switch are propagated to all switches in the fabric. • Manages domain ID assignment within the fabric.
1 Device login • F_Port — A fabric port is assigned to fabric-capable devices, such as SAN storage devices. • EX_Port — A type of E_Port that connects a Fibre Channel router to an edge fabric. From the point of view of a switch in an edge fabric, an EX_Port appears as a normal E_Port. It follows applicable Fibre Channel standards as other E_Ports. However, the router terminates EX_Ports rather than allowing different fabrics to merge as would happen on a switch with regular E_Ports.
High availability of daemon processes 1 Duplicate Port World Wide Name According to Fibre Channel standards, the Port World Wide Name (PWWN) of a device cannot overlap with that of another device, thus having duplicate PWWNs within the same fabric is an illegal configuration. If a PWWN conflict occurs with two devices attached to the same domain, Fabric OS handles device login in such a way that only one device may be logged in to the fabric at a time.
1 High availability of daemon processes TABLE 1 14 Daemons that are automatically restarted (Continued) Daemon Description webd Webserver daemon used for WebTools (includes httpd as well). weblinkerd Weblinker daemon provides an HTTP interface to manageability applications for switch management and fabric discovery.
Chapter 2 Performing Basic Configuration Tasks In this chapter • Fabric OS overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Fabric OS command line interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Password modification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • The Ethernet interface on your switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Date and time settings .
2 Fabric OS command line interface Although many different software and hardware configurations are tested and supported by Brocade Communications Systems, Inc., documenting all possible configurations and scenarios is beyond the scope of this document. In some cases, earlier releases are highlighted to present considerations for interoperating with them. The hardware reference manuals for Brocade products describe how to power up devices and set their IP addresses.
Fabric OS command line interface TABLE 2 2 Terminal port parameters (Continued) Parameter Value Stop bits 1 Flow control None • In a UNIX environment, enter the following string at the prompt: tip /dev/ttyb -9600 If ttyb is already in use, use ttya instead and enter the following string at the prompt: tip /dev/ttya -9600 Telnet or SSH sessions Connect to the Fabric OS through a Telnet or SSH connection or through a console session on the serial port.
2 Fabric OS command line interface Switches in the fabric that are not connected through the Ethernet port can be managed through switches that are using IP over Fibre Channel. The embedded port must have an assigned IP address. 3. Log off the switch’s serial port. 4. From a management station, open a Telnet connection using the IP address of the switch to which you want to connect. The login prompt is displayed when the Telnet connection finds the switch in the network. 5.
Password modification 2 Password modification The switch automatically prompts you to change the default account passwords after logging in for the first time. If you do not change the passwords, the switch prompts you after each subsequent login until all the default passwords have been changed. NOTE The default account passwords can be changed from their original values only when prompted immediately following the login; the passwords cannot be changed using the passwd command later in the session.
2 The Ethernet interface on your switch The Ethernet interface on your switch The Ethernet (network) interface provides management access, including direct access to the Fabric OS CLI, and allows other tools, such as Web Tools, to interact with the switch. You can use either Dynamic Host Configuration Protocol (DHCP) or static IP addresses for the Ethernet network interface configuration.
The Ethernet interface on your switch 2 Displaying the network interface settings If an IP address has not been assigned to the network interface (Ethernet), you must connect to the Fabric OS CLI using a console session on the serial port. For more information, see “Console sessions using the serial port” on page 16. Otherwise, connect using SSH. 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the ipAddrShow command.
2 The Ethernet interface on your switch Static Ethernet addresses Use static Ethernet network interface addresses on Brocade DCX and DCX-4S Backbones, and in environments where DHCP service is not available. To use static addresses for the Ethernet interface, you must first disable DHCP. You can enter static Ethernet information and disable DHCP at the same time. For more information, refer to “DHCP activation” on page 23.
The Ethernet interface on your switch 2 Setting the static addresses for the chassis management IP interface 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the ipAddrSet -chassis command. switch:admin> ipaddrset -chassis Ethernet IP Address [192.168.166.148]: Ethernet Subnetmask [255.255.255.0]: Committing configuration...Done. 3. Enter the network information in dotted-decimal notation for the Ethernet IPv4 address or in semicolon-separated notation for IPv6.
2 The Ethernet interface on your switch 4. Enable DHCP by entering on. switch:admin> ipaddrset Ethernet IP Address [10.1.2.3]: Ethernet Subnetmask [255.255.255.0]: Fibre Channel IP Address [220.220.220.2]: Fibre Channel Subnetmask [255.255.0.0]: Gateway IP Address [10.1.2.1]: DHCP [Off]:on Disabling DHCP When you disable DHCP, enter the static Ethernet IP address and subnet mask of the switch and default gateway address.
Date and time settings 2 When IPv6 autoconfiguration is enabled, the platform engages in stateless IPv6 autoconfiguration. When IPv6 autoconfiguration is disabled, the platform relinquishes usage of any autoconfigured IPv6 addresses that it may have acquired while it was enabled.
2 Date and time settings • yy is the year, valid values are 00 through 37 and 70 through 99 (year values from 70 through 99 are interpreted as 1970 through 1999, year values from 00 through 37 are interpreted as 2000 through 2037). Example of showing and setting the date switch:admin> date Fri Sep 29 17:01:48 UTC 2007 Stealth200E:admin> date "0204101008" Mon Feb 4 10:10:00 UTC 2008 Time zone settings You can set the time zone for a switch by name.
Date and time settings 2 • Use tsTimeZone with no parameters to display the current time zone setting. • Use --interactive to list all of the time zones supported by the firmware. • Use timeZone_fmt to set the time zone by Country/City or by time zone ID, such as Pacific Standard Time (PST).
2 Domain IDs Synchronizing the local time with an external source The tsClockServer command accepts multiple server addresses in IPv4, IPv6, or Domain Name System (DNS) name formats. When multiple NTP server addresses are passed, tsClockServer sets the first obtainable address as the active NTP server. The rest are stored as backup servers that can take over if the active NTP server fails. The principal or primary FCS switch synchronizes its time with the NTP server every 64 seconds. 1.
Domain IDs 2 Displaying the domain IDs 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the fabricShow command. Example output of fabric information, including the domain ID (D_ID) The principal switch is determined by the arrow ( > ) next to the name of the switch. In this output, the principal switch appears in blue boldface.
2 Switch names 3. Enter the configure command. 4. Enter y after the Fabric Parameters prompt. Fabric parameters (yes, y, no, n): [no] y 5. Enter a unique domain ID at the Domain prompt. Use a domain ID value from 1 through 239 for normal operating mode (FCSW-compatible). Domain: (1..239) [1] 3 6. Respond to the remaining prompts, or press Ctrl-D to accept the other settings and exit. 7. Enter the switchEnable command to re-enable the switch.
Fabric name 2 Fabric name You can assign a alphanumeric name to identify and manage a logical fabric that formerly could only be identified by a fabric ID. The fabric name does not replace the fabric ID or its usage. The fabric continues to have a fabric ID, in addition to the assigned alphanumeric fabric name. Note the considerations: • Each name must be unique for each logical switch within a chassis; duplicate fabric names are not allowed.
2 Switch activation and deactivation Switch activation and deactivation By default, the switch is enabled after power is applied and diagnostics and switch initialization routines have finished. You can disable and re-enable the switch as necessary. Disabling a switch 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the switchDisable command. All Fibre Channel ports on the switch are taken offline. If the switch is part of a fabric, the fabric is reconfigured.
Basic connections 2 The system is halted flushing ide devices: hda Power down. 5. Power off the switch. Powering off a Brocade Backbone 1. From the active CP in a dual-CP platform, enter the sysShutdown command. NOTE When the sysShutdown command is issued on the active CP, the active CP, the standby CP, and any application blades are all shut down. 2. Enter y at the prompt. 3.
2 Basic connections Switch connection See the hardware reference manual of your specific switch for ISL connection and cable management information. The standard or default ISL mode is L0. ISL mode L0 is a static mode, with the following maximum ISL distances: • • • • • • 10 km at 1 Gbps 5 km at 2 Gbps 2.
Chapter 3 Performing Advanced Configuration Tasks In this chapter • PIDs and PID binding overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Blade terminology and compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Enabling and disabling blades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3 PIDs and PID binding overview Core PID addressing mode Core PID is the default PID format for Brocade platforms. It uses the entire 24-bit address space of the domain, area ID, and AL_PA to determine an object’s address within the fabric.
PIDs and PID binding overview 3 • Shared area limitations are removed on 48-port and 64-port blades. • Any port on a 48-port or 64-port blade can support up to 256 NPIV devices (in fixed addressing mode, only 128 NPIV devices are supported in non-VF mode and 64 NPIV devices in VF mode on a 48-port blade). • Any port on a 48-port blade can support loop devices. • Any port on a 48-port or 64-port blade can support hard port zoning. • Port index is not guaranteed to be equal to the port area ID.
3 PIDs and PID binding overview Virtual Fabrics considerations for WWN-based PID assignment WWN-based PID assignment is disabled by default and is supported in the default switch on the Brocade DCX and DCX 8510 Backbone families. This feature is not supported on application blades such as the FS8-18, FX8-24, and the FCOE10-24. The total number of ports in the default switch must be 256 or less.
Ports 3 Clearing PID binding 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the wwnAddress -unbind command to clear the PID binding for the specified WWN. Showing PID assignments 1. Connect to the switch and log in using an account with admin permissions. 2. Based on what you want to display, enter the appropriate command: • wwnAddress –show displays the assigned WWN-PID bindings. • wwnAddress –findPID wwn displays the PID assigned to the device WWN specified.
3 Ports When you have port blades with different port counts in the same Backbone (for example, 16-port blades and 32-port blades, or 16-port blades and 18-port blades with 16 FC ports and 2 GbE ports, or 16-port and 48-port blades), the area IDs no longer match the port numbers. Table 6 on page 45 lists the port numbering schemes for the blades. Setting port names Perform the following steps to specify a port name. For Backbones, specify the slot number where the blade is installed. 1.
Ports 3 Port identification by index With the introduction of 48-port blades, indexing was introduced. Unique area IDs are possible for up to 255 areas, but beyond that there needed to be some way to ensure uniqueness. A number of fabric-wide databases supported by Fabric OS (including ZoneDB, the ACL DDC, and Admin Domain) allow a port to be designated by the use of a “D,P” (domain,port) notation.
3 Ports Port activation and deactivation By default, all licensed ports are enabled. You can disable and re-enable them as necessary. Ports that you activate with the Ports on Demand license must be enabled explicitly, as described in “Ports on Demand” on page 385. If ports are persistently disabled and you use the portEnable command to enable a disabled port, the port will revert to being disabled after a power cycle or a switch reboot.
Ports 3 Port decommissioning Fabric OS 7.0.0 and later provides an automated mechanism to remove an E_Port or E_Port trunk port from use. This feature identifies the target port and communicates the intention to decommission the port to those systems within the fabric affected by the action. Each affected system can agree or disagree with the action, and these responses are automatically collected before a port is decommissioned.
3 Blade terminology and compatibility The following example sets the speed for all ports on the switch to autonegotiate: switch:admin> switchcfgspeed 0 Committing configuration...done. Setting port speed for a port octet You can use the portCfgOctetSpeedCombo command to configure the speed for a port octet. Note that in a Virtual Fabrics environment, this command applies chassis-wide and not just to the logical switch. 1. Connect to the switch and log in using an account with admin permissions. 2.
Blade terminology and compatibility TABLE 6 3 Port blade terminology, numbering, and platform support Supported on: Blade Blade ID DCX (slotshow) family DCX 8510 family Ports Definition FC8-161 21 Yes No 16 8-Gbps port blade supporting 1, 2, 4, and 8 Gbps port speeds. Ports are numbered from 0 through 15 from bottom to top. FC8-321 55 Yes No 32 8-Gbps port blade supporting 1, 2, 4, and 8 Gbps port speeds.
3 Blade terminology and compatibility TABLE 6 Port blade terminology, numbering, and platform support (Continued) Supported on: Blade Blade ID DCX (slotshow) family DCX 8510 family Ports Definition FC16-48 96 Yes 48 A 48-port, 16-Gbps port blade supporting 2, 4, 8, 10, and 16 Gbps port speeds. No NOTE: 10 Gbps speed for FC16-xx blades requires the 10G license.
Blade terminology and compatibility 3 CP blades The control processor (CP) blade provides redundancy and acts as the main controller on the Brocade Backbone. The Brocade DCX and DCX 8510 Backbone families support the CP8 blades. The CP blades in the Brocade DCX and DCX 8510 Backbone families are hot-swappable. The CP8 blades are fully interchangeable among Brocade DCX, DCX-4S, DCX 8510-4, and DCX 8510-8 Backbones.
3 Enabling and disabling blades TABLE 7 Blade compatibility within Brocade Backbone families Intelligent blade Fabric OS v6.3.0 Fabric OS v6.4.0 Fabric OS v7.0.0 DCX DCX-4S DCX DCX-4S DCX DCX-4S DCX 8510-8 DCX 8510-4 FR4-18i1 8 4 8 4 8 4 0 0 FS8-18 4 4 4 4 4 4 4 4 FCOE10-242 2 2 2 2 4 4 0 0 2 4 4 4 4 4 4 4 FX8-24 3 1. The iSCSI function over FCIP is not supported, but the FCIP link is the same as other FC E_Ports. This is not restricted by software. 2.
Enabling and disabling blades 3 Enabling blades 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the bladeEnable command with the slot number of the port blade you want to enable. ecp:admin> bladeenable 3 Slot 3 is being enabled FC8-48, FC8-48E, FC8-64, and FC16-48 port blade enabling exceptions Because the area IDs are shared with different port IDs, the FC8-48, FC8-48E, FC8-64, and FC16-48 blades support only F_ and E_Ports. They do not support FL_Ports.
3 Blade swapping • When an FR4-18i blade is replaced by an FC8-16, FC8-32, FC8-48, or FC8-64 blade, then the EX_Port configuration is retained, but the ports are persistently disabled. All remaining port configurations are retained. NOTE The FC10-6 blade does not support EX_Ports. Disabling blades 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the bladeDisable command with the slot number of the port blade you want to disable.
Blade swapping 3 How blades are swapped The bladeSwap command performs the following operations: 1. Blade selection The selection process includes selecting the switch and the blades to be affected by the swap operation. Figure 2 shows the source and destination blades identified to begin the process. FIGURE 2 Identifying the blades 2. Blade validation The validation process includes determining the compatibility between the blades selected for the swap operation: • Blade technology.
3 Blade swapping FIGURE 3 Blade swap with Virtual Fabrics during the swap 4. Port swapping The swap ports action is effectively an iteration of the portSwap command for each port on the source blade to each corresponding port on the destination blade. In Figure 4 shows Virtual Fabrics, where the blades can be carved up into different logical switches as long as they are carved the same way. If slot 1 and slot 2 ports 0-7 are all in the same logical switch, then blade swapping slot 1 to slot 2 will work.
Power management 3 3. Once the command completes successfully, move the cables from the source blade to the destination blade. 4. Enter the bladeEnable command on the destination blade to enable all user ports. Power management All blades are powered on by default when the switch chassis is powered on. Blades cannot be powered off when POST or AP initialization is in progress.
3 Equipment status Equipment status You can check the status of switch operation, High Availability features, and fabric connectivity. Checking switch operation 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the switchShow command. This command displays a switch summary and a port summary. 3. Check that the switch and ports are online. 4. Use the switchStatusShow command to further check the status of the switch.
Track and control switch changes 3 Verifying fabric connectivity 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the fabricShow command. This command displays a summary of all the switches in the fabric. The output of the fabricShow command is discussed in “Domain IDs” on page 28. Verifying device connectivity 1. Connect to the switch and log in using an account with admin permissions. 2.
3 Track and control switch changes Enabling the track changes feature 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the trackChangesSet 1 command to enable the track changes feature. A message displays, verifying that the track changes feature is on: switch:admin> trackchangesset 1 Committing configuration...done. 3. View the log using the commands errDump |more to display a page at a time or errShow to view one line at a time.
Track and control switch changes WWN CP Blade CoreBlade Flash MarginalPorts FaultyPorts MissingSFPs ErrorPorts Number of ports: 4 0 0 0 0 0 0.00%[0] 0.00%[0] 0.00%[0] 0.00%[0] 3 0 0 0 0 0 0.00%[0] 0.00%[0] 0.00%[0] 0.00%[0] Setting the switch status policy threshold values 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the switchStatusPolicySet command. The current switch status policy parameter values are displayed.
3 Audit log configuration Bad PowerSupplies contributing to MARGINAL status: (0..2) [1] Bad Temperatures contributing to DOWN status: (0..4) [2]1 Bad Temperatures contributing to MARGINAL status: (0..4) [1]2 Bad Fans contributing to DOWN status: (0..2) [2] Bad Fans contributing to MARGINAL status: (0..2) [1] (output truncated) On the Brocade Backbones, the command output includes parameters related to CP blades.
Audit log configuration 3 NOTE Only the active CP can generate audit messages because event classes being audited occur only on the active CP. Audit messages cannot originate from other blades in a Backbone. Switch names are logged for switch components and Backbone names for Backbone components. For example, a Backbone name may be FWDL or RAS and a switch component name may be zone, name server, or SNMP. Pushed messages contain the administrative domain of the entity that generated the event.
3 Configuring FLOGI-time handling of duplicate PWWN 4. Enter the auditCfg --show command to view the filter configuration and confirm that the correct event classes are being audited, and the correct filter state appears (enabled or disabled). switch:admin> auditcfg --show Audit filter is enabled. 2-SECURITY 4-FIRMWARE 5. Issue the auditDump -s command to confirm that the audit messages are being generated. Example of the syslog (system message log) output for audit logging Oct 10 08:52:06 10.3.220.
Configuring FLOGI-time handling of duplicate PWWN TABLE 9 Duplicate PWWN behavior: Second login overrides existing login Input port Duplicate found on same F_Port FLOGI received 1 2 Duplicate found on different F_Port Implicit logout. 1 Send FLOGI ACC. 2 3 4 FDISC received 3 N/A 1 2 3 4 Duplicate found on same NPIV port Duplicate found on different NPIV port Logout different F-port. N/A Persistently disable different F-port. RASLog Duplicate Found. FLOGI ACC to Input port.
3 62 Configuring FLOGI-time handling of duplicate PWWN Fabric OS Administrator’s Guide 53-1002446-01
Chapter 4 Routing Traffic In this chapter • Routing overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Inter-switch links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Gateway links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Routing policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4 Routing overview Paths and route selection Paths are possible ways to get from one switch to another. Each inter-switch link (ISL) has a metric cost based on bandwidth. The cumulative cost is based on the sum of all costs of all traversed ISLs. Route selection is the path that is chosen. Paths that are selected from the routing database are chosen based on the minimal cost.
Routing overview 4 NOTE FSPF only supports 16 routes in a zone, including Traffic Isolation Zones. FSPF makes minimal use of the ISL bandwidth, leaving virtually all of it available for traffic. In a stable fabric, a switch transmits 64 bytes every 20 seconds in each direction. FSPF frames have the highest priority in the fabric. This guarantees that a control frame is not delayed by user data and that FSPF routing decisions occur very quickly during convergence.
4 Inter-switch links Inter-switch links An inter-switch link (ISL) is a link between two switches, E_Port-to-E_Port. The ports of the two switches automatically come online as E_Ports once the login process finishes successfully. For more information on the login process, refer to Chapter 1, “Understanding Fibre Channel Services”. FIGURE 6 New switch added to existing fabric You can expand your fabric by connecting new switches to existing switches.
Inter-switch links 4 Buffer credits In order to prevent the dropping of frames in the fabric, a device can never send frames without the receiving device being able to receive them, so an end-to-end flow control is used on the switch. Flow control in Fibre Channel uses buffer-to-buffer credits, which are distributed by the switch. When all buffer-to-buffer credits are utilized, a device waits for a VC_RDY or an R_RDY primitive from the destination switch before resuming I/O.
4 Gateway links FIGURE 7 Virtual channels on a QoS-enabled ISL Gateway links A gateway merges SANs into a single fabric by establishing point-to-point E_Port connectivity between two Fibre Channel switches that are separated by a network with a protocol such as IP or SONET. Except for link initialization, gateways are transparent to switches; the gateway simply provides E_Port connectivity from one switch to another. Figure 8 shows two separate SANs, A-1 and A-2, merged together using a gateway.
Gateway links FIGURE 8 4 Gateway link merging SANs By default, switch ports initialize links using the Exchange Link Parameters (ELP) mode 1. However, gateways expect initialization with ELP mode 2, also referred to as ISL R_RDY mode. Therefore, to enable two switches to link through a gateway, the ports on both switches must be set for ELP mode 2.
4 Routing policies Example of enabling a gateway link on slot 2, port 3 ecp:admin> portcfgislmode 2/3, 1 Committing configuration...done. ISL R_RDY Mode is enabled for port 3. Please make sure the PID formats are consistent across the entire fabric. Routing policies By default, all routing protocols place their routes into a routing table.
Routing policies 4 Exchange-based routing The choice of routing path is based on the Source ID (SID), Destination ID (DID), and Fibre Channel originator exchange ID (OXID) optimizing path utilization for the best performance. Thus, every exchange can take a different path through the fabric. Exchange-based routing requires the use of the Dynamic Load Sharing (DLS) feature; when this policy is in effect, you cannot disable the DLS feature.
4 Route selection Routing in Virtual Fabrics Virtual Fabrics support DPS on all partitions. DPS is limited where multiple paths are available for a logical fabric frame entering a Virtual Fabrics chassis from a base fabric that is sent out using one of the dedicated ISLs in a logical switch. The AP policy affecting the DPS behavior, whether it is exchange-based, device-based, or port-based, is configured on a per-logical switch basis.
Route selection 4 Dynamic Load Sharing The exchange-based routing policy depends on the Fabric OS Dynamic Load Sharing (DLS) feature for dynamic routing path selection. When using the exchange-based routing policy, DLS is enabled by default and cannot be disabled. In other words, you cannot enable or disable DLS when the exchange-based routing policy is in effect. When the port-based policy is in force, you can enable DLS to optimize routing.
4 Frame order delivery Frame order delivery The order of delivery of frames is maintained within a switch and determined by the routing policy in effect. The frame delivery behaviors for each routing policy are: • Port-based routing All frames received on an incoming port destined for a destination domain are guaranteed to exit the switch in the same order in which they were received.
Lossless Dynamic Load Sharing on ports 4 Lossless Dynamic Load Sharing on ports Lossless Dynamic Load Sharing (DLS) allows you to rebalance port paths without causing input/output (I/O) failures. For devices where in-order delivery (IOD) of frames is required, you can set IOD separately.
4 Lossless Dynamic Load Sharing on ports Lossless DLS does the following whenever paths need to be rebalanced: 1. Pauses ingress traffic by not returning credits. Frames that are already in transit are not dropped. 2. Changes the existing path to a more optimal path. 3. If IOD is enabled, waits for sufficient time for frames already received to be transmitted. This is needed to maintain IOD. 4. Resumes traffic. Table 10 shows the effect of frames when you have a specific routing policy turned on with IOD.
Enabling forward error correction 4 Configuring Lossless Dynamic Load Sharing You configure Lossless DLS switch- or chassis-wide by using the dlsSet command to specify that no frames are dropped while rebalancing or rerouting traffic. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the appropriate dlsSet command to enable or disable Lossless Dynamic Load Sharing.
4 Frame Redirection FEC is useful when broadcasting data to many destinations simultaneously from a single source, when retransmissions might be costly. Use the portCfgFec command to enable and disable FEC on a port, as shown in the following examples. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the portcfgfec command, specifying the port or range of ports on which FEC is to be enabled.
Frame Redirection FIGURE 9 4 Single host and target Figure 9 demonstrates the flow of Frame Redirection traffic. A frame starts at the host with a destination to the target. The port where the appliance is attached to the host switch acts as the virtual initiator and the port where the appliance is attached to the target switch is the virtual target.
4 Frame Redirection Viewing redirect zones 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the cfgShow command.
Chapter 5 Managing User Accounts In this chapter • User accounts overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Local database user accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Local account database distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Password policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • The boot PROM password . . . . . . .
5 User accounts overview Fabric OS provides three options for authenticating users—remote RADIUS services, remote LDAP service, and the local switch user database. All options allow users to be centrally managed using the following methods: • Remote RADIUS server: Users are managed in a remote RADIUS server. All switches in the fabric can be configured to authenticate against the centralized remote database. • Remote LDAP server: Users are managed in a remote LDAP server.
User accounts overview 5 If some Admin Domains have been defined for the user and all of them are inactive, the user will not be allowed to log in to any switch in the fabric. If no Home Domain is specified for a user, the system provides a default home domain. The default home domain for the predefined account is AD0. For user-defined accounts, the default home domain is the Admin Domain in the user’s Admin Domain list with the lowest ID.
5 User accounts overview TABLE 13 Maximum number of simultaneous sessions Role name Maximum sessions Admin 2 BasicSwitchAdmin 4 FabricAdmin 4 Operator 4 SecurityAdmin 4 SwitchAdmin 4 User 4 ZoneAdmin 4 Managing user-defined roles Fabric OS provides an extensive toolset for managing user defined roles: • The roleConfig command is available for defining new roles, deleting created roles, or viewing information about user-defined roles.
Local database user accounts 5 > classConfig --showroles security Roles that have access to RBAC Class ‘security’ are: Role Name --------User Admin Factory Root SwitchAdmin FabricAdmin BasicSwitchAdmin SecurityAdmin mysecurityrole Permissions ----------O OM OM OM O OM O OM O To delete a user-defined role, use the roleConfig --delete command.
5 Local database user accounts TABLE 14 Default local user accounts Account name Role Admin Domain Logical Fabric Description admin Admin AD0-255 home: 0 LF1-128 home: 128 Most commands have observe-modify permission. factory Factory AD0-255 home: 0 LF1-128 home: 128 Reserved. root Root AD0-255 home: 0 LF1-128 home: 128 Reserved. user User AD0 home: 0 LF-128 home: 128 Most commands have observe-only permission.
Local database user accounts 5 Deleting an account This procedure can be performed on local user accounts. 1. Connect to the switch and log in using an account with admin permissions, or an account associated with a user-defined role with permissions for the UserManagement class of commands. 2. Enter the userConfig --delete command. NOTE You cannot delete the default accounts. An account cannot delete itself. All active CLI sessions for the deleted account are logged out. 3.
5 Local account database distribution Changing the password for a different account 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the passwd command specifying the name of the account for which the password is being changed. 3. Enter the requested information at the prompts. Local account database distribution Fabric OS allows you to distribute the user database and passwords to other switches in the fabric.
Password policies 5 Rejecting distributed user databases on the local switch 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the fddCfg --localreject PWD command. Password policies The password policies described in this section apply to the local switch user database only. Configured password policies (and all user account attribute and password state information) are synchronized across CPs and remain unchanged after an HA failover.
5 Password policies • Punctuation Specifies the minimum number of punctuation characters that must appear in the password. All printable, non-alphanumeric punctuation characters except the colon ( : ) are allowed. The default value is zero. The maximum value must be less than or equal to the MinLength value. • MinLength Specifies the minimum length of the password. The minimum can be from 8 to 40 characters. New passwords must be between the minimum length specified and 40 characters.
Password policies 5 Password expiration policy The password expiration policy forces expiration of a password after a configurable period of time, and is enforced across all user accounts. A warning that password expiration is approaching is displayed when the user logs in. When a user’s password expires, he or she must change the password to complete the authentication process and open a user session. You can specify the number of days prior to password expiration during which warnings will commence.
5 Password policies The following commands are used to manage the account lockout policy. • userConfig • passwdCfg --change account_name -u --disableadminlockout Note that the account-locked state is distinct from the account-disabled state. Use the following attributes to set the account lockout policy: • LockoutThreshold Specifies the number of times a user can attempt to log in using an incorrect password before the account is locked.
The boot PROM password 5 The boot PROM password The boot PROM password provides an additional layer of security by protecting the boot PROM from unauthorized use. Setting a recovery string for the boot PROM password enables you to recover a lost boot PROM password by contacting your switch service provider. Without the recovery string, a lost boot PROM password cannot be recovered.
5 The boot PROM password 5. Enter the recovery password (string). The recovery string must be between 8 and 40 alphanumeric characters. A random string that is 15 characters or longer is recommended for higher security. The firmware prompts for this password only once. It is not necessary to remember the recovery string because it is displayed the next time you enter the command shell. The following prompt displays: New password: 6. Enter the boot PROM password, then re-enter it when prompted.
The boot PROM password 5 6. Enter the recovery password (string). The recovery string must be between 8 and 40 alphanumeric characters. A random string that is 15 characters or longer is recommended for higher security. The firmware only prompts for this password once. It is not necessary to remember the recovery string because it is displayed the next time you enter the command shell. The following prompt displays: New password: 7. Enter the boot PROM password, then re-enter it when prompted.
5 The boot PROM password 5. At the shell prompt, enter the passwd command. NOTE The passwd command only applies to the boot PROM password when it is entered from the boot interface. 6. Enter the boot PROM password at the prompt, then re-enter it when prompted. The password must be eight alphanumeric characters (any additional characters are not recorded). Record this password for future use. 7. Enter the saveEnv command to save the new password. 8. Reboot the switch by entering the reset command.
The authentication model using RADIUS and LDAP 5 9. Enter the saveEnv command to save the new password. 10. Reboot the standby CP blade by entering the reset command. 11. Connect to the active CP blade by serial or Telnet and enter the haEnable command to restore high availability; then fail over the active CP blade by entering the haFailover command. Traffic resumes flowing through the newly active CP blade after it has completed rebooting. 12.
5 The authentication model using RADIUS and LDAP To enable the secure LDAP service, you need to install a certificate from the Microsoft Active Directory server. By default, the LDAP service does not require certificates. The configuration applies to all switches and on a Backbone the configuration replicates itself on a standby CP blade if one is present. It is saved in a configuration upload and applied in a configuration download.
5 The authentication model using RADIUS and LDAP TABLE 15 Authentication configuration options (Continued) aaaConfig options Description Equivalent setting in Fabric OS v5.1.0 and earlier --radius --switchdb1 --authspec “ldap” Authenticates management connections against any LDAP databases only. If LDAP service is not available or the credentials do not match, the login fails. n/a n/a --authspec “ldap; local” Authenticates management connections against any LDAP databases first.
5 The authentication model using RADIUS and LDAP syntax error in the attributes, the password expiration warning will not be issued. If your RADIUS server maintains its own password expiration attributes, you must set the exact date twice to use this feature, once on your RADIUS server and once in the VSA attribute. If the dates do not match, then the RADIUS server authentication fails. The syntax used for assigning VSA-based account switch roles on a RADIUS server is described in Table 16.
The authentication model using RADIUS and LDAP FIGURE 10 5 Windows 2000 VSA configuration Linux FreeRadius server For the configuration on a Linux FreeRadius server, define the values outlined in Table 17 in a vendor dictionary file called dictionary.brocade. TABLE 17 Entries in dictionary.
5 The authentication model using RADIUS and LDAP RADIUS configuration with Admin Domains or Virtual Fabrics When configuring users with Admin Domains or Virtual Fabrics, you must also include the Admin Domain or Virtual Fabric member list. This section describes the way that you configure attribute types for this configuration.
The authentication model using RADIUS and LDAP 5 In the next example, on a Linux FreeRadius Server, the user has the “zoneAdmin” permissions, with VFlist 2, 4, 5, 6, 7, 8, 10, 11, 12, 13, 15 17, 19, 22, 23, 24, 25, 29, 31 and HomeLF 1.
5 The authentication model using RADIUS and LDAP # attributes # ATTRIBUTE ATTRIBUTE ATTRIBUTE ATTRIBUTE ATTRIBUTE ATTRIBUTE ATTRIBUTE Brocade-Auth-Role Brocade-AVPairs1 Brocade-AVPairs2 Brocade-AVPairs3 Brocade-AVPairs4 Brocade-Passwd-ExpiryDate Brocade-Passwd-WarnPeriod 1 2 3 4 5 6 7 string string string string string string string Brocade Brocade Brocade Brocade Brocade Brocade Brocade This defines the Brocade vendor ID as 1588, the Brocade attribute 1 as Brocade-Auth-Role and 6 as Brocade-Passwd-E
The authentication model using RADIUS and LDAP 5 Enabling clients Clients are the switches that will use the RADIUS server; each client must be defined. By default, all IP addresses are blocked. The Brocade Backbones send their RADIUS requests using the IP address of the active CP. When adding clients, add both the active and standby CP IP addresses so that, in the event of a failover, users can still log in to the switch. 1. Open the $PREFIX/etc/raddb/client.
5 The authentication model using RADIUS and LDAP 3. Configuring a user IAS is the Microsoft implementation of a RADIUS server and proxy. IAS uses the Windows native user database to verify user login credentials; it does not list specific users, but instead lists user groups. Each user group should be associated with specific switch role.
The authentication model using RADIUS and LDAP 5 RSA SecurID with an RSA RADIUS server is used for user authentication. The Brocade switch does not communicate directly with the RSA Authentication Manager, so the RSA RADIUS server is used in conjunction with the switch to facilitate communication. To learn more about how RSA SecurID works, visit www.rsa.com for more information.
5 The authentication model using RADIUS and LDAP ########################################################################### # brocade.dct -- Brocade Dictionary # # (See readme.dct for more details on the format of this file) ########################################################################### # # Use the Radius specification attributes in lieu of the Brocade one: # @radius.
The authentication model using RADIUS and LDAP 5 d. Add the Brocade profile. e. In RSA Authentication Manager, edit the user records that will be authenticating using RSA SecurID. LDAP configuration and Microsoft Active Directory LDAP provides user authentication and authorization using the Microsoft Active Directory service in conjunction with LDAP on the switch. There are two modes of operation in LDAP authentication, FIPS mode and non-FIPS mode.
5 The authentication model using RADIUS and LDAP For instructions on how to create a user, refer to www.microsoft.com or Microsoft documentation to create a user in your Active Directory. 3. Create a group name that uses the switch’s role name so that the Active Directory group’s name is the same as the switch’s role name. or Use the ldapCfg -–maprole ldap_role_name switch_role command to map an LDAP server role to one of the default roles available on the switch. 4.
The authentication model using RADIUS and LDAP 5 Adding an Admin Domain or Virtual Fabric list 1. From the Windows Start menu, select Programs> Administrative Tools> ADSI.msc ADSI is a Microsoft Windows Resource Utility. This will need to be installed to proceed with the rest of the setup. For Windows 2003, this utility comes with Service Pack 1 or you can download this utility from the Microsoft website. 2. Go to CN=Users. 3. Right click on select Properties. Click the Attribute Editor tab. 4.
5 The authentication model using RADIUS and LDAP Authentication servers on the switch At least one RADIUS or LDAP server must be configured before you can enable RADIUS or LDAP service. You can configure the RADIUS or LDAP service even if it is disabled on the switch. You can configure up to five RADIUS or LDAP servers. You must be logged in as admin or switchAdmin to configure the RADIUS service.
The authentication model using RADIUS and LDAP 5 Changing a RADIUS or LDAP server configuration 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the aaaConfig --change command. Changing the order in which RADIUS or LDAP servers are contacted for service 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the aaaConfig --move command. When the command succeeds, the event log indicates that a server configuration is changed.
5 114 The authentication model using RADIUS and LDAP Fabric OS Administrator’s Guide 53-1002446-01
Chapter 6 Configuring Protocols In this chapter • Security protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Secure Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Secure Shell protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Secure Sockets Layer protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6 Secure Copy TABLE 18 Secure protocol support (Continued) Protocol Description SSH Secure Shell (SSH) is a network protocol that allows data to be exchanged over a secure channel between two computers. Encryption provides confidentiality and integrity of data. SSH uses public-key cryptography to authenticate the remote computer and allow the remote computer to authenticate the user, if necessary. SSL Fabric OS uses secure socket layer (SSL) to support HTTPS.
Secure Shell protocol 6 3. Type y or yes at the cfgload attributes prompt. 4. Type y or yes at the Enforce secure configUpload/Download prompt. Example of setting up SCP for configUpload/download switch:admin> configure Not all options will be available on an enabled switch. To disable the switch, use the "switchDisable" command. Configure...
6 Secure Shell protocol DSA, the authentication protocols are based on a pair of specially generated cryptographic keys, called the private key and the public key. The advantage of using these key-based authentication systems is that in many cases, it is possible to establish secure connections without having to depend on passwords for security. RSA asynchronous algorithms are FIPS-compliant. Incoming authentication is used when the remote host needs to authenticate to the switch.
Secure Shell protocol 6 Configuring outgoing SSH authentication After the allowed-user is configured, the remaining setup steps must be completed by the allowed-user. To configure outgoing authentication, follow these steps: 1. Log in to the switch as the default admin. 2. Change the allowed-user’s permissions to admin, if applicable.
6 Secure Sockets Layer protocol Deleting private keys on the switch 1. Log in to the switch as the allowed-user. 2. Use the sshUtil delprivkey command to delete the private key. For more information on IP Filter policies, refer to Chapter 7, “Configuring Security Policies”. Secure Sockets Layer protocol Secure sockets layer (SSL) protocol provides secure access to a fabric through Web-based management tools like Web Tools. SSL support is a standard Fabric OS feature.
Secure Sockets Layer protocol 6 Configuring for SSL involves these main steps, which are shown in detail in the next sections. 1. Choose a certificate authority (CA). 2. Generate the following items on each switch: a. A public and private key by using the secCertUtil genkey command. b. A certificate signing request (CSR) by using the secCertUtil gencsr command. 3. Store the CSR on a file server by using the secCertUtil export command. 4. Obtain the certificates from the CA.
6 Secure Sockets Layer protocol 3. Respond to the prompts to continue and select the key size. Example of generating a key Continue (yes, y, no, n): [no] y Select key size [1024 or 2048]: 1024 Generating new rsa public/private key pair Done. Because CA support for the 2048-bit key size is limited, you should select 1024 in most cases. Generating and storing a CSR After generating a public/private key, perform this procedure on each switch. 1.
Secure Sockets Layer protocol 6 Obtaining certificates Check the instructions on the CA website; then, perform this procedure for each switch. 1. Generate and store the CSR as described in “Generating and storing a CSR” on page 122. 2. Open a Web browser window on the management workstation and go to the CA website. Follow the instructions to request a certificate. Locate the area in the request form into which you are to paste the CSR. 3.
6 Secure Sockets Layer protocol Checking and installing root certificates on Internet Explorer 1. Select Tools > Internet Options. 2. Click the Content tab. 3. Click Certificates. 4. Click the Intermediate or Trusted Root tabs and scroll the list to see if the root certificate is listed. Take the appropriate following action based on whether you find the certificate: • If the certificate is listed, you do not need to install it. You can skip the rest of this procedure.
Simple Network Management Protocol 6 Example of installing a root certificate C:\Program Files\Java\j2re1.6.0\bin> keytool -import -alias RootCert -file RootCert.crt -keystore ..
6 Simple Network Management Protocol If you use both SW-MIB and FA-MIB, you may receive duplicate information. You can disable the FA-MIB, but not the SW-MIB. You can also use these additional MIBs and their associated traps: • FICON-MIB (for FICON environments) • SW-EXTTRAP Includes the swSsn (Software Serial Number) as a part of Brocade SW traps. For information on Brocade MIBs, see the Fabric OS MIB Reference.
Telnet protocol 6 Attributes that are specific to each logical switch belong to the switch category. These attributes are available in the Virtual Fabrics context and not available in the Chassis context. Attributes that are common across the logical switches belong to the chassis level. These attributes are accessible to users having the chassis-role permission. When a chassis table is queried the context is set to chassis context, if the user has the chassis-role permission.
6 Telnet protocol 5. Add a rule to the policy, by typing the ipFilter --addrule command. switch:admin> ipfilter --addrule BlockTelnet -rule 1 -sip any -dp 23 -proto tcp -act deny ATTENTION The rule number assigned has to precede the default rule number for this protocol. For example, in the defined policy, the Telnet rule number is 2, therefore to effectively block Telnet, the rule number to assign must be 1.
Listener applications 6 Unblocking Telnet 1. Connect to the switch through a serial port or SSH and log in as admin. 2. Type in the ipfilter --delete command. Refer to “Deleting a rule to an IP Filter policy” on page 159 for more information on deleting IP filter rules. 3. To permanently delete the policy, type the ipfilter --save command. ATTENTION If you deleted the rule to permit Telnet, you will need to add a rule to permit Telnet.
6 Ports and applications used by switches TABLE 23 Access defaults Access default Hosts Any host can access the fabric by SNMP. Any host can Telnet to any switch in the fabric. Any host can establish an HTTP connection to any switch in the fabric. Any host can establish an API connection to any switch in the fabric. Devices All devices can access the management server. Any device can connect to any FC port in the fabric. Switch access Any switch can join the fabric.
Chapter 7 Configuring Security Policies In this chapter • ACL policies overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • ACL policy management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • FCS policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • DCC policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7 ACL policy management Policies with the same state are grouped together in a Policy Set. Each switch has the following two sets: • Active policy set, which contains ACL policies being enforced by the switch. • Defined policy set, which contains a copy of all ACL policies on the switch. When a policy is activated, the defined policy either replaces the policy with the same name in the active set or becomes a new active policy.
ACL policy management 7 Displaying ACL policies You can view the active and defined policy sets at any time. Additionally, in a defined policy set, policies created in the same login session also appear but these policies are automatically deleted if the you log out without saving them. 1. Connect to the switch and log in using an account with admin permissions, or an account with O permission for the Security RBAC class of commands. 2. Type the secPolicyShow command.
7 ACL policy management Example of deleting an ACL policy switch:admin> secpolicydelete "DCC_POLICY_010" About to delete policy Finance_Policy. Are you sure (yes, y, no, n):[no] y Finance_Policy has been deleted. Adding a member to an existing ACL policy As soon as a policy has been activated, the aspect of the fabric managed by that policy is enforced. 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the Security RBAC class of commands.
FCS policies 7 Example of aborting unsaved changes switch:admin> secpolicyabort Unsaved data has been aborted. All changes since the last time the secPolicySave or secPolicyActivate commands were entered are aborted. FCS policies Fabric configuration server (FCS) policy in base Fabric OS may be performed on a local switch basis and may be performed on any switch in the fabric. The FCS policy is not present by default, but must be created.
7 FCS policies Table 27 shows the commands for switch operations for Primary FCS enforcement.
FCS policies 7 3. To save or activate the new policy, enter either the secPolicySave or the secPolicyActivate command. Once the policy has been activated you can distribute the policy. NOTE FCS policy must be consistent across the fabric. If the policy is inconsistent in the fabric, then you will not be able to perform any fabric-wide configurations from the primary FCS. Modifying the order of FCS switches 1.
7 DCC policies Only the Primary FCS switch is allowed to distribute the database. The FCS policy may need to be manually distributed across the fabric using the distribute -p command. Since this policy is distributed manually, the command fddCfg –-fabwideset is used to distribute a fabric-wide consistency policy for FCS policy in an environment consisting of only Fabric OS v6.2.0 and later switches.
DCC policies TABLE 29 7 DCC policy states Policy state Characteristics No policy Any device can connect to any switch port in the fabric. Policy with no entries Any device can connect to any switch port in the fabric. An empty policy is the same as no policy. Policy with entries If a device WWN or Fabric port WWN is specified in a DCC policy, that device is only allowed access to the switch if connected by a switch port listed in the same policy.
7 DCC policies 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the Security RBAC class of commands. 2. Enter the secPolicyCreate “DCC_POLICY_nnn” command. DCC_POLICY_nnn is the name of the DCC policy; nnn is a string consisting of up to 19 alphanumeric or underscore characters to differentiate it from any other DCC policies. 3.
DCC policies 7 DCC policy behavior with Fabric-Assigned PWWNs A DCC policy check is always performed for the physical port WWN of a device when the HBA has established that the device is attempting a normal FLOGI and has both a fabric-assigned port WWN (FA-PWWN) and a physical port WWN. DCC policies created with FA-PWWNs will result in the disabling of FA-PWWN assigned ports on subsequent FLOGI.
7 SCC Policies TABLE 31 DCC policy behavior when created manually with PWWN Configuration WWN seen on DCC policy list Behavior when DCC policy activates Behavior on portDisable and portEnable • FA-PWWN has logged into the switch. DCC policy creation manually with physical PWWN of device. DCC policy activation. PWWN Traffic will not be disrupted. Ports will come up without security issues. DCC policy creation. manually with physical PWWN FA-PWWN has logged into the switch. DCC policy activation.
Authentication policy for fabric elements 7 Creating an SCC policy 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the Security RBAC class of commands. 2. Enter the secPolicyCreate “SCC_POLICY” command. 3. Save or activate the new policy by entering either the secPolicySave or the secPolicyActivate command. If neither of these commands is entered, the changes are lost when the session is logged out.
7 Authentication policy for fabric elements Key database on switch Local secret A Peer secret B Switch A FIGURE 13 Key database on switch Local secret B Peer secret A Switch B DH-CHAP authentication If you use DH-CHAP authentication, then a secret key pair must be installed only in connected fabric elements. However, as connections are changed, new secret key pairs must be installed between newly connected elements.
Authentication policy for fabric elements 7 Virtual Fabrics considerations: The switch authentication policy applies to all E_Ports in a logical switch. This includes ISLs and extended ISLs. Authentication of extended ISLs between two base switches is considered peer-chassis authentication. Authentication between two physical entities is required, so the extended ISL which connects the two chassis needs to be authenticated.
7 Authentication policy for fabric elements Re-authenticating E_Ports Use the authUtil --authinit command to re-initiate the authentication on selected ports. It provides flexibility to initiate authentication for specified E_Ports, a set of E_Ports, or all E_Ports on the switch. This command does not work on loop, NPIV and FICON devices, or on ports configured for in-flight encryption. The command authUtil can re-initiate authentication only if the device was previously authenticated.
Authentication policy for fabric elements 7 Virtual Fabrics considerations: Because the device authentication policy has switch and logical switch-based parameters, each logical switch is set when Virtual Fabrics is enabled. Authentication is enforced based on each logical switch’s policy settings. Configuring device authentication 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the Authentication RBAC class of commands. 2.
7 Authentication policy for fabric elements Authentication protocols Use the authUtil command to perform the following tasks: • Display the current authentication parameters. • Select the authentication protocol used between switches. • Select the DH (Diffie-Hellman) group for a switch. Run the authUtil command on the switch you want to view or change. Below are the different options to specify which DH group you want to use.
Authentication policy for fabric elements 7 Secret key pairs for DH-CHAP When you configure the switches at both ends of a link to use DH-CHAP for authentication, you must also define a secret key pair—one for each end of the link. Use the secAuthSecret command to perform the following tasks: • View the WWN of switches with a secret key pair. • Set the secret key pair for switches. • Remove the secret key pair for one or more switches.
7 Authentication policy for fabric elements Example of setting a secret key pair switchA:admin> secauthsecret --set This command is used to set up secret keys for the DH-CHAP authentication. The minimum length of a secret key is 8 characters and maximum 40 characters. Setting up secret keys does not initiate DH-CHAP authentication. If switch is configured to do DH-CHAP, it is performed whenever a port or a switch is enabled. Warning: Please use a secure channel for setting secrets.
Authentication policy for fabric elements 7 You can request a certificate from a CA through a Web browser. After you request a certificate, the CA either sends certificate files by e-mail (public) or gives access to them on a remote host (private). Typically, the CA provides the certificate files listed in Table 33. ATTENTION Only the .pem file is supported for FCAP authentication. TABLE 33 FCAP certificate files Certificate file Description nameCA.pem The CA certificate.
7 Authentication policy for fabric elements Enter Login Name: jdoe jdoe@10.1.2.3's password: Success: exported FCAP CA certificate Importing CA for FCAP Once you receive the files back from the Certificate Authority, you will need to install or import them onto the local and remote switches. 1. Log in to the switch using an account with admin permissions, or an account associated with the chassis role and having OM permissions for the PKI RBAC class of commands. 2.
IP Filter policy 7 Fabric-wide distribution of the Auth policy The AUTH policy can be manually distributed to the fabric by command; there is no support for automatic distribution. To distribute the AUTH policy, see “Distributing the local ACL policies” on page 162 for instructions. Local Switch configuration parameters are needed to control whether a switch accepts or rejects distributions of the AUTH policy using the distribute command and whether the switch may initiate distribution of the policy.
7 IP Filter policy Cloning an IP Filter policy You can create an IP Filter policy as an exact copy of an existing policy. The policy created is stored in a temporary buffer and has the same type and rules as the existing defined or active policy. 1. Log in to the switch using an account with admin permissions, or an account associated with the chassis role and having OM permissions for the IPfilter RBAC class of commands. 2. Enter the ipFilter --clone command.
IP Filter policy 7 1. Log in to the switch using an account with admin permissions, or an account associated with the chassis role and having OM permissions for the IPfilter RBAC class of commands. 2. Enter the ipFilter –-activate command. Deleting an IP Filter policy You can delete a specified IP Filter policy. Deleting an IP Filter policy removes it from the temporary buffer. To permanently delete the policy from the persistent database, run ipfilter --save.
7 IP Filter policy For an IP Filter policy rule, you can only select port numbers in the well-known port number range, between 0 and 1023, inclusive. This means that you have the ability to control how to expose the management services hosted on a switch, but not the ability to affect the management traffic that is initiated from a switch. A valid port number range is represented by a dash, for example 7-30. Alternatively, service names can also be used instead of port number.
IP Filter policy TABLE 34 7 Supported services (Continued) Service name Port number shell 514 uucp 540 biff 512 who 513 syslog 514 route 520 timed 525 kerberos4 750 rpcd 897 securerpcd 898 Protocol TCP and UDP protocols are valid protocol selections. Fabric OS v6.2.0 and later do not support configuration to filter other protocols. Implicitly, ICMP type 0 and type 8 packets are always allowed to support ICMP echo request and reply on commands like ping and traceroute.
7 IP Filter policy Default policy rules A switch with Fabric OS v6.2.0 or later will have a default IP Filter policy for IPv4 and IPv6. The default IP Filter policy cannot be deleted or changed. When an alternative IP Filter policy is activated, the default IP Filter policy becomes deactivated. Table 36 lists the rules of the default IP Filter policy.
IP Filter policy 7 Adding a rule to an IP Filter policy There can be a maximum of 256 rules created for an IP Filter policy. The change to the specified IP Filter policy is not saved to the persistent configuration until a save or activate subcommand is run. 1. Log in to the switch using an account with admin permissions, or an account associated with the chassis role and having the OM permissions for the IPfilter RBAC class of commands. 2. Enter the ipFilter --addrule command.
7 Policy database distribution Managing filter thresholds Fabric OS v7.0.0 allows you to configure filter thresholds using the fmMonitor command. 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the FabricWatch RBAC class of commands. 2. Enter the fmMonitor command.
Policy database distribution TABLE 37 7 Interaction between fabric-wide consistency policy and distribution settings Distribution setting Fabric-wide consistency policy Absent (default) Tolerant configuration.1 Reject Database is protected, it cannot be overwritten. May not match other databases in the fabric. Invalid Accept (default) Database is not protected, the database can be overwritten.
7 Policy database distribution Example shows the database distribution settings switch:admin> fddcfg --showall Local Switch Configuration for all Databases:DATABASE - Accept/Reject --------------------------------SCC accept DCC accept PWD accept FCS accept AUTH accept IPFILTER accept Fabric Wide Consistency Policy:- "" Enabling local switch protection 1.
Policy database distribution 7 Fabric-wide enforcement The fabric-wide consistency policy enforcement setting determines the distribution behavior when changes to a policy are activated. Using the tolerant or strict fabric-wide consistency policy ensures that changes to local ACL policy databases are automatically distributed to other switches in the fabric. NOTE To completely remove all policies from a fabric enter the fddCfg --fabwideset "” command.
7 Policy database distribution Setting the fabric-wide consistency policy 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the FabricDistribution RBAC class of commands. 2. Enter the fddCfg --fabwideset command. Example shows how to set a strict SCC and tolerant DCC fabric-wide consistency policy.
Policy database distribution 7 Use the fddCfg –-fabwideset command on either this switch or the fabric to set a matching strict SCC, DCC, or FCS fabric-wide consistency policy. Use ACL policy commands to delete the conflicting ACL policy from one side to resolve ACL policy conflict. If neither the fabric nor the joining switch is configured with a fabric-wide consistency policy, there are no ACL merge checks required. The descriptions above also apply to joining two fabrics.
7 Management interface security TABLE 41 Examples of strict fabric merges Fabric-wide consistency policy setting Strict/Tolerant Strict/Absent Expected behavior Fabric A Fabric B SCC:S;DCC:S SCC;DCC:S SCC;DCC:S SCC:S;DCC SCC:S;DCC SCC:S Ports connecting switches are disabled. SCC:S;DCC:S SCC:S DCC:S Strict/Strict SCC:S DCC:S Table 42 has a matrix of merging fabrics with tolerant and absent policies.
Management interface security 7 • Automated Key Management—Automates the process, as well as manages the periodic exchange and generation of new keys. Using the ipsecConfig command, you must configure multiple security policies for traffic flows on the Ethernet management interfaces based on IPv4 or IPv6 addresses, a range of IPv4 or IPv6 addresses, the type of application, port numbers, and protocols used (UDP/TCP/ICMP).
7 Management interface security FIGURE 15 Gateway tunnel configuration Endpoint-to-gateway tunnel In this scenario, a protected endpoint (typically a portable computer) connects back to its corporate network through an IPsec-protected tunnel. It might use this tunnel only to access information on the corporate network, or it might tunnel all of its traffic back through the corporate network in order to take advantage of protection provided by a corporate firewall against Internet-based attacks.
Management interface security 7 IPsec protocols use a sliding window to assist in flow control, The IPsec protocols also use this sliding window to provide protection against replay attacks in which an attacker attempts a denial of service attack by replaying an old sequence of packets. IPsec protocols assign a sequence number to each packet. The recipient accepts each packet only if its sequence number is within the window. It discards older packets.
7 Management interface security TABLE 43 Algorithms and associated authentication policies (Continued) Algorithm Encryption Level Policy Description 3des_cbc 168-bit ESP Triple DES is a more secure variant of DES. It uses three different 56-bit keys to encrypt blocks of 64-bit plain text. The algorithm is FIPS-approved for use by Federal agencies. blowfish_cbc 64-bit ESP Blowfish is a 32-bit to 448-bit keyed, symmetric block cipher.
Management interface security 7 Key management The IPsec key management supports Internet Key Exchange or Manual key/SA entry. The Internet Key Exchange (IKE) protocol handles key management automatically. SAs require keying material for authentication and encryption. The managing of keying material that SAs require is called key management. The IKE protocol secures communication by authenticating peers and exchanging keys. It also creates the SAs and stores them in the SADB.
7 Management interface security Creating the tunnel Each side of the tunnel must be configured in order for the tunnel to come up. Once you are logged into the switch, do not log off as each step requires that you are logged in to the switch. IPsec configuration changes take effect upon execution and are persistent across reboots.
Management interface security 7 8. Create an IPsec transform on each switch using the ipSecConfig --add command. Example of creating an IPsec transform This example creates an IPsec transform TRANSFORM01 to use the transport mode to protect traffic identified for IPsec protection and use IKE01 as key management policy. switch:admin> ipsecconfig --add policy ips transform –t TRANSFORM01 \ -mode transport -sa-proposal IPSEC-AH \ -action protect –ike IKE01 9.
7 Management interface security Example of an End-to-end transport tunnel mode This example illustrates securing traffic between two systems using AH protection with MD5 and configure IKE with pre-shared keys. The two systems are a switch, BROCADE300 (IPv4 address 10.33.74.13), and an external host (10.33.69.132). NOTE A backslash ( \ ) is used to skip the return character so you can continue the command on the next line without the return character being interpreted by the shell. 1.
Management interface security 7 9. Create traffic selectors to select the outbound and inbound traffic that needs to be protected. switch:admin> ipsecconfig --add policy ips selector \ -t SELECTOR-OUT -d out -l 10.33.74.13 -r 10.33.69.132 \ -transform TRANSFORM01 switch:admin> ipsecconfig --add policy ips selector \ -t SELECTOR-IN -d in -l 10.33.69.132 -r 10.33.74.13 \ -transform TRANSFORM01 10. Verify the IPsec SAs created with IKE using the ipsecConfig --show manual-sa –a command. 11.
7 176 Management interface security Fabric OS Administrator’s Guide 53-1002446-01
Chapter 8 Maintaining the Switch Configuration File In this chapter • Configuration settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Configuration file backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Configuration file restoration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Configurations across a fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8 Configuration settings If your user account has chassis account permissions, you can use any of the following options when uploading or downloading a configuration file: -fid To upload the specified FID configuration. -all To upload all of the system configuration, including the chassis section and all switch sections for all logical switches. Note: Use this parameter when obtaining a complete capture of the switch configuration in a switch that has Virtual Fabric mode disabled.
Configuration settings 8 [Active Security policies] [cryptoDev] [FICU SAVED FILES] [Banner] [End] [Switch Configuration End : 0] date = Tue Mar 1 21:28:52 2011 [Switch Configuration Begin : 1] SwitchName = switch_2 Fabric ID = 1 [Boot Parameters] [Configuration] [Bottleneck Configuration] [Zoning] [Defined Security policies] [Active Security policies] [cryptoDev] [FICU SAVED FILES] [Banner] [End] [Switch Configuration End : 1] Chassis section There is only one chassis section within a configuration.
8 Configuration file backup • • • • • • LicensesLservc – Sentinel License configuration GE blade mode – GigE Mode Configuration FWD CHASSIS CFG – Fabric watch configuration FRAME LOG – Frame log configuration (enable/disable) DMM_TB – Data migration manager configuration MOTD – Message of the day Switch section There is always at least one switch section for the default switch or a switch that has Virtual Fabric mode disabled, and there are additional sections corresponding to each additionally defined
Configuration file restoration 8 Before you upload a configuration file, verify that you can reach the FTP server from the switch. Using a Telnet connection, save a backup copy of the configuration file from a logical switch to a host computer. Secure File Transfer Protocol is now an option when uploading a configuration file. SFTP is analogous to SCP (secure copy) and appears an option for the configupload/download, supportsave, auto FFDC/trace upload (supportftp) commands.
8 Configuration file restoration CAUTION Make sure that the configuration file you are downloading is compatible with your switch model. Downloading a configuration file from a different switch model or from a different firmware could cause your switch to fail. CAUTION If you have Virtual Fabrics enabled, you must follow the procedure in “Configuration management for Virtual Fabrics” on page 186 to restore the logical switches.
Configuration file restoration -all 8 The number of switches or FIDs defined in the downloaded configuration file must match the number of switches or FIDs currently defined on the switch. The switches must be disabled first. If they are not, the configDownload command will download the configuration for as many switches as possible until a non-disabled switch is found. Then it will stop. Before running the configDownload command, verify if any switches need to be disabled.
8 Configuration file restoration Configuration download without disabling a switch You can download configuration files to a switch while the switch is enabled; that is, you do not need to disable the switch for changes in SNMP, Fabric Watch, or ACL parameters. However, if there is any changed parameter that does not belong to SNMP, Fabric Watch, or ACL, then you must disable the switch. When you use the configDownload command, you are prompted to disable the switch only when necessary.
Configurations across a fabric 8 Section (all|chassis|FID# [all]): all *** CAUTION *** This command is used to download a backed-up configuration for a specific switch. If using a file from a different switch, this file's configuration settings will override any current switch settings. Downloading a configuration file, which was uploaded from a different type of switch, may cause this switch to fail. A switch reboot might be required for some parameter changes to take effect.
8 Configuration management for Virtual Fabrics Do not download a configuration file from one switch to another switch that is a different model or runs a different firmware version, because it can cause the switch to fail. If you need to reset affected switches, issue the configDefault command after download is completed but before the switch is enabled. If a switch is enabled with a duplicate domain ID, the switch becomes segmented.
Configuration management for Virtual Fabrics 8 Potentially remote file may get overwritten Section (all|chassis|FID# [all]): Password: configUpload complete: All selected config parameters are uploaded Example of configUpload of a logical switch configuration DCX_80:FID128:admin> configupload -vf Protocol (scp, ftp, sftp, local) [ftp]: Server Name or IP Address [host]: 10.1.2.3 User Name [user]: anonymous Path/Filename [/config.
8 Configuration management for Virtual Fabrics Example of configDownload on a switch 5100:FID128:admin> configdownload -vf Protocol (scp, ftp, sftp, local) [ftp]: Server Name or IP Address [host]: 10.1.2.3 User Name [user]: UserFoo Path/Filename [/config.txt]: 5100_FID89.txt *** CAUTION *** This command is used to download the VF configuration to the switch. Afterwards, the switch will be automatically rebooted and the new VF settings will be used.
Brocade configuration form 8 Brocade configuration form Use the form in Table 45 as a hard copy reference for your configuration information. In the hardware reference manuals for the Brocade DCX and DCX-4S Backbones, there is a guide for FC port setting tables. The tables can be used to record configuration information for the various blades.
8 190 Brocade configuration form Fabric OS Administrator’s Guide 53-1002446-01
Chapter 9 Installing and Maintaining Firmware In this chapter • Firmware download process overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Preparing for a firmware download . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Firmware download on switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Firmware download on a Backbone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Firmware download from a USB device . . . . . . . . . . .
9 Firmware download process overview You can download Fabric OS to a Backbone, which is a chassis; and to a nonchassis-based system, also referred to as a fixed-port switch. The difference in the download process is that Backbones have two CPs and fixed-port switches have one CP. Use the firmwareDownload command to download the firmware from either an FTP or SSH server by using either the FTP, SFTP, or SCP protocol to the switch. Or you can use a Brocade-branded USB device.
Firmware download process overview 9 In most cases, you will be upgrading firmware; that is, installing a newer firmware version than the one you are currently running. However, some circumstances may require installing an older version; that is, downgrading the firmware. The procedures in this section assume that you are upgrading firmware, but they work for downgrading as well, provided the old and new firmware versions are compatible.
9 Preparing for a firmware download TABLE 46 Backbone HA sync states (Continued) Active CP Fabric OS version Standby CP Fabric OS version HA sync state Remedy v6.3.0 v6.3.0 inSync n/a v6.3.0 v6.4.0 inSync n/a v6.4.0 v6.3.0 inSync Run firmwareDownload -s on the standby CP and upgrade it to v6.4.0. v6.4.0 v6.4.0 inSync n/a v7.0.0 v6.4.0 inSync Run firmwareDownload -s on the standby CP to upgrade it to v7.0.0 v7.0.0 v7.0.
Preparing for a firmware download 9 Connected switches Before you upgrade the firmware on your switch, you need to check the connected switches to ensure compatibility and that any older versions are supported. Refer to the Fabric OS Compatibility section of the Brocade Fabric OS Release Notes, for the recommended firmware version. NOTE Go to http://www.brocade.com to view end-of-life policies for Brocade products. Navigate to the Support tab, then select Policies and Locations.
9 Firmware download on switches Firmware download on switches Brocade fixed-port switches maintain primary and secondary partitions for firmware. The firmwareDownload command defaults to an autocommit option that automatically copies the firmware from one partition to the other. NOTE This section only applies when upgrading from Fabric OS v6.1.x to v6.2.0, or from different versions of v6.2.0, such as patch releases. If you are downgrading from v6.2.0 to v6.1.
Firmware download on a Backbone 9 2. Obtain the firmware file from the Brocade website at http://www.brocade.com and store the file on the FTP or SSH server or the USB memory device. 3. Unpack the compressed files preserving directory structures. The firmware is in the form of RPM packages with names defined in a .plist file. The .plist file contains specific firmware information and the names of packages of the firmware to be downloaded. 4.
9 Firmware download on a Backbone the CPs are not in sync, you can run firmwareDownload –s on each of the CPs to upgrade them. These operations are disruptive. Or if the CPs are not in sync, run the haSyncStart command. If the problem persists, refer to the Fabric OS Troubleshooting and Diagnostics Guide. If the troubleshooting information fails to help resolve the issue, contact your switch service provider.
Firmware download on a Backbone 9 Upgrading firmware on Backbones (including blades) There is only one chassis management IP address for the Brocade Backbones. NOTE By default, the firmwareDownload command automatically upgrades both the active and the standby CP and all co-CPs on the CP blades in the Brocade Backbones. It automatically upgrades all AP blades in the Brocade Backbones using auto-leveling. 1. Verify that the Ethernet interfaces located on CP0 and CP1 are plugged into your network. 2.
9 Firmware download on a Backbone If an AP blade is present: At the point of the failover an autoleveling process is activated. Autoleveling is triggered when the active CP detects a blade that contains a different version of the firmware, regardless of which version is older. Autoleveling downloads firmware to the AP blade, swaps partitions, reboots the blade, and copies the new firmware from the primary partition to the secondary partition.
Firmware download from a USB device 9 Slot 7 (CP1, active): Firmware has been downloaded to the secondary partition of the switch. [5]: Mon Mar 22 04:37:24 2010 Slot 7 (CP1, standby): The firmware commit operation has started. This may take up to 10 minutes. [6]: Mon Mar 22 04:41:59 2010 Slot 7 (CP1, standby): The commit operation has completed successfully. [7]: Mon Mar 22 04:41:59 2010 Slot 7 (CP1, standby): Firmwaredownload command has completed successfully.
9 FIPS support Downloading from USB using the relative path 1. Log in to the switch as admin. 2. Enter the firmwareDownload -U command. ecp:admin>firmwaredownload –U v7.0.0 Downloading from USB using the absolute path 1. Log in to the switch as admin. 2. Enter the firmwareDownload command with the -U operand. ecp:admin>firmwaredownload –U /usb/usbstorage/brocade/firmware/v7.0.
FIPS support 9 NOTE If FIPS is enabled, all logins should be done through SSH or direct serial and the transfer protocol should be SCP. Updating the firmware key 1. Log in to the switch as admin. 2. Type the firmwareKeyUpdate command and respond to the prompts. The firmwareDownload command As mentioned previously, the public key file needs to be packaged, installed, and run on your switch before downloading a signed firmware.
9 Test and restore firmware on switches Power-on firmware checksum test FIPS requires the checksums of the executables and libraries on the filesystem to be validated before Fabric OS modules are launched. This is to make sure these files have not been changed after they are installed. When firmware RPM packages are installed during firmwareDownload, the MD5 checksums of the firmware files are stored in the RPM database on the filesystem. The checksums go through all of the files in the RPM database.
Test and restore firmware on switches 9 User Name: userfoo File Name: /home/userfoo/v7.0.0 Password: Do Auto-Commit after Reboot [Y]: n Reboot system after download [N]: y Firmware is being downloaded to the switch. This step may take up to 30 minutes. Checking system settings for firmwaredownload... The switch performs a reboot and comes up with the new firmware to be tested. Your current switch session automatically disconnects.
9 Test and restore firmware on Backbones Test and restore firmware on Backbones This procedure enables you to perform a firmware download on each CP and verify that the procedure was successful before committing to the new firmware. The old firmware is saved in the secondary partition of each CP until you enter the firmwareCommit command.
Test and restore firmware on Backbones 9 If an AP blade is present: At the point of the failover an autoleveling process is activated. See “Backbone firmware download process overview” on page 198 for details about autoleveling. 8. Verify the failover. a. Connect to the Backbone on the active CP, which is the former standby CP. b. Enter the haShow command to verify that the HA synchronization is complete.
9 Validating a firmware download d. Enter the haShow command to confirm that the HA state is in sync. ATTENTION Stop! If you have completed step 11, then you have committed the firmware on both CPs and you have completed the firmware download procedure. 12. Restore the firmware on the standby CP. In the current Backbone session for the standby CP, enter the firmwareRestore command. The standby CP reboots and the current Backbone session ends.
Validating a firmware download 9 NOTE When you prepared for the firmware download earlier, you issued either the supportShow or supportSave command. Although you can issue the command again and compare the output from before and after, it may take up to 30 minutes for the command to execute. To save time, it is recommended that you use the commands listed below, which are all subsets of the supportSave output.
9 210 Validating a firmware download Fabric OS Administrator’s Guide 53-1002446-01
Chapter 10 Managing Virtual Fabrics In this chapter • Virtual Fabrics overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Logical switch overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Logical fabric overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Management model for logical switches . . . . . . . . . . . . . . . . . . . . . . . . . . .
10 Logical switch overview This chapter describes the logical switch and logical fabric features. For information about device sharing with Virtual Fabrics, refer to “FC-FC routing and Virtual Fabrics” on page 501. For information about supported switches and port types, refer to “Supported platforms for Virtual Fabrics” on page 222. Virtual Fabrics and Admin Domains are mutually exclusive and are not supported at the same time on a switch.
Logical switch overview 10 After you enable Virtual Fabrics, you can create up to seven additional logical switches, depending on the switch model. Figure 18 shows a Virtual Fabrics-enabled switch before and after it is divided into logical switches. Before you create logical switches, the chassis appears as a single switch (default logical switch). After you create logical switches, the chassis appears as multiple independent logical switches.
10 Logical switch overview Physical chassis Logical switch 1 (Default logical switch) (FID = 128) Logical switch 2 (FID = 1) Logical switch 3 (FID = 15) Logical switch 4 (FID = 8) Logical switch 5 (FID = 20) FIGURE 19 Fabric IDs assigned to logical switches Port assignment in logical switches Initially, all ports belong to the default logical switch. When you create additional logical switches, they are empty and you must assign ports to those logical switches.
Logical switch overview 10 A given port is always in one (and only one) logical switch. The following scenarios refer to the chassis after port assignment in Figure 20: • If you assign P2 to logical switch 2, you cannot assign P2 to any other logical switch. • If you want to remove a port from a logical switch, you cannot delete it from the logical switch, but must move it to a different logical switch.
10 Logical fabric overview Physical chassis Logical switch 1 P1 (Default logical switch) Fabric ID 128 Logical switch 2 Fabric ID 1 H1 P2 P3 D1 P4 Logical switch 3 Fabric ID 15 Logical switch 4 Fabric ID 8 P5 P6 D2 ISL Switch FIGURE 21 Logical switches connected to devices and non-Virtual Fabrics switch Figure 22 shows a logical representation of the physical chassis and devices in Figure 21. As shown in Figure 22, the devices are isolated into separate fabrics.
Logical fabric overview 10 Logical fabric and ISLs Figure 23 shows two physical chassis divided into logical switches. In Figure 23, ISLs are used to connect the logical switches with FID 1 and the logical switches with FID 15. The logical switches with FID 8 are each connected to a non-Virtual Fabrics switch. The two logical switches and the non-Virtual Fabrics switch are all in the same fabric, with FID 8.
10 Logical fabric overview Base switch and extended ISLs Another way to connect logical switches is to use extended ISLs and base switches. When you divide a chassis into logical switches, you can designate one of the switches to be a base switch. A base switch is a special logical switch that is used for interconnecting the physical chassis. A base switch has the following properties: • ISLs connected through the base switch can be used for communication among the other logical switches.
Logical fabric overview 10 Think of the logical switches as being connected with logical ISLs, as shown in Figure 26. In this diagram, the logical ISLs are not connected to ports because they are not physical cables. They are a logical representation of the switch connections that are allowed by the XISL.
10 Logical fabric overview By default, the physical ISL path is favored over the logical path (over the XISL) because the physical path has a lower cost. This behavior can be changed by configuring the cost of the dedicated physical ISL to match the cost of the logical ISL. ATTENTION If you disable a base switch, all of the logical ISLs are broken and the logical switches cannot communicate with each other unless they are connected by a physical ISL.
Management model for logical switches 10 Management model for logical switches You can use one common IP address for the hardware that is shared by all of the logical switches in the chassis and you can set up individual IPv4 addresses for each Virtual Fabric. For a management host to manage a logical switch using the Internet Protocol over Fibre Channel (IPFC) IP address, it must be physically connected to the Virtual Fabric using a host bus adapter (HBA).
10 Supported platforms for Virtual Fabrics Supported platforms for Virtual Fabrics The following platforms are Virtual Fabrics-capable: • • • • • • • Brocade 5100 Brocade 5300 Brocade 6510 Brocade VA-40FC, in Native mode only Brocade DCX Brocade DCX-4S Brocade DCX 8510 family Some restrictions apply to the ports, depending on the port type and blade type. The following sections explain these restrictions.
Supported platforms for Virtual Fabrics TABLE 47 10 Blade and port types supported on logical switches (Continued) Blade type Default logical switch User-defined logical switch Base switch FR4-18i: FC ports GE ports Yes (F, E) Yes (VE) No Yes (VE) No Yes (VE, VEX) ICL ports Yes Yes Yes 1. In the Brocade DCX and DCX 8510-8, ports 56–63 of the FC8-64 blade are not supported as E_Ports on the default logical switch. The Brocade DCX-4S and DCX 8510-4 do not have this limitation. 2.
10 Limitations and restrictions of Virtual Fabrics TABLE 48 Virtual Fabrics interaction with Fabric OS features (Continued) Fabric OS feature Virtual Fabrics interaction FC-FC Routing Service All EX_Ports must reside in a base switch. You cannot attach EX_Ports to a logical switch that has XISL use enabled. You must use ISLs to connect the logical switches in an edge fabric. NOTE: FC-FC Routing is not supported on a Brocade 6510 with more than 3 logical switches.
Enabling Virtual Fabrics mode 10 Restrictions on XISLs The Allow XISL Use option, available under the configure command, allows a logical switch to use XISLs in the base switch as well as any standard ISLs that are connected to that logical switch. To allow or disallow XISL use for a logical switch, see “Configuring a logical switch to use XISLs” on page 234. Following are restrictions on XISL use.
10 Disabling Virtual Fabrics mode 3. Delete all Admin Domains, as described in “Deleting all user-defined Admin Domains non-disruptively” on page 354. 4. Enter the following command to enable VF mode: fosconfig --enable vf 5. Enter y at the prompt. Example The following example checks whether VF mode is enabled or disabled and then enables it.
Configuring logical switches to use basic configuration values 10 Example The following example checks whether VF mode is enabled or disabled and then disables it.
10 Creating a logical switch or base switch NOTE Domain ID conflicts are detected before fabric ID conflicts. If you have both a domain ID conflict and a fabric ID conflict, only the domain ID conflict is reported. 1. Connect to the physical chassis and log in using an account with the chassis-role permission. 2.
Executing a command in a different logical switch context 10 Executing a command in a different logical switch context This procedure describes how to execute a command for a logical switch while you are in the context of a different logical switch. You can also execute a command for all the logical switches in a chassis. The command is not executed on those logical switches for which you do not have permission. 1.
10 Deleting a logical switch "fabricshow" on FID 4: Switch ID Worldwide Name Enet IP Addr FC IP Addr Name ------------------------------------------------------------------------14: fffc0e 10:00:00:05:1e:82:3c:2b 10.32.79.105 0.0.0.0 >"switch_4" (output truncated) Deleting a logical switch You must remove all ports from the logical switch before deleting it. You cannot delete the default logical switch.
Displaying logical switch configuration 10 NOTE If you are deploying ICLs in the base switch, all ports associated with those ICLs must be assigned to the base switch. If you are deploying ICLs to connect to default switches (that is, XISL use is not allowed), the ICL ports should be assigned (or left) in the default logical switch. 1. Connect to the physical chassis and log in using an account with the chassis-role permission. 2.
10 Changing the fabric ID of a logical switch Changing the fabric ID of a logical switch The following procedure describes how you can change the fabric ID of an existing logical switch. The fabric ID indicates in which fabric the logical switch participates. By changing the fabric ID, you are moving the logical switch from one fabric to another. Changing the fabric ID requires permission for chassis management operations. You cannot change the FID of your own logical switch context.
Changing a logical switch to a base switch 10 5. Enable the switch. switchenable Example of changing the logical switch with FID 7 to a base switch sw0:FID128:admin> setcontext 7 switch_25:FID7:admin> switchshow switchName: switch_25 switchType: 66.
10 Setting up IP addresses for a Virtual Fabric Setting up IP addresses for a Virtual Fabric NOTE IPv6 is not supported when setting the IPFC interface for Virtual Fabrics. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the ipAddrSet -ls command. For the --add parameter, specify the network information in dotted-decimal notation for the Ethernet IPv4 address with a Classless Inter-Domain Routing (CIDR) prefix.
Changing the context to a different logical fabric 10 Changing the context to a different logical fabric You can change the context to a different logical fabric. Your user account must have permission to access the logical fabric. 1. Connect to the physical chassis and log in using an account with the chassis-role permission. 2.
10 Creating a logical fabric using XISLs c. Create a base switch and assign it a fabric ID that will become the FID of the base fabric. See “Creating a logical switch or base switch” on page 227 for instructions on creating a base switch. For the example shown in Figure 28, you would create a base switch with fabric ID 8. d. Assign ports to the base switch, as described in “Adding and moving ports on a logical switch” on page 230. e.
Chapter 11 Administering Advanced Zoning In this chapter • Special zones. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Zoning overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Broadcast zones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Zone aliases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11 Zoning overview • QoS zones Assign high or low priority to designated traffic flows. QoS zones are regular zones with additional QoS attributes specified by adding a QOS prefix to the zone name. See “QoS: SID/DID traffic prioritization” on page 419 for more information. • Traffic Isolation zones (TI zones) Isolate inter-switch traffic to a specific, dedicated path through the fabric. See Chapter 12, “Traffic Isolation Zoning,” for more information.
Zoning overview 11 Blue Zone Server 2 Server 1 Storage 2 Red Zone Storage 1 RAID Green Zone Storage 3 FIGURE 29 Server 3 Zoning example Approaches to zoning Table 50 lists the various approaches you can take when implementing zoning in a fabric. TABLE 50 Approaches to fabric-based zoning Zoning approach Description Recommended approach Single HBA Fabric OS Administrator’s Guide 53-1002446-01 Zoning by single HBA most closely re-creates the original SCSI bus.
11 Zoning overview TABLE 50 Approaches to fabric-based zoning (Continued) Zoning approach Description Alternative approaches Application Zoning by application typically requires zoning multiple, perhaps incompatible, operating systems into the same zones. This method of zoning creates the possibility that a minor server in the application suite could disrupt a major server (such as a Web server disrupting a data warehouse server).
Zoning overview 11 The types of zone objects used to define a zone can be mixed. For example, a zone defined with the zone objects 2,12; 2,14; 10:00:00:80:33:3f:aa:11 contains the devices connected to domain 2, ports 12 and 14, and a device with the WWN 10:00:00:80:33:3f:aa:11 (either node name or port name) that is connected on the fabric.
11 Zoning overview The different types of zone configurations are: • Defined Configuration The complete set of all zone objects defined in the fabric. • Effective Configuration A single zone configuration that is currently in effect. The effective configuration is built when you enable a specified zone configuration. • Saved Configuration A copy of the defined configuration plus the name of the effective configuration, which is saved in flash memory.
Zoning overview 11 Identifying the enforced zone type 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the portZoneShow command, using the following syntax: portzoneshow Considerations for zoning architecture Table 51 lists considerations for zoning architecture. TABLE 51 Considerations for zoning architecture Item Description Type of zoning enforcement: frameor session-based If security is a priority, frame-based hardware enforcement is recommended.
11 Broadcast zones Best practices for zoning The following are recommendations for using zoning: • Always zone using the highest Fabric OS-level switch. Switches with earlier Fabric OS versions do not have the capability to view all the functionality that a newer Fabric OS provides, as functionality is backwards compatible but not forwards compatible. • Zone using the core switch versus an edge switch. • Zone using a Backbone rather than a switch.
Broadcast zones 11 Figure 30 illustrates how broadcast zones work with Admin Domains. Figure 30 shows a fabric with five devices and two Admin Domains, AD1 and AD2. Each Admin Domain has two devices and a broadcast zone.
11 Zone aliases High availability considerations with broadcast zones If a switch has broadcast zone-capable firmware on the active CP (Fabric OS v5.3.x or later) and broadcast zone-incapable firmware on the standby CP (Fabric OS version earlier than v5.3.0), then you cannot create a broadcast zone because the zoning behavior would not be the same across an HA failover. If the switch failed over, then the broadcast zone would lose its special significance and would be treated as a regular zone.
Zone aliases 11 Creating an alias 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the aliCreate command, using the following syntax: alicreate "aliasname", "member[; member...]" 3. Enter the cfgSave command to save the change to the defined configuration. The cfgSave command ends and commits the current zoning transaction buffer to nonvolatile memory.
11 Zone aliases Removing members from an alias 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the aliRemove command, using the following syntax: aliremove "aliasname", "member[; member...]" 3. Enter the cfgSave command to save the change to the defined configuration. The cfgSave command ends and commits the current zoning transaction buffer to nonvolatile memory.
Zone creation and maintenance 11 Viewing an alias in the defined configuration 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the aliShow command, using the following syntax alishow "pattern"[, mode] If no parameters are specified, the entire zone database (both the defined and effective configuration) is displayed. Example The following example shows all zone aliases beginning with “arr”.
11 Zone creation and maintenance Adding devices (members) to a zone 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the zoneAdd command, using the following syntax: zoneadd "zonename", "member[; member...]" 3. Enter the cfgSave command to save the change to the defined configuration. The cfgSave command ends and commits the current zoning transaction buffer to nonvolatile memory.
Zone creation and maintenance 11 Deleting a zone 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the zoneDelete command, using the following syntax: zonedelete "zonename" 3. Enter the cfgSave command to save the change to the defined configuration. The cfgSave command ends and commits the current zoning transaction buffer to nonvolatile memory.
11 Default zoning mode 1,0; loop1 zone: White_zone 1,3; 1,4 alias: array1 21:00:00:20:37:0c:76:8c; 21:00:00:20:37:0c:71:02 alias: array2 21:00:00:20:37:0c:76:22; 21:00:00:20:37:0c:76:28 alias: loop1 21:00:00:20:37:0c:76:85; 21:00:00:20:37:0c:71:df 3. Enter the zone --validate command to list all zone members that are not part of the current zone enforcement table. Note that zone configuration names are case-sensitive; blank spaces are ignored. switch:admin> zone --validate "White_zone" 4.
Default zoning mode 11 Typically, when you disable the zoning configuration in a large fabric with thousands of devices, the name server indicates to all hosts that they can communicate with each other. In fact, each host can receive an enormous list of PIDs, and ultimately cause other hosts to run out of memory or crash. To ensure that all devices in a fabric do not see each other during a configuration disable operation, set the default zoning mode to No Access.
11 Zone database size Viewing the current default zone access mode 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the defZone --show command. NOTE If you perform a firmware download of an older release, then the current default zone access state will appear as it did prior to the download. For example, if the default zoning mode was No Access before the download, it will remain as No Access afterward.
Zone configurations 11 You can use the cfgSize command to check both the maximum available size and the currently saved size on all switches. If you think you are approaching the maximum, you can save a partially completed zone configuration and use the cfgSize command to determine the remaining space. The cfgSize command reports the maximum available size on the current switch only. It cannot determine the maximum available size on other switches in the fabric.
11 Zone configurations Example switch:admin> cfgadd "newcfg", "bluezone" switch:admin> cfgsave You are about to save the Defined zoning configuration. This action will only save the changes on the Defined configuration. Any changes made on the Effective configuration will not take effect until it is re-enabled.
Zone configurations 11 Example switch:admin> cfgenable "USA_cfg" You are about to enable a new zoning configuration. This action will replace the old zoning configuration with the current configuration selected. If the update includes changes to one or more traffic isolation zones, the update may result in localized disruption to traffic on ports associated with the traffic isolation zone changes.
11 Zone configurations Example switch:admin> cfgdelete "testcfg" switch:admin> cfgsave You are about to save the Defined zoning configuration. This action will only save the changes on the Defined configuration. Any changes made on the Effective configuration will not take effect until it is re-enabled.
Zone configurations 11 1,2 21:00:00:20:37:0c:76:22 21:00:00:20:37:0c:76:28 zone: Purple_zone 1,0 21:00:00:20:37:0c:76:85 21:00:00:20:37:0c:71:df Viewing selected zone configuration information 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the cfgShow command and specify a pattern.
11 Zone object maintenance and configurations in the Defined configuration. Run cfgSave to commit the transaction or cfgTransAbort to cancel the transaction. Do you really want to clear all configurations? (yes, y, no, n): [no] 3. Enter one of the following commands, depending on whether an effective zone configuration exists: • If no effective zone configuration exists, enter the cfgSave command.
Zone object maintenance 11 Deleting a zone object The following procedure removes all references to a zone object and then deletes the zone object. The zone object can be a zone member, a zone alias, or a zone. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the cfgShow command to view the zone configuration objects you want to delete.
11 Zone configuration management Renaming a zone object 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the cfgShow command to view the zone configuration objects you want to rename.
Zone merging 11 Brocade Advanced Zoning is configured on the primary fabric configuration server (FCS). The primary FCS switch makes zoning changes and other security-related changes. The primary FCS switch also distributes zoning to all other switches in the secure fabric. All existing interfaces can be used to administer zoning. You must perform zone management operations from the primary FCS switch using a zone management interface, such as Telnet or Web Tools.
11 Zone merging If you have implemented default zoning you must set the switch you are adding into the fabric to the same default zone mode setting as the rest of the fabric to avoid segmentation. • Merging rules Observe these rules when merging zones: - Local and adjacent configurations: If the local and adjacent zone database configurations are the same, they will remain unchanged after the merge.
Zone merging 11 NOTE If the zoneset members on two switches are not listed in the same order, the configuration is considered a mismatch, resulting in the switches being segmented from the fabric. For example: cfg1 = z1; z2 is different from cfg1 = z2; z1, even though members of the configuration are the same. If zoneset members on two switches have the same names defined in the configuration, make sure zoneset members are listed in the same order.
11 Zone merging TABLE 52 Zone merging scenarios: Defined and effective configurations (Continued) Description Switch A Switch B Expected results Switch A does not have a defined configuration. Switch B has a defined configuration. defined: none effective: none defined:cfg1 zone1: ali1; ali2 effective: cfg1 Switch A will absorb the configuration from the fabric, with cfg1 as the effective configuration. Switch A and Switch B have the same defined configuration.
Zone merging TABLE 54 11 Zone merging scenarios: Different names (Continued) Description Switch A Switch B Expected results Same content, different alias name. defined: cfg1 ali1: A; B effective: irrelevant defined:cfg1 ali2: A; B effective: irrelevant Fabric segments due to: Zone Conflict content mismatch Same alias name, same content, different order.
11 Zone merging TABLE 56 Zone merging scenarios: Default access mode (Continued) Description Switch A Switch B Expected results Effective zone configuration. No effective configuration. defzone = allaccess effective: cfg2 defzone: allaccess or noaccess Clean merge — effective zone configuration and defzone mode from Switch B propagates to fabric. Effective zone configuration. No effective configuration.
Chapter 12 Traffic Isolation Zoning In this chapter • Traffic Isolation Zoning overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Enhanced TI zones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Traffic Isolation Zoning over FC routers . . . . . . . . . . . . . . . . . . . . . . . . . . . . • General rules for TI zones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12 Traffic Isolation Zoning overview Figure 31 shows a fabric with a TI zone consisting of the following: • N_Ports: • E_Ports: “1,7”, “1,8”, “4,5”, and “4,6” “1,1”, “3,9”, “3,12”, and “4,7” The dotted line indicates the dedicated path between the initiator in Domain 1 to the target in Domain 4.
Traffic Isolation Zoning overview TABLE 58 12 Traffic behavior when failover is enabled or disabled in TI zones Failover enabled Failover disabled If the dedicated path is not the shortest path or if the dedicated path is broken, the TI zone traffic will use a non-dedicated path instead. If the dedicated path is not the shortest path or if the dedicated path is broken, traffic for that TI zone is halted until the dedicated path is fixed.
12 Traffic Isolation Zoning overview • Ensure that there are multiple paths between switches. Disabling failover locks the specified route so that only TI zone traffic can use it. Non-TI zone traffic is excluded from using the dedicated path. • You should enable failover-enabled TI zones before enabling failover-disabled TI zones, to avoid dropped frames. When you issue the cfgEnable command to enable the zone configuration, if you have failover disabled zones, do the following: 1.
Traffic Isolation Zoning overview 12 If the dedicated ISL is not the lowest cost path ISL, then the following rules apply: • If failover is enabled, the traffic path for the TI zone is broken, and TI zone traffic uses the lowest cost path instead. • If failover is disabled, the TI zone traffic is blocked. If the dedicated ISL is the only lowest cost path ISL, then the following rules apply: • If failover is enabled, non-TI zone traffic as well as TI zone traffic uses the dedicated ISL.
12 Enhanced TI zones Domain 1 8 Domain 3 1 9 9 14 12 3 15 7 16 6 = Dedicated Path = Ports in the TI zone 5 Domain 4 Domain 2 FIGURE 34 Dedicated path is not the shortest path NOTE For information about setting or displaying the FSPF cost of a path, see the linkCost and topologyShow commands in the Fabric OS Command Reference. Enhanced TI zones Prior to Fabric OS v6.4.0, a port could be in only one TI zone at a time. Starting in Fabric OS v6.4.
Enhanced TI zones 12 Illegal configurations with enhanced TI zones When you create TI zones, ensure that all traffic from a port to all destinations on a remote domain have the same path. Do not create separate paths from a local port to two or more ports on the same remote domain. If the TI zones are configured with failover disabled, some traffic will be dropped.
12 Traffic Isolation Zoning over FC routers In this example traffic from the Target to Domain 2 is routed correctly. Only one TI zone describes a path to Domain 2. However, both TI zones describe different, valid paths from the Target to Domain 1. Only one path will be able to get to (1,1). Traffic from port (3,8) cannot be routed to Domain 1 over both (3,6) and (3,7), so one port will be chosen. If (3,7) is chosen, frames destined for (1,1) will be dropped at Domain 1.
Traffic Isolation Zoning over FC routers Edge fabric 1 Backbone fabric 12 Edge fabric 2 = Dedicated path set up by TI zone in edge fabric 1 = Dedicated path set up by TI zone in edge fabric 2 = Dedicated path set up by TI zone in backbone fabric FIGURE 38 Traffic Isolation Zoning over FCR In addition to setting up TI zones, you must also ensure that the devices are in an LSAN zone so that they can communicate with each other.
12 Traffic Isolation Zoning over FC routers In the TI zone, when you designate E_Ports between the front and xlate phantom switches, you must use -1 in place of the “I” in the D,I notation. Both the front and xlate domains must be included in the TI zone.
General rules for TI zones 12 Using D,I and port WWN notation, the members of the TI zone in Figure 40 are: 1,1 (EX_Port for FC router 1) 1,4 (VE_Port for FC router 1) 2,7 (VE_Port for FC router 2) 2,1 (EX_Port for FC router 2) 10:00:00:00:00:01:00:00 (Port WWN for the host) 10:00:00:00:00:02:00:00 (Port WWN for target 1) 10:00:00:00:00:03:00:00 (Port WWN for target 2) Limitations of TI zones over FC routers Be aware of the following when configuring TI zones over FC routers: • A TI zone defined within
12 Supported configurations for Traffic Isolation Zoning • Routing rules imposed by TI zones with failover disabled override regular zone definitions. Regular zone definitions should match TI zone definitions. • FSPF supports a maximum of 16 paths to a given domain. This includes paths in a TI zone. • Each TI zone is interpreted by each switch and each switch considers only the routing required for its local ports.
Limitations and restrictions of Traffic Isolation Zoning 12 Additional configuration rules for enhanced TI zones Enhanced TI zones (ETIZ) have the following additional configuration rules: • Enhanced TI zones are supported only if every switch in the fabric is ETIZ capable. A switch is ETIZ capable if it meets the following qualifications: - The switch must be one of the supported platforms, as listed in “Supported hardware and software” on page xxxiv. - The switch must be running Fabric OS v6.4.
12 Admin Domain considerations for Traffic Isolation Zoning • TI zones that have members with port index greater than 511 are not supported with Fabric OS versions earlier than v6.4.0. If such a TI zone and Fabric OS version combination is detected, a warning is issued. These configurations are not prevented, but their behavior is unpredictable.
12 Virtual Fabric considerations for Traffic Isolation Zoning Host Domain 8 8 9 1 2 5 6 3 4 8 7 LS3, FID1 Domain 3 Chassis 1 Target Domain 9 LS1, FID1 Domain 5 Domain 7 LS4, FID3 Domain 4 10 Base switch Domain 1 11 12 XISL XISL 14 13 15 XISL 16 XISL 17 Chassis 2 LS2, FID3 Domain 6 Base switch Domain 2 = Dedicated Path = Ports in the TI zones FIGURE 42 Dedicated path with Virtual Fabrics Figure 43 shows a logical representation of FID1 in Figure 42.
12 Traffic Isolation Zoning over FC routers with Virtual Fabrics Using D,I notation, the port numbers for the TI zones in the logical fabric and base fabric are as follows: Port members for the TI zone in logical fabric Port members for the TI zone in base fabric 8,8 8,1 3,3 3,10 5,16 5,8 9,5 9,9 1,3 1,10 7,12 7,14 2,16 2,8 F_Port E_Port E_Port E_Port E_Port E_Port E_Port F_Port E_Port for ISL in logical switch E_Port for XISL E_Port for XISL E_Port for XISL E_Port for XISL E_Port for ISL in logical
Creating a TI zone 12 Edge fabric Fabric 1 1 SW3 3 10 2 12 4 5 SW1 FIGURE 46 SW6 11 6 15 13 7 Backbone fabric Edge fabric Fabric 3 16 SW2 14 Logical representation of TI zones over FC routers in logical fabrics Creating a TI zone You create and modify TI zones using the zone command. Other zoning commands, such as zoneCreate, aliCreate, and cfgCreate, cannot be used to manage TI zones. When you create a TI zone, you can set the state of the zone to activated or deactivated.
12 Creating a TI zone Be aware of the ramifications if you create a TI zone with failover mode disabled. See “TI zone failover” on page 270 for information about disabling failover mode. 3. Perform the following steps if you have any TI zones with failover disabled. If all of your TI zones are failover-enabled, skip to step 4. a. Change the failover option to failover enabled. This is a temporary change to avoid frame loss during the transition. zone --add -o f name b. Enable the zones.
Creating a TI zone 12 To create TI zones in a logical fabric, such as the one shown in Figure 43 on page 283: Log in to the logical switch FID1, Domain 7 and create a TI zone in the logical fabric with FID=1: LS1> zone --create -t ti -o f "ti_zone1" -p "8,8; 8,1; 3,3; 3,10; 5,16; 5,8; 9,5; 9,9" Then create a TI zone in the base fabric, as described in “Creating a TI zone in a base fabric”.
12 Modifying TI zones Example The following example creates TI zones in the base fabric shown in Figure 44 on page 283: BS_D1> BS_D1> BS_D1> 2,8" BS_D1> zonecreate "z1", "1,1" cfgcreate "base_cfg", z1 zone --create -t ti -o f "ti_zone2" -p "1,3; 1,10; 7,12; 7,14; 2,16; cfgenable "base_config" Modifying TI zones Using the zone --add command, you can add ports to an existing TI zone, change the failover option, or both.You can also activate or deactivate the TI zone.
Changing the state of a TI zone c. 12 Reset the failover option to failover disabled. Then continue with step 4. zone --add -o n name 4. Enter the cfgEnable command to reactivate your current effective configuration and enforce the TI zones.
12 Deleting a TI zone Example of setting the state of a TI zone To change the state of the existing TI zone bluezone to activated, type: switch:admin> zone --activate bluezone To change the state of the existing TI zone greenzone to deactivated, type: switch:admin> zone --deactivate greenzone Remember that your changes are not enforced until you enter the cfgEnable command. Deleting a TI zone Use the zone --delete command to delete a TI zone from the defined configuration.
Troubleshooting TI zone routing problems 12 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the zone --show command.
12 Setting up TI over FCR (sample procedure) Following is an example report that would be generated for the illegal configuration shown in Figure 36 on page 275. switch:admin> zone --showTIerrors My Domain: 3 Error type: ERROR Affected Remote Domain: 1 Affected Local Port: 8 Affected TI Zones: etiz1, etiz2 Affected Remote Ports: 1, 2, 3, 4 Setting up TI over FCR (sample procedure) The following example shows how to set up TI zones over FCR to provide a dedicated path shown in Figure 47.
Setting up TI over FCR (sample procedure) 12 1. In each edge fabric, set up an LSAN zone that includes Host 1, Target 1, and Target 2, so these devices can communicate with each other. See Chapter 24, “Using FC-FC Routing to Connect Fabrics,” for information about creating LSAN zones. 2. Log in to the edge fabric 1 and set up the TI zone. a. Enter the fabricShow command to display the switches in the fabric. From the output, you can determine the front and translate domains.
12 Setting up TI over FCR (sample procedure) 3. Log in to the edge fabric 2 and set up the TI zone. a. Enter the fabricShow command to display the switches in the fabric. From the output, you can determine the front and translate domains. E2switch:admin> fabricshow Switch ID Worldwide Name Enet IP Addr FC IP Addr Name ------------------------------------------------------------------------1: fffc01 50:00:51:e3:95:36:7e:09 0.0.0.0 0.0.0.0 "fcr_fd_1" 4: fffc04 50:00:51:e3:95:48:9f:a1 0.0.0.0 0.0.0.
Setting up TI over FCR (sample procedure) b. 12 Enter the following commands to reactivate your current effective configuration and enforce the TI zones. BB_DCX_1:admin> cfgactvshow Effective configuration: cfg: cfg_TI zone: lsan_t_i_TI_Zone1 10:00:00:00:00:00:02:00:00 10:00:00:00:00:00:03:00:00 10:00:00:00:00:00:08:00:00 BB_DCX_1:admin> cfgenable cfg_TI You are about to enable a new zoning configuration. This action will replace the old zoning configuration with the current configuration selected.
12 296 Setting up TI over FCR (sample procedure) Fabric OS Administrator’s Guide 53-1002446-01
Chapter 13 Bottleneck Detection In this chapter • Bottleneck detection overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Supported configurations for bottleneck detection . . . . . . . . . . . . . . . . . . • Advanced bottleneck detection settings . . . . . . . . . . . . . . . . . . . . . . . . . . . • Enabling bottleneck detection on a switch . . . . . . . . . . . . . . . . . . . . . . . . . • Excluding a port from bottleneck detection. . . . . . . . . . . . . . . . . . .
13 Bottleneck detection overview You configure bottleneck detection on a per-switch basis, with optional per-port exclusions. NOTE Bottleneck detection is disabled by default. Best practice is to enable bottleneck detection on all switches in the fabric, and leave it on to continuously gather statistics. Bottleneck detection does not require a license.
Bottleneck detection overview 13 • How many affected seconds are needed to generate the alert. • How long to stay quiet after an alert Changing alerting parameters affects RASlog alerting as well as SNMP traps. Using alerting parameters to determine whether alerts are generated You have the option of receiving per-port alerts based on the latency and congestion history of the port. Alerts are generated based on the number of affected seconds over a specified period of time.
13 Supported configurations for bottleneck detection Supported configurations for bottleneck detection Note the following configuration rules for bottleneck detection: • Bottleneck detection is supported only on Fibre Channel ports and FCoE F_Ports. • Bottleneck detection is supported only on the following port types: - E_Ports - EX_Ports - F_Ports - FL_Ports • F_Port and E_Port trunks are supported. • Long distance E_Ports are supported. • FCoE F_Ports are supported.
Advanced bottleneck detection settings 13 Trunking considerations for bottleneck detection A trunk behaves like a single port. Both latency and congestion bottlenecks are reported on the master port only, but apply to the entire trunk. For masterless trunking, if the master port goes offline, the new master acquires all the configurations and bottleneck history of the old master and continues with bottleneck detection on the trunk.
13 Enabling bottleneck detection on a switch The sub-second latency criterion parameters are always applicable. These parameters affect alerts and, even if alerting is not enabled, they affect the history of bottleneck statistics. The sub-second latency criterion parameters are the following, with default values in parentheses: • -lsubsectimethresh (0.8) is similar to the -lthresh alerting parameter, except on a sub-second level. The default value of 0.
Excluding a port from bottleneck detection 13 By default, alerts are not sent unless you specify the alert parameter; however, you can view a history of bottleneck conditions for the port as described in “Displaying bottleneck statistics” on page 307. 3. Repeat step 1 and step 2 on every switch in the fabric. NOTE Best practice is to use the default values for the alerting and sub-second latency criterion parameters.
13 Displaying bottleneck detection configuration details Displaying bottleneck detection configuration details 1. Connect to the switch and log in using an account with admin permissions. 2.
Changing bottleneck parameters 13 Changing bottleneck parameters When you enable bottleneck detection, you can configure switch-wide alerting and sub-second latency criterion parameters that apply to every port on the switch. After you enable bottleneck detection, you can change the alerting parameters on the entire switch or on individual ports. You can change the sub-second latency criterion parameters on individual ports only.
13 Changing bottleneck parameters ==== 2 3 4 Example The following example changes alerting parameters for the entire logical switch. switch:admin> bottleneckmon --config -alert -lthresh .97 -cthresh .8 -time 5000 switch:admin> bottleneckmon --status Bottleneck detection - Enabled ============================== Switch-wide sub-second latency bottleneck criterion: ==================================================== Time threshold - 0.800 Severity threshold - 0.
Displaying bottleneck statistics Alerts Latency threshold for alert Congestion threshold for alert Averaging time for alert Quiet time for alert - 13 Yes 0.100 0.800 300 seconds 300 seconds Per-port overrides for sub-second latency bottleneck criterion: =============================================================== Port TimeThresh SevThresh ================================= 6 0.600 40.
13 Disabling bottleneck detection on a switch Disabling bottleneck detection on a switch When you disable bottleneck detection on a switch, all bottleneck configuration details are discarded, including the list of excluded ports and non-default values of alerting parameters. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the bottleneckmon --disable command to disable bottleneck detection on the switch.
Chapter In-flight Encryption and Compression 14 In this chapter • In-flight encryption and compression overview . . . . . . . . . . . . . . . . . . . . . . 309 • Configuring encryption and compression . . . . . . . . . . . . . . . . . . . . . . . . . . 312 • Encryption and compression example . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14 In-flight encryption and compression overview on En cr yp tio si es pr FIGURE 49 om 16G C n 16G Compression/Encryption 16G Encryption and compression on 16 Gbps ISLs The encryption and compression features are designed to work only with E_Ports. Encryption and compression are also compatible with the following features: • • • • E_Ports with trunking, QoS, or long distance features enabled. Flow control modes R_RDY, VC_RDY, and EXT_VC_RDY. XISL ports in VF mode.
In-flight encryption and compression overview 14 How encryption and compression are enabled This feature provides encryption and decryption or compression and decompression between two E_Ports across an ISL. You can enable encryption or compression or both on an E_Port on a per port basis. By default, this feature is disabled on all ports on a switch. Encryption and compression capabilities and configurations from each end of the ISL are exchanged during E_Port initialization.
14 Configuring encryption and compression Virtual Fabrics considerations The E_Ports in the user-created logical switch, base switch, or default switch can support encryption and compression. You can configure encryption on XISL ports, but not on LISL ports. However, frames from the LISL ports are implicitly encrypted or compressed as they pass through encryption/compression enabled XISL ports.
Configuring encryption and compression 14 These steps summarize how to enable encryption or compression on a port: 1. Use the portEncCompShow command to determine which ports are available for encryption or compression. 2. If you are enabling encryption on the port, configure port level authentication for the port using the secAuthSecret and authUtil commands. Omit this step if you want to enable only compression on the port. 3. Use the portCfgEncrypt command to enable encryption on the port.
14 Configuring encryption and compression 22 No No No No 23 No No No No 144 No No No No 145 No No No No 146 No No No No 147 No No No No 148 No No No No 149 No No No No 150 No No No No 151 No No No No ----------------------------------------------------88 No No No No 89 No No No No 90 No No No No 91 No No No No 92 No No No No 93 No No No No 94 No No No No 95 No No No No 208 No No No No 209 No No No No 210 No No No No 211 No No No No 212 No No No No 213 No No No No 214 No No No No 215 No No No No 344 No No
Configuring encryption and compression 14 3. Enter the authUtil command to set the switch policy mode to Active or On: authutil --policy -sw active or: authutil --policy -sw on 4. Enable the DH-CHAP authentication protocol: authutil --set -a dhchap or: authutil --set -a all 5. Enable authentication with DH group 4 or “*”: authutil --set -g 4 DH Group was set to 4. or authutil --set -g "*" DH Group was set to 0,1,2,3,4.
14 Configuring encryption and compression Configuring compression NOTE Before performing this procedure, it is recommended that you check for port availability using the portEncCompShow command. See “Viewing the encryption and compression configuration” on page 313 for details. To configure compression on a port, follow these steps: 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the SwitchPortConfiguration RBAC class of commands. 2.
Encryption and compression example 14 Disabling compression To disable compression on a port, follow these steps: 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the SwitchPortConfiguration RBAC class of commands. 2. Disable the port on which you want to disable compression. Use the portDisable command.Enter the portCfgCompress --disable command.
14 Encryption and compression example Example of enabling encryption and compression on a port This example configures and enables encryption and compression on a given port. Authentication and secret key must also be configured as these are required before configuring encryption. The commands in this example are shown entered on the Brocade 6510 named myswitch. The same commands must also be entered on the peer switch.
Encryption and compression example 14 2. Peer secret: The secret of the peer that authenticates to peer. 3. Local secret: The local secret that authenticates peer. Press enter to start setting up secrets >1 Enter peer WWN, Domain, or switch name (Leave blank when done): 10:00:00:05:1e:e5:cb:00 Enter peer secret: Re-enter peer secret: Enter local secret: Re-enter local secret: Enter peer WWN, Domain, or switch name (Leave blank when done): Are you done? (yes, y, no, n): [no] y Saving data to key store...
14 Encryption and compression example Frame Shooter Port D-Port mode: Compression: Encryption: FEC: myswitch:root> OFF OFF OFF ON OFF Finally, you enable compression on the same port. The subsequent portCfgShow command shows both encryption and compression to be enabled on the port.
Encryption and compression example 14 Next, disable compression: myswitch:root> portdisable 0 myswitch:root> portcfgcompress --disable 0 myswitch:root> portenable 0 Now use the portCfgShow command to check the results: myswitch:root> portcfgshow 0 Area Number: 0 Octet Speed Combo: 3(16G,10G) Speed Level: AUTO(SW) AL_PA Offset 13: OFF Trunk Port ON Long Distance OFF VC Link Init OFF Locked L_Port OFF Locked G_Port OFF Disabled E_Port OFF Locked E_Port OFF ISL R_RDY Mode OFF RSCN Suppressed OFF Persistent
14 322 Encryption and compression example Fabric OS Administrator’s Guide 53-1002446-01
Chapter 15 NPIV In this chapter • NPIV overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Configuring NPIV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Enabling and disabling NPIV. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Viewing NPIV port configuration information. . . . . . . . . . . . . . . . . . . . . . . .
15 NPIV overview Index Port Address Media Speed State Proto ============================================== 0 0 010000 id N4 Online FC F-Port 1 1 010100 id N4 Online FC F-Port 2 2 010200 id N4 Online FC F-Port 3 3 010300 id N4 Online FC F-Port 20:0c:00:05:1e:05:de:e4 0xa06601 1 N Port + 4 NPIV public 1 N Port + 119 NPIV public 1 N Port + 221 NPIV public On the Brocade DCX and DCX-4S with the FC8-64 blade, the base port is not included in the NPIV device count.
Configuring NPIV TABLE 60 15 Number of supported NPIV devices (Continued) Platform Virtual Fabrics Logical switch type NPIV support DCX-4S Enabled Logical switch Yes, 255 virtual device limit.3 DCX-4S Enabled Base switch No. 1. Maximum limit support takes precedence if user-configured maximum limit is greater. This applies to shared areas on the FC4-48, FC8-48 and FC8-64 port blades. 2.
15 Enabling and disabling NPIV VC Link Init Locked L_Port Locked G_Port Disabled E_Port Locked E_Port ISL R_RDY Mode RSCN Suppressed Persistent Disable LOS TOV enable NPIV capability QOS E_Port Port Auto Disable: Rate Limit EX Port Mirror Port Credit Recovery F_Port Buffers Fault Delay: NPIV PP Limit: CSCTL mode: Frame Shooter Port D-Port mode: Compression: Encryption: FEC: OFF OFF OFF OFF OFF OFF OFF OFF OFF ON AE OFF OFF OFF OFF ON OFF 0(R_A_TOV) 128 OFF OFF OFF OFF OFF ON Enabling and disabling NPIV
Viewing NPIV port configuration information 15 Viewing NPIV port configuration information 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the portCfgShow command to view the switch ports information.
15 Viewing NPIV port configuration information portName: 02 portHealth: HEALTHY Authentication: None portDisableReason: None portCFlags: 0x1 portFlags: 0x24b03 PRESENT ACTIVE F_PORT G_PORT NPIV LOGICAL_ONLINE LOGIN NOELP LED ACCEPT portType: 10.0 portState: 1Online portPhys: 6In_Sync portScn: 32F_Port port generation number: 148 portId: 630200 portIfId: 43020005 portWwn: 20:02:00:05:1e:35:37:40 portWwn of device(s) connected: c0:50:76:ff:fb:00:16:fc c0:50:76:ff:fb:00:16:f8 ...
Chapter 16 Dynamic Fabric Provisioning: Fabric-Assigned WWN In this chapter • Introduction to Dynamic Fabric Provisioning using FA-PWWN . . . . . . . . . . • User- and auto-assigned FA-PWWN behavior . . . . . . . . . . . . . . . . . . . . . . . • Configuring FA-PWWNs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Supported switches and configurations for FA-PWWN . . . . . . . . . . . . . . . . • Configuration upload and download considerations for FA-PWWN . . . . . .
16 User- and auto-assigned FA-PWWN behavior NOTE For the server to use the FA-PWWN feature, it must be using a Brocade HBA or Adapter. Refer to the release notes for the HBA or Adapter versions that support this feature. Some configuration of the HBA must be performed to use the FA-PWWN. User- and auto-assigned FA-PWWN behavior An FA-PWWN can be either user-generated or automatically assigned by the fabric.
Configuring FA-PWWNs 16 This section includes an FA-PWWN configuration procedure for each of the following two topologies: • An FA-PWWN for an HBA device that is connected to an Access Gateway switch. • An FA-PWWN for an HBA device that is connected directly to an edge switch. These topologies are shown in Figure 50. Access Gateway Switch Edge Switch running FOS 7.0.0 running FOS 7.0.
16 Configuring FA-PWWNs 10:00:00:05:1e:d7:3d:dc/9 20 20:09:00:05:1e:d7:2b:73 \ 10:00:00:05:1e:d7:3d:dc/16 ---:--:--:--:--:--:--:-- \ -----------------------------------------------------------Virtual Port WWN PID Enable MapType -----------------------------------------------------------52:00:10:00:00:0f:50:30 -Yes AG/Auto 11:22:33:44:55:66:77:88 11403 Yes AG/User 52:00:10:00:00:0f:50:32 2:00:10:00:00:0f:50:33 11404 Yes AG/Auto 52:00:10:00:00:0f:50:38 -Yes AG/Auto 4. Enable the FA-PWWN on the HBA.
Supported switches and configurations for FA-PWWN 16 4. Enable the FA-PWWN on the HBA. The following steps are to be executed on the server and not the switch. a. Log in to the server as root. b. Enter the following command: bcu port -faa port_id --enable c. Enter the following command: bcu port -faa port_id --query Once the Brocade HBA has been assigned the FA-PWWN, the HBA retains the FA-PWWN until it is rebooted. This means you cannot unplug and plug the cable into a different port on the switch.
16 Configuration upload and download considerations for FA-PWWN Configuration upload and download considerations for FA-PWWN The configuration upload and download utilities can be used to import and export the FA-PWWN configuration. ATTENTION Brocade recommends you delete all FA-PWWNs from the switch with the configuration being replaced before you upload or download a modified configuration. This is to ensure no duplicate FA-PWWNs in the fabric.
Restrictions of FA-PWWN 16 Restrictions of FA-PWWN Note the following restrictions when using the FA-PWWN feature: • FA-PWWN is supported only on Brocade HBAs and Adapters. Refer to the release notes for the supported Brocade HBA or Adapter versions.
16 336 Access Gateway N_Port failover with FA-PWWN Fabric OS Administrator’s Guide 53-1002446-01
Chapter Managing Administrative Domains 17 In this chapter • Administrative Domains overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 • Admin Domain management for physical fabric administrators . . . . . . . . 346 • SAN management with Admin Domains . . . . . . . . . . . . . . . . . . . . . . . . . . .
17 Administrative Domains overview NOTE Do not confuse an Admin Domain number with the domain ID of a switch. They are two different identifiers. The Admin Domain number identifies the Admin Domain and has a range from 0 through 255. The domain ID identifies a switch in the fabric and has a range from 1 through 239. Figure 51 shows a fabric with two Admin Domains: AD1 and AD2.
Administrative Domains overview 17 Admin Domain features Admin Domains allow you to do the following: • Define the scope of an Admin Domain to encompass ports and devices within a switch or a fabric. • Share resources across multiple Admin Domains. For example, you can share array ports and tape drives between multiple departments. In Figure 51 on page 338, one of the storage devices is shared between AD1 and AD2. • Have a separate zone database for each Admin Domain.
17 Administrative Domains overview Table 61 lists each Admin Domain user type and describes its administrative access and capabilities. TABLE 61 AD user types User type Description Physical fabric administrator User account with admin permissions and with access to all Admin Domains (AD0 through AD255). Creates and manages all Admin Domains. Assigns other administrators or users to each Admin Domain. The default admin account is the first physical fabric administrator.
Administrative Domains overview 17 For example, if DeviceA is not a member of any user-defined Admin Domain, then it is an implicit member of AD0. If you explicitly add DeviceA to AD0, then DeviceA is both an implicit and an explicit member of AD0. AD0 implicit members DeviceA AD0 explicit members DeviceA AD2 members none If you add DeviceA to AD2, then DeviceA is deleted from the AD0 implicit membership list, but is not deleted from the AD0 explicit membership list.
17 Administrative Domains overview FIGURE 53 Fabric with AD0 and AD255 Home Admin Domains and login You are always logged in to an Admin Domain, and you can view and modify only the devices in that Admin Domain. If you have access to more than one Admin Domain, one of them is designated as your home Admin Domain, the one you are automatically logged in to.
Administrative Domains overview 17 Admin Domain member types You define an Admin Domain by identifying members of that domain. Admin Domain members can be devices, switch ports, or switches. Defining these member types is similar to defining a traditional zone member type. An Admin Domain does not require or have a new domain ID or management IP address linked to it.
17 Administrative Domains overview Switch members Switch members are defined by the switch WWN or domain ID, and have the following properties: • A switch member grants administrative control to the switch. • A switch member grants port control for all ports in that switch. • A switch member allows switch administrative operations such as disabling and enabling a switch, rebooting, and firmware downloads. • A switch member does not provide zoning rights for the switch ports or devices.
Administrative Domains overview FIGURE 54 17 Fabric showing switch and device WWNs Figure 55 shows the filtered view of the fabric as seen from AD3 and AD4. The switch WWNs are converted to the NAA=5 syntax; the device WWNs and domain IDs remain the same.
17 Admin Domain management for physical fabric administrators Admin Domain compatibility, availability, and merging Admin Domains maintain continuity of service for Fabric OS features and operate in mixed-release Fabric OS environments. High availability is supported with some backward compatibility. When an E_Port comes online, the adjacent switches merge their AD databases.
Admin Domain management for physical fabric administrators 17 1. Log in to the switch with the appropriate RBAC role. 2. Ensure you are in the AD0 context by entering the ad --show command to determine the current Admin Domain. If necessary, switch to the AD0 context by entering the ad --select 0 command. 3. Set the default zoning mode to No Access, as described in “Setting the default zoning mode” on page 253.
17 Admin Domain management for physical fabric administrators 5. Enter the ad --create command using the -d option to specify device and switch port members and the -s option to specify switch members: ad --create ad_id -d "dev_list" -s "switch_list" 6. Enter the appropriate command based on whether you want to save or activate the Admin Domain definition: • To save the Admin Domain definition, enter ad --save.
Admin Domain management for physical fabric administrators 17 Creating a new user account for managing Admin Domains 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the userConfig --add command using the -r option to set the role, the -a option to provide access to Admin Domains, and the -h option to specify the home Admin Domain.
17 Admin Domain management for physical fabric administrators Removing an Admin Domain from a user account When you remove an Admin Domain from an account, all of the currently active sessions for that account are logged out. 1. Connect to the switch and log in using an account with admin permissions. 2.
Admin Domain management for physical fabric administrators 17 Deactivating an Admin Domain If you deactivate an Admin Domain, the members assigned to the Admin Domain can no longer access their hosts or storage unless those members are part of another Admin Domain. You cannot log in to an Admin Domain that has been deactivated. You must activate an Admin Domain before you can log in to it. 1. Connect to the switch and log in using an account with admin permissions. 2.
17 Admin Domain management for physical fabric administrators 4. Enter the appropriate command based on whether you want to save or activate the Admin Domain definition: • To save the Admin Domain definition, enter ad --save. • To save the Admin Domain definition and directly apply the definition to the fabric, enter ad --apply.
Admin Domain management for physical fabric administrators 17 3. Enter the ad --rename command with the present name and the new name. ad --rename present_name new_name 4. Enter the appropriate command based on whether you want to save or activate the Admin Domain definition: • To save the Admin Domain definition, enter ad --save. • To save the Admin Domain definition and directly apply the definition to the fabric, enter ad --apply. The Admin Domain numbers remain unchanged after the operation.
17 Admin Domain management for physical fabric administrators Deleting all user-defined Admin Domains When you clear the Admin Domain configuration, all user-defined Admin Domains are deleted, the explicit membership list of AD0 is cleared, and all fabric resources (switches, ports, and devices) are returned to the implicit membership list of AD0. You cannot clear the Admin Domain configuration if zone configurations exist in any of the user-defined Admin Domains.
Admin Domain management for physical fabric administrators 17 3. Enter the zone --copy command to copy the zones from all user-defined Admin Domains to AD0. zone --copy source_AD.source_name dest_name In this syntax, source_AD is the name of the user-defined AD from which you are copying the zone, source_name is the name of the zone to be copied, and dest_name is the name to give to the zone after it is copied to AD0. 4. Copy the newly added zones in AD0 to the zone configuration.
17 Admin Domain management for physical fabric administrators FIGURE 56 AD0 and two user-defined Admin Domains, AD1 and AD2 At the conclusion of the procedure, all devices and zones are moved to AD0, and the user-defined Admin Domains are deleted, as shown in Figure 57.
Admin Domain management for physical fabric administrators 17 10:00:00:00:02:00:00:00; 10:00:00:00:03:00:00:00 Effective configuration: cfg: AD1_cfg zone: AD1_BlueZone 10:00:00:00:02:00:00:00 10:00:00:00:03:00:00:00 Zone CFG Info for AD_ID: 2 (AD Name: AD2, State: Active) : Defined configuration: cfg: AD2_cfg AD2_GreenZone zone: AD2_GreenZone 10:00:00:00:04:00:00:00; 10:00:00:00:05:00:00:00 Effective configuration: cfg: AD2_cfg zone: AD2_GreenZone 10:00:00:00:04:00:00:00 10:00:00:00:05:00:00:00 sw0:adm
17 SAN management with Admin Domains Validating an Admin Domain member list You can validate the device and switch member list. You can list non-existing or offline Admin Domain members. You can also identify misconfigurations of the Admin Domain. The Admin Domain validation process is not applicable for AD0, because AD0 implicitly contains all unassigned online switches and their devices. 1. Connect to the switch and log in using an account with admin permissions. 2.
SAN management with Admin Domains 17 CLI commands in an AD context The CLI command input arguments are validated against the AD member list; they do not work with input arguments that specify resources that are not members of the current Admin Domain. All commands present filtered output, showing only the members of the current Admin Domain. For example, switchShow displays details for the list of AD members present in that switch.
17 SAN management with Admin Domains • AD0–AD254: The membership of the current Admin Domain is displayed. • AD0: The device and switch list members are categorized into implicit and explicit member lists. 1. Connect to the switch and log in as any user type. 2. Enter the ad --show command. ad --show If you are in the AD0 context, you can use the -i option to display the implicit membership list of AD0; otherwise, only the explicit membership list is displayed.
SAN management with Admin Domains 17 Example of switching to a different Admin Domain context The following example switches to the AD12 context and back. Note that the prompt changes to display the Admin Domain. switch:admin> ad --select 12 switch:AD12:admin> logout switch:admin> Admin Domain interactions with other Fabric OS features The Admin Domain feature provides interaction with other Fabric OS features and across third-party applications.
17 SAN management with Admin Domains TABLE 63 Admin Domain interaction with Fabric OS features (Continued) Fabric OS feature Admin Domain interaction FICON Admin Domains support FICON. However, you must perform additional steps because FICON management requires additional physical control of the ports. You must set up the switch as a physical member of the FICON AD.
SAN management with Admin Domains 17 The AD zone database also has the following characteristics: - Each zone database has its own name space. For example, you can define a zone name of test_z1 in more than one Admin Domain. - There is no zone database linked to the physical fabric (AD255) and no support for zone database updates. In the physical fabric context (AD255), you can only view the complete hierarchical zone database, which is all of the zone databases in AD0 through AD254.
17 SAN management with Admin Domains LSAN zone names in AD0 are never converted for backward-compatibility reasons. The auto-converted LSAN zone names might collide with LSAN zone names in AD0 (in the example, if AD0 contains lsan_for_linux_farm_AD005, this causes a name collision). Fabric OS does not detect or report such name clashes. LSAN zone names greater than 57 characters are not converted or sent to the FCR phantom domain.
Section Licensed Features II This section describes optionally licensed Brocade Fabric OS features and includes the following chapters: • • • • • • • Chapter 18, “Administering Licensing” Chapter 19, “Inter-chassis Links” Chapter 20, “Monitoring Fabric Performance” Chapter 21, “Optimizing Fabric Behavior” Chapter 22, “Managing Trunking Connections” Chapter 23, “Managing Long Distance Fabrics” Chapter 24, “Using FC-FC Routing to Connect Fabrics” Fabric OS Administrator’s Guide 53-1002446-01 365
366 Fabric OS Administrator’s Guide 53-1002446-01
Chapter 18 Administering Licensing In this chapter • Licensing overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Brocade 7800 Upgrade license . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • ICL licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • 8G licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
18 Licensing overview Table 65 lists the optionally licensed features that are available in Fabric OS 7.0.1: TABLE 65 Available Brocade licenses License Description 10 Gigabit FCIP/Fibre Channel (10G license) • • • • 7800 Upgrade • • • Allows 10 Gbps operation of FC ports on the Brocade 6510 switch or the FC ports of FC16-32 or FC16-48 port blades installed on a Brocade DCX 8510 Backbone.
Licensing overview TABLE 65 18 Available Brocade licenses (Continued) License Description Brocade Advanced Performance Monitoring • Brocade Extended Fabrics Provides greater than 10 km of switched fabric connectivity at full bandwidth over long distances (depending on the platform, this can be up to 3000 km). • Enables performance monitoring of networked storage resources. Includes the Top Talkers feature.
18 Licensing overview TABLE 65 Available Brocade licenses (Continued) License Description FCoE Included with the Brocade 8000 switch; enables Fibre Channel over Ethernet (FCoE) functions. FICON Management Server (Also known as Control Unit Port or “CUP”) Enables host-control of switches in mainframe environments. High Performance Extension over FCIP/FC (formerly known as “FC-IP Services”) Includes the IPsec capabilities. Applies to the FR4-18i blade.
Licensing overview TABLE 66 18 License requirements and location name by feature Feature License Where license should be installed Adaptive Rate Limiting Advanced Extension Local switch. Administrative Domains No license required. N/A Bottleneck Detection No license required. N/A Configuration up/download No license required. N/A Converged Enhanced Ethernet Requires FCoE base license and POD1 license. Brocade Network Advisor No license required for base use.
18 Licensing overview TABLE 66 License requirements and location name by feature (Continued) Feature Full fabric connectivity License Where license should be installed Full Fabric Local switch. May be required on attached switches. NOTE: Also called the Fabric license (visible in licenseShow output) and the E_Port Upgrade license. In-flight encryption and compression No license required. N/A Inband Management No license required.
Licensing overview TABLE 66 18 License requirements and location name by feature (Continued) Feature License Where license should be installed Ports • Local switch. • • • • Ports on Demand licenses required, applicable to a select set of switches only. 7800 Upgrade license for the 7800 switches to use all ports. 10 Gigabit FCIP/Fibre Channel license to use 10Gb FC ports on FC16-32 blades, FC16-48 blades, and the Brocade 6510.
18 Brocade 7800 Upgrade license TABLE 66 License requirements and location name by feature (Continued) Feature License Where license should be installed Virtual Fabrics No license required. N/A Web Tools No license required. Local and any switch you will be managing using Web Tools. Zoning No license required. N/A Brocade 7800 Upgrade license The Brocade 7800 has four Fibre Channel (FC) ports and two GbE ports active by default.
ICL licensing 18 On the Brocade DCX 8510-8, this license enables QSFP ports 0–7; QSFP ports 8–15 are disabled. (QSFP ports 0–7 correspond to core blade port numbers 0–31, and QSFP ports 8–15 correspond to core blade port numbers 32–63, as observed in switchShow output.) This license allows you to purchase half the bandwidth of the Brocade DCX 8510-8 ICL ports initially and upgrade with an additional ICL license to use the full ICL bandwidth later.
18 8G licensing 8G licensing ATTENTION This license is installed by default and you should not remove it. Port operation may become disrupted, and ports may be prevented from operating at 8 Gbps when the license is removed. The 8 Gbps license applies to the Brocade 300, 5100, 5300, and VA-40FC switches and the 8 Gbps embedded switches; this license does not apply to the Brocade 6505 or 6510.
10G licensing 18 Once a license is assigned to a slot, whether it has been automatically assigned or manually assigned, the assignment will remain until you manually reassign the license to another slot. This design allows for various maintenance operations to occur without having the license move around to other slots. For a slot-based licensed feature to be active, follow these steps: 1.
18 10G licensing This 10G license is applied as a slot-based license on the FC16-32 and FC16-48 port blades and on the FX8-24 extension blade; generic rules for adding slot-based licenses apply, as described in “Slot-based licensing” on page 376. When this license is applied to the Brocade 6510 switch, it is applied to the whole chassis.
10G licensing 18 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the license and switchportconfiguration classes of RBAC commands. 2. Use the licenseAdd command to add the 10G license. 3. Use the licenseShow command to verify the license. Bladed platforms only: If the results of the automatic license assignment are not what you intended, use the licenseSlotCfg command to reassign the license to the desired blades. 4.
18 10G licensing Enabling the 10-GbE ports on an FX8-24 blade To enable the 10-GbE ports on an FX8-24 blade, follow these steps. 1. Connect to the Brocade Backbone and log in using an account with admin permissions, or an account with OM permissions for the license class of RBAC commands. 2. Use the licenseAdd command to add the 10G license. 3. Use the licenseShow command to check the results of automatic license assignment.
Temporary licenses 18 Temporary licenses A temporary license applies a “try-before-you-buy” approach to certain features so that you can experience the feature and its capabilities prior to buying the license. Once you have installed the license, you are given a time limit to use the feature. A temporary license can be either a regular temporary license or a universal temporary license: • A regular temporary license is available on a per-switch basis.
18 Temporary licenses Date change restriction Once the temporary license is installed, you cannot change the time of the switch until the temporary license is removed. To change the time, you must remove the license, change the date, and then re-install the license on the switch. However, if there is any other mechanism that exists to change the time, such as NTP, then it is not blocked.
Viewing installed licenses 18 Extending a universal temporary license Extending a universal temporary license is done by adding a temporary license with an expiry date after the universal temporary license expiry date, or by adding a permanent license. Re-applying an existing universal temporary license is not allowed. Universal temporary license shelf life All universal temporary licenses are encoded with a “shelf life” expiration date.
18 Removing a licensed feature For the Brocade Backbones, licenses are effective on both CP blades, but are valid only when the CP blade is inserted into a Backbone that has an appropriate license ID stored in the WWN card. If a CP is moved from one Backbone to another, the license works in the new Backbone only if the WWN card is the same in the new Backbone.
Ports on Demand 18 3. Remove the license key using the licenseRemove command. The license key is case-sensitive and must be entered exactly as given. The quotation marks are optional. After removing a license key, the licensed feature is disabled when the switch is rebooted or when a switch disable and enable is performed. 4. Enter the licenseShow command to verify the license is disabled.
18 Ports on Demand Table 68 shows the ports that are enabled by default and the ports that can be enabled after you install the first and second Ports on Demand licenses for each switch type.
Ports on Demand 18 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the licenseshow command.
18 Ports on Demand If the switch detects more active links than allowed by the current POD licenses, then some ports will not be assigned a POD license. Ports that do not receive a POD assignment have a state of No Sync or In Sync; these ports are not allowed to progress to the online state. Ports that cannot be brought online because of insufficient POD licenses have a state of (No POD License) Disabled. You can use the switchShow command to display the port states.
Ports on Demand 18 12 port assignments are provisioned by a full POD license 8 ports are assigned to installed licenses: 8 ports are assigned to the base switch license 0 ports are assigned to the full POD license Ports assigned to the base switch license: 1, 2, 5, 6, 8*, 21, 22, 23 Ports assigned to the full POD license: None Ports not assigned to a license: 0, 3, 4, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 16 license reservations are still available for use by unassigned ports 1 license assignme
18 Ports on Demand 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the licensePort --show command to verify there are port reservations available.
Ports on Demand 18 12 port assignments are provisioned by the base switch license 12 port assignments are provisioned by a full POD license 10 ports are assigned to installed licenses: 10 ports are assigned to the base switch license 0 ports are assigned to the full POD license Ports assigned to the base switch license: 1*, 2*, 3*, 4*, 5*, 6*, 8*, 21, 22, 23 Ports assigned to the full POD license: None Ports not assigned to a license: 0, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 6.
18 392 Ports on Demand Fabric OS Administrator’s Guide 53-1002446-01
Chapter 19 Inter-chassis Links In this chapter • Inter-chassis links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • ICLs for the Brocade DCX 8510 Backbone family . . . . . . . . . . . . . . . . . . . . • ICLs for the Brocade DCX Backbone family . . . . . . . . . . . . . . . . . . . . . . . . . • Virtual Fabrics considerations for ICLs . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Supported topologies for ICL connections. . . . . . . . . . . . . . .
19 ICLs for the Brocade DCX 8510 Backbone family Refer to the hardware reference manuals for additional information about LED status and ICL connections, including instructions on how to cable ICLs. ICLs for the Brocade DCX 8510 Backbone family Each ICL connects the core blades of two Brocade DCX 8510 chassis and provides up to 64 Gbps of throughput within a single cable.
ICLs for the Brocade DCX Backbone family 19 NOTE QSFP ICLs and ISLs in the same switch and connected to the same neighboring switch are not supported. This is a topology restriction with 16 Gbps ICLs and any ISLs that are E_Ports or VE_Ports. ICL trunking on the Brocade DCX 8510-8 and DCX 8510-4 ICL trunks automatically form on the ICLs if the ISL Trunking license is installed on each platform. Each QSFP has four ports, each terminating on a different ASIC.
19 Virtual Fabrics considerations for ICLs FIGURE 59 DCX-4S allowed ICL connections The following ICL connections are not allowed: • ICL0 ports to ICL0 ports • ICL1 ports to ICL1 ports ICL trunking on the Brocade DCX and DCX-4S On the Brocade DCX and DCX-4S, trunks are automatically formed on the ICLs when you install the ISL Trunking license on each platform. The ICLs are managed the same as ISL trunks. • On the Brocade DCX, each ICL is managed as two 8-port ISL trunks.
Supported topologies for ICL connections 19 Supported topologies for ICL connections You can connect the Brocade Backbones in a mesh topology and a core-edge topology. A brief description of each follows. The illustrations in this section show sample topologies. Refer to the Brocade SAN Scalability Guidelines for details about maximum topology configurations. Mesh topology You can connect the Brocade Backbones in a mesh topology, in which every chassis is connected to every other chassis.
19 Supported topologies for ICL connections Core-edge topology You can also connect the Brocade DCX 8510 Backbones in a core-edge topology. For example, Figure 61 shows six chassis connected in a core-edge topology (four edges and two cores). Although Figure 61 shows only the Brocade DCX 8510-8, each chassis can be either a Brocade DCX 8510-4 or a DCX 8510-8. Each line in Figure 61 represents four QSFP cables. The cabling scheme should follow the parallel example shown in Figure 58.
Chapter 20 Monitoring Fabric Performance In this chapter • Advanced Performance Monitoring overview . . . . . . . . . . . . . . . . . . . . . . . • End-to-end performance monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Frame monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Top Talker monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Trunk monitoring . . . . . . . . . . . . . . .
20 Advanced Performance Monitoring overview Restrictions for installing monitors • Advanced Performance Monitoring is not supported on VE_Ports and EX_Ports. If you issue commands for any Advanced Performance Monitors on VE_Ports or EX_Ports you will receive error messages. • For the Brocade 8000, performance monitoring is supported only on the FC ports and not on the CEE ports. • All monitor types are allowed only on physical ports.
End-to-end performance monitoring 20 Access Gateway considerations for Advanced Performance Monitoring EE monitors and frame monitors are supported on switches in Access Gateway mode. Top Talker monitors are not supported on these switches. EE monitors must be installed on F_Ports. Frame monitors can be installed on F_Ports or N_Ports. See the Access Gateway Administrator’s Guide for additional information.
20 End-to-end performance monitoring Supported port configurations for EE monitors You can configure EE monitors on F_Ports and, depending on the switch model, on E_Ports. The following platforms support EE monitors on E_Ports: • Brocade 6505 • Brocade 6510 • Brocade DCX 8510 family Identical EE monitors cannot be added to the same port. Two EE monitors are considered identical if they have the same SID and DID values after applying the end-to-end mask.
End-to-end performance monitoring 20 Example of monitoring the traffic from Dev B to Host A On Domain 2, add a monitor to the F_Port as follows: switch:admin> perfaddeemonitor 2/14 "0x021e00" "0x011200" This monitor (Monitor 4) counts the frames that have an SID of 0x021e00 and a DID of 0x011200. For Monitor 4, RX_COUNT is the number of words from Dev B to Host A, and TX_COUNT is the number of words from Host A to Dev B.
20 End-to-end performance monitoring Figure 63 shows the mask positions in the command. A mask (“ff”) is set on slot 1, port 2 to compare the AL_PA fields on the SID and DID in all frames (transmitted and received) on port 2. The frame SID and DID must match only the AL_PA portion of the specified SID-DID pair. Each port can have only one EE mask. The mask is applied to all end-to-end monitors on the port. Individual masks for each monitor on the port cannot be specified.
End-to-end performance monitoring 20 Example of displaying an end-to-end monitor on a port at 10-second intervals switch:admin> perfMonitorShow --class EE 4/5 10 Showing EE monitors 4/5 10: Tx/Rx are # of bytes 0 1 2 3 4 --------- --------- --------- --------- --------Tx Rx Tx Rx Tx Rx Tx Rx Tx Rx ========= ========= ========= ========= ========= 0 0 0 0 0 0 0 0 0 0 53m 4.9m 53m 4.9m 53m 4.9m 53m 4.9m 53m 0 53m 4.4m 53m 4.4m 53m 4.4m 53m 4.4m 53m 0 53m 4.8m 53m 4.8m 53m 4.8m 53m 4.8m 53m 0 53m 4.6m 53m 4.
20 Frame monitoring Frame monitoring Frame monitoring counts the number of times a frame with a particular pattern is transmitted by a port and generates alerts when thresholds are crossed. Frame monitoring is achieved by defining a filter, or frame type, for a particular purpose. The frame type can be a standard type (for example, a SCSI read command filter that counts the number of SCSI read commands that have been transmitted by the port) or a user-defined frame type customized for your particular use.
Frame monitoring 20 You can specify up to four values to compare against each offset. If more than one offset is required to properly define a filter, the bytes found at each offset must match one of the given values for the filter to increment its counter. If one or more of the given offsets does not match any of the given values, the counter does not increment. The value of the offset must be between 0 and 63, in decimal format.
20 Frame monitoring Adding frame monitors to a port If the switch does not have enough resources to add a frame monitor to a port, then other frame monitors on that port might have to be deleted to free resources. 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the fmMonitor --addmonitor command to add a frame monitor to one or more ports.
Frame monitoring 20 Displaying frame monitors 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the fmmonitor --show command.
20 Top Talker monitors Example This example clears the counters for the ABTS monitor from ports 7 through 10. switch:admin> fmmonitor --clear ABTS -port 7-10 Top Talker monitors Top Talker monitors determine the flows (SID/DID pairs) that are the major users of bandwidth (after initial stabilization). Top Talker monitors measure bandwidth usage data in real-time and relative to the port on which the monitor is installed.
Top Talker monitors 20 How do Top Talker monitors differ from EE monitors? EE monitors provide counter statistics for traffic flowing between a given SID-DID pair. Top Talker monitors identify all possible SID-DID flow combinations that are possible on a given port and provides a sorted output of the top talking flows.
20 Top Talker monitors Edge fabric E_Port FC router EX_Port Backbone fabric FIGURE 64 Fabric mode Top Talker monitors on FC router do not monitor any flows Edge fabric E_Port E_Port E_Port FC router EX_Port Backbone fabric FIGURE 65 Fabric mode Top Talker monitors on FC router monitor flows over the E_Port Limitations of Top Talker monitors Be aware of the following when using Top Talker monitors: • • • • • 412 Top Talker monitors cannot detect transient surges in traffic through a given
Top Talker monitors 20 Adding a Top Talker monitor to a port (port mode) 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the perfTTmon --add command.
20 Top Talker monitors The output is sorted based on the data rate of each flow. If you do not specify the number of flows to display, then the command displays the top 8 flows or the total number of flows, whichever is less.
Trunk monitoring 20 For example, to delete the monitor on port 7: perfttmon --delete 7 To delete the monitor on slot 2, port 4 on a Backbone: perfttmon --delete 2/4 Deleting all fabric mode Top Talker monitors 1. Connect to the switch and log in using an account with admin permissions. 2. Enter the perfTTmon --delete fabricmode command. perfttmon --delete fabricmode All Top Talker monitors are deleted.
20 Performance data collection When there are more than 512 monitors in the system, monitors are saved to flash memory in the following order: • The EE monitors for each port (from 0 to MAX_PORT) • The frame monitors for each port EE monitors get preference saving to flash memory when the total number of monitors in a switch exceeds 512.
Chapter 21 Optimizing Fabric Behavior In this chapter • Adaptive Networking overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Ingress Rate Limiting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • QoS: SID/DID traffic prioritization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • CS_CTL-based frame prioritization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21 Ingress Rate Limiting • Ingress Rate Limiting Ingress rate limiting restricts the speed of traffic from a particular device to the switch port. Ingress rate limiting requires an Adaptive Networking license. See “Ingress Rate Limiting” on page 418 for more information about this feature. • Quality of Service (QoS) SID/DID Traffic Prioritization SID/DID traffic prioritization allows you to categorize the traffic flow between a host and target has having a high or low priority.
QoS: SID/DID traffic prioritization 21 Virtual Fabrics considerations: If Virtual Fabrics is enabled, the rate limit configuration on a port is on a per-logical switch basis. That is, if a port is configured to have a certain rate limit value, and the port is then moved to a different logical switch, it would have no rate limit applied to it in the new logical switch. If that same port is moved back to the original logical switch, it would have the original rate limit take effect again.
21 CS_CTL-based frame prioritization TABLE 72 Comparison between CS_CTL-based and QoS zone-based prioritization CS_CTL-based frame prioritization QoS zone-based traffic prioritization Requires Adaptive Networking license. Requires Adaptive Networking license. Must be manually enabled after you install the license. Automatically enabled when you install the license. No zones are required. Requires you to create QoS zones. Enabled on F/FL_Ports. Enabled on E_Ports.
Enabling CS_CTL-based frame prioritization 21 Supported configurations for CS_CTL-based frame prioritization • CS_CTL-based frame prioritization is supported on all 8-Gbps and 16-Gbps platforms. • All switches in the fabric should be running Fabric OS v6.0.0 or later. NOTE If a switch is running a firmware version earlier than Fabric OS v6.0.0, the outgoing frames from that switch lose their priority.
21 QoS zone-based traffic prioritization High, medium, and low priority flows are allocated to different virtual channels (VCs). High priority flows receive more VCs than medium priority flows, which receive more VCs than low priority flows. The virtual channels are allocated as shown in Table 74.
QoS zone-based traffic prioritization 21 3. Identify E_Ports on which QoS should be manually disabled. In the islshow output, these ports have all of the following characteristics: • 8 Gbps or 16 Gbps ports • Trunking is enabled • QoS is disabled 4. Check whether QoS is enabled on each port identified in step 3 using the following command: portcfgshow In the output, the value of QOS E_Port is AE if QoS is automatically enabled by default, ON if QoS is enabled manually, and OFF or "..
21 QoS zones NPIV capability ON ON ON ON NPIV PP Limit 126 126 126 126 QOS E_Port AE AE AE AE EX Port .. .. .. .. Mirror Port ON .. .. .. Rate Limit .. .. .. .. Credit Recovery ON ON ON ON Fport Buffers .. .. .. .. Port Auto Disable .. .. .. .. CSCTL mode .. .. .. .. ON ON ON ON 126 126 126 126 AE AE AE AE .. .. .. .. ON .. .. .. .. .. .. .. ON ON ON ON .. .. .. .. .. .. .. .. .. .. .. .. ON ON ON ON 126 126 126 126 .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ON ON ON ON .. .. .. .. .. .. .. .. .. .
QoS zones 21 For example, Figure 66 shows a fabric with two hosts (H1, H2) and three targets (S1, S2, S3). The traffic prioritization is as follows: • Traffic between H1 and S1 is high priority. • Traffic between H1 and S3 and between H2 and S3 is low priority. • All other traffic is medium priority, which is the default.
21 QoS zones Domain 1 H1 Domain 3 1 9 14 H2 3 13 12 15 8 7 = Low priority = Medium priority = High priority = E_Ports with QoS enabled FIGURE 67 S1 S3 16 Domain 2 S2 Domain 4 QoS with E_Ports enabled You need to enable QoS on the E_Ports on both ISLs between Domain 3 and Domain 4 because either path might be selected to carry the traffic.
QoS zones 21 • QoS over FC routers is supported for the following configurations: - Edge-to-edge fabric configuration: supported on all platforms. - Backbone-to-edge fabric configuration: supported on 16-Gbps-capable platforms only (Brocade 6510 and Brocade DCX 8510 family), and only if the setup contains no other platforms. For all other platforms, you cannot prioritize the flow between a device in an edge fabric and a device in the backbone fabric.
21 QoS zones High availability considerations for QoS zone-based traffic prioritization If the standby CP is running a Fabric OS version earlier than 6.3.0 and is synchronized with the active CP, then QoS zones using D,I notation cannot be created. If the standby CP is not synchronized or if no standby CP exists, then the QoS zone creation succeeds.
Setting QoS zone-based traffic prioritization 21 • Traffic prioritization is not supported in McDATA Fabric Mode (interopmode 2) or Open Fabric Mode (interopmode 3). • • • • You must be running Fabric OS v6.3.0 or later to create QoS zones using D,I notation. QoS zones using D,I notation are not supported for QoS over FCR. QoS zones using D,I notation should not be used for loop or NPIV ports. If QoS is enabled, an additional 16 buffer credits are allocated per port for 8-Gbps ports in LE mode.
21 Setting QoS zone-based traffic prioritization NOTE QoS is enabled by default on all ports (except long-distance ports). If you use the portCfgQos command to enable QoS on a specific port, the port is toggled to apply this configuration, even though the port already has QoS enabled. The port is toggled because the user configuration changed, even though the actual configuration of the port did not change.
Setting QoS zone-based traffic prioritization over FC routers 21 Setting QoS zone-based traffic prioritization over FC routers 1. Connect to the switch in the edge fabric and log in using an account with admin permissions. 2. Create QoS zones in the edge fabric. The QoS zones must have WWN members only, and not D,I members. See “Setting QoS zone-based traffic prioritization” on page 429 for instructions. 3. Create LSAN zones in the edge fabric.
21 432 Disabling QoS zone-based traffic prioritization Fabric OS Administrator’s Guide 53-1002446-01
Chapter 22 Managing Trunking Connections In this chapter • Trunking overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Requirements for trunk groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Supported configurations for trunking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Supported platforms for trunking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Recommendations for trunking groups . . .
22 Trunking overview Types of trunking Trunking can be between two switches, between a switch and an Access Gateway module, or between a switch and a Brocade adapter. The types of trunking are as follows: • ISL trunking, or E_Port trunking, is configured on an inter-switch link (ISL) between two Fabric OS switches and is applicable only to E_Ports. • ICL trunking is configured on an inter-chassis link (ICL) between two Backbones and is applicable only to ports on the core blades.
Requirements for trunk groups 22 License requirements for trunking All types of trunking require the Trunking license. This license must be installed on each switch that participates in trunking. ATTENTION After you add the Trunking license, to enable trunking functionality, you must disable and then re-enable each port to be used in trunking, or disable and re-enable the switch.
22 Supported configurations for trunking • Trunking cannot be done if ports are in ISL R_RDY mode. (You can disable this mode using the portCfgIslMode command.) • Trunking is supported only on FC ports. Virtual FC ports (VE_ or VEX_Ports) do not support trunking. Supported configurations for trunking • Trunk links can be 2 Gbps, 4 Gbps, 8 Gbps, 10 Gbps, or 16 Gbps depending on the Brocade platform. • The maximum number of ports per trunk and trunks per switch depends on the Brocade platform.
Recommendations for trunking groups 22 Recommendations for trunking groups To identify the most useful trunking groups, consider the following recommendations along with the standard guidelines for SAN design: • Evaluate the traffic patterns within the fabric. • Place trunking-capable switches adjacent to each other. This maximizes the number of trunking groups that can form.
22 Enabling trunking on a port or switch To re-initialize the ports, you can either disable and then re-enable the switch, or disable and then re-enable the affected ports. 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the islShow command to determine which ports are used for ISLs. 3. Enter the portDisable command for each port to be used in a trunk group. Alternatively, you can enter the switchDisable command to disable all ports on the switch. 4.
Displaying trunking information 22 Displaying trunking information You can use the trunkShow command to view the following information: • • • • All the trunks and members of a trunk. Whether the trunking port connection is the master port connection for the trunking group. That trunks are formed correctly. Trunking information for a switch that is part of an FC Router backbone fabric interlinking several edge fabrics.
22 ISL trunking over long distance fabrics Tx: Bandwidth 16.00Gbps, Throughput 1.67Gbps (12.12%) Rx: Bandwidth 16.00Gbps, Throughput 1.66Gbps (12.11%) Tx+Rx: Bandwidth 32.00Gbps, Throughput 3.33Gbps (12.11%) ISL trunking over long distance fabrics In long-distance fabrics, if a port speed is set to autonegotiate, then the maximum speed, which is 16 Gbps, is assumed for reserving buffers for the port. If the port is only running at 2 Gbps, this wastes buffers.
EX_Port trunking 22 The FC router front domain has a higher node WWN—derived from the FC router—than that of the edge fabric. Therefore, the FC router front domain initiates the trunking protocol on the EX_Port. After initiation, the first port from the trunk group that comes online is designated as the master port. The other ports that come online on the trunk group are considered the slave ports.
22 F_Port trunking Backward compatibility support For backward compatibility, an FC router that supports EX_Port trunking can continue to interoperate with older FC routers and all previously supported Brocade switches in the backbone fabric or Brocade edge fabric. Configuring EX_Port trunking With EX_Port trunking, you use the same CLI commands as you do for E_Port trunking. See “Configuring trunk groups” on page 437 for instructions. Displaying EX_Port trunking information 1.
F_Port trunking 22 F_Port trunking for Access Gateway You can configure trunking between the F_Ports on an edge switch and the N_Ports on an Access Gateway module. NOTE You cannot configure F_Port trunking on the F_Ports of an Access Gateway module. F_Port trunking keeps F_Ports from becoming disabled when they are mapped to an N_Port on a switch in Access Gateway mode.
22 F_Port trunking NOTE You do not need to manually map the host to the master port because Access Gateway will perform a cold failover to the master port. See “Configuring F_Port trunking for Access Gateway” on page 447 for instructions on configuring F_Port trunking.
F_Port trunking TABLE 76 22 F_Port masterless considerations (Continued) Category Description configdownload If you issue the configDownload command for a port configuration that is not compatible with F_Port trunking, and the port is Trunk Area-enabled, then the port will be persistently disabled. F_Port trunks will never be restored through configDownload. NOTE: Configurations that are not compatible with F_Port trunking are long distance, port mirroring, non-CORE_PID, and Fast Write.
22 F_Port trunking TABLE 76 F_Port masterless considerations (Continued) Category Description Port Swap When you assign a Trunk Area to a trunk group, the Trunk Area cannot be port swapped; if a port is swapped, then you cannot assign a Trunk Area to that port. Port Types Only F_Port trunk ports are allowed on a Trunk Area port. All other port types are persistently disabled. PWWN The entire Trunk Area trunk group shares the same Port WWN within the trunk group.
Configuring F_Port trunking for Access Gateway 22 F_Port trunking in Virtual Fabrics F_Port trunking functionality performs the same in Virtual Fabrics as it does in non-virtual fabric platforms except for the Brocade DCX and DCX 8510-8. Fabric OS uses a 10-bit addressing model, which is the default mode for all dynamically created logical switches in the DCX platform. In the DCX and DCX 8510 platforms, F_Port trunk ports dynamically receive an 8-bit area address that remains persistent.
22 Configuring F_Port trunking for Brocade adapters 3. Enter the portDisable command for each port to be included in the TA. 4. Enter the portTrunkArea --enable command to enable the trunk area. For example, the following command creates a TA for ports 36-39 with index number 37. switch:admin> porttrunkarea --enable 36-39 -index 37 Trunk index 37 enabled for ports 36, 37, 38 and 39. When you assign a trunk area on a port, it enables trunking on the F_Ports automatically.
Disabling F_Port trunking 22 Port Type State Master TI DI ------------------------------------36 F-port Master 36 37 36 37 F-port Slave 36 37 37 38 F-port Slave 36 37 38 39 F-port Slave 36 37 39 • Enter the porttrunkarea --show trunk command to display the trunking information. switch:admin> porttrunkarea --show trunk Trunk Index 37: 39->0 sp: 8.000G bw: 16.000G deskew 15 MASTER Tx: Bandwidth 16.00Gbps, Throughput 1.63Gbps (11.84%) Rx: Bandwidth 16.00Gbps, Throughput 1.62Gbps (11.
22 Enabling the DCC policy on a trunk area 3. Turn on the trunk ports. Trunk ports should be turned on after issuing the secPolicyActivate command to prevent the ports from becoming disabled in the case where there is a DCC security policy violation. You can configure authentication on all Brocade trunking configurations. For more information on authentication, see Chapter 7, “Configuring Security Policies”.
Chapter 23 Managing Long Distance Fabrics In this chapter • Long distance fabrics overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Extended Fabrics device limitations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Long distance link modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Configuring an extended ISL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Buffer credit management . . . . . . . .
23 Extended Fabrics device limitations Extended Fabrics device limitations Note the limitations regarding the following platforms: • Brocade 8000 FCoE switch Extended Fabrics is not supported on this platform. • FC8-64 port blade Brocade recommends that you do not use the FC8-64 port blade for long distance, due to limited buffers. This blade does not support LWL and supports limited distance.
Configuring an extended ISL 23 • Static Long-Distance Mode (LS) - LS calculates a static number of BB credits based only on a user-defined desired_distance value. LS mode also assumes that all FC payloads are 2112 bytes. Specify LS mode to configure a static long distance link with a fixed buffer allocation greater than 10 km. Up to a total of 1452 full-size frame buffers are reserved for data traffic, depending on the specified desired_distance value.
23 Configuring an extended ISL Example The following example configures slot 1, port 2 to support a 100 km link in LS mode and be initialized using the extended link initialization sequence. This example is for an 8 Gbps platform. switch:admin> portcfgfillword 1/2 3 switch:admin> portcfglongdistance 1/2 LS 1 100 Reserved Buffers = 406 Warning: port may be reserving more credits depending on port speed.
Buffer credit management 23 3. Disable the credit recovery; credit recovery is not compatible with the IDLE mode. If you do not disable the credit recovery, it continues to perform a link reset. switch:admin> portcfgcreditrecovery --disable [slot/]port 4. Configure the port to support long-distance links.
23 Buffer credit management Upon arrival at a receiver, a frame goes through several steps. It is received, deserialized, decoded, and is stored in a receive buffer where it is processed by the receiving port. If another frame arrives while the receiver is processing the first frame, a second receive buffer is needed to hold this new frame.
Buffer credit management 23 Fibre Channel gigabit values reference definition Before you can calculate the buffer requirement, note the following Fibre Channel gigabit values reference definition: • • • • • • 1.0625 for 1 Gbps 2.125 for 2 Gbps 4.25 for 4 Gbps 8.5 for 8 Gbps 10.625 for 10 Gbps 17 for 16 Gbps Allocating buffer credits based on full-size frames Assuming that the frame size is full, one buffer credit allows a device to send one payload up to 2112 bytes (2148 with headers).
23 Buffer credit management payloads consistently being 2,112 bytes is not realistic in practice. To gain the proper number of BB credits using the LS mode, there must be enough BB credits available in the pool because Fabric OS will check before accepting a value. NOTE The portCfgLongDistance command’s desired_distance parameter is the upper limit of the link distance and is used to calculate buffer availability for other ports in the same port group.
Buffer credit management 23 Example Consider the Brocade 300, which has a single 24-port port group and a total of 676 buffer credits for that port group. The maximum remaining number of buffer credits for the port group, after each port reserves its eight buffer credits, is: 676 – (24 * 8) = 484 unreserved buffer credits Where: 24 = the number of user ports in a port group retrieved from Table 79 on page 461. 8 = the number of reserved credits for each user port.
23 Buffer credit management 4. Use the following formula to calculate the number of buffer-to-buffer credits to allocate: BB credits = roundup [desired_distance * (data_rate / 2.125)] Using the values for desired_distance and data_rate from step 1 and step 3, the value for BB credits is calculated as follows: BB credits = roundup [(207 * 8.5) / 2.125] = 828 NOTE This formula does not work with LD mode because LD mode checks the distance and limits the estimated distance to the real value of 100 km.
Buffer credit management 23 Buffer credits for each switch model Table 79 shows the total ports in a switch or blade, number of user ports in a port group, and the unreserved buffer credits available per port group. The number in the Unreserved buffers column is the number with QoS enabled. This number is higher if QoS is not enabled.
23 Buffer credit management Maximum configurable distances for Extended Fabrics Table 80 shows the maximum supported extended distances (in kilometers) that can be configured for one port on a specific switch or blade at different speeds.
Buffer credit recovery 23 To get an estimated maximum equally distributed distance for n number of ports at a particular ("X") speed, divide the 1-port maximum distance of the switch at X speed by n. For example, for three ports running at 2 Gbps on a 300 switch, the maximum equally distributed distance is calculated as 486 / 3 = 164 km. Buffer credit recovery Buffer credit recovery does not require configuration.
23 464 Buffer credit recovery Fabric OS Administrator’s Guide 53-1002446-01
Chapter 24 Using FC-FC Routing to Connect Fabrics In this chapter • FC-FC routing overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Fibre Channel routing concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Setting up FC-FC routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Backbone fabric IDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24 FC-FC routing overview For more information about M-EOS connectivity, refer to Appendix A, “Interoperation of Fabric OS and M-EOS Fabrics Using FC Router”. A Fibre Channel router (FC router) is a switch running the FC-FC routing service. The FC-FC routing service can be simultaneously used as an FC router and as a SAN extension over wide area networks (WANs) using FCIP. You can set up QoS traffic prioritization over FC routers.
FC-FC routing overview 24 • VEX_Ports are supported on the FR4-18i Router Blade, but EX_Ports are not supported. The FR4-18i blade is not supported in the same chassis as the FX8-24 blade. • The Backbones have a limit of 128 EX_Ports for each chassis. Refer to the Network OS Administrator’s Guide for supported Network OS platforms. Supported configurations for FC-FC routing FC-FC routing supports the following configurations: • • • • • • FC router connected to a Fabric OS nonsecured edge fabric.
24 Fibre Channel routing concepts Fibre Channel routing concepts Fibre Channel routing introduces the following concepts: • Fibre Channel router (FC router) A switch running the FC-FC routing service. Refer to “Supported platforms for FC-FC routing” on page 466 for a list of platforms that can be FC routers.
Fibre Channel routing concepts 24 • Logical SANs (LSANs) An LSAN is defined by zones in two or more edge or backbone fabrics that contain the same devices. You can create LSANs that span fabrics. These LSANs enable Fibre Channel zones to cross physical SAN boundaries without merging the fabrics while maintaining the access controls of zones. An LSAN device can be a physical device, meaning that it physically exists in the fabric, or it can be a proxy device.
24 Fibre Channel routing concepts • Fabric ID (FID) Every EX_Port and VEX_Port uses the fabric ID (FID) to identify the fabric at the opposite end of the inter-fabric link. The FID for every edge fabric must be unique from the perspective of each backbone fabric. - If multiple EX_Ports (or multiple VEX_Ports) are attached to the same edge fabric, they must be configured with the same FID.
Fibre Channel routing concepts 24 ISL FC router FC router EX_Port EX_Port Backbone fabric IFL IFL E_Port E_Port Edge SAN 1 Edge SAN 2 = LSAN FIGURE 74 Edge SANs connected through a backbone fabric • Phantom domains A phantom domain is a domain emulated by the Fibre Channel router. The FC router can emulate two types of phantom domains: front phantom domains and translate phantom domains. For detailed information about phantom domains, refer to “Phantom domains” on page 473.
24 Fibre Channel routing concepts Proxy host (imported device) Host Proxy target (imported device) Target Fabric 1 Fabric 2 E_Port IFL E_Port EX_Port IFL FC router FIGURE 75 MetaSAN with imported devices FC-FC routing topologies The FC-FC routing service provides two types of routing: • Edge-to-edge Occurs when devices in one edge fabric communicate with devices in another edge fabric through one or more FC routers.
Fibre Channel routing concepts 24 Phantom domains A phantom domain is a domain created by the Fibre Channel router. The FC router creates two types of phantom domains: front phantom domains and translate phantom domains. A front phantom domain, or front domain, is a domain that is projected from the FC router to the edge fabric. There is one front phantom domain from each FC router to an edge fabric, regardless of the number of EX_Ports connected from that router to the edge fabric.
24 Fibre Channel routing concepts Host 1 Fabric 1 Front domain 1 (FC router 1) Front domain 2 (FC router 2) Xlate domain 1 (Fabric 2) Xlate domain 2 (Fabric 3) Target 1' FIGURE 77 Target 2' Target 3' EX_Port phantom switch topology All EX_Ports or VEX_Ports connected to an edge fabric use the same xlate domain ID for an imported edge fabric; this value persists across switch reboots and fabric reconfigurations.
Setting up FC-FC routing 24 1. Connect to the FC router and log in using an account with admin permissions. 2. Enter the fcrXlateConfig --show command to identify any stale xlate domains. 3. Enter the fcrXlateConfig --del command to delete the stale xlate domains.
24 Setting up FC-FC routing 2. If you are configuring a Backbone, enter the slotShow command to verify that either the FR4-18i or FX8-24 blade is present or an 8-Gbps or 16-Gbps port blade is present. The following example shows slots 1, 2, 3, 9, 10, and 12 with 8-Gbps port blades enabled.
Backbone fabric IDs 24 FC-FC routing and fabric mode Top Talker monitors are not concurrently supported on 8-Gbps platforms. FC-FC routing and fabric mode Top Talker monitors are concurrently supported only on the Brocade 6510 and on the Brocade DCX Backbone family with only 16-Gbps-capable ports. Backbone fabric IDs If your configuration has only one backbone fabric, then you do not need to assign a backbone fabric ID because the backbone fabric ID in this situation defaults to a value of 128.
24 FCIP tunnel configuration FC Router service is disabled switch:admin> fcrconfigure FC Router parameter set.
Inter-fabric link configuration 24 2. Configure each port that connects to an edge fabric as an EX_Port or VEX_Port. Note the following: • portCfgVEXPort works only on VE_Ports. • portCfgEXPort (only on the FC ports on the FC router) commands work only on ports that are capable of FC-FC routing. Use the portCfgEXPort or portCfgVEXPort command to: • Enable or disable EX_Port or VEX_Port mode. • Set the fabric ID (avoid using fabric IDs 1 and 128, which are the default IDs for backbone connections).
24 Inter-fabric link configuration 4. (Optional) Set up ISL or EX_Port trunking. For information on trunking setup, refer to “Configuring EX_Port trunking” on page 442. 5. Enter the portEnable command to enable the ports that you disabled in step 1. switch:admin> portenable 7/10 6. Physically attach ISLs from the Fibre Channel router to the edge fabric. 7. Enter the portCfgShow command to view ports that are persistently disabled.
24 Inter-fabric link configuration State: NOT OK Pid format: Not Applicable Operate mode: Brocade Native Edge Fabric ID: 30 Preferred Domain ID: 160 Front WWN: 50:06:06:9e:20:38:6e:1e Fabric Parameters: Auto Negotiate R_A_TOV: Not Applicable E_D_TOV: Not Applicable Authentication Type: None DH Group: N/A Hash Algorithm: N/A Edge fabric's primary wwn: N/A Edge fabric's version stamp: N/A switch:admin_06> portshow 7/10 portName: portHealth: OFFLINE Authentication: None EX_Port Mode: Fabric ID: Front Phanto
24 FC router port cost configuration Overrun: Suspended: Parity_err: 2_parity_err: CMI_bus_err: 0 0 0 0 0 Lr_in: Lr_out: Ols_in: Ols_out: 0 0 0 0 Port part of other ADs: No 10. Enter the switchShow command to verify the EX_Port (or VEX_Port), edge fabric ID, and name of the edge fabric switch (containing the E_Port or VE_Port) are correct. 11. Enter the fcrFabricShow command to view any edge fabric switch names and ensure links are working as expected.
FC router port cost configuration 24 Every IFL has a default cost. The default router port cost values are: • 1000 for a legacy (v5.1 or XPath FCR) IFL • 1000 for an EX_Port IFL • 10,000 for a VEX_Port IFL The FC router port cost settings are 0, 1000, or 10,000. If the cost is set to 0, the default cost will be used for that IFL. The FC router port cost is persistent and is saved in the existing port configuration file. FC router port cost is passed to other routers in the same backbone.
24 EX_Port frame trunking configuration For details about the use of any of the following commands, refer to the Fabric OS Command Reference. 1. Enter the portDisable command to disable any port on which you want to set the router port cost. switch:admin> portdisable 7/10 2. Enable EX_Port or VEX_Port mode with the portCfgEXPort or portCfgVEXPort command. switch:admin> portcfgexport 7/10 -a 1 3. Enter the fcrRouterPortCost command to display the router port cost for each EX_Port.
LSAN zone configuration 24 After initiation, the first port from the trunk group that comes online is designated as the master port. The other ports that come online on the trunk group are considered the slave ports. Adding or removing a slave port does not cause frame drop; however, removing a slave port causes the loss of frames in transit.
24 LSAN zone configuration Zones that contain hosts and targets that are shared between the two fabrics must be explicitly coordinated. To share devices between any two fabrics, you must create an LSAN zone in both fabrics containing the port WWNs of the devices to be shared. Although an LSAN is managed using the same tools as any other zone on the edge fabric, two behaviors distinguish an LSAN from a conventional zone: • A required naming convention. The name of an LSAN begins with the prefix “LSAN_”.
LSAN zone configuration 24 switch:admin> nsshow { Type Pid COS PortName NodeName TTL(sec) N 060f00; 2,3; 10:00:00:00:c9:2b:c9:0c; 20:00:00:00:c9:2b:c9:0c; na FC4s: FCP NodeSymb: [35] "Emulex LP9002 FV3.91A3 DV5-5.20A6 " Fabric Port Name: 20:0f:00:05:1e:37:00:44 Permanent Port Name: 10:00:00:00:c9:2b:c9:0c The Local Name Server has 1 entry } 3. Enter the zoneCreate command to create the LSAN lsan_zone_fabric75, which includes the host.
24 LSAN zone configuration zone: lsan_zone_fabric2 10:00:00:00:c9:2b:c9:0c; 50:05:07:61:00:5b:62:ed; 50:05:07:61:00:49:20:b4 Effective configuration: no configuration in effect 10. Enter the cfgAdd and cfgEnable commands to create and enable the LSAN configuration. switch:admin> cfgadd "zone_cfg", "lsan_zone_fabric2" switch:admin> cfgenable "zone_cfg" You are about to enable a new zoning configuration. This action will replace the old zoning configuration with the current configuration selected.
LSAN zone configuration 24 Configuring backbone fabrics for interconnectivity If you want devices in backbone fabrics to communicate with devices in edge fabrics, set up the LSANs as described in the section “Controlling device communication with the LSAN” on page 486. However, instead of configuring the LSAN in the second edge fabric, configure the LSAN in the backbone fabric.
24 LSAN zone configuration LSAN zone policies using LSAN tagging You can create tags for LSAN zones to give them a special meaning. LSAN zones are zones with names that start with the “lsan_” prefix. You can specify a tag to append to this prefix that causes the LSAN zone to be treated differently. You can specify two types of tags: • Enforce tag – Specifies which LSANs are to be enforced in an FC router. • Speed tag – Specifies which LSANs are to be imported or exported faster than other LSANs.
LSAN zone configuration 24 For example, in Figure 78 on page 491, assume that the host, H1, needs fast access to target devices D1 and D2. You could set up the Speed tag as follows: 1. In FC router 1 and FC router 2, configure the Speed tag as “super”. 2. In Edge fabric 2, configure two LSANs: lsan_f2_f1 (H1, D1) lsan_f2_f3 (H1, D2) The LSAN in the host fabric does not need the tag. 3. In Edge fabric 1, configure the following LSAN: lsan_super_f1_f2 (H1, D1) 4.
24 LSAN zone configuration • The LSAN tags are configured per FC router, not per fabric. If the backbone fabric has multiple FC routers, it is recommended that you configure the LSAN tags on all of the FC routers. • The FC router must be disabled before you configure the Enforce tag. Configuring the Speed tag does not require that the FC router be disabled; however, after configuring the Speed tag, you must toggle the host or target port to trigger the fast import process.
LSAN zone configuration 24 Removing an LSAN tag Use the following procedure to remove an LSAN tag. This procedure does not remove the LSAN zone; it deactivates the tag so that LSAN zones with this tag in the name now behave as regular LSAN zones. You must disable the switch before removing an Enforce LSAN tag. You do not need to disable the switch to remove a Speed LSAN tag. 1. Log in to the FC router as admin. 2. Enter the fcrlsan --remove command to remove an existing LSAN tag.
24 LSAN zone configuration Without LSAN zone binding, every FC router in the backbone fabric maintains the entire LSAN zone and device state database. The size of this database limits the number of FC routers and devices you can have. With LSAN zone binding, each FC router in the backbone fabric stores only the LSAN zone entries of the remote edge fabrics that can access its local edge fabrics. The LSAN zone limit supported in the backbone fabric is not limited by the capability of one FC router.
LSAN zone configuration TABLE 81 24 LSAN information stored in FC routers, with and without LSAN zone binding WIthout LSAN zone binding With LSAN zone binding FC router 1 FC router 2 FC router 3 FC router 4 FC router 1 FC router 2 FC router 3 FC router 4 LSAN 1 LSAN 2 LSAN 3 LSAN 4 LSAN 1 LSAN 2 LSAN 3 LSAN 4 LSAN 1 LSAN 2 LSAN 3 LSAN 4 LSAN 1 LSAN 2 LSAN 3 LSAN 4 LSAN 1 LSAN 2 LSAN 2 LSAN 3 LSAN 4 LSAN 4 LSAN zone binding considerations • Without LSAN zone binding, the maximum number
24 LSAN zone configuration FC router matrix definition Depending on the structure of the backbone fabric, you can specify pairs of FC routers that can access each other.
LSAN zone configuration 24 Setting up LSAN zone binding 1. Log in to the FC router as admin. 2. Enter the following command to add a pair of FC routers that can access each other: FCR:Admin> fcrlsanmatrix --add -fcr wwn1 wwn2 The variables wwn1 and wwn2 are the WWNs of the FC routers. 3. Enter the following command to add a pair of edge fabrics that can access each other: FCR:Admin> fcrlsanmatrix --add -lsan fid1 fid2 The variables fid1 and fid2 are the fabric IDs of the edge fabrics. 4.
24 Proxy PID configuration Proxy PID configuration When an FC router is first configured, the PIDs for the proxy devices are automatically assigned. Proxy PIDs (as well as phantom domain IDs) persist across reboots. The most common situation in which you would set a proxy PID is when you replace a switch. If you replace the switch and want to continue using the old PID assignments, you can configure it to do so; this value remains in the system even if the blade is replaced.
Inter-fabric broadcast frames 24 Inter-fabric broadcast frames The FC router can receive and forward broadcast frames between edge fabrics and between the backbone fabric and edge fabrics. Many target devices and HBAs cannot handle broadcast frames. In this case, you can set up broadcast zones to control which devices receive broadcast frames. (Refer to “Broadcast zones” on page 244 for information about setting up broadcast zones.
24 Resource monitoring Resource monitoring It is possible to exhaust resources, such as proxy PIDs. Whenever a resource is exhausted, Fabric OS generates an error message. The messages are described in the Fabric OS Message Reference. You can monitor FC router resources using the fcrResourceShow command.
FC-FC routing and Virtual Fabrics 12 13 14 15 16 17 18 19 20 21 22 23 | | | | | | | | | | | | 6 6 6 6 8 8 8 8 8 8 8 8 24 34 34 34 34 34 34 34 34 34 34 34 34 FC-FC routing and Virtual Fabrics If Virtual Fabrics is not enabled, FC-FC routing behavior is unchanged. If Virtual Fabrics is enabled, then in the FC-FC routing context, a base switch is like a backbone switch and a base fabric is like a backbone fabric.
24 FC-FC routing and Virtual Fabrics • Backbone-to-edge routing is not supported in the base switch. Refer to “Backbone-to-edge routing with Virtual Fabrics” on page 503 for information about how to configure legacy FC routers to allow backbone-to-edge routing with Virtual Fabrics. • All FCR commands can be executed only in the base switch context. • The fcrConfigure command is not allowed when Virtual Fabrics is enabled. Instead, use the lsCfg command to configure the FID.
FC-FC routing and Virtual Fabrics 24 Edge fabric Fabric 128 Edge fabric Fabric 15 SW3 SW5 E SW1 SW7 E EX SW2 EX Fabric 1 SW4 Backbone fabric Fabric 8 FIGURE 81 SW6 SW8 Logical representation of EX_Ports in a base switch Backbone-to-edge routing with Virtual Fabrics Backbone-to-edge routing is not supported in the base switch, unless you use a legacy FC router. A legacy FC router is an FC router configured on a Brocade 7500 switch or an FR4-18i blade.
24 Upgrade and downgrade considerations for FC-FC routing Physical chassis 2 Physical chassis 1 IFL E Logical switch 1 E (Default logical switch) Fabric ID 128 ISL B E Logical switch 5 F (Default logical switch) Fabric ID 128 Logical switch 2 Fabric ID 1 Allows XISL use Edge fabric FID 20 Logical switch 6 Fabric ID 1 Allows XISL use C F Logical switch 3 Fabric ID 15 E ISL E E E Logical switch 7 Fabric ID 15 IFL IFL EX Logical switch 4 EX (Base switch) E Fabric ID 8 XISL E Logical switc
24 Displaying the range of output ports connected to xlate domains Displaying the range of output ports connected to xlate domains The edge fabric detects only one front domain from an FC router connected through multiple output ports. The output port of the front domain is not fixed to 0; the values can be in a range from 129 through 255. The range of the output ports connected to the xlate domain is from 1 through 128. This range enables the front domain to connect to 127 remote xlate domains. 1.
24 506 Displaying the range of output ports connected to xlate domains Fabric OS Administrator’s Guide 53-1002446-01
Appendix Interoperation of Fabric OS and M-EOS Fabrics Using FC Router A In this appendix • Interoperability overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507 • Establishing interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509 • Fabric configurations for interconnectivity. . . . . . . . . . . . . . . . . . . . . . . . . .
A Interoperability overview TABLE 82 Fabric OS and M-EOSc interoperability compatibility matrix1 Fabric OS Versions of M-EOSc v6.2.0 v7.1.3x v8.0 v9.2.0 v9.6.2 v9.7 v5.1.02 v9.8 v9.9 Yes No No No No No v5.2.0 No Yes Yes No No No v5.3.0 No No Yes Yes No No v6.0.0 No No No No Yes No v6.1.0 No No No No Yes Yes v6.2.0 Yes Yes v6.3.0 Yes Yes v6.4.0 Yes Yes v7.0.0 and later3 Yes Yes v6.1.1 v6.1.1_enc 1. Both Open and McDATA Fabric modes are supported.
Establishing interoperability A Features of Connected SANs Connected SANs provide additional features not possible with segregated SANs. Some of these features are listed below: • Island consolidation—Uses the Fabric OS v6.0 or later FC router to connect isolated M-EOS and Fabric OS fabrics to share devices. • Backup consolidation—Consolidates backup solutions across Fabric OS and M-EOS fabrics. • Manageable large-scale storage network—Uses the Fabric OS v6.
A Fabric configurations for interconnectivity When configuring an EX_Port, you have the option to request a front domain with the portCfgEXPort -d command. If you request a front domain that is not within the valid range for M-EOSc, then the Fibre Channel router will internally request a valid M-EOSc domain ID. For M-EOSc switches, after the port is properly configured and connected, running switchShow on the FC router displays the M-EOSc switch that is connected.
Fabric configurations for interconnectivity A Configuring the FC router When configuring a fabric on which Fabric OS is installed to connect to a Native McDATA fabric, you must configure the FC router in advance. The following procedure shows how to connect an EX_Port of an FC router to a Native McDATA fabric configured in Fabric mode. NOTE For additional information on configuring the FC router, refer to Chapter 24, “Using FC-FC Routing to Connect Fabrics”. 1.
A Fabric configurations for interconnectivity 9. Capture a SAN profile of the M-EOS and Fabric OS SANs, identifying the number of devices in each SAN. By projecting the total number of devices and switches expected in each fabric when the LSANs are active, you can quickly determine the status of the SAN by issuing the commands nsAllShow and fabricShow on the Fabric OS fabric. The nsAllShow displays the global name server information and fabricShow displays the fabric membership information.
Fabric configurations for interconnectivity A Correcting errors if LSAN devices appear in only one of the fabrics If the LSAN devices appear in only one of the fabrics in a multiple-fabric SAN, use the following procedure to correct the problem. 1. Log in to each fabric and verify that all of the devices are physically logged in. 2. Verify that the devices are properly configured in the LSAN zone in both edge fabrics. 3. Enter the fabricShow command on the Fabric OS fabric. 4.
A Fabric configurations for interconnectivity 3. Physically connect the configured FC router EX_Port to the M-EOS switch, and issue the switchShow command on the Brocade FC router. New domains should be visible for each IFL (front domain) that connects the Fabric OS switch to the FC router and one domain for the xlate domain. 4. Start Brocade Network Advisor and select the fabric for the M-EOS switch. 5. View the fabric topology.
Fabric configurations for interconnectivity A Permanent Port Name: 10:00:00:00:00:03:00:00 Port Index: na Share Area: No Device Shared in Other AD: No All of the devices from both LSANs should appear in the output. If the devices do not appear in the output, issue the cfgShow command to verify your zone configuration. Use the cfgActvShow command to display the zone configuration currently in effect. The following example illustrates the use of cfgActvShow.
A 516 Fabric configurations for interconnectivity Fabric OS Administrator’s Guide 53-1002446-01
Appendix B Port Indexing This appendix shows how to use the switchShow command to determine the mapping among the port index, slot/port numbers, and the 24-bit port ID (PID) on any Brocade Backbone. Enter the switchShow command without parameters to show the port index mapping for the entire platform. Enter the switchShow -slot command for port mapping information for the ports on the blade in a specific slot. Include the --qsfp option to list also the QSFP number, for slots that contain core blades.
B Port Indexing 739 3 19 4 740 3 20 5 741 3 21 5 742 3 22 5 743 3 23 5 744 3 24 6 745 3 25 6 746 3 26 6 747 3 27 6 748 3 28 7 10:00:00:05:1e:39:e4:5a 749 3 29 7 10:00:00:05:1e:39:e4:5a 750 3 30 7 10:00:00:05:1e:39:e4:5a 751 3 31 7 10:00:00:05:1e:39:e4:5a --------------------------------------------------trunkmaster -----trunkmaster -----trunkmaster -----trunkmaster -16G No_Module -16G No_Module -16G No_Module -16G No_Module -16G No_Module -16G No_Module -16G No_Module -16G No_Module -16G No_Module id 16
Port Indexing B Example of port indexing on an FC8-64 blade on a Brocade DCX-4S Backbone. The Brocade DCX-4S does not need a mapping of ports on port blades because it is a one-to-one mapping. The order is sequential starting at slot 1 port 0 all the way through slot 8 port 255 for the FC8-64 blade. For core blades, the port index mapping for the blade in slot 3 begins with port index 256, and port index mapping for the core blade in slot 6 begins with port index 736.
B Port Indexing Example of port indexing on an FS8-18 blade on a DCX 8510-8 Backbone This example shows the truncated switchShow output for an FS8-18 encryption blade on the Brocade DCX 8510-8 Backbone. The assignment of port index numbers to PIDs will vary depending on blade type, platform type, and slot number.
Appendix C FIPS Support In this appendix • FIPS overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Zeroization functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • FIPS mode configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Preparing the switch for FIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
C Zeroization functions TABLE 85 Zeroization behavior (Continued) Keys Zeroization CLI Description FCSP Challenge Handshake Authentication Protocol (CHAP) Secret secAuthSecret –-remove value | –-all The secAuthSecret --remove value command is used to remove the specified keys from the database. When the secAuthSecret command is used with the --remove –-all option, then the entire key database is deleted.
FIPS mode configuration C The results of the POST and conditional tests are recorded in the system log or are output to the local console. This action includes logging both passing and failing results. Refer to the Fabric OS Troubleshooting and Diagnostics Guide for instructions on how to recover if your system cannot get out of the conditional test mode. FIPS mode configuration By default, the switch comes up in non-FIPS mode.
C FIPS mode configuration LDAP in FIPS mode You can configure your Microsoft Active Directory server to use the Lightweight Directory Access Protocol (LDAP) while in FIPS mode. There is no option provided on the switch to configure TLS ciphers for LDAP in FIPS mode. However, the LDAP client checks if FIPS mode is set on the switch and uses the FIPS-compliant TLS ciphers for LDAP.
FIPS mode configuration C Specify the DNS IP address using either IPv4 or IPv6. This address is needed for the switch to resolve the domain name to the IP address because LDAP initiates a TCP session to connect to your Microsoft Active Directory server. A Fully Qualified Domain Name (FQDN) is needed to validate the server identity as mentioned in the common name of the server certificate. 3. Set the switch authentication mode and add your LDAP server by using the commands shown in the following example.
C FIPS mode configuration LDAP certificates for FIPS mode To utilize the LDAP services for FIPS between the switch and the host, you must generate a certificate signing request (CSR) on the Active Directory server and import and export the CA certificates. To support server certificate validation, it is essential to have the CA certificate installed on the switch and Microsoft Active Directory server. Use the secCertUtil command to import the CA certificate to the switch.
Preparing the switch for FIPS C Deleting an LDAP switch certificate This procedure deletes the LDAP CA certificate from the switch. 1. Connect to the switch and log in using an account with admin permissions, or an account with OM permissions for the PKI RBAC class of commands. 2. Enter the secCertUtil show -ldapcacert command to determine the name of the LDAP certificate file. 3. Enter the secCertUtil delete -ldapcacert file_name command, where file_name is the name of the LDAP certificate on the switch.
C Preparing the switch for FIPS • • • • • • Disable in-flight encryption. Disable IPsec for Ethernet and IPsec for FCIP. Disable in-band management. Disable root access. Enable the KATs and the conditional tests. Enable FIPS. Enabling FIPS mode 1. Log in to the switch using an account with securityadmin permissions. 2. Enter the sshutil delpubkeys and sshutil delprivkey commands to remove legacy OpenSSH DSA keys. These keys, which were previously the default, do migrate to Fabric OS v7.0.
Preparing the switch for FIPS C ipfilter --addrule policyname -rule rule_number -sip source_IP -dp dest_port -proto protocol -act deny • The -sip option can be given as any. • The -dp option for the port numbers for Telnet, HTTP, and RPC are 23, 80, and 898, respectively. • The -proto option should be set to tcp. c. Activate each IP Filter policy. Refer to “Activating an IP Filter policy” on page 154. d. Save each IP Filter policy. Refer to “Saving an IP Filter policy” on page 154.
C Preparing the switch for FIPS 11. Enter the portCfgEncrypt --disable command to disable in-flight encryption. You must first disable the port. Example myswitch:root> portdisable 0 myswitch:root> portcfgencrypt --disable 0 myswitch:root> portenable 0 12. Enter the ipSecConfig --disable command to disable Ethernet IPsec. 13. Disable IPsec for FCIP connections. The procedure depends on the type of extension blade used.
Appendix Hexadecimal Conversion D Hexadecimal overview Hexadecimal, also known as hex, is a numeral system with a base of 16, usually written using unique symbols 0–9 and A–F, or a–f. Its primary purpose is to represent the binary code that computers interpret in a format easier for humans to remember. It acts as a form of shorthand, in which one hexadecimal digit takes the place of four binary bits.
D Hexadecimal overview TABLE 89 532 Decimal to hexadecimal conversion table Decimal 01 02 03 04 05 06 07 08 09 10 Hex 01 02 03 04 05 06 07 08 09 0a Decimal 11 12 13 14 15 16 17 18 19 20 Hex 0b 0c 0d 0e 0f 10 11 12 13 14 Decimal 21 22 23 24 25 26 27 28 29 30 Hex 15 16 17 18 19 1a 1b 1c 1d 1e Decimal 31 32 33 34 35 36 37 38 39 40 Hex 1f 20 21 22 23 24 25 26 27 28 Decimal 41 42 43 44 45 46 47 48 49 50 Hex 2
D Hexadecimal overview TABLE 89 Decimal to hexadecimal conversion table (Continued) Decimal 181 182 183 184 185 186 187 188 189 190 Hex b5 b6 b7 b8 b9 ba bb bc bd be Decimal 191 192 193 194 195 196 197 198 199 200 Hex bf c0 c1 c2 c3 c4 c5 c6 c7 c8 Decimal 201 202 203 204 205 206 207 208 209 210 Hex c9 ca cb cc cd ce cf d0 d1 d2 Decimal 211 212 213 214 215 216 217 218 219 220 Hex d3 d4 d5 d6 d7 d8 d9 da db dc Decimal
D 534 Hexadecimal overview Fabric OS Administrator’s Guide 53-1002446-01
Index A AAA service requests, 97 access browser support, 120 changing account parameters, 87 CP blade, 103 creating accounts, 86 deleting accounts, 87 IP address changes, 17 log in fails, 17 NTP, 28 password, changing, 19 remote access policies, 106 secure, HTTPS, 120 secure, SSL, 120 SNMP ACL, 125 accessing switches and fabrics, 129 account ID, 18 accounts changing parameters, 87 creating, 86 deleting, 87 displaying information, 86 lockout policy, 91 lockout policy, duration, 92 lockout policy, threshold,
Admin Domains about, 337 access levels, 339 activating, 350 AD0, 340 AD255, 341 adding members, 351 ADList, 102 assigning users to, 348 configupload, download, 364 configuration, displaying, 359 creating, 347 deactivating, 351 defined AD configuration, 346 deleting, 353, 354 effective AD configuration, 346 homeAD, 102, 342 implementing, 346 interaction with Fabric OS features, 361 logging in to, 342 LSAN zones, 363 member types, 343 numbering, 337 physical fabric administrator, 339 removing from user accoun
changing an account password, 89 FID of logical switch, 232 logical switch to base switch, 232 RADIUS configuration, 113 RADIUS servers, 113 clearing performance monitor counters, 405 clearing zone configurations, 259 command line interface, 16 compression, in-flight, 309 configuration file backing up, 180 chassis section, 179 configDownload, 182 configdownload in Admin Domain context, 364 configupload in Admin Domain context, 364 configUpload in interactive mode, 181 display settings, 177 format, 178 infor
dictionary.
FCAP, 143 FC-FC Routing, 142 FC-FC Routing and Virtual Fabrics, 501 FC-FC routing service, 465 FCIP link, 509 FCR and traffic isolation, 276 FCS policy modifying, 135 feature licenses, 367 Fibre Channel NAT, 65 Fibre Channel over IP, 478 Fibre Channel protocol auto discovery process, 12 Fibre Channel routing, 468 Fibre Channel services, 3 FICON-MIB, 126 FIPS certificates, installing, 526 firmwareDownload, 203 LDAP certificates, displaying and deleting, 526 firmware download, 192 auto-leveling, 207 Backbones
IPsec algorithms, 169 Authentication Header protocol, 168 configuration on the management interface, 166 Encapsulating Security Payload protocol, 168 flushing SAs, 175 IKE policies, 170 key management, 171 manual key entry, 171 policies, 170 pre-shared key, 171 sa-proposal, 169 security association, 169 security certificate, 171 traffic selector, 170 transform set, 170 ISL, 34 J Java support, SSL, 120 Java version, 120 logical switches about, 212 allowing XISL use, 234 changing FID, 232 changing to a base
N NAT, 65 network address translation, see NAT Network OS connectivity, 465 network security, 117 NPIV 10-bit addressing mode, 324 disabling, 326 enabling, 326 viewing PID login information, 328 NTP access, 28 P password, 18 boot PROM, 93 changing, 88 changing defaults, 19 limits, 19 recovery string, 95 recovery string, boot PROM password, 93 rules, 87 password expiration policy, 91 password policies, 88 password policy account lockout, 91 password strength policy, 89 permissions and roles, 83 phantom doma
RBAC, 82 Registered State Change Notification, 12 remote access policies, 106 remove feature, 384 removing Admin Domain members, 352 Admin Domains from user accounts, 350 alias members, 248 frame monitors, 408 licensed feature, 384 LSAN tags, 493 members from a zone configuration, 256 ports from logical switches, 230 zone configurations, 256 zone members, 250 renaming Admin Domains, 352 requirements Admin Domains, 339 restoring monitor configuration, 415 Role-Based Action Control. See RBAC.
support FC router, 142 Java version, 120 SNMPv3 and v1, 125 SW-EXTTRAP, 126 switch access methods, Web Tools, 15 certificates, installing, 123 certificates, installing for FIPS, 526 configuring, 111 deleting RADIUS configuration, 112 disabling port, 42 displaying RADIUS configuration, 113 name limitations, 30 RADIUS client, 105 RADIUS configuration, disabling, 112 user-defined accounts, 85 switch access, 129 switch firmware version, finding, 195 switch names, 30 switch WWN in Admin Domains, 344 system-defin
Virtual Fabrics and FC-FC Routing, 501 and ingress rate limiting, 419 base switches, about, 218 base switches, creating, 227 ContextRoleList, 102 date settings, 25 default logical switch, 212 disabling, 226 enabling, 225 extended ISL (XISL), 218 F_Port trunking, 447 FID, changing, 232 HomeContext, 102 logical fabric context change, 235 logical fabrics, about, 216 logical ISLs (LISL), 219 logical switch configuration, displaying, 231 logical switch to base switch change, 232 logical switches, about, 212 logi
zone configurations creating, 255 deleting, 257 disabling, 257 enabling, 256 removing, 256 zone database and Admin Domains, 362 zone, broadcast, 244 zones QoS zones, 424 TI zones, 269 Fabric OS Administrator’s Guide 53-1002446-01 545
546 Fabric OS Administrator’s Guide 53-1002446-01
