HP StorageWorks Fabric OS 6.
Legal and notice information © Copyright 2008 Hewlett-Packard Development Company, L.P. © Copyright 2008 Brocade Communications Systems, Incorporated. Hewlett-Packard Company makes no warranty of any kind with regard to this material, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose.
Contents About this guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Supported Fabric OS 6.x HP StorageWorks hardware . . . . . . . . . . . . . . . . . . . . . . Intended audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Related documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Document conventions and symbols . . . . . . . . . . . . . .
Disabling and enabling ports . . . . . . . . . . . . . . . . . . . . . . . Making basic connections . . . . . . . . . . . . . . . . . . . . . . . . . Connecting to devices . . . . . . . . . . . . . . . . . . . . . . . . . Connecting to other switches. . . . . . . . . . . . . . . . . . . . . Linking through a gateway . . . . . . . . . . . . . . . . . . . . . . . . Checking status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tracking and controlling switch changes . . . . . . . . . . . . .
3 Configuring standard security features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Secure protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ensuring network security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring the Telnet protocol. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
IP Filter policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating an IP Filter policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cloning an IP Filter policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Displaying an IP Filter policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Admin Domains and switch WWN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Admin Domain compatibility and availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Admin Domains and merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Compatibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Configuring Directors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Changing a Director’s name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identifying ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Director port numbering schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Defining and naming zones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LSAN zones and fabric-to-fabric communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LSAN zone binding (optional) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dual backbone configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Port and switch naming standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adding and removing FICON CUP licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zoning and PDCM considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Zoning and link incident reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15Administering NPIV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 About NPIV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Enabling and disabling NPIV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring NPIV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Determining how many ports can be configured for long distance . . . . . . . . . . . . . . . . . . . . . . . . . . Displaying the remaining buffers in a port group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fabric considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Long distance link initialization activation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Virtual port types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Traffic shaping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . QoS concepts and implementation over FCIP . . . . . . . . . . . . . . . . . . . . . . . . . . . .
How FC fastwrite works . . . . . . . . . . . . . . . . FC fastwrite flow configuration requirements . . Hardware considerations for FC fastwrite . . . . Configuring and enabling FC fastwrite . . . . . . . . Disabling FC fastwrite on a blade or switch . . Disabling FC fastwrite on a port . . . . . . . . . . .. .. .. .. .. .. . . . . . . .. .. .. .. .. .. .. .. .. .. .. .. . . . . . . ............................... ............................... ............................... ........................
Enabling McDATA Open Fabric mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Enabling McDATA Fabric mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Enabling Brocade Native mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Enabling Fabric OS L2 SANtegrity (Fabric Binding) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
34 35 36 37 38 39 40 41 42 43 44 45 46 Tables 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 16 Zoning with hardware assist (mixed-port and WWN zones) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Session-based hard zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Broadcast zones and Admin Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 AD user types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Ports and devices in CLI output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Admin Domain interaction with Fabric OS features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
103 104 105 106 107 108 109 110 111 112 113 114 18 McDATA-aware features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478 McDATA-unaware features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478 Complete feature compatibility matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480 Supported optional features . . . . . . . . . . . . . . . . .
About this guide This guide provides information about: • Installing and configuring Fabric OS 6.x • Managing user accounts • Using licensed features Supported Fabric OS 6.x HP StorageWorks hardware Table 1 lists Brocade and HP StorageWorks product models supported by Fabric OS 6.x.
Intended audience This guide is intended for system administrators with knowledge of: • Storage area networks • HP StorageWorks Fibre Channel SAN switches Related documentation The following documents provide related information: • HP StorageWorks Fabric OS 6.x release notes • HP StorageWorks DC SAN Backbane Director hardware reference guide You can find these documents from the Manuals page of the HP Business Support Center website: http://www.hp.
NOTE: TIP: Provides additional information. Provides helpful hints and shortcuts. Rack stability Rack stability protects personnel and equipment. WARNING! To reduce the risk of personal injury or damage to equipment: • Extend leveling jacks to the floor. • Ensure that the full weight of the rack rests on the leveling jacks. • Install stabilizing feet on the rack. • In multiple-rack installations, secure racks together. • Extend only one rack component at a time.
Subscription service HP recommends that you register your product at the Subscriber's Choice for Business website: http://www.hp.com/go/e-updates. After registering, you will receive e-mail notification of product enhancements, new driver versions, firmware updates, and other product resources. HP websites For additional product information, see the following HP websites: • http://www.hp.com • http://www.hp.com/go/storage • http://www.hp.
1 Standard features This chapter describes how to configure your HP StorageWorks SAN using the Fabric OS Command Line Interface (CLI). Before you can configure a Storage Area Network (SAN), you must power-up the Director or switch and blades, and then set the IP addresses of those devices. Although this chapter focuses on configuring a SAN using the CLI, you can also use the following methods to configure a SAN: • Web Tools For Web Tools procedures, see the Web Tools Administrator’s Guide.
The following commands provide help files for specific topics to understand configuring your SAN: diagHelp ficonHelp fwHelp iscsiHelp licenseHelp perfHelp routeHelp trackChangesHelp zoneHelp Diagnostic help information FICON help information Fabric Watch help information iSCSI help informations License help information Performance Monitoring help information Routing help information Track Changes help information Zoning help information Connecting to the CLI Read this section for procedures.
4. Verify that the login was successful. The prompt displays the switch name and user ID to which you are connected. login: admin password: xxxxxxx switch:admin> Using a console session on the serial port Note the following behaviors for serial connections: • Some procedures require that you connect through the serial port; for example, setting the IP address or setting the boot PROM password. • If you are using a Fabric OS version prior to 6.
Every logical switch (domain) has a set of default accounts. The root and factory default accounts are reserved for development and manufacturing. The user account is primarily used for system monitoring. For more information on default accounts, see ”About the default accounts” on page 59. Table 3 describes the default administrative accounts for switches by model number.
Password changed. Saving password to stable storage. Password saved to stable storage successfully. switch:admin> Configuring the Ethernet interface You can use Dynamic Host Configuration Protocol (DHCP) for the Ethernet network interface configuration. The Ethernet (network) interface provides management access, including direct access to the Fabric OS CLI, and allows other tools, such as Web Tools, to interact with the switch.
Setting static Ethernet addresses Use static Ethernet network interface addresses on HP StorageWorks 2/128, 4/256 SAN Director, DC Director models, and in environments where DHCP service is not available. To use static addresses for the Ethernet interface, you must first disable DHCP. You may enter static Ethernet information and disable DHCP at the same time. Refer to ”Configuring DHCP” on page 29 for more information.
Configuring DHCP By default, some HP switches have DHCP enabled; check the latest Fabric OS 6.x release notes for a complete list of switches. The 4/256 SAN Director and DC SAN Backbone Director (short name, DC Director) do not support DHCP.
4. When you are prompted for DHCP[On], disable it by entering off. switch:admin> ipaddrset Ethernet IP Address [192.168.74.102]: Ethernet Subnetmask [255.255.255.0]: Fibre Channel IP Address [220.220.220.2]: Fibre Channel Subnetmask [255.255.0.0]: Gateway IP Address [192.168.74.1]: DHCP [On]:off Setting the date and time Switches maintain the current date and time inside a battery-backed real-time clock (RTC) circuit. Date and time are used for logging events.
IMPORTANT: If you are downgrading to a Fabric OS version earlier than 6.x, or retaining the offset format, see prior versions of the Fabric OS Administrator’s Guide for detailed information about setting time zones using the offset format. See ”About the firmware download process” on page 163 for time zone downgrading considerations. You can set the time zone for a switch using the tsTimeZone command.
The following procedure describes how to set the current time zone using interactive mode to Pacific Standard Time. To set the time zone interactively: 1. Type the tsTimeZone command as follows: switch:admin> tstimezone --interactive 2. You are prompted to select a general location. Please identify a location so that time zone rules can be set correctly. 3. Enter the appropriate number or Ctrl-D to quit. 4. At the prompt, select a country location. 5.
The following example shows how to set up more than one NTP server using a DNS name: switch:admin> tsclockserver "10.32.170.1;10.32.170.2;ntp.localdomain.net" Updating Clock Server configuration...done. Updated with the NTP servers Changes to the clock server value on the principal or primary FCS switch are propagated to all switches in the fabric.
The Fabric has 4 switches The fields in the fabricShow display are: Switch ID — The switch Domain_ID and embedded port D_ID Worldwide Name — The switch WWN Enet IP Addr — The switch Ethernet IP address for IPv4 and IPv6 configured switches. For IPv6 switches, only the static IP address displays FC IP Addr —The switch FC IP address Name —The switch symbolic name. An arrow (>) indicates the principal switch. To set the Domain ID: 1. Connect to the switch and log in using an admin account. 2.
Generating a license key To generate a license key: 1. If you already have a license key, go to ”Activating a license key” on page 35 to activate. If you do not have a license key, launch an Internet browser and go to: http://webkey.external.hp.com/welcome.asp The Hewlett-Packard Authorization Center website main menu displays. 2. Click Generate a license key. The HP StorageWorks Software License Key instruction page opens:h 3. Enter the information in the required fields. 4.
3. Verify that the license was added by entering the licenseShow command. The licensed features currently installed on the switch display. If the feature is not listed, enter the licenseAdd command again. Some features may require additional configuration, or you may need to disable and reenable the switch to make them operational; see the feature documentation for details.
Features and required licenses Table 4 lists the licenses that should be installed on the local switch and any connecting switches for a particular feature. Table 4 License requirements Feature License Where license should be installed Administrative Domains No license required. n/a Configuration up/download No license required. Configupload or configdownload is a command and comes with the OS on the switch. n/a Diagnostic tools No license required.
Table 4 License requirements Feature License Where license should be installed Ports Ports on demand licenses. This license applies to a select set of switches. Local switch QoS Adaptive Networking Local switch and attached switches. RADIUS No license required. n/a RBAC No license required. n/a Routing traffic No license required. This includes port-based or exchanged-based routing, static routes, frame-order deliver, and dynamic routes. n/a Security No license required.
Time-based licenses A time-based license applies a try-before-you-buy approach to certain features so that you can experience the feature and its capabilities prior to buying the license. Once you have installed the license, you are given a time limit to use the feature. The following lists the types of licenses that have this feature: • FCIP • Fabric • Extended Fabric • Trunking • Advanced Performance Monitoring If you downgrade your switch to a version earlier than 6.
After you install a license key, you must enable the ports to complete their activation. You can do so without disrupting switch operation by issuing the portEnable command on each port. Alternatively, you can disable and reenable the switch to activate ports. NOTE: If you enable or disable an active port you will disrupt any traffic and potentially lose data flowing on that port.
Displaying the port license assignment Use the licensePort --show command to display the available licenses, the current port assignment of those licenses, and the POD method state (dynamic or static). To display the port licenses: 1. Connect to the switch and log in using an admin account. 2. Enter the licensePort --show command.
1, 2, 5, 6, 8*, 21, 22, 23 Ports assigned to the full POD license: None Ports not assigned to a license: 0, 3, 4, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 16 license reservations are still available for use by unassigned ports 1 license assignment is held by an offline port (indicated by *) Disabling Dynamic Ports on Demand Disabling the Dynamic POD feature (changing the POD method to static), erases any prior port license associations or assignments the next time the switch is rebooted.
12 port assignments are provisioned by a full POD license 10 ports are assigned to installed licenses: 10 ports are assigned to the base switch license 0 ports are assigned to the full POD license Ports assigned to the base switch license: 1*, 2*, 3*, 4*, 5*, 6*, 8*, 21, 22, 23 Ports assigned to the full POD license: None Ports not assigned to a license: 0, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 3.
Ports assigned to the base switch license: 1*, 2*, 3*, 4*, 5*, 6*, 8*, 21, 22, 23 Ports assigned to the full POD license: None Ports not assigned to a license: 0, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 switch:admin> 6. Enter the switchEnable command to bring the switch back online. 7. Enter the switchShow command to verify the switch state is now online.
For 4/256 SAN Director and DC Director: Enter the following command: switch:admin> portenable slotnumber/portnumber where slotnumber and portnumber are the slot and port numbers of the port you want to enable. (Slots are numbered 1 through 4 and 7 through 10, counting from left to right.) If the port is connected to another switch, the fabric may be reconfigured. If the port is connected to one or more devices, these devices become available to the fabric.
Any number of E_Ports in a fabric can be configured for gateway links, provided the following rules are followed: • All switches in the fabric must be upgraded to Fabric OS 5.2.0 or later. • All switches in the fabric are using the core PID format. • The switches connected to both sides of the gateway are included when determining switch count maximums.
3. Enter the haShow to verify that HA is enabled, the heartbeat is up, and that the HA state is synchronized between the active and standby CP blades. 4. Enter the slotShow to display the inventory and the current status of each slot in the system. To verify fabric connectivity: 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the fabricShow command. This command displays a summary of all the switches in the fabric.
Tracking and controlling switch changes The track changes feature allows you to keep a record of specific changes that may not be considered switch events, but may provide useful information. The output from the track changes feature is dumped to the system messages log for the switch. Use the errDump or errShow command to view the log. Items in the log created from the Track changes feature are labeled TRCK.
To view the switch status policy threshold values: 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the switchStatusPolicyShow command. Whenever there is a switch change, an error message is logged and an SNMP connUnitStatusChange trap is sent.
3. Verify the threshold settings you have configured for each parameter. Enter the switchStatusPolicyShow command to view your current switch status policy configuration.
be easily distinguished from other system message log events that occur in the network. Then, at some regular interval of your choosing, you can review the audit events to look for unexpected changes. Before you configure audit event logging, familiarize yourself with the following audit event log behaviors and limitations: • By default, all event classes are configured for audit; to create an audit event log for specific events, you must explicitly set a filter with the class operand and then enable it.
NOTE: Only the active CP can generate audit messages because event classes being audited occur only on the active CP. Audit messages cannot originate from other blades in a Director. Audit events have the following message format: AUDIT, , [], , , ///,/,, Switch names are logged for switch components and Director names for Director components.
Jun 5 08:15:32 [10.32.248.73.2.2] raslogd: AUDIT, 2006/06/05-13:38:17, [SEC-1000], WARNING, SECURITY, JaneDoe/root/192.168.132.19/ telnet, Domain A/DoeSwitch, , Incorrect password during login attempt. Shutting down switches and Directors To avoid corrupting your file system, HP recommends that you perform graceful shutdowns of switches and Directors. The following procedure describes how to gracefully shut down a switch. To power off a switch: 1.
Schedule downtime and reboot the switch at your convenience. Table 6 lists the daemons that are considered non-critical and are automatically restarted on failure. Table 6 54 Daemons that are automatically restarted Daemon Description Arrd Asynchronous Response Router (used to send management data to hosts when the switch is accessed through the APIs (FA API or SMI-S). Cald Common Access Layer Daemon (used by Manageability Applications).
2 Managing user accounts This chapter provides information and procedures on managing authentication and user accounts for the switch management channel. Overview In addition to the default accounts—root, factory, admin, and user—Fabric OS supports up to 252 additional user-defined accounts in each logical switch (domain). These accounts expand your ability to track account access and audit administrative activities.
Using Role-Based Access Control (RBAC) Role-Based Action Control (RBAC) defines the capabilities that a user account has based on the role the account has been assigned. For each role, there is a set of pre-defined permissions on the jobs and tasks that can be performed on a fabric and its associated fabric elements. Fabric OS 6.x uses RBAC to determine which commands a user can issue. When you log in to a switch, your user account is associated with a pre-defined role.
Role permissions Table 9 describes the types of permissions that are assigned to roles. Table 9 Permission types Abbreviation Definition Description O Observe The user can run commands using options that display information only, such as running userConfig --show -a to show all users on a switch. M Modify The user can run commands using options that create, change, and delete objects on the system, such as running userconfig --change username -r rolename to change a user’s role.
Table 10 RBAC permissions matrix (continued) Category Role permission User Operator Switch admin Zone admin Fabric admin Basic switch admin Admin Security admin HA (High Availability) O O OM N OM O OM O iSCSI O O O O OM O OM N License O OM OM O OM O OM O LDAP N N N N N N OM OM Local User Environment OM OM OM OM OM OM OM OM Logging O OM OM O OM O OM OM Management Access Configuration O OM OM N OM O OM N Management Server O OM OM O
Managing the local database user accounts User add, change, and delete operations are subject to the subset rule: an admin with ADlist 0-10 cannot perform operations on an admin, user, or any role with an ADlist 11-25. The user account being changed must have an ADlist that is a subset of the account that is making the change. About the default accounts Fabric OS provides the following predefined accounts in the local switch user database.
To create an account: 1. Connect to the switch and log in using an admin account. 2. Enter the following command: userConfig --add -r [-h ] [-a ] [-d ] [-x] username Specifies the account name, which must begin with an alphabetic character. The name can be from 8 to 40 characters long. It is case-sensitive and can contain alphabetic and numeric characters, the period (.) and the underscore ( _ ).
To change account parameters: When changing account parameters, if you change the ADlist for the user account, all of the currently active sessions for that account will be logged out. 1. Connect to the switch and log in using an admin account. 2.
Recovering accounts The following conditions apply to recovering user accounts: • The attributes in the backup database replace the attributes in the current account database. • An event is stored in the system message log, indicating that accounts have been recovered. To recover an account: 1. Connect to the switch and log in using an admin account. 2. If a backup database exists, enter the following command.
Configuring the local user database This section covers the following topics: • ”Distributing the local user database” on page 61 • ”Protecting the local user database from distributions” on page 61 • ”Configuring password policies” on page 62 Distributing the local user database Distributing the local switch user database and passwords to other switches in the fabric causes the distributed database to replace (overwrite) the database on the target switch.
Configuring password policies The password policies described in this section apply to the local switch user database only. Configured password policies (and all user account attribute and password state information) are synchronized across CPs and remain unchanged after an HA failover. Password policies can also be manually distributed across the fabric (see ”Distributing the local user database” on page 61).
• Sequence Specifies the length of sequential character sequences that will be disallowed. A sequential character sequence is defined as a character sequence in which the ASCII value of each contiguous character differs by one. The ASCII value for the characters in the sequence must all be increasing or decreasing. For example, if the “sequence” value is set to 3, a password “passABCword” is disallowed because it contains the sequence “ABC”.
Upgrade and downgrade considerations If you are upgrading from a 5.3.x environment to 6.x, the existing password databases do not contain the state information that implements password expiration. So, when the password expiration policy is first set after an upgrade to 6.x, any user who has not changed their password will have their password expiration period set to the maximum password expiration period.
To disable the admin lockout policy: 1. Log in to the switch using an admin or securityAdmin account. 2. Type passwdCfg --disableadminlockout. The policy is now disabled. Denial of service implications The account lockout mechanism may be used to create a denial of service condition by repeatedly attempting to log in to an account using an incorrect password. Selected privileged accounts are exempted from the account lockout policy to prevent them from being locked out from a denial of service attack.
Consider the following effects of the use of RADIUS or LDAP service on other Fabric OS features: • When RADIUS or LDAP service is enabled, all account passwords must be managed on the RADIUS or LDAP server. The Fabric OS mechanisms for changing switch passwords remain functional; however, such changes affect only the involved switches locally. They do not propagate to the RADIUS or LDAP server, nor do they affect any account on the RADIUS or LDAP server.
Table 12 Authentication configuration options (continued) aaaConfig options Description Equivalent setting in Fabric OS 5.1.0 and earlier --radius --switchdb1 --authspec “ldap” Authenticates management connections n/a against any LDAP database(s) only. If LDAP service is not available or the credentials do not match, the login fails. n/a --authspec “ldap; local” Authenticates management connections against any LDAP database first.
Table 13 Syntax for VSA-based account roles (continued) Item Value Description Vendor type 1 1 octet, Brocade-Auth-Role; valid attributes for the Brocade-Auth-Role are: SwitchAdmin ZoneAdmin FabricAdmin BasicSwitchAdmin Operator User Admin 2 Optional: Specifies the Admin Domain member list. For more information, see ”RADIUS configuration and Admin Domains” on page 69.
Linux FreeRadius server For the configuration on a Linux FreeRadius server, define the following in a vendor dictionary file called dictionary.brocade. Include the values outlined in Table 14. Table 14 dictionary.brocade file entries Include Key Value VENDOR Brocade 1588 ATTRIBUTE Brocade-Auth-Role 1 string Brocade AdminDomain After you have completed the dictionary file, define the role for the user in a configuration file.
Configuring the RADIUS server You must know the switch IP address, in either IPv4 or IPv6 notation, or name to connect to switches. Use the ipAddrShow command to display a switch IP address. For Directors (chassis-based systems), the switch IP addresses are aliases of the physical Ethernet interfaces on the CP blades. When specifying client IP addresses for the logical switches in such systems, make sure the CP blade IP addresses are used.
To create the user: • Open the $PREFIX/etc/raddb/user file in a text editor and add user names and roles for users who will be accessing the switch and authenticating RADIUS. The user will log in using the role specified with Brocade-Auth-Role. The valid roles include Root, Admin, SwitchAdmin, ZoneAdmin, SecurityAdmin, BasicSwitchAdmin, FabricAdmin, Operator and User. You must use quotation marks around “password” and “role”.
Each user group should be associated with a specific switch login role. For example, you should configure a user group for root, admin, factory, switchadmin, and user, and then add any users whose logins you want to associate to the appropriate group. • Configuring the server To enable CHAP: 1. From the Windows Start menu, select Programs > Administrative Tools > Local Security Policy to open the Local Security Settings window. 2.
6. In the Add Remote Access Policy window, enter an easily identifiable Policy friendly name that will enable you to see the switch login for which the policy is being created; then click Next. 7. After the Add Remote Access Policy window refreshes, click Add. 8. In the Select Attribute window, select Windows Groups and click Add. 9. In the Groups window, click Add. 10. In the Select Groups window, select the user-defined group for which you are creating a policy and click Add.
To set up LDAP: 1. Install a certificate on the Windows Active Directory server for LDAP. Create a user in Microsoft Active Directory server. For instructions on how to create a user, refer to www.microsoft.com or Microsoft documentation to create a user in your Active Directory. 2. Create a group name that uses the switch’s role name so that the Active Directory group’s name is the same as the switch’s role name. 3. Associate the user to the group by adding the user to the group.
Configuring authentication servers on the switch RADIUS and LDAP configuration of the switch is controlled by the aaaConfig command. At least one RADIUS or LDAP server must be configured before you can enable RADIUS or LDAP service. You can configure the RADIUS or LDAP service even if it is disabled on the switch. You can configure up to five RADIUS or LDAP servers. You must be logged in as admin or switchadmin to configure the RADIUS service.
To add a RADIUS server to the switch configuration: 1. Connect to the switch and log in using an admin account. 2. Enter this command: switch:admin> aaaConfig --add [-p port] [-s secret] [-t timeout] [-a pap | chap | peap-mschapv2] server Enter either a server name or IPv4 or IPv6 address. Avoid duplicating server listings (that is, listing the same server once by name and again by IP address). Up to five servers can be added to the configuration. -p port Optional: Enter a server port.
NOTE: When the RADIUS authentication mode is set to radius;local, you cannot downgrade the Fabric OS to any version earlier than 5.2.0. Previous versions do not support the radius;local mode. When the LDAP authentication mode is set to ldap;local, you cannot downgrade the Fabric OS to any version earlier than 6.x. Previous versions do not support the ldap;local mode. To enable and disable a RADIUS or LDAP server: 1. Connect to the switch and log in using an admin account. 2.
To change an LDAP server configuration: 1. Connect to the switch and log in using an admin account. 2. Enter this command: switch:admin> aaaConfig --change server [-p port] [-t timeout] [-d domain_name] server -p port -t timeout -d domain_name Enter either a server name or IPv4 address. Microsoft’s Active Directory does not support IPv6 addresses. Avoid duplicating server listings (that is, listing the same server once by name and again by IP address).
Setting the boot PROM password with a recovery string To set the boot PROM password with a recovery string, refer to the section that applies to your switch model. NOTE: Setting the boot PROM password requires accessing the boot prompt, which stops traffic flow through the switch until the switch is rebooted. You should perform this procedure during a planned down time.
4/256 SAN Director and DC SAN Backbone Director (short name, DC Director) The boot PROM and recovery passwords must be set for each CP blade on the 4/256 SAN Director or DC Director. To set the boot PROM password for a Director with a recovery string: 1. Connect to the serial port interface on the standby CP blade. 2. Connect to the active CP blade by serial or Telnet and enter the haDisable command to prevent failover during the remaining steps. 3.
Setting the boot PROM password without a recovery string Although you can set the boot PROM password without also setting the recovery string, it is strongly recommended that you set both the password and the string as described in ”Setting the boot PROM password with a recovery string” on page 79. If your site procedures dictate that you must set the boot PROM password without the string, follow the procedure that applies to your switch model.
The following options are available: Option Description 1 2 3 Start system. Continues the system boot process. Recovery password. Lets you set the recovery string and the boot PROM password. Enter command shell. Provides access to boot parameters. 6. Enter 3. 7. Enter the passwd command at the shell prompt. NOTE: The passwd command only applies to the boot PROM password when it is entered from the boot interface. 8. Enter your boot PROM password at the prompt, then re-enter it when prompted.
3 Configuring standard security features This chapter provides information and procedures for configuring standard Fabric OS security features such as protocol and certificate management. IMPORTANT: Secure Fabric OS is no longer supported in Fabric OS 6.x. Secure protocols Fabric OS supports the secure protocols shown in Table 15. Table 15 Secure protocol support Protocol Description SSL Supports SSLv3, 128-bit encryption by default. Fabric OS uses SSL to support HTTPS.
The security protocols are designed with the four main usage cases described in Table 17. Table 17 Main security scenarios Fabric Management interfaces Comments Nonsecure Nonsecure No special setup is needed to use Telnet or HTTP. Nonsecure Secure Secure protocols may be used. An SSL switch certificate must be installed if HTTPS is used. Secure Secure Secure protocols are supported on Fabric OS v4.1.0 and later switches.
Configuring the Telnet protocol Telnet is enabled by default. To prevent users from passing clear text passwords over the network when they connect to the switch, you can block the Telnet protocol using an IP Filter policy. NOTE: Before blocking Telnet, make sure you have an alternate method of establishing a connection with the switch. Blocking Telnet To block Telnet: 1. Connect to the switch and log in as admin. Connect through some means other than Telnet: for example, through SSH. 2.
Blocking listeners HP switches block Linux subsystem listener applications that are not used to implement supported features and capabilities. Table 18 lists the listener applications that Brocade switches either block or do not start.
Port configuration The following Table provides information on ports that the switch uses. When configuring the switch for various policies, take into consideration firewalls and other devices that may sit between switches in the fabric and your network or between the managers and the switch. Port Type Common use Comment 22 TCP SSH 23 TCP Telnet 123 TCP NTP 80 TCP HTTP Use the ipfilter command to block the port. 111 TCP sunrpc This port is used by Platform API.
Summary of SSL procedures You configure for SSL by obtaining, installing, and activating digital certificates for SSL support. Certificates are required on all switches that are to be accessed through SSL. You also need to install a certificate in the Java Plug-in on the management workstation, and you may need to add a certificate to your Web browser. Configuring for SSL involves these major steps, which are shown in detail in the next sections. 1. Choose a Certificate Authority (CA). 2.
IMPORTANT: limited. HP recommends selecting 1024 in most cases. CA support for the 2048-bit key size is Generating and storing a CSR After generating a public/private key, perform this procedure on each switch. 1. Connect to the switch and log in as admin. 2. Enter this command: switch:admin> seccertutil gencsr 3.
It may take several days to receive the certificates. If the certificates arrive by e-mail, save them to an FTP server. If the CA provides access to the certificates on an FTP server, make note of the path name and make sure you have a login name and password on the server. Installing a switch certificate Perform this procedure on each switch. 1. Connect to the switch and log in as admin. 2. Enter this command: switch:admin> seccertutil import 3.
Configuring the browser The root certificate may already be installed on your browser, but if not, you must install it. To see whether it is already installed, check the certificate store on your browser. The next procedures are guides for installing root certificates to Internet Explorer and Mozilla browsers. For more detailed instructions, refer to the documentation that came with the certificate. To check and install root certificates on Internet Explorer: 1.
Trust this certificate? [no]: yes Certificate was added to keystore In the example, changeit is the default password and RootCert is an example root certificate name. Displaying and deleting certificates Table 21 summarizes the commands for displaying and deleting certificates. For details on the commands, see the Fabric OS Command Reference.
Configuring for SNMP You can configure for the automatic transmission of SNMP information to management stations. SNMPv3 and SNMPv1 are supported. The configuration process involves configuring the SNMP agent and configuring SNMP traps. The following commands are used in the process: • Use the configure command to set the security level. You can specify no security, authentication only, or authentication and privacy.
webtools attributes (yes, y, no, n): [no] System (yes, y, no, n): [no] No changes. Using the snmpConfig command 4. Use the snmpConfig --set command to change either the SNMPv3 or SNMPv1 configuration. You can also change access control, MIB capability, and system group. Sample SNMPv3 configuration switch:admin> snmpconfig --set snmpv3 SNMPv3 user configuration: User (rw): [snmpadmin1] adminuser Auth Protocol [MD5(1)/SHA(2)/noAuth(3)]: (1..
Sample SNMPv1 configuration switch:admin> snmpconfig --set snmpv1 SNMP community and trap recipient configuration: Community (rw): [Secret C0de] admin Trap Recipient's IP address in dot notation: [0.0.0.0] 10.32.225.1 Trap recipient Severity level : (0..5) [0] 1 Community (rw): [OrigEquipMfr] Trap Recipient's IP address in dot notation: [10.32.225.2] Trap recipient Severity level : (0..5) [1] Community (rw): [private] Trap Recipient's IP address in dot notation: [10.32.225.
connUnitStatusChange: YES connUnitEventTrap: YES connUnitSensorStatusChange: YES connUnitPortStatusChange: YES SW-EXTTRAP: NO FICON-TRAP: YES linkRNIDDeviceRegistration: YES linkRNIDDeviceDeRegistration: YES linkLIRRListenerAdded: YES linkLIRRListenerRemoved: YES linkRLIRFailureIncident: YES HA-TRAP: YES fruStatusChanged: YES cpStatusChanged: YES fruHistoryTrap: YES FCIP-TRAP: NO Sample systemGroup configuration (default) switch:admin> snmpconfig --default systemGroup ***** This command will reset the agent
4 Configuring advanced security features This chapter provides information and procedures for configuring advanced Fabric OS security features such as Access Control List (ACL) policies, authentication policies, and IP Filtering for HP’s Fibre Channel switches. NOTE: Run all commands, with the suggested role, in this chapter by logging in to Administrative Domain (AD) 255 or, if Administrative Domains have not been implemented, log in to AD 0.
and active sets but they have different values, then the policy has been modified but the changes have not been activated. Admin Domain considerations: ACL management can be done on AD255 and in AD0 only if other there are no user-defined Admin Domains. Both AD0 (when no other user-defined Admin Domains exist) and AD255 provide an unfiltered view of the fabric.
• ”Configuring the database distribution settings” on page 122 Configure a switch to accept or reject the distribution of polices. • ”Distributing ACL policies to other switches” on page 123 Configure the distribution of policies to switches within the fabric. Displaying ACL policies Use the secPolicyShow command to display the active and defined policy sets.
distribution, Fabric OS 6.0.0 switches may enforce FCS policy and perform database distribution among 5.3.0 and 6.0.0 switches while still allowing pre-5.3.0 switches to join the fabric. • Distribution to pre-5.3.0 switches with specific Domain IDs When specific Domain IDs are given for the distribution, all domains must be on a switch with Fabric OS 5.3.0 or later. If one of the domains is pre-5.3.0 the distribution operation will fail. • Distribution to pre-5.3.
Overview of steps to create and manage the FCS policies Whether your intention is to create new FCS policies or manage your current FCS policies, you must follow certain steps to ensure the domains throughout your fabric have the same policy. The local-switch WWN cannot be deleted from the FCS policy. 1. Set the pre-6.0 switches in the fabric to accept the FCS policy using the fddcfg --localaccept/localreject command. 2. Create the FCS policy using the secPolicyCreate command. 3.
For example, to move a backup FCS switch from position 2 to position 3 in the FCS list, using interactive mode: primaryfcs:admin> secpolicyfcsmove Pos Primary WWN DIdswName. ================================================= 1 Yes 10:00:00:60:69:10:02:181switch5. 2 No 10:00:00:60:69:00:00:5a2switch60. 3 No 10:00:00:60:69:00:00:133switch73. Please enter position you’d like to move from : (1..3) [1] 2 Please enter position you’d like to move to : (1..
switch. Setting the configuration parameter to accept indicates distribution of the policy will be accepted and distribution may be initiated using the distribute -p command. Setting the configuration parameter to reject indicates the policy distribution is rejected and the switch may not distribute the policy. The default value for the distribution configuration parameter is accept, which means the switch accepts all database distributions and is able to initiate a distribute operation for all databases.
• You cannot manage proxy devices with DCC policies. Proxy devices are always granted full access, even if the DCC policy has an entry that restricts or limits access of a proxy device. Creating a DCC policy DCC policies must follow the naming convention “DCC_POLICY_nnn,” where nnn represents a unique string. The maximum length is 30 characters, including the prefix DCC_POLICY_. To save memory and improve performance, one DCC policy per switch or group of switches is recommended.
Examples of creating DCC policies To create the DCC policy “DCC_POLICY_server” that includes device 11:22:33:44:55:66:77:aa and port 1 and port 3 of switch domain 1: switch:admin> secpolicycreate "DCC_POLICY_server", "11:22:33:44:55:66:77:aa;1(1,3)" DCC_POLICY_server has been created To create the DCC policy “DCC_POLICY_storage” that includes device port WWN 22:33:44:55:66:77:11:bb, all ports of switch domain 2, and all currently connected devices of switch domain 2: switch:admin> secpolicycreate "DCC_POLI
For example, to create an SCC policy that allows switches that have Domain IDs 2 and 4 to join the fabric: switch:admin> secpolicycreate "SCC_POLICY", "2;4" SCC_POLICY has been created 3. To save or activate the new policy, enter either the secPolicySave or the secPolicyActivate command. If neither of these commands is entered, the changes are lost when the session is logged out.
Removing a member from an ACL policy To remove a member from an ACL policy: 1. Connect to the switch and log in using an account assigned to the admin role. 2. Type secPolicyRemove “policy_name”, “member;...;member”. where policy_name is the name of the ACL policy. member is the device or switch to be removed from the policy, identified by IP address, switch Domain ID, device or switch WWN, or switch name. 3. To implement the change immediately, enter the secPolicyActivate command.
authutil –-set to set the authentication protocol which can then be verified using the command authutil –-show CLI. NOTE: The standards-compliant DH-CHAP and FCAP authentication protocols are not compatible with the SLAP protocol that was the only protocol supported in earlier Fabric OS releases 4.2, 4.1, 3.1, 2.6.x. Fabric OS 6.0.0 switch-to-switch authentication implementation is fully backward compatible with 3.2, 4.2, 4.4, 5.0, 5.1, 5.2, and 5.3.0.
WARNING! If data input has not been completed and a failover occurs, the command is terminated without completion and the entire user input is lost. If data input has completed, the enter key pressed, and a failover occurs, data may or may not be replicated to the other CP depending on the timing of the failover. Log in to the other CP after the failover is complete and verify the data was saved. If data was not saved, run the command again.
Device authentication policy Device authentication policy can also be categorized as an HBA authentication policy. Fabric wide distribution of the device authentication policy is not supported since the device authentication requires manual interaction in setting the HBA shared secrets and switch shared secrets, and most of the HBAs do not support the defined DH groups for use in the DH-CHAP protocol.
Selecting authentication protocols Use the authUtil command to perform the following tasks: • Display the current authentication parameters • Select the authentication protocol used between switches • Select the Diffie-Hellman (DH) group for a switch Run the authUtil command on the switch you want to view or change.
WARNING! correctly. This command may bring down the E_Port(s) if the DH-CHAP shared secrets are not installed To re-authenticate E_Ports: 1. Log in to the switch using an account assigned to the admin role. 2. On a switch running Fabric OS 5.3.0 and later, type the following command: $authutil –-authinit Example $authutil –-authinit 2,3,4 Example $authutil –-authinit allE (all E_ports in the switch) For directors, use the slot/port format for specifying the port number.
To set a secret key pair: 1. Log in to the switch using an account assigned to the admin role. 2. On a switch running Fabric OS 4.x, 5.x, or 6.0, type secAuthSecret --set; on a switch running Fabric OS 3.x, type secAuthSecret "--set". The command enters interactive mode. The command returns a description of itself and needed input; then it loops through a sequence of switch specification, peer secret entry, and local secret entry. To exit the loop, press Enter for the switch name; then type y.
Accept distributions configuration parameter Local Switch configuration parameters are needed to control whether a switch accepts or rejects distributions of the AUTH policy using the distribute command and whether the switch may initiate distribution of the policy. To set the local switch configuration parameter, refer to ”Configuring the database distribution settings” on page 122.
Displaying an IP Filter policy Displays the IP Filter policy content for the specified policy name, or all IP Filter policies if policy name is not specified. For each IP Filter policy, the policy name, type, persistent state and policy rules are displayed. The policy rules are listed by the rule number in ascending order. There is no pagination stop for multiple screens of information. Pipe the output to the more command to achieve this.
Deleting an IP Filter policy You can delete a specified IP Filter policy. Deleting an IP Filter policy will remove it from the temporary buffer. To permanently delete the policy from persistent database, run ipfilter --save. An active IP Filter policy cannot be deleted. To delete an IP Filter policy: 1. Log in to the switch using an account assigned to the admin role. 2. Type in the following command: ipfilter –delete where is the name of the policy. 3.
Table 30 Supported services (continued) Service name Port number telnet 23 www 80 TCP and UDP protocols are valid selections. Fabric OS 5.3.0 and later does not support configuration to filter other protocols. Implicitly, ICMP type 0 and type 8 packets are always allowed to support ICMP echo request and reply on commands like ping and traceroute. For the action, only “permit” and “deny” are valid.
If none of the rules in the policy matches the incoming packet, the two implicit rules will be matched to the incoming packet. If the rules still do not match the packet, the default action, which is to deny, will be taken. When the IPv4 or IPv6 address for the management interface of a switch is changed through the ipAddrSet command or manageability tools, the active IP Filter policies will automatically become enforced on the management IP interface with the changed IP address.
To abort a transaction associated with IP Filter: 1. Log in to the switch using an account assigned to the admin role. 2. Type in the following command: ipfilter –-transabort IP Filter policy distributions The IP Filter policy is manually distributed, using the distribute --p “IPFILTER” command. The distribution includes both active and defined IP Filter policies. All policies are combined as a single entity to be distributed and cannot be selectively distributed.
Table 33 explains how the local database distribution settings and the fabric-wide consistency policy affect the local database when the switch is the target of a distribution command. Table 33 Interaction between fabric-wide consistency policy and distribution settings Distribution setting Fabric-wide consistency policy Absent (default) Tolerant Strict Reject Database is protected, it cannot be overwritten. May not match other databases in the fabric. Invalid configuration.1 Invalid configuration.
2. Enter the following command: switch:admin> fddcfg --showall Local Switch Configuration for all Databases:DATABASE - Accept/Reject --------------------------------SCC accept DCC accept PWD accept FCS accept AUTH accept IPFILTER accept Fabric Wide Consistency Policy:- "" To enable local switch protection: 1. Connect to the switch and log in using an account assigned to the admin role. 2.
Table 35 describes how the target switch database distribution settings affect the distribution. Table 35 ACL policy database distribution behavior Target switch Distribution Results Fabric OS version Database setting 5.1.0 or earlier NA Fails An error is returned. The entire transaction is aborted and no databases are updated. 5.2.0 Reject Fails The target switch explicitly refuses the distribution. The entire transaction is aborted and no databases are updated.
To display the fabric-wide consistency policy: 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the fddCfg --showall command. The following example shows policies for a fabric where no consistency policy is defined.
Under both conflicting conditions, secPolicyActivate is blocked in the merged fabric.Use fddcfg –fabwideset command to resolve the fabric-wide consistency policy conflicts. Use the distribute command to explicitly resolve conflicting ACL policies. When a switch is joined to a fabric with a strict SCC or DCC fabric-wide consistency policy, the joining switch must have a matching fabric-wide consistency policy.
Non-matching fabric-wide consistency policies You may encounter one of the following two scenarios: Merging a fabric with a strict policy to a fabric with an absent, tolerant, or non-matching strict policy. The merge fails and the ports are disabled. Table 38 shows merges that are not supported.
Zeroization functions Explicit zeroization can be done at the discretion of the security administrator. These functions clear the passwords and the shared secrets. The following table lists the various keys used in the system that will be zeroized in a FIPS compliant FOS module. Table 40 Zeroization behavior Keys Zeroization CLI Description DH Private keys No CLI required Keys will be zeroized within code before they are released from memory.
Conditional tests These tests are for the random number generators and are executed to verify the randomness of the random number generator. The conditional tests are executed each time prior to using the random number provided by the random number generator. The results of all self-tests, for both power-up and conditional, are recorded in the system log or are output to the local console. This includes logging both passing and failing results. To enter into single-user mode: 1.
loading kernel . . . kjournald starting. Commit interval 5 seconds EXT3-fs: mounted filesystem with ordered data mode. VFS: Mounted root (ext3 filesystem) readonly. Trying to move old root to /initrd ... okay Freeing unused kernel memory: 108k init INIT: version 2.78 booting sh-2.04# 5. On all platforms, from the shell prompt, enter the following commands: mount -o remount,rw,noatime / mount /dev/hda2 /mnt 6.
Table 41 FIPS mode restrictions Features FIPS mode Non-FIPS mode DH-CHAP/FCAP hashing algorithms SHA-1 MD5 and SHA-1 Signed firmware Mandatory firmware signature validation Optional firmware signature validation Configupload/ download/ supportsave/ firmwaredownload SCP only FTP and SCP IPsec Usage of AES-XCBC, MD5 and DH group 1 are blocked No restrictions Radius auth protocols PEAP-MSCHAPv2 CHAP, PAP, PEAP-MSCHAPv2 Preparing the switch for FIPS The following functionalities are blocked
b. Add a rule to the IP Filter policy, see ”To add a rule to an IP Filter policy:” on page 120. You can use the following modifications to the rule: ipfilter --addrule -rule -sip -dp -proto -act • -sip option can be given as any • -dp option for the port numbers for Telnet, HTTP, and RPC are 23, 80, and 898 respectively • -proto option should be set to tcp c.
6. Optional: Use the configure command to set switch to use non-signed firmware. By keeping the switch set to use signed firmware, all firmware downloaded to the switch will have to be signed with a key. 7. Disable selftests by typing the following command: fipscfg --disable selftests 8. Disable IPFilter policies that were created to enable FIPS. 9. Optional: Configure RADIUS server authentication protocol. 10. Reboot the switch. To zeroize for FIPS: 1.
Configuring advanced security features
5 Maintaining configurations This chapter provides procedures for basic switch configuration maintenance. Maintaining consistent configuration settings It is important to maintain consistent configuration settings on all switches in the same fabric because inconsistent parameters (such as inconsistent PID formats) can cause fabric segmentation.
4. Respond to the Protocol (scp or ftp) Server Name or IP Address User name File name Password prompts as follows: If your site requires the use of Secure Copy, specify scp. Otherwise, specify FTP. Enter the name or IP address of the server where the file is to be stored; for example, 192.1.2.3. You can enter a server name if DNS is enabled. For details about the dnsConfig command, see the Fabric OS Command Reference. Enter the user name of your account on the server; for example, JohnDoe.
Restoring switch information Run the commands listed in Table 42 and save the output in a file format. Store the files in a safe place for emergency reference. Table 42 CLI commands to display switch configuration information Command Displays configShow System configuration parameters and settings, including license information, zoning, and licensing information. ipAddrShow The IP address.
To restore a configuration: 1. Verify that the FTP service is running on the server where the backup configuration file is located. 2. Connect to the switch and log in as admin. 3. If there are any changed parameters in the configuration file that do not belong to SNMP, Fabric Watch, or ACL, disable the switch by entering the switchDisable command. 4. Enter the configDownload command. The command becomes interactive and you are prompted for the required information. 5.
The following example shows configDownload run on a switch with Admin Domains: switch:AD5:admin>configdownload Protocol (scp or ftp) [ftp]: Server Name or IP Address [host]: 10.1.2.3 User Name [user]: JohnDoe File Name [config.txt]: /pub/configurations/config.txt *** CAUTION *** This command is used to download a backed-up configuration for a specific switch. If using a file from a different switch, this file's configuration settings will override any current switch settings.
There may be some restrictions if you are using Admin Domains. See ”Managing administrative domains” on page 143 for details. Messages captured in the logs Configuration download generates both RASLog and Audit log messages resulting from execution of the configDownload command.
To download a configuration file from one switch to another same model switch: 1. Configure one switch first. 2. Use the configUpload command to save the configuration information. See ”Backing up a configuration” on page 135. 3. Run configDefault on each of the target switches, and then use the configDownload command to download the configuration file to each of the target switches. See ”Restoring a configuration” on page 137.
Maintaining configurations
6 Managing administrative domains This chapter provides procedures for using administrative domains (Admin Domain or AD). An Admin Domain is a logical grouping of fabric elements that defines what switches, ports, and devices you can view and modify. An Admin Domain is a filtered administrative view of the fabric. NOTE: If you do not implement Admin Domains, the feature has no impact on users and you do not need to learn how to use this functionality.
AD1 AD2 Figure 2 Fabric with two Admin Domains Figure 3 shows how users get a filtered view of this fabric, depending on which Admin Domain they are in. As shown in Figure 4, users can see all switches and E_Ports in the fabric, regardless of their Admin Domain; however, the switch ports and end devices are filtered based on Admin Domain membership.
Admin Domain features Admin Domains allow you to: • Define the scope of an Admin Domain to encompass ports and devices within a switch or a fabric. • Share resources across multiple Admin Domains. For example, you can share array ports and tape drives between multiple departments. In Figure 2, one of the storage devices is shared between AD1 and AD2. • Have a separate zone database for each Admin Domain. See ”Admin Domains, zones, and zone databases” on page 162 for more information.
AD0 AD0 is a system-defined Admin Domain that, in addition to containing members you explicitly added (similar to user-defined Admin Domains), contains all online devices, switch ports, and switches that have not been assigned to any user-defined Admin Domain. Unlike user-defined Admin Domains, AD0 has an implicit and an explicit membership list. User-defined Admin Domains have only explicit members.
AD1 AD255 AD0 AD2 Figure 4 Fabric with AD0 and AD255 Admin Domain access levels Admin Domains offer a hierarchy of administrative access. To manage Admin Domains, you must be a physical fabric administrator. A “physical fabric administrator” is a user with the Admin role and access to all Admin Domains (AD0 through AD255). Only a physical fabric administrator can perform Admin Domain configuration and management. Other administrative access is determined by your defined RBAC role and AD membership.
Admin Domains and login You are always logged in to an Admin Domain, and you can view and modify only the devices in that Admin Domain. If you have access to more than one Admin Domain, one of them will have been specified as your “home Admin Domain,” the one you are automatically logged in to. If your home Admin Domain is deleted or deactivated, then by default you are logged in to the lowest numbered active Admin Domain in your Admin Domain List.
Switch port members Switch port members are defined by switch domain,port. A switch port member: • Grants port control rights and zoning rights for that switch port. • Grants view access and zoning rights to the device connected to that switch port. • Allows you to share domain,port members across multiple Admin Domains. In each Admin Domain, you can also zone shared devices differently. • Implicitly includes all devices connected to the specified domain,port members in the Admin Domain membership.
Figure 5 shows an unfiltered view of a fabric with two switches, three devices, and two Admin Domains. The devices are labeled with device WWNs and the switches are labeled with Domain ID and switch WWNs.
AD database exactly matches both the defined and effective configurations of the local AD database. If the AD database merge fails, the E_Port is segmented with “AD conflict” error code. Compatibility Admin Domains can be implemented in fabrics with a mix of AD-aware switches and AD-unaware switches.
Understanding the AD transaction model You use the ad command to perform most of the tasks in this section. This command follows a batched-transaction model, which means that changes to the Admin Domain configuration occur in the transaction buffer. An Admin Domain configuration can exist in several places: • Effective configuration—The Admin Domain configuration that is currently in effect. • Defined configuration—The Admin Domain configuration that is saved in flash memory.
Creating an Admin Domain To create an Admin Domain, you must specify an Admin Domain name, number, or both. • If you create an Admin Domain using only a number, the Admin Domain name is automatically assigned to be “ADn”, where n is the number you specified. For example, if you specify AD number = 4, then AD name is set to “AD4”.
Assigning a user to an Admin Domain After you create an Admin Domain, you can specify one or more user accounts as the valid accounts who can use that Admin Domain. You create these user accounts using the userConfig command. User accounts have the following characteristics with regard to Admin Domains: • A user account can only have a single role. You can choose roles from one of the seven types of roles, either the existing user and administrator role or one of the other RBAC roles.
where username is the name of the account and home_AD is the home Admin Domain. The following example creates new user account pf_admin1 with an admin role, access to all Admin Domains (AD0 through AD255), and home Admin Domain set to 255. This user account is now a physical fabric administrator. sw5:admin> userconfig --add pf_admin1 -r admin -h 255 -a "0-255" Activating and deactivating Admin Domains An Admin Domain can be in either an active or inactive state.
Adding and removing Admin Domain members Use the following procedures to add or remove members of an Admin Domain. NOTE: If you remove the last member of an Admin Domain, that Admin Domain is automatically deleted. To add members to an existing Admin Domain: 1. Connect to an AD-aware switch and log in as admin. 2. Switch to the AD255 context, if you are not already in that context. ad --select 255 3.
The rename operation does not take effect if the Admin Domain you want to rename is part of the effective configuration and thus enforced. 4. To end the transaction now, enter ad --save to save the Admin Domain definition or enter ad --apply to save the Admin Domain definition and directly apply the definitions to the fabric. The Admin Domain numbers remain unchanged after the operation. The following example changes the name of Admin Domain Eng_AD to Eng_AD2.
Validating an Admin Domain member list The ad --validate option allows you to validate the device and switch member list and flag all resources that are from AD-unaware switches. You can use the validate option to list Admin Domain members from AD-unaware switches and non-existing or offline Admin Domain members. You can use the validate option to identify misconfigurations of the Admin Domain.
Table 46 Ports and devices in CLI output For Condition domain,port The port is specified in the domain,port member list of the Admin Domain. One or more WWNs specified in the AD member list is attached to the domain,port. Device WWN The device WWN is specified in the AD WWN member list. The device WWN is attached to one of the domain,port specified in the AD member list. RASLog and SYSlog output is not filtered based on AD membership.
The following example displays membership information about AD1. sw5:AD1:admin> ad --show Current AD Number: 1 AD Name: TheSwitches Effective configuration: -----------------------AD Number: 1 AD Name: TheSwitches Switch WWN members: State: Active 50:06:06:99:00:2a:e9:01; 50:00:51:e0:23:36:f9:01; 50:06:06:98:05:be:99:01; Switching to a different Admin Domain context The ad --select option is used to switch between different Admin Domain contexts.
Table 47 lists some of the Fabric OS features and considerations that apply when using Admin Domains. Table 47 Admin Domain interaction with Fabric OS features Fabric OS feature Admin Domain interaction ACLs If no user-defined Admin Domains exist, you can run ACL configuration commands in only AD0 and AD255. If any user-defined Admin Domains exist, you can run ACL configuration commands only in AD255.
Admin Domains, zones, and zone databases Each Admin Domain has its own zone database, with both defined and effective zone configurations and all related zone objects (zones, zone aliases, and zone members). Within an Admin Domain, you can configure zoning only with the devices that are present in that Admin Domain.
The auto-converted LSAN zone names might collide with LSAN zone names in AD0 (for example, in the above example, if AD0 contains lsan_for_linux_farm_AD005, this would cause a name collision). Fabric OS does not detect or report such name clash. LSAN zone names greater than 57 characters are not converted or sent to the FCR phantom domain.
Managing administrative domains
7 Installing and maintaining firmware This chapter provides procedures for installing and maintaining firmware. Fabric OS 6.0 provides nondisruptive firmware installation. This chapter refers to the following specific types of blades inserted into either the 4/256 SAN Director or DC SAN Backbone Director (short name, DC Director): • FC blades or port blades contain only Fibre Channel ports: FC4-16/32/48, FC10-6, and FC8-16/32/48. FC8-32 & FC8-48 aren't supported in 4/256 until 6.1.0.
The command supports both non-interactive and interactive modes. If the firmwareDownload command is issued without any operands, or if there is any syntax error in the parameters, the command enters an interactive mode, in which you are prompted for input. TIP: For each switch in your fabric, complete all firmware download changes on the current switch before issuing the firmwareDownload command on the next switch. This process ensures nondisruption of traffic between switches in your fabric.
Preparing for firmware downloads Before executing a firmware download, it is recommended that you perform the tasks listed in this section. In the unlikely event of a failure or time-out, the preparation tasks that are described in this section will enable you to provide HP the information required to perform advanced troubleshooting. It is recommended that you perform a configUpload to back up the current configuration before you download firmware to a switch.
Checking connected switches When checking connected switches, ensure that any older versions are supported. See the recommended version (shown in Table 50) before upgrading firmware on the switch. Go to http://www.hp.com to view end-of-life policies. Table 50 Recommended firmware Switch model Earliest compatible version HP StorageWorks 1 Gb Switch Not supported in same fabric with 6.x Recommended version for interoperating with Fabric OS 6.x switches. 3.2.1b 3.2.1b 5.0.5c 5.0.5f 5.1.1b 5.3.1 5.
Refer to the Fabric OS Compatibility section of the HP StorageWorks Fabric OS 6.x release notes, for the recommended firmware version. If the 4/8 SAN Switch, 4/16 SAN Switch, SAN Switch 2/8V, SAN Switch 2/16V, SAN Switch 2/32, Brocade 4Gb SAN Switch for HP p-Class BladeSystem, Brocade 4Gb SAN Switch for HP c-Class BladeSystem, SAN Switch 4/32, 4/64 SAN Switch, SAN Switch 4/32B, or 400 MP Router switches are adjacent and you start firmware downloads on them at the same time, there may be traffic disruption.
HP StorageWorks 4/8 SAN Switch, 4/16 SAN Switch, Brocade 4Gb SAN Switch for HP p-Class BladeSystem, Brocade 4Gb SAN Switch for HP c-Class BladeSystem, SAN Switch 4/32, 4/64 SAN Switch, SAN Switch 4/32B, 400 MP Router, and firmware download The upgrade process first downloads and then commits the firmware to the switch. While the upgrade is proceeding, you can start a session on the switch and use the firmwareDownloadStatus command to observe the upgrade progress if you wish.
Network protocol Password Specify the file transfer protocol used to download the firmware from the file server. Valid values are FTP and SCP. The Values are not case-sensitive. If “-p” is not specified, firmwareDownload will determine the protocol automatically by checking the config.security parameter on the switch. Enter the password for the server. This operand can be omitted if firmware is accessible through a local directory, or if no password is required by the FTP or SCP server.
Overview of the firmware download process on directors The following summary describes the default behavior of the firmwareDownload command (without options) on the 4/256 SAN Director and DC Director. After you enter the firmwareDownload command on the active CP blade the following actions occur: 1. The standby CP blade downloads firmware. 2. The standby CP blade reboots and comes up with the new Fabric OS. 3. The active CP blade synchronizes its state with the standby CP blade. 4.
CP blades must be synchronized and running Fabric OS 4.2.0 or later to provide a nondisruptive download. If the two CP blades are not synchronized, enter the haSyncStart command to synchronize them. If the CPs still are not synchronized, contact HP. 7. Enter the firmwareDownload –s command. 8. Respond to the prompts as follows: Server Name Enter the name or IP address of the FTP server, or SSH server for SCP, where or IP Address the firmware file is stored; for example, 192.1.2.3.
Autoleveling takes place in parallel with the firmware download being performed on the CPs, but does not impact performance. Fibre Channel traffic is not disrupted during autoleveling, but GbE traffic on AP blades may be affected. sw77:admin> firmwaredownload Type of Firmware (FOS, SAS, or any application) [FOS]: Server Name or IP Address: 192.168.32.10 Network Protocol (1-auto-select, 2-FTP, 3-SCP) [1]: User Name: userfoo File Name: /home/userfoo/5.3.
[8]: Thu Jul 28 00:37:50 2005 Slot 7 : Firmware commit is started. [9]: Thu Jul 28 00:37:50 2005 Slot 2 : Firmware commit has completed. [10]: Thu Jul 28 00:37:50 2005 Slot 7 : Firmware commit has completed. (Firmwaredownload has completed.) 11. Enter the firmwareShow command to display the new firmware versions. Following is an example of firmwareShow output on the 4/256 SAN Director.
firmwaredownload from a USB device The DC Director supports a firmware download from the USB device attached to the active CP. NOTE: The USB device ships with the DC Director only. Before the USB device can be accessed by the firmwaredownload command, it must be enabled and mounted as a file system. The firmware images to be downloaded must be stored under the /firmware directory in the USB file system. Multiple images can be stored under this directory.
FIPS Support Federal information processing standards (FIPS) specify the security standards needed to satisfy a cryptographic module utilized within a security system for protecting sensitive information in the computer and telecommunication systems. For more information about FIPS, refer to ”Configuring advanced security features” on page 17. The 6.0 firmware is digitally signed using the OpenSSL utility to provide FIPS support.
3. Respond to the Server Name or IP Address Download from USB Network protocol User name File name Password prompts as follows: Enter the name or IP address of the FTP server, or SSH server for SCP, where the firmwarekey file is stored; for example, 192.1.2.3. Optional: -U (upper case) Specify this option if you want to download from the USB device attached to the active CP. Specify the file transfer protocol used to download the firmware from the file server. Valid values are FTP and SCP.
Power-on firmware checksum test FIPS requires the checksums of the executables and libraries on the filesystem to be validated before Fabric OS modules are launched. This is to make sure these files have not been changed after they are installed. When firmware RPM packages are installed during firmwareDownload, the MD5 checksums of the firmware files are stored in the RPM database on the filesystem. This will go through all of the files in the RPM database.
IMPORTANT: Stop! If you want to restore the firmware, stop here and skip ahead to step 9; otherwise, continue to step 8 to commit the firmware on the switch, which completes the firmware download operations. 8. Commit the firmware. a. Enter the firmwareCommit command to update the secondary partition with new firmware. Note that it takes several minutes to complete the commit operation. b. Enter the firmwareShow command to confirm both partitions on the switch contain the new firmware.
6. Update the firmware on the standby CP: a. Connect to the switch and log in as admin to the standby CP. b. Enter the firmwareDownload -s command and respond to the prompts. At this point, the firmware should download to the standby CP only. When it has completed the download to that CP, reboot it. The current switch session will be disconnected. 7. Fail over to the standby CP. a. Connect to the switch on the active CP. b. Enter the haShow command to verify that HA synchronization is complete.
11. Perform a commit on the active CP. a. From the current switch session on the active CP, enter the firmwareShow command and confirm that only the active CP secondary partition contains the old firmware. b. Enter the firmwareCommit command to update the secondary partition with the new firmware. It takes several minutes to complete the commit operation. Do not do anything on the switch while this operation is in process. c.
Validating firmwareDownload Validate the firmware download by running the following commands: firmwareShow, firmwareDownloadStatus, nsShow, nsAllShow, and fabricShow. NOTE: When you prepared for the firmware download earlier, you issued either the supportShow or supportSave command. Although you can issue the command again and compare the output from before and after, it may take up to 30 minutes for the command to execute.
Troubleshooting firmwareDownload Starting in Fabric OS 5.2.0 a network diagnostic script and preinstallation check was added as a part of the firmwareDownload procedure. The script and preinstallation check performs troubleshooting and automatically checks for any blocking conditions. However, you should follow these best practices for firmware download before you start the procedure: • Keep all session logs.
• If LDAP is configured on the switch, delete the LDAP configuration. Preinstallation messages The messages in this section are displayed if an exception case is encountered during firmware download from Fabric OS 5.2.0. The example earlier shows feature-related messages that you may see if you were downgrading from 5.2.0 to 5.1.0: The following items need to be addressed before downloading the specified firmware: Port mirror connections detected.
Use the slotShow command to display which slot the FC4-16IP port blade is in. Physically remove the blade(s) from the chassis, or use the micro-switch to turn the blade off. Retry the firmware download operation. Message AP Blade type 33 is inserted. Please use slotshow to find out which slot it is in and remove it. Cannot downgrade due to the presence of AP BLADE type 33. Remove or power off these blades before proceeding.
Message SW Blade type 51 is inserted. Please use slotshow to find out which slot it is in and remove it. Probable cause and recommended action The firmware download operation was attempting to downgrade a system to Fabric OS 5.3.0 or earlier with one or more FC8-48 port blades (blade ID 51) in the system. FC8-48 port blades are not supported on firmware 5.3.0 or earlier, so the firmware download operation failed. Use the slotShow command to display which slot the FC8-48 port blade is in.
Disable the switch and change the routing policy selection to one of the following supported selections on firmware 5.1.0 using the aptPolicy command, and then retry the firmware download operation. The supported selections are: policy 1 Port-based routing policy With this policy, the path chosen for an incoming frame is based on: 1. Incoming port on which the frame was received 2. Destination domain for the frame The chosen path remains the same if the dynamic load sharing (DLS) feature is not enabled.
Message Cannot downgrade due to LSAN count is set to 3000, please disable it before proceeding. Probable cause and recommended action If a switch is running 5.3.0 or higher and the LSAN count is at 3000, then you will not be allowed to downgrade to 5.2.0 or earlier. Use the fcrlsanmatrix command to disable the LSAN. Message Cannot downgrade due to LSAN zone binding is enabled. Please disable it before proceeding. Probable cause and recommended action If switch is running 5.3.
Message Cannot upgrade directly to 5.3.0. Upgrade your switch to 5.1 or 5.2 first before upgrading to the requested version. Probable cause and recommended action If the switch is running 5.0.0 or earlier, you will not be allowed to upgrade directly to 5.3.0 because of the “two-version” rule. Upgrade your switch to Fabric OS version 5.1.0 or 5.2.0 before upgrading to 5.3.0 Message Cannot upgrade due to the presence of an existing zone named “broadcast”. Rename this zone before proceeding.
Message The command failed due to the current zone size is not supported by the new firmware. Reduce the size of the configuration before proceeding. Probable cause and recommended action The firmware download operation was attempting to downgrade a system to Fabric OS 5.1.0 or earlier and the current zone size is not supported by the firmware version to be downloaded, so the firmware download operation failed. Reduce the zone database size to 256 KB.
Installing and maintaining firmware
8 Configuring Directors This chapter provides procedures specific to HP StorageWorks Director models. Changing a Director’s name HP recommends that you customize the enterprise-class platform name for each platform. Some system logs identify devices by platform names, if you assign meaningful platform names, logs are more useful. To change the platform name: 1. Connect to the switch and log in using an account assigned to the admin role. 2.
Director port numbering schemes Table 51 lists the port numbering schemes for the 4/256 Director and DC Director. Table 51 Port numbering schemes for the 4/256 Director and DC Director Port blades Numbering scheme FC2-16 FC4-16 FC8-16 Ports are numbered from 0 through 15 from bottom to top. FC4-32 FC8-32 Ports are numbered from 0 through 15 from bottom to top on the left set of ports and 16 through 31 from bottom to top on the right set of ports.
A number of fabric-wide databases supported by Fabric OS (including ZoneDB, the ACL DDC, and Admin Domain) allow a port to be designated by the use of a “D,P” (domain,port) notation. While the “P” component appears to be the port number, in up to 255 ports it is actually the area assigned to that port. If the PID format is changed from Extended-edge to Core, the “P” value for ports 0-127 also changes. If two ports are changed using the portSwap command, their respective areas and “P” values are exchanged.
Table 52 Default index/area_ID core PID assignment with no port swap (continued) Port on blade Slot 1Idx/area Slot 2Idx/area Slot 3Idx/area Slot 4Idx/area Slot 7Idx/area Slot 8Idx/area Slot 9Idx/area Slot 10Idx/area 23 135/135 151/151 167/167 183/183 199/199 215/215 231/231 247/247 22 134/134 150/150 166/166 182/182 198/198 214/214 230/230 246/246 21 133/133 149/149 165/165 181/181 197/197 213/213 229/229 245/245 20 132/132 148/148 164/164 180/180 196/196 212/212
To power off a port blade: 1. Connect to the switch and log in as admin. 2. Enter the slotPowerOff command with the slot number of the port blade you want to power off. switch:admin> slotpoweroff 3 Slot 3 is being powered off switch:admin> To provide power to a port blade: 1. Connect to the switch and log in as admin. 2. Enter the slotPowerOn command with the slot number of the port blade you want to power on.
To summarize: • When an FC4-16, FC4-32, FC8-16, FC8-32, FC10-6, or FC4-16IP blade is replaced by an FR4-18i blade, the FC configuration of the previously configured FC_Ports continues to be used, and all FC_Ports on the FR4-18i blade are persistently disabled.
Blade terminology and compatibility Before configuring a chassis, familiarize yourself with the Director CP blade and port blade nomenclature, as well as the port blade compatibilities. Often in procedures, only the abbreviated names for CP and port blades are used (for example, the FC4-16 blade). Table 53 includes CP and port blade abbreviations and descriptions.
type of CP blade installed and that each CP (primary and secondary partition) maintains the same firmware version. Core blades The DC Director supports two CR8 core blades. This blade is used for intra-chassis switching as well as ICL connectivity to another DC Director chassis. The 4/256 Director does not support core blades. Port blade compatibility Table 54 identifies which port blades are supported for each Director.
Table 56 lists chassis configuration options and resulting slot configurations. Table 56 Chassis configuration options Option Result 1 One 128-port switch (Blade IDs 4, 17 on slots 1–4, 7–10. Blade ID 5 and 16 on slots 5, 6) 5 One 384-port switch (Blade IDs 4, 17, 18, 31, and 36 on slots 1–4, 7–10. Blade ID 16 on slots 5, 6) See Table 53 for details about the different blades, including their corresponding IDs. Obtaining slot information To display the status of all slots in the chassis: 1.
204 Configuring Directors
9 Routing traffic This chapter provides information on routing policies. About data routing and routing policies Data moves through a fabric from switch to switch and from storage to server along one or more paths that make up a route. Routing policies determine the correct path for each frame of data. Whatever routing policy a switch is using applies to the VE_Ports as well. See ”Using the FC-FC routing service” on page 215 for details about VE_Ports.
option 1, an error message is returned because you cannot change the routing policy. See the Fabric OS Command Reference for more details on the aptPolicy command. You must disable the switch before changing the routing policy, and re-enable it afterward.
To force in-order frame delivery across topology changes: 1. Connect to the switch and log in as admin. 2. Enter the iodSet command at the command line. NOTE: This command can cause a delay in the establishment of a new path when a topology change occurs; use it with care. 3. To confirm the in-order delivery has been set, issue the iodShow command. To restore out-of-order frame delivery across topology changes: 1. Connect to the switch and log in as admin. 2. Enter the iodSet command at the command line.
Viewing routing path information The topologyShow and uRouteShow commands provide information about the routing path. 1. Connect to the switch and log in as admin. 2. Enter the topologyShow command to display the fabric topology, as it appears to the local switch: switch:admin> topologyshow 4 domains in the fabric; Local Domain ID: 2 Domain: Metric: Name: Path Count: 1 10500 fcr_xd_1_1 1 Hops: Out Port: In Ports: Total Bandwidth: Bandwidth Demand: Flags: switch:admin> 2 39 35 56 4.
3.
Viewing routing information along a path You can display detailed routing information from a source port (or area) on the local switch to a destination port (or area) on another switch. This routing information describes the full path that a data stream travels between these ports, including all intermediate switches. 1. Connect to the switch and log in as admin. 2. Enter the pathInfo command.
The information that pathInfo provides is: Hops The number of switch-to-switch links (ISLs) traversed. The local switch is hop 0. In Port The port that the frames come in from on this path. For hop 0, the source port. Domain ID The Domain ID of the switch. Name The name of the switch. Out Port The output port that the frames use to reach the next hop on this path. For the last hop, the destination port. BW The bandwidth of the output ISL, in Gbps. It does not apply to the embedded port.
214 Routing traffic
10 Using the FC-FC routing service Supported platforms FC-FC Routing is supported on the following platforms: • 400 MP Router • 4/256 SAN Director or DC SAN Backbone Director (short name, DC Director) when it is configured with an FR4-18i blade and uses chassis configuration option 5 NOTE: The DC Director only supports chassis configuration option 5.
Figure 8 shows a metaSAN consisting of three edge fabrics connected through a 4/256 SAN Director or DC Director containing an FR4-18i with interfabric links.
VE_Port Edge fabric 2 IP cloud Edge fabric 1 Edge fabric 3 E_Port E_Port IFL IFL IFL VEX_Port 400 MP Router EX_Port (2) = LSAN Backbone fabric 26416a Figure 9 A metaSAN with edge-to-edge and backbone fabrics Figure 9 shows a metaSAN with a backbone consisting of one 400 MP Router connecting hosts in Edge Fabrics 1 and 3 with storage in Edge Fabric 2 and the backbone through the use of LSANs.
If an FR4-18i blade is attached to an edge fabric using an EX_Port, it will create translate phantom domains in the fabric corresponding to the imported edge fabrics with active LSANs defined. If you import devices into the backbone fabric, then a translate phantom domain is created in the backbone device in addition to the one in the edge fabric.
Proxy host (imported device) Host Proxy target (imported device) Target Fabric 1 Fabric 2 E_Port IFL E_Port EX_Port IFL 400 MP Router Figure 11 MetaSAN with imported devices Routing types • Edge-to-Edge Occurs when devices in one edge fabric communicate with devices in another edge fabric through one or more Fibre Channel routers. • Backbone-to-Edge Occurs when Fibre Channel routers connect to a common fabric—known as a backbone fabric—through E_Ports.
Fibre Channel fabrics require that all ports be identified by a unique PID. In a single fabric, FC protocol guarantees that Domain IDs are unique, and so a PID formed by a Domain ID and area ID is unique within a fabric. However, the Domain IDs and PIDs in one fabric may be duplicated within another fabric, just as IP addresses are unique to one private network are likely to be duplicated within another private network.
Performing verification checks Before configuring a fabric to connect to another fabric, you must perform the following verification checks on the switch or director. To perform verification checks: 1. Log in to the switch or director as admin and enter the version command. Verify that Fabric OS 6.0 is installed on the 400 MP Router, 4/256 SAN Director or DC Director with the FR4-18i blade as shown in the following example. switch:admin_06> version Kernel: 2.4.19 Fabric OS: v6.
4. Enter the interopMode command and verify that Brocade switch interoperability with switches from other manufacturers is disabled. switch:admin> interopmode InteropMode: Off Usage: InteropMode 0|1 0: to turn it off 1: to turn it on 5. Enter the msPlatShow command to verify that Management Server Platform database is disabled in the backbone fabric. switch:admin_06> msplatshow *MS Platform Management Service is NOT enabled.
To assign backbone fabric IDs: 1. Log in to the switch or director. 2. Enter the fosConfig --disable fcr command to disable the FC-FC Routing Service. See the Fabric OS Command Reference or the CLI man pages for more information about the fosConfig command. NOTE: The default state for the FCR is disabled. The fcrEnable and fcrDisable commands continue to operate as before in Fabric OS versions 5.2.0 and earlier.
fabrics. Secure Fabric OS is an optional licensed product that provides customizable security restrictions through local and remote management channels on an HP fabric. Although Secure Fabric OS is not supported in Fabric OS 6.0, you can still connect a 6.0 switch to an edge switch that participates in a Secure Fabric OS. The FC-FC Routing Service uses only the DH-CHAP shared secrets to provide switch-to-switch authentication when connecting to a Secure Fabric OS fabric.
5. When prompted, type y. The DH-CHAP secret is now stored in the secret word database and is ready for use. switch:admin> secauthsecret --set This command is used to set up secret keys for the DH-CHAP authentication. The minimum length of a secret key is 8 characters and maximum 40 characters. Setting up secret keys does not initiate DH-CHAP authentication. If switch is configured to do DH-CHAP, it is performed whenever a port or a switch is enabled.
To configure an IFL for both edge and backbone connections: 1. On the 400 MP Router, or 4/256 SAN Director or DC Director with an FR4-18i blade, disable the port that you are configuring as an EX_Port (the one connected to the Brocade switch) by issuing the portDisable command. switch:admin> portdisable 7/10 You can verify that port 7 has been disabled by issuing the portShow command for the port. 2. Configure each port that connects to an edge fabric as an EX_Port or VEX_Port.
portCfgExport options This port can now connect to another switch. The following list describes the options for the portCfgExport command. For more information about the portCfgExport and portCfgVexport commands, see the Fabric OS Command Reference. -a -f -r -e -d Sets the EX_Port to enabled (1) or disabled (2). Admin use only. Sets the fabric ID (1 to 128). Each edge fabric must have a unique ID, and EX_Ports (or VEX_Ports) connected to the same edge fabric must have the same fabric ID.
4. Enter the portCfgShow command to view ports that are persistently disabled. switch:admin> portcfgshow Area Number: Speed Level: Trunk Port Long Distance VC Link Init Locked L_Port Locked G_Port Disabled E_Port ISL R_RDY Mode RSCN Suppressed Persistent Disable NPIV capability EX Port Mirror Port FC Fastwrite 7/10 74 AUTO OFF OFF OFF OFF OFF OFF OFF OFF OFF ON ON ON ON 5.
6.
Proc_rqrd: Timed_out: Rx_flushed: Tx_unavail: Free_buffer: Overrun: Suspended: Parity_err: 2_parity_err: CMI_bus_err: 0 0 0 0 0 0 0 0 0 0 Protocol_err: Invalid_word: Invalid_crc: Delim_err: Address_err: Lr_in: Lr_out: Ols_in: Ols_out: 0 0 0 0 0 0 0 0 0 Port part of other ADs: No 7. Enter the switchShow command to verify the EX_Port (or VEX_Port), edge fabric ID, and name of the edge fabric switch (containing the E_Port or VE_Port). 8.
The FCR router port cost settings are 0, 1000, or 10,000. If the cost is set to 0, the default cost will be used for that IFL. The FC router port cost is persistent and is saved in the existing port configuration file. Router port cost is passed to other routers in the same backbone. Link costs from the front domain to the translate (xlate) domain remain at 10,000. You can use the lsDbShow from the edge fabric to display these link costs.
Port cost considerations The router port cost has the following considerations: • Router port sets are defined as follows: • 0-7 and FCIP Tunnel 16-23 • 8-15 and FCIP Tunnel 24-31 More than two router port sets can exist in a 4/256 SAN Director or DC Director with two FR4-18i blades. • The router port cost does not help distinguish one IFL (or EX_ and VEX_Port link) from another, if all the IFLs are connected to the same port set.
400 MP Router or 4/256 SAN Director or DC Director with an FR4-18i blade, use the portCfgEXPort command. If you want to change the fabric parameters of a VEX_Port, then use the portCfgVEXPort command. The PID mode for the backbone fabric PID mode and the edge fabric PID mode do not need to match, but the PID mode for the EX_Port or VEX_Port and the edge fabric to which it is attached must match. You can statically set the PID mode for the fabric by using the -p option with the portCfgEXPort command.
Supported configurations and platforms The EX_Port trunking is an FCR software feature and requires that you have a trunking license installed on the FCR switch and on the edge fabric connected to the other side of the trunked EX_Ports. EX_Port trunking is supported only with edge fabrics. You can use EX_Port frame trunking in the following configurations and cases: • Ports with speeds of 2 Gbps up to a maximum speed of 4 Gbps and trunking over long distance.
through these ports may be disrupted for a short period of time. In addition to the commands for enabling and disabling trunking, you can also use the following E_Port commands for administering EX_Port Frame Trunking: • Use portCfgSpeed and switchCfgSpeed to set speed for a port or switch. • Display lists of trunks and members of trunks with the trunkShow command. • Use trunkDebug to list link characteristics for troubleshooting.
address authority (NAA) field in the WWN to detect an FC router. LSAN zone enforcement in the local fabric occurs only if the administration domain member list contains both of the devices (local and imported device) specified in the LSAN zone. For more information, see ”Managing administrative domains” on page 143. Defining and naming zones Zones are defined locally on a switch or director.
• Target B has WWN 50:05:07:61:00:49:20:b4 (connected to switch2). The following procedure shows how to control device communication with the LSAN. To control device communication with the LSAN: 1. Log in as admin and connect to switch1. 2. Enter the nsShow command to list the WWN of the host (10:00:00:00:c9:2b:c9:0c). NOTE: The nsShow output displays both the port WWN and node WWN; the port WWN must be used for LSANs.
9. Enter the cfgShow command to verify that the zones are correct. switch:admin> cfgshow Defined configuration: zone: lsan_zone_fabric2 10:00:00:00:c9:2b:c9:0c; 50:05:07:61:00:5b:62:ed; 50:05:07:61:00:49:20:b4 Effective configuration: no configuration in effect 10. Enter the cfgAdd and cfgEnable commands to create and enable the LSAN configuration. switch:admin> cfgadd "zone_cfg", "lsan_zone_fabric2" switch:admin> cfgenable "zone_cfg" You are about to enable a new zoning configuration.
LSAN zone binding (optional) By default, the Fibre Channel routers (FCR) in the backbone maintain the entire LSAN zone and device state database. On Fibre Channel routers with Fabric OS 5.3.0 and later, the LSAN zone binding allows you to specify pairs of edge fabrics that share devices, effectively creating an LSAN fabric matrix.
--cancel --display --fabricview --verify --quickmode Clears the information from the cache and put it back to the saved value. Displays the information that is saved in the cache. Displays the static and default and dynamic binding of the backbone to show which edge fabrics can access each other. Verifies if the information in the cache is valid and will not disrupt existing import/export devices. Runs a quick mode to derive the LSAN Zone matrix from the current import/export database.
The fcrlsancount command assumes that all the FCRs in the same LSAN fabric matrix or backbone have the same maximum LSAN count defined, to protect all the FCRs from running into indefinite state. Asymmetric LSAN configurations due to different maximum LSAN counts could lead to different devices being imported on different FCRs.
In the FC router, use the command fcrbcastconfig to prevent interfabric forwarding of broadcast frames of edge or backbone fabrics. Using the fcrbcastconfig command, you can disable or enable the broadcast frame forwarding option per FID (edge fabric or backbone fabric). If you have an FID with a pre-existing IPFC data session that you want to disable then the IPFC traffic across the FCR may not stop even after disabling the broadcasting to some edge fabrics.
2. Type the following command: fcr:admin> fcrbcastconfig --disable -f where is the specified FID where you want to disable frame forwarding. This command disables the broadcast frame forwarding option for an FID (edge or backbone fabric). Monitoring resources It is possible to exhaust resources, such as proxy PIDs. Whenever a resource is exhausted, Fabric OS generates an error message. The messages are described in the Fabric OS Message Reference.
The following example shows the use of the fcrResourceShow command display per physical port (EX_Port) resources.
To check for Fibre Channel connectivity problems: 1. On the edge Fabric OS switch, make sure that the source and destination devices are properly configured in the LSAN zone before entering the fcPing command. This command performs the following functions: • Checks the zoning configuration for the two ports specified. • Generates an ELS (extended link service) ECHO request to the source port specified and validates the response.
For the exact RASLog message descriptions, see the following RASLogs: FCR_1055, FCR_1056, and FCR_1073. For further information on these messages, refer to Fabric OS Message Reference. Backward compatibility In a fabric with Secure Fabric OS enabled, the edge fabric must have Fabric OS 3.2, 4.4.0, or later because only DH-CHAP authentication is supported. For a nonsecure fabric, the hardware and firmware compatibility is described in Table 57.
The portCfgExport command has additional options to verify the front Domain ID. The portCfgExport –d option is changed to enforce use of the same front Domain ID for the EX_Ports connected to the same edge fabric. The portCfgExport display results remain the same. For more information about the portCfgExport -d option, see ”portCfgExport options” on page 226 and the command details in the Fabric OS Command Reference. The following example illustrates the use of the portcfgexport command.
To display the range of output ports connected to the xlate domains: 1. Log in to the FC router. 2. Enter the lsDbShow command on the edge fabric. The following example shows the range of output ports. linkCnt = 2, flags = 0x0 LinkId = 53, out port = 1, rem port = 35, cost = 500, costCnt = 0, type = 1 LinkId = 57, out port = 129, rem port = 18, cost = 500, costCnt = 0, type = 1 The following example also shows the use of the lsDbShow display on the edge fabric.
Interoperating with an M-EOS fabric IMPORTANT: Interoperating with an M-EOS fabric is not supported at the time of the release of this document. Please check with your sales representative or http://www.hp.com regarding HP support of the interoperability features. This section covers how to set up your B-Series SAN and M-Series SAN to route traffic without merging the two SANs. If you want to merge the SANs or use SANtegrity, refer to ”Implementing an interoperable fabric” on page 481 for more information.
The Fibre Channel routing feature for M-EOS interoperability is not a licensed feature. Table 59 Brocade-McDATA M-EOSn interoperability compatibility matrix1 Fabric OS Versions of M-EOSn (i10k) 9.2.0 9.6.2 v5.3.0 Yes No 6.0 No Yes 1. Both Open and McDATA Fabric modes are supported. Connected SANs provide additional functionality not possible with segregated SANs. Some of these functions are as follows: • Island consolidation—Uses the Fabric OS 6.
data (RNID) to obtain the information. If the command to get the switch name is successful, the RNID request is not tried and the switch name is obtained. See the following example: switch: admin> switchshow|grep EX 44 3 12 042c00 id N2 Online EX-Port 10:00:08:00:88:2c:c2:00 "McDATA:10.32.68.146" (fabric id = 12 ) 46 3 14 042e00 id N2 Online EX-Port 10:00:00:60:69:e2:18:b6 "b24000_5x_1" (fabric id = 23 )(Trunk master) switch: admin> fcrfabricshow FC router WWN: 10:00:00:60:69:80:04:0a, Dom ID: 4, Info: 10.
Connectivity modes You can connect to M-EOS fabrics in both McDATA Open mode or McDATA Fabric mode. If the mode is not configured correctly, the port is disabled because of incompatibility. To allow interconnectivity with M-EOS SANs, the command line interface (CLI) command portCfgExPort uses the -m option to indicate the connectivity mode. Table 60 lists the valid parameters to use with the -m option to set the connectivity mode.
3. On the 400 MP Router and 4/256 SAN Director or DC Director with an FR4-18i blade, use the portDisable to disable the EX_Port that you will use to connect to the M-Series switch. Ports are persistently disabled by default. Switch:admin_06> portdisable 10/13 Switch:admin_06> switchshow switchName: b48000_5x_1/PORT/P switchType: 42.
5. Enable the port by issuing the portEnable command. switch>:admin_06> portenable 10/13 If the port was persistently disabled, use the following command to enable the port: switch:admin_06> portcfgpersistentenable 10/13 • Connect IFL1 and verify EX_PORT connectivity. Repeat for all Brocade fabric IFLs. • Connect IFL (n) for the M-EOS fabric and verify EX_PORT connectivity. Repeat for all M-EOS fabric IFLs. 6.
For information about Brocade edge fabric setup on E_Ports and interswitch linking, see ”Administering ISL Trunking” on page 129. For information on EX_Port Frame trunking setup on the FCR switch, see ”Using EX_Port frame trunking” on page 234. 8. Capture a SAN profile of the McDATA and Brocade SANs, identifying the number of devices in each SAN.
Figure 13 SAN Pilot and EFCM Zone screens NOTE: The screens provided in this section are for illustrative purposes only. Depending on the M-EOS firmware release you are using, the M-EOS web-based management tool may display a user interface different from those shown. 4. Type the desired name in the Zone Name field, using the LSAN_xxxx naming schema. In EFCM, move to the Zone Name field, and enter the desired name using the LSAN_xxxx naming schema. 5.
6. To add devices that are connected to the Brocade fabric, click Edit in the Pending Zone set. 7. On the Modify Zone tab, enter the device WWN into the World Wide Name field and click Add. The Pending Zone Membership List is updated with the new Zone members. If you are using EFCM, select Potential Zone Members > New Member, enter the WWN port name, and click Add. NOTE: The procedures described in this section were current when the document was written, but may have changed since then.
Figure 14 Adding a zone set name in SAN Pilot Regardless of the method used, you should now verify that the new zone set containing your LSAN has been added. Alternately, use the following procedure: 1. Create the LSAN, using the LSAN_xxxx naming schema. 2. Append the newly-created zone set to a currently active zone set. 3. Activate the updated zone set. LSAN zoning with M-EOS An LSAN is defined by a zone in an edge fabric.
5. Move back to the 400 MP Router or 4/256 SAN Director or DC Director with an FR4-18i blade and issue the fcrProxyDevShow command on to verify that the devices are configured and exported.
6. Log in to the Brocade edge fabric switch and issue the nsAllShow or the nsCamShow command.
All of the devices from both LSANs should appear in the output. If the devices do not appear in the output, issue the cfgShow command to verify your zone configuration. Use the cfgactvshow command to display the zone configuration currently in effect. The following example illustrates the use of cfgactvshow. switch:admin> cfgactvshow Effective configuration: cfg:test zone:lsan_san 10:00:00:00:00:03:00:00 10:00:00:00:00:01:00:00 zone:lsan_test 50:06:01:60:38:e0:0b:a4 10:00:00:00:c9:44:54:04 7.
. Figure 16 Configuration during the upgrade The switch Domain ID and BB fabric ID of the new FC router can be identical. Once the metaSAN is stable, EX_Ports on the new router are ‘active’, the old router can be taken out of the setup. Redundant configuration The configuration shown in Figure 17 on page 261, shows that old routers can be removed one by one. For example, FC router 2 can be replaced with the new FC router.
Figure 18 Dual backbone fabric configuration Devices directly connected to router In the Multi-protocol Router, end devices are allowed to be directly connected, but these devices cannot be imported to other edge fabrics (using LSAN zones). During the upgrade process, these devices will face disruption unless there is redundancy support provided from the device end. 400 MP Router allows the end devices to be imported to edge fabrics. To configure the new router: 1. Log in to the new router as admin. 2.
264 Using the FC-FC routing service
11 Administering FICON fabrics This chapter provides procedures for managing FICON fabrics. Overview of Fabric OS support for FICON IBM Fibre Connection (FICON®) is an industry-standard, high-speed input/output (I/O) interface for mainframe connections to storage devices. Fabric OS supports intermix mode operations, in which FICON and Fibre Channel technology work together.
Supported switches FICON protocol is supported on the following HP StorageWorks models: DC SAN Backbone Director, short name, DC Director (FC8-16, FC8-32 port blades, FR4-18i FCIP blade and FC10-6 10 Gbit/sec port blade for ISL connections), the 4/256 SAN Director (FC4-16, FC4-32 port blades, FR4-18i FCIP blade and FC10-6 10 Gbit/sec port blade for ISL connections), SAN Switch 4/32, 4/64 SAN Switch and SAN Switch 4/32B switches.
• The FR4-18i routing blade must not be inserted in slot 10 of the chassis. (Other blades are supported in slot 10, but the FR4-18i blade is not.) FICON channels and control units can be attached only to the FC ports on this blade. This blade is advertised to the mainframe as a 16-port blade. If you have an FC4-18i blade in slot 10 in your director, the 16 virtual ports are disabled when you enable fmsmode.
For information on these tools, see: • Web Tools—Web Tools Administrator’s Guide • Fabric Manager—Fabric Manager Administrator’s Guide • SNMP Agent and FICON Management Information Base (MIB)—Fabric OS MIB Reference User security considerations To administer FICON, you must have one of the following roles: • Admin • Operator • SwitchAdmin • FabricAdmin The User and BasicSwitchAdmin roles are view-only. The ZoneAdmin and SecurityAdmin roles have no access.
Preparing a switch To verify and prepare a switch for use in a FICON environment, complete the following steps: 1. Connect to the switch and log in as admin. 2. Enter the switchShow command to verify that the switch and devices are online. 3. Change the routing policy on the switch from the default exchange-based policy to the required port-based policy for those switches with FICON devices directly attached using the aptPolicy command when working from the command line.
Figure 19 and Figure 20 show two viable cascaded configurations. These configurations require Channel A to be configured for two-byte addressing and require IDID and fabric binding. It is recommended that there are only 2 domains in a path from a FICON Channel interface to a FICON Control Unit interface.
8. Enter the switchEnable command to re-enable the switch. switch:admin> configure Configure... Fabric parameters (yes, y, no, n): [no] yes Domain: (1..239) [3] 5 R_A_TOV: (4000..120000) [10000] E_D_TOV: (1000..5000) [2000] Data field size: (256..2112) [2112] Sequence Level Switching: (0..1) [0] Disable Device Probing: (0..1) [0] Suppress Class F Traffic: (0..1) [0] VC Encoded Address Mode: (0..1) [0] Per-frame Route Priority: (0..1) [0] Long Distance Fabric: (0..1) [0] BB credit: (1..
Swapping ports If a port malfunctions, or if you want to connect to different devices without having to re-wire your infrastructure, you can move a port’s traffic to another port (swap ports) without changing the I/O Configuration Data Set (IOCDS) on the mainframe computer. To swap ports: 1. Connect to the switch and log in as admin. 2. Enter the portSwapEnable command (to enable the command for port swapping). 3. Enter the portDisable command to disable the two ports to be swapped. 4.
Setup summary To set up FICON CUP, use the following procedure and be sure to perform the steps in the order indicated. 1. For directors with at least 256 ports installed, use the PortDisable command to disable (block) ports 254 and 255. Ports 254 and 255 are not supported in a CUP environment. After fmsmode has been successfully enabled, these two ports remain disabled and cannot be used either as an F_Port or an E_Port.
• Advanced Zoning, if used, continues to be in force. If there are any differences in restrictions set up with Advanced Zoning and PDCM, the most restrictive rules are automatically applied. • RSCNs are sent to devices if PDCM results in changes to connectivity between a set of ports. Changing fmsmode from enabled to disabled triggers the following events: • A device reset is performed on the control device. • PDCM is no longer enforced.
Displaying mode register bit settings The mode register bits are described in Table 62. Table 62 FICON CUP mode register bits POSC Programmed offline state control. When this bit is set on, the host is prevented from taking the switch offline. The default setting is 1 (on). UAM User alert mode. When this bit is set on, a warning is issued when an action is attempted that will write CUP parameters on the switch. The default setting is 0 (off). ASM Active=saved mode.
Setting mode register bits Use the ficoncupset modereg command to set the FICON CUP mode register bits for the local switch. Consider the following when changing mode register bits: • As required by the CUP protocol, the UAM bit cannot be changed using this command. • All mode register bits except UAM are saved across power on/off cycles; the UAM bit is reset to 0 following a power-on. • Mode register bits can be changed when the switch is offline or online.
Port and switch naming standards Fabric OS handles differences in port and switch naming rules between CUP and itself as follows: • CUP employs 8-bit characters in port address names and switch names; Fabric OS employs 7-bit characters. When fmsmode is enabled, all characters greater than 0x40 and not equal to 0xFF (EBCIDC code page 37 [0x25]) are allowed in the name; therefore, it is possible for a channel to set a name with nonprintable characters.
Troubleshooting The following sources provide useful problem-solving information: • The standard support commands (portLogDump, supportSave, supportShow) or the Fabric Manager Event Log. By default, the FICON group in the supportShow output is disabled. To enable the capture of FICON data in the supportShow output, enter the supportshowcfgenable ficon command.
Backing up and restoring FICON configuration files The FICON file access facility is used to store configuration files. This includes IPL and other configuration files. The Fabric OS saves the IPL and all other configuration files on the switch. A maximum of 16 configuration files, including the IPL file, are supported. You can upload the configuration files saved on the switch to a management workstation using the configUpload command.
Recording configuration information You can use the following worksheet for recording FICON configuration information.
Sample IOCP configuration file The channel subsystem controls communication between a configured channel, the control unit, and the device. The I/O Configuration Dataset (IOCDS) defines the channels, control units, and devices to the designated logical partitions (LPARs) within the server; this is defined using the Input/Output Configuration Program (IOCP). The IOCP statements are typically built using the hardware configuration dialog (HCD).
280 Administering FICON fabrics
12 Configuring the Distributed Management Server This chapter provides information on enabling and disabling the platform services, configuring and controlling access to the Management Server database, and using the topology discovery feature. Introduction The Fabric OS Distributed Management Server allows a SAN management application to retrieve information and administer interconnected switches, servers, and storage devices.
3. Enter y to confirm the deactivation. switch:admin> msplmgmtdeactivate MS Platform Service is currently enabled. This will erase MS Platform Service configuration information as well as database in the entire fabric. Would you like to continue this operation? (yes, y, no, n): [no] y Request to deactivate MS Platform Service in progress......
4. Enter the WWN of the host to be added to the ACL. 5. At the prompt, enter 1 to display the access list so you can verify that the WWN you entered was added to the ACL. 6. After verifying that the WWN was added correctly, enter 0 at the prompt to end the session. 7. At the “Update the FLASH?” prompt, enter y. 8. Press Enter to update the nonvolatile memory and end the session.
To delete a member from the ACL: 1. Connect to the switch and log in as admin. 2. Enter the msConfigure command. The command becomes interactive. 3. At the select prompt, enter 3 to delete a member based on its port/node WWN. 4. At the prompt, enter the WWN of the member to be deleted from the ACL. 5. At the prompt, enter 1 to display the access list so you can verify that the WWN you entered was deleted from the ACL. 6.
Configuring the server database The Management Server database can be viewed or cleared. The command msPlClearDB is allowed only in AD0 and AD255. To view the contents of the Management Server database: 1. Connect to the switch and log in as admin. 2. Enter the msPlatShow command. The contents of the Management Server platform database are displayed. switch:admin> msplatshow -------------------------------------------------Platform Name: [9] "first obj" Platform Type: 5 : GATEWAY Number of Associated M.A.
Request to enable MS Topology Discovery Service in progress.... *MS Topology Discovery enabled locally. switch:admin> mstdenable ALL Request to enable MS Topology Discovery Service in progress.... *MS Topology Discovery enabled locally. *MS Topology Discovery Enable Operation Complete!! To disable topology discovery: 1. Connect to the switch and log in as admin. 2. Enter the appropriate following command based on how you want to disable discovery: • For the local switch, enter the mstdDisable command.
13 Working with Diagnostic Features This chapter provides information on diagnostics and how to display system, port, and specific hardware information. It also describes how to set up system logging mapping (syslogd) and how to set up offloading error messages (supportSave). About Fabric OS diagnostics The purpose of the diagnostic subsystem is to evaluate the integrity of the system hardware.
Press escape within 4 seconds to enter boot interface. Booting "Fabric Operating System" image. Linux/PPC load: BootROM command line: quiet Uncompressing Linux...done. Now booting the kernel Attempting to find a root file system on hda2... modprobe: modprobe: Can't open dependencies file /lib/modules/2.4.19/modules.dep (No such file or directory) INIT: version 2.78 booting INIT: Entering runlevel: 3 eth0: Link status change: Link Up. 100 Mbps Full duplex Auto (autonegotiation complete).
POST2: Running diagshow POST2: Script PASSED with exit status of 0 Thu Mar 31 20:13:12 GMT 2005 took (0:0:17) 2005/03/31-20:13:13, [BL-1000], 221,, INFO, Paulsa45, Initializing Ports... Enabling switch... 2005/03/31-20:13:13, [BL-1001], 222,, INFO, Paulsa45, Port Initialization Completed 2005/03/31-20:13:13, [EM-5012], 0,, INFO, SW4100_P45, EM: sent dumpready to ME., em.c, line: 2152 2005/03/31-20:13:13, [DGD-5002], 0,, INFO, SW4100_P45, Slot 0 has passed the POST tests., main.
To display switch information: 1. Connect to the switch and log in as admin. 2. Enter the switchShow command, which displays the following information for a switch: • switchname—The switch name. • switchtype—The switch model and firmware version numbers. • switchstate—The switch state: Online, Offline, Testing, or Faulty. • switchrole—The switch role: Principal, Subordinate, or Disabled. • switchdomain—The switch Domain ID. • switchid—The embedded port D_ID of the switch.
Viewing port information Use the following commands to view information about ports. To view the status of a port: 1. Connect to the switch and log in as admin. 2. Enter the portShow command, specifying the number that corresponds to the port you are troubleshooting. In this example, the status of port two is shown: switch:admin> switch:user> portshow 0 portName: portHealth: OFFLINE Authentication: None portDisableReason: None portCFlags: 0x1 portFlags: 0x4001 PRESENT U_PORT LED portType: 4.
To display the port statistics: 1. Connect to the switch and log in as admin. 2. At the command line, enter the portStatsShow command. Port statistics include information such as the number of frames received, number of frames sent, number of encoding errors received, and number of class 2 and class 3 frames received. See the Fabric OS Command Reference for additional portStatsShow command information, such as the syntax for slot or port numbering.
To display a summary of port errors for a switch: 1. Connect to the switch and log in as admin. 2. Enter the portErrShow command. See the Fabric OS Command Reference for additional portErrShow command information. switch:admin> porterrshow frames enc crc too too bad enc disc link loss loss frjt fbsy tx rx in err shrt long eof out c3 fail sync sig sig===================================================================== 0: 22 24 0 0 0 0 0 1.5m 0 7 3 0 0 0 1: 22 24 0 0 0 0 0 1.
The portErrShow command output provides one output line per port. See Table 64 for a description of the error types.
Viewing equipment status You can display status for fans, power supply, and temperature. NOTE: The number of fans, power supplies, and temperature sensors depends on the switch type. For detailed specifications on these components, refer to the switch hardware reference manual. The specific output from the status commands varies depending on the switch type. To display the status of the fans: 1. Connect to the switch and log in as admin. 2.
2. Enter the tempShow command: switch:admin> tempshow Index Status Centigrade Fahrenheit ---------------------------------------------------1 OK 21 70 2 OK 22 72 3 OK 29 84 4 OK 24 75 5 OK 25 77 switch:admin> Information displays for each temperature sensor in the switch. The possible temperature status values are: • OK—Temperature is within acceptable range. • FAIL—Temperature is outside of acceptable range. Damage might occur.
2. Enter the portLogShow command: switch:admin> portlogshow 12 time task event port cmd args ------------------------------------------------Thu Apr 14 12:07:09 2005 12:07:09.350 PORT Rx 0 02fffffd,00fffffd,0608ffff,14000000 12:07:09.350 0 0 12:07:10.812 PORT Tx 0 02fffffd,00fffffd,07feffff,14000000 40 12:07:10.813 PORT 0 0 12:07:19.492 PORT Tx 4 02fffffd,00fffffd,0800ffff,14000000 40 12:07:19.492 PORT Tx 22 02fffffd,00fffffd,0802ffff,14000000 40 12:07:19.
Because a portLogDump output is long, a truncated example is presented: switch:admin> portlogdump task event port cmd args ------------------------------------------------16:30:41.780 PORT Rx 9 40 02fffffd,00fffffd,0061ffff,14000000 16:30:41.780 PORT Tx 9 0 c0fffffd,00fffffd,0061030f 16:30:42.503 PORT Tx 9 40 02fffffd,00fffffd,0310ffff,14000000 16:30:42.505 PORT Rx 9 0 c0fffffd,00fffffd,03100062 16:31:00.464 PORT Rx 9 20 02fffc01,00fffca0,0063ffff,01000000 16:31:00.
In this example, Fabric OS messages map to local7 facility level 7 in the /etc/syslog.conf file: local7.emerg /var/adm/swcritical local7.alert /var/adm/alert7 local7.crit /var/adm/crit7 local7.err /var/adm/swerror local7.warning /var/adm/swwarning local7.notice /var/adm/notice7 local7.info /var/adm/swinfo local7.debug /var/adm/debug7 If you prefer to map Fabric OS severities to a different UNIX local7 facility level, see ”To set the facility level:” on page 299.
To remove a syslogd host from the list: 1. Connect to the switch and log in as admin. 2. Enter the syslogDipRemove command: switch:admin> syslogdipremove 10.1.2.1 3. Verify the IP address was deleted using the syslogDipShow command. Viewing and saving diagnostic information Enter the supportShow command to dump important diagnostic and status information to the session screen, where you can review it or capture its data.
4. Respond to the prompts as follows: Host Name Enter the name or IP address of the server where the file is to be stored; for example, 1080::8:800:200C:417A for a server configured for IPv6. User name Enter the user name of your account on the server; for example, “JohnDoe”. Password Enter your account password for the server. Remote Specify a path name for the remote directory. Absolute path names can be directory specified by starting the path name with a forward slash (/).
302 Working with Diagnostic Features
14 Troubleshooting This chapter provides information on troubleshooting and the most common procedures to use to diagnose and recover from problems. It also includes specific troubleshooting scenarios as examples. About troubleshooting Troubleshooting should begin at the center of the SAN—the fabric. Because switches are located between the hosts and storage devices and have visibility into both sides of the storage network, starting with them can help narrow the search path.
Table 67 Common troubleshooting problems and tools (continued) Problem area Investigate Tools Hosts • Downlevel HBA firmware • Host operating system diagnostic tools • Incorrect device driver installation • Incorrect device driver configuration • Device driver diagnostic tools • Switch commands (for example, switchShow or nsAllShow) for diagnostics Also, make sure you use the latest HBA firmware recommended by HP or on the HBA supplier's web site Storage Management Applications • Incorrect instal
• How large is the fabric? • Is it a secure fabric? • Is the fabric redundant? 6. Run the supportSave command on both CPs if it is a director class product, for example 4/256 SAN Director or DC SAN Backbone Director (short name, DC Director). 7. Document the sequence of events by answering the following questions: • What happened prior to the problem? • Is the problem reproducible? • If so, what are the steps to produce the problem? • What configuration was in place when the problem occurred? 8.
Analyzing connection problems If a host is unable to detect its target (for example, a storage or tape device), you should begin troubleshooting the problem in the middle of the data path. Determine if the problem is above or below the starting point, then continue to divide the suspected problem path in half until you can pinpoint the problem. To check the logical connection: 1. Enter the switchShow command. 2.
Round-trip min/avg/max = 1012/1136/1442 usec Pinging 21:00:00:20:37:25:ad:05 [0x211e8] with 12 bytes of data: Request rejected Request rejected Request rejected Request rejected Request rejected 5 frames sent, 0 frames received, 5 frames rejected, 0 frames timeout Round-trip min/avg/max = 0/0/0 usec switch:admin> Following is sample output from the fcPing command in which one device accepts the request and another device does not respond to the request: switch:admin> fcping 0x020800 22:00:00:04:cf:75:63:85
To check the name server (NS): 1.
• If the device is listed in the NS, the problem is between the storage device and the host. There may be a zoning mismatch or a host/storage issue. Proceed to ”To check for zoning problems:” on page 309. 3. Enter the portLoginShow command to check the port login status. 4. Enter the fcpProbeShow command to display the FCP probing information for the devices attached to the specified F_Port or L_Port.
There are a number of settings that control the overall behavior and operation of the fabric. Some of these values, such as the Domain ID, are assigned automatically by the fabric and can differ from one switch to another in the fabric. Other parameters, such as the BB credit, can be changed for specific applications or operating environments, but must be the same among all switches to allow the formation of a fabric.
3. Compare the fabricShow output from the two fabrics. Note the number of Domain ID conflicts; there may be several duplicate Domain IDs that must be changed. Determine which switches have domain overlap and change the Domain IDs for each of those switches. 4. Choose the fabric on which to change the duplicate Domain ID; connect to the conflicting switch in that fabric. 5. Enter the switchDisable command. 6. Enter the switchEnable command.
See ”Administering Advanced Zoning” on page 403 for additional information about setting up zoning. Also, see the Fabric OS Command Reference for details about zoning commands. You can correct zone conflicts by using the cfgClear command to clear the zoning database. IMPORTANT: The cfgClear command is a disruptive procedure. To correct a fabric merge problem quickly: 1. Determine which switch(es) have the incorrect zoning configuration; then, log in to the switches as admin. 2.
Recognizing MQ-WRITE errors An MQ error is a message queue error. Identify an MQ error message by looking for the two letters M and Q in the error message: 2004/08/24-10:04:42, [MQ-1004], 218,, ERROR, ras007, mqRead, queue = raslog-teststring0123456-raslog, queue I D = 1, type = 2 MQ errors can result in devices dropping from the SNS or can prevent a switch from joining the fabric. MQ errors are rare and difficult to troubleshoot; resolve them by working with HP.
Correcting I2C bus errors I2C bus errors generally indicate defective hardware or poorly seated devices or blades; the specific item is listed in the error message. See the Fabric OS Command Reference for information specific to the error that was received. Some Chip-Port (CPT) and Environmental Monitor (EM) messages contain I2C-related information. If the I2C message does not indicate the specific hardware that may be failing, begin debugging the hardware, as this is the most likely cause.
Correcting device login issues Perform the following steps to try to pinpoint problems with device logins. 1. Log in to the switch as admin. 2. Enter the switchShow command; then, check for correct logins: switch:admin> switchshow switchName: Dazzler switchType: 26.
ISL R_RDY Mode .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. RSCN Suppressed .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Persistent Disable.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. NPIV capability ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON where AN:AutoNegotiate, ..:OFF, ??:INVALID, SN:Software controlled AutoNegotiation. 4.
8 Offline No_Light PRESENT U_PORT LED 9 Offline No_Light PRESENT U_PORT LED 10 Offline No_Module PRESENT U_PORT LED 11 Offline No_Module PRESENT U_PORT LED 12 Offline No_Module PRESENT U_PORT LED 13 Offline No_Module PRESENT U_PORT LED 14 Online In_Sync PRESENT ACTIVE F_PORT G_PORT U_PORT LOGICAL_ONLINE LOGIN NOELP LED ACCEPT 15 Online LOGIN LED In_Sync PRESENT ACTIVE E_PORT G_PORT U_PORT SEGMENTED 6.
Table 70 Component test descriptions Test name Operands Checks fporttest [-nframes count] [-ports itemlist] [-seed payload_pattern] [-width pattern_width] [-size pattern_size] Tests point-to-point path from the F_Port to the N_Port and back. Used to test online F_Port devices, N_Port devices, SFPs, and GBICs. loopporttest [-nframes count] [-ports itemlist][-seed payload_pattern] [-width pattern_width] Only tests components attached to a switch that are on an FC-AL.
See Table 71 for a list of additional tests that can be used to determine the switch components that are not functioning properly. See the Fabric OS Command Reference for additional command information. Table 71 Switch component tests Test Function portloopbacktest Performs a functional test of port N to N path. portregtest Performs a read and write test of the ASIC SRAMs and registers. spinsilk Performs a functional test of internal and external transmit and receive paths at full speed.
6. Correct the negotiation by entering the portCfgSpeed [slotnumber/]portnumber, speed_level command if the fields in step 5 do not appear. switch:admin> portcfgspeed Usage: portCfgSpeed PortNumber Speed_Level: Speed_Level 0 - Auto Negotiate 1 - 1Gbps 2 - 2Gbps 4 - 4Gbps To check for a loop initialization failure: 1. Verify the port is an L_Port. a. Enter the switchShow command. b. Check the comment field of the output to verify that the switch port indicates an L_Port.
Table 72 SwitchShow output and suggested action Output Suggested action Disabled Check the output from the switchShow command to determine whether the switch is disabled. If the port is disabled (for example, due to persistent disable or security reasons), attempt to resolve the issue and then enter the portEnable command. Bypassed Check the output from the switchShow command to determine whether the port is testing.
Table 73 Loopback modes Loopback mode Description 8 Back-end bypass & SERDES loopback 9 Back-end bypass & internal loopback 6. Check the results of the loopback test and proceed as follows: • If the loopback test failed, the port is bad. Replace the port blade. • If the loopback test did not fail, the SFP was bad. 7. Optionally, to rule out cabling issues: a. Insert a new cable in the suspected marginal port. b. Enter the portErrShow command to determine if a problem still exists.
Table 74 FTRACE configurable parameters Parameter Default Range Syntax Trace Mask 0x8000 0-0xFFFFFFFF Integer Trigger Mask 0x00000003 0-0xFFFFFFFF Integer After information is captured, you can use the portshow command to display FTRACE information on a GE port for a tunnel. You can save trace events can for future analysis.
Recognizing port initialization and FCP auto discovery process The steps in the port initialization process represent a protocol used to discover the type of connected device and establish the port type. The possible port types are as follows: • U_Port—Universal FC port. The base Fibre Channel port type and all unidentified, or uninitiated ports are listed as U_Ports. • FL_Port—Fabric Loop port. Connects both public and private loop devices. • G_Port—Generic port.
Supported hardware Port mirroring is supported on Condor-based ASIC platforms, including: • HP StorageWorks SAN Switch 4/32 and 4/32B • HP StorageWorks 4/64 SAN Switch • HP StorageWorks 400 MP Router • 4/256 SAN Director and DC Director with chassis option 5 Port mirroring can be used on the following blades within a chassis: • FC4-32 32-port blade • FC4-16 16-port blade • FC4-48 48-port blade • FC8-16 16-port blade • FC8-32 32-port blade • FC8-48 48-port blade • FC4-16IP iSCSI blade on FC ports only The FC
Port mirroring considerations Before creating port mirror connections, consider the following limitations: • A mirror port can be any port on the same switch as the source identifier port. • Only one domain can be mirrored per chip; after a domain is defined, only mirror ports on the defined domain can be used.
To delete a port mirror connection between two local switch ports or a local and a remote switch port: 1. Log in to the switch as admin. 2. Type portMirror --del SourceID DestID. For example, to delete the port mirror connection on mirror port 2, you might type: portMirror --del 0x011400 0x240400 To display port mirror connections: 1. Log in to the switch as admin. 2. Type portMirror --show.
12 12 -- N2 No_Module 13 13 -- N2 No_Module 14 14 id 15 15 id 328 Troubleshooting N2 N2 Online Online F-Port E-Port 21:00:00:e0:8b:12:8a:be segmented,(No Fabric License)
15 Administering NPIV This chapter describes the concepts and procedures for administering N-Port ID Virtualization (NPIV). About NPIV NPIV enables a single Fibre Channel protocol port to appear as multiple, distinct ports, providing separate port identification within the fabric for each operating system image behind the port (as if each operating system image had its own unique physical port). NPIV assigns a different virtual port ID to each Fibre Channel protocol device.
The following example shows the configuration of these parameters: switch:admin> switchdisable switch:admin> configure Configure... Fabric parameters (yes, y, no, n): [no] Virtual Channel parameters (yes, y, no, n): [no] F-Port login parameters (yes, y, no, n): [no] y Maximum logins per switch: (1..4032) [4032] 2048 Maximum logins per port: (1..255) [255] 126 . . .
output indicates whether or not a port is an NPIV F_Port, and identifies the number of virtual N_Ports behind it. Following is sample output from the switchShow command: switch: admin> switchshow switchName:swd77 switchType:32.
16 Optimizing fabric behavior This chapter describes the Adaptive Networking features. Introduction to adaptive networking Adaptive Networking is a suite of tools and capabilities that enable you to ensure optimized behavior in the SAN. Even under the worst congestion conditions, the Adaptive Networking features can maximize the fabric behavior and provide necessary bandwidth for high-priority, mission-critical applications and connections.
In Figure 21, all traffic entering Domain 1 from N_Port 8 is routed through E_Port 1. Similarly, traffic entering Domain 3 from E_Port 9 is routed to E_Port 12, and traffic entering Domain 4 from E_Port 7 is routed to the device through N_Port 6. Traffic coming from other ports in Domain 1 would not use E_Port 1, but would use E_Port 2 instead. Other traffic is excluded from the dedicated path as long as other equal-cost routes through the fabric exist.
In Figure 23, a dedicated path between Domain 1 and Domain 4 exists, but is not the shortest path. In this situation, if failover is enabled, the TI zone traffic uses the shortest path, even though the E_Ports are not in the TI zone. If failover is disabled, the TI zone traffic stops until the dedicated path is configured to be the shortest path.
Domain 1 8 9 Domain 3 1 9 2 10 11 8 12 7 6 = Dedicated path 5 = Ports in the TI zone Domain 4 Figure 24 TI zone misconfiguration Supported configurations for Traffic Isolation Note the following configuration rules for TI zones: • Traffic Isolation is supported only on the HP StorageWorks 4/8 SAN Switch, 4/16 SAN Switch, SAN Switch 4/32, 4/64 SAN Switch, SAN Switch 4/32B, and 400 Multi-protocol Router, 4/256 SAN Director and DC SAN Backbone Director (short name, DC Director), all configured in B
Limitations and restrictions of Traffic Isolation The following are limitations of TI zones: • A maximum of 255 TI zones can be created in one fabric. A fabric merge resulting in greater than 255 TI zones results in merge failure and the fabrics are segmented. • A TI zone can be created using D,I (Domain, Index) notation only. Use of WWNs is not allowed. • To include a trunk group in a TI zone, you must include all ports of the trunk in the TI zone.
Modifying TI zones Using the zone --add and zone --remove commands, you can add and remove ports and change the failover option of existing TI zones. If you remove the last member of a TI zone, the TI zone is deleted. To modify a TI zone: 1. Connect to the switch and log in as admin.
2. Enter the zone --add command to add ports or change the failover option for an existing TI zone. Enter the zone --remove command to remove ports from an existing TI zone. zone --add [-o optlist] name -p "portlist" zone --remove name -p "portlist" where: optlist A list of options for controlling failover mode. • Disable failover mode. • Enable failover mode. name The name of the zone to be modified. portlist The list of ports to be added to or removed from the TI zone.
Deleting a TI zone Use the zone --delete command to delete a TI zone from the defined configuration. This command deletes the entire zone; to only remove port members from a TI zone, use the zone --remove command, as described in ”Modifying TI zones” on page 337. To delete a TI zone: 1. Connect to the switch and log in as admin. 2. Enter the zone --delete command. zone --delete name where: name The name of the zone to be deleted.
To limit the traffic, you set the maximum speed at which the traffic can flow through a particular F_Port or FL_Port. For example, if you set the rate limit at 4 Gbps, then traffic from a particular device is limited to a maximum of 4 Gbps. Ingress rate limiting enforcement is needed only if the port can run at a speed higher than the rate limit. For example, if the rate limit is 4 Gbps and the port is only a 2 Gbps port, then ingress rate limiting is not enforced.
QoS zones You assign high or low priority (QoS level) using a QoS zone. A QoS zone is a special zone that indicates the priority of the traffic flow between a given host/target pair. The members of a QoS zone are WWNs of the host/target pairs. QoS zones can contain only WWN members. “Domain,Index” zoning is not supported. A QoS zone has a special name, to differentiate it from a regular zone.
QoS on E_Ports In addition to configuring the hosts and targets in a zone, you must also enable QoS on individual E_Ports that might carry traffic between the given host and target pairs. Path selection between the “host,target” pairs is governed by FSPF rules and is not affected by QoS priorities. By default, QoS is enabled on E_Ports in port configuration. For example, in Figure 26, QoS should be enabled on the encircled E_Ports.
• Traffic prioritization is not supported on mirrored ports. • Trunking considerations: If some ports in a trunk group have QoS enabled and some ports have QoS disabled, then two different trunks are formed, one with QoS enabled and one with QoS disabled. • Traffic prioritization is enforced on the egress ports only, not on the ingress ports. Setting traffic prioritization 1. Connect to the switch and log in as admin. 2.
17 Administering Advanced Performance Monitoring This chapter contains information about the Advanced Performance Monitoring licensed feature. About Advanced Performance Monitoring Based on Frame Filtering technology and a unique performance counter engine, Advanced Performance Monitoring is a comprehensive tool for monitoring the performance of networked storage resources.
Table 77 lists commands associated with Advanced Performance Monitoring. Advanced Performance Monitor commands are available only to users with the admin or switchAdmin roles. For detailed information on these commands, see the Fabric OS Command Reference. Table 77 Advanced Performance Monitoring commands Command Description perfAddEEMonitor Add an end-to-end monitor to a port. perfAddIPMonitor Add an IP monitor to a port. perfAddReadMonitor Add a SCSI Read monitor to a port.
Monitoring AL_PAs You can use the perfShowAlpaCrc command to display the CRC error count for all AL_PA devices or for a single AL_PA on a specific active L_Port. The following example displays the CRC error count for all AL_PA devices on a port: switch:admin> perfshowalpacrc 1/1 AL_PA CRC count -------------------0xd9 0 The following example displays the CRC error count for a single AL_PA device on a port: switch:admin> perfshowalpacrc 1/1, 0xd9 The CRC count at ALPA 0xd9 on port 1 is 0x000000000.
NOTE: For end-to-end monitors, CRC counters are not displayed on the 4/8 SAN Switch, 4/16 SAN Switch, SAN Switch 4/32, 4/64 SAN Switch, SAN Switch 4/32B, 400 Multi-protocol Router, 4/256 SAN Director, and DC Director switches. Adding end-to-end monitors An end-to-end monitor counts the following items for a port: number of words received, number of words transmitted, and number of CRC errors detected in frames. 4/8 SAN Switch and 4/16 SAN Switch models allow up to eight end-to-end monitors.
Monitoring the traffic from Host A to Dev B: Add Monitor 0 to slot 2, port 2 on Switch x, specifying 0x051200 as the SID and 0x111eef as the DID, as shown in the following example: switch:admin> perfaddeemonitor 2/2, "0x051200" "0x111eef" End-to-End monitor number 0 added. Monitor 0 counts the frames that have an SID of 0x051200 and a DID of 0x111eef.
AL_PA of the SIDs and DIDs for frames transmitted from and received by the port. Figure 29 shows the mask positions in the command. A mask (“ff”) is set on slot 1, port 2 to compare the AL_PA fields on the SID and DID in all frames (transmitted and received) on port 2. The frame SID and DID must match only the AL_PA portion of the specified SID-DID pair. Each port can have only one EE mask. The mask is applied to all end-to-end monitors on the port.
Monitoring filter-based performance Filter-based performance monitoring counts the number of times a frame with a particular pattern is transmitted by a port. Filter-based monitoring is achieved by configuring a filter for a particular purpose. The filter can be a standard filter (for example, a SCSI read command filter that counts the number of SCSI read commands that have been transmitted by the port) or a user-defined filter customized for your particular use.
switch:admin> perfaddscsimonitor 1/2 SCSI traffic frame monitor #3 added switch:admin> perfaddipmonitor 1/2 IP traffic frame monitor #4 added switch:admin> perfmonitorshow --class FLT 1/2 There are 5 filter-based monitors defined on port 2.
frame (SOF). When the offset is set to 0, the values 0–7 that are checked against that offset are predefined as shown in Table 79. Table 79 Predefined values at offset 0 Value SOF Value SOF 0 SOFf 4 SOFi2 1 SOFc1 5 SOFn2 2 SOFi1 6 SOFi3 3 SOFn1 7 SOFn3 If the switch does not have enough resources to create a given filter, then other filters might have to be deleted to free resources.
Identifying top bandwidth users (Top Talkers) Top Talker monitors determine the flows (SID/DID pairs) that are the major users of bandwidth (after initial stabilization). Top Talker monitors measure bandwidth usage data in real-time and relative to the port on which the monitor is installed. NOTE: Initial stabilization is the time taken by a flow to reach the maximum bandwidth. This time varies depending on the number of flows in the fabric and other factors.
Using Top Talker monitors in port mode Use the perfttmon command to add, delete, and display Top Talker monitors. Refer to the Fabric OS Command Reference for details about the perfttmon command. To add a Top Talker monitor on an F_Port: 1. Connect to the switch and log in as admin. 2. Enter the perfttmon --add command. perfttmon --add [egress | ingress] [slotnumber/]port where: slotnumber For director-class switches only (4/256 SAN Director and DC Director), the slot number. port The port number.
For example, to display the top 5 flows on port 7 in WWN (default) format: perfttmon --show 7 5 To display the top flows on slot 2, port 4 on the 4/256 SAN Director or DC Director in PID format: perfttmon --show 2/4 pid switch:admin> perfttmon --show 2/4 pid ======================================== Src_PID Dst_PID MB/sec ======================================== 0xa90800 0xa05200 6.926 0xa90800 0xa908ef 6.
To display the top flows on domain 2 in PID format: perfttmon --show dom 2 pid switch:admin> perfttmon --show dom 2 pid ======================================== Src_PID Dst_PID MB/sec ======================================== 0xa908ef 0xa05200 6.926 0xa05200 0xa908ef 6.872 0xa905ef 0xa05200 6.830 0xa909d5 0xa05200 6.772 Limitations of Top Talker monitors • Top Talker monitors cannot detect transient surges in traffic through a given flow.
slotnumber portnumber interval Specifies the slot number for a 4/256 SAN Director director. For all other switches, this operand is not required. The slot number must be followed by a slash ( / ) and the port number, so that each port is represented by both slot number (1 through 4 or 7 through 10) and port number (0 through 15). The Directors have a total of 10 slots. Slot numbers 5 and 6 are control processor blades; slots 1 through 4 and 7 through 10 are port blades.
The following example displays an end-to-end monitor on a port at 6-second intervals: switch:admin> perfMonitorShow --class EE 4/5 6 perfmonitorshow 53, 6: Tx/Rx are # of bytes and crc is # of crc errors 0 1 2 3 4 ------------- ------------- ------------- ------------- ------------crc Tx Rx crc Tx Rx crc Tx Rx crc Tx Rx crc Tx Rx ============= ============= ============= ============= ============= 0 0 0 0 0 0 0 0 0 0 0 53m 4.9m 0 53m 4.9m 0 53m 4.9m 0 0 53m 4.
The following example displays a filter-based monitor on a port at 6-second intervals: switch:admin> perfMonitorShow --class FLT 2/5 6 perfmonitorshow 21, 6 0 1 2 3 4 5 6 #Frames #Frames #Frames #Frames #Frames #Frames #Frames --------------------------------------------------------------0 0 0 0 0 0 0 26k 187 681 682 682 494 187 26k 177 711 710 710 534 176 26k 184 734 734 734 550 184 26k 182 649 649 649 467 182 26k 188 754 755 755 567 184 26k 183 716
Known display problem and workaround When two shared ports on an FC4-48 blade are receiving traffic and the primary port goes offline, all the frames that are out for delivery for the primary port are dropped, but the counters show them as dropped on the secondary port that shares the same area. Error counters increment unexpectedly for the secondary port, but the secondary port is operating properly. If this occurs, clear the counters on the secondary port after primary port goes offline.
where: slotnumber The monitor class, which can be one of EE (end-to-end), FLT (filter-based), or ISL (inter-switch link). The --class monitor_class operand is required. Specifies the slot number for a 4/256 SAN Director. For all other switches, this operand is not required. The slot number must be followed by a slash ( / ) and the port number, so that each port is represented by both slot number (1 through 4 or 7 through 10) and port number (0 through 15).
Saving and restoring monitor configurations To save the current end-to-end and filter monitor configuration settings into nonvolatile memory, use the perfCfgSave command: switch:admin> perfcfgsave This will overwrite previously saved Performance Monitoring settings in FLASH. Do you want to continue? (yes, y, no, n): [no] y Please wait ... Performance monitoring configuration saved in FLASH. To restore a saved monitor configuration, use the perfCfgRestore command.
364 Administering Advanced Performance Monitoring
18 Administering Extended Fabrics This chapter provides information on implementing Extended Fabrics software. Extended Fabrics licensing To implement long distance dynamic (LD) and long distance static (LS) distance levels, you must first install the Extended Fabrics license. Use the licenseShow command to verify that the license is present on both switches used on both ends of the extended ISL.
Table 80 describes Fibre Channel data frames Table 80 Fibre Channel data frames Start of frame 4 byes 32 bits Standard frame header 24 bytes 192 bits Data (payload) {0 - 2,112 bytes {0 - 16,896} bits CRC 4 bytes 32 bits End of frame 4 bytes 32 bits Total (Nbr bits/frame) {36 0 2,148} bytes 288 - 17, 184 bits The term byte used in Table 80 means 8 bits. The maximum fibre channel frame is 2,148 bytes.
FC switch port Buffer Credit requirements for long distance calculations You can calculate how many ports can be configured for long distance on all switch modules or ASICs except Bloom-based switches. For information on the port, speed and distance for Bloom-based ASICs, see Table 82. Following are the considerations for the calculation: • Each user port reserves eight buffers when they are not online. • Remaining buffers can be reserved by any port in the port group.
Example: Consider the 4/16 SAN Switch, which has 16 ports and total buffers of 272 The maximum remaining number of buffer credits after each port is reserved is: 272 – (16 * 8) = 144 buffers Where: 16 = the number of ports in a port group retrieved from Table 81. 8 = the number of reserved buffers 272 = a static number retrieved from Table 81. If you allocate the entire 144 + 8 reserved buffers = 152 buffers to a single port; you can have one port 146km @ 2G or 292km @ 1G.
2. Enter the portbuffershow command. switch:admin> portbuffershow 1 User Port Lx Max/Resv Port Type Mode Buffers ---------------0 U 8 1 U 8 2 U 8 3 U 8 4 U 8 5 U 8 6 U 8 7 U 8 8 U 8 9 U 8 10 U 8 11 U 8 12 8 13 8 14 8 15 8 16 U 8 17 U 8 18 U 8 19 8 20 U 8 21 U 8 22 U 8 23 U 8 switch:admin> Buffer Needed Usage Buffers ------ ------0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - Link Remaining Distance Buffers --------- ----484 Fabric OS 6.
Table 81 Switch, port speed, and distance with ASIC and buffers Switch blade model ASIC Total ports in a switch or blade Total ports in a group Reserved buffers for ports B-Series 2Gb Switches Bloom 8, 16 or 32 108/4 0 4/8 SAN Switch or 4/16 SAN Switch Golden Eye 16 272/16 8 SAN Switch 4/32 and SAN Switch 4/32B Condor 32 1000/32 8 4/64 SAN Switch Condor 64 712/16 8 400 Multi-protocol (MP) Router Condor 16 441/8 8 FR4-18i Condor 16 441/8 8 FC4-16iP Condor 8 680/8 8 F
Fabric considerations Because the number of buffer credits available for use within each port group is limited, configuring buffer credits for extended links may impact the performance of the other ports in the group used for core-to-edge connections. Balance the number of long-distance ISL connections and core-to-edge ISL connections within a switch. Configuring long-distance ISLs between core and edge switches is possible, but is not a recommended practice.
3. Enter the configure command to set the switch fabric-wide configurations. You can set the following fabric-wide settings: (* = multiplication symbol) Field Type Default Range Domain R_A_TOV E_D_TOV WAN_TOV MAX_HOPS Number Number Number Number Number 1 1000 2000 0 7 Varies E_D_TOV * 2 to 120000 1000 to R_A_TOV/2 0 to R_A_TOV/4 7 to 19 4.
Table 82 lists the extended ISL modes for switches with Bloom-based ASICs. You can configure extended ISL modes with the portCfgLongDistance command when the Extended Fabrics license is activated. Table 82 Extended ISL modes: B-Series 2Gb Switches (Bloom and Bloom II ASICs) Mode Buffer allocation Distance @ 1 Gbps Distance @ 2 Gbps Earliest Fabric OS release Extended Fabrics license required? 1 Gbps 2 Gbps L0 5 (26)1 5 (26) 10 km 5 km All No LE 13 19 n/a 10 km 3.x, 4.
Administering Extended Fabrics
19 Administering ISL Trunking This chapter contains procedures for using the ISL Trunking licensed feature, which optimizes the use of bandwidth by allowing a group of interswitch links to merge into a single logical link. About ISL Trunking ISL Trunking reduces or eliminates situations that require static traffic routes and individual ISL management to achieve optimal performance.
The maximum number of ports per trunk and trunks per switch depends on the HP model. NOTE: Director blade model FC10-6 does not support trunking. For detailed information about trunking commands, see online help or the Fabric OS Command Reference. Standard trunking criteria Observe the following criteria for standard distance trunking: • There must be a direct connection between participating switches. • Trunk ports must reside in the same port group.
• Determine the optimal number of trunking groups between each set of linked switches, depending on traffic patterns and port availability. The goal is to avoid traffic congestion without unnecessarily using ports that could be used to attach other switches or devices. Consider these points: • Each physical ISL uses two ports that could otherwise be used to attach node devices or other switches.
There are three methods of monitoring fabric traffic: • Advanced Performance Monitoring monitors traffic flow and allows you to view the impact of different fabric configurations on performance. See ”Administering Advanced Performance Monitoring” on page 361 for additional information. • Fabric Watch allows you to monitor traffic flow through specified ports on the switch and send alerts when the traffic exceeds or drops below configured thresholds.
Enabling and disabling ISL Trunking You can enable or disable ISL Trunking for a single port or for an entire switch.When you execute the commands portCfgTrunkPort or switchCfgTrunk to update the trunking configuration, the ports for which the configuration applies are disabled and re-enabled with the new trunk configuration. As a result, traffic through those ports could be disrupted. IMPORTANT: Trunking is performed based on QoS configuration on the master and the slave ports.
Setting port speeds For long-distance ports, if a port is set to autonegotiate port speed, the maximum speed (which is 4 Gbps) is assumed for reserving buffers for the port–this wastes buffers if the port is actually running at 2 Gbps. For long-distance ports, it is best to set the port speed (this applies to the 4/32 SAN Switch, 4/32B SAN Switch and the 4/256 SAN Director only).
Specifies the speed of the link: speedlevel • 0—Auto-negotiating mode. The port automatically configures for the highest speed. • 1—one Gbps mode. Fixes the port at a speed of one Gbps. Changing the speed to one Gbps causes the port to be excluded from the trunk group. • 2—two Gbps mode. Fixes the port at a speed of two Gbps. • 4—four Gbps mode. Fixes the port at a speed of four Gbps.
Trunking over Extended Fabrics In addition to the criteria listed in ”Standard trunking criteria” on page 378, observe the following criteria for trunking over Extended Fabrics: • ISL Trunking over Extended Fabrics is supported on switches running Fabric OS 4.4.0 and later. • Extended Fabrics and ISL Trunking licenses are required on all participating switches. • The vc_translation_link_init parameter must be set the same on all ports in an extended trunk.
Trunking distances Enhanced trunking support for switches with Condor ASICs is summarized in Table 82. Table 82 Trunking support for SAN Switch 4/32, 4/32B and 4/64 SAN Switch (Condor ASIC) Mode Distance Number of 2 Gbps ports Number of 4 Gbps ports LE 10 km 32 (four 8-port trunks) 32 (four 8-port trunks) L0.
• Port trunking is disabled. • The port is not an E_Port. • The port is not 2 Gbps, 4 Gbps, or 8 Gbps. • The port connects to different switches. • The ports are not the same speed, or they are not set to a valid speed. • The ports are not set to the same long distance mode. • Local or remote ports are not in the same port group. • The difference in the cable length among trunked links is greater than the allowed difference.
20 Administering Advanced Zoning About zoning Zoning enables you to partition your SAN into logical groups of devices that can access each other. A device can communicate only with other devices connected to the fabric within its specified zone. For example, you can partition your SAN into two zones, winzone and unixzone, so that your Windows servers and storage do not interact with your UNIX servers and storage.
Zone types Table 84 summarizes the types of zoning available. Table 84 Types of zoning Zone type Description Storage-based Storage units typically implement LUN-based zoning, also called LUN masking. LUN-based zoning limits access to the LUNs on the storage port to the specific WWN of the server HBA. It is needed in most SANs. It functions during the probe portion of SCSI initialization. The server probes the storage port for a list of available LUNs and their properties.
Table 85 Approaches to fabric-based zoning (continued) Zoning approach Description Alternative approaches Application Zoning by application typically requires zoning multiple, perhaps incompatible, operating systems into the same zones. This method of zoning creates the possibility that a minor server in the application suite could disrupt a major server (such as a Web server disrupting a data warehouse server).
When a zone object is the port WWN name, only the single port is in the zone. The types of zone objects used to define a zone can be mixed. For example, a zone defined with the zone objects 2,12; 2,14; 10:00:00:80:33:3f:aa:11 contains the devices connected to domain 2, ports 12 and 14, and a device with the WWN (either node name or port name) 10:00:00:80:33:3f:aa:11 that is connected on the fabric.
• Disabled Configuration—The effective configuration is removed from flash memory. When you disable the effective configuration, the Advanced Zoning feature is disabled on the fabric, and all devices within the fabric can communicate with all other devices (unless you previously set up a default zone, as described in ”Activating default zones” on page 403). This does not mean that the zoning database is deleted, however, only that there is no configuration active in the fabric.
• Is available on 1, 2, 4, 8 and 10 Gbps platforms. • Ensures that the name server does not return any information to an unauthorized initiator in response to a name server query. • Is exclusively enforced through selective information presented to end nodes through the fabric Simple Name Server (SNS). When an initiator queries the name server for accessible devices in the fabric, the name server returns only those devices that are in the same zone as the initiator.
Figure 32 shows a fabric with four hardware-enforced zones that don’t overlap. WWN_Zone1 Port_Zone1 Port_Zone2 Core Switch WWN_Zone2 Zone Boundaries 22.2b(13.2) Figure 32 Hardware-enforced nonoverlapping zones Figure 33 shows the same fabric components, but with overlapping zones. WWN_Zone1 Port_Zone1 Port_Zone2 Core Switch Zone Boundaries WWN_Zone2 22.3b(13.
Port_Zone2 Port_Zone1 WWN_Zone1 Core Switch WWN_Zone2 Zone Boundaries 22.5b(13.5) Figure 35 Session-based hard zoning In Figure 35, only the overlapping ports are software-enforced with hardware assist. Considerations for zoning architecture Table 87 lists considerations for zoning architecture. Table 87 Considerations for zoning architecture Item Description Type of zoning: hard or soft (session-based) If security is a priority, hard zoning is recommended.
Table 87 Considerations for zoning architecture (continued) Item Description Effect of changes in a production fabric Zone changes in a production fabric can result in a disruption of I/O under conditions when an RSCN is issued because of the zone change and the HBA is unable to process the RSCN fast enough. Although RSCNs are a normal part of a functioning SAN, the pause in I/O might not be acceptable.
To restrict broadcast frames reaching broadcast-incapable devices, create a broadcast zone and populate it with the devices that are capable of handling broadcast packets. Devices that cannot handle broadcast frames must be kept out of the broadcast zone so that they do not receive any broadcast frames. You create a broadcast zone the same way you create any other zone except that a broadcast zone must have the name “broadcast” (case-sensitive).
You can run zone -validate on a broadcast zone to check if it has any invalid members that cannot be enforced in the current AD context. Upgrade and downgrade considerations If you upgrade from a Fabric OS version earlier than 5.3.0 to Fabric OS 5.3.0 or later, you must rename any existing zones named “broadcast” before you upgrade. The firmware download fails if a pre-5.3.x switch has a zone with the name of “broadcast” in the effective configuration.
Creating and managing zone aliases A zone alias is a logical group of ports or WWNs. You can simplify the process of creating zones by first specifying aliases, which eliminates the need for long lists of individual zone member names. If you are creating a new alias using aliCreate w, “1,1”, and a user in another Telnet session executes cfgEnable (or cfgDisable, or cfgSave), the other user’s transaction will abort your transaction and you will receive an error message.
You are about to save the Defined zoning configuration. This action will only save the changes on the Defined configuration. Any changes made on the Effective configuration will not take effect until it is re-enabled. Do you want to save Defined zoning configuration only? (yes, y, no, n): [no] y \ To remove members from an alias: 1. Connect to the switch and log in as admin. 2. Enter the aliRemove command, using the following syntax: aliremove "aliasname", "member[; member...
3. Enter the cfgSave command to save the change to the defined configuration. switch:admin> alidelete "array1" switch:admin> cfgsave You are about to save the Defined zoning configuration. This action will only save the changes on the Defined configuration. Any changes made on the Effective configuration will not take effect until it is re-enabled. Do you want to save Defined zoning configuration only? (yes, y, no, n): [no] y To view an alias in the defined configuration: 1.
The values represent the following: zonename The name of the zone to be created. member A member or list of members to be added to the zone. A zone member can be specified by one or more of the following methods: • A domain,port pair. • Device node or device port WWN • Zone alias name To create a broadcast zone, use the reserved name “broadcast”. 3. Enter the cfgSave command to save the change to the defined configuration.
To remove devices (members) from a zone: 1. Connect to the switch and log in as admin. 2. Enter the zoneRemove command, using the following syntax: zoneremove "zonename", "member[; member...]" The values represent the following: zonename The name of the zone to be created. member A member or list of members to be removed from the zone. A zone member can be specified by one or more of the following methods: • A domain,port pair. • Device node or device port WWN • Zone alias name 3.
The values represent the following: pattern A POSIX-style regular expression used to match zone names. mode Specify 0 to display the contents of the transaction buffer (the contents of the current transaction), or specify 1 to display the contents of the nonvolatile memory. The default value is 0.
NOTE: If you performed a firmware download of an older release, then the current default zone access state will appear as it did prior to the download. For example, if the d_efault_Cfg was in effect before the download, it will remain in effect afterward. See the Fabric OS Command Reference for additional information on the defZone command. Merging zones Table 88 presents zoning database size limitations for various Fabric OS release versions.
Table 89 Resulting database size: 0 to 96K Fabric OS 2.6 Fabric OS 3.1 Fabric Fabric OS 3.2 OS 4.0/ 4.1/ 4.2 Fabric OS 4.3/ 4.4.0 Fabric OS 5.0.0/ 5.0.1/ 5.1.0 Fabric OS 5.2.0/ 5.3.0 Fibre Channel Router XPath OS 7.3 Fabric OS 2.6/3.1 Join Join Join Join Join Join Join Join Join Fabric OS 3.2 Join Join Join Join Join Join Join Join Join Fabric OS 4.0/ 4.1/4.2 Join Join Join Join Join Join Join Join Join Fabric OS 4.4.
2 Table 91 Resulting database size: 128K to 256K Receiver Fabric OS Fibre 5.2.0/ Channel 5.3.0 Router XPath 7.3 Fabric OS 3.1 Fabric OS 2.6/3.1 Segment Segment Segmen Segment Segmen Segment Segment t t Join Segment Fabric OS 3.2 Segment Segment Join Join Segment Fabric OS 4.0/4.1/4.2 Segment Segment Segmen Segment Segmen Segment Segment t t Segment Segment Fabric OS 4.4.0 Segment Segment Join Segment Join Join Join Join Segment Fabric OS 5.0.0/5.0.
Table 92 Resulting database size: 256K to 1M (continued) Receiver Fabric OS 2.6 Fabric OS 3.1 Initiator Fabric OS 3.2 Fabric OS 4.0/v 4.1/ 4.2 Fabric OS 4.3/ 4.4.0 Fabric OS 5.0.0/ 5.0.1/ 5.1.0 Fabric OS 5.2.0/ 5.3.0 Fibre Channel Router XPath 7.3 Join Segment Fibre Channel Router Segment Segment Segment Segment Segment Join Join XPath 7.3 Segment Segment Segment Segment Segment Segment Segmen Segment Segment t Fabric OS 6.
Creating and modifying zoning configurations You can store a number of zones in a zoning configuration database. The maximum number of items that can be stored in the zoning configuration database depends on the following criteria: • Number of switches in the fabric. • Whether or not interoperability mode is enabled. • Number of bytes per item. The number of bytes required for an item depends on the specifics of the fabric, but cannot exceed 64 bytes per item.
The values represent the following: cfgname The name of the zone configuration. member The zone name or list of zone names to be added to the configuration. 3. Enter the cfgSave command to save the change to the defined configuration. switch:admin> cfgadd "newcfg", "bluezone" switch:admin> cfgsave You are about to save the Defined zoning configuration. This action will only save the changes on the Defined configuration.
Do you want to save Defined zoning configuration only? (yes, y, no, n): [no] y To clear changes to a configuration: 1. Enter the cfgTransAbort command. When this command is executed, all changes since the last save operation (performed with the cfgSave command) are cleared.
For example, to display all zone configurations that start with “Test”: switch:admin> cfgshow "Test*" cfg: Test1 Blue_zone cfg: Test_cfg Red_zone; Blue_zone To view a configuration in the effective zone database: 1. Connect to the switch and log in as admin. 2. Enter the cfgActvShow command.
4. Enter the cfgShow command to verify the new zone object is present. switch:admin> cfgshow "Test*" cfg: Test1 Blue_zone cfg: Test_cfg Red_zone; Blue_zone switch:admin> cfgShow "US_Test1" cfg: Test1 Blue_zone cfg: Test_cfg Red_zone; Blue_zone 5. If you want the change preserved when the switch reboots, enter the cfgSave command to save it to nonvolatile (flash) memory. 6. Enter the cfgEnable command for the appropriate zone configuration to make the change effective. To delete a zone object: 1.
To rename a zone object: 1. Connect to the switch and log in as admin. 2. Enter the cfgShow command to view the zone configuration objects you want to rename.
4. To validate all zones in the zone database in the defined configuration.
Before the new fabric can merge successfully, it must pass the following criteria: • Before merging zones To facilitate merging, check the following before merging switches or fabrics: • Zoning licenses: All switches must have a Zoning license enabled. • Native operating mode: All switches must be in the native operating mode. • Secure Fabric OS: The switch being merged into the existing fabric must not have Secure Fabric OS enabled.
A merge is not possible if any of the following conditions exist: • Configuration mismatch: Zoning is enabled in both fabrics and the zone configurations that are enabled are different in each fabric. • Type mismatch: The name of a zone object in one fabric is used for a different type of zone object in the other fabric. • Content mismatch: The definition of a zone object in one fabric is different from the definition of zone object with the same name in the other fabric.
IMPORTANT: Use caution using the cfgClear command because it deletes the defined configuration. Fabric OS 6.
418 Administering Advanced Zoning
21 Configuring and monitoring FCIP extension services This chapter describes the FCIP concepts, configuration procedures, and tools and procedures for monitoring network performance. Commands described in this chapter require Admin or root user access. See the Fabric OS Command Reference for detailed information on command syntax. FCIP services licensing Most of the FCIP extension services described in this chapter require the High Performance Extension over FCIP/FC license.
Platforms that support SAN extension over IP Fabric OS supports SAN extension between 400 Multi-protocol Routers or between FR4-18i blades installed on 4/256 SAN Directors or DC SAN Backbone Directors. The 400 Multi-protocol Router and FR4-18i blade integrate sixteen physical Fibre Channel ports and two physical GbE ports as illustrated in Figure 37 and Figure 38.
FCIP concepts Fibre Channel over IP (FCIP) enables you to connect Fibre Channel SANs over IP-based networks. 400 Multi-protocol Router and FR4-18i blades use FCIP to encapsulate Fibre Channel frames within IP frames that can be sent over an IP network to a partner 400 Multi-protocol Router or FR4-18i blade. When the IP packets are received, the Fibre Channel frames are reconstructed. The Fibre Channel fabric and all Fibre Channel targets and initiators are unaware of the presence of the IP network.
Fibre Channel initiator Fibre Channel initiator Office FC SAN Data Center FC SAN IP WAN Network VE_Port VE_Port 400 MP Router VE_Port 4/256 SAN Director with FR4-18i Blade 400 MP Router VE_Port Office FC SAN Office FC SAN Fibre Channel Target Figure 39 Network using FCIP 4/256 SAN Director with FR4-18i Blade Fibre Channel Target 26412a Compression Data compression can be enabled or disabled on FCIP tunnels. The default setting is to disable compression.
Layer three DiffServ Code Points (DSCP) Layer three class of service DiffServ Code Points (DSCP) refers to a specific implementation for establishing QoS policies as defined by RFC2475. DSCP uses six bits of the Type of Service (TOS) field in the IP header to establish up to 64 different values to associate with data traffic priority. DSCP settings are useful only if IP routers are configured to enforce QoS policies uniformly within the network.
IPSec concepts and implementation over FCIP Internet Protocol security (IPSec) uses cryptographic security to ensure private, secure communications over Internet Protocol networks. IPSec supports network-level data integrity, data confidentiality, data origin authentication, and replay protection.
Table 95 IPSec terminology (continued) Term HMAC SA Definition A stronger MAC because it is a keyed hash inside a keyed hash. Security Association is the collection of security parameters and authenticated keys that are negotiated between IPSec peers. The following limitations apply to using IPSec: • IPv6, NAT, and AH are not supported. • You can only create a single secure tunnel on a port; you cannot create a nonsecure tunnel on the same port as a secure tunnel.
Constraints for FCIP fastwrite and tape pipelining Consider the constraints described in Table 96 when configuring tunnels to use either of these features. Table 96 Using FCIP fastwrite and tape pipelining FCIP fastwrite Tape pipelining Each GbE port supports up to 2048 simultaneous accelerated exchanges, which means a total of 2048 simultaneous exchanges combined for fastwrite and tape pipelining.
Connection can be VE-VE or VEX-VE Figure 40 Single tunnel, fastwrite and tape pipelining enabled Figure 41 Multiple tunnels to multiple ports, fastwrite and tape pipelining enabled on a per-tunnel/per-port basis Unsupported configurations The following configurations are not supported with fastwrite and tape pipelining. These configurations use multiple equal-cost paths. Fabric OS 6.
VE-VE or VEX-VEX Figure 42 Unsupported configurations with fastwrite and tape pipelining FICON emulation concepts FICON emulation supports FICON traffic over IP WANs using FCIP as the underlying protocol. FICON emulation can be extended to support performance enhancements for specific applications.
XRC emulation The eXtended Remote Copy (XRC) application is a DASD application that implements disk mirroring, as supported by the disk hardware architecture and a host software component called System Data Mover (SDM). The primary volume and the secondary mirrored volume may be geographically distant across an IP WAN. The latency introduced by greater distance creates delays in anticipated responses to certain commands.
FCIP services configuration guidelines There are multiple configuration requirements and options associated with FCIP services. The following general guidelines may be helpful. The steps are presented in an order that minimizes the number of times ports need to be disabled and enabled. In practice, the steps do not have to be taken in this order. 1. Determine if you are implementing IPSec.
Table 97 Command checklist for configuring FCIP links (continued) Step Command 3. If a VEX port is to be implemented, configure the appropriate virtual port as a VEX_Port. portcfgvexport 4. Configure the IP interface for both ports of a tunnel. portcfg ipif 5. Verify the IP interface for both ports of a tunnel. portshow ipif 6.Create one or more IP routes connecting the IP interfaces across the IP network. portcfg iproute 7.
IPSec policies are managed using the policy command. You can configure up to 32 IKE and 32 IPSec policies. Policies cannot be modified; they must be deleted and recreated in order to change the parameters. You can delete and recreate any policy as long as the policy is not being used by an active FCIP tunnel. Each FCIP tunnel is configured separately and may have the same or different IKE and IPSec policies as any other tunnel. Only one IPSec tunnel can be configured for each GbE port.
Managing policies Use the policy command to create, delete, and show IKE and IPSec policies. To create a new policy: 1. Log in to the switch as admin. 2. At the command prompt, type: policy --create type number [-enc encryption_method][-auth authentication_algorithm] [-pfs off|on] [-dh DH_group] [-seclife secs] where: type and number The type of policy being created (IKE or IPSec) and the number for this type of policy.
The example below shows all of the IKE policies defined; in this example, there are two IKE policies.
SACK on Min Retransmit Time 100 Keepalive Timeout 80 Max Retransmissions 9 Status : Active Uptime 1 day, 23 hours, 24 minutes, 46 seconds IKE Policy 7 ----------------------------------------Authentication Algorithm: MD5 Encryption: 3DES Perfect Forward Secrecy: off Diffie-Hellman Group: 1 SA Life (seconds): 200000 IPSec Policy 7 ----------------------------------------Authentication Algorithm: AES-XCBC Encryption: 3DES SA Life (seconds): 1500000 Pre-Shared Key 1234567890123456 Persistently disabling ports
The following example configures a port as a VEX_Port for slot number 8 in port number 18, enables admin, and specifies fabric ID 2 and preferred Domain ID 220: switch:admin06> portcfgvexport 8/18 -a 1 -f 2 -d 220 Configuring IP interfaces and IP routes The IP network connection between two 400 Multi-protocol Router or two FR4-18i blades is configured by defining IP interfaces for origin and destination virtual ports, and then defining one or more IP routes to connect them. 1.
The following example verifies that the two routes have been successfully created: switch:admin06> portshow iproute 8/ge0 Slot: 8 Port: ge0 IP Address Mask Gateway Metric Flags ----------------------------------------------------------------192.168.100.0 Interface 255.255.255.0 192.168.100.40 0 192.168.100.0 Interface 255.255.255.0 192.168.100.41 0 192.168.11.0 255.255.255.0 192.168.100.1 1 192.168.12.0 255.255.255.0 192.168.100.1 1 3.
The following example tests the connection between 192.175.5.100 and 192.175.5.200, switch:admin06> portcmd --ping ge0 -s 192.175.5.100 -d 192.175.5.200 Pinging 192.175.5.200 from ip interface 192.175.5.100 on 0/ge0 with 64 bytes of data Reply from 192.175.5.200: bytes=64 rtt=1ms ttl=64 Reply from 192.175.5.200: bytes=64 rtt=0ms ttl=64 Reply from 192.175.5.200: bytes=64 rtt=0ms ttl=64 Reply from 192.175.5.200: bytes=64 rtt=1ms ttl=64 Ping Statistics for 192.175.5.
-f Enables FCIP fastwrite. -M Enables VC QoS mapping. -t Enables tape pipelining. If tape pipelining is enabled, fastwrite must also be enabled. -n remote_wwn The remote-side FC entity WWN. -k timeout The keep-alive timeout in seconds. The range of valid values is 8 through 7,200 sec and the default is 10. If tape pipelining is enabled both the default and minimum values are 80 sec. -r retransmissions The maximum number of retransmissions on the existing FCIP tunnel.
Where: slot The number of a slot in a 4/256 SAN Director or DC Director chassis that contains an FR4-18i blade. This parameter does not apply to the stand-alone 400 Multi-protocol Router. ge0|ge1 The Ethernet port used by the tunnel (ge0 or ge1). tunnel_id The tunnel number (0 - 7). config The config option creates a configuration. -x 1|0 Enables or disables XRC emulation. 1 is enable, O is disable. -w 1|0 Enables or disables tape write pipelining. 1 is enable, O is disable.
Configuring FTRACE FTRACE is a support tool primarily for use by Tech Support personnel. FTRACE includes the ability to freeze traces on certain events, and to retain the trace information for future examination.
The following example shows an active tunnel with FCIP fastwrite and tape pipelining enabled: switch:admin06> portshow fciptunnel ge0 all ------------------------------------------Tunnel ID 0 Remote IP Addr 10.0.10.224 Local IP Addr 10.0.10.225 Remote WWN Not Configured Local WWN 10:00:00:05:1e:37:91:dd Compression on Fastwrite on Tape Pipelining on Uncommitted bandwidth, minimum of 1000 Kbps (0.
To verify that a VE_Port or VEX_Port is online, use the switchShow command to view and verify that the FCIP tunnel is online. switch:admin06> portenable 8/18 switch:admin06> portenable 8/19 switch:admin06> switchshow switchName:switch switchType:42.
Enabling persistently disabled ports Before an FCIP tunnel can be used, the associated ports must be persistently enabled. NOTE: VEX_Port Users: If the fabric is already connected, you must leave the ge0 and ge1 ports disabled until after you have configured the VEX_Port; this will prevent unintentional merging of the two fabrics. To enable a persistently disabled port: 1. Enter the portCfgShow command to view ports that are persistently disabled. 2.
3.
Modify and delete command options Command options are available that allow you to modify or delete configured elements. NOTE: Using the Modify option disrupts traffic on the specified FCIP tunnel for a brief period of time. Modifying FCIP tunnels The portCfg fcipTunnel command to modify FCIP tunnels (you must specify at least one characteristic to modify).
-p control_L2Cos The layer 2 class of service used for control traffic. -P data_L2Cos The layer 2 class of service used for data traffic. The following example shows two FCIP tunnels created on slot 8, port ge0; the first with an uncommitted bandwidth (0), and the second with a committed bandwidth of 10000 Kb/sec: switch:admin06> portcfg fciptunnel 8/ge0 create 2 192.168.100.50 192.168.100.40 0 switch:admin06> portcfg fciptunnel 8/ge0 create 3 192.168.100.51 192.168.100.
modify The modify option changes the FICON emulation configuration options and parameters. The following options turn features on and off. The associates tunnels must be disabled to modify the option settings. If you attempt to do them on an enabled tunnel, the operation is not allowed, and you are prompted to disable the tunnel. -x 1|0 Enables or disables XRC emulation. 1 is enable, O is disable. -w 1|0 Enables or disables tape write pipelining. 1 is enable, O is disable.
wrtMaxChains value Defines the maximum amount of data that can be contained in a single CCW chain. If this value is exceeded, emulation is suspended. oxidBase value Defines the base value of an entry pool of 256 OXIDs supplied to emulation generated exchanges. It should fall outside the range used by FICON channels and devices to avoid conflicts. The default value is 0x1000. The range is 0x0000 to 0xF000. dbgFlags value Defines optional debug flags.
NOTE: If you do not specify a destination IP address, the destination address defaults to 0.0.0.0, and all frames are tagged with the associated VLAN tag. FCIP and ipPerf create and maintain entries in the VLAN tag table through their own configuration procedures. Manual entries are needed on both the local and remote sides for portcmd ping and portcmd traceroute commands when they are used to test and trace routes across a VLAN when no FCIP tunnel is active.
WAN performance analysis tools Introduced in Fabric OS 5.2.0, WAN analysis tools are designed to test connections, trace routes, and estimate the end-to-end IP path performance characteristics between a pair of HP FCIP port endpoints. WAN tools include the following commands and options: • portCmd ipPerf—Characterizes end-to-end IP path performance between a pair of HP FCIP ports. You can use the WAN tool ipPerf only on the FR4-18i or 400 Multi-protocol Router FCIP ports running Fabric OS 5.2.
WAN tool performance characteristics Table 100 lists the end-to-end IP path performance characteristics that you can display using the portCmd ipPerf command and option. All four of the base ipPerf performance characteristics (bandwidth, loss, RTT, PMTU) are provided in the command output in Fabric OS 5.2.0 or later. Table 100 WAN tool performance characteristics Characteristic Description Bandwidth Indicates the total packets and bytes sent.
To start an ipPerf session: 1. Configure the receiver test endpoint using the CP CLI. The syntax for invoking the receiver test endpoint using ipPerf for slot8, port ge0 on an FR4-18i is as follows: portcmd --ipperf 8/ge0 -s 192.168.255.10 -d 192.168.255.100 -R 2. Configure the sender test endpoint using a similar CP CLI. The syntax for invoking the sender test endpoint using ipPerf for slot8, port ge0 on an FR4-18i is as follows: portcmd --ipperf 8/ge0 -s 192.168.255.100 -d 192.168.255.
• Default size—1MSS Following is the syntax for portCmd ipPerf to display end-to-end IP path performance statistics: portCmd --ipPerf [slot]/ge0|ge1 -s source_ip -d destination_ip -S|-R [-r rate] [-z size] [-t time] [-i interval] [-p port] [-q diffserv] [-v vlan_id] [-c L2_Cos] Where: -s source_ip The source IP address. -d destination_ip The destination IP address. -S Operates the WAN tool FCIP port-embedded client in the sender mode.
Where: slot The number of a slot in a 4/256 SAN Director or DC Director chassis that contains an FR4-18i blade. This parameter does not apply to the stand-alone 400 Multi-protocol Router. ge0|ge1 The Ethernet port used by the tunnel (ge0 or ge1) -s source_ip The source IP interface that originates the ping request. -d destination_ip The destination IP address for the ping request. -n num-requests Generates a specified number of ping requests. The default is 4. -q diffserv The DiffServ QoS.
-h max_hops The maximum number of IP router hops allowed for the outbound probe packets. If this value is exceeded, the probe is stopped. The default is 30. -f first_ttl The initial time to live value for the first outbound probe packet. The default value is 1. -q diffserv The DiffServ QoS. The default is 0 (zero). The value must be an integer in the range from 0 through 255. -w wait-time The time to wait for the response of each ping request.
2013762456 compressed Bytes 33208083 Bps 30s avg, 4760667 Bps lifetime avg 7.35 compression ratio FC control traffic TCP connection: Local 192.175.4.100:4139, Remote 192.175.4.200:3225 Performance stats: 849 output packets 0 pkt/s 30s avg, 2 pkt/s lifetime avg 173404 output Bytes 39 Bps 30s avg, 409 Bps lifetime avg 0 packets lost (retransmits) 0.
Uptime 7 minutes, 3 seconds FC control traffic TCP connection: Local 192.175.4.100:4139, Remote 192.175.4.200:3225 Runtime parameters: Send MSS 1456 Bytes Sender stats: smoothed roundtrip 50 ms, variance 0 peer advertised window 1874944 Bytes negotiated window scale (shift count) 9 congestion window 149649 Bytes slow start threshold 1875000 Bytes operational mode: slow start 2 packets queued: TCP sequence# MIN(2950582519) MAX(2950582655) NXT(2950582655) 2 packets in-flight Send.
FICON performance statistics You can use the portShow fcipTunnel command to view the performance statistics and monitor the behavior of an online FCIP tunnel. This additional information is reported in the details of the command output. portshow ficon [Slot/]ge0|ge1 all|tunnel_id [arguments] Where: slot The slot number of a blade in a multi-slot chassis. Does not apply to the 400 Multi-protocol Router. ge0|ge1 The Ethernet port (ge0 or ge1). tunnel_id Tunnel number (0-7).
2 OFF OFF OFF OFF OFF OFF 3 OFF OFF OFF OFF OFF OFF 4 OFF OFF OFF OFF OFF OFF 5 OFF OFF OFF OFF OFF OFF 6 OFF OFF OFF OFF OFF OFF 7 OFF OFF OFF OFF OFF OFF PARAMETERS TunnelId WrtPipe RdPipe WrtDevs RdDevs WrtTimer WrtChain OxidBase DebugFlags 0 000 000 00 00 0000 0000000 0x0000 0x00000000 1 000 000 00 00 0000 0000000 0x0000 0x00000000 2 000 000 00 00 0000 0000000 0x0000 0x00000000 3 000 000 00 00 0000 0000000 0x0000 0x00000000 4
Enable Enable traces for a tunnel. Filter Display the active trace filters. Help Display this menu. HEX Display traces in HEX. INDex xx Display traces starting at index 'xx'. INOxid 0-FFFF Set/Reset the inbound FC OXID filter. Lcontrol Set/Reset the FICON link control frame filter. N Display the next trace records. NCfg Display a new FCTRACE configuration. OUtoxid 0-FFFF Set/Reset the outbound FC OXID filter. OXid 0-FFFF Set/Reset the inbound and outbound FC OXID filters.
Channel ISLs implemented through the FC-FC Routing Service (FRS) rather than FCIP. FC fastwrite is supported in Fabric OS 5.3.x and later. 400 MP Router 400 MP Router Figure 43 Typical network topology for FC fastwrite Platforms and OS requirements for FC fastwrite Fabric OS supports FC fastwrite between two 400 Multi-protocol Router or two 4/256 SAN Directors with FR4-18i blades connected by a Fibre Channel network. FC fastwrite is a new feature beginning with Fabric OS release 5.3.0.
4. The PI continues to stage data received from the initiator, respond locally to Transfer Ready, and send the data to the target device until the target device sends an FCP_RSP. Figure 44 How FC fastwrite works FC fastwrite can improve Write performance. Read performance is unaffected. The gains seen from enabling FC fastwrite depend on several factors, including the following: • The size of I/O vs. Transfer Ready.
Configuring and enabling FC fastwrite The FC-FC (Fibre Channel) Routing Service provides Fibre Channel routing between two or more fabrics without merging those fabrics. The FC-FC Routing Service can be simultaneously used as a Fibre Channel router and for SAN extension over wide area networks (WANs) using FCIP. Take the following steps to configure and enable FC fastwrite. 1. Create a zone configuration to filter FC fastwrite flows.
4. Repeat steps 1 through 3 for the blade or switch on the other end of the FC fastwrite path. 5. Use the portshow command to verify that FC fastwrite is enabled. rack1_6a1:root> portshow 3/3 portName: portHealth: HEALTHY Authentication: None portDisableReason: None portCFlags: 0x1 portFlags: 0x20b03 PRESENT ACTIVE F_PORT G_PORT U_PORT LOGICAL_ONLINE LOGIN NOELP ACCEPT portType: 10.
Disabling FC fastwrite on a blade or switch Disable FC fastwrite using the fastwritecfg command. Disabling FC fastwrite with this command disrupts data traffic. For the FR4-18i blade, the command powers the blade off and back on. In the case of the 400 Multi-protocol Router, the switch is rebooted. The process takes up to five minutes. #fastwritecfg —disable slot# Where slot# is the slot in which the FR4-18i blade is installed. A slot number is not required for the 400 Multi-protocol Router.
A Configuring the PID format Port identifiers (called PIDs) are used by the routing and zoning services in Fibre Channel fabrics to identify ports in the network. All devices in a fabric must use the same PID format, so when you add new equipment to the SAN, you might need to change the PID format on legacy equipment. NOTE: Fabric OS 6.0 only supports PID format 1 (Core PID).
In addition to the PID formats list here, Interoperability mode supports additional PID formats that are not discussed in this guide. Impact of changing the fabric PID format If your fabric contains switches that use Native PID, it is recommended that you change the format to Core PID before you add the new, higher port count switches and directors. Also, it is recommended that you use Core PID when upgrading the Fabric OS version on 1Gb and 2Gb series switches.
Changes to configuration data Table 101 lists various combinations of before-and-after PID formats, and indicates whether the configuration is affected. NOTE: After changing the fabric PID format, if the change invalidates the configuration data (see Table 101 to determine this), do not download old (pre-PID format change) configuration files to any switch on the fabric.
Table 102 shows various combinations of existing fabrics, new switches added to those fabrics, and the recommended PID format for that combination. The criteria for the recommendations are first to eliminate host reboots, and second to minimize the need for a host reboot in the future. Table 102 PID format recommendations for adding new switches Existing Fabric OS versions; PID format Switch to be added Recommendations (in order of preference) 2.6.2 and later/3.1.2 and later; Native PID 2.6.
1. Collect device, software, hardware, and configuration data. The following is a non-comprehensive list of information to collect: • HBA driver versions • Fabric OS versions • RAID array microcode versions • SCSI bridge code versions • JBOD drive firmware versions • Multipathing software versions • HBA time-out values • Multipathing software timeout values • Kernel timeout values • Configuration of switch 2. Make a list of manually configurable PID drivers.
If either of the first two options are used, the procedures should again be validated in the test environment. Determine the behavior of multipathing software, including but not limited to: • HBA time-out values • Multipathing software time-out values • Kernel time-out values Planning the update procedure Whether it is best to perform an offline or online update depends on the uptime requirements of the site. • An offline update must have all devices attached to the fabric be offline.
Offline update The following steps are intended to provide SAN administrators a starting point for creating site-specific procedures. 1. Schedule an outage for all devices attached to the fabric. 2. Back up all data and verify backups. 3. Shut down all hosts and storage devices attached to the fabric. 4. Disable all switches in the fabric. 5. Change the PID format on each switch in the fabric. 6. Reenable the switches in the updated fabric one at a time.
Before changing the PID format, determine if host reboots will be necessary. The section ”Host reboots” on page 468 summarizes the situations that may require a reboot. switch:admin> switchdisable switch:admin> configure Configure... Fabric parameters (yes, y, no, n): [no] y Domain: (1..239) [1] BB credit: (1..27) [16] R_A_TOV: (4000..120000) [10000] E_D_TOV: (1000..5000) [2000] WAN_TOV: (1000..120000) [0] Data field size: (256..2112) [2112] Sequence Level Switching: (0..1) [0] Disable Device Probing: (0..
! 31 143 47 159 63 175 79 191 30 142 46 158 62 174 78 190 29 141 45 157 61 173 77 28 140 44 156 60 172 27 139 43 155 59 26 138 42 154 25 137 41 24 136 40 ! 111 223 127 239 15 255 94 206 110 222 126 238 14 254 189 93 205 109 221 125 237 13 253 76 188 92 204 108 220 124 236 12 252 171 75 187 91 203 107 219 123 235 11 251 58 170 74 186 90 202 106 218 122 234 10 250 153 57 169 73 185 89 201 105 217 121
Performing PID format changes There are several routine maintenance procedures which might result in a device receiving a new PID.
9. Enter the switchEnable command to re-enable the switch. For example: switch:admin> switchdisable switch:admin> configure Configure... Fabric parameters (yes, y, no, n): [no] yes Domain: (1..239) [1] R_A_TOV: (4000..120000) [10000] E_D_TOV: (1000..5000) [2000] Data field size: (256..2112) [2112] Sequence Level Switching: (0..1) [0] Disable Device Probing: (0..1) [0] Suppress Class F Traffic: (0..1) [0] SYNC IO mode: (0..1) [0] Core Switch PID Format: (0..2) [0] 1 Per-frame Route Priority: (0..
14. Change to /dev and untar the file that was tared in step 4. For example: tar –xf /tmp/jbod.tar 15. Import the volume groups using vgimport. The proper usage would be vgimport –m . For example: vgimport –m /tmp/jbod_map /dev/jbod /dev/dsk/c64t8d0 /dev/dsk/c64t9d0 16. Activate the volume groups using vgchange. The proper usage would be vgchange –a y . For example: vgexport –a y /dev/jbod 17.
AIX procedure This procedure is not intended to be comprehensive. It provides a starting point from which a SAN administrator can develop a site-specific procedure for a device that binds automatically by PID, and cannot be rebooted due to uptime requirements. 1. Back up all data. Verify backups. 2. If you are not using multipathing software, stop all I/O going to all volumes connected through the switch or fabric to be updated. 3. If you are not using multipathing software, vary the volume groups offline.
Swapping port area IDs If a device that uses port binding is connected to a port that fails, you can use port swapping to make another physical port use the same PID as the failed port. The device can then be plugged into the new port without the need to reboot the device. Use the following procedure to swap the port area IDs of two physical switch ports. In order to swap port area IDs, the port swap feature must be enabled, and both switch ports must be disabled.
B Implementing an interoperable fabric This appendix provides information on setting up a heterogeneous fabric that includes Fabric OS switches and McDATA Enterprise OS switches (M-EOS). IMPORTANT: These features are not supported at the time of the release of this document. Please check with your sales representative or http://www.hp.com regarding HP support of the interoperability features. In Fabric OS 6.
Table 103 McDATA-aware features (continued) Feature Behavior FCR Fabric OS L2 SANtegrity Supported only in McDATA Fabric mode. Name server Displays the device PID with domain offset. For example, a host attached to a switch with domain value 1 will have a default PID of 0x61AAPP. (nsShow, nsCamShow, nsAllShow) ESS Displays the firmware version in the McDATA format as 9.6.2, and the Brocade format as 6.0.
Supported Connectivity for Fabric 6.0 Brocade switches can directly connect to the following Brocade M-series (formerly McDATA) directors: Mi10k, M6140, M6064 and switches: 4700, 4400, 4500, 4300, 3232,3216, 3032, 3016. Other M-EOS 9.6.2 products can reside in the same fabric as a Brocade switch but cannot directly connect to it. M-EOS 9.6.2 must be running on all Fibre Channel switches in the fabric.
Table 105 Complete feature compatibility matrix (continued) Config Download/Upload Yes DHCP Yes Environmental Monitor Yes Error Event Management Yes Extended Fabrics Yes Fabric Device Management Interface (FDMI) Yes Fabric Watch Yes FICON (includes CUP) No High Availability Yes Interoperability • Fabric OS Native Mode: No Displays the credit number in the configure command. HCL (Hot Code Load) in Fabric OS 6.
Table 105 Complete feature compatibility matrix (continued) Speed Negotiation Yes Syslog Daemon Yes Trunking • Frame-level ISL Trunking from Brocade to Brocade: Yes - McDATA Fabric Mode only • Frame-level ISL Trunking from Brocade to McDATA: No • Load balancing from Brocade to Brocade using DLS or DPS: Yes • Load balancing from Brocade to McDATA using DLS or DPS: Yes ValueLineOptions (Static POD, DPOD) Yes Web Tools Yes Launch from EFCM with non fabric-wise configuration Zoning Yes for McDATA
Trunking HP switches support trunking when participating in an M-EOS Native fabric. Trunk ports (bandwidth aggregation) only apply to ISL between two HP switches. NOTE: Trunking is allowed between Brocade switches in Native mode only. Trunking is disabled between Brocade switches running in McDATA Open Fabric mode. • Fabric OS frame-based trunking is supported for frame-based ISLs. Multiple ISLs between a Brocade switch and an M-EOS type switch are allowed, but no frame-based trunking will occur.
The following licensed features are not supported with Fabric OS 6.0: Table 107 Unsupported features License Feature Advanced Zoning In Fabric 0S 6.0, Advance zoning does require a license. Zoning is configured through M-EOS switches. Fabric Manager Enables administration, configuration, and maintenance of fabric switches and SANs with host-based software. Only supports Brocade switches. Advanced Performance Monitoring Enables performance monitoring of networked storage resources.
Supported features McDATA Fabric mode (interopmode 2) The following features are supported in Fabric OS 6.0: • Zone activation Zoning managed through EFCM • ESA frame support • Coordinated Hot Code Load • FCR E_Port SANtegrity • Fabric OS L2 SANtegrity Support • No limitations on the number of E_Port connections to any Brocade switch • HA support for zoning Supported features McDATA Open Fabric mode (interopmode 3) The following features are supported in Fabric OS 6.
McDATA Open Fabric mode configuration restrictions • Maximum 200 devices. • Maximum 4 switch (Domain ID) limitation. • Domain IDs must be in the 97 to 127 value range on Fabric OS switches for successful connection to McDATA switches. The firmware automatically assigns a valid Domain ID, if necessary. If Fabric OS 6.0 is installed on a Brocade switch, and when McDATA Open Fabric is enabled on the switch, then from a McDATA perspective, 97-127 appear as 1-31.
• Brocade switches connected to McDATA switches receive the effective configuration when a zone merge occurs. (McDATA only has an effective zone configuration and discards the defined zone configuration when it sends merge information to the Brocade switch.) However, a zone update sends the defined and effective configuration to all switches in the fabric. • Use the cfgsaveactivetodefined command to move the effective configuration to the defined configuration database.
Safe zone Safe zoning is a fabric-wide parameter that ensures that the resulting zone set of two merged fabrics is consistent with the pre-merged zone sets. When you enable Safe zone, the Default Zone must be disabled and the zoning configuration of neighboring switches must match completely before the zoning can merge. To allow a Brocade switch into an M-EOS Native fabric, safe zoning must be disabled. This allows the Brocade switch to join the fabric although the zone sets do not match.
To view zoning configurations: • Enter the cfgShow command to view the zoning configuration. switch:admin cfgShow Default Zone: OFF Safe Zone: OFF Defined configuration: cfg: switch set switch1; sqitch2; switch3; switch4 zone: switch1 dd:dd:dd:dd:aa:aa:aa:aa; bb:bb:bb:cc:cc:cd:dd:dd zone: switch2 23:34:87:23:50:72:35:07; 12,64 [output truncated] ...
Moving to McDATA Open Fabric mode from earlier Fabric OS versions To move from interopmode 1 under Fabric OS 5.3 to Open Fabric mode: 1. Enter the switchDisable command to disable the switch. switch:admin> switchdisable 2. Enter the interopmode 0 command (native Brocade mode). 3. Upgrade to Fabric OS 6.0. 4. Enter the interopmode 3 command to configure the switch to Open Fabric mode. 5.
Enabling McDATA Fabric mode When McDATA Fabric mode is turned on, the OUI portion of the switch WWN is no longer replaced with a McDATA OUI. All existing zoning configurations will be cleared. To enable McDATA Fabric mode 1. Verify that you have implemented all the Brocade prerequisites necessary to enable interopmode 2 on the fabric (see ”McDATA Fabric mode configuration restrictions” on page 488.) 2. Connect to the switch and log in as admin. Ensure that the switch is disabled or offline. 3.
Enabling Brocade Native mode When you change the mode from McDATA Fabric or McDATA Open Fabric mode to Brocade Native mode, existing configurations will be erased and the switch must assume the zone configuration from the fabric it joins or a new configuration must be configured. When you change the switch to Brocade Native mode, all configuration parameters return to their default states and can be modified using the configure command.
NOTE: on. Turning off McDATA Enterprise Fabric mode does NOT turn off any of the features that it turned Enabling Fabric Binding using EFCM will automatically enable Insistent Domain ID on all Fabric OS and McDATA switches in the fabric. Disabling Fabric Binding does not turn off Insistent Domain ID. EFCM automates the Fabric Binding configuration process. FCR SANtegrity (Fabric Binding) The support for FCR SANtegrity in this release is for Fabric Binding.
1. On the FCR, enter the portcfgexport command to configure the preferred Domain ID.This preferred Domain ID will become Insistent whenever Fabric Binding is enabled. If the port is not already set to McDATA Fabric mode, this command may also be used to set it. 2. Enable the EX_Port configured in the previous step. 3. Use EFCM to create the Fabric Binding list and to enable Fabric Binding. NOTE: The front port preferred Domain ID will behave as insistent while Fabric Binding is enabled.
Table 109 Hot Code upgrade considerations Fabric OS Versions Notes Upgrading from any other down-level release • Must upgrade to Brocade Native mode, and then change the interopmode; cannot be in McDATA Open Fabric mode before the upgrade. • The upgrade is disruptive or has the potential to be disruptive. Upgrade to higher release than Fabric OS 6.0 or Patch release of Fabric OS 6.0 • Will be nondisruptive Activating Hot Code Load 1. Enter the firmwaredownload command without any option.
C Understanding legacy password behavior This appendix provides password information for early versions of Fabric OS firmware. Password management information Table 110 describes the password standards and behaviors between various versions of firmware. Table 110 Account/password characteristics matrix Topic 4.0.0 4.1.0 to 4.2.0 4.4.0 and later Number of default accounts on the switch 4, chassis-based Core Switch 2/64 8 for the director, 4 per switch. All other switches and directors - 4.
Table 110 Account/password characteristics matrix (continued) Topic 4.0.0 4.1.0 to 4.2.0 4.4.0 and later Can passwd change higher-level passwords? For example, can admin change root password? Yes, but will ask for the “old password” of the higher-level account (example “root”). Yes; if users connect as admin, they can change the root, factory, and admin passwords. However, if one connects as user, one can only change the user password. 4.4.0 to 5.1.
Password migration during firmware changes Table 112 describes the expected outcome of password settings when upgrading or downgrading firmware for various Fabric OS versions. Table 112 Password migration behavior during firmware upgrade/downgrade Topic 4.4.0 to 5.0.1 5.0.1 and later Passwords used when upgrading to a newer firmware release for the first time. Default accounts and passwords are preserved. Default accounts and passwords are preserved.
502 Understanding legacy password behavior
D Using Remote Switch This appendix prrovides infromation on the Remote Switch feature. About Remote Switch The Remote Switch feature, which aids in ensuring gateway compatibility, was formerly a licensed feature. Its functionality is now available as part of the Fabric OS standard feature set through the use of the portCfgIslMode command, which is described in ”Linking through a gateway” on page 44.
You may be required to reconfigure the following parameters, depending on the gateway requirements: NOTE: Consult your gateway vendor for supported and qualified configurations. • R_A_TOV: Specify a Resource Allocation Timeout Value compatible with your gateway device. • E_D_TOV: Specify an Error Detect Timeout Value compatible with your gateway device. • Data field size: Specify the maximum Fibre Channel data field reported by the fabric. Verify the maximum data field size the network-bridge can handle.
E Zone merging scenarios Table 114 provides information on merging zones and the expected results. Table 114 Zone merging scenarios Description Switch A Switch B Expected results Switch A has a defined configuration. defined: cfg1: zone1: ali1; ali2 effective: none defined: none effective: none Configuration from Switch A to propagate throughout the fabric in an inactive state, because the configuration is not enabled.
Table 114 Zone merging scenarios (continued) Description Switch A Switch B Expected results Effective configuration mismatch. defined: cfg1 zone1: ali1; ali2 effective: cfg1 zone1: ali1; ali2 defined: cfg2 zone2: ali3; ali4 effective: cfg2 zone2: ali3; ali4 Fabric segments due to: Zone Conflict cfg mismatch Configuration content mismatch.
Table 114 Zone merging scenarios (continued) Description Switch A Switch B Expected results Same default zone access mode settings. defzone: allaccess defzone: allaccess Clean merge — defzone configuration is allaccess in the fabric. Same default zone access mode settings. defzone: noaccess defzone: noaccess Clean merge — defzone configuration is noaccess in the fabric. Effective zone configuration. No effective configuration.
508 Zone merging scenarios
Index A AAA service requests 67 aaaConfig command 69, 77, 79, 80 access browser support 89 changing account parameters 61 control 97 CP blade 72 creating accounts 60 deleting accounts 60 IP address changes 24 log in fails 24 NTP 32 other devices 45 other switches 45 password, changing 26 recovering accounts 62 remote access policies 74 secure, HTTPS 89 secure, SSL 89 SNMP ACL 85 access methods configuration, Fabric Manager 23 configuration, Web Tools 23 accessControl 97 accessing switches and fabrics 88 acc
auto-leveling, FR4-18i blade 173, 181 B backbone fabric ID 218 backbone-to-edge routing 215, 218 backing up a configuration 135 basic card management 196 PID procedure 472 basic connections 45 blocking listeners 88 boot password 80, 81, 83 boot PROM password 80 broadcast zones 391 Brocade Vendor-Specific Attribute 69 browser configuring 93 troubleshooting certificates 94 browser and Java support 89 browser, configuration 93 browsers support 89 C card management 196 certificates activating 92 browser, conf
secPolicyShow 103 slotshow 217 supportsave 244 supportShow 244 userConfig 57 version 217 configuration FICON environment switched point-to-point 262 FICON environment, cascaded 262 high-integrity fabric 265 save to a host 135 settings, FICON environment 264 configure 95, 98 configuring access control 97 access methods, Fabric Manager 23 access methods, Web Tools 23 authentication 67 browser 93 browser, certificates 93 certificates 89 changing RADIUS servers 80 date and time 30 FibreAlliance MIB 95 FICON env
zone configurations 405 zones 398 deleting end-to-end monitors 348 deleting filter-based monitors 351 designing fabric for trunking 374 deskew 231 deskew values displaying 379 devices, connecting 45 devices, proxy 214 DH-CHAP 219 DHCHAP 220 DH-CHAP secret 220 dictionary.
fddCfg 218 fddCfg command 218 feature licenses 34 Fibre Channel NAT 215 Fibre Channel over IP 219, 417 Fibre Channel routing 211 FICON 323 FICON environment cascaded configuration 262 configuration settings 264 disabling IDID mode 263 displaying link incidents 263 registered listeners for link incidents 267 enabling IDID mode 263 high integrity fabric 262 identifying port swapping nodes 268 monitoring FRU failures 267 node identification data, displaying 267 switched point-to-point configuration 262 switche
K key transaction, for licensed features 34 keys purchasing 39 L legacy FCR switches 241 license ID 34 license key activating 35 licenseadd command 35 licensed features 34 licenseIdShow 34 licenseremove command 36 licenses license ID 34 overview 34 purchasing keys 39 remove feature 36 transaction key 34 link incidents displaying in a FICON environment 263, 267 linking through a gateway 45 Linux, configuring RADIUS on 72 listing link characteristics 381 local account disabling 61 enabling 61 local authentic
recovery string 81 rules 62 set PROM 81, 82 password expiration policy 65 password management information 495 password migration during firmware changes 497 password policies 63 password prompting behaviors 496 password recovery options 497 password strength policy 64 passwords recovering forgotten passwords 84 perfaddeemonitor command 346 perfaddIPmonitor command 349 perfaddusermonitor command 350 perfcfgrestore command 361 perfcfgsave command 361 perfdeleemonitor command 348 perfdelfiltermonitor command 3
end-to-end monitors 348 filter-based monitors 351 licensed feature 36 members from a zone configuration 405 zone members 398 renaming Admin Domains 156 resolving zone conflicts 412 restoring a configuration 137 restoring a segmented fabric 307 restoring configurations in a FICON environment 140 restoring monitor configuration 361 restoring the system configuration settings 137 Role-Based Action Control. See RBAC.
certificates, security 85 SSL protocol configuring 89 standard filter-based monitors 349 standard trunking criteria 374 standby CP blade 72 static PID mapping errors 464 static route 204 storage-based zoning 384 Subscriber’s choice, HP 22 summary of PID formats 463 summary of SSL procedures 90 support FC router 107 Java version 89 SNMPv3 and v1 95 supported optional licensed features 482 Supported Services 118 supportSave command 244 supportsave command 298 supportShow command 244 swapping port area IDs 476
certificates 89 using dynamic load sharing 205 using FICON CUP 268 using legacy commands for SNMPv1 98 using the snmpconfig command 96 using zoning to administer security 412 V validating Admin Domain members 158 VC_RDY 369 VE_Port 322 verification check 217 verify device connectivity 45 high availability (HA) 46 version command 217 VEX_Port 417 viewing alias 396 fan status 293 port status 289 power supply status 293 temperature status 293 zones 398 viewing and saving diagnostic information 298 viewing equ
