53-1002748-01 14 December 2012 Fabric OS FCIP Administrator’s Guide Supporting Fabric OS v7.1.
Copyright © 2009-2011 Brocade Communications Systems, Inc. All Rights Reserved. Brocade, the B-wing symbol, BigIron, DCX, Fabric OS, FastIron, NetIron, SAN Health, ServerIron, and TurboIron are registered trademarks, and Brocade Assurance, Brocade NET Health, Brocade One, CloudPlex, MLX, VCS, VDX, and When the Mission Is Critical, the Network Is Brocade are trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries.
Document History Title Publication number Summary of changes Date Fabric OS FCIP Administrator’s Guide 53-1001349-01 New document. July 2009 Fabric OS FCIP Administrator’s Guide 53-1001349-02 Various changes and corrections. October 2009 Fabric OS FCIP Administrator’s Guide 53-1001755-01 New document for Fabric OS January 2010 version 6.3.1. Fabric OS FCIP Administrator’s Guide 53-1001766-01 New document for Fabric OS March 2010 version 6.4.0.
iv Fabric OS FCIP Administrator’s Guide 53-1002748-01
Contents About This Document How this document is organized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Supported hardware and software . . . . . . . . . . . . . . . . . . . . . . . . . . . ix What’s new in this document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x Document conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Text formatting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
FCIP Trunking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Design for redundancy and fault tolerance . . . . . . . . . . . . . . . . 17 FCIP tunnel restrictions for FCP and FICON acceleration features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 FCIP circuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 FCIP circuit failover capabilities . . . . . . . . . . . . . . . . . . . . .
Modifying an FCIP tunnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Modifying an FCIP circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Deleting an IP interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Deleting an IP route . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Deleting an FCIP tunnel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
viii Fabric OS FCIP Administrator’s Guide 53-1002748-01
About This Document • How this document is organized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix • Supported hardware and software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix • What’s new in this document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x • Document conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi • Notice to the reader . . . . . . . . . . . . . . . . . . .
What’s new in this document Major new additions or deletions in this document support the following: • General - Removed all mention of FR4-18i blade since it is not supported as of this release. • Preface. - Removed FR4-18i from “Supported hardware and software” on page ix.
• Chapter 3 - Removed chapter 3 from previous release, titled “FCIP on the FR4-18i blade.” Chapter 3 now becomes “FCIP Management and Troubleshooting.” - Removed “portCmd --ipperf” under “WAN performance analysis tools” on page 66 since this is a tool specific to FR4-18i blade. Also, the command in last bullet in this section was changed to portShow fcipTunnel --perf.
Command syntax conventions Command syntax in this manual follows these conventions: command Commands are printed in bold. --option, option Command options are printed in bold. -argument, arg Arguments. [] Optional element. variable Variables are printed in italics. In the help pages, variables are underlined or enclosed in angled brackets < >. ... Repeat the previous element, for example “member[;member...]” value Fixed values following arguments are printed in plain font.
Key terms For definitions specific to Brocade and Fibre Channel, see the technical glossaries on MyBrocade. See “Brocade resources” on page xiii for instructions on accessing MyBrocade. For definitions of SAN-specific terms, visit the Storage Networking Industry Association online dictionary at: http://www.snia.org/education/dictionary Notice to the reader This document may contain references to the trademarks of the following corporations.
http://www.t11.org For information about the Fibre Channel industry, visit the Fibre Channel Industry Association website: http://www.fibrechannel.org Getting technical help Contact your switch support supplier for hardware, firmware, and software support, including product repairs and part ordering. To expedite your call, have the following information available: 1.
Document feedback Quality is our first concern at Brocade and we have made every effort to ensure the accuracy and completeness of this document. However, if you find an error or an omission, or you think that a topic needs further development, we want to hear from you. Forward your feedback to: documentation@brocade.com Provide the title and version number of the document and as much detail as possible about your comment, including the topic heading and page number and your suggestions for improvement.
xvi Fabric OS FCIP Administrator’s Guide 53-1002748-01
Chapter 1 FCIP Overview • FCIP platforms and supported features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 • FCIP concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 • IP WAN network considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1 FCIP concepts TABLE 1 FCIP capabilities by platform (Continued) Capabilities 7800 switch FX8-24 blade • Enforcement 802.
IP WAN network considerations 1 FC-IP FC-IP FC-2 TCP TCP FC-2 FC-1 IP IP FC-1 FC-0 LINK LINK FC-0 PHY WAN PHY To Fibre Channel To Fibre Channel FCIP Tunnel FIGURE 1 FCIP tunnel concept and TCP/IP layers Each FCIP tunnel is assigned to a single VE_Port and supports a single circuit. The circuit provides the link for traffic between the source and destination IP addresses on either end of the tunnel. Since a switch can support multiple VE_Ports, multiple FCIP tunnels can be created.
1 4 IP WAN network considerations Fabric OS FCIP Administrator’s Guide 53-1002748-01
Chapter FCIP on the 7800 Switch and FX8-24 Blade 2 In this chapter • 7800 switch hardware overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 • 7800 switch license options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 • FX8-24 blade hardware overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 • FX8-24 blade license options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 7800 switch hardware overview 7800 switch hardware overview Figure 2 shows the FC ports and GbE ports on the 7800 switch. There are 16 FC ports, numbered 0 through 15. The FC ports can operate at 1, 2, 4, or 8 Gbps. There are six GbE ports, numbered 0 through 5. Ports 0 and 1 are available as either RJ-45 ports or small form factor pluggable (SFP) transceiver ports. Only six total GbE ports can be used. The 6 GbE ports together can provide up to 6 Gbps total bandwidth.
7800 switch license options 2 7800 switch license options Some of the capabilities of the Brocade 7800 switch require the following feature licenses, as described in Table 2.
2 FX8-24 blade hardware overview VE_Ports and FCIP tunnels on the 7800 switch A 7800 switch can support eight VE_Ports. VE_Ports are numbered from 16 through 23. Each FCIP tunnel is identified with a VE_Port number. Up to eight FCIP tunnels can be created. The 7800 switch supports VEX_Ports to avoid the need to merge fabrics. Consider the following when using tunnels and VE_Ports: • On a 7800, the total bandwidth limit is 6 Gbps for VE_Ports.
FX8-24 blade hardware overview 6 2 7 2 3 1 5 4 1 10GbE ports (Labeled xge0 and xge1 on the sticker.
2 FX8-24 blade license options Removing FX8-24 blades ATTENTION If you are permanently removing a blade from a DCX, DCX-4S, DCX 8510-8, or DCX 8510-4 chassis to relocate to another slot in the chassis or you are removing the blade from the chassis entirely, you must follow these procedures before removing the blade. • Remove all FCIP configuration settings for the blade. If there are residual configuration settings, they may cause issues with future configurations and upgrades.
FX8-24 blade license options 2 VE_Ports and FCIP tunnels on the FX8-24 blade An FX8-24 blade can support 20 VE_Ports, and therefore 20 FCIP tunnels. There are two VE_Port groups, numbered 12 through 21 and 22 through 31. Each FCIP tunnel is associated with a specific VE_Port. VE_Ports do not have to be associated with a particular GbE port on FX8-24 blades and the 7800 switch.
2 FX8-24 blade license options 10 GbE port considerations Enhanced 10GbE port operation is different than 1 GbE port operation and requires special considerations when configuring circuits, tunnels, failover operations, and bandwidth. Multigigabit circuits For each 10 GbE port, you can configure multigigabit circuits. For example, a single 10 Gbps circuit or two 5 Gbps circuits can be configured per port. A limit of 10 FCIP circuits can be configured on a single port.
FX8-24 blade license options 2 Back-end bandwidth Back-end port bandwidth allocation is calculated as follows: • Back-end bandwidths are always rounded up to the nearest 1 Gbps. For example, 1.5 Gbps actually consumes 2 Gbps of back-end bandwidth. • Each VE_Port group is allocated 10 Gbps of back-end bandwidth (10 Gbps for the VE_Port 12-21 group and 10 Gbps for the VE_Port 22-31 group).
2 FX8-24 blade license options or portcfg ipif 8/xge0 create 192.168.11.20 255.255.255.0 1500 –x Delete the crossport address using the delete option instead of the create option for the portcfg ipif command. portcfg ipif 8/xge0 delete 192.168.11.20 255.255.255.0 1500 –x NOTE If the crossport or x option is not specified and the address is on the crossport, the command will fail with an unknown IP address.
FX8-24 blade license options 2 For more information on configuring an IP route, refer to “Configuring an IP route” on page 41. For more information on using Fabric OS commands, optional arguments, and command output refer to the Fabric OS Command Reference Manual. NOTE If an XGE port has both regular and crossport addresses configured on it, and they use the same IP route, then two routes will need to be configured—a regular route and an identical route on the cross port.
2 FCIP Trunking Using traceroute with crossports You can trace a route to a crossport address, as in the following example. Note that if the crossport or x options are not specified and the address is on the crossport, the portcmd command will fail with an unknown IP address.The command will also fail if the x option is specified and the address is not on the crossport. portcmd --traceroute 8/xge0 -s 192.168.11.20 -d 1.1.1.1 –x or portcmd --traceroute 8/xge0 -s 192.168.11.20 -d 1.1.1.
FCIP Trunking 2 WAN IP Router 10.0.0.1 IP Router 10.0.1.1 FCIP Circuits 10.0.0.2 10.0.0.3 10.0.0.4 10.0.0.5 FIGURE 4 FCIP Circuits FCIP Tunnel 10.0.1.2 10.0.1.3 10.0.1.4 10.0.1.5 FCIP tunnel and FCIP circuits Design for redundancy and fault tolerance Multiple FCIP tunnels can be defined between pairs of 7800 switches or FX8-24 blades, but doing so defeats the benefits of a multiple circuit FCIP tunnel.
2 FCIP Trunking • When configuring tunnels to support large numbers of devices, consider memory limitations of the Brocade 7800 switch and FX8-24 blade if you are enabling any type of emulation feature, such as FCP or FICON. If too many devices are present or activated at one time, emulation operations can be negatively impacted. Refer to “Memory use limitations for large-device tunnel configurations” on page 35.
FCIP Trunking - A limit of 10 FCIP circuits can be configured on a single 10 GbE port. - For a FX8-24 blade with a VE_Port group on a 10GbE port, the sum of the maximum committed rates of that group's circuits cannot exceed 10 Gbps. 2 A limit of 20 FCIP circuits can be configured per VE port group (12 through 21 or 22 through 31) when using a 10G port.
2 FCIP Trunking Circuit 1 - Metric 0 - Active 7800 7800 Circuit 2 - Metric 1 - Standby FIGURE 6 Failover to a higher metric standby circuit 10GbE Lossless Link Loss Circuit failover is supported between 10GbE circuits on FX8-24 blades when both 10GbE ports are on the same logical switch and are operating in 10 Gbps mode. Besides configuring secondary circuits for failover, you can configure a set of IP addresses for circuit failover on crossports.
FCIP Trunking 2 Active-active configuration The following examples shows an active-active configuration in which two circuits are configured with the same metric, one circuit going over xge0 and the other circuit going over the crossport using xge1 as the external port. The metric values of both the circuits are the same (default value), so both circuits send data. The load is balanced across these circuits. Effective bandwidth of the tunnel in this example is 2 Gbps.
2 FCIP Trunking 5. Display local and crossport interface details for xge0. portshow ipif 8/xge0 NOTE If the source and destination addresses are on different subnets, you must configure IP routes to the destination addresses. Refer to “Configuring an IP route” on page 41. NOTE For more information on using Fabric OS commands, optional arguments, and command output refer to the Fabric OS Command Reference Manual.
Adaptive Rate Limiting 2 Adaptive Rate Limiting Adaptive Rate Limiting (ARL) is performed on FCIP circuits to change the rate in which the FCIP tunnel transmits data through the IP network. ARL uses information from the TCP connections to determine and adjust the rate limit for the FCIP circuit dynamically. This allows FCIP connections to utilize the maximum available bandwidth while providing a minimum bandwidth guarantee.
2 PP-TCP-QoS priorities over an FCIP trunk • QoS medium - The default value is 30 percent of the available bandwidth. • QoS low - The default value is 20 percent of the available bandwidth. QoS priority is based on the VC (Virtual Circuit) that carries data into the FCIP Engine. For example, if data enters on a high VC, it is placed on a high TCP connection; if it enters on a low VC then it is placed on the low TCP circuit. Data is assigned to the proper VC based on zone name prefix.
QoS, DSCP, and VLANs 2 • QoS priority settings must be the same on each end of the tunnel. NOTE Priorities are enforced only when there is congestion on the network. If there is no congestion, all traffic is handled at the same priority. Following are some examples of setting QoS priority levels on VE_Port 12: • The following command sets the QoS high priority to 60 percent. portcfg fciptunnel 1/12 create --qos-high 60 • The following command sets the QoS medium priority to 30 percent.
2 QoS, DSCP, and VLANs DSCP settings are useful only if IP routers are configured to enforce QoS policies uniformly within the network. IP routers use the DSCP value as an index into a Per Hop Behavior (PHB) table. Control connections and data connections can be configured with different DSCP values. Before configuring DSCP settings, determine if the IP network you are using implements PHB, and consult with the WAN administrator to determine the appropriate DSCP values.
QoS, DSCP, and VLANs TABLE 4 2 Default mapping of DSCP priorities to L2CoS priorities (Continued) DSCP priority/bits L2CoS priority/bits Assigned to: 59 / 111011 4 / 100 High QoS 63 / 111111 0 / 000 Reserved DSCP and VLAN support on FCIP circuits When VLAN tag is created on an FCIP circuit, all traffic over that circuit will use the specified VLAN.
2 QoS, DSCP, and VLANs The following example creates an additional FCIP circuit with a different VLAN tag. switch:admin> portcfg fcipcircuit 16 create 1 192.168.2.21 192.168.2.11 100000 -v 200 Operation Succeeded The following example shows the fcipcircuit modify command that changes the VLAN tag and L2CoS levels for circuit 0. Parameters are the same for both the create and modify options.
Compression options 2 The following example adds an entry that tags all frames from IP address 192.168.10.1 destined for IP address 192.168.20.1 with a VLAN ID of 100, and a L2CoS value of 3. switch:admin> portcfg vlantag 8/ge0 add 192.168.10.1 100 3 192.168.20.1 The following example adds an entry that tags al frames from a crossport with local address 192.168.11.20, VLAN ID of 200, and a LSCoS value of 1. switch:admin> portcfg vlantag 8/xge0 add 192.168.11.20 200 1 –x 3.
2 IPsec implementation over FCIP tunnels IPsec implementation over FCIP tunnels Internet Protocol security (IPsec) uses cryptographic security to ensure private, secure communications over Internet Protocol networks. IPsec supports network-level data integrity, data confidentiality, data origin authentication, and replay protection. It helps secure your SAN against network-based attacks from untrusted computers. The following describes the sequence of events that invokes the IPsec protocol. 1.
IPsec implementation over FCIP tunnels 2 • IKEv2 uses UDP port 500 to communicate between the peer switches or blades. • All IKEv2 traffic is protected using AES-GCM-ESP encryption. • Authentication requires the generation and configuration of 32-byte pre-shared secrets for each tunnel. • An SHA-512 hash message authentication code (HMAC) is used to check data integrity and detect third-party tampering. • Pseudo-random function (PRF) is used to strengthen security.
2 Open Systems Tape Pipelining portcfg fciptunnel 17 create 192.168.0.91 -K12345678901234567890123456789012 -l portcfg fcipcircuit 17 create 1 192.168.1.91 portcfg fcipcircuit 17 create 2 192.168.2.91 portcfg fcipcircuit 17 create 3 192.168.3.91 portcfg fcipcircuit 17 create 4 192.168.4.91 portcfg fcipcircuit 17 create 5 192.168.5.91 192.168.0.81 50000 -x 0 -d c0 -i 192.168.1.81 192.168.2.81 192.168.3.81 192.168.4.81 192.168.5.
2 Open Systems Tape Pipelining T0 H1 FCIP tunnel FW=1, TA=1 H2 FC SAN .. . GE0 GE1 Hn T1 FC SAN GE0 Connection can be VE-VE or VEX-VE GE1 Ta Hn 172.0.1.
2 Support for IPv6 addressing In some cases, traffic isolation zoning (TIZ) or VF LS/LF configurations may be used to control the routing of SID/DID pairs to individual tunnels. This provides deterministic flows between the switches and allows the use of ECMP. Refer to the Fabric OS Administrator’s Guide for more information about TIZ. Support for IPv6 addressing The IPv6 implementation is a dual IP layer operation implementation as described in RFC 4213.
Memory use limitations for large-device tunnel configurations 2 • The Neighbor Discovery ICMPv6 Solicitations and Advertisements are transmitted to the Layer 2 Ethernet multicast MAC address derived from the IPv6 source address (RFC 2464). • ICMPv6 message types in RFC 4443 and ICMPv6 message types used for Neighbor Discovery are supported. • Path MTU Discovery (RFC 1981) is not supported on this implementation, requiring static configuration of MTU size.
2 Memory use limitations for large-device tunnel configurations Control blocks created during FCP traffic flow For FICON traffic flows, FCIP tunnel processing creates control block structures based upon the SID/DID pairs called a FICON device port, path block (FDPB). If any FICON emulation feature is enabled, additional control blocks are created for each SID/DID pair, LPAR number (FCHB structure), LCU Number (FCUB structure) and for each individual FICON device address on those LCUs (FDCB structure).
Configuration preparation 2 NOTE The 7800 is expected to support no more than about 120,000 extended device images (FICON or FCP). A single FX8-24 DP is expected to support no more than 160,000 extended device images. Configuration preparation Before you begin to configure FCIP, do the following: • Determine the amount of bandwidth that will be required for the RDR, FICON, or tape application to be deployed. • • • • • • The WAN link has been provisioned and tested for integrity.
2 Configuration steps Configuration steps The following is a list of the major steps for configuring FCIP on the 7800 switch or FX8-24 blade: • • • • • • • Persistently disable VE_Ports. If required, configure VEX_Ports. For the 7800 switch, set the media type for GbE ports 0 and 1. For the FX8-24 blade, set the GbE or XGE port operating mode. Assign IP addresses to the GbE or XGE ports using the portCfg ipif command. Create one or more IP routes using the portCfg iproute command.
Configuration steps 2 You can determine if fmsmode is enabled by using the ficoncupshow fmsmode command. Configuring VEX_Ports If you are going to use a VEX_Port in your tunnel configuration, use the portCfgVEXPort command to configure the port as a VEX_Port. VEX_Ports can be used to avoid merging fabrics over distance in FCIP implementations. If the fabric is already connected, disable the GbE ports and do not enable them until after you have configured the VEX_Port.
2 Configuration steps Port ge1 is configured in optical mode Setting the GbE port operating mode (FX8-24 blade only) The GbE ports on an FX8-24 blade can operate in one of three ways: • 1 Gbps mode. GbE ports 0 through 9 may be enabled as GbE ports, with the XGE ports disabled. The 10GbE (FTR_10G) license is not required. • 10 Gbps mode. 10GbE ports xge0 and xge1 may be enabled, with GbE ports 0 through 9 disabled.
2 Configuration steps The following command creates an IP interface for port ge0 on the Brocade 7800 switch. switch:admin> portcfg ipif ge0 create 192.168.1.78 255.255.255.0 1500 Use the following command to display current configuration details for all interfaces. portshow ipf all NOTE For full details on syntax and using this command, refer to the Fabric OS Command Reference Manual.
2 Configuration steps Storage Server Brocade DCX-4S with FX8-24 Blade Gateway 192.168.1.1 Gateway 192.168.11.1 WAN VE_Port 192.168.1.24 FIGURE 11 Brocade 7800 VE_Port 192.168.11.78 Configuring an IP route For information on configuring IP routes using crossport addresses, refer to “Configuring IP routes with crossports” on page 14.
Configuration steps 2 The following command creates the 7800 end of the tunnel. VE_Port 16 is specified. Circuit parameters are included to create circuit 0 on the 7800. The circuit parameters must match up correctly with the circuit parameters on the FX8-24 end of the circuit. The FX8-24 destination address is specified first, followed by the 7800 switch source address. Matching ARL minimum and maximum committed rates must be specified on both ends of circuit 0.
2 Configuration steps TABLE 7 . Tunnel options Option Arguments Disruptive Description Compression Short option: -c Long option: --compression Operands: 0|1|2|3|4| Yes Enables compression on an FCIP tunnel. Compression is set by the portCfg fciptunnel create or modify command, and applies to traffic over all circuits in the tunnel. Compression cannot be set or modified by the portCfg fcipcircuit create or modify command. The argument values have the following meanings.
Configuration steps TABLE 7 2 Tunnel options (Continued) Option Arguments Disruptive Description Remote FC WWN Short Option: -n Long Option: --remote-wwn Yes This is a fabric security feature that allows you to only allow the FCIP tunnel to come up when the correct remote WWN is entered. If the WWN of the remote side does not match the value entered here, the FCIP tunnel will not initiate.
2 Configuration steps TABLE 8 . Circuit options Option Argument Disruptive Description Committed rate committed rate Yes Create behavior: Sets the minimum and maximum committed rate to the value specified for committed rate. Short option: -b and-B This option may be used on a portcfg fciptunnel create command or on the portcfg fcipcircuit create command to set a committed rate for an FCIP circuit.
Configuration steps TABLE 8 2 Circuit options (Continued) Option Argument Disruptive Description Selective acknowledgement Short option: -s Long option: --sack Operands (modify only): 0|1 Yes Disables or enables selective acknowledgement. Selective acknowledgement allows a receiver to acknowledge multiple lost packets with a single ACK response. This results in better performance and faster recovery time. Selective acknowledgement is initially turned on.
2 Configuration steps TABLE 8 Circuit options (Continued) Option Argument Disruptive Description Specify connection type Short option: -C Long option: --connection-type Operands: default|listener|initiator Yes Allows you to specify which side of the circuit is the listener or initiator. If this is not specified, the initiator or listener are automatically selected based on the lower and higher-order IP address.
Configuration steps 2 Creating additional FCIP circuits If the Advanced Extension (FTR_AE) license is enabled, additional FCIP circuits can be created and added to an FCIP tunnel (VE_Port) using the portCfg fcipcircuit create command. The following examples add a circuit to the tunnel in the basic sample configuration (refer to Figure 12 on page 43). The following command creates circuit 1 on the FX8-24 end of the tunnel. switch:admin> portcfg fcipcircuit 8/12 create 1 192.168.11.79 192.168.1.
2 Creating a multicircuit tunnel (example) Enabling persistently disabled ports It is strongly recommended to disable ports while they are being configured to prevent unwanted fabric merges. 1. Connect to the switch and log in using an account assigned to the admin role. 2. Enter the portCfgShow command to view ports that are persistently disabled. 3. After identifying the ports, enter the portCfgPersistentEnable command to enable the ports. 4.
Creating a multicircuit tunnel (example) 2 • “Creating an FCIP tunnel” on page 42 • “Creating additional FCIP circuits” on page 49 To create a tunnel between two switches or blades, you must first understand the IP network infrastructure between the sites. Each circuit requires a pair of IP interface addresses (either IPv4 or IPv6). Therefore, to create an FCIP Tunnel with six circuits, you need 12 IP addresses: six for the site A switch and six for the site B switch.
2 Creating a multicircuit tunnel (example) Site B portcfg fciptunnel 16 create --ficon -c 1 • To use this tunnel for FCP with Fastwrite and Open Systems Tape Pipelining traffic, and hardware compression, create it using the following commands.
Creating a multicircuit tunnel (example) 2 Site B portcfg fcipcircuit 16 create 0 192.168.0.63 192.168.0.64 -b 1000000 -B 1000000 portcfg fcipcircuit 16 create 1 192.168.1.63 192.168.1.64 -b 1000000 -B 1000000 portcfg fcipcircuit 16 create 2 192.168.2.63 192.168.2.64 -b 1000000 -B 1000000 portcfg fcipcircuit 16 create 3 192.168.3.63 192.168.3.64 -b 1000000 -B 1000000 portcfg fcipcircuit 16 create 4 192.168.4.63 192.168.4.64 -b 1000000 -B 1000000 portcfg fcipcircuit 16 create 5 192.168.5.63 192.168.5.
2 Modifying an FCIP tunnel Modifying an FCIP tunnel FCIP tunnel characteristics and options can be modified as needed, using the portCfg fcipTunnel command with the modify option. The command syntax is as follows: portCfg fciptunnel ve_port modify Where: ve_port Each tunnel is assigned to a specific VE_Port. The VE_Port number serves as the tunnel ID. The range is 16 through 23 for a 7800 switch and 12 through 31 for the FX8-24 blade.
Deleting an IP interface 2 Deleting an IP interface You can delete an IP interface using the portcfg ipif command with the delete option. The command syntax is as follows: portcfg ipif [slot/]ge n delete ipaddr For full details on syntax and using this command, refer to the Fabric OS Command Reference Manual. NOTE You cannot delete an IP interface if there is a tunnel or circuit configured to use it. Be sure to delete all tunnels, circuits, and IP routes using an interface before deleting it.
2 Deleting an FCIP circuit Deleting an FCIP circuit You can delete individual FCIP circuits using the portCfg fcipcircuit command with the delete option. The command syntax is as follows: portcfg fcipcircuit ve_port delete circuit_id For full details on syntax and using this command, refer to the Fabric OS Command Reference Manual. Virtual Fabrics The 1GbE ports (7800 switch and FX8-24 blade), 10GbE ports (FX8-24 blade only), and VE_Ports can be part of any logical switch.
Virtual Fabrics 2 • Logical switch 1 has VE17, which has a circuit over GbE0 • Logical switch 2 has VE18, which also has a circuit over GbE0 All of the committed-rate restrictions and bandwidth sharing of the GbE ports for ARL remain the same for shared ports in the logical switches. VE_Ports created from shared GbE ports initiate as regular VE ISLs in their respective logical switches.
2 58 Virtual Fabrics Fabric OS FCIP Administrator’s Guide 53-1002748-01
Chapter 3 FCIP Management and Troubleshooting In this chapter • Inband management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • WAN performance analysis tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • Portshow command usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • FCIP tunnel issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • FCIP links . . . . .
3 Inband management IP routing The inband management interfaces are separate from the existing IP interfaces currently used for FCIP. These interfaces exist on the CP and are added and maintained on the CP routing table to ensure end-to-end connectivity. Because this routing table will be shared among all devices on the CP, including the management interface, precautions must be taken to ensure that proper connectivity is maintained.
Inband management 3 7800 L1 Configure the inband management interfaces. portcfg mgmtif ge0 create 192.168.3.10 255.255.255.0 7800 R1 Configure the inband management interfaces. portcfg mgmtif ge0 create 192.168.3.20 255.255.255.0 Management station Access the Brocade 7800 switches through the external inband management station. telnet 192.168.3.
3 Inband management 7800 L1 1. Configure the inband management interfaces. portcfg mgmtif ge0 create 192.168.1.10 255.255.255.0 2. Configure the inband management route for the management station. portcfg mgmtroute ge0 create 192.168.3.0 255.255.255.0 192.168.1.250 7800 R1 1. Configure the inband management interfaces. portcfg mgmtif ge0 create 192.168.2.20 255.255.255.0 2. Configure the inband management route for the management station. portcfg mgmtroute ge0 create 192.168.3.0 255.255.255.0 192.168.
Inband management Management Workstation 3 Router C 192.168.3.250 172.0.1.3 192.168.3.30 192.168.3.31 Router A Router B 172.0.1.1 172.0.1.0 Subnet 172.0.1.2 192.168.1.250 192.168.4.250 192.168.1.10 192.168.4.10 192.168.2.20 192.168.5.21 7800 L1 10.1.1.10 FIGURE 16 7800 R1 10.1.2.20 Redundant connection to management station 7800 L1 1. Configure the inband management interfaces. portcfg mgmtif ge0 create 192.168.1.10 255.255.255.0 portcfg mgmtif ge1 create 192.168.4.10 255.255.255.0 2.
3 Inband management route add 192.168.5.0 netmask 255.255.255.0 gw 192.168.3.250 2. Access the 7800 switches through the external inband management interfaces. telnet 192.168.1.10 VLAN tagging support To add VLAN tag entries to the VLAN tag table for inband management interfaces, use the --mgmt or -m option with the portcfg vlantag command. Complete the following steps: 1. Configure an IP addresses and route for an Inband Management interface using the following command format.
Inband management 3 For this example, you must configure the following: • On the management station: - IP address 10.1.1.1/24 (defined) - IP route to 192.168.3.20/32 via 10.1.1.10 • On the 7800 L1: - CP Management address 10.1.1.10/24 - Inband management address 192.168.3.10/24 - IP filter forward rule with destination IP address 192.168.3.20 • On the 7800 R1: - CP Management address 10.1.2.20/24 - Inband management address 192.168.3.20/24 - Inband management route to 10.1.1.1/32 via 192.168.3.
3 WAN performance analysis tools WAN performance analysis tools WAN analysis tools are designed to test connections, trace routes, and estimate the end-to-end IP path performance characteristics between a pair of Brocade FCIP port endpoints. These tools are available as options on the portCmd command.
WAN performance analysis tools 3 16 0 ge0 Up ---4--s 1h21m34s 0.00 0.00 2 500/500 0 16 1 ge1 Up ---4--s 1h21m43s 0.00 0.
3 WAN performance analysis tools Using ping to test a connection The portCmd --ping command tests the connection between the IP address of a local Ethernet port and a destination IP address. If you want to use this command to test a VLAN connection when you do not have an active FCIP tunnel, you must manually add entries to the VLAN tag table on both the local and remote sides of the route, using portCfg vlantag command.
Portshow command usage 3 Portshow command usage Use the portshow command to display port operational information on Brocade 7800 switches, and FX8-24 blades. The Fabric OS Command Reference Manual provides complete descriptions of portshow command syntax and options. The following sections identify a few specific outputs that may be useful for maintenance and troubleshooting. Displaying IP interfaces The following example displays IP interface information for a 7800 switch.
3 Portshow command usage Displaying performance statistics Display a summary of performance statistics for tunnels and circuits using the circuit, perf, and summary options as in the following example. switch:admin> portshow fciptunnel all --circuit --perf --summary For details of command syntax and output examples, refer to the Fabric OS Command Reference Manual. Displaying QoS statistics Display QoS statistics for tunnels using the qos and summary options as in the following example.
Portshow command usage 3 Displaying FCIP tunnel performance The following example will display performance statistics for tunnel associated with port 17 on a 7800 switch. switch:admin> portshow fciptunnel 17 --perf For details of command syntax and output examples, refer to the Fabric OS Command Reference Manual. Displaying FCIP tunnel TCP statistics The following example will display TCP connections for a tunnel associated with port 17 on a 7800 switch.
3 Portshow command usage Displaying TCP statistics for circuits The following example displays TCP statistics for circuits associated with VE_Port 12 of a FX8-24 blade. switch:admin>portshow fcipcircuit 3/12 --tcp You can reset statistics counters to zero to display only new statistics with the --tcp option from the time you issue the reset using the following command.
FCIP tunnel issues 3 For details of command syntax and output examples, refer to the Fabric OS Command Reference Manual. FCIP tunnel issues The following are common FCIP tunnel issues and recommended actions for you to follow to fix the issue. Symptom FCIP tunnel does not come Online. Probable cause and recommended action Confirm the following steps. 1. Confirm GE port is online. portshow ge1 Eth Mac Address: 00.05.1e.37.93.
3 FCIP links 7. Get a GE Ethernet sniffer trace. Rule out all possible blocking factors. Routers and firewalls that are in the data path must be configured to pass FCIP traffic (TCP port 3225) and IPsec traffic, if IPsec is used (UDP port 500). If possible blocking factors have been rule out, simulate a connection attempt using the portCmd --ping command, from source to destination, and then take an Ethernet trace between the two endpoints.
FCIP links 3 • If an FCIP tunnel fails with the “Disabled (Fabric ID Oversubscribed)” message, the solution is to reconfigure the VEX_Port to the same Fabric ID as all of the other ports connecting to the edge fabric. • Because of an IPsec RASLog limitation, you may not be able to determine an incorrect configuration that causes an IPsec tunnel to not become active. This misconfiguration can occur on either end of the tunnel.
3 FTRACE concepts FTRACE concepts FTRACE is a support tool used primarily by your switch support provider. FTRACE can be used in a manner similar to that of a channel protocol analyzer. FTRACE may be used to troubleshoot problems using a Telnet session rather than sending an analyzer or technical support personnel to the site. CAUTION FTRACE is meant to be used solely as a support tool and should be used only by Brocade support personnel, or at the request of Brocade support personnel.
Index Numerics E 10GbE lossless failover, 20 7800 switch, 6 configuring a GbE port, 40 configuring an IP route, 41 creating and FCIP circuit, 49 extended interswitch link (XISL), 39 A Adaptive Rate Limiting (ARL), 23 B backend bandwidth, 13 bandwidth backend, 13 frontend, 13 C circuit metrics, 19 committed rate, 46 configuration hirarchy, 38 configuring IP routes for crossport addresses, 14 creating a multicircuit FCIP tunnel, 51 creating an FCIP tunnel, 42 crossport active-active configuration, 21 a
G Q GbE port mode on the FX8-24 blade, 40 QoS configuring priority percentages QoS SID/DID priorities, 23 QoS implementation in FCIP, 25 QoS priorities per FCIP circuit, 23 I inband management, 59 configuring IP addresses and routes, 60 IP routing and subnets, 60 VLAN tagging support, 64 IPsec limitations for 7800 and FX8-24, 30 NAT limitation for 7800 and FX8-24, 30 L License requirements 7800 switch, 7 FX8-24 blade, 10 Load leveling and failover, 19 lossless failover, 20 lossless link loss (LLL) defi
