HP Integrity Servers with Microsoft® Windows Server™ 2003 and Windows Server™ 2008 HP Integrity Network Adapter Teaming White Paper HP Part Number: 5992-4497 Published: September 2008
© Copyright 2008 Hewlett-Packard Development Company, L.P. Legal Notices Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Table of Contents 1 Introduction......................................................................................................................9 Abstract...................................................................................................................................................9 How to Read This Document.................................................................................................................9 Section Layout.................................................
Network Adapter Miniport Driver............................................................................................30 Teaming Intermediate Driver.....................................................................................................30 HP Network Configuration Utility............................................................................................30 HP Teaming and Layer 2 Versus Layer 3 Addresses.......................................................................
Dual Channel Applications........................................................................................................78 Recommended Configurations for Dual Channel Environments.............................................78 802.3ad Dynamic Dual Channel Load Balancing............................................................................79 802.3ad Dynamic Dual Channel Configuration.........................................................................79 Automatic...........................
List of Figures 3-1 3-2 3-3 3-4 3-5 3-6 4-1 4-2 4-3 4-4 4-5 4-6 4-7 4-8 4-9 4-10 4-11 4-12 4-13 4-14 4-15 4-16 4-17 4-18 4-19 4-20 4-21 4-22 4-23 4-24 4-25 4-26 4-27 A-1 A-2 6 HP Integrity Network Adapter Teaming......................................................................................15 Using Multi-homing for server network redundancy..................................................................16 Using NIC teaming for server network redundancy.............................................
List of Tables 3-1 3-2 4-1 4-2 4-3 4-4 4-5 4-6 4-7 4-8 4-9 4-10 4-11 4-12 4-13 4-14 4-15 4-16 5-1 Basic versus Advanced teaming features......................................................................................22 Dual Channel capabilities comparison..........................................................................................23 Heartbeat Frame Format...............................................................................................................36 802.
1 Introduction Abstract This white paper provides a high- and low-level discussion of the technology behind HP Integrity Network Adapter Teaming for HP Integrity servers running Microsoft® Windows®. HP Integrity Network Adapter Teaming is software-based technology used by server administrators and network administrators to increase a server’s network availability and performance.
This white paper specifically discusses HP Integrity Network Adapter Teaming for Windows Server 2003 and Windows Server 2008. For technical information on network adapter teaming for other operating systems, refer to www.hp.com.
2 An Executive Overview of Teaming What is HP Integrity Network Adapter Teaming? HP Integrity Network Adapter Teaming is software-based technology used by server administrators and network administrators to increase a server’s network availability and performance. HP Integrity Network Adapter Teaming provides network adapter, network port, network cable, switch, and communication path fault recovery technology, in addition to, transmit and receive load balancing technology.
• administrator to provide a switch port to the server that has 100% uptime. While network administrators can achieve high availability in the network, they can’t promise 100% availability for the switch and switch port to which the server’s network adapter port is connected. Switch failures, like server failures, do occur. Switch misconfigurations, like server misconfigurations, do occur.
today’s servers ship standard with at least two network adapter ports, many built directly onto the motherboard (in other words, LAN On Motherboard or LOM). The SA need only install HP Integrity Network Adapter Teaming software, select at least two network adapter ports, and create a team. Everything else works the same, both inside the server (server’s protocol stack) and outside of the server (the server’s clients).
3 Teaming Fundamentals for the First-Time User A Technical Overview of HP Integrity Network Adapter Teaming HP Integrity Network Adapter Teaming provides fault tolerance and load balancing across a team of two or more network adapter ports. The term “team” refers to the concept of multiple network adapters (teamed ports) working together as a single network adapter, commonly referred to as a virtual network adapter or virtual NIC interface.
Figure 3-2 Using Multi-homing for server network redundancy 1.1.1.3 1.1.1.1 1.1.1.4 1.1.1.2 HP Integrity Server Server has redundant path for outgoing traffic. From the server’s IP perspective, either adapter provides access to the network. 1.1.1.5 Switch Clients How do clients know the server has two or more IP addresses? Which IP address represents the server for DNS/NetBIOS/etc.
Figure 3-3 Using NIC teaming for server network redundancy Integrity Server Teaming Driver 1.1.1.1 Clients 1.1.1.3 Switch Co ns o le 1.1.1.4 1.1.1.5 •Server has redundant path for outgoing traffic. From the Server’s IP perspective, single “virtual” adapter provides access to the network. •Provides redundant connectivity from Clients to Server for same Server IP address. Server is known by single IP for DNS/NetBIOS/etc.
Figure 3-4 Teaming types and functionality Advanced Teaming Basic Teaming Receive Load Balancing Basic Teaming Basic Teaming Transmit Load Balancing Switch Fault Tolerance* Switch Fault Tolerance* Network Adapter Fault Tolerance Network Fault Tolerance Only (NFT) Network Fault Tolerance Only with Preference Order Transmit Load Balancing with Fault Tolerance (TLB) Transmit Load Balancing with Fault Tolerance and Preference Order Switch-assisted Load Balancing with Fault Tolerance (SLB) 802.
(for example, higher receive/transmit descriptors or buffers, interrupt coalescence, etc.), or preference for the team’s Primary port to be located on a specific switch. NFT with Preference Order is a basic feature of HP Integrity Network Adapter Teaming. Refer to “Basic Teaming Versus Advanced Teaming” for more information. Transmit Load Balancing with Fault Tolerance (TLB) Transmit Load Balancing with Fault Tolerance (TLB) is a team type that allows the server to load balance its transmit traffic.
or switch redundancy (NFT, TLB). Dual Channel allows the user to create two teams, called groups, inside of a single team. Each group (A or B) is assigned one or more teamed ports. Also, each group can be connected to a different switch to provide switch fault tolerance. Full inbound and outbound load balancing is provided across both groups. Should all members in one group completely fail, the team remains available via the other group.
Figure 3-5 How to choose the best team type What type of NIC Team do I Need? NIC Teaming is not needed I need NIC Fault Tolerance and/or Load Balancing NO Team Type Automatically set to IEEE 802.3ad YES YES Choose Team Type “Network Fault Tolerant Only” Choose Team Type “Network Fault Tolerant Only with Preference Order” NO I want to manually configure the Team type I need to designate the order in which ports become Primary NO All NICs connected to a single IEEE 802.
Basic Teaming Versus Advanced Teaming You can choose to use the basic HP Integrity Network Adapter Teaming features like NFT, TLB, and SLB, or the advanced teaming features such as Dual Channel, Dynamic Dual Channel, Active Path, and Fast Path. The following table summarizes these Basic and Advanced features. Table 3-1 Basic versus Advanced teaming features Basic Teaming Features NFT √ NFT with Preference √ TLB √ TLB with Preference √ SLB √ 802.
ports have become segregated onto different broadcast domains (because of link loss on an upstream switch uplink, server network adapter ports are connected to the wrong switch ports, VLAN misconfiguration on a switch, etc.) Also, Fast Path is used by HP Integrity Network Adapter Teaming to determine which teamed port is the optimal port to use as the team’s Primary port among one or more teamed ports with validated connectivity to the preferred Spanning Tree root switch.
Launch The NCU can be launched from the tray icon (unless disabled) or from the Control Panel applet labeled HP Network. Overview of the HP Network Configuration Utility HP Integrity Network Adapter Teaming is configured and monitored through a graphical user interface entitled the HP Network Configuration Utility (NCU). NCU is also used for configuring and monitoring individual network adapters.
Team Properties Page The Team Properties page (refer to Figure 3-6) is used to manage and monitor all team-specific settings and consists of the following individual tabs. Team Properties Page Tabs • • • • • • • Teaming Controls – This tab is used to change the team name, select team type, select the Transmit Load Balancing algorithm, change port preference order for NFT with Preference teams, and to assign group membership for Dual Channel teams.
• • Information – This tab is only used for monitoring adapter status information. Examples of information that can be monitored are current teamed status, current speed/duplex, current MAC address, driver version, part number, bus/slot/port information, and so on. Diagnostics – This tab is used for running manual diagnostics on a per-adapter level. The diagnostic tests are manually selected and executed by clicking the Run Tests button.
3. Set the team type to Automatic. a. Click the Properties icon on the NCU main page for the newly formed team (white team icon). b. Under Team Type Selection on the Teaming Controls tab, notice that Automatic (Recommended) is already chosen. This is the default setting for new teams. c. Note the name assigned to the team in the Team Name window (for example, HP Network Team #1). d. OPTIONAL STEP: Rename the team by changing the text in the Team Name window. 4. 4.
4 The Mechanics of Teaming for the Advanced User Section Objectives and Prerequisites This section is intended to provide an in-depth technical discussion of the mechanics of HP Integrity Network Adapter Teaming for the advanced user. This section assumes the reader has read the previous sections in this white paper and has hands-on experience in the deployment and usage of HP Integrity Network Adapter Teaming.
in a single server, there will be more than one instance of the HP network adapter team and an individual protocol address will be assigned to each instance. Teaming Software Components HP Integrity Network Adapter Teaming consists of three components: the Miniport driver, Intermediate driver, and configuration GUI. Network Adapter Miniport Driver For Microsoft Windows Server 2003, the Miniport driver used with the HP network adapter will be e1000645.sys or b57xp64.sys, depending on the adapter in use.
When a failover event occurs, the MAC addresses of the current Primary adapter and one of the Non-Primary adapters are swapped. The former Non-Primary adapter becomes the new Primary adapter and the former Primary adapter becomes a Non-Primary adapter. By swapping the MAC addresses in this manner, the HP network adapter team is always known by one MAC address and one protocol address.
Figure 4-1 Overview of NFT communication ARP Table: Destination MAC/Source MAC/Destination IP/Source IP ARP Table: 1.1.1.2 = A 1.1.1.3 = B 1.1.1.4 = C 1.1.1.5 = D E A 1.1.1.1 1.1.1.2 E B 1.1.1.1 1.1.1.3 E C 1.1.1.1 1.1.1.4 E D 1.1.1.1 1.1.1.5 Client A IP Address = 1.1.1.2 MAC = A Client B Destination MAC/Source MAC/Destination IP/Source IP A E 1.1.1.2 1.1.1.1 B E 1.1.1.3 1.1.1.1 C E 1.1.1.4 1.1.1.1 D E 1.1.1.5 1.1.1.1 IP Address = 1.1.1.
Figure 4-2 Scenario 4–A: A device pings an NFT team on the same Layer 2 network Ethernet MAC = A IP = 1.1.1.1 MAC = B Primary MAC = E Non-Primary Red NFT IP = 1.1.1.2 Blue (server) 1. Red transmits a broadcast ARP request asking for Blue’s MAC address. A user on Red issues the command ping 1.1.1.2 to initiate a ping to Blue. First, Red determines whether or not Blue is on the same Layer 2 network.
4. Blue transmits a broadcast ARP request asking for Red’s MAC address. NOTE: The following step may not occur if Blue’s ARP table still contains an entry for Red as a result of Steps 1 and 2. Blue checks its ARP cache for a MAC address entry that matches 1.1.1.1. If Blue does not find one, then Blue broadcasts an ARP request asking for Red’s MAC address. 5. Red transmits a unicast ARP reply to Blue providing its MAC address. NOTE: The following step will not occur if Step 4 does not take place.
• teamed port under test is able to receive the heartbeat frame, the test succeeds. This feature can be disabled, but fault tolerance will be reduced. Switch MAC table updates with team MAC address – The Primary Port must ensure that the switch has the team’s MAC address on the correct port, and that the switch keeps the team’s MAC address in its MAC table [or Content Addressable Memory (CAM) table].
frame. Therefore, a transmit path validation heartbeat frame may be transmitted once every 6 seconds (if the transmit path validation interval timer is set to the default of 3 seconds) on a port that has not transmitted anything. If the port does successfully transmit a heartbeat or any other kind of frame, its internal status is reset to 0.
Table 4-2 802.1Q Tagged Heartbeat Frame Format (continued) Source MAC address “MAC address of Teamed port” 6 bytes 802.
For example, refer to the diagram of an NFT team in Figure 4-4. The HP Integrity server’s Primary team member port is connected to Switch A. Switch A’s uplink to the Core Switch has been unplugged/disconnected. This results in the HP Integrity server being effectively disconnected from the entire network except for Switch A.
probes and receive responses on each individual member port. Additionally, Active Path must accomplish this without using the team’s IP address. The team’s IP address cannot be used since the Echo Node will always resolve the team’s IP address to a single port in the team, which would prevent Echo Node probe responses from being sent to every port in the team. Active Path uses a special frame as the Echo Node probe.
Node and a failure recovery would not be possible. Second, the frame type had to be supported by as many IP protocol stacks as possible, so no special configuration or modifications to the designated Echo Node were required. Third, a frame type had to be chosen that would not cause loss of communication for normal data traffic between the Echo Node and the team since, in many environments, the Echo Node could be another server, another client, a switch, or a router functioning as the team’s gateway.
Table 4-4 Directed ARP Echo Node Probe REPLY Frame Format (continued) Protocol address length 0x04 1 byte ARP operation 0x002 (ARP response) 2 bytes Source MAC address “Echo Node’s MAC Address” 6 bytes Source IP address “Echo Node’s IP Address” 4 bytes Destination MAC address “MAC address of Teamed port” 6 bytes Destination IP address 0.0.0.
and all members in a particular team. This is accomplished by the Echo Node transmitting a single Echo Node probe reply that is received by all teamed ports configured to monitor the particular Echo Node. In contrast to Directed ARP, each individual teamed port is not necessarily required to transmit its own Echo Node request, yet all teamed ports individually receive an Echo Node probe reply.
1. 2. 3. Validate individual connectivity for a teamed port There is no special configuration on the designated Echo Node The Echo Node probe frame can’t cause loss of connectivity for the team Community Address ARP was designed to work with IP stacks that fail to respond to the Directed ARP probe packets. Community Address ARP allows practically any network device to function as the Echo Node. Although Directed ARP works with most IP stacks, certain routers fail to respond to the Directed ARP probes.
Table 4-6 Community Address ARP Echo Node Probe REPLY Frame Format (continued) Protocol address length 0x04 1 byte ARP operation 0x002 (ARP response) 2 bytes Source MAC address “Echo Node’s MAC Address” 6 bytes Source IP address “Echo Node’s IP Address” 4 bytes Destination MAC address “Community Probe MAC from GUI” 6 bytes Destination IP address “Community Probe IP from GUI” 4 bytes Checksum “varies” 4 bytes Community Address ARP’s Designated Echo Node: An Echo Node is any network device
Active Path Configuration To use Active Path, an SA must enable it on a per-team basis from the Advanced Redundancy tab located on the HP Integrity Network Adapter Teaming configuration GUI. Next, the SA should select which Active Path mechanism to use – Directed ARP or Community Address ARP from the Echo Node Response Mechanism drop-down box. For both Directed ARP and Community Address ARP, an IP address for an appropriate Echo Node must be provided in the Echo Node IP Address field.
Figure 4-5 Active Path failover configuration: Directed ARP Community Address ARP Configuration: To enable Active Path’s Community Address ARP mechanism, select Community Address ARP from the Echo Node Response Mechanism drop-down box. In addition to supplying the Echo Node IP address, it is necessary to supply a Community Probe IP Address. Community Address ARP guidelines: • Use the same Echo Node, Community Probe IP, and Community Probe MAC address for as many servers as possible on the same subnet.
Figure 4-6 Active Path failover configuration: Community Address ARP Fast Path Fast Path is a mechanism used by HP Integrity Network Adapter Teaming to intelligently and proactively determine the best teamed ports to use to provide optimum network connectivity for the server.
• • • Link loss detection – “Do I have physical link?” Transmit path validation – “ Can I successfully transmit a frame onto the network?” Receive path validation – “Can I successfully receive any kind of frame from the network?” These basic redundancy mechanisms do not provide for redundancy beyond the network adapter port in the team.
Figure 4-7 Upstream link failures cause server isolation 50 USERS ACCESS TO SERVER 250 USERS NO ACCESS TO SERVER Co ns o le Switch A HP NIC TEAMING IP= 1.1.1.1 NIC 1 Primary Co ns o le Core Switch NO ACCESS TO SERVER Switch B NIC 2 Non-Primary External Network o le ns Co Router NO ACCESS TO SERVER Server 50 USERS NO ACCESS TO SERVER In this case, the Fast Path mechanism, designed and developed by HP Engineering is an alternative solution.
Figure 4-8 Upstream link failures cause server receive bandwidth bottleneck Link 5 Gigabit Ethernet HP NIC TEAMING Co ns o le Switch A NIC 1 Primary Gigabit Ethernet Link 2 100 Mbps Link 3 NIC 2 Non-Primary Link 1 Gigabit Ethernet Gigabit Ethernet Link 4 Core Switch Co ns o le Switch B Server The Fast Path mechanism is a solution to this problem.
The solution to this problem is the Fast Path mechanism designed and developed by HP Engineering. In addition to Fast Path detecting full connectivity loss, as described in the first example, Fast Path can also detect when a server’s teamed ports have different hop count paths to the Core Switch (root switch).
Figure 4-10 Fast Path operation overview Fast Path Configuration Fast Path is enabled from the Advanced Redundancy tab in NCU by checking the Fast Path Failover checkbox (refer to Figure 4-11). Fast Path can also be monitored from the Advanced Redundancy tab. Under the Team Members section, the Port/Path Cost column provides real-time monitoring information. The Port Cost value (in other words, 4 in Figure 4-11) is the Spanning Tree cost associated with the current speed of the teamed port.
Figure 4-11 Fast Path configuration and monitoring Fast Path has two Spanning Tree Protocol configuration options: • Enabled or disabled, as discussed in the previous section • Spanning Tree Protocol (STP) type Both of these settings are located on the Advanced Redundancy tab in NCU. Refer to Figure 4-11. There are two predominate Spanning Tree Protocols (STP) in use today. They are IEEE 802.1D Spanning Tree Protocol and Cisco® Per VLAN Spanning Tree Plus (PVST+) Protocol.
1. 2. 3. 4. 5. Switch ports connected to teamed ports should be configured to bypass Spanning Tree’s block/listen/learn stages. These stages are not needed for ports connected to end node devices (for example, servers, PCs, etc.) Switch ports connected to teamed ports MUST transmit BPDUs. Without BPDUs, Fast Path will not have the information it needs to make decisions.
Table 4-7 Redundancy mechanism comparison chart (continued) Detects good ports versus best ports Mechanism can be manually disabled √ √ √ √ √ Failover Events Link Loss Heartbeats (Receive and Transmit Validation) When a network port is a member of a team and loses physical link (in other words, wire fault, lost link light), the teaming driver prevents the port from being used by the team.
is the preferred root Fast Path also can choose the best teamed port from a set of teamed ports that all have validated connectivity to the main network segment. As a result, Fast Path can initiate a failover from one teamed port operating as the Primary port to another non-Primary teamed port for either of two reasons.
requires fault tolerance in case of a network adapter malfunction, but does not have a demand for receiving or transmitting more than the capacity of the Primary port.) Recommended Configurations for NFT Environments HP recommends the following: • Heartbeats should be enabled (default). • MAC addresses should not be manually set to a locally administered address (LAA) via the Microsoft Adapter Properties User Interface.
However, traffic received by the server is not load balanced, meaning the Primary teamed port is responsible for receiving all traffic destined for the server (refer to Figure 4-12). As with NFT, there are two types of team members, Primary and Non-Primary ports. The Primary port transmits and receives frames and the non-Primary ports only transmit frames. Figure 4-12 Overview of TLB communication ARP Table: Client A 1.1.1.1 = E IP Address = 1.1.1.2 1.1.1.3 = B 1.1.1.4 = C 1.1.1.
Figure 4-13 Scenario 4–B: A device pings a TLB team on the same Layer 2 network Ethernet MAC = A IP = 1.1.1.1 MAC = B Primary MAC = E Non-Primary Red TLB IP = 1.1.1.2 Blue (server) 1. Red transmits a broadcast ARP request asking for Blue’s MAC address. A user on Red issues the command ping 1.1.1.2 to initiate a ping to Blue. First, Red determines whether or not Blue is on the same Layer 2 network.
4. Blue transmits a broadcast ARP request asking for Red’s MAC address. NOTE: The following step may not occur if Blue’s ARP table still contains an entry for Red as a result of Steps 1 and 2. Blue checks its ARP cache for a MAC address entry that matches 1.1.1.1. If Blue does not find one, then Blue broadcasts an ARP request asking for Red’s MAC address. 5. Red transmits a unicast ARP reply to Blue providing its MAC address. NOTE: The following step will not occur if Step 4 does not take place.
Figure 4-14 Transmit Load Balancing Method Configuration Table 4-8 Transmit Load Balancing method comparison Preserves Frame Transmission Order Load Balances through Router Automatic √ √ TCP Connection √ √ Destination IP address √ √ Destination MAC address √ Round Robin Guarantees Equal Load Balancing HP Recommendation √ √ TLB Automatic Method Automatic is a load-balancing method that is designed to preserve frame ordering.
the Automatic mode. By deploying this method now, future upgrades will automatically take advantage of the new intelligence. TLB TCP Connection Method TCP Connection is also a load-balancing method that is designed to preserve frame ordering. This method will load balance outbound traffic based on the TCP port information in the frame’s TCP header. This load-balancing method combines the TCP source and destination ports to identify the TCP conversation.
Table 4-10 Load Balancing based on Destination IP Address (four- and five-port teams) (continued) 000 network port 1 000 network port 1 001 network port 2 001 network port 2 010 network port 3 010 network port 3 011 network port 4 011 network port 4 100 network port 1 100 network port 5 101 network port 2 101 network port 1 110 network port 3 110 network port 2 111 network port 4 111 network port 3 If the Destination IP Address algorithm is chosen, and the frame doesn’t have
Figure 4-15 Scenario 4-C: TLB team using IP address for load balancing algorithm MAC = C IP = 1.1.1.3 Green (Router) MAC = D IP = 2.2.2.3 Network 1.0.0.0 Network 2.0.0.0 Ethernet MAC = F IP = 1.1.1.4 Yellow Ethernet MAC = B Primary MAC = A IP = 1.1.1.1 MAC = E Non-Primary TLB IP = 2.2.2.1 Red Blue (server) TLB Destination MAC Address Method Destination MAC Address is another load-balancing method that will attempt to preserve frame ordering.
Table 4-11 Load balancing based on Destination MAC Address (two- and three-port teams) (continued) 011 network port 2 011 network port 1 100 network port 1 100 network port 2 101 network port 2 101 network port 3 110 network port 1 110 network port 1 111 network port 2 111 network port 2 Table 4-12 Load balancing based on Destination MAC Address (four- and five-port teams) Four-Port Team Five-Port Team Destination MAC Transmitting Port Destination MAC Transmitting Port 000 network
Figure 4-16 Scenario 4-D: TLB team using MAC address for load-balancing algorithm Ethernet MAC = F IP = 1.1.1.4 MAC = A IP = 1.1.1.1 MAC = B Primary Yellow MAC = E Non-Primary Red TLB IP = 1.1.1.2 Blue (server) In summary, special consideration should be given when choosing the MAC address-based load-balancing algorithm in an environment where the server and clients are separated by a Layer 3 device such as a router.
ports are equally used. HP recommends that the implications of this method of load balancing be carefully considered before deployment. TLB Redundancy Mechanisms The basic and advanced redundancy mechanisms used by TLB are identical to NFT. Refer to the section, “NFT Redundancy Mechanisms”, for more information. TLB Network Adapter Failure Recovery With TLB, the recovery mechanism provided is very similar to the NFT failover methods discussed previously.
• switches. This helps prevent switch uplink failure scenarios that leave team members in separate broadcast domains. TLB teams that communicate with TCP/IP network devices via a router should use the Automatic, TCP port or, IP address-based load balancing algorithm (configured via the NCU).
Figure 4-17 Overview of SLB communication ARP Table: Client A 1.1.1.2 = A 1.1.1.3 = B 1.1.1.4 = C 1.1.1.5 = D IP Address = 1.1.1.2 Destination MAC/Source MAC/Destination IP/Source IP E A 1.1.1.1 1.1.1.2 N100NT5.SYS MAC = E MAC = A Destination MAC/Source MAC/Destination IP/Source IP A E 1.1.1.2 1.1.1.1 Client B E B 1.1.1.1 Destination MAC/Source MAC/Destination IP/Source IP B E 1.1.1.3 1.1.1.1 Destination MAC/Source MAC/Destination IP/Source IP N100NT5.SYS MAC = E E C 1.1.1.1 1.1.1.
As an example, a detailed discussion of Cisco’s EtherChannel load-balancing algorithm is described below. Most of the information about how Cisco switches load-balance traffic on a port trunk is applicable to other switch vendors’ Port Trunking technology. Cisco EtherChannel Cisco's Fast EtherChannel (FEC) and Gigabit EtherChannel (GEC) technology is a MAC layer (Layer 2) load-balancing technology using two or more switch ports grouped together as one logical switch port.
Table 4-14 Example of load-balancing algorithms #2 (continued) Packet 2 - Destination MAC address: 00-00-00-00-00-01 frame is transmitted out port 2 Last binary bit = 0000 0001 Method 2: SOURCE-BASED ALGORITHM Packet 1 - Source MAC address: 00-00-00-00-00-03 frame is transmitted out port 2 Last binary bit = 0000 0001 Packet 2 - Source MAC address: 00-00-00-00-00-02 frame is transmitted out port 1 Last binary bit = 0000 0000 Method 3: XOR ALGORITHM (best method to use on switch) Packet 1 - Source MAC a
SLB Redundancy Mechanisms The only redundancy mechanisms used by SLB are link loss and transmit validation heartbeats. None of the advanced redundancy mechanisms (for example, Active Path and Fast Path) are supported by SLB. The advanced redundancy mechanisms require the ability to predict and control which ports in a team will receive a particular frame. Since the switch connected to an SLB team controls how the team receives frames, the advanced redundancy mechanisms would not function reliably.
• technology has been designed to allow for flexibility. Therefore, the NCU may allow configuration of an SLB team that will not work correctly with a particular vendor’s switch. The switch’s load-balancing algorithm should be set to XOR or SOURCE-BASED but not DESTINATION-BASED (refer to the sections, “Switch-assisted Load Balancing with Fault Tolerance (SLB)” and “Cisco EtherChannel”).
Figure 4-18 TLB does not provide receive load balancing Switch NIC 1 (A) Client ARP Table: 1.1.1.1 = A PRIMARY HP Network Adapter Team Client ARP Table: 1.1.1.1 = A Switch Redundancy NIC 2 (B) SECONDARY (TLB) Client ARP Table: 1.1.1.1 = A NIC 3 (C) SECONDARY 1.1.1.1 Switch NIC 4 (D) SECONDARY HP Integrity Server • Maximum Server Bandwidth Transmit = 4000 Mbit Receive = 1000 Mbit Client ARP Table: 1.1.1.
With Dual Channel, a server can have full transmit and receive load balancing with switch redundancy (refer to Figure 4-20). Figure 4-20 Dual Channel & Dynamic Dual Channel provide full load balancing and switch redundancy Switch Client ARP Table: 1.1.1.1 = A Port Channel NIC 1 (A) Client 1 HP Network PRIMARY Adapter Team (Dual Channel) 1.1.1.1 Client ARP Table: 1.1.1.1 = A Client 2 NIC 2 (A) SECONDARY Switch Switch Redundancy Client ARP Table: Port Channel NIC 3 (B) SECONDARY 1.1.1.
functional (just like SLB). If all of the teamed ports in a group fail, the team is still available via the other group. Since the IP address is “owned” by the team instead of either group (A or B), the server maintains network connectivity (with the same IP address) as long as at least one teamed port in at least one group is functional. Figure 4-21 Overview of Dual Channel communication ARP Table: ARP Table: Client A 1.1.1.2 = A 1.1.1.3 = B 1.1.1.4 = C 1.1.1.5 = D IP Address = 1.1.1.
Figure 4-22 Dual Channel configuration Dual Channel Transmit Balancing Algorithm The transmit load-balancing algorithms used by Dual Channel are the same methods used by TLB. Refer to “Transmit Load Balancing with Fault Tolerance (TLB)” for a detailed discussion.
The client’s ARP table entries expire after a few minutes; as a result, the clients will re-ARP for the server’s IP address. The clients may or may not get the same IP address for the server depending on the results from the load-balancing algorithm. • Periodic non-cross trunk load balancing when server transmits ARP request: Periodically, the server must broadcast an ARP request for the MAC address of an IP address to which it needs to talk.
• • • Make sure that each group in a Dual Channel team is connected to a different switch. If both groups are attached to the same switch, switch redundancy is lost. Whenever a Dual Channel team is created with an even number of teamed ports, evenly distribute the ports between both groups. Ensure all ports in the same group are connected to the same switch. 802.3ad Dynamic Dual Channel Load Balancing 802.3ad Dynamic Dual Channel Load Balancing (Dynamic Dual Channel) functions exactly like Dual Channel.
Team Types and Redundancy Mechanisms Interoperation Team Type and Redundancy Mechanisms Compatibility Chart Table 4-16 Team type and redundancy mechanism compatibility NFT TLB SLB √ 802.
Figure 4-23 Active Path and Fast Path disagree on Primary teamed port Link 5 Gigabit Ethernet Root Switch 2 (Bridge Priority 32768) Co ns o le Switch A NIC 1 Active Path (Passed) Fast Path (Failed) Echo Node Root Switch 1 (Bridge Priority 16384) HP NIC Teaming NIC 2 Fast Path (Passed) Active Path (Failed) Link 4 Gigabit Ethernet Link 1 Gigabit Ethernet Co ns o le HP Integrity Server Switch B Core Switch Because Active Path and Fast Path disagree, the implementer needs to designate which mech
Figure 4-24 Mechanisms priority configuration: Fast Path prioritized higher than Active Path Team Status and Icons Network Adapter Teamed Status The NCU reports a “Teamed Status” on the Information tab of each teamed port in the server. This status allows the user to determine the condition of the teamed port’s teamed status and take corrective action if necessary.
Figure 4-25 Teamed port Information tab Listed below are the possible status conditions with definitions: • Available – The team member is functioning normally. • Not Teamed – The adapter is not part of the team. The most likely cause is adding an adapter to the team but not applying the change. • Unknown – The team member's status could not be determined. • Wire Fault – The member does not have link. • Not Joined – The member cannot be joined in the team because it has an incompatible setting.
• Failed (LACP Standby) – The team member has failed because the team has more members than the switch supports in the LACP protocol. The port is blocked by the switch. Failed (Rx Heartbeat) – The team member is not receiving frames. Failed (Tx Heartbeat) – A failure occurred while attempting to send a frame on the team member. Failed (Multiple) – The team member has multiple failed conditions. • • • Team State The NCU reports a Team State on the Information tab of each configured team.
HP Integrity Network Adapter Teaming and Advanced Networking Features When adapter ports are members of a team, some advanced features are no longer configurable on the port’s Advanced Settings tab. These advanced features are either promoted to the team’s Advanced Settings tab because the features are compatible and supported by all team members, or not promoted because one or more ports do not support the features.
order to improve performance. If all teamed ports in the same team support jumbo frames, the feature is promoted to the team’s Advanced Settings tab with the lowest size configured for the team members. For example, if there are two adapters, one configured for 4088 bytes and the other for 9014 bytes, and they are teamed, the Maximum Frame Size feature is set to 4088 bytes. A setting of 1514 bytes is equal to Jumbo Frames being disabled.
4–7 HP Integrity Network Adapter Teaming Configuration and Deployment with Integrity Essentials Rapid Deployment Pack (RDP) One of the methods of deployment for HP Integrity Network Adapter Teaming is to use HP Integrity Essential’s Rapid Deployment Pack (RDP) remote scripting feature. This feature allows an SA to deploy a specific teaming configuration to any number of target servers in a single action.
5 Teaming Feature Matrix Table 5-1 Teaming feature matrix NFT & NFT with Preference TLB & TLB with Preference SLB & 802.
Table 5-1 Teaming feature matrix (continued) Can team ports operating at different speeds as long as the ports support a common speed √ √ √ √ Can team ports of different media √ √ √ √ Load balances TCP/IP √ √ √ Load balances non-IP traffic √ √ √ Supports load balancing by destination IP address, destination MAC address, TCP port, and Round Robin √ √ √ √ √* √ √ All teamed ports within a team transmit frames with the same MAC address All ports within a team utilize the same IP addres
A – Overview of Network Addressing and Communication Understanding the concepts of network addressing is the key to understanding how HP’s Network Adapter Teaming works. This section provides a brief overview of network addressing as a baseline for explaining how HP’s Network Adapter Teaming can create one logical network port from a team of two or more ports. Layer 2 Versus Layer 3 Devices on a computer network use unique addresses, much like telephone numbers, to communicate with each other.
These scenarios provide a baseline of typical network addressing and communication using IP. This baseline will be referred to later in this document to differentiate how HP Integrity Network Adapter Teaming functions in these same scenarios. By understanding the differences in simple examples such as these (without HP’s Network Adapter Teaming technology involved), implementers will have a better understanding of how HP’s Network Adapter Teaming technology may work in their environment.
NOTE: This step may not occur if Blue’s ARP table still contains an entry for Red as a result of Steps 1 and 2. Blue checks its ARP cache for a MAC address entry that corresponds to 1.1.1.1. If Blue does not find one (in other words, ARP cache timed out since last communication with Red), then Blue broadcasts an ARP request asking for Red’s MAC address. 5. Red transmits a unicast ARP reply to Blue providing its MAC address (A). NOTE: The following step will not occur if Step 4 does not take place.
is. First, Red checks its own ARP cache for an entry that matches 1.1.1.3. If Red does not have an entry cached, then it must broadcast an ARP request frame on network 1.0.0.0 asking Green to respond and provide its MAC address. 2. Green transmits a unicast ARP reply to Red providing its MAC address (C). Green sees the ARP request and responds with a unicast ARP reply to Red. Also, Green enters Red’s MAC address and IP address into its ARP cache.
address as the source MAC address and Blue’s IP address as the source IP address. Green receives the ping reply and determines that the frame is meant for Red because of the Layer 3 address (IP). 10. Green transmits a broadcast ARP request on Network 1.0.0.0 asking for Red’s MAC address. NOTE: This step will not occur if Green’s ARP table still contains an entry for Red resulting from Steps 1 and 2. Green looks in its ARP cache for a MAC address for Red.
B – Frequently Asked Questions B.1 HP Integrity Network Adapter Teaming Frequently Asked Questions B.B.1.1 Why is traffic not being load balanced out of my server? (or) Why is traffic not being load balanced during backups? Either TLB or SLB is required for load balancing of transmit traffic. NFT will not provide for any type of load balancing.
B.B.1.9 B.B.1.10 B.B.1.11 B.B.1.12 B.B.1.13 How do I uninstall HP Integrity Network Adapter Teaming? HP Integrity Network Adapter Teaming can be uninstalled by opening the properties page of any network interface under Network and Dial-up Connections (Microsoft UI). Select HP Network Teaming and Configuration and click the UNINSTALL button. Always dissolve any existing teams using the NCU before attempting an uninstall of HP Integrity Network Adapter Teaming.
• Server Clusters: Network Configuration Best Practices for Windows 2000 and Windows Server 2003: http://technet2.microsoft.com/windowsserver/en/library/ 7b1c90d1-877e-4c55-ab8c-9c65101a503d1033.mspx • Understanding Requirements for Failover Clusters (Windows Server 2008): http://technet2.microsoft.com/windowsserver2008/en/library/ 6fe86b30-6820-43a2-8321-3e7b51cbe7631033.mspx • Network Adapter Teaming and Server Clustering: http://support.microsoft.com/kb/254101 B.B.1.
B.B.1.23 B.B.1.24 B.B.1.25 B.B.1.26 B.B.1.27 B.B.1.28 B.B.1.29 100 The heartbeat timers should be increased when heartbeat failures are caused by latency in the network infrastructure. This is an extremely rare problem. I need NIC redundancy, switch redundancy and Rx/Tx load balancing. What teaming type should I use? When NIC redundancy, switch redundancy, and load balancing are all required, the only option is Dual Channel or Dynamic Dual Channel.
C – Overview of Utilities Included with HP Integrity Network Adapter Teaming In the Component Pack (CPnnnnn.exe) for HP Integrity Network Adapter Teaming there are several command line utilities provided for special teaming-related management functions. Below is a list of the utilities with a brief description of each. Consult the latest HP Integrity Network Adapter Teaming Component Pack for the latest utilities and associated documentation. • Cqniccmd.
D – Glossary The following networking and teaming terms are used in this document: 802.1D Refers to the IEEE 802.1D specification. This is the original Spanning Tree specification. 802.3ad Dynamic Refers to the IEEE 802.3ad specification. This specification provides for manual and automatic (dynamic) port grouping for fault tolerance and load balancing between two network devices.
GEC Gigabit EtherChannel. A method of load balancing that both transmits and receives traffic across multiple Gigabit Ethernet (1000 Mbps) connections between two devices. Developed by Cisco Systems. Refer to SLB. GUI Graphical User Interface HP Teaming and Configuration GUI Now referred to as the HP Network Configuration Utility (NCU). Refer to NCU. IEEE Institute of Electrical and Electronics Engineers. A standards body for, among other things, network communications and protocols.
NFT Network Fault Tolerance. A team of network ports that transmits and receives on only one port with all other ports in standby. NIC Network Interface Card. Synonymous with network adapter. NIC teaming A phrase referring to HP Integrity Network Adapter Teaming. OSI Model Open Systems Interconnect Model. The seven-layer model developed by the International Standards Organization (ISO) that outlines the mechanisms used by networked devices to communicate with each other.
