Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.0 User’s Guide Information furnished in this manual is believed to be accurate and reliable. However, QLogic Corporation assumes no responsibility for its use, nor for any infringements of patents or other rights of third parties which may result from its use. QLogic Corporation reserves the right to change product specifications at any time without notice.
Table of Contents Preface Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Related Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . What’s in This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Documentation Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . License Agreements. . . . . . . . . . . . . . . . . . . . . . . . . .
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.0 User’s Guide Using the QLogic Fast!UTIL BIOS Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . Verifying That the Driver is Loaded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring the Driver Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Deploying NPIV for VMware ESX/ESXi 4.0 . . . . . . . . . . . . . . . . . . . . . . . . . Monitoring NPIV Virtual Ports from SANsurfer . . .
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.0 User’s Guide A-6 A-7 A-8 Common Driver Log Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Proc Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Frequently Seen Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.
Preface QLogic provides a portfolio of Fibre Channel Adapters, Intelligent Ethernet Adapters, and Converged Network Adapters on the VMware® ESX/ESXi 4.0 platform: QLogic Fibre Channel Adapters include 2Gb, 4Gb, and 8Gb adapter offerings on VMware ESX/ESXi 4.0 in PCI-X™, PCI Express®, and PCI Express ExpressModule™ form factors. QLogic Intelligent Ethernet Adapters include the 1/10Gb QLE3000 Series adapters in PCI Express form factors.
SANsurfer FC HBA CLI User’s Guide (part number SN0054614-00) SANsurfer FC HBA CLI release notes and ReadMe file Related VMware documentation: Online VMware Compatibility Guide located here: http://www.vmware.com/go/hcl VMware Fibre Channel SAN Configuration Guide located here: http://www.vmware.com/pdf/vsphere4/r40/vsp_40_san_cfg.pdf Other technical documentation for VMware products is located here: http://www.vmware.
2 Installation details the supported features and covers installation of hardware, drivers, and SANsurfer® management tools. 3 Configuration covers how to identify QLogic adapters, start Fast!UTIL, verify that the driver is loaded, configure driver parameters, deploy N_Port ID virtualization (NPIV), monitor NPIV from the SANsurfer management tools, and configure the boot from SAN functionality. A Troubleshooting details troubleshooting methods, including LEDs and logs.
Text within square brackets indicates optional parameters. For example iscli [optional parameters] [Command Variable] Text in blue font indicates a hyperlink (jump) to a figure, table, chapter, or section in this guide, and links to Web sites are shown in underlined blue. For example: Table 4-1 lists the configurable driver parameters. See “Configuring Boot from SAN” on page 4-10. For more information, visit www.qlogic.com.
Visit the QLogic support Web site listed in Contact Information for the latest firmware and software updates. Availability QLogic Technical Support for products under warranty is available during local standard working hours excluding QLogic Observed Holidays. Training QLogic offers training for technical professionals for all iSCSI, InfiniBand, and Fibre Channel products. From the main QLogic Web page, www.qlogic.
Notes xii FC0054607-00 A
1 Hardware and Software Components This chapter provides details about QLogic Fibre Channel and Converged Network Adapters in VMware ESX/ESXi, including: Hardware Components “Software Components” on page 1-3 “Supported QLogic Adapters” on page 1-4 Hardware Components A typical QLogic Fibre Channel or Converged Network Adapter includes the following major components: RISC processor Static random access memory (SRAM) Non-volatile memory (Flash) Non-volatile random access memory (NV
1–Hardware and Software Components Hardware Components 5 1 NVRAM: Non-Volatile Random Access Memory (aka EEPROM). Where some of the adapter parameter settings are stored. Because NVRAM is a non-volatile type of RAM, data is retained even if the power is shut off. 2 Transceivers: Short for transmitterreceiver, a device (copper or optical) that transmits and receives signals. 4 LEDs: Light Emitting Diodes. Display various active/inactive states of an adapter.
1–Hardware and Software Components Software Components Table 1-1. Adapter Component Details (Continued) Callout Number Description 4 LEDs indicate the real-time status of the adapter, Fibre Channel or Ethernet link status, and link activity. LED patterns are described in detail in “Troubleshooting with LEDs” on page A-1. 5 Non-volatile random access memory (NVRAM) is used to store the per-port adapter configuration parameters.
1–Hardware and Software Components Supported QLogic Adapters NOTE: All inbox drivers for ESX/ESXi are distributed by VMware. Firmware The firmware module performs most of the Fibre Channel and FCoE protocol-related functions, thus freeing the system CPU and memory resources from protocol processing. This efficiency keeps the QLogic Fibre Channel and Converged Network Adapter drivers thin, which saves runtime system resources such as CPU and memory.
2 Installation This chapter provides the following installation information: Supported Features “Installing the Hardware” on page 2-2 “Installing the Driver” on page 2-3 “Installing SANsurfer Management Tools” on page 2-3 Supported Features When properly installed, QLogic Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.0 support the following features: VMware ESX/ESXi Release Support—4.0. Topologies—FC-AL, FC-AL2, point-to-point, and switched fabric.
2–Installation Installing the Hardware Support for MSI-X—on 4/8Gb Fibre Channel and Converged Network Adapters. Support for VMware SCSI path failover—for Active/Active and Active/Passive storage devices. Support for VMware VMotion™ and VMware Storage VMotion Remote SAN and adapter management—using the SANsurfer FC HBA Manager tool (see “Installing SANsurfer FC HBA Manager” on page 2-4).
2–Installation Installing the Driver 5. Refasten the adapter's retaining bracket using the existing screw or lever. 6. Close the computer cover. 7. Plug the Fibre Channel or Ethernet cable into the adapter. 8. Plug in the power cable and turn on the computer. 9. Verify the LEDs on the adapter to ensure proper operation. The LED patterns are described in “Troubleshooting with LEDs” on page A-1. Installing the Driver The QLogic Fibre Channel driver ships “inbox” with VMware ESX/ESXi 4.0.
2–Installation Installing SANsurfer Management Tools Installing SANsurfer FC HBA Manager To run SANsurfer FC HBA Manager on a VMware ESX 4.0 server requires a two-step installation process, whereby: 1. You must install the SANsurfer back end, commonly called the qlremote agent, on the ESX server. 2. You must download the SANsurfer FC HBA Manager GUI from the QLogic Web site and install it on a Windows® or Linux® system. SANsurfer FC HBA Manager allows you to remotely manage QLogic adapters.
2–Installation Installing SANsurfer Management Tools 2. Click the by Operating System button, and then select Fibre Channel Adapters, VMware, and VMware ESX/ESXi, as shown in Figure 2-2. Figure 2-2 Downloading SANsurfer FC HBA Manager—VMware 3. Click Go. 4. On the VMware ESX/ESXi download page under Management Tools, click SANsurfer FC HBA Manager (x86/x64). 5. On the End User Software License Agreement page, click Agree. 6.
2–Installation Installing SANsurfer Management Tools c. In the command window, type hostname, and then press ENTER. 9. On the ESX Host, add the IP address and the “hostname” of the remote system where you plan to run SANsurfer FC HBA Manager to the following files: /etc/hosts /etc/hosts.allow 10. On the remote system, add the IP address and “hostname” of the ESX Host, based on the operating system: For Linux, modify the /etc/hosts file.
2–Installation Installing SANsurfer Management Tools 3. Click Go. 4. Under Management Tools, click SANsurfer FC HBA Manager (x86/x64). 5. On the End User Software License Agreement page, click Agree. 6. Save the zipped file and then use it to install SANsurfer FC HBA Manager by following the installation wizard. After you complete the preceding installation steps, you must connect SANsurfer FC HBA Manager from the Linux or Windows host to the qlremote agent running on the ESX host.
2–Installation Installing SANsurfer Management Tools 4. On the VMware ESX/ESXi download page under Management Tools, click SANsurfer FC HBA CLI (x86/x64). 5. On the End User Software License Agreement page, click Agree. 6. Save the scli-x.xx.xx-xx.i386.rpm.gz file to your local machine. (In the file name, x.xx.xx.xx indicates the current version of SANsurfer FC HBA CLI). Then use it to install SANsurfer FC HBA CLI by following the installation wizard. 7.
3 Configuration QLogic Fibre Channel and Converged Network Adapters on VMware ESX/ESXi 4.0 configuration procedures include the following: Identifying QLogic Adapters on VMware ESX/ESXi 4.0 “Using the QLogic Fast!UTIL BIOS Utility” on page 3-5 “Verifying That the Driver is Loaded” on page 3-6 “Configuring the Driver Parameters” on page 3-7 “Deploying NPIV for VMware ESX/ESXi 4.
3–Configuration Identifying QLogic Adapters on VMware ESX/ESXi 4.0 To identify QLogic Fibre Channel and Converged Network Adapters using vSphere: 1. Start vSphere Client and select the host you want. 2. Click the Configuration tab to bring the Configuration page to the front. 3. Under Hardware on the left, click Storage Adapters. A list of all storage adapters configured on the system appears, including QLogic Fibre Channel and Converged Network Adapters, as shown in Figure 3-1.
3–Configuration Identifying QLogic Adapters on VMware ESX/ESXi 4.0 To identify QLogic Fibre Channel and Converged Network Adapters using console OS: 1. Discover the device using lspci. 2. To list the subdirectories, run the following command: ls /proc/scsi: If the qla2xxx directory is present, QLogic adapters are installed on the VMware ESX/ESXi 4.0 system. 3.
3–Configuration Identifying QLogic Adapters on VMware ESX/ESXi 4.0 Figure 3-2 Finding Adapters on VMware ESX/ESXi 4.0 Driver information includes the following: 3-4 Host adapter: loop state specifies the current state of the adapter port, including: READY—Driver is ready to handle IO commands. UPDATE—Driver is updating its target database because of a change in the fabric. DEAD—No connection to a fabric or target.
3–Configuration Using the QLogic Fast!UTIL BIOS Utility NPIV Supported indicates whether the adapter port supports NPIV. If Yes, the maximum number of supported virtual ports on that specific adapter port is shown in the next line, Max Virtual Ports. SCSI Device Information lists the adapters’ world wide names (WWNs) and Fibre Channel IDs. FC Target-Port List lists the target devices currently attached to the adapter port.
3–Configuration Verifying That the Driver is Loaded Select Host Adapter setting enables you to select a specific adapter, if you have multiple Fibre Channel or Converged Network Adapters in your system, and then configure or view the adapter’s settings. The Fast!UTIL BIOS utility is described in detail in the BIOS Readme file on the QLogic Driver Downloads / Documentation page (in the Boot Code section) for your adapter: http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/default.
3–Configuration Configuring the Driver Parameters Figure 3-3 Verifying That the Driver is Loaded Configuring the Driver Parameters Use the esxcfg-module command to configure the QLogic Fibre Channel and Converged Network Adapter driver parameters. To configure driver parameters: 1. To list the driver parameters, execute the following command: # esxcfg-module -g qla2xxx 2.
3–Configuration Configuring the Driver Parameters where is one of the configurable parameters listed in Table 3-1 on page 3-8. For example, the following command sets the maximum device queue depth reported by the QLogic to a value of 32: # esxcfg-module -s ql2xmaxqdepth=32 qla2xxx 3. To update the ESX RAMdisk image, execute following command: # esxcfg-boot -b 4.
3–Configuration Deploying NPIV for VMware ESX/ESXi 4.0 Table 3-1. Configurable Driver Parameters (Continued) Driver Parameter Description ql2xloginretrycount Option to modify the login retry count. ql2xlogintimeout Login timeout value in seconds. Default is 20. ql2xmaxqdepth Maximum queue depth to report for target devices. Default is 32. ql2xmaxsgs Maximum scatter or gather entries per I/O request. Default is 32.
3–Configuration Monitoring NPIV Virtual Ports from SANsurfer VMware ESX/ESXi 4.0 enables NPIV support on QLogic 4Gb/8Gb Fibre Channel Adapters and QLE8000 Series Converged Network Adapters. No additional software or hardware installation is required to support NPIV. Note that VMware ESX/ESXi 4.0 supports NPIV only on RDM (Raw Device Mapping) devices. To configure NPIV, the Fibre Channel-FCoE switch port must be configured to enable NPIV.
3–Configuration Monitoring NPIV Virtual Ports from SANsurfer Figure 3-4 SANsurfer FC HBA Manager Showing NPIV When you select an individual virtual port on the Topology page, SANsurfer FC HBA Manager displays additional details about the selected virtual port on four tabbed pages. The Information page (see Figure 3-5) displays port attributes such as Port ID, Firmware Version, and Actual Data Rate.
3–Configuration Configuring Boot from SAN Figure 3-5 Virtual Port Information Page in SANsurfer Configuring Boot from SAN Boot from SAN refers to installing and booting a server’s operating system from a storage device attached to the SAN instead of from a locally attached storage device. The OS is installed on one or more LUNs in the SAN array and the Fibre Channel and Converged Network Adapters are configured to boot from the specified LUN.
3–Configuration Configuring Boot from SAN VMware ESX/ESXi 4.0 provides boot from SAN capability on QLogic Fibre Channel Adapters. For storage array and Fibre Channel switch configuration, refer to the appropriate user manual for the device. For VMware ESX/ESXi 4.0-specific configuration requirements, refer to the VMware Fibre Channel SAN Configuration Guide located at the following URL: http://www.vmware.com/pdf/vsphere4/r40/vsp_40_san_cfg.
3–Configuration Configuring Boot from SAN 7. 8. After the Host Adapter BIOS is enabled, select the boot device as shown in the following table: Enable Selectable Boot WWPN/LUN Boot List No x Yes None specified BIOS configures the first disk drive it finds that is also a LUN 0 as boot device. Yes Specified BIOS scans through the specified boot WWPN or LUN list until it finds a disk drive, and then configures it as the boot device.
A Troubleshooting This appendix provides the following methods of determining the status of the adapters and finding solutions to common problems: Troubleshooting with LEDs “Dynamic Extended Logging” on page A-3 “Troubleshooting with Driver Logs” on page A-4 “Debugging Using Proc Information” on page A-9 “Collecting System Logs for Troubleshooting” on page A-10 “Frequently Seen Issues” on page A-10 Troubleshooting with LEDs QLogic Fibre Channel and Converged Network Adapters have LE
A–Troubleshooting Troubleshooting with LEDs Table A-2.
A–Troubleshooting Dynamic Extended Logging Table A-4.
A–Troubleshooting Troubleshooting with Driver Logs When the logging is enabled, the driver version in the /proc/scsi/qla2xxx/ is appended with a -debug flag as shown below: # cat /proc/scsi/qla2xxx/5 QLogic PCI to Fibre Channel Host Adapter for QLE2462: Firmware version 4.04.09 [IP] [Multi-ID] [84XX], Driver version 8.02.01-k1-vmw41-debug BIOS version 2.10 FCODE version 2.04 EFI version 2.04 Flash FW version 4.06.
A–Troubleshooting Troubleshooting with Driver Logs Aug 18 15:15:13 localhost vmkernel: 0:00:00:34.550 cpu6:4109)<6>qla2xxx 0000:0a:00.0: Allocated (1413 KB) for firmware dump... Aug 18 15:15:13 localhost vmkernel: 0:00:00:34.572 cpu6:4109)IDT: 942: 0x71 sharable (entropy source), flags 0x10 Aug 18 15:15:13 localhost vmkernel: 0:00:00:34.572 cpu6:4109)VMK_VECTOR: 137: Added handler for shared vector 113, flags 0x10 Aug 18 15:15:13 localhost vmkernel: 0:00:00:34.
A–Troubleshooting Troubleshooting with Driver Logs Table A-6 lists the more common driver log messages and provides an explanation of each. Table A-6. Common Driver Log Messages "qla2xxx : Found an ISPXXXX, irq yyy, iobase zzz" Driver is reporting which adapter it has found during initialization. "qla2xxx : Verifying loaded RISC code..." Driver is reporting that it has verified the RISC code and it is running. "qla2xxx : LIP reset occured (f8f7).
A–Troubleshooting Troubleshooting with Driver Logs Table A-6. Common Driver Log Messages (Continued) "scsi(%d): RSCN,..." Driver received a registered state change notification (RSCN) async event from the firmware. Additional information follows the message (that is, mailbox values from the firmware). "scsi(%d): Cannot get topology - retrying" Firmware return status indicating it is busy. "qla2x00: Status Entry invalid handle" Driver detected an invalid entry in the ISP response queue from the firmware.
A–Troubleshooting Troubleshooting with Driver Logs Table A-6. Common Driver Log Messages (Continued) "WARNING qla2xxx: Couldn't allocate host from scsi layer!" Indicates the driver could not register with the SCSI layer, usually because it could not allocate the memory required for the adapter. "WARNING scsi(%d): [ERROR] Failed to allocate memory for adapter\n" Indicates the driver could not allocate all the kernel memory it needed.
A–Troubleshooting Debugging Using Proc Information Debugging Using Proc Information The driver /proc/scsi/qla2xxx/ provides debugging information that can help you troubleshoot problems with the driver. The /proc entry provides firsthand information about the current state of the driver. Table A-8 lists the /proc output fields and provides a description of each. Table A-7. Proc Output Output Field Description Boot Code Version Provides the version of the boot code present on the adapter.
A–Troubleshooting Collecting System Logs for Troubleshooting Collecting System Logs for Troubleshooting To collect all relevant data on the running system into one tar file, execute the following command: # vm-support Frequently Seen Issues Table A-8 lists some common issues and questions and provides tips for their resolution. Table A-8. Frequently Seen Issues Issue Troubleshooting Tip What is the SANsurfer FC HBA Manager password? The password is config.
Glossary adapter The board that interfaces between the host system and the target devices. Adapter is synonymous with host bus adapter (HBA), host channel adapter (HCA), host adapter, and adapter board. arbitrated loop A circular (ring) topology (versus point-to-point) where two or more ports can be interconnected, but only two ports can communicate at a time. All communication passes through all ports connected to the loop. adapter port A port on the adapter board.
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.0 User’s Guide cache A temporary high-speed storage (memory) area where recently used or frequently accessed data is stored for rapid access, thus increasing the efficiency of processor operations. Cache contents duplicate recently-used values stored on disk or elsewhere. Cache is quick to access relative to the original data which either takes more time due to slow disc-access time, or is slow to compute.
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.0 User’s Guide fabric switch Also, switched fabric. Connects multiple devices from independent Fibre Channel-arbitrated loops (FC-ALs) and point-to-point topologies into a fabric using Fibre Channel switches. Fast!UTIL QLogic Fast!UTIL™ Fibre Channel Adapter BIOS utility. FC Fibre Channel is a high-speed serial interface technology that supports other higher layer protocols such as SCSI and IP. FCode Forth code.
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.0 User’s Guide ISL Inter-switch link. The connection between two switches using E_Ports. ISP Intelligent storage peripheral. QLogic trademark and family of Fibre Channel and SCSI controller chips that replace network interface chips in network adapters, servers, and storage. LAN Local area network. Network typically with transmissions less than 5 km. L_Port Loop port. Does arbitrated loop functions and protocols.
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.0 User’s Guide N_Port Node port. A port that connects by a point-to-point link to either a single N_Port or a single F_Port. N_Ports handle creation, detection, and flow of message units to and from the connected systems. N_Ports are end ports in virtual point-to-point links through a fabric, for example N_Port to F_Port to F_Port to N_Port using a single Fibre Channel fabric switch. See also FL_Port. N_Port ID virtualization See NPIV.
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.0 User’s Guide This is important to the driver when selecting the proper method of failover notification. It can make a difference to the target device, which might have to take different actions when receiving retries of the request from another initiator or on a different port. PLOGI Port login.
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.0 User’s Guide target The storage-device endpoint of a SCSI session. Initiators request data from targets (usually disk-drives, tape-drives, or other media devices). Typically, a SCSI peripheral device is the target but an adapter may, in some cases, be a target. A target can contain many LUNs. A target is a device that responds to a request by an initiator (the host system).
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.
Index A acronyms, definitions of Glossary-1 adapter port beacon, definition of Glossary-1 adapter port, definition of Glossary-1 adapters definition of Glossary-1 identifying on VMware ESX 3-1 installing hardware 2-2 supported 1-4 agreements, license x arbitrated loop, definition of Glossary-1 audience, intended vii automated local host management, supported 2-2 B basic input output system, See BIOS BIOS definition of Glossary-1 searching for SCSI devices 3-13 utility, Fast!UTIL 3-5 blue text, meaning of x
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.
Fibre Channel and Converged Network Adapters for VMware ESX/ESXi 4.
Corporate Headquarters QLogic Corporation 26650 Aliso Viejo Parkway Aliso Viejo, CA 92656 949.389.6000 Europe Headquarters QLogic (UK) LTD. Quatro House Lyon Way, Frimley Camberley Surrey, GU16 7ER UK www.qlogic.com +44 (0) 1276 804 670 © 2009 QLogic Corporation. Specifications are subject to change without notice. All rights reserved worldwide. QLogic and the QLogic logo are registered trademarks of QLogic Corporation.
