Emulex® Drivers for VMware ESXi User Guide Release 12.
Broadcom, the pulse logo, Connecting everything, Avago Technologies, Avago, the A logo, Brocade, Emulex, ExpressLane, LightPulse, and OneCommand are among the trademarks of Broadcom and/or its affiliates in the United States, certain other countries, and/or the EU. Copyright © 2003–2020 Broadcom. All Rights Reserved. The term “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. For more information, please visit www.broadcom.com.
Emulex Drivers for VMware ESXi User Guide Table of Contents Chapter 1: Introduction ...................................................................................................................... 5 1.1 ESXi Compatibility ....................................................................................................................................................5 1.2 Supported Guest Operating Systems ...............................................................................................
Emulex Drivers for VMware ESXi User Guide Chapter 4: Troubleshooting the FC Driver ..................................................................................... 43 4.1 FC Driver Log Messages ........................................................................................................................................45 4.2 FC Driver Error Log Messages and Their Descriptions ......................................................................................
Emulex Drivers for VMware ESXi User Guide Chapter 1: Introduction This product supports the following Emulex® HBAs: LPe12000-series adapters LPe16000-series adapters LPe31000-series adapters LPe32000-series adapters LPe35000-series adapters The VMware ESXi drivers support the FC protocol. In addition, ESXi 7.0 supports NVMe over FC on Emulex LPe31000-series, LPe32000-series, and LPe35000-series adapters in initiator systems.
Emulex Drivers for VMware ESXi User Guide 1.2 Supported Guest Operating Systems The Emulex drivers support the following guest operating systems: CentOS 7.x CentOS 8.x Oracle Linux 7.x and Oracle Linux 8.x Oracle UEKR4 and R5 OVM 3.3.3 OVM 3.4.1 RHEL 7.7 and subsequent releases RHEL 8.0 and subsequent releases SLES 12 SP4 and subsequent releases SLES 15 SP1 and subsequent releases Windows Server 2012 and R2 Windows Server 2016 Windows Server 2019 XenServer Citrix 7.0 1.
Emulex Drivers for VMware ESXi User Guide Table 1: Acronyms and Abbreviations (Continued) Acronym/Abbreviation Description GA general availability GB gigabyte Gb/s gigabits per second GPT GUID partition table GUID globally unique identifier I/O input/output ID identifier IOCB input/output control block IOCTL input/output control IODM I/O Device Management KV key value LBA logical block address LIF logical interface LIP Loop Initiatialization Primitive LPFC LightPulse Fibre Ch
Emulex Drivers for VMware ESXi User Guide Chapter 2: Installation This section provides information for installing the ESXi driver for the FC protocol. 2.1 Installing the FC Driver This section provides installation information for the driver and the Emulex CIM Provider using the FC interface on ESXi systems. Before using this product, you need a working knowledge of FC storage devices. 2.1.
Emulex Drivers for VMware ESXi User Guide 3. Restart the hostd using the following command: /etc/init.d/hostd restart After the hostd has restarted, the elxfc namespace appears under esxcli. To install the esxcli elxfc plug-in for ESXi 7.0 and subsequent operating system releases, perform these steps: 1. On the ESXi host, install the component as follows: esxcli software component apply -d //Broadcom-ELX-esxclielxfc_-1OEM.700.0.0.-signed_component.
Emulex Drivers for VMware ESXi User Guide Table 2: Available Namespaces (Continued) Namespace Description iostat I/O statistics information of an Emulex FC HBA kv KV page information of an Emulex FC HBA logmessage Log message of an Emulex FC HBA in vmkernel.
Emulex Drivers for VMware ESXi User Guide 2.4.2 Updating Drivers with the VMware Component VMware recommends using the component to update software on VMware ESXi 7.0 and subsequent operating system platforms. To update a driver using the component, perform the following steps: 1. Depending on the driver, run the appropriate command to update the driver. To update the FC driver, run the following command: esxcli software component apply -d Broadcom-ELX-lpfc_1OEM.700.1.0.
Emulex Drivers for VMware ESXi User Guide Chapter 3: Configuration This chapter provides configuration information for the ESXi driver. 3.1 ESXi Command Line Tool Transition Beginning with the ESXi 6.5 release, VMware transitioned from esxcfg commands to esxcli commands. This section describes the transition to the esxcli commands. NOTE: Both sets of commands are supported in the ESXi 6.5 and later operating system releases. 3.1.1 ESXi CLI Implementation For ESXi 6.
Emulex Drivers for VMware ESXi User Guide 3.2.1 Configuration Methods for FC Driver Parameters Configure the driver parameters using any of the following methods: Permanent (global) Permanent (per adapter) Temporary (global) Temporary (per adapter) The Emulex HBA Manager application supports all four configuration methods for driver parameters. Refer to the Emulex HBA Manager Application User Guide or the Emulex HBA Manager Application for VMware vCenter User Guide for more information.
Emulex Drivers for VMware ESXi User Guide 3. To reboot the server, type the following command: reboot The new setting is used when the driver reloads. To verify the setting, type the following command: esxcli system module parameters list -m lpfc 3.2.1.1.2 Examples of Permanent Per-Adapter Configuration The following example sets lun_queue_depth to 20 (the default is 64) for adapter 1. 1.
Emulex Drivers for VMware ESXi User Guide The following information applies to the ExpressLane feature: ExpressLane is not supported on LPe12000-series adapters. An ExpressLane LUN must be enabled in the driver before it can be used by the Emulex HBA Manager application for VMware vCenter. Additionally, the priority value to mark each of the ExpressLane frames must be specified to the FC driver through the appropriate driver parameters.
Emulex Drivers for VMware ESXi User Guide 3.2.4 Trunking Support Trunking enables you to combine multiple physical FC links to form a single logical link (aggregated port). The aggregated port’s maximum link speed is the sum of the maximum link speeds of the individual physical links comprising the aggregated port. For example, an aggregated port comprised of two physical links running at 32Gb/s each will have a potential logical (aggregate) link speed of 64Gb/s.
Emulex Drivers for VMware ESXi User Guide Following is an example of an LPe35004 adapter, showing 2-lane port aggregation with a single SCSI host.
Emulex Drivers for VMware ESXi User Guide 3.2.7 FC Driver Module Parameters Table 4 lists the FC driver module parameters, their descriptions, and their corresponding values in ESXi native mode. Dynamic parameters do not require a system reboot for changes to take effect. NOTE: Refer to the VMware documentation for quick boot instructions. Table 4: FC Driver Parameters Module Parameter Description Values lpfc_compression_log Defines how often the Default = 300 compression logs are written Min.
Emulex Drivers for VMware ESXi User Guide Table 4: FC Driver Parameters (Continued) Module Parameter Description Values lpfc_enable_fc4_type Enables the FC4 type Default = 3 supported. Valid values are: Min. = 1 1 = Register FCP only Max. = 3 3 = Register both FCP and NVMe This parameter requires a system reboot or quick boot. lpfc_enable_fcp_priority Enables or disables FCP priority. This parameter requires a system reboot or quick boot.
Emulex Drivers for VMware ESXi User Guide Table 4: FC Driver Parameters (Continued) Module Parameter Description Values Comments lpfc_enable_SmartSAN Enables HPE Smart SAN functionality. The function works with FDMI-2 to provide enhanced fabric diagnostics. This parameter requires a system reboot or quick boot. 0 = Disable (default) 1 = Enable If lpfc_enable_SmartSAN is set to 1, the driver automatically supports FDMI-2.
Emulex Drivers for VMware ESXi User Guide Table 4: FC Driver Parameters (Continued) Module Parameter Description lpfc_link_speed Selects link speed. Valid values Default = 0 are: Min. = 0 0 = Auto select Max. = 32 2 = 2 Gigabaud 4 = 4 Gigabaud 8 = 8 Gigabaud 16 = 16 Gigabaud 32 = 32 Gigabaud This parameter is dynamically updated. Setting this option incorrectly can cause the adapter to fail to initialize. lpfc_log_verbose Verbose logging bit-mask.
Emulex Drivers for VMware ESXi User Guide Table 4: FC Driver Parameters (Continued) Module Parameter Description lpfc_max_scsicmpl_time Uses the SCSI command Default = 0 completion time to control Min. = 0 queue depth to the device. Max. = 60,000 0 = SCSI command completion time is not used for controlling I/O queue depth. N = I/O queue depth is controlled to limit the I/O completion time to N ms. This parameter is dynamically updated.
Emulex Drivers for VMware ESXi User Guide Table 4: FC Driver Parameters (Continued) Module Parameter Description Values lpfc_ras_fwlog_buffsize Buffer size to be allocated in host memory for firmware logging. This parameter requires a system reboot or quick boot. This parameter is supported only on Default = 0 LPe31000-series, LPe32000-series, and Min. = 0 LPe35000-series adapters. Max.
Emulex Drivers for VMware ESXi User Guide Table 4: FC Driver Parameters (Continued) Module Parameter Description Values Comments lpfc_suppress_link_up Suppresses link up at initialization: 0x0 = Bring up link 0x1 = Do not bring up link 0x2 = Never bring up link This parameter requires a system reboot or quick boot. Default = 0 Min. = 0 Max. = 2 Change this parameter to assist with SAN issues during ESXi boot. lpfc_suppress_rsp Negotiates whether to Default = 1 suppress the RSP Min.
Emulex Drivers for VMware ESXi User Guide Table 4: FC Driver Parameters (Continued) Module Parameter Description Values lpfc_use_adisc Uses address discovery on 0 = Disable rediscovery, initiated by RSCN, (default) to authenticate FCP devices 1 = Enable instead of port login. This parameter is dynamically updated. — lpfc_use_msi Uses preferred MSI-X interrupt Default = 2 mode, if possible. Min. = 0 0 = MSI disabled (INTx Max.
Emulex Drivers for VMware ESXi User Guide Table 4: FC Driver Parameters (Continued) Module Parameter Description lpfc_xri_split When enabled, this parameter Default = 50% defines the division of XRI Min. = 10% resources between SCSI and Max. = 90% NVMe. This parameter is only used if the value of the lpfc_enable_fc4_type parameter is 3—register both FCP and NVMe and the port is not configured for NVMeT. The supported values are in percentages.
Emulex Drivers for VMware ESXi User Guide Table 6: Adapter Attributes for the lpfc_fdmi_on Parameter Hexadecimal Value Information Type 0x001 Node Name 0x002 Manufacturer 0x003 Serial Number 0x004 Model 0x005 Model Description 0x006 Hardware Version 0x007 Driver Version 0x008 Option ROM Version (boot code) 0x009 Firmware Version 0x00A Operating System Name and Version 0x00B Maximum Common Transport Payload Length 0x00C Node Symbolic Name 3.2.
Emulex Drivers for VMware ESXi User Guide Functional Nonfunctional VPD Firmware Parameters Trunking Instead of Emulex HBA Manager for Windows, use the Emulex HBA Manager for VMware CLI (elxvcpcmd) or the esxcli elxmgmt utilities to manage HBAs installed on ESXi 6.7 Update 1. 3.2.9.2 Using the Emulex HBA Manager for VMware vCenter The Emulex HBA Manager application for VMware vCenter uses the standard CIM interface to manage the adapters and supports CIM-based device and adapter management.
Emulex Drivers for VMware ESXi User Guide NOTE: The following information applies to vPorts: Ensure you are using the latest recommended firmware for vPort functionality. Check the Broadcom website for the latest firmware. Loop devices and NPIV are not supported on the same port at the same time. If you are running a loop topology and you create a vPort, the vPorts link state is offline. VMware ESXi supports fabric mode only. You can create vPorts only on 8, 16, and 32 GFC adapters.
Emulex Drivers for VMware ESXi User Guide 3.2.13 Adjusting the LUN Queue Depth Inadequate queue depth is one of the reasons for low performance. If you are using VVols and you experience low performance, you might want to adjust the queue depth. To change the LUN queue depth, perform the following steps: 1.
Emulex Drivers for VMware ESXi User Guide To configure NVMe over FC on a NetApp target, perform the following steps: 1. Create a virtual server by typing the following command: vserver create -vserver 2. Display the available protocols by typing the following command: vserver show-protocols -vserver Information similar to the following is displayed: Vserver: Protocols: nfs, cifs, fcp, iscsi, ndmp, nvme 3.
Emulex Drivers for VMware ESXi User Guide 9. Verify the sizes of the created volumes by typing the following command: volume show -vserver Information similar to the following is displayed. Vserver Volume Aggregate State Type Size Available Used% ------------- ---------- ---------- ---------- ----- ----- ---------- ------- vserver_name vol_name1 aggr1 online RW 5GB 4.75GB 0% vserver_name vol_name2 aggr2 online RW 5GB 4.
Emulex Drivers for VMware ESXi User Guide 16. Verify the mapping by typing the following command: vserver nvme subsystem map show -vserver Information similar to the following is displayed: Vserver Subsystem NSID Namespace Path ----------------------------------------------------------------------------nvme_test nvme_ss1 00000001h /vol/nvme_vol/nvme_ns nvme_ss2 00000001h /vol/nvme_vol2/nvme_ns2 2 entries were displayed. NVMe over FC is configured on the Netapp target.
Emulex Drivers for VMware ESXi User Guide 6. Verify the connected controller list by typing the following command: esxcli nvme controller list Information similar to the following is displayed: Con-tro ller Number Adapter Name Trans -port Is Type Online -------------------------------------------------------- ------ ------ ---- ----nqn.1992-08.com.netapp:sn.54a936ec00ac11ea829d00a0986e444a:subsystem. 262 nvmevs_paramesxi7_ss01#vmhba64#203c00a0986e4449:203e00a0986e4449 vmhba64 FC true nqn.1992-08.
Emulex Drivers for VMware ESXi User Guide You must now create datastores on the NVMe namespaces. To create datastores using the partedUtil and vmkfstools CLI commands, follow the instructions in Section 3.4.1, Creating Datastores Using CLI Commands. To create datastores using the vSphere Client GUI, go to Section 3.4.2, Creating Datastores Using the vSphere Client GUI. 3.4.
Emulex Drivers for VMware ESXi User Guide 2. Select Create new VMFS datastore and click Next. The Select device dialog appears.
Emulex Drivers for VMware ESXi User Guide 3. In the Name field, type the name of the datastore you want to create. Select an unused namespace on which to create the datastore. Click Next. The Select partitioning options dialog appears.
Emulex Drivers for VMware ESXi User Guide 4. Select the desired partitioning options and click Next. The Ready to complete dialog displays information about the selections you have made. 5. Click Finish. The Datastores tab appears. Repeat this procedure for each namespace on your server, and then proceed to Section 3.4.3, Completing the Initiator Configuration.
Emulex Drivers for VMware ESXi User Guide 3.4.3 Completing the Initiator Configuration Create VMs on the each datastore that you created in Section 3.4.1, Creating Datastores Using CLI Commands or Section 3.4.2, Creating Datastores Using the vSphere Client GUI. Each VM must have at least four CPUs and 4 GB of memory. Perform the following steps using the vSphere Client GUI: 1. To create a new virtual machine, use the New Virtual Machine wizard following the instructions provided by VMware.
Emulex Drivers for VMware ESXi User Guide 4. Click Add other device and select NVMe controller to add an NVMe controller on the VM. 5. Click Add hard disk to attach the NVMe datastore as a new standard hard disk.
Emulex Drivers for VMware ESXi User Guide 6. Select the NVMe controller on which to attach the new namespace, and click Next.
Emulex Drivers for VMware ESXi User Guide 7. Verify your selections, and click Finish. Repeat this procedure for each VM, and power-on the VMs. NVMe over FC is configured on the initiator system.
Emulex Drivers for VMware ESXi User Guide Chapter 4: Troubleshooting the FC Driver Your system may operate in an unexpected manner in certain circumstances. This chapter explains many of these circumstances and offers one or more workarounds for each situation. Table 7 identifies some of the common situations and their potential resolutions for the FC driver. Table 7: Troubleshooting the FC Driver Situation Resolution The port link fails to come up.
Emulex Drivers for VMware ESXi User Guide Table 7: Troubleshooting the FC Driver (Continued) Situation Resolution The FC driver does not discover all remote ports in the configuration switch zone. Some initiators or targets may appear to be missing. Evaluate your switch zone. Count how many entries are there and add at least seven more (to account for fabric logins). If the sum exceeds 128, you must increase the driver's node table size. The following commands increase it to 200 entries. See Section 3.
Emulex Drivers for VMware ESXi User Guide b.
Emulex Drivers for VMware ESXi User Guide Table 8: Message Log Table (Continued) LOG Message Verbose Mask Definition Verbose Bit Verbose Description LOG_FCP_ERROR 0x00001000 Log errors, not underruns LOG_LIBDFC 0x00002000 Libdfc events LOG_VPORT 0x00004000 NPIV events LOG_MEM_SLAB 0x00008000 Memory tracker slab logging LOG_EVENT 0x00010000 Common transport, TEMP, DUMP, logging LOG_AUTH 0x00020000 DH-CHAP logging LOG_FCP_UNDER 0x00040000 FCP underruns errors LOG_TASKMGMT 0x00100000
Emulex Drivers for VMware ESXi User Guide 3042 No Authentication configuration found for request No authentication configured. ACTION: Need to configure authentication, Refer to the Emulex Drivers for VMware ESXi User Guide for instructions. 0264 FDMI cmdcode not supported Driver sends an invalid FDMI command. ACTION: Report the error to Broadcom Technical Support.
Emulex Drivers for VMware ESXi User Guide 9021 Failed to issue IOCB Failed to issue IOCB command to the firmware. ACTION: Report the error to Broadcom Technical Support. 7062 VMID cmdcode not supported Invalid VMID Common transport command. ACTION: Report the error to Broadcom Technical Support. 7157 Failed Gen_Req status Failed to send the Common transport command. ACTION: Check the port connection and the switch configuration.
Emulex Drivers for VMware ESXi User Guide 0136: PLOGI completes to NPort completion PLOGI completes to N_Port completion. ACTION: No action required. 0137: No retry ELS command to remote No retry Extended Link Service command that is sent to remote node ACTION: No action required. 0237: Pending Link Event during Discovery: State Received link event during discovery. Causes discovery restart. ACTION: No action required, unless this issue persists. If persistent, check cabling.
Emulex Drivers for VMware ESXi User Guide 2852 Failed to allocate mbox memory Mailbox allocation error. ACTION: No action required 3334 Delay fc port discovery for seconds Delay the node port discovery since fabric parameters changed. ACTION: No action required. 2556: UNREG_VFI mbox allocation failed HBA state Could not allocate memory for UNREG_VFI mailbox command. ACTION: No action required.
Emulex Drivers for VMware ESXi User Guide 2831 FLOGI response with cleared Fabric bit fcf_index Switch Name Fabric Name When the driver completed an FLOGI, the common service parameters did not indicate an F_Port or N_Port remote node. The driver treats this issue as an error. ACTION: Validate the external cable connection and F_Port/N_Port configuration. If this issue persists, report the error to Broadcom Technical Support.
Emulex Drivers for VMware ESXi User Guide 2754: PRLI failure DID: Status:/. A process login to failed either by the driver, the firmware, or the target. The and indicate why the process login failed. ACTION: Check the remote N_Port functionality. If the issue persists, report the error to Broadcom Technical Support. 2755: ADISC failure DID: Status:/.
Emulex Drivers for VMware ESXi User Guide 7105 failed to allocate mailbox memory); Memory allocation for mailbox failed. ACTION: No action required. 0820 FLOGI Failed BBCredit Not Supported An FLOGI failed as the other endpoint was not ready/busy. ACTION: No action required. 1305 Link Down Event received Data: Link down event was received. ACTION: Check the connectivity between the fabric nodes.
Emulex Drivers for VMware ESXi User Guide 0542 lpfc_create_static_vport failed to allocate mailbox memory); Out of memory while allocating mailbox command for vPort information. ACTION: No action required. 2018 REG_VFI mbxStatus error x HBA state x Mailbox command returned error. ACTION: This error could indicate a software driver or firmware issue. If this issue persists, report the error to Broadcom Technical Support.
Emulex Drivers for VMware ESXi User Guide 2554 Could not allocate memory for fcf record The memory allocation to store fcf_record failed. ACTION: No action required. 2555 UNREG_VFI mbxStatus error HBA state The UNREG_VFI mailbox command failed. ACTION: No action required. 2798 Unreg_vpi failed vpi mb status The UNREG_VPI mailbox command failed. ACTION: No action required. 3274 ScsiNotifyPathStateChangeAsync SAdapter Num TID , DID .
Emulex Drivers for VMware ESXi User Guide 2606 No NPIV Fabric support The fabric does not support NPIV port. ACTION: No action required. 2607 Failed to allocate init_vpi mailbox The adapter failed to allocate the init_vpi mailbox command. ACTION: No action required. 0260 Register NameServer error: The REG_LOGIN mailbox command has failed for the NameServer. ACTION: If the issue persists, report the error to Broadcom Technical Support.
Emulex Drivers for VMware ESXi User Guide 3298 ScsiNotifyPathStateChangeAsync SAdapter Num TID DID Path change notification is successful. ACTION: No action required. 3299 ScsiNotifyPathStateChangeAsync Failed: SAdapter Num TID, DID Path change notification has failed. ACTION: Report the error to Broadcom Technical Support. 0258 Register Fabric login error: The REG_LOGIN for the fabric failed.
Emulex Drivers for VMware ESXi User Guide 0259 No NPIV Fabric support Switch does not support NPIV. ACTION: Check the switch configuration. 2731 Cannot find fabric controller node Fabric controller ndlp is not present. ACTION: Report the error to Broadcom Technical Support. 1815 Could not issue unreg_did (default rpis) status ; UNREG_DID command failure. ACTION: Report the error to Broadcom Technical Support.
Emulex Drivers for VMware ESXi User Guide 0319 READ_SPARAM mbxStatus error hba state The driver issued a READ_SPARAM mailbox command to the HBA that failed. ACTION: This error could indicate a firmware or hardware issue. Report the error to Broadcom Technical Support. 3233 Error while allocating MSIX interrupts fallback to MSI MSIX interrupt registration fails. Falling back to MSI. ACTION: No action required.
Emulex Drivers for VMware ESXi User Guide 0499 Failed allocate fast-path FCP CQ () The completion queue event for the fast path could not be allocated. ACTION: Check the ESX host configuration and memory usage to free up some resources. 3258 PathClaim thread death pending.... LPFC worker thread is going down. ACTION: No action required. 3259 PathClaim Thread create in state LPFC thread create state. ACTION: No action required.
Emulex Drivers for VMware ESXi User Guide 3149 Fast-path FCP WQ array not allocated Fast-path FCP WQ array not allocated. ACTION: Check the system configuration and memory usage. 2562 Failure to allocate an ELS sgl entry: Failure to allocate an Extended Link Service SGL entry. ACTION: Report the error to Broadcom Technical Support. 0505 Failed allocate slow-path MQ Failed allocate slow-path multiqueue. ACTION: Check the system configuration and memory usage.
Emulex Drivers for VMware ESXi User Guide 0420 MSI-X alloc cookie MSI-X allocation failed for LPe12000-series adapter. ACTION: Check the ESX configuration, if issue is still seen contact VMware or Broadcom support team. 9035 Destroying Physical port refcnt Destroying physical port. ACTION: No action required. 3296 LPFC_SLI_EVENT_TYPE_MISCONFIGURED event: Invalid link Invalid link number for LPFC_SLI_EVENT_TYPE_MISCONFIGURED event.
Emulex Drivers for VMware ESXi User Guide 2563 Failure to allocate an ELS mbuf Failed to allocate an Extended Link Service memory buffer. ACTION: Check the system configuration and memory usage. 2889 Port Overtemperature event, taking port offline Data: " Temperature of the adapter is very high. ACTION: Check the cooling mechanism of server. Ensure that the server fans are not blocked. Shut down the server if the airflow is restricted. Contact Broadcom Technical Support.
Emulex Drivers for VMware ESXi User Guide 2813 Mgmt I/O is Blocked - mbox cmd still active. The HBA management interface is marked as blocked in preparation for an online or offline state transition. All user space access to the HBA through the libdfc interface will be blocked. ACTION: No action required. Notification of a run-state change only.
Emulex Drivers for VMware ESXi User Guide 2894 SLI_INTF reg contents invalid sli_intf reg SLI_INTF reg contents invalid sli_intf reg ACTION: No action required. 2897 The mboxq allocation failed The driver could not allocate mbox queue. ACTION: No action required. 0479 Deferred Adapter Hardware Error Data: An adapter hardware error was sent to the driver. ACTION: Perform a dump using hbacmd.
Emulex Drivers for VMware ESXi User Guide 0391 Error during rpi post operation The driver while trying to post pages to the firmware to keep target login information encountered a failure. ACTION: Unload and reload the driver. 0493: SLI_CONFIG_SPECIAL mailbox failed with status Mailbox command failed. ACTION: Ensure that the adapter firmware is current. Unload and reload the driver. 0492 Unable to allocate memory for issuing SLI_CONFIG_SPECIAL mailbox command Mailbox memory pool allocation error.
Emulex Drivers for VMware ESXi User Guide 0525: Failed to setup FOF EQ lock The driver failed to set up the FOF event queue lock. ACTION: Unload and reload the driver. 0401 Link fault code: The link went into fault mode. ACTION: Check the cables and reload the driver. 3231 Error while allocating MSI interrupt fallback to INTx The driver failed to allocate MSI interrupts, so falling back to INTx. ACTION: No action required.
Emulex Drivers for VMware ESXi User Guide 0714 SCSI layer issued Bus Reset Data The SCSI layer is requesting the driver to abort all I/Os to all targets on this HBA. ACTION: Check the state of the targets in question. 0524: Failed to setup EQ lock The driver failed to setup the event queue lock. ACTION: Unload and reload the driver. 0022 FATAL - unsupported SLI4 interface type - The adapter is not supported by the driver.
Emulex Drivers for VMware ESXi User Guide 0427: Cannot re-enable interrupt after slot reset The driver was not able to enable the interrupt after an adapter reset. ACTION: Unload and reload the driver. 0434 MIS-X Interrupt Name Properties failed MSIx initialization failed. ACTION: Unload and reload the driver. If the issue persists, report the error to Broadcom Technical Support. 0487 PCI enable MSI-X failed Enabling MSIx failed. ACTION: Unload and reload the driver.
Emulex Drivers for VMware ESXi User Guide 4667 Error reading Cgn Params Unable to read congestion firmware parameters from adapter. ACTION: No action required. If the issue persists, report the error to Broadcom Technical Support. 1431: Invalid HBA PCI-device group: An invalid adapter PCI-device group was detected. ACTION: No action required.
Emulex Drivers for VMware ESXi User Guide 2605: lpfc_dump_static_vport: memory allocation failed Memory allocation failed. ACTION: Try closing other applications to free some memory. If this issue persists, report the error to Broadcom Technical Support. 3569 dump type 3 page 0xA0 allocation failed Memory allocation failure. ACTION: Try closing other applications to free some memory. If this issue persists, report the error to Broadcom Technical Support.
Emulex Drivers for VMware ESXi User Guide 3200 No IO Space Err. MaxMappable Not enough I/O address space available to map elements. ACTION: This is a software error. If this issue persists, report the error to Broadcom Technical Support. 3374 Slab Obj for ID Alloc Free Slab memory to be freed not found. ACTION: No action required.
Emulex Drivers for VMware ESXi User Guide 2945: SLI_CONFIG(mse) rd, ext_buf_cnt() out of range() SLI_CONFIG read mailbox failed. ACTION: No action required. 3383 OAS detected unsupported string version Unsupported OAS version. ACTION: This is a software error. If this issue persists, report the error to Broadcom Technical Support. 3381 No OAS version pointer from caller No OAS version passed. ACTION: This is a software error.
Emulex Drivers for VMware ESXi User Guide 3377 OAS slab empty OAS slab memory allocation failure. ACTION: This is a software driver error. If this issue persists, report the error to Broadcom Technical Support. 3444 Failed SLI_CONFIG(hbd) wr Write to interface type. ACTION: This is a software error. If this issue persists, report the error to Broadcom Technical Support. 3032 SLI_CONFIG(hbd) rd to interface type: Read to interface type. ACTION: This is a software error.
Emulex Drivers for VMware ESXi User Guide 3172 Attempted to add already registered entry with vmid: Attempted to re-register a VMID entry. ACTION: This is a software driver error. If this issue persists, report the error to Broadcom Technical Support. 3125: Not receiving unsolicited event Mgmt Loopback Test. IOCB timed out. ACTION: This is a software driver error. If this issue persists, report the error to Broadcom Technical Support.
Emulex Drivers for VMware ESXi User Guide 3170 Update process failed for entry[] with status: VMID update process failed for entry. ACTION: This is a software driver error. If this issue persists, report the error to Broadcom Technical Support. 1936 VMK_NO_MEMORY Kernel resource unavailale); The driver failed to allocate a DMA buffer during a loopback test. ACTION: This is a software driver error. If this issue persists, report the error to Broadcom Technical Support.
Emulex Drivers for VMware ESXi User Guide 3252 WQ[] doorbell offset not supported: Unsupported WQ doorbell offset. ACTION: Report the error to Broadcom Technical Support. ():2529 Mailbox command cannot issue Data: Cannot issue mailbox because another mailbox command is already active. ACTION: Report these errors to Broadcom Technical Support.
Emulex Drivers for VMware ESXi User Guide 2707 Ring handler: Failed to allocate iocb Rctl Type received IOCB is not available to process unsolicited requests. ACTION: Report these errors to Broadcom Technical Support. 2508 WQ_DESTROY mailbox failed with status add_status , mbx status WQ_DESTROY mailbox command failure. ACTION: Report the error to Broadcom Technical Support.
Emulex Drivers for VMware ESXi User Guide 0436 Adapter failed to init, timeout, status reg , FW Data: A8 AC Adapter initialization failed. ACTION: Report the error to Broadcom Technical Support. 0602 Failed to allocate CQ_EVENT entry The completion queue event pool is empty. ACTION: Report the error to Broadcom Technical Support. 1419 Invalid HBA PCI-device group: Invalid PCI device group.
Emulex Drivers for VMware ESXi User Guide 3471 : RAS FW-Logging Enable Failed : RAS FW logging is not enabled. ACTION: No action required. () 0323 Unknown Mailbox command (/) Cmpl Unknown mailbox command. ACTION: Report the error to Broadcom Technical Support. 2983 Allocated DMA memory size () is less than the requested DMA memory size () Allocated mailbox command memory size is less than requested size.
Emulex Drivers for VMware ESXi User Guide 0442 Adapter failed to init, mbxCmd x CONFIG_PORT, mbxStatus x Adapter initialization failed when issuing a CONFIG_PORT mailbox command. ACTION: This error could indicate a hardware or firmware issue. If this issue persists, report the error to Broadcom Technical Support. 0443 Adapter did not grant BlockGuard Adapter does not support the BlockGuard feature. ACTION: No action required.
Emulex Drivers for VMware ESXi User Guide 3262 RQ doorbell format not supported: The doorbell format is not supported for this receive queue. ACTION: Report the error to Broadcom Technical Support. 4111 UNREG cmpl deferred clr on NPort Data: There is some deferred event to be processed after UNREG_LOGIN mailbox completion. ACTION: No action required.
Emulex Drivers for VMware ESXi User Guide 0451 Adapter did not grant BlockGuard The adapter firmware did not allow the driver to enable BlockGuard support. It is not a fatal error. ACTION: No action required. 0599 Firmware Dump Image Present on Adapter "Driver identified that a firmware dump image is present. ACTION: No action required. 2616 txcmplq not empty after The completion queue is not yet drained after a particular time. ACTION: No action required.
Emulex Drivers for VMware ESXi User Guide 3161 Failure to post els sgl to port. The driver has failed to post the Extended Link Service SGL to the firmware. ACTION: Report the error to Broadcom Technical Support. 0343 Ring Cannot find buffer for an unsolicited iocb. tag The driver does not have a buffer for processing an unsolicited command. ACTION: No action required. 0342 Ring Cannot find buffer for an unsolicited iocb.
Emulex Drivers for VMware ESXi User Guide 0349 rc should be MBX_SUCCESS The next mailbox command on the mailbox queue has failed. ACTION: No action required. 3367 LWPD Memory Allocation Failed for FW Logging The DMA memory allocation for getting firmware logging has failed. ACTION: No action required. 2564 POST_SGL_BLOCK mailbox command failed status add_status mbx status The list of XRI SGEs failed to be registered with the firmware. ACTION: No action required.
Emulex Drivers for VMware ESXi User Guide 0410 Cannot find virtual addr for mapped buf on ring Data The driver cannot find the specified buffer in its mapping table. Therefore, it cannot find the virtual address needed to access the data. ACTION: This error could indicate a software driver or firmware issue. If this issue persists report these errors to Broadcom Technical Support.
Emulex Drivers for VMware ESXi User Guide 0370 Invalid completion queue type The event queue entry is not for a mailbox or a work queue entry. ACTION: No action required. 3370 FW LOG mailbox completed with status add_status , mbx status Enabling firmware logging in host memory failed. ACTION: No action required. 2752 KILL_BOARD command failed retval The KILL_BOARD mailbox command is failed by the firmware.
Emulex Drivers for VMware ESXi User Guide 0390 iotag does not map to outstanding IO. last cmd , XRI , wqidx , flag The I/O tag does not map to the outstanding I/O list of the driver. ACTION: This error could indicate a software driver, firmware, or a hardware issue. Report the error to Broadcom Technical Support. 1807 IOCB <> failed. No vport The IOCB does not have the vPort reference in it. ACTION: This is a software issue.
Emulex Drivers for VMware ESXi User Guide Chapter 5: Troubleshooting the NVMe Driver NVMe over FC on VMware ESXi 7.0 might operate in an unexpected manner in certain circumstances. This chapter describes such situations and their potential resolutions. This chapter also provides information about NVMe-related esxcli commands that are used in resolving issues. 5.
Emulex Drivers for VMware ESXi User Guide Table 9: Troubleshooting the NVMe Driver (Continued) Situation Resolution A newly added NVMe controller is not visible. Stale entries cannot be removed by a rescan. Perform an initiator port link reset. For example, you can issue a LIP to an initiator port by typing the following command: /usr/lib/vmware/vmkmgmt_keyval/vmkmgmt_keyval -i / Emulex -k adapter -s lip where is the HBA number. 5.
Emulex Drivers for VMware ESXi User Guide 2. To view the NVMe controller status, type the following command: esxcli nvme controller list Information similar to the following is displayed: Name Controller Number Adapter Trans -port Is Type Online -------------------------------------------------------- ------ ------ ---- ----nqn.1992262 08.com.netapp:sn.54a936ec00ac11ea829d00a0986e444a:subsystem.nvmevs_pa ramesxi7_ss01#vmhba64#203c00a0986e4449:203e00a0986e4449 vmhba64 FC true nqn.1992264 08.com.
Emulex Drivers for VMware ESXi User Guide Information similar to the following is displayed: Key ‘node’: lpfc Node page: WWNN WWPN SCSI ID DID Type Status 10:00:00:05:33:7f:2f:56 20:0e:00:05:33:7f:2f:56 xfffffe Fabric Node ok 10:00:00:05:33:7f:2f:56 21:fc:00:05:33:7f:2f:56 xfffffc Fabric Node ok 20:3c:00:a0:98:6e:44:49 20:3e:00:a0:98:6e:44:49 xcf1503 Target Node ok 5.2.
Emulex Drivers for VMware ESXi User Guide 5.2.5 Viewing Multipath Information To view multipath information, type the following command: esxcfg-mpath -b Information similar to the following is displayed: uuid.a597cb7a665c410e88e59f3a3d4c134e : NVMe Fibre Channel Disk (uuid.
Emulex Drivers for VMware ESXi User Guide Appendix A: esxcli Management Tool A.1 Usage The esxcli management tool supports both local and remote management.
Emulex Drivers for VMware ESXi User Guide A.2 esxcli Management Commands The commands listed in this section are for local management only. You can add the remote management parameters to the local commands for remote management. The following table contains a list of esxcli management commands and their descriptions. Table 11: esxcli Management Commands Command Description authconfig delete -w Sets delete authentication for a specified HBA port.
Emulex Drivers for VMware ESXi User Guide Table 11: esxcli Management Commands (Continued) Command Description authconfig set -w -d -b -p -s -m -r -a -t Sets the authentication configuration parameters for a specified HBA port. d specifies the destination WWPN. It must be ff:ff:ff:ff:ff:ff:ff:ff for a switch or the actual WWPN for a target.
Emulex Drivers for VMware ESXi User Guide Table 11: esxcli Management Commands (Continued) Command Description bootparams set -w -p Sets the boot parameter and boot type for the specified HBA port. -t -v - b p specifies the parameter name. t specifies the boot type: {X86, EFI, OB}. v specifies the value to be set for the parameter.
Emulex Drivers for VMware ESXi User Guide Table 11: esxcli Management Commands (Continued) Command Description – EnableFABL EFI { State } (0=Disable, 1=Enable) – MaxLunsPerTarget EFI { Value } – DelayDeviceDiscovery EFI { Value } – SfsFlag OB – { State } (0=Disable, 1=Enable) Boot Device Parameters – D_ID All {Value [BootDev Value2]} – LUN All {Value [BootDev Value2]} – TargetWWPN All {Value [BootDev Value2]} – TargetID OB {Value } b specifies the boot device entry number { 0 to 7 } as shown in the outp
Emulex Drivers for VMware ESXi User Guide Table 11: esxcli Management Commands (Continued) Command Description dumpdirectory set -d Sets the dump directory. A dump directory must be set before a dump can be created. NOTE: The dump directory must have a subdirectory under the /vmfs/volumes directory to store Emulex HBA Manager dumps and the directory must exist at the time of setting. The directory path must not contain spaces.
Emulex Drivers for VMware ESXi User Guide Table 11: esxcli Management Commands (Continued) Command Description hbaport allnodeinfo list -w Shows the target node information for each target accessible by the adapter. hbaport attributes -w Lists port attributes of an Emulex FC HBA. hbaport bootcode -w -s Enables or disables boot code on the adapter port. s specifies the state of the boot code. Possible values are E and D. – E = Enable. – D = Disable.
Emulex Drivers for VMware ESXi User Guide Table 11: esxcli Management Commands (Continued) Command Description hbaport loopbacktest start Runs loopback tests on FC functions. -w -d d specifies the destination WWPN. It must be ff:ff:ff:ff:ff:ff:ff:ff -c -s [-p ][-o
Emulex Drivers for VMware ESXi User Guide Table 11: esxcli Management Commands (Continued) Command Description wwn read -w -t Reads the current WWPN for a specified HBA port. t specifies the type: (0, 1, 2, 3, 4) – 0 = Volatile. – 1 = Non-Volatile. – 2 = Factory Default – 3 = Current. – 4 = Configured. wwn restore -w -t Restores the WWPN for a specified HBA port. t specifies the type: (0, 1) – 0 = Restore Default WWNs. – 1 = Restore NVRAM WWNs.
Emulex Drivers for VMware ESXi User Guide Appendix B: lpfc Driver BlockGuard Functionality This appendix describes how to enable BlockGuard and set lpfc driver module parameters. B.1 Overview The BlockGuard feature checks the integrity of data read and written between the host and the disk through the SAN. This check is implemented through the DIF defined in the ANSI T10 standard. The Emulex lpfc driver supports T10 DIF Type 1.
Emulex Drivers for VMware ESXi User Guide B.2 Enabling BlockGuard BlockGuard is disabled by default. To enable it, the parameter lpfc_enable_bg must be passed to the driver as follows: esxcli system module parameters set -m lpfc -p "lpfc_enable_bg=1" Additional module parameters can be added to this line, separated by spaces. B.
Emulex Drivers for VMware ESXi User Guide Table 12: Initiator Operations (Continued) Initiator Operation Initiator Host <-> Adapter Adapter <-> Target Comment READ_STRIP Unprotected Protected Reads the data and protection data from the target. The adapter verifies data integrity, discards protection data, and transfers only the data to the initiator host. It does not send the protection data to the initiator host. Protection data is sent only on the SAN.
Emulex Drivers for VMware ESXi User Guide The following table shows how protection data gets placed for each supported profile. Table 14: Protection Data Placement for Supported Profiles Flag Value BlockGuard Profile Operation VMK_SCSI_TYPE1_PROT 1 A1 READ_STRIP/WRITE_INSERT VMK_SCSI_DIX_TYPE0_PROT 8 AST2 READ_INSERT/WRITE_STRIP VMK_SCSI_DIX_TYPE1_PROT VMK_SCSI_TYPE1_PROT 17 AST1 / C1 READ_PASS/WRITE_PASS B.4.
Emulex Drivers for VMware ESXi User Guide Appendix C: Using the VMID Feature on a Brocade Switch This appendix describes the setup required for the Broadcom ECD VMID feature to work on a Brocade switch. VMID requires a VMID-supported target and initiator to work. NOTE: The Broadcom ECD VMID feature is not supported on the LPe12000-series adapters. 1. Set up the Brocade switch. a. Install firmware version 8.0.1 or later on the switch. b.
Emulex Drivers for VMware ESXi User Guide Output similar to the following is shown: -----------------------------------------------------------Displaying results for Fabric -----------------------------------------------------------N_Port ID : 7e1200 Entity Name : Entity ID (ASCII) : 52 e1 b0 6f 8c 5f 66 6a-79 c3 a9 f6 32 4d 19 20 Entity ID (Hex) : 0x35322065312062302036662038632035662036362036612d37392063332061392066362033322034642031392032300 0 Application ID : 0x00000209h (521) -------------------------
Emulex Drivers for VMware ESXi User Guide Appendix D: Using the VMID Feature on a Cisco Switch This appendix describes the setup required for the Broadcom ECD VMID feature to work on a Cisco switch. VMID requires a VMID-supported target and initiator to work. NOTE: The Broadcom ECD VMID feature is not supported on the LPe12000-series adapters. 1. Set up the Cisco switch. a. Install firmware version 8.2 or later on the switch. b. The Broadcom ECD VMID feature is not enabled by default on Cisco switches.
Emulex Drivers for VMware ESXi User Guide 7.
Emulex Drivers for VMware ESXi User Guide Appendix E: NPIV Configuration This appendix describes how to configure NPIV on an ESXi host. Most switches come with NPIV support and the feature is enabled by default. To enable NPIV on an ESXi host using the VMware vCenter Server, perform the following steps: 1. Make sure a VM exists with at least one Mapped Raw LUN (RDM) assigned. Select Edit Settings > Virtual Hardware. Click New Hard disk. The existing RDMs are displayed, see Figure 3.
Emulex Drivers for VMware ESXi User Guide NOTE: You must have at least one WWNN and you can have one or many WWPNs. For every physical port that exists in the ESXi host, one NPIV port is created. Figure 4: Assign NPIV Ports f. Make a note of the newly assigned WWNNs and WWPNs (NPIV ports). 3. Manually configure the same WWNNs and WWPNs in the switch zone. 4. Configure the same WWNNs and WWPNs in the array management software. 5. Power-on the VM. The NPIV ports are created on power-on.
Emulex Drivers for VMware ESXi User Guide An output similar to the following is shown: Dumping all key-value instance names: Key Value Instance: vmhba67/Emulex Key Value Instance: vmhba66/Emulex Key Value Instance: vmhba5/Emulex Key Value Instance: vmhba4/Emulex Key Value Instance: vmhba72/Emulex Key Value Instance: vmhba2/Emulex Key Value Instance: vmhba1/vmw_ahci Key Value Instance: vmhba0/vmw_ahci Key Value Instance: lsi_mr3-54cd98f0a033bc00/LSI Incorporation NOTE: If no NPIV ports are shown, it indic
Emulex Drivers for VMware ESXi User Guide Interrupt Type: MSIX BlockGuard Disabled External DIF Enabled BBCredit Recovery Supported BBSCN default:8 min:1 max:15 [root@dhcp-10-123-183-217:~] lun -g Key 'lun': lpfc Lun Path Stats Page: /usr/lib/vmware/vmkmgmt_keyval/vmkmgmt_keyval -i vmhba66/Emulex -k LUN[0:3] WWNN 2f:f7:00:02:ac:01:90:40 WWPN 21:01:00:02:ac:01:90:40 path is ok qdepth 64 fcperr 0001 abts issue 000000 cnt 000000 lun_rst 0000 tx_cnt 00000008 partial_sns_cnt 0000 LUN[1:3] WWNN 2f:f7:00:02:ac
Emulex Drivers for VMware ESXi User Guide Appendix F: License Notices F.1 OpenSSL Notice This is a copy of the current LICENSE file inside the CVS repository. LICENSE ISSUES ============== The OpenSSL toolkit stays under a dual license, i.e. both the conditions of the OpenSSL License and the original SSLeay license apply to the toolkit. See below for the actual license texts. Actually both licenses are BSD-style Open Source licenses.
Emulex Drivers for VMware ESXi User Guide */ Original SSLeay License ----------------------/* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com) * All rights reserved. * * This package is an SSL implementation written * by Eric Young (eay@cryptsoft.com). * The implementation was written so as to conform with Netscapes SSL. * * This library is free for commercial and non-commercial use as long as * the following conditions are aheared to.
