Users Guide
Table Of Contents
- Revision History
- Table of Contents
- Regulatory and Safety Approvals
- Functional Description
- Network Link and Activity Indication
- Features
- Software and Hardware Features
- Virtualization Features
- VXLAN
- NVGRE/GRE/IP-in-IP/Geneve
- Stateless Offloads
- UDP Fragmentation Offload
- Stateless Transport Tunnel Offload
- Multiqueue Support for OS
- SR-IOV Configuration Support Matrix
- SR-IOV
- Network Partitioning (NPAR)
- RDMA over Converged Ethernet – RoCE
- Supported Combinations
- Installing the Hardware
- Software Packages and Installation
- Windows Driver Advanced Properties and Event Log Messages
- Teaming
- System-level Configuration
- ISCSI Boot
- VXLAN: Configuration and Use Case Examples
- SR-IOV: Configuration and Use Case Examples
- NPAR – Configuration and Use Case Example
- RoCE – Configuration and Use Case Examples
- DCBX – Data Center Bridging
SR-IOV: Configuration and Use Case ExamplesNetXtreme-E User’s Manual
September 4, 2019 • NetXtreme-E-UG103 Page 64
6. Enable Virtual Functions through Kernel parameters:
a. Once the driver is installed, lspci will display the NetXtreme-E NICs present in the system. Bus, device,
and Function are needed for activating Virtual functions.
b. To activate Virtual functions, enter the command shown below:
echo X >/sys/bus/pci/device/0000\:Bus\:Dev.Function/sriov_numvfs
A typical example would be:
echo 4 > /sys/bus/pci/devices/0000\:04\:00.0/sriov_numvfs
7. Check the PCI-E virtual functions:
a. The
lspci command will display the virtual functions with DID set to 16D3 for BCM57402/BCM57404/
BCM57406, 16DC for non-RDMA BCM57412/BCM57414/BCM57416, and 16C1 or RDMA enabled
BCM57412/BCM57414/BCM57416.
8. Use the Virtual Manager to install a Virtualized Client system (VMs).
Refer to the Linux documentation for Virtual Manager installation. Ensure that the hypervisor’s built in driver
is removed. An example would be
NIC:d7:73:a7 rtl8139. Remove this driver.
9. Assign a virtual function to the guest VMs.
a. Assign this adapter to a guest VM as a physical PCI Device. Refer to the Linux documentation for
information on assigning virtual functions to a VM guest.
10. Install
bnxt_en drivers on VMs:
a. On the guest VMs, copy the
netxtreme-bnxt_en-<version>.tar.gz source file and extract the tar.gz file.
Change directory to each driver and run
make; make install; modprobe bnxt_en (and bnxt_re if enabling
RDMA). Make sure that the driver loads properly by checking the interface using modinfo command. The
user may need to run
modprobe -r bnxt_en to unload existing or inbox bnxt_en module prior to loading
the latest built module.
11. Test the guest VM connectivity to external world:
a. Assign proper IP address to the adapter and test the network connectivity.
Windows Case
1. Enable SR-IOV in the NIC cards:
a. SR-IOV in the NIC card can be enabled using the HII menu. During the system boot, access the system
BIOS -> Device Settings -> NetXtreme-E NIC -> Device Level Configuration.
b. Set the Virtualization mode to SR-IOV.
Note: Use netxtreme-bnxt_en<version>.tar.gz to install both bnxt_re and bnxt_en for RDMA
functionality on SRIOV VFs.
Note: Ensure that the PF interfaces are up. VFs are only created if PFs are up. X is the number of VFs
that will be exported to the OS.