Users Guide
Table Of Contents
- Revision History
- Table of Contents
- Regulatory and Safety Approvals
- Functional Description
- Network Link and Activity Indication
- Features
- Software and Hardware Features
- Virtualization Features
- VXLAN
- NVGRE/GRE/IP-in-IP/Geneve
- Stateless Offloads
- UDP Fragmentation Offload
- Stateless Transport Tunnel Offload
- Multiqueue Support for OS
- SR-IOV Configuration Support Matrix
- SR-IOV
- Network Partitioning (NPAR)
- RDMA over Converged Ethernet – RoCE
- Supported Combinations
- Installing the Hardware
- Software Packages and Installation
- Windows Driver Advanced Properties and Event Log Messages
- Teaming
- System-level Configuration
- ISCSI Boot
- VXLAN: Configuration and Use Case Examples
- SR-IOV: Configuration and Use Case Examples
- NPAR – Configuration and Use Case Example
- RoCE – Configuration and Use Case Examples
- DCBX – Data Center Bridging
FeaturesNetXtreme-E User’s Manual
September 4, 2019 • NetXtreme-E-UG103 Page 29
SR-IOV
The PCI-SIG defines optional support for Single-Root IO Virtualization (SR-IOV). SR-IOV is designed to allow
access of the VM directly to the device using Virtual Functions (VFs). The NIC Physical Function (PF) is divided
into multiple virtual functions and each VF is presented as a PF to VMs.
SR-IOV uses IOMMU functionality to translate PCI-E virtual addresses to physical addresses by using a
translation table.
The number of Physical Functions (PFs) and Virtual Functions (VFs) are managed through the UEFI HII menu,
the CCM, and through NVRAM configurations. SRIOV can be supported in combination with NPAR mode.
Network Partitioning (NPAR)
The Network Partitioning (NPAR) feature allows a single physical network interface port to appear to the system
as multiple network device functions. When NPAR mode is enabled, the NetXtreme-E device is enumerated as
multiple PCI-E physical functions (PF). Each PF or “partition” is assigned a separate PCI-E function ID on initial
power on. The original PCI-E definition allowed for eight PFs per device. For Alternative Routing-ID (ARI)
capable systems, Broadcom NetXtreme-E adapters support up to 16 PFs per device. Each partition is assigned
its own configuration space, BAR address, and MAC address allowing it to operate independently. Partitions
support direct assignment to VMs, VLANs, etc., just as any other physical interface.
RDMA over Converged Ethernet – RoCE
Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) is a complete hardware offload
feature in the BCM5741X that allows RDMA functionality over an Ethernet network. RoCE functionality is
available in user mode and kernel mode application. RoCE Physical Functions (PF) and SRIOV Virtual
Functions (VF) are available in single function mode and in mutli-function mode (NIC Partitioning mode).
Broadcom supports RoCE in Windows, Linux, and VMWare.
Please refer to the following links for RDMA support for each operating system:
Windows
https://technet.microsoft.com/en-us/library/jj134210(v=ws.11).aspx
Redhat Linux
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Networking_Guide/ch-
Configure_InfiniBand_and_RDMA_Networks.html
VMware
https://pubs.vmware.com/vsphere-65/index.jsp?topic=%2Fcom.vmware.vsphere.networking.doc%2FGUID-
4A5EBD44-FB1E-4A83-BB47-BBC65181E1C2.html
Note: In the System Setup > Device Settings > [Broadcom 5741x Device] > Device Level
Configuration page, the user can enable NParEP to allow the NXE adapter to support up to 16 PFs
per device. For 2 port devices, this means up to eight PFs for each port.