Users Guide
Table Of Contents
- Revision History
- Table of Contents
- Regulatory and Safety Approvals
- Functional Description
- Network Link and Activity Indication
- Features
- Software and Hardware Features
- Virtualization Features
- VXLAN
- NVGRE/GRE/IP-in-IP/Geneve
- Stateless Offloads
- UDP Fragmentation Offload
- Stateless Transport Tunnel Offload
- Multiqueue Support for OS
- SR-IOV Configuration Support Matrix
- SR-IOV
- Network Partitioning (NPAR)
- RDMA over Converged Ethernet – RoCE
- Supported Combinations
- Installing the Hardware
- Software Packages and Installation
- Windows Driver Advanced Properties and Event Log Messages
- Teaming
- System-level Configuration
- ISCSI Boot
- VXLAN: Configuration and Use Case Examples
- SR-IOV: Configuration and Use Case Examples
- NPAR – Configuration and Use Case Example
- RoCE – Configuration and Use Case Examples
- DCBX – Data Center Bridging
RoCE – Configuration and Use Case ExamplesNetXtreme-E User’s Manual
September 4, 2019 • NetXtreme-E-UG103 Page 74
Windows
Kernel Mode
Windows Server 2012 R12 and beyond invokes the RDMA capability in the NIC for SMB file traffic if both ends
are enabled for RDMA. Broadcom NDIS miniport bnxtnd.sys v20.6.2 and beyond support RoCEv1 and RoCEv2
via the NDKPI interface. The default setting is RoCEv1.
To enable RDMA:
1. Upgrade the NIC NVRAM using the appropriate board packages. In CCM or in UEFI HII, enable support for
RDMA.
2. Go to the adapter Advanced Properties page and set NetworkDirect Functionality to Enabled for each
BCM5741X miniport, or using PowerShell window, run the following command:
Set-NetAdapterAdvancedProperty -RegistryKeyword *NetworkDirect -RegistryValue 1
3. The following PowerShell commands returns true if NetworkDriect is enabled.
a. Get-NetOffLoadGlobalSetting
b. Get-NetAdapterRDMA
Verifying RDMA
To verify RDMA:
1. Create a file share on the remote system and open that share using Windows Explorer or "net use ….". To
avoid hard disk read/write speed bottleneck, a RAM disk is recommended as the network share under test.
2. From PowerShell run the following commands:
Get-SmbMultichannelConnection | fl *RDMA*
ClientRdmaCapable : True
ServerRdmaCapable : True
If both Client and Server show True, then any file transfers over this SMB connection use SMB.
3. The following commands can be used to enable/disable SMB Multichannel:
Server Side:
• Enable: Set-SmbServerConfiguration -EnableMultiChannel $true
• Disable: Set-SmbServerConfiguration -EnableMultiChannel $false
Client Side:
• Enable: Set-SmbClientConfiguration -EnableMultiChannel $true
• Disable: Set-SmbClientConfiguration -EnableMultiChannel $false