Users Guide

Broadcom NetXtreme-E-UG304-2CS
121
NetXtreme-E User Guide User Guide for Dell Platforms
Congestion control performance is measured by these network traffic performance metrics during periods of congestion:
 Fairness of bandwidth allocation between qp
 Link utilization
 Latency
Under heavy congestion, NCC can enforce fairness at a per-QP level, with low variation between streams. Streams not
crossing the point of network contention are not affected.
Performance may vary depending on the topology, number of flows, traffic types, and the application used. Careful
consideration and understanding should be used when adjusting the different NCC parameters. As a default, deterministic
marking with DCTCP is recommended and can be configured using the setup script provided.
14.2.5.11.1 Priority Flow Control
Although Priority Flow Control (PFC) is not required, it is recommended to enable PFC to ensure that packets are not
dropped in bursty or transient scenarios. In cases where PFC is not being utilized, adjustments of the cc tunables are
required to ensure the ramp up and ramp down is appropriate based on the user’s scenario.
14.2.5.11.2 CNP Traffic Classes
It is recommended that CNPs are classified onto a different traffic class on both the NIC and the switch for optimal
performance to ensure CNP are not utilizing the same buffer as the RoCE packets.
14.2.5.11.3 Disable CPU Power Saving
Intel CPU power saving technology can cause packet RoCE drops with bursty traffic. It is highly recommended to set BIOS
power saving mode to maximum performance, and disable c-states and p-states with kernel options by editing the /etc/
default/grub file, and appending the following to GRUB_CMDLINE_LINUX_DEFAULT:
intel_pstate=disable processor.max_cstate=1 intel_idle.max_cstate=0
Rebuild the grub configuration using the following command:
sudo /usr/sbin/grub-mkconfig* -o /boot/grub/grub.cfg
14.2.5.12 Limitations
In dual-port NICs, if both ports are on same subnet, RDMA perftest commands may fail. The possible cause is due to an arp
flux issue in the Linux OS. To work around this limitation, use multiple subnets for testing or bring the second port/interface
down.
14.2.5.13 Known Issues
Bnxt_en and Bnxt_re are designed to function as a pair. Older Bnxt_en drivers prior to version 1.7.x do not support
RDMA and cannot be loaded at the same time as the Bnxt_re (RDMA) driver. The user may experience a system crash
and reboot if Bnxt_re is loaded with older Bnxt_en drivers. It is recommend that the user load the Bnxt_en and Bnxt_re
module from the same netxtreme-bnxt_en-<1.7.x>.tar.gz bundle.
To prevent mismatching a combination of bnxt_en and bnxt_re from being loaded, the following is required:
 If RedHat/CentOS 7.2 OS was installed to the target system using PXEboot with bnxt_en DUD or a kernel module
RPM, delete the file bnxt_en.ko found in /lib/modules/$(uname -r)/extra/bnxt_en/bnxt_en.ko or edit /etc/depmod.d/.