RoCE – Configuration and Use Case Examples
NetXtreme-E User’s Manual
February 26, 2018 • NetXtreme-E-UG100
Page 67
To compile bnxt_re:
$make
•
Distros that need external OFED to be installed:
SLES11SP4
Please refer OFED release notes from the following link and install OFED before compiling bnxt_re driver.
http://downloads.openfabrics.org/downloads/OFED/release_notes/OFED_3.18-2_release_notes
To compile bnxt_re:
$export OFED_VERSION=OFED-3.18-2
$make
Installation
To install RoCE in Linux:
1.
Upgrade the NIC NVRAM using the RoCE supported firmware packages from Software Release
20.06.04.01 or newer.
2.
In the OS, uncompress, build, and install the BCM5741X Linux L2 and RoCE drivers.
a. # tar -xzf netxtreme-bnxt_en-1.7.9.tar.gz
b. # cd netxtreme-bnxt_en-bnxt_re
c. # make build && make install
3.
Uncompress, build, and install the NetXtreme-E Linux RoCE User Library.
a. # tar xzf libbnxtre-0.0.18.tar.gz
b. #cd libbnxtre-0.0.18
c. # configure && make && make install.
d. # cp bnxtre.driver /etc/libibverbs.d/
e. # echo "/usr/local/lib" >> /etc/ld.so.conf
f. # ldconfig -v
Please refer to the bnxt_re README.txt for more details on configurable options and recommendations.
Limitations
In dual port NICs, if both ports are on same subnet, rdma perftest commands may fail. The possible cause is
due to an arp flux issue in the Linux OS. To workaround this limitation, use multiple subnets for testing or bring
the second port/interface down.
Known Issues
Bnxt_en
and
Bnxt_re
are designed to function in pair. Older
Bnxt_en
drivers prior to version 1.7.x do not support
RDMA and cannot be loaded at the same time as the
Bnxt_re
(RDMA) driver. The user may experience a
system crash and reboot if
Bnxt_re
is loaded with older
Bnxt_en
drivers. It is recommend that the user load the
Bnxt_en
and
Bnxt_re
module from the same
netxtreme-bnxt_en-<1.7.x>.tar.gz
bundle.
