7–RoCE Configuration
Configuring DCQCN
164
BC0154501-00 P
All traffic of the specified priority is affected, even if there is a subset of
specific connections that are causing the congestion.
PFC is a single-hop mechanism. That is, if a receiver experiences
congestion and indicates the congestion through a PFC packet, only the
nearest neighbor will react. When the neighbor experiences congestion
(likely because it can no longer transmit), it also generates its own PFC. This
generation is known as
pause propagation
. Pause propagation may cause
inferior route utilization, because all buffers must congest before the
transmitter is made aware of the problem.
DCQCN addresses all of these disadvantages. The ECN delivers congestion
indication to the reaction point. The reaction point sends a CNP packet to the
transmitter, which reacts by reducing its transmission rate and avoiding the
congestion. DCQCN also specifies how the transmitter attempts to increase its
transmission rate and use bandwidth effectively after congestion ceases. DCQCN
is described in the 2015 SIGCOMM paper,
Congestion Control for Large-Scale
RDMA Deployments
, located here:
http://conferences.sigcomm.org/sigcomm/2015/pdf/papers/p523.pdf
DCB-related Parameters
Use DCB to map priorities to traffic classes (priority groups). DCB also controls
which priority groups are subject to PFC (lossless traffic), and the related
bandwidth allocation (ETS).
Global Settings on RDMA Traffic
Global settings on RDMA traffic include configuration of vLAN priority, ECN, and
DSCP.
Setting vLAN Priority on RDMA Traffic
Use an application to set the vLAN priority used by a specified RDMA Queue Pair
(QP) when creating a QP. For example, the
ib_write_bw
benchmark controls
the priority using the
-sl
parameter. When RDMA-CM (RDMA Communication
Manager) is present, you may be unable to set the priority.
Another method to control the vLAN priority is to use the
rdma_glob_vlan_pri
node. This method affects QPs that are created after setting the value. For
example, to set the vLAN priority number to 5 for subsequently created QPs,
issue the following command:
./debugfs.sh -n eth0 -t rdma_glob_vlan_pri 5
