Figure 4: CTPIO Apertures on the X2522 and X3522
NIC port
Kernel
App 1
App 2
App 3
...
CTPIO
X2522
NIC port
Kernel
App 1
App 2
...
App 15
CTPIO
CTPIO
CTPIO
...
CTPIO
X3522
X26636-050922
One of the CTPIO apertures on the X3522 is used by the Linux kernel, and so 15 are available for
other senders. Onload uses one of these apertures per stack.
In situations where there are more than 16 senders, either some will not be able to perform
accelerated sends, or they will need to share access to an aperture. If processes are sharing
transmit apertures they must lock any access themselves entirely within software, and this does
not give optimal performance. It would make sense for such processes to have similar locality to
the shared receive queues, so apertures are shared between processes running on the same CPU
cluster.
Each transmit aperture has an associated event queue. The common use case is for each transmit
aperture to have a separate event queue that is consumed by the user-level sender. This means
that a sending thread can process transmit events without having to handle receive events or
other traffic. Event queues can also in theory be shared between transmit apertures.
The X3522 includes 32 KB of buffering per transmit aperture to handle bursts of packets and
reordering of any out-of-order PCIe transfers.
The X3522 allows warming the code path, with a flag not to send data.
As it is fully cut-through, the X3522 does not support TCP/IP checksum offload on transmit.
Like its predecessors, the X3522 sends a poisoned frame on failure, but its improved design
means this is significantly less likely to occur.
Chapter 3: What's New
UG1523 (v1.0) October 18, 2022
Alveo X3522 User Guide
16
