MSI-X
BCM5718 Programmer’s Guide
Broadcom
®
January 29, 2016 • 5718-PG108-R
Page 240
MSI-X
The BCM5718 family introduces PCIe
™
-compliant message signal interrupts (MSI-X), supporting a maximum
of 17 vectors (BCM5718).
This is accomplished without disturbing the legacy interrupt system model. The introduction of interrupt
vectoring provides per-receive-queue indication and per-transmit-queue completion to the device driver. The
maximum number of MSI-X vectors attainable is indirectly determined by certain controller modes of operation.
The existence of multiple receive queues (Rx return rings) in the BCM5718 family stems either from the Receive
Side Scaling (RSS) feature or from the I/O Virtualization (IOV) feature.
When both RSS and IOV are disabled, all of the received packets are posted in the default receive queue (Rx
Return Ring 0). This default Rx return ring is sometimes referred to as “the Receive Return Ring”. The number
of transmit queues is also limited to one in this use case.
In RSS enabled mode, receive traffic classification (also known as sorting) is done purely based on the RSS
hash lookup table. Four RSS receive queues and a single transmit queue exist in this mode for the controller.
Therefore, up to 5 MSI-X vectors are offered in RSS mode.
In IOV mode, receive traffic classification is performed by VRQ filters. Sorted Rx traffic is routed among 17
Virtual Receive queues (VRQs). The transmit side offers up to 16 transmit queues. Therefore, while in IOV
mode, there is an opportunity to map packet indication and completions into 17 MSI-X vectors.
The MSI-X specification allows a device to advertise the availability of a chosen number of vectors in its PCIe
capabilities list; however, it does not specify a mechanism for an operating system (OS) to negotiate the number
of vectors down to the number it would actually want to use. So, one simple way for an OS to allocate fewer
vectors than a device has asked for is to fill out all of the vector table entries advertised by a device. In such an
event, the device would first need to identify the situation, and then reassign or regroup the internal interrupt
sources into the limited number of table entries allocated by the OS. But, the device hardware would not possess
all of the information necessary to accomplish either task. This means that device driver involvement is required.
In the BCM5718 family, it has been assumed that, in most cases, an OS would allocate the requested number
of vectors. A deviation from that would be considered an exception; in that case, because support for MSI-X has
already been advertised to the BIOS, there would be no way to fall back to INTx mode. This situation could be
handled by forcing everything into MSI-X Vector#0 and not using other vectors, disregarding however many of
the originally requested vectors the OS actually allocated.
Thus, the BCM5718 family offers two vector modes within the MSI-X mode:
• Single Vector mode
• Multivector mode
Each of these two modes in turn offers the following submodes:
• Single Vector mode (Restrict to Vector#0)
– Single Vector RSS mode
Note:
RSS and IOV are mutually exclusive—only one of these features may be enabled in the
BCM5718 family at any time.