MSI-X Plumbing
BCM5718 Programmer’s Guide
Broadcom
®
January 29, 2016 • 5718-PG108-R
Page 251
MSI-X Data Structures
Full specification of MSI-X is available in PCI Specification rev. 2.3; it is not repeated here. The MSI-X Table
hosted by BCM5718 family is described here:
• This structure is placed at offset 0x0 pointed by BAR4 and BAR5. The content of the MSI-X Table structure
is shown in
• Depending on whether IOV mode is selected, the PCIe core advertises either a 17-entry MSI-X table or a
5-entry MSI-X table. The advertisement choice is made only once following POR and cannot be changed
afterwards (see the Table Size field). Selection of Single-Vector/Multivector mode does not affect Table
Size.
• This table comprises multiple 4 DWORD-long entries and each entry corresponds to one MSI-X vector.
Thus, in the BCM5718 family, there is a maximum of 17 such entries.
• Each entry consists of four fields:
– The Message Address High and Message Address Low fields points to a 64-bit Host Address where the
corresponding vector message must be posted.
– Message Data contains a 32-bit vector data. Every time the BCM5718 family wants to send an interrupt
message corresponding to this vector, it writes the value provided by this field into the address pointed
by Message Address field(s).
– Host software is allowed to replicate the same physical host address into multiple entries; that amounts
to interrupt vector aliasing. The BCM5718 family can handle such aliasing.
– Only one bit, bit[0] of the Vector Control field, is implemented by PCI/PCIe. This is the (per vector) Mask
bit. When this bit is 1, the device function must not send a corresponding interrupt vector message to
the host. Instead, the function must set the corresponding Pending bit in the PBA. When this bit is 1 and
a 0 is written to it, a device must schedule an interrupt vector in case one was already “Pending”.
• The PBA Structure in the BCM5718 family is only 4 DWORDs or 128 bits wide, out of which bits [16:0] are
useful, while bits [127:17] are reserved for future use. Each PBA bit index corresponds to the respective
MSI-X vector#—that is why only 17 bits are implemented in the BCM5718 family. The PBA is placed at the
offset 0x120 relative to the addresses pointed by BAR4 and BAR5. See
• There is room for expansion of up to 18 MSI-X vectors in future NetXtreme controllers.
• In the Message Control Register, only three fields are important to the respective MAC Core:
– MSI-X Enable: This is the feature enable/disable bit. The MAC Core must dynamically snoop CFG
writes to this bit in order to determine if MSI-X gets enabled or not. When MSI-X is enabled, Line
Interrupt Message and MSI Message must be gated off by MAC Core. (Though PCIe allows host
software to enable MSI and MSI-X concurrently, albeit erratically, MSI is preempted in such a scenario.)
– Function Mask: This bit acts like a device function-wide vector mask. When this bit is 1, all vectors in the
function, i.e., 0 through 16 in the BCM5718 family must be masked. If any interrupt vector event occurs
while Function Mask is 1, the corresponding Pending bit in PBA must be set. When this bit is 0, the per-
vector Mask bits found in each Table Entry determine whether a vector is masked or not.
– Table Size: This field may declare a value of either 5 or 17. This is accomplished by Boot Code
programming the appropriate Private Register of the PCIe core.
Note:
PCI 2.3 specification notes that “For all accesses to MSI-X Table and MSI-X PBA fields, software
must use aligned full DWORD or aligned full QWORD transactions; otherwise, the result is undefined”.