Issue 11
© Solarflare Communications 2014
287
Solarflare Server Adapter
User Guide
Receive Side Scaling (RSS)
Solarflare adapters support Receive Side Scaling (RSS). RSS enables packet receive-processing to
scale with the number of available CPU cores. RSS requires a platform that supports MSI-X
interrupts.
When RSS is enabled the controller uses multiple receive queues into which to deliver incoming
packets. The receive queue selected for an incoming packet is chosen in such a way as to ensure that
packets within a TCP stream are all sent to the same receive queue – this ensures that packet-
ordering within each stream is maintained. Each receive queue has its own dedicated MSI-X
interrupt which ideally should be tied to a dedicated CPU core. This allows the receive side TCP
processing to be distributed amongst the available CPU cores, providing a considerable performance
advantage over a conventional adapter architecture in which all received packets for a given
interface are processed by just one CPU core.
RSS is enabled by default in the sfxge driver. To limit or disable RSS:
Uncomment the following line in:
/kernel/drv/sfxge.conf
rx_scale_count=<number of MSI-X interrupts requested>
Limitations of Solaris MSI-X interrupt allocation are:
1
All network drivers share 32 MSI-X interrupts.
2
A single NIC can only use 2 MSI-X interrupts (this restriction can be lifted with the
ddi_msix_alloc_limit
setting below).
To lift the restriction of 2 MSI-X interrupts, add the following line to
/etc/system
and reboot.
set ddi_msix_alloc_limit=8
If no MSI/MSI-X interrupts are available then the driver will fall-back to use a single legacy interrupt.
RSS will be unavailable for that port.
Other RSS Settings
You should add the following lines to
/etc/system
:
set pcplusmp:apic_intr_policy=1
This sets the interrupt distribution method to round robin.
set ip:ip_squeue_fanout=1
This determines the mode for associating TCP/IP connections with queues. For details refer to the
following:
http://docs.oracle.com/cd/E19082-01/819-2724/chapter4-7/index.html
NOTE:
RSS also works for UDP packets. For UDP traffic the Solarflare adapter will select the Receive
CPU based on IP source and destination addresses. SFN5xxx adapters support IPv4 and IPv6 RSS,
while SFN4xxx adapters support just IPv4 RSS.