Issue 11
© Solarflare Communications 2014
91
Solarflare Server Adapter
User Guide
Interrupt and Irqbalance Service
Interrupt affinity describes the set of host cpus that may service a particular interrupt.
This affinity therefore dictates the CPU context where received packets will be processed and where
transmit packets will be freed once sent. If the application can process the received packets in the
same CPU context by being affinitized to the relevant CPU, then latency and CPU utilization can be
improved. This improvement is achieved because well tuned affinities reduce inter-CPU
communication.
Tuning interrupt affinity is most relevant when MSI-X interrupts and RSS are being used. The
irqbalance service, which typically runs by default in most Linux distributions, is a service that
automatically changes interrupt affinities based on CPU workload.
In many cases the irqbalance service hinders rather than enhances network performance. It is
therefore necessary to disable it and then set interrupt affinities.
To disable irqbalance permanently, run:
/sbin/chkconfig -level 12345 irqbalance off
To see whether irqbalanace is currently running, run:
/sbin/service irqbalance status
To disable irqbalance temporarily, run:
/sbin/service irqbalance stop
Once the irqbalance service has been stopped, the Interrupt affinities can be configured manually.
Example 1:
How affinities should be manually set will depend on the application. For a single streamed
application such as Netperf, one recommendation would be to affinitize all the Rx queues and the
application on the same CPU. This can be achieved with the following steps:
1
Determine which interrupt line numbers the network interface uses. Assuming the interface is
eth0, this can be done with:
# cat /proc/interrupts | grep eth0-
NOTE:
The Solarflare driver will evenly distribute interrupts across the available host CPUs (based
on the
rss_cpus
module parameter).
To use the Solarflare driver default affinities (recommended), the irqbalance service must be
disabled before the Solarflare driver is loaded (otherwise it will immediately overwrite the affinity
configuration values set by the Solarflare driver).
123:
13302
0
0
0
PCI-MSI-X eth0-0
131:
0
24
0
0
PCI-MSI-X eth0-1
139:
0
0
32
0
PCI-MSI-X eth0-2
147:
0
0
0
21
PCI-MSI-X eth0-3