Qlogic OFED+ Host, Software User'S Manual

Page: 108 / 284

5 – Using Other MPIs
Improving Performance of Other MPIs Over IB Verbs
5-12 IB0054606-02 A
Improving Performance of Other MPIs Over IB
Verbs
Performance of MPI applications when using an MPI implementation over IB
Verbs can be improved by tuning the IB MTU size.
The maximum supported MTU size of IB adapter cards is 4K.
Support for 4K IB MTU requires switch support for 4K MTU. The method to set the
IB MTU size varies by MPI implementation:
 Open MPI defaults to the lower of either the IB MTU size or switch MTU size.
 MVAPICH defaults to an IB MTU size of 1024 bytes. This can be over-ridden
by setting an environment variable:
$ export VIADEV_DEFAULT_MTU=MTU2048
Valid values are
MTU256
,
MTU512
,
MTU1024
,
MTU2048
and
MTU4096
. This
environment variable must be set for all processes in the MPI job. To do so,
use
~/.bashrc
or use of
/usr/bin/env
.
 MVAPICH2 defaults to an IB MTU size of 2048 bytes, which should be
sufficient for most applications.
 Platform MPI over IB Verbs automatically determines the IB MTU size.
 Intel MPI over uDAPL (which uses IB Verbs) automatically determines the IB
MTU size.
NOTE
No manual tuning is necessary for PSM-based MPIs, since the PSM layer
determines the largest possible IB MTU for each source/destination path.