Qlogic Fast Fabric, Руководство пользователя

Страница: 119 / 174

5 – Detailed Descriptions of Command LineTools
D000006-000 Rev A 5-65
Q
5.5.1.5
sacache
This verifies the given hosts have properly communicated with the SA and
cached paths to each other. To run this command, InfiniBand must be installed
and running on the given hosts. The subnet manager and switches must be up.
If this test fails,
cmdall 'cat /proc/driver/ics_dsc/gids'
can be run
against any problem nodes to see what they have cached.
5.5.1.6
ipoibping
This verifies IPoIB basic operation by ensuring that the host can ping all other
nodes via IPoIB. To run this command InfiniBand must be installed, IPoIB must
be configured and running on the host and the given hosts, the SM and switches
must be up. The
-i
option can specify an alternate IPoIB hostname suffix.
5.5.1.7
mpiperf
Verifies that MPI is operational and checks MPI end-to-end latency and
bandwidth between pairs of nodes (e.g., 1-2, 3-4, 5-6). This can be used to verify
switch latency/hops, PCI bandwidth and overall MPI performance. The
test.res
file will have the results of each pair of nodes tested.
To obtain accurate results, this test should be run at a time when no other stressful
applications (e.g., MPI jobs, high stress filesystem operations) are running on
the given hosts.
The following is a sample of expected MPI bandwidths for various server slot
speeds:
❥
PCI-X 66 Mhz (32 bit) - 250 MB/s or less
❥
PCI-X 66Mhz - 400-450 MB/s or less
❥
PCI-X 100 Mhz - 600-700 MB/s
❥
PCI-X 133 Mhz - 800-900 MB/s
❥
PCIe x8 SDR HCA - 900+ MB/s
❥
PCIe x8 DDR HCA - 1300+ MB/s
Bandwidth issues typically indicate server configuration issues (e.g., incorrect
slot used, incorrect BIOS settings, incorrect HCA model), or fabric issues (e.g.,
symbol errors, incorrect link width or speed). Assuming
iba_report
has
previously been used to check for link errors and link speed issues, the server
configuration should be verified.
Note that BIOS settings and differences between server models can account for
10-20% differences in bandwidth. The numbers above are conservative