pshpstuningguidewp040105.doc
Page
15
3.5 Large pages and IP support
One of the most important ways to improve IP performance on the HPS is to ensure that large
pages are enabled. Large pages are required to allocate a number of large pages which will used
by the HPS IP driver at boot time.
Each snX needs one large page for the IP FIFO, plus the number of send pools and receive pools
shared among all adapters. Here is the formula for the number of large pages, assuming that the
send pool and receive pool each need two pages.
(N_adapters*(1 + 2send pools + 2receive pools))
To check whether the driver is using large pages, run the following command:
/usr/sbin/ifsn_dump -a | grep use_lg_pg
If large pages are being used, you should see this result:
use_lg_pg 0x00000001
If large pages are not used for the IP pools, the ifsn_dump -r traces (discussed later) report this
with the following messages:
[02] 1099351956s 0831707096ns 0x000c sn_setup_if_env: large page check failed!
lgpg_size=0x1000000, lgpg_cnt=[. . .]
[02] 1099351956s 0831708002ns 0x000d sn_setup_if_env: large page check failed!
num_pages=0x4, lgpg_numfrb=[. . .]
3.6 Memory affinity for a single LPAR
If you are running with one big LPAR containing all processors on a p690 machine, you need to
ensure that memory affinity is set correctly. To do this using vmo, set
memory_affinity =
1
.
This works with the AIX 5L shell variable MEMORY_AFFINITY and with the MPI variable
MP_TASK_AFFINITY described earlier.
3.7 Amount of memory available
To properly size the number of large pages on a system, or to determine the largest problem size
that an MPI task can run, you need to determine the amount of configured memory in your
LPAR. To do this, run the following command:
lsattr -E -l sys0 -a realmem
To find the actual physical real memory installed on you CEC or LPAR, run the following
command:
lscfg -vp | grep GB
