IBM Elastic Storage System 3000, Service Manual

Page: 46 / 80

[root@ess3k5b ~]# nmcli c
NAME UUID TYPE DEVICE
bond-bond0 6756fa88-eb54-44cb-a63a-69a56c81d0c0 bond bond0
bond-slave-ib2 3dbd639d-4de2-41a9-ae2d-195b03d2da71 infiniband ib2
bond-slave-ib3 1cfb3056-4017-4ff8-8930-7244e58f38d9 infiniband ib3
bond-slave-ib4 30d93d60-0846-46c6-8baf-53e1093e67eb infiniband ib4
bond-slave-ib5 a1c3bdfc-1b49-49a5-8966-19d6e91c4c56 infiniband ib5
enp29s0f0 ada957e2-feba-d40d-5e1d-5639f49aa2d4 ethernet enp29s0f0
enp1s0 38f8c207-f692-4130-89e3-8ac716e82725 ethernet --
enp29s0f1 4aa0cd6a-f093-4dc9-a865-7e66448190a9 ethernet --
enp29s0f2 0c258db4-35f6-44db-b93d-d2f6875e6cc2 ethernet --
enp29s0f3 1242cc02-68d6-4fc9-b6ad-e62cafd415c5 ethernet --
ethernet-enp1s0 77b8269d-733f-422d-8dbf-500b2b9a9b89 ethernet --
Do not proceed until these nmcli commands are run on both canisters.
After these nmcli commands to add new interfaces to the bond, there must be additional bond-
slave devices that are associated with bond0 . For a system with three network cards, you should
find a total of six bond-slave devices that are associated with bond0 .
Note: In this example where the command output was captured, the system does have a total of
six interfaces with three adapters. However, ib0 and ib1 did not have the active links connected.
Therefore, in the output only four master-slave devices are listed.
e. Do the following extra steps on only one node canister, if MES is for EC64 (InfiniBand):
1) To update the verbs port list, first start GPFS manually.
2) To identify the node class name associated with the target ESS 3000, issue the following
command:
# mmlsnodeclass
3) Ensure that the node servers are active before you do the next step by issuing the following
commands:
# mmstartup -N <nodeclass name>
# mmgetstate -a
4) Ensure that correct entries are listed in verbsPorts for the target node class by issuing the
following command:
/opt/ibm/ess/tools/samples/essServerConfig.sh <node class name>
# mmlsconfig -Y | grep -i verbsPort
Example
[root@ess3k5a ~]# mmlsconfig -Y | grep -i verbsPort
mmlsconfig::0:1:::verbsPorts:mlx5_1/1::
mmlsconfig::0:1:::verbsPorts:mlx5_0/1 mlx5_1/1:bodhi1-ib,gssio2-ib,bodhi_nc1,ems:
mmlsconfig::0:1:::verbsPorts:mlx5_0/1 mlx5_1/1 mlx5_2/1 mlx5_3/1:ess_x86_64:
mmlsconfig::0:1:::verbsPorts:mlx5_2/1 mlx5_3/1 mlx5_4/1 mlx5_5/1:ess_x86_64_mmvdisk_5:
mmlsconfig::0:1:::verbsPorts:mlx5_2/1 mlx5_3/1:ess_x86_64_mmvdisk_6:
mmlsconfig::0:1:::verbsPorts:mlx5_0/1:gss_ppc64:
8. Restore GPFS normal operational mode by doing the following steps on both canisters (customer
task):
a. Restart the server by issuing the following command:
# systemctl reboot
b. When the server is up again, do a basic ping test between the canister over the high-speed
interface.
c. If the ping is successful, start GPFS again by issuing the following command:
# mmstartup -N <node class name>
34 IBM Elastic Storage System 3000: Service Guide