CHAPTER
EIGHT
IPU-POD
128
NETWORK CONFIGURATION
This section describes how to set up the network configuration on the management server and IPU-M2000s for
your IPU-POD
128
.
8.1 Overview
This section describes how two IPU-POD
64
racks can be merged into a new IPU-POD
128
. The two IPU-POD
64
racks will be referred to as lrack1 and lrack2 in this section of the document for illustration purposes. Lrack1 has
the management server role for the BMC and IPU-Gateway management network.
An RNIC spine switch can be used in an IPU-POD
128
if required. With an RNIC spine switch installed in the
system, all the IPU-M2000s in the IPU-POD
128
can be accessed by both the lrack1 and lrack2 servers in the
RDMA network (data plane). Without the spine switch, only local IPU-POD servers can access their local IPU-
M2000s over the RDMA network - so the lrack1 server can only access the IPU-M2000s in lrack1 and the lrack2
server can only access the IPU-M2000s in lrack2. Since the lrack1 server is used as the management server, it
can access the lrack2 IPU-M2000s using the management network, with or without a spine switch.
• You need to upgrade the IP addressing on both IPU-POD
64
racks to the IP address scheme required for IPU-
PODs larger than IPU-POD
64
. The default factory IPU-POD
64
setup has no rack number indication in the
IP addresses, however this is required for larger IPU-PODs. With this updated IP address scheme multiple
IPU-POD
64
racks can be connected together and can also form part of a larger IPU-POD later on without
having to change their IP addresses.
–
The default factory scheme for IPU-POD
64
racks as shipped is
10.1.x.z.
–
The z values are for the IPU-M2000s in the rack
–
The updated IP address scheme required for the IPU-POD
128
uses
10.x.y.z
where
y = the logical rack
number (lrack)
which is normally identical to the IPU-POD number
–
The x values stay the same: x=1 for BMC, x=2 for IPU-Gateway, x=3 for management server port, x=5
for data RNIC
• Lrack1 hosts a management server running the V-IPU Management Controller, 1GbE management DHCP
server, as well as NTP and syslog servers for the IPU-POD
128
.
• Lrack2 has a limited management server role if no spine switch is being used to fully connect all the
IPU-POD
128
servers with all the IPU-M2000s across the two racks. In this case (not fully connected) the
RNIC interfaces on the IPU-M2000s are still served by their local (existing) DHCP server on each IPU-POD
64
rack, but this time with a new IP address that identifies the lrack number.
Note:
If there is no spine switch you will not be able to reach the RNICs on lrack2 when checking the status with
rack_tool
from lrack1 since there is no inter-rack 100GbE connectivity.
Note:
If any of your IPU-POD
64
racks have DHCP config files named
vlan-14.conf
these should be changed to
vlan-13.conf
. You should also check the port names for the rack switches - any that are named
vlan-14
should
Version: latest (2021-11-25)
88