IBM 24R9718, User Manual

Page: 46 / 59

2
Each Server Switch Module includes 14 1x ports to the backplane and 4 4x ports (in the form of 1 4x
connector and 1 12x connector) on the front panel. (In the case of the BladeCenter T, the internal ports
beyond the first 8 appear in the CLI but do not function.) The Server Switch Module provides fully
non-blocking switching for all 18 ports. All internal 1x ports provide 2.5 Gbps connections to the HCA
expansion cards on server blades. All external 4x connectors provide 10 Gbps connections to the
outside IB network and can autonegotiate connection speed.
Server Switch Modules transmit information to and from BladeCenter Management Modules over
Ethernet (via an internal Ethernet switch) to facilitate setup and management. After you set up a Server
Switch Module and bring it online, the on-board Topspin Subnet Manager brings distributed
intelligence to the IB network.
Topspin Server Switch Modules allow you to perform a field upgrade to load the latest firmware on the
Server Switch Module. You can perform a firmware upgrade even following a failed upgrade. CLI and
GUI interfaces both support upgrades. For details, refer to the
Command Line Interface Reference
Guide or InfiniBand User Guide .
Server Switch Module and HCA Expansion
Cards
Within the BladeCenter unit, Server Switch Modules manage traffic to and from HCA expansion cards
on the BladeCenter hosts. Each HCA expansion card adds two IB ports to a BladeCenter host. Each
HCA port connects via the unit backplane to a particular Server Switch Module bay. The first IB port of
each HCA card (ib0) connects to the Server Switch Module in bay 3, and the second IB port of each
HCA card (ib1) connects to bay 4. For detailed information on the HCA expansion card, refer to the
InfiniBand Host Channel Adapter Expansion Card for IBM eServer BladeCenter User Guide .
Topologies
With the Server Switch Module and HCA expansion cards, you can create a non-redundant, single
switch topology or a redundant, dual-switch topology.
Single-Switch Topology
When you populate just one BladeCenter module bay with a Server Switch Module, you create a
bi-sectional bandwidth topology. However, this topology does
not provide redundant links from the
HCA expansion cards to the Server Switch Module. We strongly recommend that you implement a
“Dual-Switch Topology” to avoid single points of failure.
NOTE: Specific operating systems on the blade servers may prefer that the single Server Switch
Module reside in bay 3, and others may prefer bay 4. The preference depends on which HCA port the
OS identifies as ib0. If your single-switch topology does not run traffic, place the module in the
alternate bay.
Dual-Switch Topology
To enable IB redundancy on the BladeCenter chassis, you must install one Server Switch Module in
each available bay. HCA expansion cards do not support redundant links to a single Server Switch
Module bay. When you add a second Server Switch Module to the BladeCenter chassis, each port of
each HCA expansion card connects to a Server Switch Module.