Oracle Exalogic Elastic Cloud, Owner'S Manual

Page: 93 / 342

6.6.1 InfiniBand Fabric
Exalogic machines use a unified 40 Gb per second InfiniBand quad data rate (QDR)
fabric for internal communication.
Applications running on compute nodes communicate with applications on other
compute nodes using this InfiniBand network. Exalogic machines communicate with
Oracle Exadata Database Machines for database connectivity via IPoIB. Exalogic
machines can be connected to an external network, including a standard database
hosted on a machine outside of the Exalogic machine, via the InfiniBand-to-10 Gb
Ethernet gateways using Ethernet over InfiniBand (EoIB). Each Exalogic machine
configuration includes at least 2 such gateways, which also act as InfiniBand switches
connecting all compute nodes and the storage appliance within the Exalogic machine.
6.6.2 InfiniBand Switches
Sun Network QDR InfiniBand Gateway Switches (part number NM2-GW) are used as
the leaf switches in the Exalogic machine. They connect to the Host Channel Adapters
(HCAs) of Exalogic compute nodes.
These switches (NM2-GW) also act as Ethernet gateways to connect your Exalogic
machine to the external LAN over Ethernet. For more information, see Connectivity
Between Exalogic Machine and External LAN Through Sun Network QDR InfiniBand
Gateway Switch .
Sun Datacenter InfiniBand Switch 36 (part number NM2-36P) is used in multirack
configuration scenarios (an Exalogic machine to another Exalogic machine, and an
Exalogic machine to an Oracle Exadata Database Machine) only. This switch is not
connected or used in an Exalogic machine single rack.
Note:
In the Exalogic machine, InfiniBand switches (both leaf and spine switches)
are automatically configured to separate the IP over InfiniBand (IPoIB) traffic
and the Ethernet over InfiniBand (EoIB) traffic.
6.6.3 Default Bonded Interfaces
After the Sun Network QDR InfiniBand Gateway Switches are connected to Exalogic
compute nodes, the following bonded interfaces are configured:
• IP over InfiniBand (IPoIB) - bond0 link (
ib0
/
ib1
for Oracle Linux, and
ibp0
/
ibp1
for Oracle Solaris)
ib0
or
ibp0
represents the HCA port 0 of compute nodes, and
ib1
or
ibp1
represents the HCA port 1 of compute nodes.
Note:
Depending on your application deployment and isolation requirements, you
can create additional bonded IP subnet interfaces over this default IPoIB link.
For more information, see the "Application Isolation by Subnetting over IPoIB"
topic in the Oracle Exalogic Enterprise Deployment Guide.
Introduction to Oracle Exalogic Network Configuration
Understand Network Requirements and Configuration 6-11