Oracle Exalogic Elastic Cloud Скачать руководство пользователя

Страница: 307 / 342

Table E-8 (Cont.) InfiniBand Network Cabling for Exalogic Machine Half Rack

From
Rack Unit
(Sun
Network
QDR
InfiniBand
Gateway
Switch)

Port

To Rack
Unit

Type of
Equipment

Port

Description

U23

U27

Compute node

PCIe 2, P1

2 meter QDR

InfiniBand cable

U23

U22

Storage appliance

server head

PCIe 0, P2

2 meter QDR

InfiniBand cable

U23

U21

Storage appliance

server head

PCIe 0, P2

2 meter QDR

InfiniBand cable

U23

U15

Compute node

PCIe 2, P1

2 meter QDR

InfiniBand cable

U23

U14

Compute node

PCIe 2, P1

2 meter QDR

InfiniBand cable

U23

10A

U13

Compute node

PCIe 2, P1

2 meter QDR

InfiniBand cable

U23

10B

U12

Compute node

PCIe 2, P1

2 meter QDR

InfiniBand cable

U23

11A

U11

Compute node

PCIe 2, P1

3 meter QDR

InfiniBand cable

U23

11B

U10

Compute node

PCIe 2, P1

3 meter QDR

InfiniBand cable

U23

12A

Compute node

PCIe 2, P2

3 meter QDR

InfiniBand cable

U23

12B

Compute node

PCIe 2, P2

3 meter QDR

InfiniBand cable

U23

13A

Compute node

PCIe 2, P2

3 meter QDR

InfiniBand cable

U23

13B

Compute node

PCIe 2, P2

3 meter QDR

InfiniBand cable

U23

14A

Compute node

PCIe 2, P2

3 meter QDR

InfiniBand cable

U23

14B

Compute node

PCIe 2, P2

3 meter QDR

InfiniBand cable

U23

15A

Compute node

PCIe 2, P2

3 meter QDR

InfiniBand cable

InfiniBand Network Cabling Tables

Cabling Tables E-15

Содержание Exalogic Elastic Cloud

Страница 1: ... Cloud Machine Owner s Guide Release 12 2 1 0 0 E18478 21 April 2017 This document describes how to manage and configure an Exalogic machine Release 12 2 1 0 0 is for Oracle Exalogic Elastic Cloud X6 2 X5 2 X4 2 X3 2 and X2 2 ...

Страница 2: ...erms and license restrictions applicable to the programs No other rights are granted to the U S Government This software or hardware is developed for general use in a variety of information management applications It is not developed or intended for use in any inherently dangerous applications including applications that may create a risk of personal injury If you use this software or hardware in ...

Страница 3: ...1 3 1 4 Operational Procedures for Exalogic Machines 1 4 1 4 1 Non emergency Power Procedure 1 5 1 4 2 Emergency Power off Considerations 1 6 1 4 3 Cautions and Warnings 1 6 2 Site Requirements 2 1 Environmental Requirements 2 1 2 2 Space Requirements 2 1 2 2 1 Receiving and Unpacking Requirements 2 2 2 2 2 Maintenance Access Requirements 2 2 2 3 Flooring Requirements 2 3 2 4 Electrical Power Requ...

Страница 4: ...er Rack 4 6 4 4 Exalogic Machine Eighth Rack 4 9 4 5 Default Port Assignments 4 12 5 Add the Exalogic Machine to Your Network 5 1 Default State of the Exalogic Machine Network Configuration 5 1 5 2 Verify the Factory Configuration 5 1 5 3 Prerequisites for Connecting Exalogic Machine to External Ethernet Network 5 5 5 4 Initial Network Configuration of Exalogic Machine 5 5 5 4 1 Configuring the Ci...

Страница 5: ...ement Network Diagram for Exalogic Machine 7 2 7 4 ILOM IP Addresses for Exalogic Machine Components 7 3 7 5 Connect to ILOM via the Network 7 3 7 5 1 Connect to the CLI 7 4 7 5 2 Connect to the Web GUI 7 4 7 5 3 Launch a Remote KVM Session 7 4 7 6 Connect to ILOM via a Serial Connection 7 5 7 6 1 Connect to the ILOM of a Compute Node 7 5 7 6 2 Connect to the ILOM of a Sun Network QDR InfiniBand G...

Страница 6: ...ntroduction to Virtual NICs VNICs 10 1 10 1 1 VNIC Resource Limit 10 1 10 2 Set Up Ethernet Over InfiniBand EoIB on Oracle Linux 10 2 10 3 Set Up Ethernet Over InfiniBand EoIB on Oracle Solaris 10 7 10 3 1 Set Up Ethernet Over InfiniBand on Oracle Solaris 11 1 10 7 10 3 2 Set Up Ethernet Over InfiniBand on Oracle Solaris 11 2 10 12 11 Set Up Virtual LANs 11 1 Introduction to VLAN 11 1 11 2 Example...

Страница 7: ...Subnet Manager in Multirack Configuration Scenarios 14 4 14 5 Check Link Status 14 5 14 6 Verify the InfiniBand Fabric in a Multirack Configuration 14 5 14 6 1 Discover the InfiniBand Network Topology in a Multirack Configuration 14 6 14 6 2 Perform Diagnostics on the InfiniBand Fabric in a Multirack Configuration 14 6 14 6 3 Check for Errors in the InfiniBand Fabric in a Multirack Configuration 1...

Страница 8: ...tches in the Fabric 16 5 16 3 4 Determine the SM Priority on an InfiniBand Switch 16 5 16 3 5 Log In to the InfiniBand Switch That Runs Master SM 16 5 16 4 Move from a Default Partition to a Custom Partition 16 5 16 5 Create an IPoIB Partition and Adding Ports 16 6 16 6 Delete a Partition 16 9 16 7 Create a Partition for EoIB and Associating the pkey with a VNIC and VLAN 16 9 16 8 Perform the Post...

Страница 9: ...se Manager Ops Center 17 32 17 17 2 Viewing Service Requests 17 33 17 17 3 Filing a Service Request 17 35 18 Install the ASR Software 18 1 About Oracle Auto Service Request ASR 18 1 18 2 Recommended Configuration 18 2 18 3 Before You Begin 18 2 18 4 Prerequisites for Installing ASR Manager 18 2 18 5 Install ASR Manager on a Standalone System 18 2 18 5 1 Install Service Tags for Oracle Linux 18 2 1...

Страница 10: ... 2 Compute Nodes D 8 D 9 Parts for Sun Server X3 2 Compute Nodes D 9 D 10 Parts for X4170 M2 Compute Nodes D 10 D 11 Parts for Sun Network QDR InfiniBand Gateway Switch for Exalogic X5 2 D 11 D 12 Parts for Sun Network QDR InfiniBand Gateway Switch for Exalogic X4 2 D 12 D 13 Parts for Sun Network QDR InfiniBand Gateway Switch for Exalogic X2 2 and X3 2 D 12 D 14 Parts for Sun Datacenter InfiniBan...

Страница 11: ...Cabling Full Rack E 27 E 3 6 Three Phase PDU Power Cabling Half Rack E 28 E 3 7 Three Phase PDU Power Cabling Quarter Rack E 29 E 3 8 Three Phase PDU Power Cabling Eighth Rack E 30 F Manage Solaris Zones on Exalogic F 1 Requirements F 1 F 2 Terminology F 1 F 3 Create a Solaris Zone F 2 F 3 1 Prerequisites F 2 F 3 2 Set Up a Solaris Zone F 8 F 4 Migrate a Zone to a New Host F 12 G Customize Linux o...

Страница 12: ...xii ...

Страница 13: ...l Port Connection 7 8 8 1 Network Ports on the Storage Appliance 8 4 8 2 Network Configuration Screen 8 5 8 3 Network Interface Settings 8 6 8 4 igb2 and igb3 in an IPMP Group 8 9 8 5 Creating a New IPMP Group Interface 8 10 8 6 IPMP Network Interface Settings 8 12 8 7 Create Filesystem 8 18 9 1 NIS Service 9 2 9 2 NFS Service 9 3 9 3 NIS Service 9 4 17 1 Oracle Enterprise Manager Home Page 17 4 1...

Страница 14: ...gs Page 18 6 18 3 ASR Activation Asset Screen in My Oracle Support 18 7 C 1 Exalogic Eighth Rack Network Connectivity C 2 C 2 Exalogic Quarter Rack Network Connectivity C 3 C 3 Exalogic Half Rack Network Connectivity C 3 C 4 Exalogic Full Rack Network Connectivity C 4 xiv ...

Страница 15: ...nd Permissions 8 18 8 4 Case Sensitivity Values 8 19 8 5 Normalization Settings 8 20 12 1 NM2 GW Specifications 12 1 13 1 Running the Subnet Manager in Different Rack Configurations 13 2 14 1 Sun Datacenter InfiniBand Switch 36 Specifications 14 1 D 1 Rack Level Replacement Parts for EL X5 2 D 2 D 2 Rack Level Replacement Parts for EL X4 2 D 2 D 3 Rack Level Replacement Parts for EL X2 2 and X3 2 ...

Страница 16: ...Machine Half Rack E 17 E 10 InfiniBand Network Cabling for Exalogic Machine Quarter Rack E 18 E 11 InfiniBand Gateway Switch Inter Connections for Exalogic Machine Quarter Rack E 20 E 12 InfiniBand Network Cabling for Exalogic Machine Eighth Rack E 21 E 13 InfiniBand Gateway Switch Inter Connections for Exalogic Machine Eighth Rack E 22 E 14 Single Phase PDU Power Cabling for Exalogic Machine Full...

Страница 17: ...ems that may result from following the average specifications in this document For detailed information about preparing your site for Oracle Exalogic machine deployment consult your hardware specification This preface contains the following sections Audience Revision History Documentation Accessibility Related Documents Conventions Audience This guide is intended for Oracle Exalogic machine custom...

Страница 18: ...key Allocation in a Hybrid Rack E18478 14 August 2013 Default Port Assignments Added default ports assignments of the Exalogic rack Initial Configuration Updated note about the initial configuration of the storage appliance Set Up Ethernet Over InfiniBand on Oracle Solaris 11 1 Added Oracle Solaris 11 1 procedure for setting up Ethernet over InfiniBand Oracle Solaris Creating VNICs and Associating...

Страница 19: ...pter 16 Monitoring the Exalogic Machine Using Oracle Enterprise Manager Ops Center Added information up front to clarify the scope of the content in this chapter Replacement Units Added replacement parts list for the new hardware components in X3 2 Rebranded the document cater to both X3 2 and X2 2 E18478 09 September 2012 Configure Ethernet Over InfiniBand Updated the procedure to set up EoIB on ...

Страница 20: ...trs if you are hearing impaired Related Documents For more information see the following documents Oracle Exalogic Release Notes Oracle Exalogic Machine Multirack Cabling Guide Conventions The following text conventions are used in this document Convention Meaning boldface Boldface type indicates graphical user interface elements associated with an action or terms defined in text or the glossary i...

Страница 21: ...leverage an internal InfiniBand fabric that connects all of the processing storage memory and external network interfaces within an Exalogic machine to form a single large computing device Each Exalogic machine is connected to the customer s data center networks via 10 GbE traffic and GbE management interfaces Customers can integrate Exalogic machines with an Exadata machine or additional Exalogic...

Страница 22: ... 2 Sun Server X4 2 and Oracle Server X5 2 30 16 8 4 Storage Appliance X6 2 Oracle ZS5 ES storage appliance X2 2 and X3 2 Dual controller Sun ZFS Storage 7320 appliance 60 TB X4 2 and X5 2 Oracle ZS3 ES storage appliance 1 1 1 1 Sun Network QDR InfiniBand Gateway Switches NM2 GW 4 2 2 2 Sun Datacenter InfiniBand Switch 36 NM2 36P 1 1 0 0 Cisco Ethernet management switch X2 2 48 port Cisco Catalyst ...

Страница 23: ... the spares kit available to them Note that any parts used by Oracle Service personnel from the spares kit will be replenished by Oracle 1 2 3 Pre cabling for Exalogic X4 2 and newer Machines The eighth quarter and half rack X4 2 and newer machines are pre cabled with a complete set equivalent in number to a full rack of InfiniBand 92 Ethernet 42 and power cables All the cables are connected to th...

Страница 24: ...one unit filler NM2 GW InfiniBand Switch Gateway U25 X X X X Vented one unit filler Cisco Ethernet Management Switch U24 X Vented one unit filler NM2 GW InfiniBand Switch Gateway U23 X X X X Vented one unit filler NM2 GW InfiniBand Switch Gateway U21 U22 X X X X Storage Appliance Server Head Storage Appliance Server Head U17 to U20 X X X X Storage Appliance Disk Shelf Storage Appliance Disk Shelf ...

Страница 25: ...data center 1 4 1 1 Power On Sequence The power on sequence is as follows 1 Rack including switches Ensure that the switches have had power applied for a few minutes to complete power on configuration before starting the storage controllers and compute nodes 2 Storage server heads attached to the chassis of the storage device Wait for a few minutes for the storage appliance to boot and start NFS s...

Страница 26: ... off Switch Emergency power off EPO switches are required when computer equipment contains batteries capable of supplying more than 750 volt amperes for more than five minutes Systems that have these batteries include internal EPO hardware for connection to a site EPO switch or relay Use of the EPO switch will remove power from the Exalogic machine 1 4 3 Cautions and Warnings The following caution...

Страница 27: ...e com technetwork server storage engineered systems exalogic index html 2 2 Space Requirements All Exalogic machines use the same rack and have the same space requirements The space requirements are as follows Height 1998 mm 78 66 inches Width 600 mm with side panels 23 62 inches Depth front door handle to rear door handle 1200 mm 47 24 inches Depth doors removed 1112 mm 43 78 inches The minimum c...

Страница 28: ...les before entering the data center The entire access route to the installation site should be free of raised pattern flooring that can cause vibration Allow enough space for unpacking it from its shipping cartons Ensure that there is enough clearance and clear pathways for moving the Exalogic machine from the unpacking location to the installation location Table 2 1 lists the access route require...

Страница 29: ...nt including PDUs 1004 71 kg 2215 lbs Note Open tiles are required for electrical access 2 4 Electrical Power Requirements Exalogic Machine can operate effectively over a wide range of voltages and frequencies However it must have a reliable power source Damage may occur if the ranges are exceeded Electrical disturbances such as the following may damage Exalogic Machine Fluctuations caused by brow...

Страница 30: ...a Center Receptacle NEMA L6 30R IEC309 3P4W IP67 60A 250V AC 3 ph IEC309 60A 3ph 4 Wire Hubbell HBL460R C9W or equivalent Hubbell CS8265C IEC309 3P4W IP67 60A 250V AC 3 ph IEC309 60A 3ph 4 Wire Hubbell HBL460R C9W or equivalent Number of Outlets Per Rack 6 2 6 4 Table 2 5 lists the PDU high voltage requirements Table 2 5 PDU Requirements for High Voltage Specification 15 kVA 3 ph 22 kVA 1 ph 24 kV...

Страница 31: ...uit AC current rating PDU power cords are 4 m 13 12 feet long and 1 to 1 5 m 3 3 to 4 9 feet of the cord will be routed within the rack cabinet The installation site AC power receptacle must be within 2 m 6 6 feet of the rack 2 4 2 Circuit Breaker Requirements To prevent catastrophic failures the design of your power system must ensure that adequate power is provided to all of the compute nodes Us...

Страница 32: ...3 percent relative humidity For optimal data center rack cooling 45 to 50 percent non condensing Altitude 3048 meters 10000 feet maximum 12000 meters 40000 feet Ambient temperature is reduced by 1 degree Celsius per 300 m above 900 m altitude above sea level Set conditions to the optimal temperature and humidity ranges to minimize the chance of downtime due to component failure Operating an Exalog...

Страница 33: ...ely affect air flow and cooling within the rack Relative humidity is the percentage of the total water vapor that can exist in the air without condensing and is inversely proportional to air temperature Humidity goes down when the temperature rises and goes up when the temperature drops For example air with a relative humidity of 45 percent at a temperature of 24 degrees Celsius 75 degrees Fahrenh...

Страница 34: ...gher inlet temperature in the servers due to exhaust air recirculation The following is the recommended number of floor tiles Four floor tiles for Exalogic machine full rack Three floor tiles for Exalogic machine half rack One floor tile for Exalogic machine quarter rack and for Exalogic machine eighth rack Figure 2 1 shows a typical installation of the floor tiles for an Exalogic machine full rac...

Страница 35: ...tallation of the Exalogic Machine 3 Unpack the Exalogic machine 4 Place the Exalogic machine in its allocated space 5 Perform preliminary checks before connecting the power cords 6 Perform a visual inspection of the hardware 7 Supply rack power and perform the power on self test device checks 8 Switch on the six PDU circuit breakers located on the rear of the PDU A 9 Wait three to five minutes for...

Страница 36: ...Go to http download oracle com docs cd E19657 01 index html to download these guides Exalogic machine cabinets can enclose a variety of rack mountable Sun servers storage products and other third party equipment 3 2 2 Exalogic Installation Safety Guidelines Before the Exalogic machine arrives the following safety precautions should be reviewed to ensure the site is safe as well as ready for delive...

Страница 37: ...ditioner welding machine or any other equipment that generates loud electronic noises Avoid static electricity at the installation location Static electricity transferred to the Exalogic machine can cause malfunctions Static electricity is often generated on carpets Confirm that the supply voltage and frequency match the electrical ratings indicated on the Exalogic machine Do not insert anything i...

Страница 38: ... used to secure the rack to the shipping pallet You can use these mounting brackets to secure the rack permanently to the installation site floor Do not dispose of these brackets as you will not be able to order replacement brackets Figure 3 1 Unpacking Exalogic Machine Rack 3 2 4 Tools for Installation The following tools are required for installation Screwdriver handle magnetic T 30 Torx wrench ...

Страница 39: ...her installed equipment Note For more information see Environmental Requirements 4 Install all necessary electrical equipment and ensure that sufficient power is provided See Also The Sun Rack II Power Distribution Units User s Guide for the Sun Rack II Power Distribution Unit PDU power requirements 5 Ensure that the installation site provides adequate air conditioning 6 Operate the air conditioni...

Страница 40: ... by pushing on the side panels Pushing on the side panels can tip the rack over This action can cause serious personal injury or death as well as damage to the equipment Never tip or rock the Exalogic machine because the rack can fall over 3 2 6 2 Stabilize the Exalogic Machine After moving the Exalogic machine to the installation site stabilize the rack to ensure that it does not move or tip over...

Страница 41: ... feet to the rear rack surface is 32 5 mm 1 28 inches 7 Distance from the center of front casters to the side of the rack is 86 7 mm 3 41 inches 8 Width between the center of the front casters is 426 6 mm 16 80 inches 9 Distance from the center of the rear casters to the rear of the rack is 173 7 mm 6 83 inches 10 Depth between the front and rear casters is 828 6 mm 32 62 inches 11 Distance betwee...

Страница 42: ...ite floor using the same mounting brackets that secured the rack to the shipping pallet The rack is secured to the pallet with four mounting brackets Use the front and rear brackets to stabilize the rack to the installation floor Prepare the installation site by drilling four holes into the floor Before permanently stabilizing the Exalogic machine with the mounting brackets pre drill the mounting ...

Страница 43: ... different bolt types and strengths 2 Position the Exalogic machine over the pre drilled holes Figure 3 5 shows the bottom view of the Exalogic machine and the location for the mounting hole and floor cut dimensions Figure 3 5 Bottom View of Exalogic Machine Showing Mounting Hole and Floor Cutout Dimensions 1 Distance from mounting bracket to the edge of the rack is 113 mm 4 45 inches Prepare to I...

Страница 44: ...es If you plan to route data or PDU power cords down through the bottom of the rack then you need to cut a hole in the installation floor site Cut a rectangular hole below the rear portion of the rack between the two rear casters and behind the rear RETMA Radio Electronics Television Manufacturers Association rails Figure 3 6 shows the base positions of the rack from the bottom Figure 3 6 Base Pos...

Страница 45: ...logic machine power distribution units PDUs achieve earth ground through their power cords Final chassis ground is achieved by way of the ground prong when you connect the power cord to a socket For additional grounding attach a chassis earth ground cable to the Exalogic machine The additional ground point enables electrical current leakage to dissipate more efficiently WARNING The PDU power input...

Страница 46: ...e 3 8 The attachment point is an adjustable bolt that is inside the rear of the Exalogic machine cabinet on the right side Figure 3 8 Earth Ground Attachment Bolt Location 3 3 Power on the System the First Time Before powering on the system for the first time it is necessary to inspect the machine and connect the power cords This section contains the following topics Inspect the Machine After It I...

Страница 47: ...the front of the Exalogic machine See Also Ventilation and Cooling Requirementsfor more information 3 3 2 Connect the Power Cords The following procedure describes how to connect power cords to the Exalogic machine 1 Open the rear cabinet door 2 Ensure that the correct power connectors have been supplied 3 Unfasten the power cord cable ties The ties are for shipping only and are no longer needed 4...

Страница 48: ...six power distribution unit PDU circuit breakers located on the rear of the main PDU PDU A inside the Exalogic machine The circuit breakers are on the rear of the Exalogic machine cabinet as shown in Figure 3 11 Press the ON side of the toggle switch Note The 24 kVA PDU has 4 plugs and the 15 kVA PDU has 2 plugs In either case you can use the instructions described in this section to power on your...

Страница 49: ...cessor SP powers on and starts up to manage the system The main power is supplied for the remainder of the chassis and fans when the power button on the front of the server chassis is pressed The power button is disabled while the SP is starting The start up state is indicated by a steady one second on one second off blinking pattern of the Power OK LED on the front of the system After the SP has ...

Страница 50: ...begins to start the Power OK LED illuminates and remains a steady green 7 Press the soft switches located on the front of the two storage heads in the storage appliance as shown in Figure 3 13 In addition see Figure 3 14 Figure 3 13 Soft Switches on the Server Heads in the Sun ZFS Storage 7320 appliance Figure 3 14 shows the soft switch and LED lights on each of the server heads in the storage app...

Страница 51: ...king sure that the storage appliance is up and running press the soft switches located on the front of the compute nodes in your Exalogic machine as shown in Figure 3 13 After power on the LED indicators turn green as shown in Figure 3 14 Note If you are using an Exalogic machine full rack you must switch on thirty compute nodes If you are using an Exalogic machine half rack you must switch on six...

Страница 52: ...the Exalogic machine as follows 1 Ensure that you have a laptop with functional USB and network ports 2 Ensure that you have a Category 5E patch cable of maximum length 25 feet and a serial cable of maximum length 15 feet 3 Open the rear cabinet door of the rack 4 Connect the network port of your laptop into an unused input port in the Cisco Ethernet switch This switch is inside a vented filler pa...

Страница 53: ...igured using the Oracle Exalogic Configuration Utility For the list of default NET0 IP addresses see Default IP Addresses and Ports If you are using a Linux operating system on your laptop do the following a Log in as a root user b At the command prompt enter the following command to display the network devices such as ETH0 attached to the Exalogic machine ifconfig a The list of network devices or...

Страница 54: ...run the Oracle Exalogic Configuration Utility set of tools and scripts you can use the network IP address that you provided as input to the Exalogic Configurator spreadsheet For more information about Oracle Exalogic Configuration Utility see Configure the Exalogic Machine Using ECU 3 5 What Next After powering on the Exalogic machine including compute nodes and storage appliance proceed to config...

Страница 55: ... hardware components in an Exalogic machine Full Rack Table 4 1 Default ILOM NET0 and IB Bonded IP for Exalogic Full Rack Unit Component Front View ILOM IP Address NET0 IP Address IB Bonded IP Address 42 Compute Node 192 168 1 132 192 168 1 32 192 168 10 32 41 Compute Node 192 168 1 131 192 168 1 31 192 168 10 31 40 Compute Node 192 168 1 130 192 168 1 30 192 168 10 30 39 Compute Node 192 168 1 12...

Страница 56: ...tch 192 168 1 202 Not applicable Not applicable 22 Server head for the storage appliance 192 168 1 116 192 168 1 16 192 168 10 16 21 Server head for the storage appliance 192 168 1 115 192 168 1 15 192 168 10 15 20 Disk storage tray for the storage appliance Not applicable Not applicable Not applicable 19 18 17 16 Sun Network QDR InfiniBand Gateway Switch 192 168 1 201 Not applicable Not applicabl...

Страница 57: ... 168 1 205 Not applicable Not applicable PDU A left from rear view 192 168 1 210 Not applicable Not applicable PDU B right from rear view 192 168 1 211 Not applicable Not applicable Note The following table shows the hardware that is included in different Exalogic versions Ex alo gic Ra ck Compute Node s Server Cisco Management Switch Storage Appliance X6 2 Oracle Server X6 2 Cisco 4948E Oracle ZS...

Страница 58: ... 4U Solid Filler Not applicable Not applicable Not applicable 37 Not applicable Not applicable Not applicable 36 Not applicable Not applicable Not applicable 35 Not applicable Not applicable Not applicable 34 4U Solid Filler Not applicable Not applicable Not applicable 33 Not applicable Not applicable Not applicable 32 Not applicable Not applicable Not applicable 31 Not applicable Not applicable N...

Страница 59: ...applicable 15 Compute Node 192 168 1 114 192 168 1 14 192 168 10 14 14 Compute Node 192 168 1 113 192 168 1 13 192 168 10 13 13 Compute Node 192 168 1 112 192 168 1 12 192 168 10 12 12 Compute Node 192 168 1 111 192 168 1 11 192 168 10 11 11 Compute Node 192 168 1 110 192 168 1 10 192 168 10 10 10 Compute Node 192 168 1 109 192 168 1 9 192 168 10 9 9 Compute Node 192 168 1 108 192 168 1 8 192 168 ...

Страница 60: ...ode s Server Cisco Management Switch Storage Appliance X6 2 Oracle Server X6 2 Cisco 4948E Oracle ZS5 ES X5 2 Oracle Server X5 2 Cisco 4948E Oracle ZS3 ES X4 2 Sun Server X4 2 Cisco 4948E F S Oracle ZS3 ES X3 2 Sun Server X3 2 Cisco 4948E F S Sun ZFS 7320 X2 2 Sun Fire X4170 M2 Cisco 4948 Sun ZFS 7320 4 3 Exalogic Machine Quarter Rack Table 4 3 lists the default ILOM NET0 and InfiniBand Bonded IP ...

Страница 61: ... applicable Not applicable 35 Not applicable Not applicable Not applicable 34 4U Solid Filler Not applicable Not applicable Not applicable 33 Not applicable Not applicable Not applicable 32 Not applicable Not applicable Not applicable 31 Not applicable Not applicable Not applicable 30 2U Solid Filler Not applicable Not applicable Not applicable 29 Not applicable Not applicable Not applicable 28 2U...

Страница 62: ...ch 192 168 1 201 Not applicable Not applicable 15 2U Solid Filler Not applicable Not applicable Not applicable 14 Not applicable Not applicable Not applicable 13 4U Solid Filler Not applicable Not applicable Not applicable 12 Not applicable Not applicable Not applicable 11 Not applicable Not applicable Not applicable 10 Not applicable Not applicable Not applicable 9 Compute Node 192 168 1 108 192 ...

Страница 63: ...e that is included in different Exalogic versions Ex alo gic Ra ck Compute Node s Server Cisco Management Switch Storage Appliance X6 2 Oracle Server X6 2 Cisco 4948E Oracle ZS5 ES X5 2 Oracle Server X5 2 Cisco 4948E Oracle ZS3 ES X4 2 Sun Server X4 2 Cisco 4948E F S Oracle ZS3 ES X3 2 Sun Server X3 2 Cisco 4948E F S Sun ZFS 7320 X2 2 Sun Fire X4170 M2 Cisco 4948 Sun ZFS 7320 4 4 Exalogic Machine ...

Страница 64: ... Not applicable 35 Not applicable Not applicable Not applicable 34 4U Solid Filler Not applicable Not applicable Not applicable 33 Not applicable Not applicable Not applicable 32 Not applicable Not applicable Not applicable 31 Not applicable Not applicable Not applicable 30 2U Solid Filler Not applicable Not applicable Not applicable 29 Not applicable Not applicable Not applicable 28 2U Solid Fill...

Страница 65: ...6 Sun Network QDR InfiniBand Gateway Switch 192 168 1 201 Not applicable Not applicable 15 2U Solid Filler Not applicable Not applicable Not applicable 14 Not applicable Not applicable Not applicable 13 4U Solid Filler Not applicable Not applicable Not applicable 12 Not applicable Not applicable Not applicable 11 Not applicable Not applicable Not applicable 10 Not applicable Not applicable Not app...

Страница 66: ...ot applicable Note The following table shows the hardware that is included in different Exalogic versions Ex alo gic Ra ck Compute Node s Server Cisco Management Switch Storage Appliance X6 2 Oracle Server X6 2 Cisco 4948E Oracle ZS5 ES X5 2 Oracle Server X5 2 Cisco 4948E Oracle ZS3 ES X4 2 Sun Server X4 2 Cisco 4948E F S Oracle ZS3 ES X3 2 Sun Server X3 2 Cisco 4948E F S Sun ZFS 7320 X2 2 Sun Fir...

Страница 67: ...mpute nodes storage appliance heads and InfiniBand ILOMs HTTPS over TCP 443 Web user configurable Any Compute nodes storage appliance heads and InfiniBand ILOMs IPMI over UDP 623 IPMI Intelligent Platform Management Interface Any Compute nodes and storage appliance heads ILOMs TCP 5120 ILOM remote console CD Any Compute nodes and storage appliance heads ILOMs TCP 5121 ILOM remote console keyboard ...

Страница 68: ...vial File Transfer Protocol Compute nodes storage appliance heads and InfiniBand ILOMs Any NTP over UDP 123 Outgoing NTP Compute nodes storage appliance heads and InfiniBand ILOMs Any IPMI over UDP 162 Outgoing IPMI Intelligent Platform Management Interface Platform Event Trap PET Compute nodes storage appliance heads and InfiniBand ILOMs Any Syslog over UDP 514 Outgoing Syslog Compute nodes stora...

Страница 69: ...interface Storage management Any NTP over UDP 123 Outgoing NTP Any PDU HTTP over TCP 80 Browser interface Any PDU SNMP over UDP 161 SNMP user configurable Any PDU HTTPS over TCP 443 Browser interface PDU Any SNMP over UDP 162 Outgoing SNMPv2 traps PDU Any Syslog over UDP 514 Outgoing Syslog PDU Any DHCP over UDP 546 DHCP Dynamic Host Configuration Protocol client Default Port Assignments Default I...

Страница 70: ...Default Port Assignments 4 16 Machine Owner s Guide ...

Страница 71: ...ND0 interfaces Note Exalogic compute nodes are not pre configured with the BOND1 interface for the Ethernet over InfiniBand EoIB connectivity You can configure BOND1 for compute notes during the initial configuration of the Exalogic machine Storage heads configured with NET0 ILOM and BOND0 interfaces Sun Network QDR InfiniBand Gateway Switches referred to as leaf switches configured with the ILOM ...

Страница 72: ...1 103 cnode4_host_name el01cn04 cnode4_private_host_name el01cn04 priv cnode4_eth0_IPADDR 192 168 1 4 cnode4_bond0_IPADDR 192 168 10 4 cnode4_SPNET0_IPADDR 192 168 1 104 cnode5_host_name el01cn05 cnode5_private_host_name el01cn05 priv cnode5_eth0_IPADDR 192 168 1 5 cnode5_bond0_IPADDR 192 168 10 5 cnode5_SPNET0_IPADDR 192 168 1 105 cnode6_host_name el01cn06 cnode6_private_host_name el01cn06 priv c...

Страница 73: ...168 1 17 cnode15_bond0_IPADDR 192 168 10 17 cnode15_SPNET0_IPADDR 192 168 1 117 cnode16_host_name el01cn16 cnode16_private_host_name el01cn16 priv cnode16_eth0_IPADDR 192 168 1 18 cnode16_bond0_IPADDR 192 168 10 18 cnode16_SPNET0_IPADDR 192 168 1 118 cnode17_host_name el01cn17 cnode17_private_host_name el01cn17 priv cnode17_eth0_IPADDR 192 168 1 19 cnode17_bond0_IPADDR 192 168 10 19 cnode17_SPNET0...

Страница 74: ...27 cnode27_private_host_name el01cn27 priv cnode27_eth0_IPADDR 192 168 1 29 cnode27_bond0_IPADDR 192 168 10 29 cnode27_SPNET0_IPADDR 192 168 1 129 cnode28_host_name el01cn28 cnode28_private_host_name el01cn28 priv cnode28_eth0_IPADDR 192 168 1 30 cnode28_bond0_IPADDR 192 168 10 30 cnode28_SPNET0_IPADDR 192 168 1 130 cnode29_host_name el01cn29 cnode29_private_host_name el01cn29 priv cnode29_eth0_IP...

Страница 75: ...xadata Database Machine rack Note You can use the Oracle Exalogic Configuration Utility set of tools and scripts to configure the compute nodes storage heads and Sun Network QDR InfiniBand Gateway Switches For more information about Exalogic Configuration Utility see Configure the Exalogic Machine Using ECU You must configure the Cisco Ethernet Switch manually For more information see Configuring ...

Страница 76: ...word For the default password contact Oracle Support Switch enable 4 Configure the network for a single VLAN The following is an example of the configuration Switch configure terminal Enter configuration commands one per line End with CNTL Z Switch config interface vlan 1 Switch config if ip address 10 7 7 34 255 255 255 0 Switch config if end Switch Sep 15 14 12 06 309 SYS 5 CONFIG_I Configured f...

Страница 77: ...nfig enable password Your_Password Switch config enable secret Your_Password The enable secret you have chosen is thesame as your enable password This is not recommended Re enter the enable secret Switch config end Switch write memory Sep 15 14 25 05 893 SYS 5 CONFIG_I Configured from console by console Building configuration Compressed configuration from 2502 bytes to 1085 bytes OK 8 Set the pass...

Страница 78: ...ffset is the hours offset from UTC and minutes offset is the minutes offset from UTC To set summer time hours use the following command clock summer time zone recurring week day month hh mm week day month hh mm offset In the preceding command zone is the time zone to be displayed when summer time is in effect week is the week of the month 1 to 5 or last day is the day of the week month is the mont...

Страница 79: ... should show the NTP server synchronized to local time if the Cisco switch is connected to the network and has access to NTP 12 Verify the configuration using the following command Switch show running config The following is an example of the output Building configuration Current configuration 2654 bytes version 12 2 no service pad service timestamps debug datetime msec service timestamps log date...

Страница 80: ...rface GigabitEthernet1 46 media type rj45 interface GigabitEthernet1 47 media type rj45 interface GigabitEthernet1 48 media type rj45 interface Vlan1 ip address 10 7 7 34 255 255 255 0 interface Vlan48 no ip address shutdown ip default gateway 10 7 7 1 ip http server control plane line con 0 stopbits 1 line vty 0 4 password Your_Password login line vty 5 15 password Your_Password login ntp server ...

Страница 81: ...mory 13 Save the current configuration using the following command Switch copy running config startup config 14 Exit from the session using the following command Switch exit 15 Disconnect the cable from the Cisco console Note To check the configuration attach a laptop computer to port 48 and ping the IP address of the internal management network Initial Network Configuration of Exalogic Machine Ad...

Страница 82: ...Initial Network Configuration of Exalogic Machine 5 12 Machine Owner s Guide ...

Страница 83: ...e nodes storage appliance and equipment to connect the compute nodes to your network The network connections allow the servers to be administered remotely enable clients to connect to the compute nodes and enable client access to the storage appliance The following table describes the network components and interfaces for each compute node and the storage appliance Table 6 1 Available network comp...

Страница 84: ...ts The Cisco Ethernet switch supplied with the Exalogic machine is minimally configured during installation The minimal configuration disables IP routing and sets the following Host name IP address Subnet mask Default gateway Domain name Domain Name Server NTP server Time Time zone Additional configuration such as defining multiple virtual local area networks VLANs or enabling routing may be requi...

Страница 85: ...iBand private network This required network connects the compute nodes and the storage appliance through the BOND0 interface to the InfiniBand switches gateways on the Exalogic rack It is the default IP over InfiniBand IPoIB subnet created automatically during the initial configuration of the Exalogic machine Note This network is either based on the default InfiniBand partition or based on a parti...

Страница 86: ...6 compute nodes connects 2 Sun Network QDR InfiniBand Gateway Switches in an active passive bond Each compute node is connected to two separate Sun Network QDR InfiniBand Gateway Switches for HA Note that each compute node requires a bond for each external network physical network or VLAN Figure 6 1 shows the network diagram for the Exalogic machine with Oracle Linux operating system Figure 6 1 Ne...

Страница 87: ... switches which allow the creation of Exalogic clouds comprising hundreds of racks and tens of thousands of processors 6 3 Important Notes for Oracle Solaris Users If you are using the Oracle Solaris operating system on Exalogic compute nodes keep the following points in mind BOND0 and BOND1 two important terms used in this guide refer to the default interfaces for IP over InfiniBand IPoIB and Eth...

Страница 88: ...N VLAN in which the compute node is connected The minimum configuration requires single Bond to single LAN per compute node the number of network connections per Exalogic compute node as outlined in the subsequent table for Full Rack Half Rack and Quarter Rack configurations The following table describes the number of required connections Type of Exalogic Machine Bonded Network Configuration Exalo...

Страница 89: ...he network configuration such as host names and IP addresses used during installation is generated from information you supply to your Oracle technical representative in the completed configuration worksheet See Network Configuration Worksheets Configure the new IP addresses in your existing networks only after you have completed the configuration worksheets and received the installation Template ...

Страница 90: ...or JMS public and a floating IP address for HTTP private to the Load Balancer In addition T3 typically uses a minimum of two IP addresses One of these IP addresses is private to the Load Balancer and the other floating IP address is exposed to application server clients An application server client initially accesses the application server through a well known virtual IP address such as T3 floatin...

Страница 91: ...te node Exalogic machine half rack contains 16 compute nodes One compute node can be a member of only one private network of an application domain quarter rack Minimum of 22 IP addresses At least 2 IP addresses per management network for compute nodes 1 IP address for ILOM 1 IP address for ETH0 2 IP addresses for controllers server heads in the storage appliance 1 IP address per Sun Network QDR In...

Страница 92: ...dresses are assigned to Exalogic compute nodes at the time of manufacturing However you can reconfigure the IP addresses by using the Oracle Exalogic Configuration Utility The Oracle Exalogic Configuration Utility and its associated scripts reconfigure InfiniBand Bonded IP addresses based on the input you provide to the Exalogic Configurator spreadsheet For a list of default InfiniBand Bonded IP a...

Страница 93: ...en Exalogic Machine and External LAN Through Sun Network QDR InfiniBand Gateway Switch Sun Datacenter InfiniBand Switch 36 part number NM2 36P is used in multirack configuration scenarios an Exalogic machine to another Exalogic machine and an Exalogic machine to an Oracle Exadata Database Machine only This switch is not connected or used in an Exalogic machine single rack Note In the Exalogic mach...

Страница 94: ... for IPMP configuration Note For information about the in mpathd daemon and the mpathd configuration file see the in mpathd 1M man page on the Oracle Solaris operating system installed on Exalogic compute nodes For information about ifconfig see the ifconfig 1M man page 6 6 4 2 IPMP Groups An IP multipathing group or IPMP group consists of one or more physical interfaces on the same system that ar...

Страница 95: ...hey support eight 10 GB Ethernet ports These ports can be accessed by Exalogic compute nodes through the InfiniBand network through EoIB You can create multiple VLANs per each of these Ethernet ports Each Exalogic compute node can access one or more Ethernet ports on two Sun Network QDR InfiniBand Gateway Switches NM2 GW for HA purposes An Exalogic machine full rack includes 4 gateway switches The...

Страница 96: ...y default each Exalogic compute node is configured with one bonded EoIB interface ethX for one external LAN It is BOND1 vnic0 vnic1 which connects to one external LAN such as LAN1 If a vNIC is created at one of the Sun Network QDR InfiniBand Gateway Switches the ethX interface is associated with the vNIC automatically Note You can configure additional EoIB network interfaces for connecting to addi...

Страница 97: ...ed by the switch vendor Note Exalogic ships with QSFP transceivers by default Customers may use them on the data center switch side if they use a Sun Oracle 10GbE switch such as the Sun Network 10 GbE Switch 72p 6 6 7 Additional InfiniBand Network Requirements and Specifications Table 6 4 lists additional InfiniBand specifications and cable requirements Table 6 4 HCA Port Specifications and Cable ...

Страница 98: ...ine location 8 Inform your Oracle representative when you have completed these steps 6 8 Subnet Manager Requirements for Connecting Exalogic to Exadata See Subnet Manager Operation in Different Rack Configurations 6 9 Network Configuration Worksheets The following networks are used with the Exalogic machine Management network Client access network InfiniBand private network IPoIB The InfiniBand pr...

Страница 99: ...gic machine resides Time Zone Valid time zone IP address of the Domain Name Server IP address of one or more network name servers up to four servers Example 10 25 45 123 10 25 45 125 NTP Server IP address of one or more Network Time Protocol servers up to four servers Example 10 12 13 14 10 12 13 15 Search Domains A list of search domains for name lookup up to four domains Example example com exam...

Страница 100: ...rksheet includes fields in the Exalogic Configurator spreadsheet that typically require data Table 6 6 Management Network Configuration Worksheet Item Entry Description and Example First Management IP address or First ILOM IP address First IP address in the sequential range used for management network interfaces This network must be distinct from all other networks on the Exalogic machine Example ...

Страница 101: ...ration worksheet Fill out this worksheet and provide it to the Oracle representative or use as a reference to complete the initial configuration of your Exalogic machine Information entered in this worksheet is used as input to Oracle Exalogic Configuration Utility This worksheet includes fields in the Exalogic Configurator spreadsheet that typically require data Table 6 8 Private InfiniBand Netwo...

Страница 102: ...Network Configuration Worksheets 6 20 Machine Owner s Guide ...

Страница 103: ... hardware and software is preinstalled on these components ILOM enables you to actively manage and monitor compute nodes in the Exalogic machine independently of the operating system state providing you with a reliable Lights Out Management LOM system With ILOM you can proactively Learn about hardware errors and faults as they occur Remotely control the power state of your compute node View the gr...

Страница 104: ...tions The web interface provides an easy to use browser interface that enables you to log in to the SP then to perform system management and monitoring The command line interface enables you to operate ILOM using keyboard commands and adheres to industry standard DMTF style CLI and scripting protocols ILOM supports SSH v2 0 and v3 0 for secure access to the CLI Using the CLI you can reuse existing...

Страница 105: ...guration Utility The Oracle Exalogic Configuration Utility and its associated scripts reconfigure ILOM IP addresses based on the input you provide to the Exalogic Configurator spreadsheet If the Exalogic Configuration Utility fails to reconfigure IP addresses or misconfigures IP addresses you can configure IP addresses manually as described in Configure ILOM IP Addresses Manually 7 5 Connect to IL...

Страница 106: ...when prompted The default user name is root The CLI command prompt is displayed 7 5 2 Connect to the Web GUI 1 In the address bar of a web browser type the ILOM IP address of an Exalogic compute node or hardware component and press Enter The login screen is displayed 2 Enter the user name and the password The default user name is root 3 Click Log In The web GUI is displayed 7 5 3 Launch a Remote K...

Страница 107: ...om a terminal or a PC running terminal emulation software to the SER MGT port of an Exalogic compute node The cable should be of length 15 feet or less 2 Verify that your terminal or laptop is operational 3 Configure the terminal device or the terminal emulation software to use the following settings 8N1 eight data bits no parity one stop bit 9600 baud default but can be set to any standard rate u...

Страница 108: ...re nm2name is the host name of the management controller The name might be the word hostname Even if you do not see the text go to Step 5 5 Log in as the root user The prompt is displayed Note If you do not see this output or prompt there is a problem with the serial configuration the USB to Serial connector or the CLI 7 7 Reconfigure the Network Access This step is required only if your network a...

Страница 109: ...ng commands at the ILOM command prompt set pendingipdiscovery static set pendingipaddress ip_address set pendingipnetmask ip_netmask set pendingipgateway ip_gateway set pendingmanagementport SYS MB NET0 set commitpending true 7 7 3 Use the Ipmitool Commands when SP Network Information is Lost If the service processor SP network information gets lost you can use an SSH shell to connect to the ETH0 ...

Страница 110: ...al port connection as follows a Establish a serial connection to the SP of a compute node by inserting a serial cable into the SER MGT port on the rear of the compute node b In the terminal window configure the following settings 8N1 Eight data bit no parity one stop bit 9600 Baud Disable hardware control Disable software control c At the login prompt log in as the root user d From your present wo...

Страница 111: ...ay set pendingmanagementport SYS MB NET0 set commitpending true 7 8 What Next After configuring ILOM for your Exalogic machine verify the various configurations and proceed to complete the initial configuration of the storage appliance which is included in the Exalogic machine What Next Set Up ILOM on the Compute Nodes 7 9 ...

Страница 112: ...What Next 7 10 Machine Owner s Guide ...

Страница 113: ... Machine Gathering information such as IP address IP net mask Host name Domain Name Server DNS domain name DNS server IP address Default router IP address and Password for configuring an Ethernet interface on the storage controllers Running the Oracle Exalogic Configuration Utility to reconfigure IP addresses and other network parameters for the storage appliance 8 2 Getting Started You can access...

Страница 114: ...appear e Apply the values by pressing ESC 1 or the F1 key or by pressing Enter after confirming the password The final shell configuration screen appears confirming that your appliance is ready for further configuration using the browser user interface BUI 4 Configure the remaining system parameters through a browser running on any client on the same network as the initial interface The management...

Страница 115: ...ppliance File systems export a file based hierarchy and can be accessed over NFS over IPoIB in the case of Exalogic machines The project share tuple is a unique identifier for a share within a pool Multiple projects can contain shares with the same name but a single project cannot contain shares with the same name A single project can contain both file systems and LUNs and they share the same name...

Страница 116: ...T0 igb0 NET1 igb1 and NET2 igb2 ports on the storage heads are connected to the Cisco management switch which is included in the Exalogic machine The igb0 and igb1 interfaces are reserved for administrative access such as access via a web browser or via command line This configuration ensures that the storage heads are always reachable independent of the load on the network data interfaces and ind...

Страница 117: ...rking configuration is displayed as shown in Figure 8 2 Note The interface names and IP addresses shown on the screens in this chapter are examples only You must verify the interface names in your environment and use them accordingly Figure 8 2 Network Configuration Screen The Interfaces section shows the configured network interfaces The green icon indicates that an interface is active on the sto...

Страница 118: ...work Interface Settings Note The interface names and IP addresses shown on the screens in this chapter are examples only You must verify the interface names in your environment and use them accordingly 8 4 3 Cluster Network Configuration The cluster is set up in an active passive configuration All resources data interface links and storage pool are owned by the active storage head When the active ...

Страница 119: ...e see the Network topic in the Oracle ZFS Storage Appliance Administration Guide 8 4 4 Network Configuration Options You can choose any of the following network configuration options for the storage appliance based on your specific requirements Configure Option 1 ASR Support and Separate Paths for Management and Disaster Recovery Configure Option 2 ASR Support and Shared Path for Management and Di...

Страница 120: ...d igb1 Separates the disaster recovery path from the management path Note Ensure that the free hanging cable from the igb3 port is connected to your data center network switch Typically for high availability purposes this cable is connected to a data center switch other than the one that Exalogic s Cisco Management Switch is connected to The bonded interface is a new interface such as dr repl inte...

Страница 121: ...ment and use them accordingly In the Properties section if you select the Allow Administration option management is enabled on the interface To create an IPMP Group with two interfaces such as igb2 and igb3 you must click the icon next to Interfaces on the Figure 8 2 The Network Interface screen is displayed as shown in Figure 8 5 Configuration Overview Configure the Storage Appliance 8 9 ...

Страница 122: ...on your stand by storage head head 2 is used and the management option is disabled The igb2 and igb3 ports are bonded with IP Multipathing IPMP and the management option is enabled on both igb2 and igb3 This configuration option offers the following benefits Supports Automated Service Request ASR for the storage appliance included in the Exalogic machine using ports igb0 and igb1 Supports disaster...

Страница 123: ...paddress 215 or https hostname 215 The login screen appears 3 Type root into the Username field and the administrative password that you entered into the appliance shell kit interface and press the Enter key The Welcome screen is displayed 4 Click the Configuration tab and click NETWORK The default networking configuration is displayed 5 On the network configuration screen Figure 8 2 click the pen...

Страница 124: ...low Administration option to enable management traffic on both igb2 and igb3 interfaces 7 Click APPLY 8 4 4 3 Configure Option 3 ASR Support and No Disaster Recovery But with Single Management URL In this custom configuration the igb0 port on your active storage head head 1 is used and the management option is enabled The igb0 port on your stand by storage Configuration Overview 8 12 Oracle Exalog...

Страница 125: ... IP address or host name you assigned to the NET0 port of either storage head as follows https ipaddress 215 or https hostname 215 The login screen appears 3 Type root into the Username field and the administrative password that you entered into the appliance shell kit interface and press the Enter key The Welcome screen is displayed 4 Click the Configuration tab and click NETWORK The default netw...

Страница 126: ... configuration Common project common Shares NODE_SHARES dumps and general These shares are at the compute node level COMMON_SHARES common patches common general and common images Note This table represents the default configuration of the storage appliance before the Exalogic machine rack configuration is modified at the customer s site Oracle Exalogic Configuration Utility does not alter this con...

Страница 127: ...erall settings for the project that are independent of any particular protocol and are not related to access control or snapshots While the CLI groups all properties in a single list this section describes the behavior of the properties in both contexts The project settings page contains three sections Space Usage Users and Groups Inherited Properties and Default Settings File systems and LUNs Tab...

Страница 128: ...hecksum Controls the checksum used for data blocks Cache device usage Controls whether cache devices are used for the share Synchronous write bias Controls the behavior when servicing synchronous writes By default the system optimizes synchronous writes for latency which leverages the log devices to provide fast response times Database record size Controls the block size used by the file system Th...

Страница 129: ...rty allows the volume size to exceed the amount of available space When set the LUN will consume only the space that has been written to the LUN While this allows for thin provisioning of LUNs most file systems do not expect to get out of space from underlying devices and if the share runs out of space it may cause instability or a corruption on clients or both Volume block size The native block s...

Страница 130: ... following permissions can be granted R Read Permission to list the contents of the directory W Write Permission to create files in the directory X Execute Permission to look up entries in the directory If users have execute permissions but not read permissions they can access files explicitly by name but not list the contents of the directory Group Group that is the current group of the directory...

Страница 131: ...col being used For NFS FTP and HTTP lookups are case sensitive This is default and prioritizes conformance of the various protocols over cross protocol consistency Insensitive All lookups are case insensitive even over protocols such as NFS that are traditionally case sensitive This setting should only be used where CIFS is the primary protocol and alternative protocols are considered second class...

Страница 132: ...Oracle through which you can manage your Sun gear Registration is also a prerequisite for using the PhoneHome service The PhoneHome service communicates with Oracle support to provide Fault reporting the system reports active problems to Oracle for automated service response Depending on the nature of the fault a support case may be opened Details of these events can be viewed in Problems Heartbea...

Страница 133: ... appliance will validate the credentials and allow you to choose which of your inventory teams to register with The default team for each account is the same as the account user name prefixed with a 3 Commit your changes Note You can see a log of PhoneHome events in Maintenance Logs PhoneHome If the phone home service is enabled before a valid Oracle Single Sign On account has been entered it will...

Страница 134: ...Use the Phone Home Service to Manage the Storage Appliance 8 22 Machine Owner s Guide ...

Страница 135: ...port file locking There are known limitations on the NLM based locking and NFSv3 it can cause lock ownership issues under certain multihomed system configurations where the source IP addresses are not deterministic between lock and unlock requests 9 1 2 NFSv4 Unlike NFSv3 NFSv4 is a stateful protocol An NFS client obtains a client ID assigned during the initial negotiation phase and uses it for al...

Страница 136: ...isplayed as shown in Figure 9 1 3 Note down the domain name and the server IP address listed on the page You require this information when setting up NIS clients on Exalogic compute nodes If you do not see configuration information on this page it means that you should install an NIS Server Figure 9 1 uses an example NIS domain nisdomain example com Figure 9 1 NIS Service 9 3 Configure the Storage...

Страница 137: ...uration on the home page 2 Expand Services on the left navigation menu 3 Click NFS The NFS service configuration page is displayed as shown in Figure 9 2 Figure 9 2 NFS Service Enter information in the fields as follows Minimum supported version NFSv3 Maximum supported version NFSv4 Configure the Storage Appliance Configure NFS Version 4 on Exalogic 9 3 ...

Страница 138: ...sdomain example com is entered Figure 9 3 NIS Service The NIS service must be up and running For Server select the Use listed servers option and enter the host name or IP address of the NIS server 9 4 Configure an Exalogic Linux Compute Node to Use NFSv4 Configure an Exalogic Linux compute node as follows 1 Log in to the compute node as root 2 Edit the etc idmapd conf configuration file vi etc idm...

Страница 139: ...example domainname nisdomain example com 6 Edit the etc nsswitch conf configuration file vi etc nsswitch conf Change the following entries passwd files passwd files nis shadow files shadow files nis group files group files nis automount files nis nisplus aliases files nis nisplus 7 Restart the rpcidmapd service service rpcidmapd restart 8 Restart the ypbind service by running the following command...

Страница 140: ...ommon patches 2 On the compute node create a new directory as follows mkdir p u01 common patches 3 Edit the etc fstab configuration file vi etc fstab Add the following line and enter the correct values for nfs4 and proto tcp el01sn01 export common patches u01 common patches nfs4 rw bg hard nointr rsize 131072 wsize 131072 proto tcp 4 Mount all shared volumes by running the following command mount ...

Страница 141: ...DR InfiniBand gateway switch you can have a maximum of eight 10 GbE uplinks In most scenarios the number of 10 GbE uplinks is less than 8 One or more VNICs can be assigned to a compute node If more than one VNIC is assigned they can be different connectors on the same Sun Network QDR InfiniBand Gateway Switch or on different Sun Network QDR InfiniBand Gateway Switches The default configuration is ...

Страница 142: ...H 4 Bridge 0 1 up Enabled Bridge 0 Port 1A ETH 1 Bridge 1 2 down Enabled Bridge 0 Port 1A ETH 2 Bridge 1 2 down Enabled Bridge 0 Port 1A ETH 3 Bridge 1 1 up Enabled Bridge 0 Port 1A ETH 4 Bridge 1 1 up Enabled From this output identify the uplinks You can determine that you can use any of the following Ethernet connectors for creating a VNIC 0A ETH 1 0A ETH 2 0A ETH 3 0A ETH 4 1A ETH 3 1A ETH 4 No...

Страница 143: ...node You can also specify the bonded IPoIB address of the compute node Example el01cn01 iblinkinfo pl R grep el01cn01 65 15 4X 10 0 Gbps Active LinkUp 121 2 el01cn01 EL C 192 168 10 29 HCA 1 Could be 5 0 Gbps 64 15 4X 10 0 Gbps Active LinkUp 120 1 el01cn01 EL C 192 168 10 29 HCA 1 Could be 5 0 Gbps From the output of the iblinkinfo command note the switch lid value 65 in first column associated wi...

Страница 144: ...ress unicast The above address is an example only 6 As ilom admin log in to the gateway switch el01gw04 that you identified in Step 4 7 Run the following command to create a VLAN hostname createvlan connector vlan vlan_ID pkey default Example e101gw04 createvlan 1A ETH 3 vlan 0 pkey default 8 Run the following command to create a VNIC hostname createvnic connector guid compute_node_port_GUID mac u...

Страница 145: ...e in the interface file must not be based on the kernel assigned ethX interface name eth4 eth5 and so on Instead Oracle recommends that the interface file name and value of the DEVICE directive in the interface file be derived from the EPORT_ID and IOA_PORT values as follows Note Any other unique naming scheme is also acceptable a Run the following command to find the EPORT_ID mlx4_vnic_info i eth...

Страница 146: ...ctive specify the dummy MAC address created in step 5 14 Create an interface file for the second VNIC say eth5 Be sure to name the interface file and specify the DEVICE directive by using a derived interface name and not the kernel assigned name as described earlier In addition be sure to specify the relevant dummy MAC address for the HWADDR directive 15 After creating the interface files create t...

Страница 147: ...ort 0A ETH 1 Bridge 0 2 up Enabled Bridge 0 Port 0A ETH 2 Bridge 0 2 up Enabled Bridge 0 Port 0A ETH 3 Bridge 0 1 up Enabled Bridge 0 Port 0A ETH 4 Bridge 0 1 up Enabled Bridge 0 Port 1A ETH 1 Bridge 1 2 down Enabled Bridge 0 Port 1A ETH 2 Bridge 1 2 down Enabled Bridge 0 Port 1A ETH 3 Bridge 1 1 up Enabled Bridge 0 Port 1A ETH 4 Bridge 1 1 up Enabled From this example identify the uplinks In this...

Страница 148: ...he switch lids by comparing the first column of the iblinkinfo output to the lid value of the ibswitches command as follows a On the compute node run the ibswitches command on the command line The example output of this command is as follows el01cn02 ibswitches Switch 0x002128548042c0a0 ports 36 SUN IB QDR GW switch el01gw03 enhanced port 0 lid 63 lmc 0 Switch 0x002128547f22c0a0 ports 36 SUN IB QD...

Страница 149: ...stconfig 8 To create a VLAN run the following command e101gw05 createvlan 1A ETH 3 vlan 1706 pkey default 9 Note the ID of the VLAN you created by running the showvlan command as follows showvlan Connector LAG VLN PKEY 1A ETH 3 0 ffff 1A ETH 3 1706 ffff In this example the VLAN ID is 1706 10 Run the following command to create a VNIC on the switch el01gw05 createvnic 1A ETH 3 guid 00 21 28 00 01 A...

Страница 150: ...teps described in Oracle Solaris Creating VNICs and Associating Them with VLANs 13 Create another VNIC for the same compute node using a connector on a different gateway switch by following steps 1 to 12 Note the name of this interface and its corresponding link For example eoib1 interface with the link net8 14 Delete the following files etc hostname bond1 etc hostname eoib0 etc hostname eoib0 15 ...

Страница 151: ...aces for the VNICs you created in step 16 by running the following commands hostname ipadm create ip interface_name Example el01cn02 ipadm create ip eoib0 el01cn02 ipadm create ip eoib1 e Set one of the interfaces as a standby for the bonded interface by running the following command hostname ipadm set ifprop p standby on m ip interface_name Example e101cn02 ipadm set ifprop p standby on m ip eoib...

Страница 152: ... Up Ethernet Over InfiniBand on Oracle Solaris 11 2 To set up Ethernet over InfiniBand connectivity for Exalogic compute nodes running Oracle Solaris 11 2 Base Image of EECS 2 0 6 2 0 perform the following procedure 1 Use an SSH client such as PuTTY to log in to a compute node as root For this example log in to el01cn16 as root 2 Run the following command to verify that the image version is EECS 2...

Страница 153: ...AGUID PORTGUID PORT STATE GWNAME GWPORT PKEYS net4 2128000 21280001EFF369 1 up el01sw ib02 0a eth 1 7FFF 8206 FFFF el01sw ib02 0a eth 2 el01sw ib02 0a eth 3 root el01cn16 dladm show ib net5 LINK HCAGUID PORTGUID PORT STATE GWNAME GWPORT PKEYS net5 2128000 21280001EFF36A 2 up el01sw ib02 0a eth 1 7FFF 8206 FFFF el01sw ib02 0a eth 2 el01sw ib02 0a eth 3 Make a note of the data that the command displ...

Страница 154: ... ib03 port2 net5 el01sw ib02 8 Run the following commands to create the EoIB datalink over net4 and display the results of the procedure root el01cn16 dladm create eoib l net4 g el01sw ib03 c 0A ETH 1 eoib0 root el01cn16 dladm show eoib LINK GWNAME GWPORT GWID FLAGS SPEED MACADDRESS OVER eoib0 el01sw ib03 0a eth 1 506 aH 10000 0 0 0 0 0 0 net4 9 Open another terminal and log in as root to the swit...

Страница 155: ...80001eff36a mac 00 14 4F 09 16 02 vNIC created Run the following command to display the result of the creation of the VNIC root el01sw ib02 showvnics grep cn16 108 WAIT IOA N 0021280001EFF36A el01cn16 EL C 192 168 10 16 0000 00 14 4F 09 16 02 NO 0xffff 0A ETH 1 13 From the compute node session run the following command to create a VNIC with a VLAN tag root el01cn16 dladm create vnic l eoib1 v 3066...

Страница 156: ...UP N 0021280001EFF369 el01cn16 EL C 192 168 10 16 31744 80 C0 A0 09 16 01 NO 0xffff 0A ETH 1 106 UP H 0021280001EFF369 el01cn16 EL C 192 168 10 16 64513 02 08 20 42 A1 F1 3066 0x8206 0A ETH 1 16 On the switch el01sw ib02 the passive VNIC is not expected to appear until the IPMP group failover process runs From the session to the switch el01sw ib02 run the following command to verify the state of t...

Страница 157: ...net link must be configured to support the defined VLANs The benefits include a logical division of workload enforcing security isolation and splitting traffic across several manageable broadcast domains VLANs allow traffic separation from the 10 GbE switch to compute nodes By design Ethernet traffic on one VLAN cannot be seen by any host on a different VLAN To enable communication between two VLA...

Страница 158: ...h an external router if they are required to be seen by each other Note On a single Ethernet connection using the connector on the gateway switch you can create up to 4094 VLANs 11 3 Tag the Ethernet Connectors With a VLAN Identifier To tag an Ethernet connector on the gateway switch with a VLAN identifier you must run the createvlan command on the gateway switch that the VLAN will be associated w...

Страница 159: ... steps 1 through 6 as described in Set Up Ethernet Over InfiniBand EoIB on Oracle Linux 2 Run the following command to create a VNIC and associate it with a VLAN for example with VLAN 10 as shown in Tag the Ethernet Connectors With a VLAN Identifier createvnic 1A ETH 3 GUID 00 21 28 56 d0 a2 c0 a0 mac a2 c0 a0 a8 1 1 vlan 10 pkey default Where 1A ETH 3 is the Ethernet connector 00 21 28 56 d0 a2 c...

Страница 160: ...ation Be sure to create a bonded interface comprising two VNICs on Oracle Linux for high availability purposes 11 5 Oracle Solaris Creating VNICs and Associating Them with VLANs If you wish to associate a VNIC with a VLAN you should provide a VLAN identifier when mapping the MAC address To create a VNIC and associate with a VLAN use the following example procedure 1 On the gateway switch CLI as ro...

Страница 161: ... in Set Up Ethernet Over InfiniBand EoIB on Oracle Solaris The two new host administered interfaces are the ones with IDs 1 and 2 with VLANs 10 and 11 respectively Tip After creating the interfaces you can run the dladm command to verify the MAC address on the compute node For example to verify the new interface and its MAC address run one of the following commands on the Oracle Solaris compute no...

Страница 162: ...Oracle Solaris Creating VNICs and Associating Them with VLANs 11 6 Machine Owner s Guide ...

Страница 163: ...ollowing topics Physical Specifications Access the Command Line Interface CLI of a Gateway Switch Verify the Status of a Gateway Switch Start the Subnet Manager Manually Check Link Status Verify the InfiniBand Fabric Monitor a Gateway Switch Using Web Interface 12 1 1 Physical Specifications This section introduces Sun Network QDR InfiniBand Gateway Switches which are also referred to as leaf swit...

Страница 164: ...ollows a Connect a serial terminal terminal server or workstation with a TIP connection to the USB to serial adapter Configure the terminal or terminal emulator with these settings 115200 baud 8 bits No parity 1 Stop bit and No handshaking b Press the Return or Enter key on the serial device several times to synchronize the connection You might see text similar to the following CentOS release 5 2 ...

Страница 165: ...asured V1P2 AND 1 16 V Measured 1 2V BridgeX 1 21 V Measured 1 8V 1 80 V Measured 1 2V Standby 1 20 V Voltage test returned OK Starting PSU test PSU 0 present PSU 1 present PSU test returned OK Starting Temperature test Back temperature 23 00 Front temperature 32 62 SP temperature 26 12 Switch temperature 45 maxtemperature 45 Bridge 0 temperature 41 maxtemperature 42 Bridge 1 temperature 43 maxtem...

Страница 166: ...Different Rack Configurations For example to set the SM on a gateway switch to priority 5 run the following command setsmpriority 5 The following output is displayed OpenSM 3 2 6_20090717 Reading Cached Option File etc opensm opensm conf Loading Cached Option routing_engine ftree Loading Cached Option sminfo_polling_timeout 1000 Loading Cached Option polling_retry_number 3 Command Line Arguments P...

Страница 167: ...p Enabled Bridge 0 0 Port 0A ETH 3 up Enabled Bridge 0 0 Port 0A ETH 4 up Enabled Connector 1A ETH Present Bridge 1 1 Port 1A ETH 1 up Enabled Bridge 1 1 Port 1A ETH 2 up Enabled Bridge 1 0 Port 1A ETH 3 up Enabled Bridge 1 0 Port 1A ETH 4 up Enabled Connector 0B Present Switch Port 19 up Enabled Connector 1B Present Switch Port 21 up Enabled Connector 15B Not present 12 1 6 Verify the InfiniBand ...

Страница 168: ...id 14 4xQDR vendid 0x2c9 devid 0x673c sysimgguid 0x3ba000100e38b caguid 0x3ba000100e388 Ca 2 H 0003ba000100e388 nsn33 43 HCA 1 2 3ba000100e38a S 0021283a8389a0a0 23 lid 14 lmc 0 Sun DCS 36 QDR switch localhost lid 15 4xQDR Note The actual output for your InfiniBand fabric will differ from that in the example 12 1 6 2 Perform Diagnostics on the InfiniBand Fabric To perform a collection of tests on ...

Страница 169: ...InfiniBand Fabric Use the ibcheckerrors command that uses the topology file to scan the InfiniBand fabric and validate the connectivity as described in the topology file and to report errors as indicated by the port counters On the command line interface CLI enter the following command ibcheckerrors Summary 4 nodes checked 0 bad nodes found 34 ports checked 0 ports have errors beyond threshold Not...

Страница 170: ...itch is installed with the default vNICs vnic0 and vnic1 configured on separate Sun Network QDR InfiniBand Gateway Switches for the Ethernet over InfiniBand EoIB BOND1 interface Optionally you can create VLANs and vNICs using the InfiniBand gateway switches What Next 12 8 Oracle Exalogic Elastic Cloud Machine Owner s Guide ...

Страница 171: ...g system and file system From the prompt on the CLI you can type hardware commands to perform some administrative and management tasks Hardware commands are user friendly and can perform some testing upon the switch chip enabling greater control of a gateway switch and its operation After you log in to the root account the shell prompt appears and you can enter shell commands Enter the hardware co...

Страница 172: ...ut only one SM is active at a time The active SM is Master SM others are Standby SMs If the master SM shuts down or fails a standby SM will automatically become the master SM Note In the Exalogic machine the InfiniBand switches both leaf and spine are automatically configured to separate the IP over InfiniBand IPoIB traffic and the Ethernet over InfiniBand EoIB traffic 13 2 2 Subnet Manager Operat...

Страница 173: ...sary to benefit from the features of the latest firmware Note that the SM should run on at least two switches in the fabric Consider a configuration that consists of three or more spine switches for example two Exalogic machines connected to two Exadata machines but with varying firmware versions If two or more of the available spine switches across the configuration have the highest firmware vers...

Страница 174: ...n the command line interface CLI run the following command getmaster l The output is displayed as in the following example getmaster l Last ring buffer history listed whereismaster daemon is running 20091204 15 00 53 whereismaster started 20091204 15 00 55 No OpenSM Master seen in the system 20091204 15 06 19 OpenSM Master on Switch 0x0002c9000100d050 ports 36 Sun DCS 36 QDR switch o4nm2 36p 2 nor...

Страница 175: ... If the master SM is not running on a spine switch log in as a root user to the leaf switch where the master SM is located 3 Disable SM on the switch as described in Disable Subnet Manager on a Switch This step relocates the master SM to another switch in the network 4 Perform the above steps until the master SM relocates to the spine switch Sun Datacenter InfiniBand Switch 36 5 Enable SM on the l...

Страница 176: ...icast domain When you create IP subnets ensure that each of the interfaces per Exalogic compute node for these additional IP subnets above the default IPoIB subnet is bonded for high availability HA purposes For more information see the Application Isolation by Subnetting over IPoIB topic in the Oracle Exalogic Enterprise Deployment Guide 13 3 2 Verify the Default Partition You can verify the defa...

Страница 177: ...hines together on the same InfiniBand fabric This chapter contains the following topics Physical Specifications Access the CLI of a Sun Datacenter InfiniBand Switch 36 Verify the Switch Status Start the Subnet Manager in Multirack Configuration Scenarios Check Link Status Verify the InfiniBand Fabric in a Multirack Configuration Monitor the Spine Switch Using Web Interface What Next 14 1 Physical ...

Страница 178: ...ial communication with the command line interface CLI as follows a Connect a serial terminal terminal server or workstation with a TIP connection to the USB to serial adapter Configure the terminal or terminal emulator with these settings 115200 baud 8 bits No parity 1 Stop bit and No handshaking b Press the Return or Enter key on the serial device several times to synchronize the connection You m...

Страница 179: ... BridgeX 1 21 V Measured 1 8V 1 80 V Measured 1 2V Standby 1 20 V Voltage test returned OK Starting PSU test PSU 0 present PSU 1 present PSU test returned OK Starting Temperature test Back temperature 23 00 Front temperature 32 62 SP temperature 26 12 Switch temperature 45 maxtemperature 45 Bridge 0 temperature 41 maxtemperature 42 Bridge 1 temperature 43 maxtemperature 44 Temperature test returne...

Страница 180: ...Manager on the spine switch to priority 8 run the following command on the CLI of the spine switch setsmpriority 8 The following output is displayed OpenSM 3 2 6_20090717 Reading Cached Option File etc opensm opensm conf Loading Cached Option routing_engine ftree Loading Cached Option sminfo_polling_timeout 1000 Loading Cached Option polling_retry_number 3 Command Line Arguments Priority 8 Creatin...

Страница 181: ... 22 up Enabled Connector 2A Present Switch Port 24 up Enabled Connector 15A Not present Connector 0A ETH Present Bridge 0 1 Port 0A ETH 1 up Enabled Bridge 0 1 Port 0A ETH 2 up Enabled Bridge 0 0 Port 0A ETH 3 up Enabled Bridge 0 0 Port 0A ETH 4 up Enabled Connector 1A ETH Present Bridge 1 1 Port 1A ETH 1 up Enabled Bridge 1 1 Port 1A ETH 2 up Enabled Bridge 1 0 Port 1A ETH 3 up Enabled Bridge 1 0...

Страница 182: ...id 0x2c9 devid 0xbd36 sysimgguid 0x21283a8389a0a3 switchguid 0x21283a8389a0a0 21283a8389a0a0 Switch 36 S 0021283a8389a0a0 Sun DCS 36 QDR switch localhost enhanced port 0 lid 15 lmc 0 23 H 0003ba000100e388 2 3ba000100e38a nsn33 43 HCA 1 lid 14 4xQDR vendid 0x2c9 devid 0x673c sysimgguid 0x3ba000100e38b caguid 0x3ba000100e388 Ca 2 H 0003ba000100e388 nsn33 43 HCA 1 2 3ba000100e38a S 0021283a8389a0a0 2...

Страница 183: ...mp ibdiagnet log for complete log I Done Run time was 1 seconds Note The actual output for your InfiniBand fabric will differ from that in the example 14 6 3 Check for Errors in the InfiniBand Fabric in a Multirack Configuration Use the ibcheckerrors command that uses the topology file to scan the InfiniBand fabric and validate the connectivity as described in the topology file and to report error...

Страница 184: ...the root user 3 Click the Switch Fabric Monitoring Tools tab 4 Click Launch Sun DCS GW Monitor The Fabric Monitor is displayed 14 8 What Next After setting up the Sun Datacenter InfiniBand Switch 36 in a multirack configuration scenario you can proceed to monitor and control the InfiniBand fabric Monitor the Spine Switch Using Web Interface 14 8 Oracle Exalogic Elastic Cloud Machine Owner s Guide ...

Страница 185: ...r a Node Display Data Counters for a Node Display Low Level Detailed Information for a Node Display Low Level Detailed Information for a Port Map LIDs to GUIDs Perform Comprehensive Diagnostics for the Entire Fabric Perform Comprehensive Diagnostics for a Route Determine Changes to the InfiniBand Topology Determine Which Links Are Experiencing Significant Errors Check All Ports 15 1 1 Identify All...

Страница 186: ...in the following example Ca 0x0003ba000100e388 ports 2 nsn33 43 HCA 1 Ca 0x5080020000911310 ports 1 nsn32 20 HCA 1 Ca 0x50800200008e532c ports 1 ib 71 HCA 1 Ca 0x50800200008e5328 ports 1 ib 70 HCA 1 Ca 0x50800200008296a4 ports 2 ib 90 HCA 1 Note The output in the example is just a portion of the full output and varies for each InfiniBand topology 15 1 3 Display the InfiniBand Fabric Topology To un...

Страница 187: ...nformation by displaying the GUIDs ports and LIDs of the nodes On the command line interface CLI run the following command ibtracert slid dlid where slid is the LID of the source node and dlid is the LID of the destination node in the fabric The output is displayed as in the following example ibtracert 15 14 From switch 0x0021283a8389a0a0 portnum 0 lid 15 15 Sun DCS 36 QDR switch localhost 23 ca p...

Страница 188: ...ort info Lid 15 DR path slid 15 dlid 65535 0 23 LinkState Active PhysLinkState LinkUp LinkWidthSupported 1X or 4X LinkWidthEnabled 1X or 4X LinkWidthActive 4X LinkSpeedSupported 2 5 Gbps or 5 0 Gbps or 10 0 Gbps LinkSpeedEnabled 2 5 Gbps or 5 0 Gbps or 10 0 Gbps LinkSpeedActive 10 0 Gbps Note The actual output for your InfiniBand fabric will differ from that in the example 15 1 6 Display Counters ...

Страница 189: ...de in the fabric and port is the port of the node For example ibdatacounts 15 23 XmtData 6048 RcvData 6048 XmtPkts 84 RcvPkts 84 Note The actual output for your InfiniBand fabric will differ from that in the example 15 1 8 Display Low Level Detailed Information for a Node If intensive troubleshooting is necessary to resolve a problem you can use the smpquery command to display very detailed inform...

Страница 190: ...e a problem you can use the smpquery command to display very detailed information about a port On the command line interface CLI enter the following command smpquery portinfo lid port where lid is the LID of the node in the fabric For example smpquery portinfo 15 23 Mkey 0x0000000000000000 GidPrefix 0x0000000000000000 Lid 0x0000 SMLid 0x0000 CapMask 0x0 DiagCode 0x0000 MkeyLeasePeriod 0 LocalPort ...

Страница 191: ...Band fabric Note Creation of the mapping file is not a requirement for InfiniBand administration The following procedure creates a file that lists the LID in hexadecimal the GUID in hexadecimal and the node description 1 Create an inventory file osmtest f c i inventory txt The inventory txt file can be used for other purposes too besides this procedure 2 Create a mapping file cat inventory txt gre...

Страница 192: ... in the fabric and dlid is the LID of the destination node The ibdiagpath log file contains the log of the testing 15 1 13 Determine Changes to the InfiniBand Topology If your fabric has a number of nodes that are suspect the osmtest command enables you to take a snapshot inventory file of your fabric and at a later time compare that file to the present conditions Note Although this procedure is m...

Страница 193: ... to port0x21283a8620b0f0 Aug 13 19 45 02 375997 B7EF96C0 0x02 osmtest_validate_sa_class_port_info SA Class Port Info base_ver 1 class_ver 2 cap_mask 0x2602 cap_mask2 0x0 resp_time_val 0x10 Aug 13 19 45 02 378991 B7EF96C0 0x01 osmtest_validate_node_data Checkingnode 0x0021283a8620b0a0 LID 0x14 Aug 13 19 45 02 379172 B7EF96C0 0x01 osmtest_validate_node_data Checkingnode 0x0021283a8620b0b0 LID 0x15 A...

Страница 194: ...In addition in the output of the ibdiagnet command search for the link_error_recovery_counter string That line contains the recovery error count in hexadecimal The preceding lines identify the node and port with the errors Recovery errors are major errors and the respective links must be investigated for the cause of the rapid symbol error propagation Additionally the ibdiagnet log file contains t...

Страница 195: ... the command line interface CLI run the following command ibclearerrors The output is displayed as in the following example Summary 5 nodes cleared 0 errors 15 2 2 Clear Data Counters When you are optimizing the InfiniBand fabric for performance you might want to know how the throughput increases or decreases according to changes you are making to the fabric and Subnet Manager The ibclearcounters ...

Страница 196: ...ymbol error generation The ibportstate command can set the speed to 2 5 5 0 or 10 0 GB sec On the command line interface CLI run the following command ibportstate lid port speed value where lid is the LID of the node in the fabric port is the port of the node and value is the speed you want to set Note Adding speed values enables either speed For example speed 7 is 2 5 5 0 and 10 0 GB sec For exam...

Страница 197: ... of unreliable operation Partition A connector and port pair are identified as being isolated from the InfiniBand fabric Both the Blacklist and Partition passphrases survive reboot You unlock these passphrases using the enableswitchport command with the reason option Note State changes made with the ibportstate command are not recognized by the disableswitchport enableswitchport or listlinkup comm...

Страница 198: ... the port 1 36 For example enableswitchport reason Blacklist 14A Enable Switch port 7 Initial PortInfo Port info DR path slid 65535 dlid 65535 0 port 7 LinkState Down PhysLinkState Disabled LinkWidthSupported 1X or 4X LinkWidthEnabled 1X or 4X LinkWidthActive 4X LinkSpeedSupported 2 5 Gbps or 5 0 Gbps or 10 0 Gbps LinkSpeedEnabled 2 5 Gbps or 5 0 Gbps or 10 0 Gbps LinkSpeedActive 2 5 Gbps After Po...

Страница 199: ...ions Overview of Partitioning Understand Partition Keys Before You Begin Move from a Default Partition to a Custom Partition Create an IPoIB Partition and Adding Ports Delete a Partition Create a Partition for EoIB and Associating the pkey with a VNIC and VLAN Perform the Post Configuration Steps Important Notes for Combined Exalogic Exadata Fabric Users Partitioning Limitations 16 1 Overview of P...

Страница 200: ...idelines for Managing pkey Allocation in a Hybrid Rack The term hybrid rack denotes an Exalogic machine on which half the compute nodes are in a physical configuration and the other half constitutes a virtualized data center For more information about hybrid racks see the Exalogic Elastic Cloud Release Notes On a hybrid Exalogic rack Exalogic Control ensures that a unique pkey is assigned to each ...

Страница 201: ...ardless of whether it is on the physical or virtual part will have a unique pkey 16 3 Before You Begin Before you can start creating unique InfiniBand partitions you must complete the following tasks 1 Verify the switch firmware version 2 Gather the port GUIDs of compute nodes and BridgeX ports of gateway switches 3 Identify the InfiniBand switches in your Exalogic machine s InfiniBand fabric and ...

Страница 202: ...mpute node Alternatively you can run the following command to display only GUIDs ibstat grep Port GUID 16 3 2 1 Identify BridgeX Ports on Gateway Switches To identify the BridgeX ports on the gateway switches that are connected to your compute nodes run the following command at the command prompt on each gateway switch that your compute node is connected to showgwports This command displays the Br...

Страница 203: ...t Manager SM is running getmaster This command displays output as shown in the following example Local SM enabled and running 20111122 08 45 02 Master SubnetManager on sm lid 11 sm guid 0x21283bad45c0a0 SUN IB QDR GW switch el01gw04 10 10 10 10 16 3 5 Log In to the InfiniBand Switch That Runs Master SM After identifying the InfiniBand switch where master SM is running log in to the ILOM shell for ...

Страница 204: ...mmand smpartition start 3 Create the myIPoIB partition with the pkey 0x005 with full membership by running the following command smpartition create n myIPoIB pkey 0x8005 m full flag ipoib 4 Run the following command to add the compute node port GUIDs which you noted down in Gather Port GUIDs of Compute Nodes and BridgeX Ports of Gateway Switches to the myIPoIB partition smpartition add n myIPoIB p...

Страница 205: ...he following command to view the changed partition configuration smpartiiton list modified This command displays the new partition with its pkey ports added to the partition and membership type 8 Run the following command to confirm the partition configuration smpartition commit 9 Create interfaces for the ibp0 and the ibp1 network devices and bond them by running these steps a Log in to the Brows...

Страница 206: ...hare mounted by following these steps a SSH to a compute node that is a member of the partition as the root user b Determine the active infiniband device by running the following commands on the compute node ifconfig ib0 ifconfig ib1 The active device will have non zero Rx Tx bytes c Configure the network for the active device by running the following commands echo 0x8005 sys class net ib0 create_...

Страница 207: ...t on one of the gateway switches run the following command el01gw04 listlinkup grep Bridge The following is an example of the output of the lislinkup command Connector 0A ETH Present Bridge 0 Port 0A ETH 1 Bridge 0 2 up Enabled Bridge 0 Port 0A ETH 2 Bridge 0 2 up Enabled Bridge 0 Port 0A ETH 3 Bridge 0 1 up Enabled Bridge 0 Port 0A ETH 4 Bridge 0 1 up Enabled Bridge 0 Port 1A ETH 1 Bridge 1 2 dow...

Страница 208: ...SM lid 6 Capability mask 0x02510868 Port GUID 0x0021280001a0a366 Link layer IB In the output information about two ports is displayed Identify the GUID and Base lid of the port that you want to use for creating the VNIC For the example illustrated in this procedure we will use the port with GUID 0x0021280001a0a366 and Base lid 121 b On the same compute node run the following command to view inform...

Страница 209: ...fine a dummy MAC address in the following format last3_octets_of_switchGUID last3_octets_of_computenode_adminIP_in_hex_format Example GUID of switch 00 21 28 56 d0 a2 c0 a0 Last three octets a2 c0 a0 Administrative IP of the compute node that requires the VNIC 192 168 1 1 Last three octets 168 1 1 in hexadecimal notation a8 01 01 MAC address a2 c0 a0 a8 01 01 Note The dummy MAC address should be u...

Страница 210: ...vlan 1A ETH 3 vlan 10 pkey 0x005 Where 1A ETH 3 is the Ethernet connector on the gateway switch 10 is the VLAN identifier and 0x005 is the partition key that you created earlier 13 To verify run the following command showvlan The following information is displayed Connector LAG VLN PKEY 1A ETH 3 10 0x005 0A ETH 1 11 ffff 14 As root log in to el01gw04 that you identified in Step 4 Use its IP addres...

Страница 211: ...r on a different gateway switch Note the ethX ID of this VNIC too It is recommended that you configure the two EoIB interfaces as a bonded interface such as bond1 19 Create interface files for the VNICs on the compute node To ensure correct failover behavior the name of the VNIC interface file and the value of the DEVICE directive in the interface file must not be based on the kernel assigned ethX...

Страница 212: ... HWADDR a2 c0 a0 a8 01 01 MASTER bond1 SLAVE yes Make sure that the name of the interface file ifcfg eth331_1 in the example is the name derived in step 19 For the DEVICE directive specify the device name eth331_1 in the example derived in step 19 For the HWADDR directive specify the dummy MAC address created in step 4 21 Create an interface file for the second VNIC say eth5 Be sure to name the in...

Страница 213: ...er on the partitioned network 1 Log in as a root user 2 Run the following commands on the command line cd sys class net ib0 echo 0x8033 create_child 3 Run the following command to verify that the child interface was created ifconfig ib0 8033 4 Specify your setup for the child interface in an ibcfg ib0 8033 file in the etc sysconfig networks scripts directory Note that it is 8033 even if it might b...

Страница 214: ...itations when creating non default partitions Once a new partition configuration is successfully committed using the smpartition command on the current master SM the configuration is kept highly available among the defined set of SM instances However all Sun Network QDR InfiniBand Gateway Switches defined to have SM enabled that is defined by the smnodes command on each gateway switch must be oper...

Страница 215: ...ise Manager Ops Center 12c documentation at http docs oracle com cd E27363_01 index htm For more information about Enterprise Manager Cloud Control see the Enterprise Manager Cloud Control 12c documentation at http docs oracle com cd E24628_01 index htm Note Exalogic virtual configurations include Exalogic Control which provides the management and monitoring interface for Exalogic The information ...

Страница 216: ...ware components of the Exalogic machine 17 2 Key Features Oracle Enterprise Manager 11g Ops Center supports the following key features in the Exalogic machine environment Hardware lifecycle management InfiniBand fabric views and Ethernet network view Console access to launch the browser user interface for managing the storage appliance and the InfiniBand switches Serial console access to the servi...

Страница 217: ...11857_01 nav management htm Note After installing Oracle Enterprise Manager Ops Center you can deploy the enterprise controller and proxy controllers on the same machine For more information about site preparation and installation see the Site Preparation and Installation sections in the Oracle Enterprise Manager Ops Center documentation Verifying the IP addresses assigned to each of the managemen...

Страница 218: ... 6 Understanding the Workflow Figure 17 2 illustrates the workflow for the management of Exalogic machine hardware components using Oracle Enterprise Manager Ops Center Understanding the Workflow 17 4 Oracle Exalogic Elastic Cloud Machine Owner s Guide ...

Страница 219: ...Figure 17 2 Management of Exalogic Machine Hardware Using Oracle Enterprise Manager Ops Center Understanding the Workflow Monitoring the Exalogic Machine Using Oracle Enterprise Manager Ops Center 17 5 ...

Страница 220: ...Management topic in the Administration section in the Oracle Enterprise Manager Ops Center 11g documentation This topic describes the following tasks Adding a user Deleting a user Roles and authorizations Assigning a role Configuring a notification profile Deleting a notification profile 17 8 Discovering and Managing Exalogic Machine Hardware This section describes how to discover and manage the h...

Страница 221: ...riteria for the following assets in your Exalogic machine individually Compute Nodes Storage appliance Sun Network QDR InfiniBand Gateway Switches Sun Datacenter InfiniBand Switch 36 Note This switch is used in multirack configuration scenarios only It is not connected in a Exalogic machine single rack configuration Discovering and Managing Exalogic Machine Hardware Monitoring the Exalogic Machine...

Страница 222: ... the Exalogic machine asset categories name and save the criteria You can check Save Criteria for Future Use to save the criteria for future use Note You can launch the discoveries all at once by multiple selection of the saved criteria 5 Click Discover Assets When the assets are discovered they are listed under the Available to be Managed Assets tab on the home page 6 To add assets to the Assets ...

Страница 223: ...extend or modify the groups at any time For more information see Discovering and Managing Exalogic Machine Hardware In addition verify that all Exalogic specific assets are listed under Managed Assets as shown in the example Figure 17 9 17 9 2 Creating the Exalogic Top Level Group To create a top level group for Exalogic machine assets complete the following steps 1 On the right navigation pane na...

Страница 224: ...f an Exalogic rack For example you can enter Exalogic el01 as a group name 3 After entering the group information and selecting the location click Next The Summary screen is displayed as in Figure 17 5 Grouping Exalogic Machine Hardware Assets 17 10 Oracle Exalogic Elastic Cloud Machine Owner s Guide ...

Страница 225: ...ne named Actions click Create Group under Organize The Configure Modify Group screen is displayed 2 In the Configure Group screen in the Group Name field enter a name for the new Exalogic group For example enter ComputeNodes Enter a description in the Description field Select the Inside a user defined Group Subgroup option as Location In the Group Selection Panel select the Exalogic top level grou...

Страница 226: ...layed 4 On the Summary screen click Finish The ComputeNodes sub group is created under the Exalogic group 17 9 4 Creating a Sub Group for the Storage Appliance After creating a top level group for the Exalogic machine you can create a sub group for the storage appliance which is included in the Exalogic machine You can create a sub group for this storage appliance as follows Grouping Exalogic Mach...

Страница 227: ...defined Group Subgroup option as Location In the Group Selection Panel select the Exalogic top level group that you created in Creating the Exalogic Top Level Group The Configure Group screen should be similar to Figure 17 7 Figure 17 7 Configure Sub Group for Storage Appliance 3 After entering the sub group information and selecting the group location click Next The Summary screen is displayed Gr...

Страница 228: ... switches as follows 1 On the right navigation pane named Actions click Create Group under Organize The Configure Modify Group screen is displayed 2 In the Configure Group screen in the Group Name field enter a name for the new Exalogic group For example enter IBswitches Enter a description in the Description field Select the Inside a user defined Group Subgroup option as Location In the Group Sel...

Страница 229: ...oup 17 9 6 Adding Assets to a Group After creating the top level Exalogic group and its sub groups you must add Exalogic specific managed assets to a group To add an Exalogic specific managed asset to a group do the following 1 On the All Assets page click the Managed Assets tab Exalogic specific managed assets are listed under All Managed Assets Grouping Exalogic Machine Hardware Assets Monitorin...

Страница 230: ...ets to Group button The selected asset is added to the selected group sub group 4 Repeat these steps for each of the Exalogic compute nodes the storage appliance and the InfiniBand switches listed under Managed Assets 17 10 Viewing Exalogic Compute Nodes To view Exalogic compute nodes do the following 1 On the left navigation pane under Assets from the drop down list select the All User Defined Gr...

Страница 231: ...formation about that compute node 4 Click the Hardware tab to view detailed information about that compute node This information includes CPU memory network adapter disk power supply and fan tray fan For example see Figure 17 10 Viewing Exalogic Compute Nodes Monitoring the Exalogic Machine Using Oracle Enterprise Manager Ops Center 17 17 ...

Страница 232: ...unch the Service Processor SP console from within this screen click the Console tab If your console connection is not enabled enable it by clicking the Enable the Console Connection icon 7 Click the Connect to the Console icon The SP console for the selected compute node is launched as shown in Figure 17 11 Viewing Exalogic Compute Nodes 17 18 Oracle Exalogic Elastic Cloud Machine Owner s Guide ...

Страница 233: ...the following 1 On the left navigation pane under Assets from the drop down list select the All User Defined Groups option 2 Select the Exalogic top level group The Exalogic group page is displayed as in Figure 17 9 3 On the left navigation pane select a particular switch under the IBswitches sub group to view information about that switch You should see a page similar to the compute node page sho...

Страница 234: ...formation about that switch This information includes ports network adapters connected to the switch and so on The hardware information about the switch is displayed as in Figure 17 13 Viewing InfiniBand Switches 17 20 Oracle Exalogic Elastic Cloud Machine Owner s Guide ...

Страница 235: ...ck the Connectivity tab 7 If you wish to view information about switch ports click the Port Monitoring tab 8 If you wish to launch the Service Processor SP console from within this interface click the Console tab If your console connection is not enabled enable it by clicking the Enable the Console Connection icon 9 Click the Connect to the Console icon The SP console for the selected switch is la...

Страница 236: ...group page is displayed as in Figure 17 9 3 On the left navigation pane select the storage appliance under the Storage sub group to view information about the storage appliance as shown in Figure 17 14 Figure 17 14 Storage Appliance View 4 Click the Hardware tab to view detailed information about the storage appliance This information includes CPU memory network adapters disks and so on Viewing th...

Страница 237: ... the corresponding storage appliance specifications in your Exalogic machine 6 Click the Storage Shares tab to view the shares exported file systems configured on the storage appliance as shown in Figure 17 16 Viewing the Storage Appliance Monitoring the Exalogic Machine Using Oracle Enterprise Manager Ops Center 17 23 ...

Страница 238: ...cting an appropriate action from the Operate actions pane as shown in Figure 17 15 17 13 Viewing the InfiniBand Fabric and Its Nodes You can view the InfiniBand fabric and its nodes in Enterprise Manager Ops Center To view the InfiniBand fabric do the following 1 On the left navigation pane under Managed Assets from the drop down list select the option that starts with the Fabric string This strin...

Страница 239: ...ticular fabric node 3 From this view you can view detailed information about the fabric node such as Summary Hardware Capabilities Connectivity Problems Monitoring Port Monitoring Console Jobs Configuration Charts and Service Requests 4 Similarly on the left navigation pane under the fabric drop down list click a gateway switch Information about the switch as a fabric node is shown as shown in Fig...

Страница 240: ...c 5 To monitor switch ports click the Port Monitoring tab Monitoring information about the switch ports is displayed as shown in Figure 17 19 Viewing the InfiniBand Fabric and Its Nodes 17 26 Oracle Exalogic Elastic Cloud Machine Owner s Guide ...

Страница 241: ...rt Monitoring Console Jobs Configuration Charts and Service Requests 7 Similarly on the left navigation pane under the fabric drop down list click the storage appliance Information about the storage appliance as a fabric node is shown as shown in Figure 17 20 Viewing the InfiniBand Fabric and Its Nodes Monitoring the Exalogic Machine Using Oracle Enterprise Manager Ops Center 17 27 ...

Страница 242: ...automate problem identification and resolution When monitoring is enabled it is connected with a problem management and notification system Monitoring includes a standard set of monitoring rules and attributes many of which are editable In addition you can add custom monitoring attributes and alert conditions A new alert is generated every time an attribute does not meet a monitoring rule When an ...

Страница 243: ...er administrator has permissions to edit and add monitoring rules and profiles The following are the main components of a complete monitoring configuration Monitoring Rules Express alerting conditions You can apply one or more rules to an asset in order to monitor the asset and raise an alert when the monitoring rule condition is met Monitoring Profiles A set of monitoring rules targeted to a spec...

Страница 244: ...ne The Add Alert Monitoring Rule Parameters window is displayed 4 Select a Rule Type from the drop down menu Threshold Boolean Control Enumerated Control or Expression 5 Select an Asset Type from the drop down menu 6 Complete the Monitored Attribute If you selected Expression the Monitored Attribute option is not available 7 Provide a name and description for the rule that will appear in the Profi...

Страница 245: ...he Firmware Compliance Report to determine if the firmware on the asset complies with your firmware profile s specifications If assets do not contain the firmware version identified in the profile you can update the firmware from the report Server provisioning reports Obtain a report of historical server provisioning actions Run this simple ad hoc report to obtain details about the Deployment Plan...

Страница 246: ... for Using Oracle Services in Oracle Enterprise Manager Ops Center Viewing Service Requests Filing a Service Request 17 17 1 Prerequisites for Using Oracle Services in Oracle Enterprise Manager Ops Center The following are the prerequisites for using Oracle Services in Oracle Enterprise Manager Ops Center Register your assets with My Oracle Support Register your user as a My Oracle Support user in...

Страница 247: ...ane click Message Center 2 Click Open Service Requests My Service Requests or Service Requests Opened by Others to display a list of requests 3 To view details of a particular service request highlight a row then click the View Service Request icon The following graphic shows an example service request Using Oracle Services in Oracle Enterprise Manager Ops Center Monitoring the Exalogic Machine Us...

Страница 248: ...Using Oracle Services in Oracle Enterprise Manager Ops Center 17 34 Oracle Exalogic Elastic Cloud Machine Owner s Guide ...

Страница 249: ...ccessfully filing a service request ticket If the asset is not registered in My Oracle Support the service request job will fail To file a service request from a problem do the following 1 Click Message Center in the Navigation pane 2 Click My Problems or Unassigned Problems 3 Highlight the problem then click the phone icon in the center pane To file a service request from an asset do the followin...

Страница 250: ...Using Oracle Services in Oracle Enterprise Manager Ops Center 17 36 Machine Owner s Guide ...

Страница 251: ...nable faster problem resolution by eliminating the need to initiate contact with Oracle Support Services for common hardware component failures reducing both the number of phone calls needed and overall phone time required ASR also simplifies support operations by using electronic diagnostic data Easily installed and deployed ASR is completely controlled by you the customer to ensure security ASR ...

Страница 252: ...of Exalogic to serve as ASR Manager Identify and verify ASR assets Ensure connectivity to the Internet using HTTPS Note For Exalogic compute nodes ASR only covers what the ILOM monitors 18 4 Prerequisites for Installing ASR Manager The following are the prerequisites for installing ASR Manager on a system outside of Exalogic Root access to the system on which ASR Manager will be installed Operatin...

Страница 253: ...n article cmd show type NOT doctype SYSTEMDOC id 1185493 1 2 As root run the following command to install the SASM package On Oracle Linux rpm i SUNWsasm version_num rpm On Oracle Solaris pkgadd d SUNWsasm version_num pkg 18 5 3 Install ASR Package To install the ASR package complete the following steps 1 Download the ASR package from the following URL and unzip https support oracle com CSP main a...

Страница 254: ...e Appliance To activate the storage appliance included in your Exalogic machine complete the following steps 1 Ensure that you have a valid My Oracle Support Account 2 Ensure that you can connect to the Internet from your storage appliance using either a direct connection or a web proxy 3 In a web browser enter the IP address or host name you assigned to the NET0 port of either storage head as fol...

Страница 255: ...Figure 18 1 Phone Home Tab on the List of Services When you click Phone Home the Phone Home page is displayed as shown in Figure 18 2 Activate the Storage Appliance Install the ASR Software 18 5 ...

Страница 256: ...ame for the accessing the web proxy server In the Password field enter your password 10 Click the Apply button 11 When the Service Enable Disable popup is presented select the Enable option Note For information about network configuration for the storage appliance see Network Configuration Options ASR activation for the storage appliance covers ILOM on the storage server heads 18 9 Approve and Ver...

Страница 257: ...18 3 You may need to update information about your Exalogic machine as required Figure 18 3 ASR Activation Asset Screen in My Oracle Support 5 Click Approve to complete activation Note An Exalogic machine asset must be in an active ASR state in My Oracle Support in order for Service Request autocreate to work 18 9 2 View and Verify ASR Assets To view and verify ASR assets Exalogic machine componen...

Страница 258: ... including compute nodes and storage server heads 2 Log in to My Oracle Support You must see the same set of ASR assets with their correct serial numbers Approve and Verify ASR Activation for Exalogic Machine Assets 18 8 Oracle Exalogic Elastic Cloud Machine Owner s Guide ...

Страница 259: ...ault network topology Compute nodes and storage heads with NET0 ILOM and BOND0 IPoIB interfaces Sun Network QDR InfiniBand Gateway Switches with the ILOM interface Note Sun Datacenter InfiniBand Switch 36 which is included in Exalogic full and half rack machines is not connected or configured by default This switch is used in multirack configuration scenarios only For more information see Use the ...

Страница 260: ...them in the configurator spreadsheet which is used as input by Exalogic Configuration Utility Ensure that all the required cabling is completed You must have connected your Exalogic machine to the data center s Ethernet device such as a switch For more information see Connectivity Between Exalogic Machine and External LAN Through Sun Network QDR InfiniBand Gateway Switch Set up VNICs as necessary ...

Страница 261: ...ist Reracking Checklists B 1 System Components Checklist Complete the following checklist to ensure that the systems component considerations have been addressed System Components Checklist Items Yes No N A Comment Is it an Exalogic machine full rack Exalogic machine half rack Exalogic machine quarter rack or Exalogic machine eighth rack How many racks will be installed If more than one rack will ...

Страница 262: ... the data center room requirements are met For information about the data center requirements see Flooring Requirements Data Center Room Checklist Items Yes No N A Comment Has the Exalogic machine location been allocated Is there a vacant location for the new equipment Does the floor layout meet the equipment maintenance access requirements Is there adequate space available for maintenance Will th...

Страница 263: ...ments Data Center Environment Considerations Yes No N A comment Does the computer room air handling meet temperature and humidity requirements Does the installation floor layout satisfy the ventilation requirements Will the equipment be positioned so the exhaust air of one rack does not enter the air intake of another rack Are the perforated floor tiles each rated at 400 CFM or greater Do the data...

Страница 264: ...yes then provide details Are there any stairs or ramps in the moving patch for the new hardware Have you confirmed that all route incline angles are within the permitted range Have you confirmed that the access route is free of any obstacles that would expose the device to shock Are all the surfaces acceptable for rolling the new unpacked and packed equipment If a pallet jack is to be use then hav...

Страница 265: ...r to Flooring Requirements for requirements B 5 Facility Power Checklist Complete the following checklist to ensure that the facility power requirements are met For information about power requirements see Electrical Power Requirements Facility Power Considerations Yes No N A Comment Do you know the required operating voltage and electric current level of the device and peripherals Will you be usi...

Страница 266: ...Power Checklist Considerations Yes No N A Comment Do you have the minimum required power sources Are power outlets available for the new equipment at the designated location Does the power frequency meet the equipment specifications Is there a UPS to power the equipment Is the capacity of the UPS sufficient for the Exalogic machine Does the power source support the power load for the new hardware ...

Страница 267: ...e data center personnel Is there security or access control for the data center Are there any security background checks or security clearances required for vendor personnel to access the data center If yes then do you have a recommended agency How many days in advance must background checks be completed Are there any additional security access issues Is computer room access available for installa...

Страница 268: ...hen do ground level deliveries require a truck with a side rail lift Is there a time constraint on dock access If yes then provide time constraints Is tail lift required on delivery carrier to unload the equipment at the delivery dock Will any of the following be required to place equipment in computer room Stair walkers Lifters Ramps Steel plates Floor covers Does the delivery carrier require any...

Страница 269: ...networking worksheets which is included in Network Configuration Worksheets and provide it to your Oracle technical representative Have you received the site specific installation template from your Oracle technical representative Did you review the installation template and consult with your Oracle technical representative regarding site specific changes if any Did you verify the IP addresses in ...

Страница 270: ...racle Installation Service Is there a cart capable of carrying the weight of the servers to move the components and associated cabling from the supplied rack to the customer supplied rack Do the target rack dimensions meet the following requirements Height 42RU shorter racks can be supported with customer provided PDUs Width 600mm Depth 1112mm without front and rear doors Is the target rack at lea...

Страница 271: ... of 600 mm is recommended to accommodate the PDU and cable harnesses on the side If the rack is less than 600 mm wide then it must have additional depth to accommodate mounting behind the server CMAs Does the target rack support Oracle cable management arms CMAs Does the target rack support installation of Oracle vented and solid filler panels Can the target rack provide tie downs along the left r...

Страница 272: ...r requirements in case one PDU fails Can the customer ensure power loads are evenly distributed across all circuits of a single PDU Can the customer provide appropriate power drops for the PDUs B 10 1 Rack Recycling The following checklist is specific to recycling the rack Recycling Considerations Yes No N A Comment Does the customer want Oracle to take back and recycle the empty rack If yes then ...

Страница 273: ...c Eighth Rack Exalogic Quarter Rack Exalogic Half Rack Exalogic Full Rack NOT_SUPPORTED For information about connecting an Exalogic machine with one or more Exalogic or Exadata machines see the Exalogic Elastic Cloud Multirack Cabling Guide C 1 Exalogic Eighth Rack Figure C 1 illustrates the cabling for a single Exalogic eighth rack Cabling Diagrams C 1 ...

Страница 274: ...o the second gateway switch The remaining half of the compute nodes are connected with their active ports to the second gateway switch and their passive ports to the first gateway switch This connection is for high availability and load distribution purposes Exalogic machine eighth racks do not contain the Sun Datacenter InfiniBand Switch 36 C 2 Exalogic Quarter Rack Figure C 2 illustrates the cab...

Страница 275: ...he second gateway switch The remaining half of the compute nodes are connected with their active ports to the second gateway switch and their passive ports to the first gateway switch This connection is for high availability and load distribution purposes Exalogic machine quarter racks do not contain the Sun Datacenter InfiniBand Switch 36 C 3 Exalogic Half Rack Figure C 3 illustrates the cabling ...

Страница 276: ...logic Full Rack Figure C 4 illustrates the cabling for a single Exalogic full rack Figure C 4 Exalogic Full Rack Network Connectivity In Figure C 4 NM2 GW represents a Sun Network QDR InfiniBand Gateway Switch leaf switches Servers refers to compute nodes in the Exalogic machine The Sun Datacenter InfiniBand Switch 36 referred to as spine switch is not connected Exalogic Full Rack C 4 Oracle Exalo...

Страница 277: ... from the spares kit will be replenished by Oracle This appendix contains the following topics Rack Level FRUs for Exalogic X5 2 Rack Level FRUs for Exalogic X4 2 Rack Level FRUs for Exalogic X2 2 and X3 2 Parts for ZS3 ES Storage Appliance for Exalogic X5 2 Parts for ZS3 ES Storage Appliance for Exalogic X4 2 Parts for Sun ZFS Storage 7320 Appliance Parts for Oracle Server X5 2 Compute Nodes Part...

Страница 278: ...99 Pwrcord Jmpr SR2 1m C14RA 10A C13 FRU 7015400 Pwrcord Jmpr SR2 2m C14RA 10A C13 FRU 530 4436 01 CBL 7 ETHRNT CAT5 CAT5E BLACK FRU 530 4502 01 CBL 10 ETHRNT CAT5 CAT5E BLUE FRU 530 4446 01 5M 10GE QSFP Passive Copp CAB FRU 350 1546 01 ASSY 1U CMA KIT EXADATA X4170 FRU 371 4919 01 ASY SLIDE RAIL LONG 1U 2U STAN FRU 135 1204 01 XCVR 40GBps SR QSFP MTO FRU D 2 Rack Level FRUs for Exalogic X4 2 Tabl...

Страница 279: ...1M QSFP Passive Copper CBL FRU D 3 Rack Level FRUs for Exalogic X2 2 and X3 2 Table D 3 lists the rack level replaceable parts for Exalogic X2 2 and X3 2 machines Note All replacement parts shown in the following table are considered FRU when covered under Premier Support of Systems warranty Table D 3 Rack Level Replacement Parts for EL X2 2 and X3 2 Part Number Description Coverage under Basic Wa...

Страница 280: ...Storage Appliance for Exalogic X5 2 Table D 4 lists the replaceable parts for the ZS3 ES storage appliance which is included in the Exalogic X5 2 machine Note All replacement parts shown in the following table are considered FRU when covered under Premier Support of Systems warranty Table D 4 Replacement Parts for Oracle ZS3 ES Appliance Part Number Description Coverage under Basic Warranty 701535...

Страница 281: ...e Appliance for Exalogic X4 2 Table D 5 lists the replaceable parts for the ZS3 ES storage appliance which is included in the Exalogic X4 2 machine Note All replacement parts shown in the following table are considered FRU when covered under Premier Support of Systems warranty Table D 5 Replacement Parts for Oracle ZS3 ES Appliance Part Number Description Coverage under Basic Warranty 7015351 CPU ...

Страница 282: ...e ZS3 ES Storage Appliance Customer Service Manual D 6 Parts for Sun ZFS Storage 7320 Appliance Table D 6 lists the replaceable parts for the Sun ZFS Storage 7320 appliance which is included in the Exalogic machine Note All replacement parts shown in the following table are considered FRU when covered under Premier Support of Systems warranty Table D 6 Replacement Parts for Sun ZFS Storage 7320 ap...

Страница 283: ... 7 Parts for Oracle Server X5 2 Compute Nodes Table D 7 lists the replaceable parts for the Oracle Server X5 2 compute nodes used in the Exalogic machine Note All replacement parts shown in the following table are considered FRU when covered under Premier Support of Systems warranty Table D 7 Replacement Parts for Oracle Server X5 2 Compute Nodes Part Number Description Coverage with Basic Warrant...

Страница 284: ...ompute nodes used in the Exalogic machine Note All replacement parts shown in the following table are considered FRU when covered under Premier Support of Systems warranty Table D 8 Replacement Parts for Sun Server X4 2 Compute Nodes Part Number Description Coverage with Basic Warranty 7058153 Motherboard Assembly FRU 7069240 FRU CPU Intel E5 2697 12Core 2 7 GHz FRU 7060951 FRU PS AC A256 F 12V 60...

Страница 285: ... parts shown in the following table are considered FRU when covered under Premier Support of Systems warranty Table D 9 Replacement Parts for Sun Server X3 2 Compute Nodes Part Number Description Coverage with Basic Warranty 7026878 1X 2 9 Ghz Intel 8 Core Xeon E5 2690 135 W FRU 7020774 Heatsink CPU X4170 M3 FRU 7018701 16 GB DIMMs FRU F375 3696 Dual port 4X QDR InfiniBand PCI E HCA CX2 FRU F375 3...

Страница 286: ...FF 8087 to SFF 8643 620 mm FRU 7027302 Front Indicator Module Cable FRU 7041996 Disk Data Cable SFF 8087 to SFF 8643 650mm FRU For more information see the Sun Server X3 2 Service Manual D 10 Parts for X4170 M2 Compute Nodes Table D 10 lists the replaceable parts for the X4170 M2 compute nodes used in the Exalogic machine Note All replacement parts shown in the following table are considered FRU w...

Страница 287: ...885 PCI E Riser X16 1U FRU F541 4275 Paddle Card CR_FANs FRU F530 4228 CBL MINI SAS 36POS LONG LY FRU F530 4431 CABLE FAN_DATA 1U FRU F530 3927 CBL PDB MB 1U 2U RIBBON FRU F530 4417 CABLE FAN PADDLE BD P FRU F541 4276 FAN MODULE CR 40X56 FRU F541 4274 FAN_DECK 1U CR_FANS FRU F541 2075 BUSS_BAR PWR 1U 2U FRU F150 3993 CR2032 Battery for M B FRU For more information see the Sun Fire X4170 M2 Server ...

Страница 288: ...ay Switches used in Exalogic X4 2 machines Note All replacement parts shown in the following table are considered FRU when covered under Premier Support of Systems warranty Table D 12 Replacement Parts for Sun Network QDR InfiniBand Gateway Switch for Exalogic X4 2 Machines Part Number Description Coverage with Basic Warranty 7054724 NM2 GW system FRU FRU 300 2299 01 PS AC A247A F 12V 760W FRU 706...

Страница 289: ...atacenter InfiniBand Switch 36 used in the Exalogic X5 2 machine full rack and half rack configurations Note All replacement parts shown in the following table are considered FRU when covered under Premier Support of Systems warranty Table D 14 Replacement Parts for Sun Datacenter InfiniBand Switch 36 Part Number Description Coverage with Basic Warranty 7052970 Assy System NM2 36P Managed FRU 7065...

Страница 290: ... parts shown in the following table are considered FRU when covered under Premier Support of Systems warranty Table D 16 Replacement Parts for Sun Datacenter InfiniBand Switch 36 Part Number Description Coverage with Basic Warranty F541 3495 03 FRU ASSY SYSTEM NM2 36P MANAGE FRU F350 1312 04 FRU FAN MODULE 1RU REVERSE FLO FRU F300 2233 02 FRU PS AC A247 F 12V 760W FRU For more information see the ...

Страница 291: ...10 0307 Cooling fan for Cisco Catalyst 4948 switch HS FRU For repair information see the Cisco Catalyst 4948E Switch Installation Guide D 19 Parts for the Gari DE2 24C Disk Enclosure for Exalogic X5 2 Table D 19 lists the replacement parts for the Gari DE2 24C disk enclosure for Exalogic X5 2 Table D 19 Replacement Parts for Gari DE2 24C Disk Enclosure Part Number Description Coverage with Basic W...

Страница 292: ...24P C I O Module IOM FRU 7043630 ASSY 2U Chassis with Midplane FRU 530 3883 ASSY 2M 4X MINI SAS CBL SHLD FRU 530 3886 ASSY 0 5M 4X MINI SAS CBL SHLD FRU 7044619 ASSY DE2 24P UNIVERSAL RAIL KIT FRU 7044396 ASSY DE2 24P SAS 73GB SSD LogZ FRU 7066831 Assy DRIVE 4T 3 5 SAS 7200 rpm DE2 24C FRU Parts for the Gari DE2 24C Disk Enclosure D 16 Oracle Exalogic Elastic Cloud Machine Owner s Guide ...

Страница 293: ...ls and lacer bars are removed the X4 2 and newer servers are installed with rack rails and cable management arms and the cables power InfiniBand and Ethernet are connected to the newly installed compute nodes The following abbreviations are used in the tables Abbreviation Description Rn Rack n where n is the number of the rack such as R1 Un Unit height in rack where n is the number such as U20 Pn ...

Страница 294: ...ute node NET0 11 7 Blue Cat 5 U34 Compute node NET0 12 7 Blue Cat 5 U33 Compute node NET0 13 7 Blue Cat 5 U32 Compute node NET0 14 7 Blue Cat 5 U31 Compute node NET0 15 7 Blue Cat 5 U30 Compute node NET0 16 7 Blue Cat 5 U29 Compute node NET0 17 7 Blue Cat 5 U28 Compute node NET0 18 7 Blue Cat 5 U27 Compute node NET0 19 7 Blue Cat 5 U26 Sun Network QDR InfiniBand Gateway Switch NET0 38 7 Black Cat ...

Страница 295: ...Compute node NET0 26 7 Blue Cat 5 U12 Compute node NET0 27 10 Blue Cat 5 U11 Compute node NET0 28 10 Blue Cat 5 U10 Compute node NET0 29 10 Blue Cat 5 U9 Compute node NET0 30 10 Blue Cat 5 U8 Compute node NET0 31 10 Blue Cat 5 U7 Compute node NET0 32 10 Blue Cat 5 U6 Compute node NET0 33 10 Blue Cat 5 U5 Compute node NET0 34 10 Blue Cat 5 U4 Compute node NET0 35 10 Blue Cat 5 U3 Compute node NET0 ...

Страница 296: ... 7 Blue Cat 5 U21 Storage appliance server head NET0 22 7 Blue Cat 5 U21 Storage appliance server head NET1 23 7 Blue Cat 5 U21 Storage appliance server head NET2 44 7 Blue Cat 5 U16 Sun Network QDR InfiniBand Gateway Switch NET0 41 7 Black Cat 5 U15 Compute node NET0 24 7 Blue Cat 5 U14 Compute node NET0 25 7 Blue Cat 5 U13 Compute node NET0 26 7 Blue Cat 5 U12 Compute node NET0 27 10 Blue Cat 5 ...

Страница 297: ...ion Not applicable External LAN NET 1 10 Blue Cat 5 Not applicable PDU A NET 2 1M Grey Cat 5 Not applicable PDU B NET 3 1M Grey Cat 5 U23 Sun Network QDR InfiniBand Gateway Switch NET0 40 7 Black Cat 5 U22 Storage appliance server head NET0 20 7 Blue Cat 5 U22 Storage appliance server head NET1 21 7 Blue Cat 5 U22 Storage appliance server head NET2 43 7 Blue Cat 5 U21 Storage appliance server head...

Страница 298: ...AN NET 1 10 Blue Cat 5 Not applicable PDU A NET 2 1M Grey Cat 5 Not applicable PDU B NET 3 1M Grey Cat 5 U23 Sun Network QDR InfiniBand Gateway Switch NET0 40 7 Black Cat 5 U22 Storage appliance server head NET0 20 7 Blue Cat 5 U22 Storage appliance server head NET1 21 7 Blue Cat 5 U22 Storage appliance server head NET2 43 7 Blue Cat 5 U21 Storage appliance server head NET0 22 7 Blue Cat 5 U21 Sto...

Страница 299: ...lists the location ports and cables for the InfiniBand connections for an Exalogic machine full rack Table E 5 InfiniBand Network Cabling for Exalogic Machine Full Rack From Rack Unit Sun Network QDR InfiniBand Gateway Switch Port To Rack Unit Type of Equipment Port Description U26 9A U42 Compute node PCIe 2 P1 3 meter QDR InfiniBand cable U26 9B U41 Compute node PCIe 2 P1 3 meter QDR InfiniBand c...

Страница 300: ...QDR InfiniBand cable U26 15A U30 Compute node PCIe 2 P2 2 meter QDR InfiniBand cable U26 15B U29 Compute node PCIe 2 P2 2 meter QDR InfiniBand cable U24 9A U42 Compute node PCIe 2 P2 3 meter QDR InfiniBand cable U24 9B U41 Compute node PCIe 2 P2 3 meter QDR InfiniBand cable U24 10A U40 Compute node PCIe 2 P2 3 meter QDR InfiniBand cable U24 10B U39 Compute node PCIe 2 P2 3 meter QDR InfiniBand cab...

Страница 301: ...U29 Compute node PCIe 2 P1 2 meter QDR InfiniBand cable U23 7A U28 Compute node PCIe 2 P1 2 meter QDR InfiniBand cable U23 7B U27 Compute node PCIe 2 P1 2 meter QDR InfiniBand cable U23 8A U22 Storage appliance server head PCIe 0 P2 2 meter QDR InfiniBand cable U23 8B U21 Storage appliance server head PCIe 0 P2 2 meter QDR InfiniBand cable U23 9A U15 Compute node PCIe 2 P1 2 meter QDR InfiniBand c...

Страница 302: ...e 2 P2 3 meter QDR InfiniBand cable U23 14B U4 Compute node PCIe 2 P2 3 meter QDR InfiniBand cable U23 15A U3 Compute node PCIe 2 P2 3 meter QDR InfiniBand cable U23 15B U2 Compute node PCIe 2 P2 3 meter QDR InfiniBand cable U16 7A U28 Compute node PCIe 2 P2 3 meter QDR InfiniBand cable U16 7B U27 Compute node PCIe 2 P2 3 meter QDR InfiniBand cable U16 8A U22 Storage appliance server head PCIe 0 P...

Страница 303: ... U9 Compute node PCIe 2 P1 2 meter QDR InfiniBand cable U16 12B U8 Compute node PCIe 2 P1 2 meter QDR InfiniBand cable U16 13A U7 Compute node PCIe 2 P1 3 meter QDR InfiniBand cable U16 13B U6 Compute node PCIe 2 P1 3 meter QDR InfiniBand cable U16 14A U5 Compute node PCIe 2 P1 3 meter QDR InfiniBand cable U16 14B U4 Compute node PCIe 2 P1 3 meter QDR InfiniBand cable U16 15A U3 Compute node PCIe ...

Страница 304: ...twork QDR InfiniBand Gateway Switch 6A 2 meter QDR InfiniBand cable U16 6B U23 Sun Network QDR InfiniBand Gateway Switch 6B 2 meter QDR InfiniBand cable U24 4A U26 Sun Network QDR InfiniBand Gateway Switch 4A 1 meter QDR InfiniBand cable U24 4B U26 Sun Network QDR InfiniBand Gateway Switch 4B 1 meter QDR InfiniBand cable U24 5A U26 Sun Network QDR InfiniBand Gateway Switch 5A 1 meter QDR InfiniBan...

Страница 305: ...d cable U16 1B U26 Sun Network QDR InfiniBand Gateway Switch 1B 2 meter QDR InfiniBand cable U16 2A U24 Sun Network QDR InfiniBand Gateway Switch 2A 2 meter QDR InfiniBand cable U16 2B U24 Sun Network QDR InfiniBand Gateway Switch 2B 2 meter QDR InfiniBand cable U16 3A U24 Sun Network QDR InfiniBand Gateway Switch 3A 2 meter QDR InfiniBand cable U16 3B U24 Sun Network QDR InfiniBand Gateway Switch...

Страница 306: ... InfiniBand Gateway Switch 3A 1 meter QDR InfiniBand cable U23 3B U26 Sun Network QDR InfiniBand Gateway Switch 3B 1 meter QDR InfiniBand cable E 2 2 Exalogic Machine Half Rack This section contains the following tables Table E 8 Table E 9 Table E 8 lists the location ports and cables for the InfiniBand connections for an Exalogic machine half rack Table E 8 InfiniBand Network Cabling for Exalogic...

Страница 307: ...U13 Compute node PCIe 2 P1 2 meter QDR InfiniBand cable U23 10B U12 Compute node PCIe 2 P1 2 meter QDR InfiniBand cable U23 11A U11 Compute node PCIe 2 P1 3 meter QDR InfiniBand cable U23 11B U10 Compute node PCIe 2 P1 3 meter QDR InfiniBand cable U23 12A U9 Compute node PCIe 2 P2 3 meter QDR InfiniBand cable U23 12B U8 Compute node PCIe 2 P2 3 meter QDR InfiniBand cable U23 13A U7 Compute node PC...

Страница 308: ...able U16 9A U15 Compute node PCIe 2 P2 2 meter QDR InfiniBand cable U16 9B U14 Compute node PCIe 2 P2 2 meter QDR InfiniBand cable U16 10A U13 Compute node PCIe 2 P2 2 meter QDR InfiniBand cable U16 10B U12 Compute node PCIe 2 P2 2 meter QDR InfiniBand cable U16 11A U11 Compute node PCIe 2 P2 2 meter QDR InfiniBand cable U16 11B U10 Compute node PCIe 2 P2 2 meter QDR InfiniBand cable U16 12A U9 Co...

Страница 309: ... an Exalogic machine half rack Table E 9 InfiniBand Gateway Switch Inter Connections for Exalogic Machine Half Rack From Rack Unit Sun Network QDR InfiniBand Gateway Switch Port To Rack Unit Type of Equipment Port Description U16 4A U23 Sun Network QDR InfiniBand Gateway Switch 4A 2 meter QDR InfiniBand cable U16 4B U23 Sun Network QDR InfiniBand Gateway Switch 4B 2 meter QDR InfiniBand cable U16 ...

Страница 310: ...Network Cabling for Exalogic Machine Quarter Rack From Rack Unit Sun Network QDR InfiniBand Gateway Switch Port To Rack Unit Type of Equipment Port Description U23 8A U22 Storage appliance server head PCIe 0 P2 2 meter QDR InfiniBand cable U23 8B U21 Storage appliance server head PCIe 0 P2 2 meter QDR InfiniBand cable U23 12A U9 Compute node PCIe 2 P2 3 meter QDR InfiniBand cable U23 12B U8 Comput...

Страница 311: ...PCIe 0 P1 3 meter QDR InfiniBand cable U16 12A U9 Compute node PCIe 2 P1 2 meter QDR InfiniBand cable U16 12B U8 Compute node PCIe 2 P1 2 meter QDR InfiniBand cable U16 13A U7 Compute node PCIe 2 P1 2 meter QDR InfiniBand cable U16 13B U6 Compute node PCIe 2 P1 3 meter QDR InfiniBand cable U16 14A U5 Compute node PCIe 2 P1 3 meter QDR InfiniBand cable U16 14B U4 Compute node PCIe 2 P1 3 meter QDR ...

Страница 312: ... Gateway Switch 5A 2 meter QDR InfiniBand cable U16 5B U23 Sun Network QDR InfiniBand Gateway Switch 5B 2 meter QDR InfiniBand cable U16 6A U23 Sun Network QDR InfiniBand Gateway Switch 6A 2 meter QDR InfiniBand cable U16 6B U23 Sun Network QDR InfiniBand Gateway Switch 6B 2 meter QDR InfiniBand cable E 2 4 Exalogic Machine Eighth Rack This section contains the following tables Table E 12 Table E ...

Страница 313: ...ompute node PCIe 2 P2 3 meter QDR InfiniBand cable U23 15B U2 Compute node PCIe 2 P2 3 meter QDR InfiniBand cable U16 8A U22 Storage appliance server head PCIe 0 P1 3 meter QDR InfiniBand cable U16 8B U21 Storage appliance server head PCIe 0 P1 3 meter QDR InfiniBand cable U16 14A U5 Compute node PCIe 2 P1 3 meter QDR InfiniBand cable U16 14B U4 Compute node PCIe 2 P1 3 meter QDR InfiniBand cable ...

Страница 314: ...QDR InfiniBand Gateway Switch 6A 2 meter QDR InfiniBand cable U16 6B U23 Sun Network QDR InfiniBand Gateway Switch 6B 2 meter QDR InfiniBand cable U16 USB Serial Not applicab le Open ended Not applicable USB Adapter U16 USB Serial Not applicab le Open ended Not applicable USB Adapter E 3 Power Distribution Unit Cabling Tables This section contains the tables for the Power Distribution Units PDU ca...

Страница 315: ...ic Machine Full Rack From Rack Unit Type of Equipment PDU A PDU B Jumper Length U42 Compute node G5 5 G0 1 2M U41 Compute node G5 4 G0 2 2M U40 Compute node G5 3 G0 3 2M U39 Compute node G5 2 G0 4 2M U38 Compute node G5 1 G0 5 2M U37 Compute node G5 0 G0 6 2M U36 Compute node G4 6 G1 0 2M U35 Compute node G4 5 G1 1 2M U34 Compute node G4 4 G1 2 2M U33 Compute node G4 3 G1 3 2M U32 Compute node G4 ...

Страница 316: ... 3 2M U10 Compute node G1 2 G4 4 2M U09 Compute node G1 1 G4 5 2M U08 Compute node G1 0 G4 6 2M U07 Compute node G0 6 G5 0 2M U06 Compute node G0 5 G5 1 2M U05 Compute node G0 4 G5 2 2M U04 Compute node G0 3 G5 3 2M U03 Compute node G0 2 G5 4 2M U02 Compute node G0 1 G5 5 2M U01 NM2 36P SW G0 0 G5 6 2M E 3 2 Single Phase PDU Power Cabling Half Rack This section contains the following tables Table ...

Страница 317: ... 2M U12 Compute node G1 4 G4 2 2M U11 Compute node G1 3 G4 3 2M U10 Compute node G1 2 G4 4 2M U09 Compute node G1 1 G4 5 2M U08 Compute node G1 0 G4 6 2M U07 Compute node G0 6 G5 0 2M U06 Compute node G0 5 G5 1 2M U05 Compute node G0 4 G5 2 2M U04 Compute node G0 3 G5 3 2M U03 Compute node G0 2 G5 4 2M U02 Compute node G0 1 G5 5 2M U01 NM2 36P SW G0 0 G5 6 2M E 3 3 Single Phase PDU Power Cabling Q...

Страница 318: ...M U03 Compute node G0 2 G5 4 2M U02 Compute node G0 1 G5 5 2M E 3 4 Single Phase PDU Power Cabling Eighth Rack This section contains the following tables Table E 17 Table E 17 lists the location ports and cables for the Single Phase PDU connections for an Exalogic machine eighth rack Table E 17 Single Phase PDU Power Cabling for Exalogic Machine Eighth Rack From Rack Unit Type of Equipment PDU A P...

Страница 319: ...ack Table E 18 Three Phase PDU Power Cabling for Exalogic Machine Full Rack From Rack Unit Type of Equipment PDU A PDU B Jumper Length U42 Compute node G5 5 G2 1 2M U41 Compute node G5 4 G2 2 2M U40 Compute node G5 3 G2 3 2M U39 Compute node G5 2 G2 4 2M U38 Compute node G5 1 G2 5 2M U37 Compute node G5 0 G2 6 2M U36 Compute node G4 6 G1 0 2M U35 Compute node G4 5 G1 1 2M U34 Compute node G4 4 G1 ...

Страница 320: ... node G1 4 G4 2 2M U11 Compute node G1 3 G4 3 2M U10 Compute node G1 2 G4 4 2M U09 Compute node G1 1 G4 5 2M U08 Compute node G1 0 G4 6 2M U07 Compute node G0 6 G3 0 2M U06 Compute node G0 5 G3 1 2M U05 Compute node G0 4 G3 2 2M U04 Compute node G0 3 G3 3 2M U03 Compute node G0 2 G3 4 2M U02 Compute node G0 1 G3 5 2M U01 NM2 36P SW G0 0 G3 6 2M E 3 6 Three Phase PDU Power Cabling Half Rack This se...

Страница 321: ...6 2M U12 Compute node G1 4 G4 2 2M U11 Compute node G1 3 G4 3 2M U10 Compute node G1 2 G4 4 2M U09 Compute node G1 1 G4 5 2M U08 Compute node G1 0 G4 6 2M U07 Compute node G0 6 G3 0 2M U06 Compute node G0 5 G3 1 2M U05 Compute node G0 4 G3 2 2M U04 Compute node G0 3 G3 3 2M U03 Compute node G0 2 G3 4 2M U02 Compute node G0 1 G3 5 2M U01 NM2 36P SW G0 0 G3 6 2M E 3 7 Three Phase PDU Power Cabling Q...

Страница 322: ...2M U03 Compute node G0 2 G3 4 2M U02 Compute node G0 1 G3 5 2M E 3 8 Three Phase PDU Power Cabling Eighth Rack This section contains the following tables Table E 21 Table E 21 lists the location ports and cables for the Three Phase PDU connections for an Exalogic machine eighth rack Table E 21 Three Phase PDU Power Cabling for Exalogic Machine Eighth Rack From Rack Unit Type of Equipment PDU A PDU...

Страница 323: ...c Machine Eighth Rack From Rack Unit Type of Equipment PDU A PDU B Jumper Length U05 Compute node G0 4 G3 2 2M U04 Compute node G0 3 G3 3 2M U03 Compute node G0 2 G3 4 2M U02 Compute node G0 1 G3 5 2M Power Distribution Unit Cabling Tables Cabling Tables E 31 ...

Страница 324: ...Power Distribution Unit Cabling Tables E 32 Machine Owner s Guide ...

Страница 325: ...tch for Zones on Shared Storage ZOSS over iSCSI available in the My Oracle Support document ID 16514816 F 2 Terminology Table F 1 describes the terms used in this appendix Table F 1 Terminology Term Description Logical Unit A logical unit is a component of a storage system A logical unit is uniquely numbered creating a Logical Unit Number LUN The storage appliance can contain many LUNs LUNs when a...

Страница 326: ...nitiator Create the Project and LUN Disable the Write Cache Format the LUN Set Up the Exclusive 10 GbE Network for the Zone F 3 1 1 Create an iSCSI Target You can create an iSCSI target by doing the following 1 Log in to the storage appliance BUI as the root user 2 Click the Configuration tab 3 Click SAN 4 Click iSCSI Targets 5 To create a new iSCSI target click the plus button next to iSCSI Targe...

Страница 327: ... F 3 1 2 Create an iSCSI Initiator You can create an iSCSI initiator by doing the following 1 Before you can create an iSCSI initiator you must identify an initiator IQN The initiator IQN is a unique reference number associated with a specific compute node To find the initiator IQN for a compute node do the following a Log in to an Exalogic compute node b Run the iscsiadm list initiator node comma...

Страница 328: ...11 Click OK 12 Add the iSCSI initiator to an iSCSI initiator group by dragging and dropping the initiator If required you can create a new iSCSI initiator group F 3 1 3 Create the Project and LUN You can create the project and LUN by doing the following 1 Create a project as described in Create Custom Projects 2 You can create the LUN by doing the following a Next to the Project click Shares b Cli...

Страница 329: ... 0 0 0 0 0 0 0 0 0 0 0 4 Restart the compute node by running the reboot command F 3 1 5 Format the LUN Before the compute node can use the LUN you must format the LUN You can format the LUN by doing the following 1 Log in to a compute node as the root user 2 Run the iscsiadm commands to discover the iSCSI targets from the compute node iscsiadm add discovery address IPoIB_address_of_the_storage_app...

Страница 330: ...un the format command to start formatting the disk as follows format e c0t600144F09C96CCA900005190BFC4000Ad0 selecting c0t600144F09C96CCA900005190BFC4000Ad0 disk formatted The format prompt appears b Enter fdisk to manipulate the partition tables as follows format fdisk No fdisk table exists The default partition for the disk is a 100 SOLARIS System partition c When prompted enter n to edit the pa...

Страница 331: ...c7t0d0 LSI MR9261 8i 2 12 28 87GB pci 0 0 pci8086 340a 3 pci1000 9263 0 sd 0 0 Specify disk enter its number F 3 1 6 Set Up the Exclusive 10 GbE Network for the Zone The zone you want to create should be given access to an exclusive network You should create the necessary VNICs for the zone by doing the following 1 Create a VLAN and VNIC Follow the procedure in Oracle Solaris Creating VNICs and As...

Страница 332: ...l and Boot Up the Zone F 3 2 1 Create a Zone You can create a zone by doing the following 1 Log in to the compute node as the root user 2 Run the zonecfg command to configure the zone as follows zonecfg z zone_name Example zonecfg z zone04 Use create to begin configuring a new zone In this example the name of the zone you are creating is zone04 3 Enter create to begin configuring the zone zonecfg ...

Страница 333: ...zonecfg zone04 remove anet zonecfg zone04 add net zonecfg zone04 net set physical vnic2_1706 zonecfg zone04 net end zonecfg zone04 add net zonecfg zone04 net set physical vnic3_1706 zonecfg zone04 net end zonecfg zone04 verify zonecfg zone04 commit 5 You can verify the details of the zone by running the info command as follows zonecfg zone04 info zonename zone04 zonepath zones zone04 brand solaris...

Страница 334: ...eating IPS image Startup linked 1 1 done Installing packages from exa family origin http localhost 1008 exa family acbd22da328c302a86fb9f23d43f5d10f13cf5a6 solaris origin http install1 release solaris DOWNLOAD PKGS FILES XFER MB SPEED Completed 185 185 34345 34345 229 7 229 7 10 6M s PHASE ITEMS Installing new actions 48269 48269 Updating package state database Done Updating image state Done Creat...

Страница 335: ...0 lo0 138 3 48 0 138 3 51 2 U 2 0 bond1 Routing Table IPv6 Destination Mask Gateway Flags Ref Use If 1 1 UH 2 0 lo0 7 Add the IP address of the bond you noted in step 5 by running the following command root zone04 route p add default IP_address_of_bond Example root zone04 route p add default 138 3 48 1 add net default gateway 138 3 48 1 add persistent net default gateway 138 3 48 1 8 Display the r...

Страница 336: ...ove one or more iSCSI discovery addresses because logical unit is in use Exported zone zpool zone04_rpool Unconfigured zone storage resource s from iscsi 192 168 14 133 luname naa 600144f09c96cca900005190bfc4000a 4 Create a directory on the storage appliance to which you can export the configuration of the zone mkdir p directory Example mkdir p u01 common general zone04 5 Export the configuration ...

Страница 337: ...o image Processing linked 1 1 done Updating non global zone Auditing packages No updates necessary for this image Updating non global zone Zone updated Result Attach Succeeded Log saved in non global zone as zones zone04 root var log zones zoneadm 20130513T135704Z zone04 attach 9 Boot up the zone by running the following command zoneadm z name_of_zone boot Example zoneadm z zone04 boot Note In som...

Страница 338: ...Migrate a Zone to a New Host F 14 Machine Owner s Guide ...

Страница 339: ...endix contains the following sections RPMs That Must Not Be Modified or Removed Prepare the Compute Nodes for Yum Updates Install Update and Remove RPMs Using Yum G 1 RPMs That Must Not Be Modified or Removed Do not modify or delete the following RPMs outside of an Exalogic upgrade patch set update PSU or patch kernel compat dapl dapl ib bonding ibacm ibutils ibsim infiniband diags libibcm libibma...

Страница 340: ...entire directive must be on one line exclude kernel compat dapl dapl ib bonding ibacm ibutils ibsim infiniband diags libibcm libibmad libibumad libibverbs libmlx4 librdmacm libsdp mpi selector mpitests_openmpi_gcc mstflint mvapich ofa ofed openmpi_gcc opensm perftest qperf rds tools sdpnetstat srptools exalogic infinibus This exclusion list includes all of the RPMs listed in RPMs That Must Not Be ...

Страница 341: ...name For the yum update or yum remove command if you specify an RPM that is in the exclusion list defined earlier the command will fail Install Update and Remove RPMs Using Yum Customize Linux on the Compute Nodes G 3 ...

Страница 342: ...Install Update and Remove RPMs Using Yum G 4 Machine Owner s Guide ...

Результаты поиска

Содержание Exalogic Elastic Cloud

Отзывы:

Похожие инструкции для Exalogic Elastic Cloud

Бренды по названию

Популярные бренды

Oracle Exalogic Elastic Cloud, Руководство пользователя

Результаты поиска

Содержание Exalogic Elastic Cloud

Отзывы:

Похожие инструкции для Exalogic Elastic Cloud

Бренды по названию

Популярные бренды