background image

Expanding the base configuration storage shelf with 18 TB HDDs for high
capacity.
 When expanding the storage shelf with HDDs, the base configurations
SSDs will be used for FLASH and the HDDs will be used for DATA/RECO.
Expansion HDDs are available in an eighteen-pack and populate the empty shelf
storage slots (slots 6-23).

The following illustration shows the expanded drive configuration for 18 TB HDDs.

 

 

Once you expand storage to support additional disks, you cannot revert to the
base configuration (no drives in slots 6-23).

If you plan on adding another storage shelf, it must be populated identically to your
first storage shelf.

The 

odaadmcli

 commands listed here must be run as the root user.

1.

Do one of the following:

If you are adding 7.68 TB SSDs to the storage shelf, leave the appliance
powered on and proceed to the next step.

If you are adding 18 TB HDDs to the storage shelf, power off the appliance
and proceed to the next step.

2.

To install the new drives into the storage shelf, do the following:

a.

Remove one of the empty slot drive filler panels by pressing its release button
[1] and removing it from the drive slot [2] as shown in the following figure.

Note:

Replace filler panels with drives one at a time to ensure proper
storage shelf chassis cooling.

Chapter 3

Oracle Database Appliance X9-2-HA Storage Shelf Hot Pluggable Components

3-8

Summary of Contents for X9-2

Page 1: ...Oracle Database Appliance X9 2 Service Manual F58388 01 June 2022 ...

Page 2: ... documentation and or iii other Oracle data is subject to the rights and limitations specified in the license contained in the applicable contract The terms governing the U S Government s use of Oracle cloud services are defined by the applicable contract for such services No other rights are granted to the U S Government This software or hardware is developed for general use in a variety of infor...

Page 3: ... Server Node Back Panel Status Indicators 1 9 Oracle Database Appliance X9 2 HA Storage Shelf Indicators 1 12 Storage Shelf Front Panel Status Indicators 1 12 Storage Shelf Back Panel Status Indicators 1 14 2 Preparing Oracle Database Appliance for Service Preparing Oracle Database Appliance X9 2 Series for Service 2 1 3 Servicing CRUs That Do Not Require System Power Off Oracle Database Appliance...

Page 4: ... Component Fault Management 5 2 6 Returning Oracle Database Appliance to Operation Returning Oracle Database Appliance X9 2 Series to Operation 6 1 Powering On Oracle Database Appliance X9 2 HA 6 2 7 Server Node Diagnostic Tools and Resetting the Service Processor Server Node Diagnostic Tools 7 1 Resetting the Service Processor 7 2 8 Server Basic Input Output System BIOS Oracle Database Appliance ...

Page 5: ... electronic support through My Oracle Support For information visit http www oracle com pls topic lookup ctx acc id info or visit http www oracle com pls topic lookup ctx acc id trs if you are hearing impaired Related Documents For more information about Oracle Database Appliance go to http www oracle com goto oda docs and click the appropriate release The following documents are published in the ...

Page 6: ... action or terms defined in the text italic Italic type indicates book titles emphasis or placeholder variables for which you supply particular values monospace Monospace type indicates commands within a paragraph URLs code in examples text that appears on the screen or text that you enter prompt The pound prompt indicates a command that is run as the root user Preface vi ...

Page 7: ...X9 2S X9 2L status indicators Oracle Database Appliance X9 2S X9 2L Server Status Indicators See a description of the front and back status indicators for the Oracle Database Appliance X9 2S X9 2L Oracle Database Appliance X9 2S X9 2L Server Status Indicators See a description of the front and back status indicators for the Oracle Database Appliance X9 2S X9 2L Server Front Panel Status Indicators...

Page 8: ...and off remotely using Oracle ILOM Pressing this button toggles the fast blink on or off 2 Service Required amber Off Normal operation On A fault is present in the server node The lighting of this indicator is always accompanied by a system console message that includes a recommended service action Chapter 1 Oracle Database Appliance X9 2S X9 2L Status Indicators 1 2 ...

Page 9: ...o standby power On The server host is powered on and is in the process of booting or is running the operating system 4 Power button Press to apply full power and start the server host Press again to transition from full power to standby power where only the Oracle ILOM SP is running 5 Top fan amber Indicates that one or more of the internal fan modules has failed Off Indicates steady state no serv...

Page 10: ...al operation On The system is not ready for service The DO NOT SERVICE indicator is application specific This indicator is only illuminated on demand by the Host application 10 Drive Ready to Remove blue Off The storage drive has not been prepared for removal On The storage drive can be removed safely during a hot plug operation 11 Drive Service Required amber Off The storage drive is operating no...

Page 11: ...s Callout Indicator LED Color State Meaning 1 PS Service Required amber Off Normal operation On The power supply PS has detected a PS fan failure PS overtemperature PS over current or PS over or under voltage 2 PS OK green Off No AC power is present On Normal operation Input AC power and DC output voltage are within specification Slow blink Normal operation Input power is within specification DC o...

Page 12: ...GbE link Yellow On 1GbE link 4 NET MGT port green 10 100 1000 BASE T Ethernet management interface Top left speed indicator Off 10 100BASE T link On Green for 1000Base T link Top right activity indicator Off No link or link down On Link up Blinking Packet activity 5 NET 0 port green 10 100 1000 BASE T Ethernet interface Bottom left activity indicator Off No link or link down On Link up Blinking Pa...

Page 13: ...d back status indicators Server Node Front Panel Status Indicators See a listing of the various font panel status indicators for the Oracle Database Appliance X9 2 HA server node Server Node Back Panel Status Indicators See a listing of the various back panel status indicators for the Oracle Database Appliance X9 2 HA server node Server Node Front Panel Status Indicators See a listing of the vario...

Page 14: ...not present or the Oracle ILOM service processor SP boot process is not complete Steady blink Standby power is on and the Oracle ILOM SP is running but the server host power is off Slow blink Startup shutdown sequence has been initiated on the server host During startup the power on self tests are running on the system During shutdown the server host is shutting down and transitioning from full po...

Page 15: ...his condition might result in the system overheating and shutting down unexpectedly 8 Service processor SP green Off The service processor SP is not running On The SP is fully operational Slow blink SP is booting 9 DO NOT SERVICE white Indicates that the system is not ready to service Off Normal operation On The system is not ready for service The DO NOT SERVICE indicator is application specific T...

Page 16: ...Input AC power and DC output voltage are within specification Slow blink Normal operation Input power is within specification DC output voltage is not enabled 3 PORT 1 and 2 private cluster interconnect 10 25 GbE SFP28 ports green yellow 10 25 GbE SFP28 interface Left link indicator Off No link Green On 25GbE link Yellow On 10GbE link Right activity indicator Off No activity Flashing Packet activi...

Page 17: ... locate service required and OK various These operate in the same way as on the front panel 7 7a Oracle Dual Port 25 Gb Ethernet Adapter SFP28 ports green yellow or 7b Oracle Quad Port 10GBase T Adapter RJ 45 ports green yellow Oracle Dual Port 25 Gb Ethernet Adapter Each of the two ports has Left link indicator Off No function Green On 25GbE link Yellow On 10GbE link Right activity indicator Off ...

Page 18: ... Panel Status Indicators See a listing of the various font panel status indicators for the Oracle Database Appliance X9 2 HA storage shelf The following figure shows the controls and indicators on the for the Oracle Database Appliance X9 2 HA storage shelf DE3 24C front panel and drives Note The 2 or 3 character LED enclosure ID display panel on the front of the storage shelf is not used with the ...

Page 19: ...mine which power supply has the fault 5 SAS I O Module fault indicator amber Off Normal operation On Indicates a fault with a SAS I O module View the LEDs at the back of the storage shelf to determine which module has the fault 6 Ready to Remove blue Off The storage drive has not been prepared for removal On The storage drive can be safely removed during a hot plug operation 7 Drive Service Requir...

Page 20: ...r fault indicator amber Off Normal operation On Indicates an AC power fault 2 Fan fail indicator amber Off Normal operation On The fan is faulty 3 Power supply status indicator green Off The power supply is unplugged or turned off On The power supply is operating normally 4 DC power fail indicator amber Off Normal operation On The power supply is faulty 5 Reserved This port is reserved for Oracle ...

Page 21: ...y to Remove blue Off The I O module has not been prepared for removal On The I O module can be safely removed during a hot plug operation 11 SAS I O Module Service Required indicator amber Off Normal operation Steady on A fault is present in the SAS I O module 12 SAS I O Module OK green Off The I O module is unplugged or turned off On The I O module is powered on and operating normally 13 Host SAS...

Page 22: ...he appliance to prepare it for service Note The links in the following table take you to instructions outside of the Oracle Database Appliance library to the Oracle Server X9 2L documentation library Use your browser s Back button to return to the Oracle Database Appliance library Oracle Database Appliance X9 2S X9 2L Oracle Database Appliance X9 2 HA Safety Precautions Safety Precautions Safety S...

Page 23: ...m hot plug tasks on an Oracle Database Appliance storage shelf Oracle Database Appliance X9 2 Server Hot Pluggable Components Provides instructions on how to perform hot plug tasks on an Oracle Database Appliance server This section describes how to service customer replaceable components that do not require server power off for an Oracle Database Appliance X9 2S X9 2L server or X9 2 HA server nod...

Page 24: ...rn to the Oracle Database Appliance library Oracle Database Appliance X9 2S X9 2L Oracle Database Appliance X9 2 HA Servicing NVMe Storage Drives CRU Caution An NVMe drive must first be powered off before removal If you do not power off the NVMe drive before removal you can crash the operating system Oracle Database Appliance uses special commands to remove or add NVMe storage Front mounted storag...

Page 25: ...dentify the NVMe drive to be replaced Type the following command at the host odaadmcli show disk For example the drive names pd_00 and pd_01 are seen by the host as NVMe0 and NVMe1 and correspond to front drive slots 0 and 1 respectively 2 Unmount and power off the drive to be replaced Do one of the following a If you are replacing a failed drive check to see if its blue ready to remove light is l...

Page 26: ...he following figure 6 Power on and remount the replacement drive Type the following command from the host odaadmcli power disk on pd_xx Where pd_xx is the name of the disk being replaced as described in step 1 It can take a few minutes for the power on operation to complete Once complete the drive s green OK indicator will light 7 To check the status of the replacement disk type the following comm...

Page 27: ...ed Once you expand storage you cannot revert to a previous storage configuration for example the default installation with only two NVMe drives 1 Remove the drive filler panel of an unpopulated slot by using your finger to pry open the panel release latch and pulling the filler out of the drive slot Note Replace filler panels with drives one at a time to ensure proper system cooling 2 Open the lat...

Page 28: ... Pluggable Components Perform hot plug tasks on an Oracle Database Appliance storage shelf Under most circumstances Oracle Database Appliance X9 2 HA storage shelf components can all be replaced while the storage shelf is powered on and running Any exceptions will be described in the following procedures Adding Optional Storage Shelf Drives CRU Add disk storage in an Oracle Database Appliance X9 2...

Page 29: ...ge and redeploy the appliance software This is required to reconfigure the appliance from the default high performance mode to high capacity mode Expanding the base configuration storage shelf with 7 68 TB SSDs for high performance The base configuration has slots 0 5 with 7 68 TB drives for DATA RECO Expansion SSDs are available in six packs and are populated in shelf storage slots from the botto...

Page 30: ...ust be populated identically to your first storage shelf The odaadmcli commands listed here must be run as the root user 1 Do one of the following If you are adding 7 68 TB SSDs to the storage shelf leave the appliance powered on and proceed to the next step If you are adding 18 TB HDDs to the storage shelf power off the appliance and proceed to the next step 2 To install the new drives into the s...

Page 31: ...e slot 1 until it engages with the backplane connector and then close the lever to fully seat the drive and lock it into place 2 as shown in the following figure Check that the drive is locked in place and flush with the other drives Chapter 3 Oracle Database Appliance X9 2 HA Storage Shelf Hot Pluggable Components 3 9 ...

Page 32: ...ly be added to a system with an identically and fully populated storage shelf you can do so after successfully completing the previous step Refer to the Oracle Database Appliance Owner s Guide for information on installing a storage expansion shelf into a rack Refer to the Oracle Database Appliance Deployment and User s Guide for information on cabling and storage expansion commands for a storage ...

Page 33: ...he replacement drive by opening the drive lever on the replacement drive Slide the new drive into the drive slot 1 until it engages with the backplane connector and then close the lever to fully seat the drive and lock it into place 2 as shown in the following figure Check that the drive is locked in place and flush with the other drives Chapter 3 Oracle Database Appliance X9 2 HA Storage Shelf Ho...

Page 34: ...wer Supply CRU Replace a power supply in an Oracle Database Appliance storage shelf The storage shelf and the storage expansion shelf each have two power supplies Indicators LEDs on the power supplies indicate their operation state The power supplies are redundant and one can be replaced while the appliance is running Example figure showing a DE3 24C power supply Chapter 3 Oracle Database Applianc...

Page 35: ...ht replace the power supply 1 Ensure the power supply on off switch is in the O off position 2 Disconnect the power cord tie strap from the power cord and unplug the power cord from the power supply 3 Remove the power supply unit by grasping the power supply handle push the release button 1 and slide the power supply out of the chassis 2 as shown in the following figure Chapter 3 Oracle Database A...

Page 36: ...s should be off Replace a Storage Shelf I O Module CRU Replace an IO module in an Oracle Database Appliance storage shelf Storage Shelf SAS I O modules are fully redundant and can be replaced while the appliance is running Note After a physical insertion or removal action of a storage component such as a disk or SAS cable you should wait before performing the next action to allow the system softwa...

Page 37: ...ors 2 Remove the I O module using your thumb and forefinger pinch the release buttons 1 together to release the locking lever 2 as shown in the following figure Grasp the lever and remove the I O module from the chassis 3 as shown in the following figure 3 Open the lever in the new I O module if it is not already open Chapter 3 Oracle Database Appliance X9 2 HA Storage Shelf Hot Pluggable Componen...

Page 38: ... O module 5 Reconnect the cables 6 Verify that the new I O module is working correctly After approximately 60 seconds the power LED should be solid green and the Fault Locate LED should be off For each port that has a cable connected all four activity LEDs should be solid green Chapter 3 Oracle Database Appliance X9 2 HA Storage Shelf Hot Pluggable Components 3 16 ...

Page 39: ... off This section describes how to service customer replaceable components that require server power off for an Oracle Database Appliance X9 2 series system Note For the Oracle Database Appliance X9 2 HA all of the CRUs on the storage shelf and the storage expansion shelf can be replaced while the power is on See Oracle Database Appliance Storage Shelf Hot Pluggable Components for storage shelf co...

Page 40: ...any optional public network cards when you initially buy the system and have the cards factory installed If you want to add or replace public network cards after your appliance has been delivered from the factory the following rules apply You can install any of the supported network cards listed in the table below You must install optional network cards in the proper order and slots refer to the t...

Page 41: ...a public network card installed in PCIe slot 6 2 A second public network card can be installed in PCIe slot 7 3 A third public network card can be installed in PCIe slot 8 1 By default the server comes with a public network card installed in PCIe slot 6 2 A second public network card can be installed in PCIe slot 4 3 A third public network card can be installed in PCIe slot 8 1 By default each ser...

Page 42: ...Oracle Database Appliance X9 series system This section describes how to service field replaceable components that require server power off for an Oracle Database Appliance X9 2 series system Only authorized Oracle Service personnel can service FRU components Note Most of the links in the following table take you to instructions outside of the Oracle Database Appliance library to the Oracle Server...

Page 43: ... in the component firmware For replaced components that do not have an embedded FRU serial number or do not automatically clear their fault state after the component has been repaired and the server is returned to service you can use Oracle ILOM command line interface to manually clear the fault state Use the following syntax set path_to_target clear_fault_action true Where path_to_target is the p...

Page 44: ...er servicing an Oracle Database Appliance X9 2 series system Note Most of the links in the following table take you to instructions outside of the Oracle Database Appliance library to the Oracle Server X9 2L documentation library Use your browser s Back button to return to the Oracle Database Appliance library Oracle Database Appliance X9 2S X9 2L Oracle Database Appliance X9 2 HA Install the Top ...

Page 45: ...f and storage expansion shelf have initialized 3 If you powered off a server node reconnect any unplugged power cords 4 Power on a server node on using one of the following methods Press the recessed Power button on the server node front panel Log in to the Oracle ILOM web interface click Host Management Power Control and select Power On from the Select Action list box Log in to the Oracle ILOM co...

Page 46: ...various Oracle Database Appliance diagnostic tools and where to obtain more information about them Resetting the Service Processor How to reset the service processor SP of a server node of an Oracle Database Appliance Server Node Diagnostic Tools See a listing of the various Oracle Database Appliance diagnostic tools and where to obtain more information about them Oracle provides a wide selection ...

Page 47: ...s supported version of Oracle ILOM Oracle Integrated Lights Out ILOM Documentation Library Power On Self Test POST Tests core components of system CPUs memory and motherboard I O bridge integrated circuits For Oracle Database Appliance X9 2 series refer to the appropriate Oracle x86 Servers Diagnostics Guide based on your appliance s supported version of Oracle ILOM Oracle x86 Servers Administrati...

Page 48: ...the SP using the Oracle ILOM web interface or the CLI for example if the SP hangs then use the SP reset button as described in the following tables Note The links in the following tables take you to instructions outside of the Oracle Database Appliance library Use your browser s Back button to return to the Oracle Database Appliance library For Oracle Database Appliance X9 2 series Oracle Database...

Page 49: ...mpted to do so by Oracle Service personnel Oracle Database Appliance X9 2 Series BIOS Utility Learn about the BIOS Setup Utility for an Oracle Database Appliance server Oracle Database Appliance X9 2 Series BIOS Utility Learn about the BIOS Setup Utility for an Oracle Database Appliance server This section describes the BIOS Setup Utility of an Oracle Database Appliance X9 2 series system All sett...

Page 50: ...S Setup Utility Tasks BIOS Setup Utility Menu Options Learn about the BIOS utility menu options for Oracle Database Appliance X9 2 series Note The links in the following table take you to instructions outside of the Oracle Database Appliance library to the Oracle Server X9 2L documentation library Use your browser s Back button to return to the Oracle Database Appliance library Oracle Database App...

Page 51: ... 5 1 CRUs component locations 4 1 CRUs continued removal that does not require system power off 3 1 servicing non hotplug components 4 1 D date and time resetting after battery replacement 4 1 diagnostic tools summary of 7 1 diagnostics available tools 7 1 DIMMs clearing faults 5 2 installing 4 1 population rules 4 1 removing 4 1 servicing 4 1 servicing PMEM NVDIMMs 4 1 disk backplane 5 1 disk dri...

Page 52: ...racle Database Appliance X9 2S X9 2L 1 5 Ethernet ports Oracle Database Appliance X9 2 HA 1 9 Oracle Database Appliance X9 2S X9 2L 1 5 fans Oracle Database Appliance X9 2 HA 1 7 Oracle Database Appliance X9 2S X9 2L 1 1 general status Oracle Database Appliance X9 2 HA 1 7 Oracle Database Appliance X9 2S X9 2L 1 1 Locate Oracle Database Appliance X9 2 HA 1 7 Oracle Database Appliance X9 2S X9 2L 1...

Page 53: ...ver Oracle Database Appliance X9 2S X9 2L 3 2 replacing in server Oracle Database Appliance X9 2S X9 2L 3 2 O option ROM settings configuring 8 2 Oracle Integrated Lights Out Manager Oracle ILOM server node power off procedures 2 1 server node power on procedures Oracle Database Appliance X9 2 HA 6 2 SP reset 7 2 used as diagnostic tool 7 1 P Pc Check 7 1 PCIe cards adding 4 2 clearing faults 5 2 ...

Page 54: ...2 replacing disk drives in a storage shelf 3 10 servicing continued replacing fans in server 3 1 replacing hard disk drives in server 3 1 replacing I O module in storage shelf 3 14 replacing power supplies in server 3 1 resetting the SP 7 1 Retimer card 5 1 returning the appliance to operation 6 1 returning the system to operation 6 1 SAS HBA cards 4 1 server node battery 4 1 troubleshooting and d...

Page 55: ... Service Required Oracle Database Appliance X9 2S X9 2L 1 5 server Service Required Oracle Database Appliance X9 2S X9 2L 1 1 Service Required Oracle Database Appliance X9 2 HA 1 7 status indicators continued Service Required continued Oracle Database Appliance X9 2S X9 2L 1 1 storage devices Oracle Database Appliance X9 2 HA 1 7 Oracle Database Appliance X9 2S X9 2L 1 1 storage adding NVMe storag...

Reviews: