manualshive.com logo in svg

Oracle EXADATA X6-2, Service Manual, Page: 22 / 284

Share
Download
Oracle EXADATA X6-2 Service Manual Download Page 22

About System Components

22

Oracle Server X6-2 Service Manual • June 2016

Related Information

■

“Customer-Replaceable Units” on page 19

■

“Illustrated Parts Breakdown” on page 17

■

“Servicing FRUs” on page 119

Summary of Contents for EXADATA X6-2

Page 1: ...Part No E62171 02 June 2016 Oracle Server X6 2 Service Manual ...

Page 2: ......

Page 3: ...s applications including applications that may create a risk of personal injury If you use this software or hardware in dangerous applications then you shall be responsible to take all appropriate fail safe backup redundancy and other measures to ensure its safe use Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous ...

Page 4: ...à risque notamment dans des applications pouvant causer des dommages corporels Si vous utilisez ce logiciel ou matériel dans le cadre d applications dangereuses il est de votre responsabilité de prendre toutes les mesures de secours de sauvegarde de redondance et autres mesures nécessaires à son utilisation dans des conditions optimales de sécurité Oracle Corporation et ses affiliés déclinent tout...

Page 5: ...and Diagnostics 23 Troubleshooting Server Component Hardware Faults 23 Troubleshooting Server Hardware Faults 24 Troubleshooting and Diagnostic Information 27 Troubleshooting Using the Server Front and Rear Panel Status Indicators 28 Troubleshooting System Cooling Issues 33 Troubleshooting Power Issues 35 Managing Server Hardware Faults Through the Oracle ILOM Fault Management Shell 37 Troubleshoo...

Page 6: ...Cables From the Server 56 Extend the Server to the Maintenance Position 56 Remove the Server From the Rack 58 Take Antistatic Measures 59 Open the Server Fan Door 59 Remove the Server Top Cover 60 Servicing CRUs That Do Not Require Server Power Off 63 Servicing Storage Drives CRU 63 Storage Drives Hot Plug Conditions 64 Storage Drive Failure and RAID 64 Storage Drive Locations and Numbering 64 Sto...

Page 7: ...a PCIe Riser From PCIe Slot 1 or 2 99 Install a PCIe Riser Into PCIe Slot 1 or 2 101 Remove the PCIe Riser From PCIe Slots 3 and 4 103 Install the PCIe Riser Into PCIe Slots 3 and 4 105 Servicing PCIe Cards CRU 108 PCIe Slot Characteristics 108 Remove a PCIe Card From PCIe Slot 1 or 2 109 Install a PCIe Card in PCIe Slot 1 or 2 110 Remove a PCIe Card From PCIe Slot 3 111 Install a PCIe Card in PCI...

Page 8: ...ing the Motherboard FRU 162 Remove the Motherboard 162 Install the Motherboard 169 Servicing the Internal HBA SAS Cable assembly 173 Remove the Internal HBA SAS Cable Assembly 173 Install the Internal HBA SAS Cable Assembly 176 Servicing the NVMe Cables 177 Removing the NVMe Cables 178 Install the NVMe Cables 180 Servicing the Power FIM Disk Backplane Data and DVD Cables 181 Removing the Power FIM...

Page 9: ...ource Allocation 209 Common BIOS Setup Utility Tasks 210 Verify BIOS Factory Default Settings 210 Select Legacy BIOS or UEFI Boot Mode 211 Select the Boot Device 213 Configure TPM Support 215 Configure SP Network Settings 217 Configure Option ROM Settings 220 Configure I O Resource Allocation 223 Exit BIOS Setup Utility 224 BIOS Setup Utility Menu Options 227 BIOS Main Menu Selections 227 BIOS Adv...

Page 10: ...Boot Menu Selections 248 BIOS Exit Menu Selections 249 Monitoring Components and Identifying SNMP Messages 251 Monitoring Component Health and Faults Using Oracle ILOM 251 Monitoring System Components 252 System Chassis Components 253 Cooling Unit Components 255 Disk Backplane Components 255 Memory Device Components 256 Power Unit Components 257 Processor Components 258 System Board Components 259...

Page 11: ...and authorized service personnel who have been instructed on the hazards within the equipment and qualified to remove and replace hardware Required knowledge Advanced experience troubleshooting and replacing hardware Product Documentation Library Documentation and resources for this product and related products are available at http www oracle com goto x6 2 docs Feedback Provide feedback about thi...

Page 12: ...12 Oracle Server X6 2 Service Manual June 2016 ...

Page 13: ...t system components About System Components on page 17 Product Description The Oracle Server X6 2 is an enterprise class one rack unit 1U server It supports the following components Up to two Intel processors Processors with the following capabilities are supported 22 core 2 2 GHz 145W processors 14 core 2 6 GHz 135W processors 10 core 2 2 GHz 85W processors 6 core 3 4 GHz 135W processors Up to 12...

Page 14: ...supplies An on board Oracle Integrated Lights Out Manager Oracle ILOM service processor SP based on the Emulex Pilot3 chip About Controls and Connectors The following sections describe the controls indicators connectors and drives located on the front and rear panels Front Panel Controls and Indicators on page 14 Server Back Panel View on page 16 Related Information About System Components on page...

Page 15: ...te 3 USB 2 0 connectors 2 4 Service Required LED amber 5 Power OK LED green 6 Power button 7 SP OK LED green 8 Top Fan Fault LED amber 9 Rear Power Supply PS Fault LED amber 10 System Over Temperature Warning LED amber 11 Storage drive 0 HDD SSD 12 Storage drive 1 HDD SSD 13 Storage drive 2 HDD SSD NVMe 14 Storage drive 3 HDD SSD NVMe 15 Storage drive 4 HDD SSD NVMe 16 Storage drive 5 HDD SSD NVMe...

Page 16: ...e Legend 1 Power Supply PS 0 2 Power Supply PS 0 status indicators Service Required LED amber AC OK LED green 3 Power Supply PS 1 4 Power Supply PS 1 status indicators Service Required LED amber AC OK LED green 5 System status indicators Locator LED white Service Required LED amber Power OK LED green 6 PCIe card slot 1 This slot is for a standard PCIe card or the optional Oracle PCIe NVMe switch c...

Page 17: ...onnector Note All of the PCIe slots comply with the PCI Express 3 0 specification and can accommodate 25 watt PCIe3 cards Related Information Front Panel Controls and Indicators on page 14 About System Components on page 17 About System Components These sections describe the components of the server Illustrated Parts Breakdown on page 17 Customer Replaceable Units on page 19 Field Replaceable Unit...

Page 18: ...About System Components 18 Oracle Server X6 2 Service Manual June 2016 FIGURE 3 Server Illustrated Parts Breakdown Callout Description 1 Storage drives 2 Front indicator module FIM 3 Disk backplane ...

Page 19: ...essor systems 17 PCIe risers 3 18 DIMMs Note A maximum of 12 DIMMs are supported in single processor systems and the DIMMs must be installed in sockets associated with processor 0 P0 No DIMM filler panels are required in vacant DIMM sockets associated with processors 0 P0 or 1 P1 19 Power supplies 20 Server chassis 21 Optional DVD drive Customer Replaceable Units The following table lists the cust...

Page 20: ...s the field replaceable units FRUs in the server and directs you to the replacement instructions FRU Description Replacement Instructions Processor and heatsink Carries out the instructions of the system Servicing Processors FRU on page 120 Oracle Storage 12 Gb s SAS PCIe RAID HBA card HDD and SSD storage drives controller Servicing the Internal HBA Card FRU on page 140 Oracle PCIe NVMe switch car...

Page 21: ...783 Eight disk backplane combo cable assembly that connects the Oracle PCIe NVMe switch card to the disk backplane See Servicing the NVMe Cables on page 177 Power cable 7064125 Connects the disk backplane to the motherboard Provides power from the motherboard to the disk backplane See Servicing the Power FIM Disk Backplane Data and DVD Cables on page 181 FIM cable 7064123 Ribbon cable that provide...

Page 22: ...About System Components 22 Oracle Server X6 2 Service Manual June 2016 Related Information Customer Replaceable Units on page 19 Illustrated Parts Breakdown on page 17 Servicing FRUs on page 119 ...

Page 23: ...mation about contacting Oracle support Getting Help on page 45 Troubleshooting Server Component Hardware Faults This section contains maintenance related information and procedures that you can use to troubleshoot and repair server hardware issues The following topics are covered Description Section Links Troubleshooting overview information and procedure Troubleshooting Server Hardware Faults on ...

Page 24: ...g the fault You can use Oracle ILOM to identify the failed component 2 Review the Oracle Server X6 2 Product Notes Once you have identified the hardware issue review the Oracle Server X6 2 Product Notes This document contains up to date information about the server including hardware related issues 3 Prepare the server for service using Oracle ILOM If you have determined that the hardware fault re...

Page 25: ...erver for service Note This procedure provides one basic approach to troubleshooting hardware faults It uses a combination of the Oracle ILOM web interface and the command line interface CLI This procedure can be performed using only the Oracle ILOM CLI interface For more information about the Oracle ILOM web interface and CLI refer to http www oracle com goto ilom docs Before You Begin Obtain the...

Page 26: ... the above example the Status screen shows that the Processor subsystem requires service This indicates that a hardware component within the subsystem is in a fault state 3 To identify the component click on Processors in the Status section The Oracle ILOM Processors screen appears The above example shows the Processors subsystem screen and indicates that CPU 0 has a fault 4 To get more informatio...

Page 27: ...ponent After servicing the component you might need to clear the fault in Oracle ILOM For more information refer the service procedures for the component Troubleshooting and Diagnostic Information The following table lists diagnostic and troubleshooting related procedures and references that can assist you with resolving server issues Description Link Diagnostic information for the x86 servers inc...

Page 28: ...ocessor SP OK Fault LED indicator and the Power OK LED indicator When AC power is connected to the server the server boots into standby power mode 1 The SP OK LED blinks slowly 0 5 seconds on 0 5 seconds off while the SP is starting and the main Power OK LED remains off until the SP is ready 2 After a few minutes the main Power OK LED flashes the standby blink pattern 0 1 seconds on 2 9 seconds of...

Page 29: ...tor lights for a replaceable component on the server Note When this indicator is lit a system console message might appear that includes a recommended service action Power OK Green Indicates the operational state of the chassis OFF AC power is not present or the Oracle ILOM boot is not complete Standby BLINK Standby power is on but the chassis power is off and the Oracle ILOM SP is running SLOW BL...

Page 30: ...ired STEADY ON The system is experiencing an overtemperature warning condition Note This is a warning indication not a fatal overtemperature Failure to correct this might result in the system overheating and shutting down unexpectedly Server Fan Status Indicators Each fan module has one status indicator LED The LEDs are located on the chassis mid wall adjacent to and aligned with the fan modules a...

Page 31: ...ower supply These indicators are visible from the rear of the server TABLE 4 Server Power Supply Indicators Status Indicator Name Icon Color State and Meaning AC OK DC OK Green OFF No AC power is present SLOW BLINK Normal operation Input power is within specification DC output voltage is not enabled STEADY ON Normal operation Input AC power and DC output voltage are within specification Service Re...

Page 32: ...llowing status indicators LEDs DIMM Fault Status Indicators Each of the 24 DIMM sockets on the motherboard has an amber fault status indicator LED associated with it If Oracle ILOM determines that a DIMM is faulty pressing the Fault Remind button on the motherboard signals the service processor to light the fault LED associated with the failed DIMM For more information on DIMM fault status indicat...

Page 33: ...re that all AC power be removed from the power supplies prior to removing the server top cover Troubleshooting System Cooling Issues Maintaining the proper internal operating temperature of the server is crucial to the health of the server To prevent server shutdown and damage to components address over temperature and hardware related issues as soon as they occur If your server has a temperature ...

Page 34: ... Areas Compromised To function properly the server has cooling areas that are maintained by an air baffle component filler panels and the server top cover These server components need to be in place for the server to function as a sealed system If internal cooling areas are compromised the server cooling system which relies on the movement of cool air through the server cannot function properly an...

Page 35: ...er on the cause of the problem might be AC Power Connection on page 35 Power Supplies PSUs on page 36 Top Cover on page 37 AC Power Connection The AC power cords are the direct connection between the server power supplies and the power sources The server power supplies need separate stable AC circuits Insufficient voltage levels or fluctuations in power can cause server power problems The power su...

Page 36: ...on Check that the AC cables are connected to both PSUs Check that the PSUs are operational the PSU indicator panel should have a lit green AC OK indicator Ensure that the PSU is properly installed A PSU that is not fully engaged with its internal connector does not have power applied and does not have a lit green AC OK indicator Prevention When a power supply fails replace it immediately To ensure...

Page 37: ...nsure that the server top cover is in place and sits flat and snug on top of the server Ensure that the interlock switch components have not been damaged removed or misaligned For instructions for properly installing the top cover see Install the Server Top Cover on page 188 Prevention After removing the top cover take care that it does not get bent When servicing the server take care that the int...

Page 38: ... table summarizes the diagnostic tools that you can use when troubleshooting or monitoring your server Diagnostic Tool Type What It Does Accessibility Remote Capability Oracle ILOM SP firmware Monitors environmental condition and component functionality sensors generates alerts performs fault isolation and provides remote access Can function on either standby power mode or full power mode and is n...

Page 39: ...s various kinds of system information Requires operating system Local and over network Oracle Linux commands Operating system software Displays various kinds of system information Requires operating system Local and over network Oracle VTS Diagnostic tool standalone software Exercises and stresses the system running tests in parallel Requires Oracle Solaris operating system Install Oracle VTS soft...

Page 40: ...onnect an Ethernet cable to the Gigabit Ethernet NET connectors as needed for OS support See Rear Panel Connector Locations on page 40 2 To connect to the service processor s Oracle ILOM over the network connect an Ethernet cable to the Ethernet port labeled NET MGT See Rear Panel Connector Locations on page 40 3 To access the Oracle ILOM command line interface CLI locally using the management por...

Page 41: ...rt is configured by default to use Dynamic Host Configuration Protocol DHCP The service processor NET MGT port uses an RJ 45 cable for a 10 100 1000BASE T connection 3 Serial management port SER MGT The service processor SER MGT port uses an RJ 45 cable and is the default connection to the Oracle ILOM service processor This port supports local connections to the server and only recognizes Oracle I...

Page 42: ...access to the server port on your server see the Oracle Integrated Lights Out Manager ILOM 3 2 Documentation Library at http www oracle com goto ilom docs You can assign serial port output using either the Oracle ILOM web interface or the command line interface CLI For instructions see the following procedures Assign Serial Port Output Using the CLI on page 42 Assign Serial Port Output Using the W...

Page 43: ... to take effect 6 Connect a serial host to the server Server Operating System Names for the NVMe Storage Drives If the optional NVMe storage drives are installed in the server s front panel they are labeled NVMe0 NVMe1 NVMe2 and NVMe3 However the server operating systems assign these storage drives different names For the corresponding names assigned by the operating systems see the following tabl...

Page 44: ...hows BIOS physical and operating system logical naming conventions used for each interface These naming conventions might vary depending on conventions of your operating system and which devices are installed in the server Port BIOS Solaris Linux Windows example default names see note below Net 0 3A00 ixgbe 0 eth0 Ethernet Net 1 3A01 ixgbe 1 eth1 Ethernet 2 Net 2 8200 ixgbe 2 eth2 Ethernet 3 Net 3...

Page 45: ...5 Locating the Chassis Serial Number on page 46 Contacting Support If the troubleshooting procedures in this chapter fail to solve your problem use the following table to collect information that you might need to communicate to support personnel System Configuration Information Needed Your Information Service contract number System model Operating environment System serial number Peripherals atta...

Page 46: ...he following resources or methods to locate your server serial number The serial number is located on the Radio frequency identification RFID label on the left side of the front panel bezel next to the general status indicators For illustrations of the server front panel see About Controls and Connectors on page 14 The serial number is recorded on the system information label that is attached to t...

Page 47: ...ithin the server power down the server and prepare for servicing Preparing the Server for Component Replacement on page 50 Related Information Returning the Server to Operation on page 187 Safety Precautions For your protection observe the following safety precautions when setting up your equipment Follow all standard cautions warnings and instructions marked on the equipment and described in the ...

Page 48: ...static Discharge Safety on page 48 Electrostatic Discharge Safety Devices that are sensitive to electrostatic discharge ESD such as the motherboard PCIe cards drives processors and memory cards require special handling Caution The boards and drives contain electronic components that are extremely sensitive to static electricity Ordinary amounts of static electricity from clothing or the work envir...

Page 49: ...Component Replacement on page 50 Returning the Server to Operation on page 187 FRU Key Identity Properties KIP Automated Update Oracle ILOM includes a key identity properties KIP auto update feature that ensures product information that is used for service entitlement and warranty coverage is accurately maintained by the server at all times including during hardware replacement activities The KIP ...

Page 50: ... Related Information Preparing for Service on page 47 Servicing CRUs That Do Not Require Server Power Off on page 63 Servicing CRUs That Require Server Power Off on page 85 Servicing FRUs on page 119 Preparing the Server for Component Replacement Before you can remove and install components that are inside the server you must perform the procedures in the following tasks Note When replacing the st...

Page 51: ...sing the Power Button on page 53 If the server is not responding or you must shut down the server quickly perform an immediate shutdown Use the Power Button for Immediate Shutdown on page 54 Use the Oracle ILOM CLI for Immediate Shutdown on page 54 Use the Oracle ILOM Web Interface for Immediate Shutdown on page 55 Related Information Power On the Server on page 194 Power Down Server Gracefully Us...

Page 52: ...tion Power Down Server Gracefully Using the Oracle ILOM Web Interface on page 52 Power Down Server Gracefully Using the Power Button on page 53 Use the Power Button for Immediate Shutdown on page 54 Power On the Server on page 194 Power Down Server Gracefully Using the Oracle ILOM Web Interface 1 Log in to the server as superuser or equivalent Depending on the nature of the problem you might want ...

Page 53: ...anel This action causes ACPI enabled operating systems to perform an orderly shutdown of the operating system Servers not running ACPI enabled operating systems shut down to standby power mode immediately When main power is off the Power OK LED on the front panel will begin flashing indicating that the server is in standby power mode See Server System Level Status Indicators on page 29 2 Disconnec...

Page 54: ...ssor remote management subsystem and power supply fans To completely power off the server you must disconnect the power cords from the power supplies Related Information Table 1 Power Down Server Gracefully Using the Oracle ILOM CLI on page 51 Power Down Server Gracefully Using the Oracle ILOM Web Interface on page 52 Power Down Server Gracefully Using the Power Button on page 53 Use the Oracle IL...

Page 55: ...nly use this procedure to power down the server after attempting the graceful power down procedure 1 Log in to the Oracle ILOM web interface using an Administrator account The Oracle ILOM web interface System Information page appears 2 In the left pane click Host Management Power Control and select Immediate Power Off in the Select Action list 3 Click Save then click OK The server powers down imme...

Page 56: ...erver 4 Disconnect all data cables from the rear of the server Related Information Server Back Panel View on page 16 Powering Down the Server on page 51 Remove the Server From the Rack on page 58 Reconnect Data Cables and Power Cords on page 193 Extend the Server to the Maintenance Position The following components can be serviced with the server in the maintenance position Storage drives Fan modu...

Page 57: ...or instructions for stabilizing the rack see the Stabilize the Rack for Installation in Oracle Server X6 2 Installation Guide 2 Verify that no cables will be damaged or will interfere when the server is extended Although the cable management arm CMA that is supplied with the server is hinged to accommodate extending the server you should ensure that all cables and cords are capable of extending 3 ...

Page 58: ...or removing the CMA see the Remove the Cable Management Arm in Oracle Server X6 2 Installation Guide 3 Extend the server to the maintenance position For instructions see Extend the Server to the Maintenance Position on page 56 4 From the front of the server pull the green slide rail release tabs toward the front of the server and pull the server out of the rack until it is free of the rack rails A...

Page 59: ...ts attach an antistatic strap to your wrist and then to a metal area on the chassis Then disconnect the power cords from the server Note An antistatic wrist strap is not included in the Ship Kit for the server However antistatic wrist straps are still included with options and components Related Information Remove Antistatic Measures on page 190 Open the Server Fan Door Servicing the server fan mo...

Page 60: ... If the top cover is removed without first removing the AC power cords the server host shuts down immediately and an event is logged to indicate that the chassis top cover has been removed 1 Ensure that the AC power cords are disconnected from the server power supplies 2 Open the server fan door See Open the Server Fan Door on page 59 3 To open the server top cover press and hold down the top cove...

Page 61: ...op Cover Preparing for Service 61 FIGURE 6 Removing the Server Top Cover 4 Lift the cover off the chassis and set it aside 2 Related Information Take Antistatic Measures on page 59 Install the Server Top Cover on page 188 ...

Page 62: ...62 Oracle Server X6 2 Service Manual June 2016 ...

Page 63: ...plies Servicing Power Supplies CRU on page 80 Related Information Servicing CRUs That Require Server Power Off on page 85 Servicing FRUs on page 119 Servicing Storage Drives CRU These sections describe how to remove and install storage drives Storage Drives Hot Plug Conditions on page 64 Storage Drive Failure and RAID on page 64 Storage Drive Locations and Numbering on page 64 Storage Drive Status...

Page 64: ...s is true then you must shut down the system before you replace the drive See Powering Down the Server on page 51 Note Replacing a storage drive does not require removing the server from a rack Storage Drive Failure and RAID A single storage drive failure does not cause a data failure if the storage drives are configured as a mirrored RAID 1 volume optional The storage drive can be removed and whe...

Page 65: ...ssigned to the NVMe storage drives by server operating systems see Server Operating System Names for the NVMe Storage Drives on page 43 Storage Drive Status Indicators The following figure and table describe the storage drive status indicators LEDs FIGURE 8 Storage Drive Status Indicators TABLE 10 Storage Drive Status Indicators Legend LED Color State and Meaning 1 OK to Remove Blue STEADY ON The ...

Page 66: ...ge drive locations see Storage Drive Locations and Numbering on page 64 3 If you are removing a HDD or SSD storage drive type the operating system commands required to stop the server from using the drive otherwise proceed to the next step Note The blue OK to Remove LED on the storage drive might not light as support for this varies depending on the operating system in use The exact commands requi...

Page 67: ...ve continue to Install a HDD or SSD Storage Drive on page 68 If you are not replacing the drive install a filler panel in the empty drive slot to maintain proper airflow and perform administrative tasks to configure the server to operate without the drive For information on how to install a storage drive filler panel see Removing and Installing Server Filler Panels on page 187 Related Information ...

Page 68: ... the drive that was removed 4 Slide the drive into the slot until the drive is fully seated 1 FIGURE 10 Installing a Storage Drive 5 Close the drive latch to lock the drive in place 2 6 Perform administrative tasks to reconfigure the drive The procedures that you perform at this point depend on how your data is configured You might need to partition the drive create file systems load data from bac...

Page 69: ...e on page 70 Install an NVMe Storage Drive in the Server on page 70 Power On an NVMe Storage Drive and Attach a Device Driver on page 71 Unmount an NVMe Storage Drive 1 Log in to Oracle Solaris that is running on the server 2 To find the NVMe drive slot number type hotplug list lc This command produces output similar to the following for each of the NVMe drives installed in the server hotplug list...

Page 70: ...of the drive slot 5 Consider your next steps If you are replacing the drive continue to Install an NVMe Storage Drive in the Server on page 70 If you are not replacing the drive install a filler panel in the empty drive slot to maintain proper airflow and perform administrative tasks to configure the server to operate without the drive For information on how to install a storage drive filler panel...

Page 71: ...s removed 4 Slide the drive into the slot until the drive is fully seated See frame 1 in Figure 10 5 Close the drive latch to lock the drive in place See frame 2 in Figure 10 Power On an NVMe Storage Drive and Attach a Device Driver 1 Type hotplug enable pcie13 2 Type hotplug list lc The following status is displayed for the NVMe storage drive hotplug list lc Connection State Description Path pcie...

Page 72: ...Verify Operation of an NVMe Storage Drive on page 75 Unmount an NVMe Storage Drive 1 Log in to Oracle Linux that is running on the server 2 Remove the NVMe storage device path a To find the PCIe addresses Bus Device Function type find sys devices egrep nvme 0 9 0 9 This commands returns output similar to the following sys devices pci0000 80 0000 80 03 0 0000 b0 00 0 0000 b1 04 0 0000 b2 00 0 misc ...

Page 73: ...unmount any file systems that are mounted on the device ii Remove the device from any multiple device md and Logical Volume Manager LVM volume using it If the device is a member of an LVM Volume group then it may be necessary to move data off the device using the pvmove command then use the vgreduce command to remove the physical volume and optionally pvremove to remove the LVM meta data from the ...

Page 74: ...ow to install a storage drive filler panel see Removing and Installing Server Filler Panels on page 187 Verify Removal of an NVMe Storage Drive 1 To check the NVMe drive enumeration to verify that the NVMe drive has been removed type lspci nnd 0953 2 View the command output and verify that the entry for the slot number that was disabled no longer appears This commands returns output similar to the...

Page 75: ...wer On an NVMe Storage Drive and Attach a Device Driver To power on the slot and attach the device driver type echo 1 sys bus pci slots slot power Where slot is the slot number for the NVMe storage drive Verify Operation of an NVMe Storage Drive To verify that an NVMe drive is operating properly do one of the following Check the var log messages log file Type ls l dev nvme Related Information Stor...

Page 76: ...cle ILOM Even if only one fan motor is faulted within the fan module the Oracle ILOM service processor detects that four fan motors have failed to spin while the fan module is removed If the fan module is not replaced within 20 seconds of removal Oracle ILOM will take the protective action to shut down the system to prevent thermal damage to the system 1 Extend the server to the maintenance positi...

Page 77: ...ber Off The fan module is correctly installed and operating within specification Amber The fan module is faulty The front TOP FAN LED and the front and rear panel Service Required LEDs are also lit if the system detects a fan module fault 4 Remove the faulty fan module a Using your thumb and forefinger grasp the fan module in the finger recesses located in the plastic between the fans ...

Page 78: ...he motherboard connectors FIGURE 12 Removing a Fan Module 5 Set the fan module aside Caution Do not service any other components in the fan compartment unless the system is shut down and the power cords are removed 6 Consider your next step If you removed the fan assembly as part of another procedure return to that procedure Otherwise continue to Install a Fan Module on page 79 Related Information...

Page 79: ...FIGURE 13 Installing a Fan Module 3 Press down on the fan module and apply firm pressure to fully seat the fan module 4 Verify that the fan module status indicator LED that aligns with the replaced fan module is extinguished 5 Close the fan door 6 Verify that the Top Fan Fault LED on the front of the server and the Service Required LEDs on the front and rear of the server are extinguished See Trou...

Page 80: ...worst case power consumption of the system could exceed the capacity of a single PSU The PSUs provide an over subscription mode which allows the system to operate with fault tolerance even with modest excursions beyond the rated capacity of a single PSU This over subscription support is accomplished using hardware signaling between the PSU and motherboard circuitry which can force the system to th...

Page 81: ...Color State Meaning 1 AC OK DC OK Green OFF No AC power is present SLOW BLINK Normal operating Input power is within specification DC output voltage is not enabled STEADY ON Normal operation Input AC power and DC output voltage are within specification 2 Service Required Amber OFF Normal operation no service required STEADY ON The power supply PS has detected a PS fan failure PS overtemperature PS...

Page 82: ...is powered on You can remove a power supply while the fans are still spinning 2 Gain access to the rear of the server where the faulty power supply is located 3 If the cable management arm CMA is installed disconnect both CMA left side connectors and move the CMA out of the way For instructions for disconnecting the CMA left side connectors refer to Remove the Cable Management Arm in Oracle Server...

Page 83: ...airflow 7 Continue to Install a Power Supply on page 83 Related Information Server Back Panel View on page 16 Power Supply Status Indicators on page 81 Install a Power Supply on page 83 Install a Power Supply Caution Always replace a faulty power supply with a power supply of the same type model 1 Remove the replacement power supply from its packaging and place it on an antistatic mat 2 Align the ...

Page 84: ...ies KIP data to the new power supply For instructions on resetting the SP see the Oracle Integrated Lights Out Manager ILOM 3 2 Configuration and Maintenance Guide at http www oracle com goto ilom docs Power Supply 1 does not contain KIP data and therefore does not require an SP reset after replacement 6 If you disconnected the two CMA left side connectors reconnect the connectors For instructions...

Page 85: ...vice the battery Servicing the Battery CRU on page 116 Related Information Servicing CRUs That Do Not Require Server Power Off on page 63 Servicing FRUs on page 119 Servicing the DIMMs CRU The Oracle Server X6 2 supports a variety of DDR4 DIMM configurations that can include quad rank QR and dual rank DR DDR4 DIMMs Caution These procedures require that you handle components that are sensitive to e...

Page 86: ...Inconsistencies Between DIMM Fault Indicators and the BIOS Isolation of Failed DIMMs on page 92 Using the Server Fault Remind Button on page 92 Identify and Remove the Failed DIMM on page 93 Install a DIMM on page 95 Related Information Servicing CRUs That Do Not Require Server Power Off on page 63 Servicing FRUs on page 119 DIMM and Processor Physical Layout The physical layout of the DIMMs and p...

Page 87: ...d Information DIMM Population Scenarios on page 87 DIMM Population Rules on page 88 Populating DIMMs for Optimal System Performance on page 88 DIMM Operating Speeds on page 92 DIMM Rank Classification Labels on page 92 Inconsistencies Between DIMM Fault Indicators and the BIOS Isolation of Failed DIMMs on page 92 DIMM Population Scenarios There are two scenarios in which you are required to popula...

Page 88: ...e server has 64 GB LRDIMMs installed you cannot install 16 GB or 32 GB RDIMMs and vice versa Within a memory channel DIMMs must be populated in the black sockets first then in the black sockets with white tabs and then in the white sockets If you are installing a mix of RDIMM sizes for example 16 GB and 32 GB install the larger size RDIMMs first then the smaller RDIMMs until all of the RDIMMs are ...

Page 89: ...0 Starting with socket P0 D0 Fill the black sockets first then the black sockets with white tabs and then the white sockets as shown in the following figure Notice that the order of populating the black sockets is Ch C first Ch A second Ch D third and Ch B last Repeat the same population sequence for the black sockets with white tabs and finally for the white sockets FIGURE 17 DIMM Population Orde...

Page 90: ...Eleventh D5 Ch D Twelfth D6 Ch B Populating DIMMs in Dual Processor Systems for Optimal System Performance In dual processor systems populate DIMMs into DIMM sockets starting with processor 0 P0 D0 then alternating between sockets associated with processor 0 P0 and matching sockets for processor 1 P1 Fill the black sockets first then the black sockets with white tabs and then the white sockets as ...

Page 91: ...or 1 P1 Memory Channels Fill the black sockets first alternating between processor 0 and processor 1 First Fill D0 first Then fill D0 Ch C Second Fill D11 first Then fill D11 Ch A Third Fill D3 first Then fill D3 Ch D Fourth Fill D8 first Then fill D8 Ch B Then fill the black sockets with white tabs alternating between processor 0 and processor 1 Fifth Fill D1 first Then fill D1 Ch C Sixth Fill D1...

Page 92: ...4 DIMM Rank Classifications Labels Rank Classification Label Quad rank LRDIMM 4Rx4 Dual rank RDIMM 2Rx4 Inconsistencies Between DIMM Fault Indicators and the BIOS Isolation of Failed DIMMs When a single DIMM is marked as failed by Oracle ILOM for example fault memory intel dimm training failed is listed in the service processor Event Log BIOS might disable the entire memory channel that contains t...

Page 93: ...can happen if the Fault Remind button is pressed for several minutes with fault LEDs lit or if power has been removed from the server for more than 15 minutes The following figure shows the location of the Fault Remind button FIGURE 19 Fault Remind Button Location Identify and Remove the Failed DIMM 1 Prepare the server for service a Power off the server and disconnect the power cords from the pow...

Page 94: ...tify and note the location of the failed DDR4 DIMM by pressing the Fault Remind button on the motherboard see Using the Server Fault Remind Button on page 92 Failed DDR4 DIMMs are identified with a corresponding amber LED on the motherboard If the DIMM fault LED is off the DIMM is operating properly If the DIMM fault LED is on amber then the DIMM has failed and should be replaced 3 To remove the f...

Page 95: ...ket 4 Replace each failed DIMM with either another DIMM of the same rank size quad rank or dual rank or leave the socket empty For DIMM replacement instructions see Install a DIMM on page 95 Related Information DIMM and Processor Physical Layout on page 86 Install a DIMM on page 95 Install a DIMM 1 Unpack the replacement DDR4 DIMM and place it on an antistatic mat 2 Ensure that the replacement DDR...

Page 96: ...ket If the notch is not aligned damage to the DIMM might occur Note If you want to move a faulty DDR4 DIMM to a new DIMM socket or re seat a DIMM for troubleshooting you must first manually clear the associated DIMM fault Otherwise the DIMM fault might follow the DIMM to the new socket location or reoccur on the current socket location possibly causing a false DIMM fault condition 4 Repeat Step 3 ...

Page 97: ...cation Labels on page 92 Identify and Remove the Failed DIMM on page 93 Servicing PCIe Risers CRU PCIe cards in all slots are installed on vertical risers You must remove the relevant riser to remove and replace a PCIe card You must remove all three PCIe risers when replacing the motherboard Caution These procedures require that you handle components that are sensitive to electrostatic discharge T...

Page 98: ...is different than the risers in PCIe slots 1 and 2 The riser for slots 3 and 4 supports two cards a standard PCIe card and the internal HBA card Do not attempt to install the PCIe riser for PCIe slots 3 and 4 in slots 1 or 2 and vice versa FIGURE 21 PCIe Riser Locations Figure Legend 1 PCIe riser and installed PCIe card or the optional Oracle PCIe NVMe switch card in slot 1 This slot is nonfunctio...

Page 99: ...nel For instructions for servicing the PCIe NVMe switch card see Servicing the Oracle PCIe NVMe Switch Card FRU on page 137 Note PCIe slot 1 is nonfunctional in single processor systems 1 Prepare the server for service a Power off the server and disconnect the power cords from the power supplies See Powering Down the Server on page 51 b Extend the server to the maintenance position See Extend the ...

Page 100: ...talled then lift the latch to release the PCIe slot filler panel 5 Lift the green tabbed riser release lever on the PCIe riser with one hand and use your other hand to remove the riser from the motherboard connector 2 3 6 If there is a PCIe card installed in the riser place the riser on an antistatic mat and note the slot in which the PCIe riser was originally installed otherwise set the PCIe rise...

Page 101: ...s 1 and 2 Do not attempt to install a PCIe riser from slot 1 or 2 into PCIe slots 3 and 4 and vice versa 1 Determine into which slot you will install the PCIe riser 2 Retrieve the PCIe riser and any PCIe cards attached to it 3 Raise the green tabbed release lever on the PCIe riser to the open up position 1 4 Gently press the riser into the motherboard connector until it is seated and then press th...

Page 102: ...osition on page 191 c Reconnect the power cords to the server power supplies and reconnect the data cables See Reconnect Data Cables and Power Cords on page 193 d Power on the server See Power On the Server on page 194 Verify that the Power OK status indicator is steady on 8 Use Oracle ILOM to clear the server PCIe riser fault If the PCIe riser fault message in Oracle ILOM is not cleared under Ope...

Page 103: ...ternal HBA card see Servicing the Internal HBA Card FRU on page 140 1 Prepare the server for service a Power off the server and disconnect the power cords from the power supplies See Powering Down the Server on page 51 b Extend the server to the maintenance position See Extend the Server to the Maintenance Position on page 56 c Attach an antistatic wrist strap to your wrist and then to a metal are...

Page 104: ... filler panel 4 To release the riser from the motherboard connector lift the green tabbed release lever on the PCIe riser to the open position 2 5 Slide the plastic PCIe card retainer which is mounted on the side of the chassis toward the front of the server to release the card s installed in the riser 3 6 Grasp the riser with both hands and remove it from the server 4 7 Disconnect the SAS storage...

Page 105: ...PCIe Slots 3 and 4 on page 105 Install the PCIe Riser Into PCIe Slots 3 and 4 Note The PCIe riser in slots 3 and 4 is different than the PCIe risers in slots 1 and 2 Do not attempt to install a PCIe riser from slot 1 or 2 into PCIe slots 3 and 4 and vice versa 1 Retrieve the PCIe riser and any PCIe cards attached to it 2 Reconnect the SAS cables to the internal host bus adapter card ...

Page 106: ...he green tabbed release lever on the PCIe riser to the open up position and then gently press the riser into the motherboard connector until it is seated 1 5 Ensure that the rear bracket on the internal HBA card in PCIe slot 4 is connected to the slot in the server chassis side wall If the bracket is not connected remove the riser and reposition it so that the rear bracket connects to the side wal...

Page 107: ... rack position See Return the Server to the Normal Rack Position on page 191 c Reconnect the power cord to the server power supply and the data cables See Reconnect Data Cables and Power Cords on page 193 d Power on the server See Power On the Server on page 194 Verify that the Power OK status indicator is steady on 11 Use Oracle ILOM to clear the server PCIe riser fault If the PCIe riser fault me...

Page 108: ...ty can cause the components to fail To avoid damage ensure that you follow antistatic practices as described in Take Antistatic Measures on page 59 Caution Ensure that all power is removed from the server before removing or installing PCIe cards You must disconnect all power cables from the system before performing these procedures Note For a complete list of supported PCIe cards refer to the Orac...

Page 109: ...lectrical 2 Low profile cards only PCIe 1 0 PCIe 2 0 PCIe 3 0 x16 mechanical x16 electrical 3 and 4 Low profile cards only PCIe 1 0 PCIe 2 0 PCIe 3 0 x8 mechanical x8 electrical Related Information Server Back Panel View on page 16 Remove a PCIe Card From PCIe Slot 1 or 2 Note Either a standard PCIe card CRU or the optional PCIe NVMe switch card can be installed in the PCIe riser for slot 1 The PC...

Page 110: ...card or a PCIe filler panel Installing PCIe filler panels in vacant PCIe slots helps reduce the level of electromagnetic interference EMI emitted by the server For instructions for installing a PCIe filler panel see Removing and Installing Server Filler Panels on page 187 Install a PCIe Card in PCIe Slot 1 or 2 Note PCIe slot 1 is nonfunctional in single processor systems 1 Retrieve the PCIe card ...

Page 111: ...he PCIe riser into the server For instructions see Install a PCIe Riser Into PCIe Slot 1 or 2 on page 101 Remove a PCIe Card From PCIe Slot 3 1 Remove the PCIe riser from the server For instructions see Remove the PCIe Riser From PCIe Slots 3 and 4 on page 103 2 Remove the PCIe card from the riser a Hold the riser in one hand and use your other hand to carefully remove the PCIe card connector from...

Page 112: ...place it with another PCIe card or a PCIe filler panel Installing PCIe filler panels in vacant PCIe slots helps reduce the level of electromagnetic interference EMI emitted by the server For instructions for installing PCIe filler panels see Removing and Installing Server Filler Panels on page 187 Install a PCIe Card in PCIe Slot 3 1 Retrieve the PCIe card and riser you want to install 2 Insert th...

Page 113: ...sitive to static discharge This sensitivity can cause the component to fail To avoid damage ensure that you follow antistatic practices as described in Take Antistatic Measures on page 59 Caution Ensure that all power is removed from the server before removing or installing a flash drive or damage to the drive might occur You must disconnect all power cables from the system before performing these...

Page 114: ...ver and disconnect the power cords from the power supplies See Powering Down the Server on page 51 b Extend the server to the maintenance position See Extend the Server to the Maintenance Position on page 56 c Attach an antistatic wrist strap to your wrist and then to a metal area on the chassis See Take Antistatic Measures on page 59 d Remove the server top cover See Remove the Server Top Cover o...

Page 115: ...ash Drive 1 Unpack the replacement USB flash drive 2 Insert the USB flash drive into the internal USB port see Figure 22 3 Return the server to operation a Install the server top cover See Install the Server Top Cover on page 188 b Close the fan door c Return the server to the normal rack position See Return the Server to the Normal Rack Position on page 191 ...

Page 116: ...not connected to a network replace the battery Caution Ensure that all power is removed from the server before removing or installing the battery You must disconnect the power cables from the system before performing these procedures This section covers the following procedures Remove the Battery on page 116 Install the Battery on page 117 Related Information Extend the Server to the Maintenance P...

Page 117: ...e riser and card to gain access to the battery See Remove a PCIe Riser From PCIe Slot 1 or 2 on page 99 3 To dislodge the battery from the retainer place your finger under the side of the battery nearest the rear of the server 1 and gently lift the battery up and out of the retainer 2 FIGURE 23 Removing the Battery Install the Battery 1 Unpack the replacement battery 2 Press the new battery into t...

Page 118: ...atus indicator is steady on Note If the service processor is configured to synchronize with a network time server using the Network Time Protocol NTP the Oracle ILOM clock will be reset as soon as the server is powered on and connected to the network otherwise proceed to the next step 5 If the service processor is not configured to use NTP do one of the following Reset the Oracle ILOM clock using ...

Page 119: ...al host bus adapter HBA Servicing the Internal HBA Card FRU on page 140 Service the disk backplane Servicing the Disk Backplane FRU on page 150 Service the DVD drive Servicing the DVD Drive FRU on page 156 Service the front indicator module Servicing the Front Indicator Module FRU on page 158 Service the motherboard Servicing the Motherboard FRU on page 162 Service the internal HBA cable assembly ...

Page 120: ... the processor socket 1 P1 To protect the delicate processor socket pins the cover that comes on the motherboard from manufacturing is left in place These procedure are provided in this section Remove a Processor on page 125 Install a Processor on page 131 Related Information Disconnect Cables From the Server on page 56 Take Antistatic Measures on page 59 Remove the Server From the Rack on page 58...

Page 121: ...mation For instructions refer to the Oracle Integrated Lights Out Manager ILOM 3 2 Documentation Library at http www oracle com goto ilom docs Or Visually check the size of the processor installed in the server This requires that you remove the processor heatsink and open the independent loading mechanism ILM assembly that holds the processor in the socket For instructions see Remove a Processor o...

Page 122: ...n a Motherboard Processor Socket Figure Legend 1 Processor alignment brackets 2 Processor left and right edges are within alignment bracket boundaries Figure 25 shows a larger processor installed Notice that the right and left edges of the processor extend beyond the boundaries of the processor alignment brackets ...

Page 123: ... bracket boundaries After you have determined the size of the processor installed in your server select the correct processor removal and replacement tool Like the processors the processor removal and replacement tool also comes in two sizes The tool for the smaller processors processors with 10 or fewer cores is color coded green The tool for the larger processors 12 or more cores is color coded ...

Page 124: ...ise the processor or the processor socket might be damaged The following table provides an easy reference for matching a processor by number of cores or model number to the correct processor removal and replacement tool The processor removal and replacement tool part number is also recorded on the tool TABLE 15 Processor Model Numbers by Number of Cores Processor Cores Processor Model Number Proce...

Page 125: ...e chassis See Take Antistatic Measures on page 59 d Remove the server top cover Remove the Server Top Cover on page 60 2 If the server has a DVD drive disconnect the DVD cable from the disk backplane carefully pull it through the chassis mid wall and move it to the side away from of the air baffle For instructions for disconnecting the DVD cable from the disk backplane see Servicing the Power FIM ...

Page 126: ...rge This can happen if the Fault Remind button is pressed for a long time with fault LEDs lit or if power has been removed from the server for more than 15 minutes The processor fault LEDs are located next to the processor s If the processor fault LED is off then the processor is operating properly If the processor fault LED is on amber then the processor is failed and should be replaced FIGURE 27...

Page 127: ... of the processor gently twist the heatsink left and right while pulling upward and then lift off the heatsink and place it upside down on a flat surface 1 A thin layer of thermal grease separates the heatsink and the processor This grease acts as an adhesive Note Do not allow the thermal grease to contaminate the work space or other components 8 Use an alcohol pad to clean the thermal grease from...

Page 128: ...ion 3 Caution Whenever you remove a processor you should replace it with another processor and reinstall the processor heatsink otherwise the server might overheat due to improper airflow For instructions for installing a processor see Install a Processor on page 131 12 To remove the processor from the processor socket acquire the processor removal and replacement tool and perform the following st...

Page 129: ...rcuit side down the installed orientation into an antistatic container 6 h Carefully clean the thermal grease off the top of the processor Caution The following figure shows the green color coded processor removal and replacement tool If you are removing a processor with 12 or more cores you must use the pink color coded processor removal and replacement tool otherwise damage might result to the p...

Page 130: ...Remove a Processor 130 Oracle Server X6 2 Service Manual June 2016 FIGURE 28 Removing a Processor Related Information Install a Processor on page 131 ...

Page 131: ...lacement processor is identical to the failed processor that was removed For a description of the processors supported by the server see Table 15 4 Ensure that the two processor ILM assembly levers and the ILM assembly load plate are in the fully open position For instructions for opening the processor ILM assembly levers and the load plate see Step 9 through Step 11 of Remove a Processor on page ...

Page 132: ...of the processor socket when viewing the server from the front and lower the tool into the processor socket e Press the center button in the tool down to release the processor so that it is placed in the socket 5 f Remove the processor removal and replacement tool 6 Caution The following figure shows the green color coded processor removal and replacement tool If you are installing a processor wit...

Page 133: ...nstall a Processor Servicing FRUs 133 FIGURE 29 Installing a Processor 6 Visually check the alignment of the processor in the socket When properly aligned the processor sits flat in the processor socket ...

Page 134: ...ate sits flat around the periphery of the processor 8 Engage the socket release lever on the left side of the socket viewing the server from the front by rotating it downward and slipping it under the catch 2 9 Engage the socket release lever on the right side of the socket viewing the server from the front by rotating it downward and slipping it under the catch 2 10 Use the syringe supplied with ...

Page 135: ...Phillips screwdriver alternately one half turn until fully seated 15 Install the air baffle 16 If the server has a switch card installed place the switch card cables in the cable troughs in the metal motherboard removal bracket and the air baffle carefully pull the cables through the chassis mid wall and reconnect the cables to the disk backplane For instructions for reconnecting the switch card c...

Page 136: ...nd type the following command to list all known faults on the system show SP faultmgmt The server lists all known faults for example show SP faultmgmt Targets 0 SYS MB P0 Properties Commands cd show Alternatively to list all known faults in the server log into the Oracle Solaris OS and issue the fmadm faulty command or log into the Oracle ILOM service processor from the Oracle ILOM Fault Managemen...

Page 137: ...h card should be serviced only by authorized Oracle Services personnel The following sections provide information to assist you when servicing the Oracle PCIe NVMe switch card Caution These procedures require that you handle components that are sensitive to electrostatic discharge This sensitivity can cause the components to fail To avoid damage ensure that you follow antistatic practices as descr...

Page 138: ...he maintenance position See Extend the Server to the Maintenance Position on page 56 c Attach an antistatic wrist strap to your wrist and then to a metal area on the chassis See Take Antistatic Measures on page 59 d Open the server fan door to gain access to the disk backplanes See Open the Server Fan Door on page 59 e Remove fan module FM0 the left most fan viewing the server from the front See R...

Page 139: ...ch card to the disk backplane 4 Hold the riser in one hand and use your other hand to carefully remove the card from the riser Install the Oracle PCIe NVMe Switch Card in PCIe Slot 1 1 Unpack the replacement PCIe NVMe switch card and retrieve the slot 1 PCIe riser 2 Reconnect the cables from the disk backplane to the switch card ...

Page 140: ...Ie Riser Into PCIe Slot 1 or 2 on page 101 5 Return the server to operation a Close the server fan door b Return the server to the normal rack position See Return the Server to the Normal Rack Position on page 191 c Reconnect the data cables to the server and reconnect the power cords to the server power supplies See Reconnect Data Cables and Power Cords on page 193 d Power on the server See Power...

Page 141: ...t 4 on page 145 Related Information Disconnect Cables From the Server on page 56 Take Antistatic Measures on page 59 Extend the Server to the Maintenance Position on page 56 Open the Server Fan Door on page 59 Remove the Internal HBA Card From PCIe Slot 4 1 Prepare the server for service a Power off the server and disconnect the power cords from the power supplies See Powering Down the Server on p...

Page 142: ...r from slots 3 and 4 For instructions see Remove the PCIe Riser From PCIe Slots 3 and 4 on page 103 3 Remove the internal host bus adapter card from the riser a Hold the riser in one hand and use your other hand to carefully remove the card from slot 4 of the riser b Disconnect the rear bracket attached to the PCIe card from the rear of the PCIe riser ...

Page 143: ...s 143 4 Disconnect the SAS cables and the super capacitor cable from the internal HBA card and place the card on an antistatic mat 5 If you are replacing the internal HBA card use a No 2 Philips screwdriver to remove the special fitted bracket from the HBA card ...

Page 144: ...ard From PCIe Slot 4 144 Oracle Server X6 2 Service Manual June 2016 You will need to install the special fitted bracket on the replacement HBA card Set aside the bracket until you are ready to install the replacement HBA card ...

Page 145: ... the super capacitor on an antistatic mat 2 8 Carefully slide the SAS cables and super capacitor cable through the chassis mid wall and remove the cables from the server For instructions see Remove the Internal HBA SAS Cable Assembly on page 173 Install the Internal HBA Card in PCIe Slot 4 1 Unpack the replacement host bus adapter HBA card the super capacitor the SAS cables and the super capacitor...

Page 146: ...Install the Internal HBA Card in PCIe Slot 4 146 Oracle Server X6 2 Service Manual June 2016 2 Using a No 2 Philips screwdriver remove the standard HBA bracket that shipped with the replacement HBA card ...

Page 147: ...ge 141 4 Carefully guide the SAS cables and super capacitor cable that connect to the disk backplane and the super cable through the chassis mid wall toward the front of the server 5 Install the super capacitor into the server a Attach the reclosable fasteners supplied with the replacement super capacitor to its underside b Connect the super capacitor cable to the super capacitor 1 ...

Page 148: ...148 Oracle Server X6 2 Service Manual June 2016 c Press the super capacitor into the server s chassis so that it is secured to the existing reclosable fastener 1 and 2 6 Connect the SAS cables and the super capacitor cable to the internal HBA card ...

Page 149: ... Install the PCIe Riser Into PCIe Slots 3 and 4 on page 105 10 Install fan module FM0 See Install a Fan Module on page 79 11 Return the server to operation a Close the server fan door b Return the server to the normal rack position See Return the Server to the Normal Rack Position on page 191 c Reconnect the data cables to the server and reconnect the power cords to the server power supplies See R...

Page 150: ...om the server before performing these procedures Remove the Disk Backplane on page 150 Install the Disk Backplane on page 153 Related Information Disconnect Cables From the Server on page 56 Take Antistatic Measures on page 59 Extend the Server to the Maintenance Position on page 56 Open the Server Fan Door on page 59 Remove the Disk Backplane Caution The disk backplane should be removed only by a...

Page 151: ... e Remove all four fan modules See Remove a Fan Module on page 76 2 Pull all storage drives out far enough to disengage them from the disk backplane See Remove a HDD or SSD Storage Drive on page 66 Note It is not necessary to completely remove the storage drives from the server simply pull them out far enough to disengage them from the disk backplane If you do remove the storage drives from the se...

Page 152: ...6 2 Service Manual June 2016 3 Remove six Philips screws from the disk cage cover and slide toward the front of the server and lift up to remove it frame 1 4 Disconnect the SAS cables and the switch card cables if present from the disk backplane 2 ...

Page 153: ...and tilt it toward the rear of the server to free it from the holding brackets and lift it out of the server 6 2 10 Place the disk backplane on an antistatic mat Related Information Install the Disk Backplane on page 153 Install the Disk Backplane Caution The disk backplane should be installed only by authorized Oracle Services personnel 1 To place the backplane into the disk cage do the following...

Page 154: ... X6 2 Service Manual June 2016 d Turn the disk backplane captive screw clockwise until the disk backplane is secure 1 5 2 Reconnect the disk backplane power cable to the disk backplane 2 3 If the server has a DVD drive perform the following steps ...

Page 155: ...es into the storage drive cage See Install a HDD or SSD Storage Drive on page 68 7 Install the disk cage cover 8 Install all four fan modules See Install a Fan Module on page 79 9 Return the server to operation a Close the server fan door b Return the server to the normal rack position See Return the Server to the Normal Rack Position on page 191 c Reconnect the data cables to the server and recon...

Page 156: ... drive might occur You must disconnect all power cables from the system before performing these procedures This section covers the following procedures Remove the DVD Drive on page 156 Install the DVD Drive on page 158 Related Information Disconnect Cables From the Server on page 56 Extend the Server to the Maintenance Position on page 56 Open the Server Fan Door on page 59 Remove the DVD Drive 1 ...

Page 157: ...60 3 Locate the DVD drive 1 4 Disconnect the cable from the rear of the DVD 2 FIGURE 30 Removing the DVD Drive 5 To disengage the DVD drive from the chassis press and hold the release tab on the rear of the DVD drive up slightly 3 6 Gently push the DVD drive toward the front of the server slide it out of the server and place it on an antistatic mat 4 ...

Page 158: ... cover See Install the Server Top Cover on page 188 b Close the fan door c Return the server to the normal rack position See Return the Server to the Normal Rack Position on page 191 d Reconnect the power cords to the server power supplies and reconnect the data cables See Reconnect Data Cables and Power Cords on page 193 e Power on the server See Power On the Server on page 194 Verify that the Po...

Page 159: ... Fan Door on page 59 Remove the Front Indicator Module Caution The front indicator module should be removed only by authorized Oracle Services personnel 1 Prepare the server for service a Power off the server and disconnect the power cords from the power supplies See Powering Down the Server on page 51 b Extend the server to the maintenance position See Extend the Server to the Maintenance Positio...

Page 160: ...icator Module 3 Loosen the screw that secures the front indicator module FIM assembly to the server chassis 2 4 Carefully slide the FIM assembly toward the rear of the server enough to lift it over the Phillips screw and out of the server 2 5 Disconnect the FIM cable from the FIM assembly and set the assembly aside 3 6 Continue to Install the Front Indicator Module on page 161 Related Information ...

Page 161: ... 2 5 Tighten the Phillips screw to secure the FIM assembly to the server chassis 2 6 Position the storage drive cage cover over the server storage drive cage so that it over hangs the front of the disk drive cage by about 1 4 inch 7 Install the six Phillips screws to secure the cover to the server 1 There are three Phillips screws on each side of the storage drive cage 8 Close the server fan door ...

Page 162: ...at you follow antistatic practices as described in Take Antistatic Measures on page 59 Remove the Motherboard on page 162 Install the Motherboard on page 169 Related Information Remove the Server From the Rack on page 58 Take Antistatic Measures on page 59 Remove the Server Top Cover on page 60 Remove the Motherboard Caution The motherboard should be serviced only by an Oracle qualified service te...

Page 163: ...quired because the power supplies must be reinstalled into the slots from which they were removed otherwise the server key identity properties KIP data might be lost When a server requires service the KIP is used by Oracle to verify that the warranty on the server has not expired For more information on KIP see FRU Key Identity Properties KIP Automated Update on page 49 a Fan modules For instructi...

Page 164: ...sconnect the disk backplane power cable from the motherboard press in on the connector latch and pull the connector out 1 4 To eject the disk backplane auxiliary power and signal cable connector open both side latches 2 5 To eject the FIM cable connector open both side latches 3 ...

Page 165: ...ver chassis b Lift up the mid wall slightly to disengage it from the raised mushroom shaped standoffs that are located on the server chassis sidewall one on each end of the mid wall then lift it out of the server and set it aside 10 Remove the motherboard from the server chassis 6 a Grasp the metal bracket located just to the rear of the DIMM sockets and slide the motherboard toward the front of t...

Page 166: ...e sockets server performance might suffer and some DIMMs might not be used by the server 14 Remove the processors from the failed motherboard For instructions see Remove a Processor on page 125 15 Remove the processor socket covers from the replacement motherboard and install the processors a Disengage the processor ILM independent loading mechanism assembly hinge lever on the right side of the pr...

Page 167: ...e cover and carefully push the cover out of the processor ILM assembly load plate 3 Caution Be careful not to allow the processor socket cover to fall into the processor socket as this could result in damage to the socket e Install a processor into the socket from which you removed the processor socket cover For instructions for installing a processor see Install a Processor on page 131 f Repeat s...

Page 168: ...ailed motherboard Frames 1 and 2 b Hold the processor ILM assembly load plate open with one hand and position the processor socket cover over the top of the ILM assembly load plate so that 1 the arrow on the processor socket cover is aligned with the arrow on the load plate and 2 the fasteners on one side of the cover the ...

Page 169: ...ed Information Install the Motherboard on page 169 Install the Motherboard Caution The motherboard should be installed only by an Oracle qualified service technician 1 Attach an antistatic wrist strap to your wrist and then to a metal area on the chassis See Take Antistatic Measures on page 59 2 Insert the motherboard into the server chassis 1 a Grasp the metal bracket located to the rear of the D...

Page 170: ...board 170 Oracle Server X6 2 Service Manual June 2016 c Ensure that the indicators controls and connectors on the rear of the motherboard fit correctly into the rear of the server chassis 3 Install the server mid wall 2 ...

Page 171: ...lly guide card cables through the chassis mid wall You will connect these cables to the switch card later 5 If the server has a DVD drive carefully guide the DVD drive cable through the mid wall and reconnect it to the motherboard 3 6 To install the front indicator module FIM cable push the side latches on the motherboard connector to the open position and push the FIM cable connector in 4 The sid...

Page 172: ...ver on page 188 b Close the server fan door c Return the server to the normal rack position See Return the Server to the Normal Rack Position on page 191 d Reconnect the data cables to the server and reconnect the power cords to the server power supplies See Reconnect Data Cables and Power Cords on page 193 e Power on the server See Power On the Server on page 194 Verify that the Power OK status i...

Page 173: ...damage to the server you must disconnect power cords before servicing the cables Tfhis section covers the following procedures Remove the Internal HBA SAS Cable Assembly on page 173 Install the Internal HBA SAS Cable Assembly on page 176 Related Information Extend the Server to the Maintenance Position on page 56 Take Antistatic Measures on page 59 Remove the Server Top Cover on page 60 Remove the...

Page 174: ...Measures on page 59 d Remove the server top cover See Remove the Server Top Cover on page 60 2 Remove all of the server fan modules See Remove a Fan Module on page 76 3 Remove the PCIe riser from PCIe slots 3 and 4 See Remove the PCIe Riser From PCIe Slots 3 and 4 on page 103 4 To disconnect each SAS cables from the disk backplane press the latch on the cable connectors and pull the connectors out...

Page 175: ... Internal HBA SAS Cable Assembly Servicing FRUs 175 FIGURE 32 Removing the SAS Cables and the Super Capacitor Cable 5 Disconnect the SAS cables and the super capacitor cable from the internal host bus adapter HBA ...

Page 176: ...Information Install the Internal HBA SAS Cable Assembly on page 176 Install the Internal HBA SAS Cable Assembly Caution The internal HBA SAS cable assembly should be installed only by an Oracle qualified service technician 1 Carefully guide SAS cables and the super capacitor cable along the side of the chassis See Figure 32 2 Connect the super capacitor cable to the super capacitor 3 3 Connect the...

Page 177: ... Remove the Internal HBA SAS Cable Assembly on page 173 Servicing the NVMe Cables Caution The NVMe cables should be serviced only by authorized Oracle Services personnel Caution These procedures require that you handle components that are sensitive to electrostatic discharge This sensitivity can cause the components to fail To avoid damage ensure that you follow antistatic practices as described i...

Page 178: ...e technician 1 Prepare the server for service a Power off the server and disconnect the power cords from the power supplies See Powering Down the Server on page 51 b Extend the server to the maintenance position See Extend the Server to the Maintenance Position on page 56 c Attach an antistatic wrist strap to your wrist and then to a metal area on the chassis See Take Antistatic Measures on page 5...

Page 179: ... pull the connectors out 1 4 Remove the PCIe riser that contains the switch card from PCIe slot 1 See Remove a PCIe Riser From PCIe Slot 1 or 2 on page 99 5 Disconnect the cables from the switch card 2 6 Carefully pull the switch card cables through chassis mid wall and remove them from the server Related Information Install the NVMe Cables on page 180 ...

Page 180: ...through the chassis mid wall and place them in cable trough in the air baffle and in the cable trough in the metal bracket just to the rear of the DIMMs 1 2 Connect the cables to the switch card 1 3 Install the PCIe riser with the switch card into PCIe slot 1 See Install a PCIe Riser Into PCIe Slot 1 or 2 on page 101 4 Connect the switch card cables to the disk backplane 2 5 Install the two middle...

Page 181: ...lated Information Install the NVMe Cables on page 180 Servicing the Power FIM Disk Backplane Data and DVD Cables Caution The Power front indicator module disk backplane data and DVD cables should be serviced only by authorized Oracle Services personnel Caution The system supplies power to the cables even when the server is powered off To avoid personal injury or damage to the server you must disco...

Page 182: ...r to the maintenance position See Extend the Server to the Maintenance Position on page 56 c Attach an antistatic wrist strap to your wrist and then to a metal area on the chassis See Take Antistatic Measures on page 59 d Remove the server top cover See Remove the Server Top Cover on page 60 2 Remove all of the server fan modules See Remove a Fan Module on page 76 3 To remove the disk backplane po...

Page 183: ...g the Power FIM Disk Backplane Data and DVD Cables Servicing FRUs 183 FIGURE 33 Removing the Power FIM Auxiliary Power Signal and DVD Cables 4 To remove the Auxiliary power and signal cable perform these steps ...

Page 184: ... perform these steps a If there is a PCIe card installed in PCIe slot 2 remove the PCIe riser and card from PCIe slot 2 See Remove a PCIe Riser From PCIe Slot 1 or 2 on page 99 b Disconnect connect the DVD cable from the motherboard 7 c Disconnect the DVD cable connector from the rear of the DVD drive and the DVD power connector from the disk backplane 8 Related Information Install the Power FIM D...

Page 185: ...ackplane ensure that the latches on the disk backplane connector are in the open position and plug the cable connector into the disk backplane connector 4 The disk backplane connector side latches will close locking the auxiliary power and signal cable connector in place b To install the auxiliary power and signal cable into the motherboard ensure that the latches on the motherboard connector are ...

Page 186: ...tall the Server Top Cover on page 188 b Close the server fan door c Return the server to the normal rack position See Return the Server to the Normal Rack Position on page 191 d Power on the server See Power On the Server on page 194 Verify that the Power OK status indicator is steady on Related Information Removing the Power FIM Disk Backplane Data and DVD Cables on page 182 ...

Page 187: ... on the server Power On the Server on page 194 Removing and Installing Server Filler Panels Each server might be shipped with module replacement filler panels for storage drives and PCIe cards These filler panels are installed at the factory and must remain in the server until you replace them with a purchased option Before you can install an optional server component into the server you must remo...

Page 188: ...late until the release lever engages with the chassis 5 Close the release lever until it clicks into place and is flush with the front of the server PCI Slot 1 Remove the server top cover 2 To remove the PCIe slot filler panel rotate the PCIe locking mechanism and then lift and remove the PCIe slot filler panel from the location into which you intend to install the PCIe card Note In single process...

Page 189: ...front of the chassis the internal latch that is located on the underside of the cover might be damaged 4 Gently slide the cover toward the front of the chassis until it locks into place with an audible click 2 As you slide the cover toward the front of the server watch the green release button You will hear an audible click when the green release button pops up indicating that the cover is locked ...

Page 190: ...server into the rack Caution The server weighs approximately 18 1 kg 40 0 lbs Two people are required to carry the chassis and install it into the rack 1 Push the slide rails into the slide rail assemblies in the rack as far as possible 2 Raise the server so that the rear ends of the mounting brackets are aligned with the slide rail assemblies that are mounted in the rack 3 Insert the mounting bra...

Page 191: ...reen 3 Slide rail lock Related Information Remove the Server From the Rack on page 58 Remove Antistatic Measures on page 190 Reconnect Data Cables and Power Cords on page 193 Return the Server to the Normal Rack Position If the server is in the extended maintenance position use this procedure to return the server to the normal rack position 1 Push the server back into the rack as described in the ...

Page 192: ...in the center of the tab not on the end and apply pressure as you pull the tab toward the front of the server FIGURE 36 Location of the Release Tabs on the Slide Rails b Continue pushing the server into the rack until the slide rail locks on the front of the server engage the slide rail assemblies You will hear an audible click when the server is in the normal rack position 2 If the CMA is not ins...

Page 193: ...r detailed information on connecting cables to the rear of the server refer to Rear Cable Connections and Ports in Oracle Server X6 2 Installation Guide Related Information Reconnect Data Cables and Power Cords on page 193 Reconnect Data Cables and Power Cords 1 Reconnect the data cables to the rear of the server as required If the cable management arm CMA is in the way extend the server approxima...

Page 194: ...the power cords are connected standby power is applied and the green Power OK status indicator on the server front panel blinks Depending on the configuration of the firmware the system might boot If it does not boot follow this procedure Power on the server by performing one of the following Press the Power button on the server front panel or Log in to the Oracle ILOM web interface click Host Man...

Page 195: ... Learn about USB ports USB Ports on page 199 Gigabit Ethernet Ports The server has four auto negotiating 100 1000 10 000BASE T gigabit Ethernet GbE system domain ports All four Ethernet ports use a standard RJ 45 connector The transfer rates are shown in the following table TABLE 16 Ethernet Port Transfer Rates Connection Type IEEE Terminology Transfer Rate Fast Ethernet 100BASE TX 100 Mbps Gigabi...

Page 196: ...rk Management Port The server has one 10 100 1000BASE T Ethernet management domain interface labeled NET MGT For information on configuring this port for managing the server with Oracle ILOM refer to the Oracle Integrated Lights Out Manager ILOM 3 2 Documentation Library at http www oracle com goto ilom docs TABLE 18 Network Management Port Signals Pin Signal Description Pin Signal Description 1 T...

Page 197: ...gure and table describe the SER MGT port pin signals TABLE 20 Serial Management Port Signals Pin Signal Description Pin Signal Description 1 Request to Send 5 Ground 2 Data Terminal Ready 6 Receive Data 3 Transmit Data 7 Data Set Ready 4 Ground 8 Clear to Send If you need to connect to the SER MGT port using a cable with either a DB 9 or a DB 25 connector follow the pin descriptions in the tables ...

Page 198: ...rossover Wiring Reference Serial Port RJ 45 Connector DB 25 Adapter Pin Signal Description Pin Signal Description 1 RTS 5 CTS 2 DTR 6 DSR 3 TXD 3 RXD 4 Signal Ground 7 Signal Ground 5 Signal Ground 7 Signal Ground 6 RXD 2 TXD 7 DSR 20 DTR 8 CTS 4 RTS Related Information Server Back Panel View on page 16 Disconnect Cables From the Server on page 56 Reconnect Data Cables and Power Cords on page 193 ...

Page 199: ...ound 14 Vertical Sync 7 Green Ground 15 N C Reserved 8 Blue Ground Related Information Server Back Panel View on page 16 Disconnect Cables From the Server on page 56 Reconnect Data Cables and Power Cords on page 193 USB Ports The server has six USB ports for attaching supported USB 2 0 compliant devices There are two USB ports on the server front panel two on the rear panel and two on the motherbo...

Page 200: ...B Ports 200 Oracle Server X6 2 Service Manual June 2016 Related Information Server Back Panel View on page 16 Disconnect Cables From the Server on page 56 Reconnect Data Cables and Power Cords on page 193 ...

Page 201: ... Menu Options on page 227 Managing the BIOS Configuration The BIOS configuration parameters on an Oracle x86 server are manageable from the BIOS Setup Utility and Oracle ILOM For information about using these tools to manage the BIOS configuration refer to Oracle ILOM Maintaining x86 BIOS Configuration Parameters in the Oracle ILOM Administrator s Guide for Configuration and Maintenance Firmware R...

Page 202: ...up Utility Menus The following table provides descriptions for the top level BIOS Setup Utility menus TABLE 25 BIOS Setup Utility Menus Summary Menu Description Main General product information including memory time date security settings system serial number and CPU and DIMM information Advanced Configuration information for the CPU trusted computing USB and other information Set the IP address f...

Page 203: ...Utility Help menu F2 Ctrl E Enter BIOS Setup Utility while the system is performing the power on self test POST Not applicable F7 Ctrl D Not applicable Discard changes Not applicable to UEFI Driver Control menu F8 Ctrl P Activate the BIOS Boot Menu Not applicable F10 Ctrl S Not applicable Activate Save and Exit pop up menu Not applicable to UEFI Driver Control menu F12 Ctrl N Activate Network boot...

Page 204: ...ain to power on the server From the Oracle ILOM web interface click Host Management Power Control and select Reset from the Select Action list From the Oracle ILOM CLI type reset System The power on self test POST sequence begins 2 To enter the BIOS Setup Utility press the F2 key Ctrl E from a serial connection when prompted and while the BIOS is running the power on self tests POST The BIOS Setup...

Page 205: ...ption user instructions for modifying the option appear in the right column of the screen If an option is a link to a sub screen a description of the sub menu content appears in the right column 4 Modify an option by pressing the or keys plus or minus keys or by pressing Enter and selecting the desired option from the pop up menus 5 Press the Esc key to return from a sub menu screen to the previou...

Page 206: ...Boot Mode or UEFI Boot Mode Legacy BIOS Boot Mode is the default If you change BIOS boot modes the boot candidates from the previously selected mode are no longer available from the Boot Options Priority List in the BIOS Setup Utility The boot candidates from the new mode appear in the Boot Options Priority List only after you select Save Changes and Reset from the BIOS Setup Utility menu Use the ...

Page 207: ...switching between Legacy BIOS Boot Mode and UEFI BIOS Boot Mode either direction BIOS settings that affect the Boot Options Priority list settings will be changed If the boot mode is changed the boot candidates from the previous boot mode disappear The boot candidates for the newly changed boot mode appear after you save your changes and reset the host and in the next boot to the BIOS Setup Utilit...

Page 208: ... to raw device labels Related Information Selecting Legacy BIOS or UEFI Boot Mode on page 206 Select the Boot Device on page 213 Exit BIOS Setup Utility on page 224 Configuration Utilities for Add In Cards The method for interacting with configuration utilities for add in cards and system resident I O adapters differs depending on whether Legacy BIOS Boot Mode or UEFI BIOS Boot Mode is used In Leg...

Page 209: ...ng message in the Oracle ILOM event log Option ROM Space Exhausted Device XXX Disabled By default all on board Legacy Options ROMs are enabled in the BIOS However you can disable most of these Option ROMs unless they are required to support booting from the associated device or to provide some other boot time function For example it is not necessary to load the Option ROM for the on board network ...

Page 210: ...ings to the optimal factory default values as well as view and edit settings as needed Any changes that you make in the BIOS Setup Utility using the F2 key persist until the next time you change the settings Before you begin ensure that the following requirements are met A hard disk drive or solid state drive is properly installed in the server A console connection is established to the server 1 R...

Page 211: ...p Utility by navigating to the Save Exit Menu and selecting Save Changes and Reset Related Information Access BIOS Setup Utility Menus on page 203 BIOS Setup Utility Menus on page 202 BIOS Key Mappings on page 203 Exit BIOS Setup Utility on page 224 Select Legacy BIOS or UEFI Boot Mode The BIOS firmware supports both Legacy BIOS Boot Mode and UEFI Boot Mode The default setting is Legacy BIOS Boot ...

Page 212: ...eboot is required to properly populate the Boot Options Priority list with devices that support the chosen boot mode 4 Use the up and down arrow keys to select the appropriate Legacy or UEFI mode and then press Enter 5 Press the F10 key to save the changes and exit the BIOS Setup Utility Related Information BIOS Setup Utility Menus on page 202 BIOS Key Mappings on page 203 Exit BIOS Setup Utility ...

Page 213: ...ent boot device specified using the F2 key will be in effect after booting from the temporary boot device 1 Reset or power on the server From the local server press the Power button on the front panel of the server to power off the server and then press the Power button again to power on the server From the Oracle ILOM web interface click Host Management Power Control and select Reset from the Sel...

Page 214: ... only the applicable devices For example when the UEFI BIOS Boot Mode is elected only UEFI BIOS boot devices are displayed in the dialog box A dialog box similar to the following appears 4 Follow the on screen instructions to install the operating system from the selected boot device Related Information BIOS Setup Utility Menus on page 202 BIOS Key Mappings on page 203 Exit BIOS Setup Utility on p...

Page 215: ...ting this feature refer to the Windows Trusted Platform Module Management documentation provided by your operating system or third party software vendor 1 Access the BIOS Setup Utility menus See Access BIOS Setup Utility Menus on page 203 2 In the BIOS Setup Utility menus navigate to the Advanced Menu The Advanced Menu screen appears 3 In the Advanced Menu screen select Trusted Computing The TPM C...

Page 216: ...Configure TPM Support 216 Oracle Server X6 2 Service Manual June 2016 A TPM Support dialog box appears 5 In the dialog box set TPM Support to Enable and then press Enter ...

Page 217: ...Microsoft s Windows Trusted Platform Module Management documentation Configure SP Network Settings Choose one of the following methods to specify service processor SP network settings BIOS Assign the IP address for the server SP from the BIOS Setup Utility on the Advanced Menu Oracle ILOM For instructions on setting the IP address for the server SP using Oracle ILOM refer to the Oracle Integrated ...

Page 218: ... BIOS Setup Utility menus See Access BIOS Setup Utility Menus on page 203 2 In the BIOS Setup Utility menus navigate to the Advanced Menu The Advanced Menu screen appears 3 In the Advanced Menu select BMC Network Configuration and then press Enter The BMC Network Configuration screen appears ...

Page 219: ...onfigure SP Network Settings Setting Up BIOS Configuration Parameters 219 The BMC is the Baseboard Management Controller 4 Select Refresh and then press Enter to display the current BMC network settings ...

Page 220: ...date the BMC network settings with the latest values 6 Press the F10 key to save the changes and exit the BIOS Setup Utility Related Information BIOS Setup Utility Menus on page 202 Exit BIOS Setup Utility on page 224 Configure Option ROM Settings 1 Access the BIOS Setup Utility menus See Access BIOS Setup Utility Menus on page 203 2 In the BIOS Setup Utility menus navigate to the IO Menu ...

Page 221: ...BIOS Configuration Parameters 221 The IO Menu screen appears 3 In the IO Menu select either Internal Devices or Add In Cards and press Enter to display the internal device or add in card slot for which you want to enable or disable Option ROM ...

Page 222: ...nfigure Option ROM Settings 222 Oracle Server X6 2 Service Manual June 2016 The Internal Devices or Add In Cards screen appears 4 Select the internal device or add In card slot that you want to configure ...

Page 223: ...d to enable the Option ROM setting Select Disabled to disable the Option ROM setting 6 Press the F10 key to save the changes and exit the BIOS Setup Utility Related Information Using BIOS for Legacy Option ROM Resource Allocation on page 209 Access BIOS Setup Utility Menus on page 203 BIOS Setup Utility Menus on page 202 Exit BIOS Setup Utility on page 224 Configure I O Resource Allocation 1 Acces...

Page 224: ... Enabled to enable I O resource allocation for the I O card Select Disabled to disable I O resource allocation for the I O card 6 Press the F10 key to save the changes and exit the BIOS Setup Utility Related Information Configure I O Resource Allocation on page 223 Access BIOS Setup Utility Menus on page 203 BIOS Setup Utility Menus on page 202 Exit BIOS Setup Utility on page 224 Exit BIOS Setup U...

Page 225: ...exit process Note After modifying any BIOS settings and selecting Save Changes and Exit from the Save Exit Menu the subsequent reboot might take longer than a typical reboot where no settings were modified The additional delay is required to ensure that changes to the BIOS settings are synchronized with Oracle ILOM Related Information Access BIOS Setup Utility Menus on page 203 BIOS Setup Utility ...

Page 226: ...226 Oracle Server X6 2 Service Manual June 2016 ...

Page 227: ...Review the BIOS IO Menu selections BIOS IO Menu Selections on page 244 Review the Boot Menu selections BIOS Boot Menu Selections on page 248 Review the BIOS Save Exit Menu selections BIOS Exit Menu Selections on page 249 Related Information Oracle X6 Series Servers Administration Guide at http www oracle com goto x86admindiag docs Setting Up BIOS Configuration Parameters on page 201 BIOS Main Menu...

Page 228: ...o reference a specific BIOS release Format is XXYYZZPP which indicates XX Unique project platform code YY BIOS major release ZZ BIOS minor release PP Build number Example 18 01 04 01 System Date Current date is displayed You can change the date setting Example Thu 06 20 2014 System Time Current time is displayed You can change the time setting Example 13 38 27 QPI Link Speed R O SLOW 6 4 GT s Inte...

Page 229: ...mation is displayed Product Name Product name is displayed Example Oracle Server X6 2 Product Serial Number Product serial number is displayed Example 1134FML00V Board Serial Number Board serial number is displayed Example 0328MSL 1132U900 CPU INFORMATION R O Attributes of a single processor CPU are defined A separate information structure is provided for each processor supported in the system Mos...

Page 230: ...ported L1 Data Cache Example 448 L1 Code Cache Example 448 L2 Cache Example 3584 KB L3 Cache Example 35840 KB Socket 1 CPU Information R O If CPU socket 1 is populated the same options as Socket 0 CPU Information are displayed Otherwise displays Not Present DIMM INFORMATION R O Memory module DIMM presence and size information are displayed Socket 0 DIMMs R O For D0 D11 if a DIMM is present memory ...

Page 231: ...dvanced menu options BIOS Advanced Menu Processor Configuration Options on page 232 BIOS Advanced Menu CPU Power Management Configuration Options on page 233 BIOS Advanced Menu Memory Configuration Option on page 234 BIOS Advanced Menu USB Ports Options on page 234 BIOS Advanced Menu Serial Port Console Redirection Options on page 235 BIOS Advanced Menu Trusted Computing Options on page 236 BIOS A...

Page 232: ...mber of cores to enable in each processor package Execute Disable Bit Disabled Enabled Enabled When disabled forces XD features flag to always return 0 Hardware Prefetcher Disabled Enabled Enabled Enable the mid level cache MLC streamer prefetcher MSR 1A4h bit 0 Adjacent Cache Prefetch Disabled Enabled Enabled Enable the mid level cache MLC streamer prefetcher MSR 1A4h bit 1 DCU Streamer Prefetche...

Page 233: ...Thermal Monitoring EMTTM be enabled on the CPU CPU C3 report Enabled Disabled Disabled Enable disable the CPU C3 ACPI C3 report to the operating system CPU C6 report Enabled Disabled Enabled Enable disable the CPU C6 ACPI C3 report to the operating system CPU C7 report Enabled Disabled Disabled Enable disable the CPU C7 ACPI C3 report to the operating system Package C State limit Enabled Disabled ...

Page 234: ...USB Ports Setup Options Options Defaults Description EHCI Hand off Disabled Enabled Disabled Enable or disable Enhanced Host Controller Interface EHCI hand off support This is a workaround for operating systems without EHCI hand off support The EHCI change of ownership should be claimed by the EHCI driver Port 60 64 Emulation Disabled Enabled Enabled Enable I O port 60h 64h emulation support Enabl...

Page 235: ...able console redirection for Windows Emergency Management Service EMS administration Console Redirection Disabled Enabled Enabled Enable or disable console redirection Terminal Type VT100 VT100 VT UTF8 ANSI VT100 Select the emulation for the terminal VT100 ASCII character set VT100 Extends VT100 to support color function keys etc VT UTF8 Uses UTF8 encoding to map Unicode characters onto one or mor...

Page 236: ...ing is 1 stop bit Communication with slow devices may require more than 1 stop bit Flow Control None Hardware RTS CTS Software Xon Xoff None Flow control can prevent data loss from buffer overflow When sending data if the receiving buffers are full a stop signal can be sent to stop the data flow Once the buffers are empty a start signal can be sent to restart the flow Hardware flow control uses tw...

Page 237: ...led Enabled Disabled Use this option to provide the current capability state of the security device Note This option is available only if TPM Support is set to enabled TPM Active Status Deactivated Activated Deactivated Use this option to provide the current capability state of the security device Note This option is available only if TPM Support is set to enabled TPM Owner Status Owned Unowned Un...

Page 238: ...C LAN information with the latest information from the service processor Active Mgmt Port NET MGT NET0 NET1 NET2 NET3 NET MGT Use to change the currently active network management port Commit NA NA Commit the current BMC network LAN information IPv4 Configuration NA NA Current configuration of the IPv4 settings is displayed Channel Number R O 1 Current channel number is displayed Current IPv4 IP A...

Page 239: ...bnet Mask NA NA If set current IPv4 Subnet Mask is displayed IPv4 Default Gateway NA NA If set current Default IPv4 Gateway is displayed Commit Commit the current BMC LAN information IPv6 Configuration Current configuration of the IPv6 settings is displayed IPv6 addresses are written with hexadecimal digits and colon separators For example 2001 0db0 000 82a1 0000 0000 1234 abcd IPv6 addresses are ...

Page 240: ...be 5000 128 Dynamic IPv6 Address 1 n R O Current dynamic IPv6 addresses are displayed Example fec0 a 8 b7 214 4fff feca 5f7e 64 Refresh Select Refresh to update to the current settings IPv6 State Disabled Enabled Disabled Set the IPv6 state to enabled or disabled Auto IPv6 Configuration Disabled Stateless Dhcpv6_stateless Dhcpv6_stateful Disabled Autoconfiguration options are Disabled When autocon...

Page 241: ...s TABLE 36 BIOS Advanced Menu iSCSI Configuration Options Setup Options Options Defaults Description iSCSI Initiator Name NA must be specified None The worldwide unique name of the iSCSI Initiator Only IQN format is accepted Add an Attempt iSCSI Attempt Name NA must be specified None The human readable name that you assigned to this attempt iSCSI Mode Disabled Enabled Enabled for MPIO Disabled Set...

Page 242: ...figurable Example Update 0ABBCCDDEEFF to 0ABBCCF07901 by inputting F07901 Enable DHCP Disabled Enabled Disabled Enable or disable DHCP Initiator IP Address NA 0 0 0 0 Use to set initiator IP address in dotted decimal notation Initiator Subnet Mask NA 0 0 0 0 Use to set initiator subnet mask IP address in dotted decimal notation Gateway NA 0 0 0 0 Use to set initiator gateway IP address in dotted d...

Page 243: ...anced Menu Ethernet Controller Options The following table lists the BIOS Advance Menu Ethernet Controller options TABLE 37 BIOS Advanced Menu Ethernet Controller Options Setup Options Options Defaults Description Port Configuration Menu NIC configuration Use to configure the network interface controller Link Speed R O Supported port speeds Auto Negotiate NA Specifies the port speed used for the s...

Page 244: ...ss R O NA NA Bus device function identifier Example format Bus Device Function Link Status Connected Disconnected Disconnected Specifies the link status of the network port MAC Address R O NA None Lists the MAC address of the network interface controller NIC Related Information BIOS Main Menu Selections on page 227 BIOS IO Menu Selections on page 244 BIOS Boot Menu Selections on page 248 BIOS Exit...

Page 245: ...Devices Options on page 247 BIOS IO Menu Add In Cards Options on page 247 BIOS IO Menu PCI Subsystem Settings Options The following table describes the BIOS IO Menu PCI Subsystem Settings Options TABLE 38 BIOS IO Menu PCI Subsystem Settings Options Setup Options Options Defaults Description PCI 64 Bit Resources Allocation Disabled Enabled Enabled Enable or disable 64 bit capable devices to be deco...

Page 246: ... O Virtualization SR IOV is used to configure devices into multiple virtual devices that can be used on virtual OS installations If supported by the hardware and set to enabled all devices within the system that are SR IOV capable are configured to support SR IOV and I O resources are allocated to the device as normal If set to disabled I O resources are not allocated to the device ARI Disabled En...

Page 247: ... Enabled Enabled Enable or disable Option ROM If set to enabled Option ROM for the card executes as normal If set to disabled Option ROM for the card is not copied into memory and the execution of the Option ROM is inhibited NET1 NET2 and NET3 Note Ethernet ports NET2 and NET3 are nonfunctional in single processor systems See NET0 description BIOS IO Menu Add In Cards Options The following table d...

Page 248: ...Boot Menu Selections This section includes a searchable text based representation and a screenshot of the BIOS Boot Menu The options that are available from the Boot Menu are described in the table that follows TABLE 43 BIOS Boot Menu Options Setup Options Options Defaults Description UEFI BIOS Boot Mode Legacy UEFI Legacy Select either Legacy BIOS or UEFI as the boot mode Enable UEFI Only UEFI BI...

Page 249: ... boots failed If set to Boot List fail over to the main Boot Options Priority list Persistent Boot Support Disabled Enabled Disabled If enabled the Persistent Boot feature is enabled If disabled system reverts back to the default boot handling from the next boot NVMe Boot Disabled Enabled Enabled BIOS supports booting to UEFI Legacy OS that is present in the NVMe device Legacy Boot Option Priority...

Page 250: ...y Discard Changes and Exit Exit the BIOS Setup Utility without saving changes Discard Changes Discard any changes made to the setup options Restore Defaults Restore load the optimal default values for all of the setup options Related Information BIOS Main Menu Selections on page 227 BIOS Advanced Menu Selections on page 231 BIOS IO Menu Selections on page 244 BIOS Boot Menu Selections on page 248 ...

Page 251: ... ILOM The Oracle ILOM 3 2 interfaces provide easy to view information about the health status of system components From the Oracle ILOM web interface or in the Oracle ILOM command line interface CLI you can collect system specific information about the server determine the health state of discrete components and view any open problems on the server Oracle ILOM automatically detects system hardware...

Page 252: ... shortened form of the component name and is indicated by the boldface portion of the component name IPMI Type Indicates the type of sensor indicator or FRU represented Description Describes the particular component name reference Values Defines the states of the sensor indicator or FRU entity and any specific units or values that are expected if applicable Note Some component names are hidden in ...

Page 253: ...er Interface hidden 01h ACPI_ON_WORKING 20h ACPI_SOFT_OFF SYS VPS Threshold sensor Virtual power sensor Watts SYS VPS_CPUS Threshold sensor Virtual power sensor CPUs Watts SYS VPS_MEMORY Threshold sensor Virtual power sensor Memory Watts SYS VPS_FANS Threshold sensor Virtual power sensor Fans Watts SYS INTSW Discrete sensor Chassis intrusion switch 01h Deasserted 02h Asserted SYS T_AMB Threshold s...

Page 254: ...rvice SYS LOCATE Indicator Locator LED Color White Location Front and rear panels Off Normal Fast blink Locate function is activated self extinguishes after 30 minutes SYS HOST_ERR Discrete sensor Host asserted error sensor 01h Deasserted 02h Asserted SYS PS_FAULT Indicator Rear Power Supply Fault LED Color Amber Location Mainboard Off Normal On General power supply fault SYS FAN_FAULT Indicator T...

Page 255: ...SNT Discrete sensor Fan module is present 01h ENTITY_ABSENT 02h ENTITY_PRESENT SYS MB FM 0 3 F 0 x TACH Threshold sensor Fan module fan speed RPM SYS MB FM 0 3 SERVICE Indicator Fan Service Required LED Color Amber Location Motherboard Off Normal On Fan module was diagnosed as faulty Related Information System Chassis Components on page 253 Disk Backplane Components on page 255 Memory Device Compo...

Page 256: ...ion Hard disk drive Off Normal On Hard disk drive was diagnosed as faulty SYS DBP HDD 0 x OK2RM Indicator OK to remove Color Blue Location Hard disk drive Off Normal On OK to remove hard disk drive SYS DBP NVME 0 x NVMe drives SYS DBP NVME 0 x PRSNT Discrete sensor NVMe drive presence 01h ENTITY_ABSENT 02h ENTITY_PRESENT Related Information System Chassis Components on page 253 Cooling Unit Compon...

Page 257: ...g Unit Components on page 255 Disk Backplane Components on page 255 Power Unit Components on page 257 Processor Components on page 258 System Board Components on page 259 System Firmware Components on page 261 Hard Disk Drive Components on page 262 Power Unit Components The following table lists the power unit components Component Name Oracle ILOM CLI Targets IPMI Type Description Values if applic...

Page 258: ...grees Celsius Related Information System Chassis Components on page 253 Cooling Unit Components on page 255 Disk Backplane Components on page 255 Memory Device Components on page 256 Processor Components on page 258 System Board Components on page 259 System Firmware Components on page 261 Hard Disk Drive Components on page 262 Processor Components The following table lists the processor CPU compo...

Page 259: ... 262 System Board Components The following table lists the system board components Component Name Oracle ILOM CLI Targets IPMI Type Description Values if applicable SYS MB FRU General host system board FRU SYS MB CPLD FRU Mainboard CPLD FPGA firmware version SYS MB NET 0 3 FRU Host Ethernet FRU SYS MB RISER 1 3 PRSNT Discrete sensor PCIe riser inserted into PCIe slot 01h ENTITY_ABSENT 02h ENTITY_P...

Page 260: ...slots Slot 1 PCIe slot 1 Slot 2 PCIe slot 2 Slot 3 PCIe slot 3 Degrees Celsius SYS MB T_IN_SLOT 1 3 Threshold sensor Cooling zone chassis inlet temperature located in front of the specified PCIe slots Slot 1 PCIe slot 1 Slot 2 PCIe slot 2 Slot 3 PCIe slot 3 Degrees Celsius SYS MB T_CORE_NET01 Threshold sensor Gigabit Ethernet controller die temperature CPU 0 NET 0 1 Degrees Celsius SYS MB T_CORE_N...

Page 261: ...57 Processor Components on page 258 System Firmware Components on page 261 Hard Disk Drive Components on page 262 System Firmware Components The following table lists the system firmware components Component Name Oracle ILOM CLI Target IPMI Type Description SYS MB BIOS FRU BIOS FRU SYS MB CPLD FRU Mainboard CPLD FPGA firmware Related Information System Chassis Components on page 253 Cooling Unit C...

Page 262: ...RE SYS DBP HDD 0 x SERVICE Indicator Hard disk drive Service Required LED Color Amber Location Hard disk drive Off Normal On Hard disk drive was diagnosed as faulty SYS DBP HDD 0 x OK2RM Indicator Hard disk drive OK to Remove LED Color Blue Location Hard disk drive Off Normal On OK to remove SYS DBP NVME 0 x FRU NVMe storage drive FRU SYS DBP NVME 0 x PRSNT Discrete sensor NVMe storage drive 01h E...

Page 263: ... events Messages and Descriptions Component Name SNMP Trap sunHwTrapTempFatalThresholdExceeded Oracle ILOM Event Message Lower fatal threshold exceeded Severity and Description Critical A temperature sensor has reported that its value has gone above an upper fatal threshold setting or below a lower fatal threshold setting The sunHwTrapThresholdType object indicates whether the threshold was an upp...

Page 264: ...rap sunHwTrapTempFatalThresholdExceeded Oracle ILOM Event Message Lower fatal threshold exceeded Severity and Description Critical A temperature sensor has reported that its value has gone above an upper fatal threshold setting or below a lower fatal threshold setting The sunHwTrapThresholdType object indicates whether the threshold was an upper or lower threshold SYS T_AMB SYS MB T_CORE_NET01 SYS...

Page 265: ...nents on page 256 Power Unit Components on page 257 Processor Components on page 258 System Board Components on page 259 Hard Disk Drive Components on page 262 Hard Disk Drive Events The following table lists hard disk drive events Messages and Descriptions Description SNMP Trap Message sunHwTrapSlotOrConnectorError Oracle ILOM Event Message Assert Severity and Description Major A sensor associate...

Page 266: ... ASSERT Severity and Description Major A power supply sensor has detected an error SYS PS 0 1 STATE SNMP Trap sunHwTrapPowerSupplyError Oracle ILOM Event Message PS_FAILURE DEASSERT Severity and Description Major A power supply sensor has detected an error SYS PS 0 1 STATE SNMP Trap sunHwTrapPowerSupplyError Oracle ILOM Event Message PS_PREDICTIVE_FAILURE ASSERT Severity and Description Major A po...

Page 267: ...S_CONFIG_ERROR DEASSERT Severity and Description Major A power supply sensor has detected an error SYS PS 0 1 STATE SNMP Trap sunHwTrapSensorNonCritThresholdExceeded Oracle ILOM Event Message Upper noncritical threshold exceeded Severity and Description Minor A sensor has reported that its value has gone above an upper noncritical threshold setting or below a lower noncritical threshold setting Th...

Page 268: ...ld was an upper or lower threshold SYS PS 0 1 P_IN SYS PS 0 1 P_OUT SNMP Trap sunHwTrapSensorCritThresholdExceeded Oracle ILOM Event Message Lower critical threshold exceeded Severity and Description Major A power supply sensor has reported that its value has gone above an upper critical threshold setting or below a lower critical threshold setting The sunHwTrapThresholdType object indicates wheth...

Page 269: ...cal threshold setting or above a lower critical threshold setting The sunHwTrapThresholdType object indicates whether the threshold was an upper or lower threshold SYS PS 0 1 V_12V SYS PS 0 1 V_12V_STBY SYS PS 0 1 V_IN SYS MB P 0 x V_DIMM SNMP Trap sunHwTrapVoltageNonCritThresholdExceeded Oracle ILOM Event Message Upper noncritical threshold exceeded Severity and Description Minor A voltage sensor...

Page 270: ...whether the threshold was an upper or lower threshold SYS MB FM 0 3 F 0 1 TACH SNMP Trap Message sunHwTrapFanSpeedFatalThresholdExceeded Oracle ILOM Event Message Lower fatal threshold exceeded Severity and Description Critical A fan speed sensor has reported that its value has gone above an upper fatal threshold setting or below a lower fatal threshold setting The sunHwTrapThresholdType object in...

Page 271: ...ted when the SNMP agent does not recognize the component type SYS VPS_CPUS SYS VPS_MEMORY SNMP Trap sunHwTrapMemoryFault Oracle ILOM Event Message event fault cpu intel quickpath link_slow The Quickpath Interconnect QPI link is operating below normal speed Severity and Description Major A memory component is suspected of causing a fault SYS MB SNMP Trap sunHwTrapMemoryFaultCleared Oracle ILOM Even...

Page 272: ...lt memory intel dimm quad rank 3rd slot A quad rank memory DIMM has been installed in the third slot of a memory channel Severity and Description Major A memory component is suspected of causing a fault SYS MB SNMP Trap sunHwTrapMemoryFaultCleared Oracle ILOM Event Message event fault memory intel dimm quad rank 3rd slot A quad rank memory DIMM has been installed in the third slot of a memory chan...

Page 273: ...MP Trap sunHwTrapMemoryFaultCleared Oracle ILOM Event Message event fault memory intel dimm sodimm unsupported An unsupported SODIMM has been detected in system Severity and Description Informational A memory component fault has been cleared SYS MB P 0 x D 0 11 SNMP Trap sunHwTrapMemoryFault Oracle ILOM Event Message event fault memory intel dimm 4gb fused An unsupported 4 gb memory DIMM has been ...

Page 274: ...ult memory intel dimm incompatible quad rank An invalid quad rank memory DIMM configuration has been detected Severity and Description Major A memory component is suspected of causing a fault SYS MB P 0 x D 0 11 SNMP Trap sunHwTrapMemoryFaultCleared Oracle ILOM Event Message event fault memory intel dimm incompatible quad rank An invalid quad rank memory DIMM configuration has been detected Severi...

Page 275: ...YS MB P 0 x D 0 11 SNMP Trap sunHwTrapMemoryFaultCleared Oracle ILOM Event Message event fault memory intel dimm population invalid Memory DIMM is improperly populated or is a type that is not compatible with other DIMMs installed Severity and Description Informational A memory component fault has been cleared SYS MB P 0 x D 0 11 SNMP Trap sunHwTrapMemoryFault Oracle ILOM Event Message event fault...

Page 276: ...le ILOM Event Message ENTITY_PRESENT ASSERT Severity and Description Major A processor sensor has detected an error Device absent SYS MB P 0 x PRSNT SNMP Trap sunHwTrapProcessorOk Oracle ILOM Event Message ENTITY_PRESENT DEASSERT Severity and Description Major A processor sensor has returned to its normal state Device present SYS MB P 0 x PRSNT SNMP Trap sunHwTrapProcessorError Oracle ILOM Event M...

Page 277: ...rusion Oracle ILOM Event Message Assert Severity and Description Major An intrusion sensor has detected that someone may have physically tampered with the system SYS INTSW SNMP Trap Message sunHwTrapSecurityIntrusion Oracle ILOM Event Message Deassert Severity and Description Major An intrusion sensor has detected that someone may have physically tampered with the system SYS INTSW ...

Page 278: ...278 Oracle Server X6 2 Service Manual June 2016 ...

Page 279: ...ting from 224 function keys 203 IO screen 244 Legacy Boot Mode 206 Main screen 227 menus accessing 203 navigating menu options 205 service processor network settings 217 setup screens summary 202 shortcut key sequences 203 TPM support 215 UEFI Boot Mode 206 UEFI compliant description 206 board components 253 boot device selecting 213 boot mode preserving settings when switching 207 selecting 211 b...

Page 280: ...view of server 17 external cables connecting 40 F fan modules description 20 installing 79 removing 76 Fault Remind button identifying failed processors 125 location 93 using 92 FIM description 20 installing 161 removing 159 servicing 159 firmware components 253 Front Indicator Module See FIM function keys within BIOS Setup Utility 203 G Gigabit Ethernet ports 195 pinouts 196 H hard disk drive com...

Page 281: ...on 21 installing 180 removing 178 178 O operating system supported by UEFI BIOS 206 Option ROMs enabling and disabling 220 Legacy BIOS allocation 209 minimizing resource exhaustion 209 Oracle Integrated Lights Out Manager Oracle ILOM key identity properties KIP for FRUID auto update feature 49 Oracle NVMe cables installing 180 Oracle PCIe NVMe switch card installing 139 removing 138 P PCIe cards d...

Page 282: ...ver disengaging 128 engaging 134 removal and replacement tool 132 processors identifying failed 125 installing 131 removing 125 PSU See power supply R rackmount stabilizing the rack 57 removing antistatic measures 190 battery 116 DIMMs 93 disk backplane 150 153 DVD drive 156 fan modules 76 FIM 159 internal HBA card 141 internal USB flash drive 114 motherboard 162 NVMe cables 178 Oracle PCIe NVMe s...

Page 283: ...tandby power 54 55 55 status indicators power supply 31 storage drive 30 stop command 52 storage drive description 19 hot plugging 64 identification 66 70 74 installing 68 71 74 latch release button 66 70 74 removing 66 status indicators 30 support for product 45 system board components 259 system chassis components 253 system components and nomenclature 252 system firmware components 261 263 265 ...

Page 284: ...284 Oracle Server X6 2 Service Manual June 2016 ...

Reviews:

No comments