background image

Contents

Safety notices........................................................................................................v

Installing PCIe adapters in the system...................................................................1

Preparing the 9080-M9S system to install a PCIe adapter........................................................................ 1

Installing a PCIe adapter in the system...................................................................................................... 8

Preparing the 9080-M9S system for operation after installing a PCIe adapter ..................................... 17

Related procedures for installing PCIe adapters......................................................................................22

Avoiding electric shock........................................................................................................................ 22

Handling static-sensitive devices........................................................................................................ 23

Updating the worldwide port name..................................................................................................... 23

Removing and replacing the tailstock on a PCIe adapter .................................................................. 24

PCIe hot-plug manager access for AIX............................................................................................... 29

Installing the AIX device driver software............................................................................................ 31

Installing or replacing a PCIe adapter with the system power turned on in a Virtual I/O Server......33

Verifying that the hot-plug PCI tools are installed for Linux...............................................................36

Checking the amount of remaining life in NVMe devices....................................................................37

Notices................................................................................................................41

Accessibility features for IBM Power Systems servers............................................................................ 42

Privacy policy considerations ................................................................................................................... 43

Trademarks................................................................................................................................................ 43

Electronic emission notices.......................................................................................................................43

Class A Notices.....................................................................................................................................43

Class B Notices.....................................................................................................................................47

Terms and conditions................................................................................................................................ 49

  iii

Содержание FC EJ05

Страница 1: ...Power Systems PCIe adapters for the 9080 M9S IBM ...

Страница 2: ...l G229 9054 and the IBM Environmental Notices and User Guide Z125 5823 This edition applies to IBM Power Systems servers that contain the POWER9 processor and to all associated models Copyright International Business Machines Corporation 2018 2019 US Government Users Restricted Rights Use duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp ...

Страница 3: ...port name 23 Removing and replacing the tailstock on a PCIe adapter 24 PCIe hot plug manager access for AIX 29 Installing the AIX device driver software 31 Installing or replacing a PCIe adapter with the system power turned on in a Virtual I O Server 33 Verifying that the hot plug PCI tools are installed for Linux 36 Checking the amount of remaining life in NVMe devices 37 Notices 41 Accessibility...

Страница 4: ...iv ...

Страница 5: ...800 300 8751 German safety information Das Produkt ist nicht für den Einsatz an Bildschirmarbeitsplätzen im Sinne 2 der Bildschirmarbeitsverordnung geeignet Laser safety information IBM servers can use I O cards or features that are fiber optic based and that utilize lasers or LEDs Laser compliance IBM servers may be installed inside or outside of an IT equipment rack DANGER When working on or aro...

Страница 6: ...To Disconnect 1 Turn off everything unless instructed otherwise 2 For AC power remove the power cords from the outlets 3 For racks with a DC power distribution panel PDP turn off the circuit breakers located in the PDP and remove the power from the Customer s DC power source 4 Remove the signal cables from the connectors 5 Remove all cables from the devices To Connect 1 Turn off everything unless ...

Страница 7: ... devices that attach to the system It is the responsibility of the customer to ensure that the outlet is correctly wired and grounded to prevent an electrical shock R001 part 1 of 2 R001 part 2 of 2 CAUTION Do not install a unit in a rack where the internal rack ambient temperatures will exceed the manufacturer s recommended ambient temperature for all your rack mounted devices Do not install a un...

Страница 8: ...ds Verify that the route that you choose can support the weight of the loaded rack cabinet Refer to the documentation that comes with your rack cabinet for the weight of a loaded rack cabinet Verify that all door openings are at least 760 x 230 mm 30 x 80 in Ensure that all devices shelves drawers doors and cables are secure Ensure that the four leveling pads are raised to their highest position E...

Страница 9: ...y position for example when working from a ladder Stability hazard The rack may tip over causing serious personal injury Before extending the rack to the installation position read the installation instructions Do not put any load on the slide rail mounted equipment mounted in the installation position Do not leave the slide rail mounted equipment in the installation position L002 L003 or or or Sa...

Страница 10: ... might be equipped with multiple AC power cords or multiple DC power cables To remove all hazardous voltages disconnect all power cords and power cables L003 L007 CAUTION A hot surface nearby L007 x Power Systems PCIe adapters for the 9080 M9S ...

Страница 11: ...king into the other end of a disconnected optical fiber to verify the continuity of optic fibers may not injure the eye this procedure is potentially dangerous Therefore verifying the continuity of optical fibers by shining light into one end and looking at the other end is not recommended To verify continuity of a fiber optic cable use an optical light source and power meter C027 CAUTION This pro...

Страница 12: ...off smooth platforms without appreciable force so take care not to push or lean Keep riser tilt adjustable angling platform option flat at all times except for final minor angle adjustment when needed Do not stand under overhanging load Do not use on uneven surface incline or decline major ramps Do not stack loads Do not operate while under the influence of drugs or alcohol Do not support ladder a...

Страница 13: ...uitable for connection to intrabuilding or unexposed wiring or cabling only The intrabuilding ports of this equipment must not be metallically connected to the interfaces that connect to the OSP outside plant or its wiring These interfaces are designed for use as intrabuilding interfaces only Type 2 or Type 4 ports as described in GR 1089 CORE and require isolation from the exposed OSP cabling The...

Страница 14: ...xiv Power Systems PCIe adapters for the 9080 M9S ...

Страница 15: ...you have the correct PCIe3 cable adapter and expansion drawer cable that works with the PCIe3 6 slot fanout module that you have in your EMX0 PCIe3 expansion drawer If you have a CCIN 50CB PCIe3 6 slot fanout module You must use one of the following PCIe3 cable adapters FC EJ08 CCIN 2CE2 FC EJ07 CCIN 6B52 FC EJ05 CCIN 2B1C You must use one of the following expansion drawer cables FC ECC6 FC ECC7 F...

Страница 16: ...s that are available in the system depends on the number of system processors in the system For the PCIe adapter placement information for the available slots in this system see PCIe adapter placement rules and slot priorities for the 9080 M9S The PCIe adapter locations have six digits The PCIe cassette locations have four digits Figure 1 Locations for the PCIe adapters and cassettes in the 9080 M...

Страница 17: ...m controls the slot continue with step 12 on page 6 7 To prepare the system to install a PCIe adapter when the system is powered off complete the following steps a Activate the identify function For instructions see Identifying a part www ibm com support knowledgecenter POWER9 p9haj sal htm b Physically verify that the slot you identified is where you want to install the adapter Use the blue ident...

Страница 18: ...disconnect both UPIC cables from the system control unit as shown in the following figure a Remove the white plastic lock A from the plug housing as shown in the following figure Figure 3 Removing the UPIC cable lock b Push down on the blue cable latch B and pull out the UPIC cable from the system control unit as shown in the following figure 4 Power Systems PCIe adapters for the 9080 M9S ...

Страница 19: ...dd a PCI Hot Plug Adapter and then press Enter e Select the appropriate PCIe slot from the list that is displayed on the screen and then press Enter f Physically verify that the slot you identified is where you want to install the adapter Use the blue identify LED on the enclosure to locate the system Ensure that the serial number of the system matches the serial number to be serviced Look for a f...

Страница 20: ...x operating system complete the following steps a Log in to the system console as the root user b Run the following command to list the available slots lsslot c pci a The following screen is an example of the information that is displayed by this command Slot Description Device s U7879 001 DQD014E P1 C1 PCI X capable 64 bit 133MHz slot Empty U7879 001 DQD014E P1 C4 PCI X capable 64 bit 133MHz slot...

Страница 21: ...ment bracket A to its raised position b Turn the quarter turn fasteners B to engage and lock the bracket into position Figure 5 Placing the cable management bracket in the raised service position 14 Attach the electrostatic discharge ESD wrist strap The ESD wrist strap must be connected to an unpainted metal surface until the service procedure is completed and if applicable until the service acces...

Страница 22: ...a PCIe adapter in the system complete the steps in this procedure Procedure 1 Ensure that you have the electrostatic discharge ESD wrist strap on and that the ESD clip is plugged into a ground jack or connected to an unpainted metal surface If not do so now 2 If necessary remove the adapter from the antistatic package Attention Avoid touching the components and gold connectors on the adapter 3 Pla...

Страница 23: ...m the 9080 M9S system 5 Remove the side cover from the cassette a Press the release button on the side cover and slide the cover in the direction that is shown b Lift the side cover off the cassette Installing PCIe adapters in the 9080 M9S system 9 ...

Страница 24: ... cassette for the 9080 M9S system 6 Remove the top cover from the cassette a Press the release button on the top cover and slide the cover in the direction that is shown b Lift the top cover off the cassette 10 Power Systems PCIe adapters for the 9080 M9S ...

Страница 25: ...ailstock filler from the cassette a Pull the tailstock filler away from the cassette in the direction that is shown b Save the tailstock filler for a future use You must have a tailstock filler in an empty PCIe cassette in a system to ensure proper airflow Installing PCIe adapters in the 9080 M9S system 11 ...

Страница 26: ... adapter retainers continue with the next step If you have PCIe adapter retainers complete the following steps 1 Ensure that the retainer latches A are in the open position by pushing up on the latch in the direction that is shown in the following figure 2 Slide up the retainers and remove them from the cassette and then discard them 12 Power Systems PCIe adapters for the 9080 M9S ...

Страница 27: ... M9S system so you can slide them off and discard them Note The PCIe adapter retainers are not needed b Press the adapter firmly into its connector as shown in the following figure ensuring that it is seated Installing PCIe adapters in the 9080 M9S system 13 ...

Страница 28: ...he adapter is secured in the adapter retainer channel 9 Replace the top cover on the cassette a Lower the top cover on the top of the cassette b Slide the top cover in the direction that is shown in the following figure until it latches securely 14 Power Systems PCIe adapters for the 9080 M9S ...

Страница 29: ... the 9080 M9S system 10 Replace the side cover on the cassette a Place the side cover on the side of the cassette b Slide the side cover in the direction that is shown in the following figure until it latches securely Installing PCIe adapters in the 9080 M9S system 15 ...

Страница 30: ...Attention Ensure proper alignment when you insert a PCI adapter cassette into the system a Align the cassette on the cassette rail in the cassette slot b Slide the cassette forward until the cassette is fully seated c To lock the adapter in its slot rotate the adapter latch A in the direction that is shown in the following figure 16 Power Systems PCIe adapters for the 9080 M9S ...

Страница 31: ...system http www ibm com support knowledgecenter POWER9 p9egp p9egp_connect_kickoff htm If you have an HMC managed 9040 MR9 or 9080 M9S system you can connect and activate the expansion drawer cable when the system power is turned on Otherwise you must connect the cable with the system power turned off The cable is activated when you start the system 3 Connect any cables or transceivers to the adap...

Страница 32: ...les are present Note Older firmware levels do not support validating the UPIC cables while the system is at FSP standby You can skip this step if it is not available c Expand System Service Aids Cable Plugging Validation In the Display Cable Status section select UPIC Cables and click Continue The system validates that the UPIC cables are installed in the correct locations and displays a table wit...

Страница 33: ...orrect locations Expand System Service Aids Cable Plugging Validation to display a table with the results Ensure that the plugging status is OK for each cable in the displayed table If the status is OK no further action is required If the status is not OK review the error logs correct the problems and repeat steps b c and d as needed until the status is OK for all cables e Exit the ASMI 10 Choose ...

Страница 34: ...e connector do not pull on the wire The cables should be firmly attached to the system b Start the system For instructions see Starting a system www ibm com support knowledgecenter POWER9 p9haj crustartsys htm c Continue with step 18 on page 21 12 If you are using the AIX operating system you must configure the adapter by completing the following steps a Install the AIX device drivers for the inst...

Страница 35: ...turn to the SST main menu e On the System Service Tools SST display select Work with disk units Work with disk configuration Work with NVM Devices Create NVM Namespaces f Select the NVMe device that you installed g On the Create NVM Namespaces display select the quantity and capacity of the namespaces disk units that you want to create on the newly installed NVMe device h Press F10 to confirm the ...

Страница 36: ...rce For racks with a DC power distribution panel PDP disconnect the customer s DC power source to the PDP When connecting power to the product ensure all power cables are properly connected For racks with AC power connect all power cords to a properly wired and grounded electrical outlet Ensure that the outlet supplies proper voltage and phase rotation according to the system rating plate For rack...

Страница 37: ... from static electricity discharge Attach a wrist strap to an unpainted metal surface of your hardware to prevent electrostatic discharge from damaging your hardware When you are using a wrist strap follow all electrical safety procedures A wrist strap is for static control It does not increase or decrease your risk of receiving electric shock when you are using or working on electrical equipment ...

Страница 38: ...ture codes a miscellaneous equipment specification MES also ships with the correct sized tailstock For repairs however replacement parts usually ship with high profile tailstock If the failed adapter was a low profile adapter installed in the system then you must exchange the tall tailstock that was shipped on the replacement card with the low profile tailstock from the failed card Note A Phillips...

Страница 39: ...rd hardware and remove the tailstock 5 If you are removing the tailstock from an Ethernet or Fibre Channel adapter that has small form factor pluggable SFP transceivers you must first remove the SFP transceivers To remove the SFP transceivers complete the following steps a Place the card flat on a static protective surface b Open the latch A by rotating it down and away from the card as shown in F...

Страница 40: ...from a new adapter 7 Ensure that you have the electrostatic discharge ESD wrist strap on and that the ESD clip is plugged into a ground jack or connected to an unpainted metal surface If not do so now 8 Using a Phillips screwdriver unscrew the two retaining screws A that attach the high profile tailstock B to the new PCIe adapter as shown in Figure 20 on page 27 26 Power Systems PCIe adapters for ...

Страница 41: ...ed in Replacing the tailstock in a new adapter 12 Ensure that you have the electrostatic discharge ESD wrist strap on and that the ESD clip is plugged into a ground jack or connected to an unpainted metal surface If not do so now 13 Place the adapter component side up on a flat antistatic surface Attention A pin on the tailstock of the adapter resembles a removable screw Do not remove this pin It ...

Страница 42: ...pter is an Ethernet or Fibre Channel adapter you must attach the SFP transceivers after you replace the tailstock To attach the SFP transceivers complete the following steps a Place the card flat on a static protective surface b Slide the transceiver A into the adapter hardware c With the latch B in the open position push the latch by rotating it up and toward the card as shown in Figure 22 on pag...

Страница 43: ...cing refer to the following placement information PCI adapter placement Accessing hot plug management functions Find information about how to use the PCI hot plug manager to service PCIe adapters with the system power turned on in AIX About this task Note Procedures that are completed with the system power turned on also known as hot plug procedures The hot plug procedure for PCIe adapter in an AI...

Страница 44: ...remove a PCI hot plug adapter Allows the user to remove an existing adapter or replace an existing adapter with an identical one For this option to work the adapter must be in the Defined state see the Unconfigure a Device option You will be asked to identify the PCI slot prior to the actual operation The selected PCI slot will go into the Action state Identify a PCI hot plug slot Allows the user ...

Страница 45: ...ng information Amber LEDs indicate a fault or identify condition If your system or one of the components on your system has an amber LED turned on or flashing identify the problem and take the appropriate action to restore the system to normal Resetting the LEDs in AIX Individual LEDs are located on or near the failing components You can use this procedure to reset the LEDs after you have complete...

Страница 46: ...played The entry fields are automatically updated Press Enter twice to confirm The Command Status window is displayed The message RUNNING is highlighted to indicate that the installation and configuration command is in progress When RUNNING changes to OK scroll to the bottom of the page and locate the installation summary After a successful installation SUCCESS is displayed in the Result column of...

Страница 47: ...ave the software required to support the new adapter and determine whether there are any existing PTF prerequisites to install For information about software prerequisites see the IBM Prerequisite website http www 912 ibm com e_dir eServerPrereq nsf About this task Follow these steps to access the PCIe Hot Plug Manager from the Virtual I O Server Procedure 1 Use the diagmenu command to open the Vi...

Страница 48: ... the Command field at the top of the screen 6 Press F4 or Esc 4 twice to return to the Hot Plug Manager 7 Select replace remove PCIe Hot Plug adapter 8 Select the slot that has the device to be removed from the system 9 Select replace A fast blinking amber LED located at the back of the machine near the adapter indicates that the slot has been identified 10 Press Enter which places the adapter in ...

Страница 49: ...tions are not available the client logical partitions can fail or they might be unable to perform I O operations for a particular application If you use the HMC to manage the system you might have redundant Virtual I O Server logical partitions which allow for Virtual I O Server maintenance and avoid downtime for client logical partitions If you are replacing an adapter on the Virtual I O Server a...

Страница 50: ...y that the hot plug PCI tools are installed rpm aq grep powerpc utils If the command does not list any powerpc utils packages the PCI Hot Plug tools are not installed 2 Enter the following command to ensure that the rpaphp driver is loaded ls l sys bus pci slots The directory should contain data If the directory is empty the driver is not loaded or the system does not contain hot plug PCI slots Th...

Страница 51: ...e become slower over time and at some point the NVMe device becomes a read only device When the operating system writes data to a read only device the write operations are rejected and the operating system considers the device as if a failure occurred To support normal write operations the NVMe device must be replaced Note Failure of IBM NVMe devices is covered in the standard warranty and during ...

Страница 52: ...and must be replaced The NVMe device will soon reach the limit for the number of write operations that are supported Write operations to the NVMe device become slower over time and at some point the NVMe device becomes a read only device When the operating system writes data to a read only device the write operations are rejected and the operating system considers the device as if a failure occurr...

Страница 53: ...9 3 The NVMe device is nearing its end of life and must be replaced The NVMe device will soon reach the limit for the number of write operations that are supported Write operations to the NVMe device become slower over time and at some point the NVMe device becomes a read only device When the operating system writes data to a read only device the write operations are rejected and the operating sys...

Страница 54: ...es data to a read only device the write operations are rejected and the operating system considers the device as if a failure occurred To support normal write operations schedule maintenance to replace the NVMe device at your earliest convenience Note Failure of IBM NVMe devices is covered in the standard warranty and during the maintenance period only for devices that have not reached the maximum...

Страница 55: ...uct s and or the program s described in this publication at any time without notice Any references in this information to non IBM websites are provided for convenience only and do not in any manner serve as an endorsement of those websites The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk IBM may use or distribute any o...

Страница 56: ...n to use information technology content successfully Overview The IBM Power Systems servers include the following major accessibility features Keyboard only operation Operations that use a screen reader The IBM Power Systems servers use the latest W3C Standard WAI ARIA 1 0 www w3 org TR wai aria to ensure compliance with US Section 508 www access board gov guidelines and standards communications a...

Страница 57: ...nologies to collect personally identifiable information If the configurations deployed for this Software Offering provide you as the customer the ability to collect personally identifiable information from end users via cookies and other technologies you should seek your own legal advice about any laws applicable to such data collection including any requirements for notice and consent For more in...

Страница 58: ...n sind die Geräte wie in den Handbüchern beschrieben zu installieren und zu betreiben Des Weiteren dürfen auch nur von der IBM empfohlene Kabel angeschlossen werden IBM übernimmt keine Verantwortung für die Einhaltung der Schutzanforderungen wenn das Produkt ohne Zustimmung von IBM verändert bzw wenn Erweiterungskomponenten von Fremdherstellern ohne Empfehlung von IBM gesteckt eingebaut werden EN ...

Страница 59: ...ctronics and Information Technology Industries Association JEITA Notice This statement applies to products less than or equal to 20 A per phase This statement applies to products greater than 20 A single phase This statement applies to products greater than 20 A per phase three phase Japan Voluntary Control Council for Interference VCCI Notice Korea Notice Notices 45 ...

Страница 60: ...ct the interference at his own expense Properly shielded and grounded cables and connectors must be used in order to meet FCC emission limits Proper cables and connectors are available from IBM authorized dealers IBM is not responsible for any radio or television interference caused by using other than recommended cables and connectors or by unauthorized changes or modifications to this equipment ...

Страница 61: ...te wie in den Handbüchern beschrieben zu installieren und zu betreiben Des Weiteren dürfen auch nur von der IBM empfohlene Kabel angeschlossen werden IBM übernimmt keine Verantwortung für die Einhaltung der Schutzanforderungen wenn das Produkt ohne Zustimmung von IBM verändert bzw wenn Erweiterungskomponenten von Fremdherstellern ohne Empfehlung von IBM gesteckt eingebaut werden Deutschland Einhal...

Страница 62: ... to products greater than 20 A per phase three phase Japan Voluntary Control Council for Interference VCCI Notice Taiwan Notice United States Federal Communications Commission FCC Notice This equipment has been tested and found to comply with the limits for a Class B digital device pursuant to Part 15 of the FCC Rules These limits are designed to provide reasonable protection against harmful 48 Po...

Страница 63: ...York 10504 Contact for FCC compliance information only fccinfo us ibm com Terms and conditions Permissions for the use of these publications are granted subject to the following terms and conditions Applicability These terms and conditions are in addition to any terms of use for the IBM website Personal Use You may reproduce these publications for your personal noncommercial use provided that all ...

Страница 64: ...50 Power Systems PCIe adapters for the 9080 M9S ...

Страница 65: ......

Страница 66: ...IBM ...

Отзывы: