Nvidia QM87 Series User Manual Download Page 2

Page: 2 / 51

Table of Contents

Introduction..................................................................................... 5

Speed and Switching Capabilities .................................................................6

Management Interfaces, PSUs and Fans ..........................................................6

Features ...............................................................................................7

Certifications .........................................................................................7

Installation ...................................................................................... 8

System Installation and Initialization .............................................................8

Safety Warnings ......................................................................................8

Air Flow................................................................................................8

Package Contents ....................................................................................9

19” System Mounting Options

.................................................................... 10

Fixed Rail Kit ....................................................................................... 10

Removing the System from the Rack........................................................ 15

Telescopic Rail Kit ................................................................................. 15

Removing the System from the Rack........................................................ 19

Cable Installation .................................................................................. 20

Splitter (Breakout) Cables and Adapters ................................................... 21

Initial Power On .................................................................................... 23

System Bring-Up of Managed Systems .......................................................... 24

Configuring Network Attributes.............................................................. 24

Remote Connection............................................................................ 27

FRU Replacements ................................................................................. 28

Power Supply ................................................................................... 28

Fans .............................................................................................. 29

Software Management .......................................................................31

InfiniBand Subnet Manager ....................................................................... 31

Upgrading Software (on Managed Systems) .................................................... 31

Updating Firmware on Externally Managed Systems ......................................... 32

Updating Firmware In-band (Typical) ....................................................... 32

Interfaces ......................................................................................34

Data Interfaces ..................................................................................... 34

Speed ............................................................................................ 34

Summary of Contents for QM87 Series

Page 1: ...QM87xx 1U HDR 200Gb s InfiniBand Switch Systems User Manual QM8700 QM8790 1U HDR 200Gb s InfiniBand Switch Systems User Manual...

Page 2: ...he Rack 15 Telescopic Rail Kit 15 Removing the System from the Rack 19 Cable Installation 20 Splitter Breakout Cables and Adapters 21 Initial Power On 23 System Bring Up of Managed Systems 24 Configur...

Page 3: ...8 Unit Identification LED 39 Port LEDs 40 Inventory Pull out Tab 41 Troubleshooting 42 Specifications 43 Appendixes 44 Accessory and Replacement Parts 44 Thermal Threshold Definitions 44 Interface Spe...

Page 4: ...itch 40 QSFP56 ports 2 Power Supplies AC unmanaged standard depth P2C airflow Rail Kit 920 9B110 00 RH 0D0 MQM8790 HS2R Mellanox Quantum HDR InfiniBand Switch 40 QSFP56 ports 2 Power Supplies AC unman...

Page 5: ...tocol SHARP technology SHARP architecture enables the usage of all active data center devices to accelerate the communications frameworks resulting in order of magnitude applications performance impro...

Page 6: ...m model System Model HDR 200Gb s QSFP56 Interfaces Max Throughput QM8700 40 16Tb s QM8790 40 16Tb s Management Interfaces PSUs and Fans The table below lists the various management interfaces and avai...

Page 7: ...n the main menu click on Products InfiniBand VPI Switch Systems and select the desired product page Certifications The list of certifications such as EMC Safety and others per system for different reg...

Page 8: ...wer On 6 Perform system bring up System Bring Up of Managed Systems 7 Optional FRU replacements FRU Replacements Safety Warnings Prior to the installation please review the Safety Warnings Note that s...

Page 9: ...o power side outlet Red latches are placed on the power inlet side OPN designation is R Power side inlet to connector side outlet Blue latches are placed on the power inlet side OPN designation is F P...

Page 10: ...ctions see Telescopic Rail Kit Fixed Rail Kit Kit OPN Legacy Kit OPN Rack Size and Rack Depth Range 930 9BRKT 00JF 000 MTEF KIT C 430 800 mm The following parts are included in the fixed rail kit see...

Page 11: ...tallation selection It is important to keep the airflow within the rack in the same direction Note that the part of the system to which you choose to attach the rails the front panel direction as demo...

Page 12: ...s Short Racks 430 580 mm Installation Side View Front side ports Rear side FRUs Standard Racks 580 800 mm Installation Side View In short racks the system s ventilation openings should be framed by th...

Page 13: ...he Rails to the Chassis Attach the left and right rack mount ears C to the switch by gently pushing the switch chassis pins through the slider key holes until locking occurs Secure the system in the b...

Page 14: ...is supporting the system s weight perform the following steps Attach the two rack mount blades B to the back side FRU side of the rack by inserting four M6 screws E in the designated cage nuts Do not...

Page 15: ...stem s weight Loosen the screws attaching the rack mount ears C to the rack Do not remove them yet Loosen the screws attaching the rack mount blades B to the rack and pull the blades towards you while...

Page 16: ...ont rail D 10x M6 Standard cage nuts E 10x M6 Standard pan head Phillips screws F 2x Phillips100 DEG F H TYPE I ST ST 6 32 X 1 4 screw with around patch G 6x Flat head 100 DEG Phillips 4 40X3 16 ST ST...

Page 17: ...thin the rack or in case more space is needed for cable bending radius it is possible to recess the connector side or the FRU side by 3 15 8cm by optional placement of the system s rails The FRU side...

Page 18: ...ng the Inner Rails for Cable Accommodation Route the power cable through either of the inner rails and reassemble the brackets by screwing the 3 screws per rail provided with the rail kit H with a tor...

Page 19: ...ening the 8 screws inserted in Step 2 with a torque of 4 5 0 5 Nm Removing the System from the Rack To remove a unit from the rack Turn off the system and disconnect it from peripherals and from the e...

Page 20: ...en the physical connection is established When a logical connection is made the relevant port LED will turn on To remove a cable disengage the locks and slowly pull the connector away from the port re...

Page 21: ...he port prior to the split and z indicating the number of the resulting single lane port 1 2 Each sub physical port is then handled as an individual port For example splitting port 5 into 2 lanes give...

Page 22: ...o two 2X HDR100 ports The following diagrams attempt to show how the logical ports map onto the physical QSFP ports as viewed by the IB tools e g ibnetdiscover Switch Profile Non Splittable Suitable f...

Page 23: ...r Inlets Electric Caution Notifications MLNX OS will refer to this 4X port as 1 13 The system platform will automatically power on when AC power is applied There is no power system Check all boards po...

Page 24: ...ind the explanation in Disable Dynamic Host Configuration Protocol DHCP sufficient In case manual configuration is required please refer to the instructions in Manual Host Configuration Disable Dynami...

Page 25: ...s described in the table below Once you perform that you should get the CLI prompt of the system Serial Terminal Program Configuration Parameter Setting Baud Rate 115200 Data bits 8 Stop bits 1 Parity...

Page 26: ...ble DHCPv6 on the MGMT0 interface Step 6 Admin password Press Enter to leave unchanged new_password Step 6 Confirm admin password new_password To avoid illegal access to the machine please type a pass...

Page 27: ...Link up yes DHCP running yes IP address 10 209 28 50 Netmask 255 255 255 0 IPv6 enabled yes Autoconf enabled no Autoconf route yes Autoconf privacy no DHCPv6 running no IPv6 addresses 1 IPv6 address...

Page 28: ...t may be extracted without bringing down the system To extract a power supply unit Remove the power cord from the power supply unit Grasping the handle with your hand push the latch release with your...

Page 29: ...ush the latch release with your thumb while pulling the handle outward As the fan unit unseats the fan unit status LEDs will turn off Remove the fan unit Fan Module Latches To remove or replace a fan...

Page 30: ...resistance is felt Continue pressing the fan unit until it seats completely The green Fan Status LED should light If not extract the fan unit and reinsert it After two unsuccessful attempts to install...

Page 31: ...e subnet Each network requires a Subnet Manager to be running in either the system itself system based or on one of the nodes which is connected to the fabric host based The subnet manager OpenSM assi...

Page 32: ...manual are available for download under https network nvidia com products adapter software firmware tools Please select the package that suits your operating system In order to obtain information rega...

Page 33: ...vidia com support firmware firmware downloads select the Quantum System page If the current version is not the latest version follow the directions in the MFT User manual to burn the new firmware inba...

Page 34: ...otocol Mellanox systems support QDR FDR EDR HDR InfiniBand FDR is an InfiniBand data rate where each lane of a 4X port runs a bit rate of 14 0625Gb s with 64b 66b encoding resulting in an effective ba...

Page 35: ...the Console connector and is located on the front side of the system the RJ45 connector It can be used with the I C DB9 to RJ45 splitting harness This interface is not found in externally managed syst...

Page 36: ...e user admin LEDs See LED Notifications LED Notifications The system s LEDs are an important tool for hardware event notification and troubleshooting LEDs Symbols Symbol Name Description Normal Condit...

Page 37: ...Solid Amber Major error has occurred For example corrupted firmware system is overheated etc If the System Status LED shows amber five minutes after starting the system unplug the system and call you...

Page 38: ...uld be replaced Power Supply Status LEDs There are two power supply inlets in the system for redundancy The system can operate with only one power supply connected Each power supply unit has a single...

Page 39: ...AC cord unplugged or AC power lost while the second power supply still has AC input power Plug in the AC cord of the faulty PSU PS failure including voltage out of range and power cord disconnected Ch...

Page 40: ...of a single 4 lane port or of the higher 2 lane split port if a splitter cable is used Splitting Indication LEDs Each time you press on the Lane Select Button the Port LEDs display will switch to a d...

Page 41: ...able is plugged into the port with the other end of the connector plugged into a functioning port When a logical connection is made the LED will change to green When data is being transferred the ligh...

Page 42: ...e power cable Replace the PSU if needed The activity LED does not light up InfiniBand Make sure that there is an SM running in the fabric System boot failure The last software upgrade failed on x86 ba...

Page 43: ...RoHS compliant Power Input Voltage 100 127VAC 50 60Hz 4 5A 200 240 50 60Hz 4 4A Global Power Consumption Full power specifications are provided in NVIDIA QM87XX Full Technical Specifications document...

Page 44: ...MTEF PSF AC C 200G 1U systems 1100W AC Power Supply w P2C airflow 930 9BPSU 00JG 0 00 MTEF PSR AC C 200G 1U systems 1100W AC Power Supply w C2P airflow HAR000631 Harness RS232 2M cable DB9 to RJ 45 fo...

Page 45: ...reshold the device will auto shutdown upon crossing the Emergency 130 C threshold Interface Specifications QSFP Pin Description QSFP Pin Description Connector Pin Number Pin Name Signal Description 1...

Page 46: ...ND Ground 21 Rx2n Receiver Inverted Data Output 3 22 Rx2p Receiver Non Inverted Data Output 3 23 GND Ground 24 Rx4n Receiver Inverted Data Output 3 25 Rx4p Receiver Non Inverted Data Output 3 26 GND G...

Page 47: ...upplied RJ45 to DB9 Harness Pinout RJ 45 Console and I C interfaces are integrated in the same connector Due to that connecting any cable other than the Mellanox supplied console cable may cause an I...

Page 48: ...uts Disposal According to the WEEE Directive 2002 96 EC all waste electrical and electronic equipment EEE should be collected separately and not disposed of with regular household waste Dispose of thi...

Page 49: ...I C under Interfaces Added power consumption measuring details in Specifications January 2020 1 6 Added a warning to Interfaces under I2C section Interface Specifications under RJ45 to DB9 Harness Pi...

Page 50: ...by customer and perform the necessary testing for the application in order to avoid a default of the application or the product Weaknesses in customer s product designs may affect the quality and rel...

Search results

Summary of Contents for QM87 Series

Reviews:

Related manuals for QM87 Series

Brands by name

Popular brands

Nvidia QM87 Series, User Manual

Search results

Summary of Contents for QM87 Series

Reviews:

Related manuals for QM87 Series

Brands by name

Popular brands