Mellanox Technologies MetroX MTX6100-2SFS Hardware User Manual Download Page 1

Page: 1 / 71

www.mellanox.com

MetroX™ InfiniBand Long Haul System

Hardware User Manual

P/N: MTX6100-2SFS

Rev 1.1

Summary of Contents for MetroX MTX6100-2SFS

Page 1: ...www mellanox com MetroX InfiniBand Long Haul System Hardware User Manual P N MTX6100 2SFS Rev 1 1...

Page 2: ...KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SE...

Page 3: ...Power Budget Classification 20 2 4 Interfaces 20 2 4 1 Port Connector Interfaces 20 2 4 2 Management and Firmware Updating Interfaces 21 2 4 3 Reset Button 22 Chapter 3 Installation 24 3 1 Installati...

Page 4: ...are 49 9 2 Switch Management 49 9 2 1 InfiniBand Subnet Manager 50 9 3 Fabric Inspector Diagnostics 50 9 3 1 Upgrading Software 50 Chapter 10 Troubleshooting 51 Appendix A Specification 54 A 1 Approve...

Page 5: ...gure 14 Top and Bottom QSFP Port Orientation 21 Figure 15 Management Interfaces 21 Figure 16 Reset Button 23 Figure 17 Rack Rail Kit Parts 28 Figure 18 Placement of Switch in Rack 29 Figure 19 Mountin...

Page 6: ...ical Link Assignments for IB 18 Table 8 Air Flow Direction 20 Table 9 Installation Kit According to Rack Size 27 Table 10 Serial Terminal Program Configuration 34 Table 11 Configuration Wizard Session...

Page 7: ...Long Haul System Hardware User Manual Rev 1 1 Mellanox Technologies 7 Revision History Table 1 Revision History Table Date Revision Description January 2013 1 1 Fixed 2 power supplies no wire bale Dec...

Page 8: ...to indicate information that is important to the user Table 2 Reference Documents and Web Sites Document Name Description InfiniBand Architecture Specifica tion Volume 1 Release 1 2 1 The InfiniBand A...

Page 9: ...EWARE This icon indicates a situation that can potentially cause personal injury or damage to hardware or software Place Field Decoder M Mellanox TX System Type MetroX based systems 6 Family generatio...

Page 10: ...nces of up to 10km The switches enable aggregate data and storage networking over a single consolidated InfiniBand fabric The long haul InfiniBand technology guarantees high performance high vol ume d...

Page 11: ...DR 6 Uplink QSFP non blocking ports with aggregate data throughput up to 240Gb s Compliant with IBTA 1 2 1 and 1 3 Virtual Lanes 2 data 1 management 1 2 Serial Number and Product Version Information T...

Page 12: ...or for connecting to a host PC 1 USB connector All switches have various status LEDs for on site switch status information Figure 3 QSFP Switch System Power and Connector Side Panels All connectivity...

Page 13: ...troubleshoot the fan module See Troubleshooting on page 51 If the PS LEDs are not green this indicates a problem with the power supplies The switch is operational only if at least one of the PS LEDs...

Page 14: ...ssignments LED Configuration STATUS System Health LED Solid Green OK The system is up and running Flashing Green The system is booting up This assignment is valid on managed systems only Solid Yellow...

Page 15: ...K All fans are up and running Solid Red Error One or more fans is not operating properly The system should be powered down and troubleshoot the fan module Off Off The fan unit is not receiving any pow...

Page 16: ...side panel with PS2 on the left side Each PS unit has a single 2 color LED on the right side of the PS unit that indicates the internal status of the unit Figure 9 PS Unit Status LEDs RST UID PS2 PS1...

Page 17: ...D LED is a debug feature that lights a blue LED on the switch connector side panel for ease in finding a particular switch Table 5 PS Unit Status LED Assignments LED Color Status Solid Green OK The Po...

Page 18: ...rt with the other end of the connector plugged into a functioning port When a logical connection is made the LED will change to green When data is being transferred the light will blink green 2 2 Air...

Page 19: ...mbering Legend on page 9 On the PS unit and fan modules the air flow direction can be seen on the power side panel Figure 12 All Components Need the Same Air Flow All PS units must have the same air f...

Page 20: ...r per port is 2 5W 2 4 Interfaces 2 4 1 Port Connector Interfaces The connector side of the switch has 12 QSFP ports The 6 on the left side numbered 1 6 are long haul ports that require 3 5W max power...

Page 21: ...connector to connect to the host PC 1 reset button 1 I2C banana connector on the power supply side for FAE use only Figure 15 Management Interfaces 2 4 2 1 RJ 45 Connector CONSOLE The port labelled C...

Page 22: ...ctor on the far left of the power side of the switch This interface is for Debug and Troubleshooting only This connector can be used to install firmware upgrades should the firmware image be damaged a...

Page 23: ...resets all of the ports by bringing them down and powering them up when the button is pushed A quick push of this button performs this reset When the button is held down for 15 sec onds the switch is...

Page 24: ...e Section G Advertencias de seguridad para la instalaci n Span ish on page 69 1 Installation Instructions 2 Over temperature 3 Stacking the Chassis 4 Redundant Power Supply Connection Electrical Hazar...

Page 25: ...ck special precautions must be taken to ensure that the system remains stable In general you should fill the rack with equip ment starting from the bottom to the top This equipment should be installed...

Page 26: ...r European connection select a power supply cord that is internationally harmonized and marked HAR 3 conductor minimum 0 75 mm2 wire rated at 300 V with a PVC insulated jacket The cord must have a mol...

Page 27: ...d using the long rail kit 3 2 Package Contents Before you install your new TX6100switch unpack the system and check to make sure that all the parts have been sent check this against the parts list bel...

Page 28: ...p Make sure the ESD strap is touching your skin and that the other end is connected to a verified ground 2 Choose which side of the switch you want even with the rack vertical support Either the side...

Page 29: ...e switch Do you want the connector side recessed in the rack to allow for a larger cable bending radius It is possible to recess the connector side by 5cm by optional placement of the switch rails See...

Page 30: ...for standard depth switches to connect each switch slide Figure 20 Screwing on the Rail 5 screws per side are needed for the short switch 7 screws per side are needed for the standard switch There are...

Page 31: ...ore caged nuts into the holes on the opposite side of the rack Check that both sides of the switch left and right are the same level in the rack 7 Slide the rail into the rail slide 8 Using two of the...

Page 32: ...connect the other end to a valid ground If you choose to not use the ground screw make sure that the rack is properly grounded and that there is a valid ground connection between the chassis of the s...

Page 33: ...indicator shows green If the fan status indicator is not green then unplug the power connection and check that the fan module is inserted properly and that the mating connector of the fan unit is fre...

Page 34: ...password This starts the Mellanox configuration wizard Step 4 Go through the Mellanox configuration wizard Table 11 shows an example of a wizard session Make sure to connect to the Console RJ 45 port...

Page 35: ...e and press Enter Step 2 Use DHCP on mgmt0 interface no Perform this step to obtain an IP address for the switch eth0 is the management port of the switch If you wish the DHCP server to assign the IP...

Page 36: ...d Enter to leave unchanged CHANGED To change an answer enter the step number to return to Otherwise hit enter to save changes and exit Choice Enter Configuration changes saved To return to the wizard...

Page 37: ...tep 8 Enable IPv6 autoconfig SLAAC on mgmt0 interface no no Step 9 Admin password Enter to leave unchanged Step 9 Confirm admin password Enter to leave unchanged You have entered the following informa...

Page 38: ...Step 6 Default gateway for example 192 168 10 1 Step 7 Primary DNS server Step 8 Domain name Step 9 Enable IPv6 yes yes Step 10 Enable IPv6 autoconfig SLAAC on mgmt0 interface no no Step 11 Admin pas...

Page 39: ...k 255 255 255 0 Speed 1000Mb s auto Duplex full auto Interface type ethernet Interface source physical MTU 1500 HW address 00 02 C9 11 2A AE Comment RX bytes 1343502058 TX bytes 313920869 RX packets 1...

Page 40: ...k machine the remote machine henceforth using a standard RJ 45 connector Step 2 Connect to the remote machine rem_mach1 is used as an example Step 3 Start a remote shell to the switch using the follow...

Page 41: ...low Direction Figure 27 Power Supply Unit Extraction Make sure that the PS unit that you are NOT replacing is showing all green for both the PS unit and status indicators Power supply units have direc...

Page 42: ...pressing the PS unit until it seats completely The latch will snap into place confirm ing the proper installation 4 Insert the power cord into the supply connector 5 Insert the other end of the power...

Page 43: ...Unit 1 Make sure the mating connector of the new unit is free of any dirt and or obstacles 2 Insert the fan unit by sliding it into the opening until slight resistance is felt Continue press ing the f...

Page 44: ...t Fanout Cables Specific ports can be split by using a QSFP 1X4 breakout cable to split one 40 GbE port into 4 lanes 4 SFP connectors 10GbE each These 4 lanes then go one lane to each of the 4 SFP con...

Page 45: ...ing port After plugging in a cable lock the connector using the latching mechanism particular to the cable vendor When a logical connection is made the LED will change to green When data is being tran...

Page 46: ...Technologies 46 6 Switch Shut Down Procedure To shut down the switch run the command Reload halt noconfirm The switch cannot be restarted remotely To restart the switch you must physically go to the s...

Page 47: ...l slides from the rack 7 Remove the ten caged nuts 7 1 Disposal Follow the instructions found in the Mellanox Web site at mellanox com for proper instructions to disassemble and dispose of the Switch...

Page 48: ...them down and powering them up when the button is pushed A quick push of this button performs this reset When the button is held down for 15 sec onds the switch is reset and the password is deleted Yo...

Page 49: ...s a CLI WebUI SNMP and chassis manage ment software and IB management software OpenSM The IB Subnet Manager running on the TX6100 switches supports up to 648 nodes If the fabric includes more than 648...

Page 50: ...st based 9 3 Fabric Inspector Diagnostics Fabric Inspector is a plug play software for fee within MLNX OS displaying and filtering all identified systems and nodes within the fabric Fabric Inspector i...

Page 51: ...that the fan mod ules and ventilation holes are not blocked especially dust over the holes 3 If you find dust blocking the holes it is recommended to clean the fan unit and remove the dust from the f...

Page 52: ...to determine the cause of the shutdown The last software update did not succeed 1 Connect the RS232 connector CONSOLE to a laptop 2 Push the reset button on the switch or management module 3 You will...

Page 53: ...elect the image to boot PCIE1 successfully set as root complex 01 00 15b3 bd34 0c06 00 Net ppc_4xx_eth0 ppc_4xx_eth1 Hit Ctrl B to stop autoboot 0 Mellanox MLNX OS Boot Menu 1 EFM_PPC_M460EX EFM_1 1 1...

Page 54: ...nd Environmental Input Voltage Management CPU Input Voltage 100 240 VAC 50 60Hz Management CPU PowerPC 460EX CPU Power numbers Power Consumption PS unit fan is always at 80 Typ Typ 56Gb s Active Cable...

Page 55: ...ssions Safety US Canada cTUVus EU IEC60950 International CB Russia GOST R Argentina S mark DoC EMC Emissions USA FCC Class A Canada ICES Class A EU EN55022 Class A EU EN55024 Class A EU EN61000 3 2 Cl...

Page 56: ...Rev 1 1 Mellanox Technologies 56 A 3 China CCC Warning Statement...

Page 57: ...Transmitter Inverted Data Input 6 Tx4p Transmitter Non Inverted Data Input 7 GND Ground 8 ModSelL Module Select 9 ResetL Module Reset 10 Vcc Rx 3 3 V Power supply receiver 11 SCL 2 wire serial interf...

Page 58: ...8 3 3 3 3 3 3 3 3 RX4p GND GND GND RX4n RX2p RX2n TX1n GND TX1p GND TX3n TX3p GND IntL ModPrsL VccTx Vcc1 LPMode 18 35 8 50 Top ResetL SDA SCL ModSelL RX3p TX2n GND VccRx 1 2 3 4 5 6 7 8 9 1 0 1 2 3...

Page 59: ...ing TXD TXD RXD GND GND RXD GND GND RJ45_RTS RJ45_DTR RJ45_TXD GND GND RJ45_RXD GND GND J34 Pin 1 Pin 8 Looking into the Socket CONSOLE TXD TXD RXD GND GND RXD GND GND RJ45_SDA GND GND GND GND RJ45_SC...

Page 60: ...side air flow MSX60 FF Fan module with connector side to power supply side air flow MSX60 FR 300W Power Supply w Power Supply Side to Connector side air flow MSX60 PF 300W Power Supply w Connector si...

Page 61: ...cette temp rature ambiante maximale est n cessaire En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouver tures de ventilation Le ch ssis ne d...

Page 62: ...tes les alimentations stabilise es Risk of electric shock and energy hazard The PSUs are all independent Disconnect all power supplies to ensure a powered down state inside of the switch platform Lors...

Page 63: ...tionaux du pays Pour l Am rique du Nord l quipement doit tre install conform ment aux sp cifications du Code Electrique National Am ricain et du Code Electrique Canadien Les c bles de branchement l un...

Page 64: ...et d une longueur maxi male de 4 5 m 18 pieds Pour les prises lectriques en Europe choisissez un cordon lectrique harmonis inter nationalement et marqu HAR 3 conducteurs d un diam tre de fil minimum d...

Page 65: ...cm 3 in Freiraum um die Bel ftungs ffnungen sein um einen einwandfreien Luftstrom zu gew hrleisten Das Chassis sollte nicht auf andere Ger te gestapelt werden Wenn das Chassis herun terf llt kann es...

Page 66: ...h oben mit Ger ten f llen Diese Ger t sollte nur von geschultem und qualifiziertem Personal installiert ausge tauscht oder gewartet werden Die Entsorgung dieses Ger ts sollte unter Beachtung aller nat...

Page 67: ...arth Verbindung bevor Sie die Verbindung von wesentlicher Bedeutung werden Bevor Sie dieses Ger t an das Stromnetz die Schutzerde Terminal Schrauben dieses Ger t muss an den Schutzleiter in der Geb ud...

Page 68: ...Rev 1 1 Mellanox Technologies 68 20 Bodily Injury Due to Weight Use enough people to safely lift this product 40 lbs 18 kgs 40 70 lbs 18 32 kgs 70 121 lbs 32 55 kgs 121 lbs 55 kgs...

Page 69: ...and should be moved using a mechanical lift to avoid injuries No se debe utilizar el equipo en un rea con una temperatura ambiente superior a la m xima recomendada 45 C Adem s para garantizar una circ...

Page 70: ...les consultar al fabricante del cable Al instalar o realizar el mantenimiento de este aparato en un bastidor es preciso adop tar precauciones especiales para garantizar que el sistema se mantenga esta...

Page 71: ...ber prover un medio para la conexi n con la puesta a tierra de protecci n y un t cnico de servicio deber conectar permanentemente el equipo a dicho medio de conexi n Un T CNICO DE SERVICIO comprobar s...

Search results

Summary of Contents for MetroX MTX6100-2SFS

Reviews:

Related manuals for MetroX MTX6100-2SFS

Brands by name

Popular brands

Mellanox Technologies MetroX MTX6100-2SFS, Hardware User Manual

Search results

Summary of Contents for MetroX MTX6100-2SFS

Reviews:

Related manuals for MetroX MTX6100-2SFS

Brands by name

Popular brands