manualshive.com logo in svg

Mellanox Technologies Mellanox ConnectX MCX556M-ECAT-S25, Руководство пользователя

Поделиться
Скачать
Mellanox Technologies Mellanox ConnectX MCX556M-ECAT-S25 Скачать руководство пользователя страница 6

Rev 1.6

4

Mellanox Technologies

List of Tables 

Table 1:

Revision History Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Table 2:

Documents List. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Table 3:

Dual-port VPI Adapter Cards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Table 4:

ConnectX-5 Socket Direct Package Contents  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Table 5:

Features  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Table 6:

Hardware and Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Table 7:

Installation Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Table 8:

mlnxofedinstall Return Codes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Table 9:

Hardware and Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Table 10: General Troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

Table 11: Linux Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Table 12: Windows Troubleshooting  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Table 13: MCX556M-ECAT-S25 Specification Table  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Table 14: Airflow Specifications  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

Table 15: Physical and Logical Link Indications (Ethernet Mode). . . . . . . . . . . . . . . . . . . . . . 62

Table 16: Physical and Logical Link Indications (InfiniBand Mode)  . . . . . . . . . . . . . . . . . . . . 62

Table 17: Connector Pin Number and Name to Signal Name Map  . . . . . . . . . . . . . . . . . . . . 66

Table 18: Slim-Line SAS Pinout Connector on the Adapter Card . . . . . . . . . . . . . . . . . . . . . . 69

Table 19: Slim-Line SAS Pinout Connector on the Auxiliary PCIe Connection Card  . . . . . . . 71

Содержание Mellanox ConnectX MCX556M-ECAT-S25

Страница 1: ...Mellanox Technologies www mellanox com Mellanox ConnectX 5 VPI Dual Port Socket Direct Adapter Card User Manual for Dual Socket Servers P N MCX556M ECAT S25 Rev 1 6...

Страница 2: ...pdated list of Mellanox trademarks visit http www mellanox com page trademarks NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNO...

Страница 3: ...2 3 PCI Express Interface 15 2 4 LED Interface 15 Chapter 3 Hardware Installation 16 3 1 System Requirements 16 3 1 1 Hardware 16 3 1 2 Operating Systems Distributions 16 3 1 3 Software Stacks 16 3 2...

Страница 4: ...1 5 4 Installing MLNX_OFED Using apt get Tool 40 4 1 5 5 Setting up MLNX_OFED apt get Repository 40 4 1 5 6 Installing MLNX_OFED Using the apt get Tool 40 4 1 5 7 Uninstalling Mellanox OFED Using the...

Страница 5: ...61 7 2 Airflow Specifications 62 7 3 Adapter Card LED Operations 62 7 4 Board Mechanical Drawing and Dimensions 63 7 5 Bracket Mechanical Drawing 65 Appendix A Interface Connectors Pinout 66 A 1 QSFP...

Страница 6: ...nd Software Requirements 44 Table 10 General Troubleshooting 58 Table 11 Linux Troubleshooting 59 Table 12 Windows Troubleshooting 60 Table 13 MCX556M ECAT S25 Specification Table 61 Table 14 Airflow...

Страница 7: ...Drawing of the Auxiliary PCIe Connection Card 63 Figure 5 Mechanical Drawing of the Slim Line SAS Cable 64 Figure 6 Adapter Card Tall Bracket 65 Figure 7 Auxiliary PCIe Connection Card Tall Bracket 65...

Страница 8: ...n page 44 January 2018 1 5 Added a note to System Requirements on page 16 Updated MCX556M ECAT S25 Specifications on page 61 Added a note to Board Mechanical Drawing and Dimensions on page 63 Updated...

Страница 9: ...irmware Utility mlxup User Manual and Release Notes Mellanox firmware update and query utility used to update the firmware See http www mellanox com Products Software Firmware Tools mlxup Firmware Uti...

Страница 10: ...purchased Mellanox products through a Mellanox approved reseller should first seek assistance through their reseller Firmware Updates The Mellanox support downloader contains software firmware and kn...

Страница 11: ...talled in two PCIe slots each card gets a unique PCI Bus and Device number Each of the PCIe x8 busses sees two network ports in effect the two 100Gb s physical ports of the ConnectX 5 Socket Direct Ad...

Страница 12: ...R QDR FDR EDR Ethernet 1 10 25 40 50 100 Gb s Network Connector Types Dual port QSFP28 located on the low profile adapter card PCI Express PCIe SerDes Speed Dual PCIe 3 0 x8 8GT s RoHS R6 Adapter IC P...

Страница 13: ...Socket Direct supporting dual socket server EDR IB 100Gb s and 100GbE dual port QSFP28 2x PCIe3 0 x8 tall bracket ROHS R6 1 SA002142 Auxiliary PCIe Connection Card 1 HAR000629 25cm Slim Line SAS Harn...

Страница 14: ...IEEE 802 3ae 10 Gigabit Ethernet IEEE 802 3ap based auto negotiation and KR startup Proprietary Ethernet protocols 20 40GBASE R2 50GBASE R4 IEEE 802 3ad 802 1AX Link Aggregation IEEE 802 1Q 802 1P VL...

Страница 15: ...ad using ASAP2 TM Flexible match action flow tables Tunneling encapsulation decapsulation Quality of Service QoS Support for port based Quality of Service enabling various application requirements for...

Страница 16: ...sive copper cable with ESD protection Powered connectors for optical and active cable support 1 6 Manageability Socket Direct technology maintains support for manageability through a BMC The Socket Di...

Страница 17: ...t standards listed in Table 5 Features on page 12 Ethernet traffic is transmitted through the cards SFP28 connectors 2 3 PCI Express Interface The ConnectX 5 adapter card supports PCI Express Gen 3 0...

Страница 18: ...inOF 2 for Windows See Chapter 4 Driver Installation 3 2 Safety Precautions 1 Remove any metallic objects from your hands and wrists 2 Make sure to use only insulated tools 3 Verify that the system is...

Страница 19: ...is section To replace the brackets you will need the following parts The new brackets of the proper height The 2 screws saved from the removal of the brackets 3 4 1 Removing the Existing Bracket 1 Rem...

Страница 20: ...Express slots on the server one for the adapter card and one for the auxiliary PCIe connection card Please note that the following figures are for illustration purposes only Use the Slim Line SAS harn...

Страница 21: ...the card insert the auxiliary PCIe connection card into the PCI Express slot until firmly seated 6 Secure the auxiliary PCIe connection card using the server s retention mechanisms Do not use excessi...

Страница 22: ...ard into the PCI Express slot adjacent to the auxiliary PCIe connection card until firmly seated 8 Secure the adapter card using the server s retention mechanisms 9 Connect the slim line SAS cable to...

Страница 23: ...cable to the adapter card Do this by using a cable holder or tying the cable to the rack b Determine the correct orientation of the connector to the card before inserting the connector Do not try and...

Страница 24: ...which allow for routing horizontally around to the side of the chassis before bending upward or downward in the rack 6 To remove a cable disengage the locks and slowly pull the connector away from the...

Страница 25: ...e from the connector on the adapter card 4 Disengage the retention mechanisms clips or screws on adapter card 5 Holding the card from its center gently pull the adapter card from the PCI Express slot...

Страница 26: ...6 24 Mellanox Technologies 7 Holding the card from its center gently pull the auxiliary PCIe connection card from the PCI Express slot 8 Disconnect the slim line SAS cable from the connector on the au...

Страница 27: ...fields VEN and DEV fields are separated by In the display example above notice the sub string PCI VEN_15B3 DEV_1003 VEN is equal to 0x15B3 this is the Vendor ID of Mellanox Technologies and DEV is eq...

Страница 28: ...nique PCI Bus and Device number Each of the PCIe x8 busses sees two network ports in effect the two 100Gb s physical ports of the ConnectX 5 Socket Direct Adapter are viewed as four net devices by the...

Страница 29: ...software stacks that are part of the standard operating system distribution or another vendor s commercial stack Installs the MLNX_OFED_LINUX binary RPMs if they are available for the current kernel I...

Страница 30: ...pecific amount of RAM For your machine to be part of the InfiniBand VPI fabric a Subnet Manager must be running on one of the fabric nodes At this point Mellanox OFED for Linux has already installed t...

Страница 31: ...s dir tmp MLNX_OFED_LINUX x x x logs This program will install the MLNX_OFED_LINUX package on your machine Note that all other Mellanox OEM OFED RDMA or Distribution IB packages will be removed Those...

Страница 32: ...cket ROHS R6 PSID MT_2190110032 PCI Device Name 0b 00 0 Base MAC 0000e41d2d5cf810 Versions Current Available FW 16 22 0228 16 22 0228 Status Up to date In case that your machine has an unsupported net...

Страница 33: ...est ofed see the file docs readme_and_user_manual hca_self_test readme After installation completion information about the Mellanox OFED installation such as prefix kernel version and installation par...

Страница 34: ...tes on SLES and Fedora Distributions lib modules uname r extra mlnx ofa_kernel on RHEL and other RedHat like Distributions Firmware The firmware of existing network adapter devices will be updated if...

Страница 35: ...ules exist in the initramfs file they can automatically be loaded by the kernel To prevent this behavior update the initramfs using the operating systems standard tools Note The process of updating th...

Страница 36: ...sical port of PCI function 0000 84 00 0 MT4119 is the PCI Device ID of the Mellanox ConnectX 5 adapters family Step 2 Check to which NUMA each netdevice is connected to Table 8 mlnxofedinstall Return...

Страница 37: ...ped 0 overruns 0 carrier 0 collisions 0 p2p1 flags 4163 UP BROADCAST RUNNING MULTICAST mtu 1500 inet 2 1 203 1 netmask 255 0 0 0 broadcast 2 255 255 255 inet6 fe80 268a 7ff fe9d 4624 prefixlen 64 scop...

Страница 38: ...CA mlx5_10 CA type MT4119 Number of ports 1 Firmware version 16 22 0228 Hardware version 0 Node GUID 0x248a0703009d4624 System image GUID 0x248a0703009d4624 Port 1 State Active Physical state LinkUp...

Страница 39: ...194 0 Connecting to www mellanox com 72 3 194 0 80 connected HTTP request sent awaiting response 200 OK Length 1354 1 3K text plain Saving to RPM GPG KEY Mellanox 100 1 354 K s in 0s 2014 04 20 13 52...

Страница 40: ...pc installer package with KMP support mlnx ofed hypervisor noarch MLNX_OFED hypervisor installer package with KMP support mlnx ofed vma noarch MLNX_OFED vma installer package with KMP support mlnx ofe...

Страница 41: ...4 301 fc21 x86_64 noarch MLNX_OFED vma installer package for kernel 3 17 4 301 fc21 x86_64 without KMP support mlnx ofed vma eth 3 17 4 301 fc21 x86_64 noarch MLNX_OFED vma eth installer package for k...

Страница 42: ...by invoking Where Step 2 Install the desired group deb file path to extracted MLNX_OFED package DEBS wget qO http www mellanox com downloads ofed RPM GPG KEY Mellanox sudo apt key add apt key list pu...

Страница 43: ...nloaded from Mellanox Technologies Web site http www mellanox com Support Firmware Download Step 1 Get the device s PSID Step 2 Download the firmware BIN file from the Mellanox website or the OEM webs...

Страница 44: ...enabled the Mellanox x 509 public key should be added to the UEFI Secure Boot key database and loaded onto the system key ring by the kernel Follow these steps below to add the Mellanox s x 509 publi...

Страница 45: ...whose key is not in the kernel key ring So it s much easier to miss this mes sage You won t see it on repeated tests where you unload and reload a kernel module until you reboot There is no way to eli...

Страница 46: ...CMD 3 Enter the following command On an x64 64 bit machine the output will be AMD64 Step 2 Go to the Mellanox WinOF 2 web page at http www mellanox com Products InfiniBand VPI Drivers Windows SW Driv...

Страница 47: ...to upgrade your firmware version1 Installing the incorrect exe file is prohibited If you do so an error message will be displayed For example if you try to install a 64 bit exe on a 32 bit machine the...

Страница 48: ...Driver Installation Rev 1 6 46 Mellanox Technologies Step 4 Click Next in the Welcome screen Step 5 Read then accept the license agreement and click Next...

Страница 49: ...Driver Installation Rev 1 6 47 Mellanox Technologies Step 6 Select the target folder for the installation...

Страница 50: ...firmware will be updated accordingly However if the user has both an OEM card and a Mellanox card only the Mellanox card will be updated Step 8 Select a Complete or Custom installation follow Step a a...

Страница 51: ...Installation Rev 1 6 49 Mellanox Technologies Diagnostic Tools installation tools used for diagnostics such as mlx5cmd b Click Next to install the desired tools Step 9 Click Install to start the insta...

Страница 52: ...tion Rev 1 6 50 Mellanox Technologies Step 10 In case that firmware upgrade option was checked in Step 7 you will be notified if a firm ware upgrade is required See Step 11 Click Finish to complete th...

Страница 53: ...Upon installation completion the inf files can be located at ProgramFiles Mellanox MLNX_WinOF2 Drivers OS To see the Mellanox network adapters display the Device Manager and pull down the Network adap...

Страница 54: ...e two 100Gb s physical ports of the ConnectX 5 Socket Direct Adapter are viewed as four netdevices by the system 4 2 5 Extracting Files Without Running Installation To extract the files without runnin...

Страница 55: ...Driver Installation Rev 1 6 53 Mellanox Technologies Step 3 Click Next to create a server image Step 4 Click Change and specify the location in which the files are extracted to...

Страница 56: ...Driver Installation Rev 1 6 54 Mellanox Technologies Step 5 Click Install to extract this folder or click Change to install to a different folder Step 6 To complete the extraction click Finish...

Страница 57: ...o upgrade firmware manually please refer to MFT User Manual www mellanox com Products Ethernet Drivers Firmware Tools 4 2 8 Deploying the Driver on a Nano Server 4 2 8 1 Offline Installation To deploy...

Страница 58: ...load the driver package Step 3 Extract the Mellanox drivers to a local directory see Section 4 2 5 Extracting Files With out Running Installation on page 52 Step 4 Open a remote connection to the Nano...

Страница 59: ...rying Mellanox devices firmware Device Type ConnectX 5 Part Number MCX556M ECAT S25 Description ConnectX 5 VPI adapter card with Multi Host Socket Direct supporting dual socket server EDR IB 100Gb s a...

Страница 60: ...Try using another cable Reinstall the drivers for the network driver files may be damaged or deleted Reboot the server Adapters stopped working after installing another adapter Try removing and re in...

Страница 61: ...x Mellanox Firmware Tool MFT Download and install MFT http www mellanox com content pages php pg management_tools menu_section 34 Refer to the User Manual for installation instructions Once installed...

Страница 62: ...ww mellanox com content pages php pg management_tools menu_section 34 Refer to the User Manual for installation instructions Once installed open a CMD window and run WinMFT mst start mst status flint...

Страница 63: ...s when connected to another Mellanox Infini Band product Ethernet 100GBASE CR4 100GBASE KR4 100GBASE SR4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25...

Страница 64: ...t been established Blinking Ambera a 1 Hz Blinking Amber occurs due to running a beacon command for locating the adapter card 6 Hz blinking Amber indicates a problem with the link Solid Green Indicate...

Страница 65: ...nsions Figure 3 Mechanical Drawing of the Dual port Adapter Card Figure 4 Mechanical Drawing of the Auxiliary PCIe Connection Card All dimensions are in millimeters All the mechanical tolerances are 0...

Страница 66: ...Specifications Rev 1 6 64 Mellanox Technologies Figure 5 Mechanical Drawing of the Slim Line SAS Cable...

Страница 67: ...Specifications Rev 1 6 65 Mellanox Technologies 7 5 Bracket Mechanical Drawing Figure 6 Adapter Card Tall Bracket Figure 7 Auxiliary PCIe Connection Card Tall Bracket...

Страница 68: ...2n Transmitter Inverted Data Input 3 Tx2p Transmitter Non Inverted Data Input 4 GND Ground 5 Tx4n Transmitter Inverted Data Input 6 Tx4p Transmitter Non Inverted Data Input 7 GND Ground 8 ModSelL Modu...

Страница 69: ...Ground 24 Rx2n Receiver Inverted Data Output 25 Rx2p Receiver Non Inverted Data Output 26 GND Ground 27 Mod PrsL Module Present 28 IntL Interrupt 29 VccTx 3 3 V Power supply transmitter 30 Vcc1 3 3 V...

Страница 70: ...port PCI Express 3 0 two PCIe x8 connectors a standard PCI Express x8 edge connector and Slim Line SAS connector pinout according to SFF 9401 standards PCIe x16 Connector Pinout Figure 9 PCIe x8 Edge...

Страница 71: ...CX_1BN A7 GND A8 I2C_SMCLK_H1 A9 I2C_SMDAT_H1 A10 GND A11 S_PERST1_CONN_L A12 S_PRSNT1_L A13 GND A14 PCIE_CPU_CX_1CP A15 PCIE_CPU_CX_1CN A16 GND A17 PCIE_CPU_CX_1D A18 PCIE_CPU_CX_1DN A19 GND A20 PCIE...

Страница 72: ..._P B12 PCEI_REFCLK1_N B13 GND B14 PCIE_CX_CPU_1CP B15 PCIE_CX_CPU_1CN B16 GND B17 PCIE_CX_CPU_1DP B18 PCIE_CX_CPU_1DN B19 GND B20 PCIE_CX_CPU_1EP B21 PCIE_CX_CPU_1EN B22 GND B23 PCIE_CX_CPU_1FP B24 PC...

Страница 73: ..._1AP A3 PCIE_CX_CPU_1AN A4 GND A5 PCIE_CX_CPU_1BP A6 PCIE_CX_CPU_1BN A7 GND A8 NC A9 S_WAKE1_CONN_L A10 GND A11 PCIE_REFCLK1_P A12 PCEI_REFCLK1_N A13 GND A14 PCIE_CX_CPU_1CP A15 PCIE_CX_CPU_1CN A16 GN...

Страница 74: ...E_CPU_CX_1BP B6 PCIE_CPU_CX_1BN B7 GND B8 I2C_SMCLK_H1 B9 I2C_SMDAT_H1 B10 GND B11 S_PERST1_CONN_L B12 S_PRSNT1_L B13 GND B14 PCIE_CPU_CX_1CP B15 PCIE_CPU_CX_1CN B16 GND B17 PCIE_CPU_CX_1D B18 PCIE_CP...

Страница 75: ...30 S_PRSNT1_R_L B31 GND B32 PCIE_CPU_CX_1GP B33 PCIE_CPU_CX_1GN B34 GND B35 PCIE_CPU_CX_1HP B36 PCIE_CPU_CX_1HN B37 GND SH1 GND SH2 GND SH3 GND SH4 GND Table 19 Slim Line SAS Pinout Connector on the A...

Страница 76: ...ial number and the card MAC for the Ethernet protocol and the card GUID for the InfiniBand protocol VPI cards have both a GUID and a MAC derived from the GUID Figure 10 MCX556M ECAT S25 Board Label Ex...

Страница 77: ...Rev 1 6 75 Mellanox Technologies Figure 12 Slim Line SAS Harness Board Label...

Страница 78: ...1 F To guarantee proper air flow allow at least 8cm 3 inches of clearance around the ven tilation openings During periods of lightning activity do not work on the equipment or connect or dis connect c...

Страница 79: ...se of controls or adjustment or performance of procedures other than those specified herein may result in hazardous radiation exposure CLASS 1 LASER PRODUCT and reference to the most recent laser stan...

Страница 80: ...maximale est n cessaire En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouver tures de ventilation Pendant un orage il ne faut pas utiliser l...

Страница 81: ...e en garde l utilisation de commandes ou de r glages ou l ex cution de proc dures autres que ce qui est sp cifi dans les pr sentes peut engendrer une exposition au rayonnement grave PRODUIT LASER DE C...

Страница 82: ...gstemperatur erforderlich Au erdem sollten mindestens 8 cm 3 in Freiraum um die Bel ftungs ffnungen sein um einen einwandfreien Luftstrom zu gew hrleisten Arbeiten Sie w hrend eines Gewitters und Blit...

Страница 83: ...ak Achtung Nutzung von Steuerungen oder Einstellungen oder Ausf hrung von Prozeduren die hier nicht spezifiziert sind kann zu gef hrlichem Strahlenkon takt f hren Klasse 1 Laserprodukt und Referenzen...

Страница 84: ...ar una circulaci n de aire adecuada se debe dejar como m nimo un espacio de 8 cm 3 pulgadas alrededor de las aberturas de ventilaci n No utilizar el equipo ni conectar o desconectar cables durante per...

Страница 85: ...ligrosos Precauci n el uso de controles o ajustes o la realizaci n de procedimientos distintos de los que aqu se especifican podr an causar exposici n a niveles de radiaci n peligrosos PRODUCTO L SER...

Отзывы:

Нет отзывов

Бренды по названию

Популярные бренды

Загрузить еще бренды