manualshive.com logo in svg

Mellanox Technologies MCX456M-ECAT, Руководство пользователя

Поделиться
Скачать
Mellanox Technologies MCX456M-ECAT Скачать руководство пользователя страница 27

Hardware Installation

Rev 1.0

27

Mellanox Technologies

3.9.2 On Linux

Get the device location on the PCI bus by running lspci and locating lines with the string “Mella-
nox Technologies”: 

The list of Mellanox Technologies PCI Device IDs can be found in the PCI ID reposi-
tory at 

http://pci-ids.ucw.cz/read/PC/15b3

. 

> lspci |grep -i Mellanox 
Network controller: Mellanox Technologies MT4115 Family [ConnectX-4]

Содержание MCX456M-ECAT

Страница 1: ...Mellanox Technologies www mellanox com ConnectX 4 VPI Dual Port QSFP28 Socket Direct Adapter Card User Manual for Dual Socket Servers P N MCX456M ECAT Rev 1 0...

Страница 2: ...tp www mellanox com page trademarks NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AN...

Страница 3: ...eability 15 Chapter 2 Interfaces 16 2 1 InfiniBand Interface 16 2 2 EthernetQSFP28 Interface 16 2 3 PCI Express Interface 16 2 4 LED Interface 16 Chapter 3 Hardware Installation 17 3 1 System Requirem...

Страница 4: ...37 4 1 6 Installing MLNX_OFED using apt get 37 4 1 6 1 Setting up MLNX_OFED apt get Repository 37 4 1 6 2 Installing MLNX_OFED using the apt get Tool 38 4 1 6 3 Uninstalling Mellanox OFED using the a...

Страница 5: ...MCX456M ECAT Specifications 61 7 2 Airflow Specifications 62 7 3 Adapter Card LED Operations 62 7 4 Board Mechanical Drawing and Dimensions 63 7 5 Bracket Mechanical Drawings 64 Appendix A Interface...

Страница 6: ...ble 4 Features 13 Table 5 Hardware and Software Requirements 28 Table 6 Hardware and Software Requirements 42 Table 7 MCX456M ECAT Specification Table 61 Table 8 Physical and Logical Link Indications...

Страница 7: ...apter Card 63 Figure 4 Mechanical Drawing of the Auxiliary PCIe Connection Card 63 Figure 5 Mechanical Drawing of the Slim Line SAS Cable 63 Figure 6 Adapter Card Tall Bracket 64 Figure 7 Auxiliary PC...

Страница 8: ...able Date Rev Comments Changes December 2016 1 1 Added Section 1 5 Manageability on page 15 Updated Section 3 1 1 Hardware on page 17 Updated Section 4 1 3 1 Installation Procedure on page 31 Added Se...

Страница 9: ...nd query utility used to update the firmware See http www mellanox com Products Software Firmware Tools mlxup Firmware Utility Mellanox OFED for Linux User Manual Document no 2877 User Manual describi...

Страница 10: ...port Services please see your contract for details regarding Technical Support Customers who purchased Mellanox products through a Mellanox approved reseller should first seek assistance through their...

Страница 11: ...le through a PCIe x8 edge connector and the other bus through an x8 parallel connector to an Auxiliary PCIe Connection Card The two cards should be installed into two adjacent PCIe x8 slots and connec...

Страница 12: ...r OPN MCX456M ECAT includes the following components Low Profile Adapter Card Dual port Auxiliary PCIe Connection Card Harness Slim Line SAS Cable Data Transmission Rate InfiniBand SDR DDR QDR EDR 100...

Страница 13: ...it Ethernet IEEE 802 3by 25 Gigabit Ethernet IEEE 802 3ae 10 Gigabit Ethernet IEEE 802 3az Energy Efficient Ethernet IEEE 802 3ap based auto negotiation and KRstartup IEEE 802 3ad 802 1AX Link Aggrega...

Страница 14: ...GPU to CPU and therefore significantly reduces application run time ConnectX 4 advanced acceleration technology enables higher cluster effi ciency and scalability to tens of thousands of nodes CPU off...

Страница 15: ...geability through a BMC The Socket Direct PCIe stand up adapter can be connected to a BMC using MCTP over SMBus or MCTP over PCIe protocols as if it is a standard Mellanox PCIe stand up adapter For co...

Страница 16: ...Table 4 Features on page 13 Ethernet traffic is transmitted through the cards QSFP28 connectors 2 3 PCI Express Interface The ConnectX 4 adapter card supports PCI Express 3 0 1 1 and 2 0 compatible th...

Страница 17: ...4 3 2 1 Software Stacks Mellanox OpenFabric software package MLNX_OFED for Linux and WinOF 2 for Windows See Chapter 4 Driver Installation on page 28 3 3 Safety Precautions 1 Remove any metallic objec...

Страница 18: ...this section To replace the brackets you will need the following parts The new bracket of the proper height The 2 screws saved from the removal of the bracket 3 5 1 Removing the Existing Bracket 1 Re...

Страница 19: ...3 Connect the slim line SAS connector male on the slim line SAS cable to the female connec tor on the auxiliary PCIe connection card 4 Locate two available PCI Express slots on the server one for the...

Страница 20: ...insert the auxiliary PCIe connection card into the PCI Express slot until firmly seated 6 Secure the auxiliary PCIe connection card using server s retention mechanisms Do not use excessive force when...

Страница 21: ...ard into the PCI Express slot adjacent to the auxiliary PCIe connection card until firmly seated 8 Secure the adapter card using the server s retention mechanisms 9 Connect the slim line SAS cable to...

Страница 22: ...termine the correct orientation of the connector to the adapter card before inserting the connector Do not try and insert the connector upside down This may damage the adapter card c Insert the connec...

Страница 23: ...a cable disengage the locks and slowly pull the connector away from the port receptacle LED indicator on the adapter card will turn off when the cable is unseated 3 8 Cards Un Installation Instruction...

Страница 24: ...connector on the adapter card 4 Disengage the retention mechanisms clips or screws on adapter card 5 Holding the card from its center gently pull the adapter card from the PCI Express slot 6 Disengage...

Страница 25: ...0 25 Mellanox Technologies 7 Holding the card from its center gently pull the auxiliary PCIe connection card from the PCI Express slot 8 Disconnect the slim line SAS cable from the connector on the au...

Страница 26: ...d properties window 4 Click the Details tab and select Hardware Ids Windows 2012 R2 from the Properties pull down menu Figure 1 PCI Device 5 In the Value display box check the fields VEN and DEV field...

Страница 27: ...he PCI bus by running lspci and locating lines with the string Mella nox Technologies The list of Mellanox Technologies PCI Device IDs can be found in the PCI ID reposi tory at http pci ids ucw cz rea...

Страница 28: ...llanox com Products Soft ware InfiniBand VPI Drivers Table 5 Hardware and Software Requirements Requirements Description Platforms A server platform with an adapter card based on one of the following...

Страница 29: ...prompted to acknowledge the deletion of the old packages If you need to install Mellanox OFED on an entire homogeneous cluster a common strategy is to mount the ISO image on one of the cluster nodes a...

Страница 30: ...information please see add kernel support option below On Ubuntu and Debian distributions drivers installation use Dynamic Kernel Module Support DKMS framework Thus the drivers compilation will take...

Страница 31: ...e ISO image on your machine Step 3 Run the installation script MLNX_OFED_LINUX x x x rhel6 3 x86_64 mlnx_add_kernel_support sh m tmp MLNX_OFED_ LINUX x x x rhel6 3 x86_64 make tgz Note This program wi...

Страница 32: ...Mellanox OFED components can be configured or reconfigured after the installation by modifying the relevant configuration files See the relevant chapters in this manual for details The list of the mod...

Страница 33: ...es are installed under lib modules uname r updates on SLES and Fedora Distributions lib modules uname r extra mlnx ofa_kernel on RHEL and other RedHat like Distributions lib modules uname r updates dk...

Страница 34: ...achine as root Step 2 Mount the ISO image on your machine and copy its content to a shared location in your net work You can download it from http www mellanox com Products Software Infini Band VPI Dr...

Страница 35: ...20 13 52 30 247 MB s RPM GPG KEY Mellanox saved 1354 1354 sudo rpm import RPM GPG KEY Mellanox warning rpmts_HdrFromFdno Header V3 DSA SHA1 Signature key ID 6224c050 NOKEY Retrieving key from file re...

Страница 36: ...d basic Installs basic packages required for running Mellanox cards mlnx ofed guest Installs packages required by guest OS mlnx ofed hpc Installs packages required for HPC mlnx ofed hypervisor Install...

Страница 37: ...ng the YUM Tool Use the script usr sbin ofed_uninstall sh to uninstall the Mellanox OFED package The script is part of the ofed scripts RPM 4 1 6 Installing MLNX_OFED using apt get This type of instal...

Страница 38: ...apt get update apt cache search mlnx ofed mlnx ofed vma eth MLNX_OFED vma eth installer package with DKMS support mlnx ofed hpc MLNX_OFED hpc installer package with DKMS support mlnx ofed vma vpi MLN...

Страница 39: ...following steps are also appropriate in case you wish to burn newer firmware that you have downloaded from Mellanox Technologies Web site http www mellanox com Support Firmware Download Step 1 Get the...

Страница 40: ...provide a comma separated list of PCI devices to exclude from the firmware update Example 4 1 8 UEFI Secure Boot All kernel modules included in MLNX_OFED for RHEL7 and SLES12 are signed with x 509 ke...

Страница 41: ...ature To remove the signature from the MLNX_OFED kernel modules Step 1 Remove the signature After the signature has been removed a message as the below will no longer be presented upon module loading...

Страница 42: ...e according to your Operating System please follow the steps below Step 1 Obtain the machine architecture For Windows Server 2012 2012 R2 8 1 Client 1 To go to the Start menu position your mouse in th...

Страница 43: ...ed Installation The following is an example of an installation session Step 1 Double click the exe and follow the GUI instructions to install MLNX_WinOF2 Step 2 Optional Manually configure your setup...

Страница 44: ...Rev 1 0 44 Mellanox Technologies Step 4 Click Next in the Welcome screen Step 5 Read then accept the license agreement and click Next...

Страница 45: ...Driver Installation Rev 1 0 45 Mellanox Technologies Step 6 Select the target folder for the installation...

Страница 46: ...ing cases If the user has an OEM card In this case the firmware will not be displayed If the user has a standard Mellanox card with an older firmware version the firmware will be updated accordingly H...

Страница 47: ...net from a host OpenSM is part of the driver and installed automatically Performances tools install the performance tools that are used to measure performance in user environment Analyze tools install...

Страница 48: ...Rev 1 0 48 Mellanox Technologies Management tools installation tools used for management such as mlxstat b Click Install to start the installation...

Страница 49: ...stallation Rev 1 0 49 Mellanox Technologies Step 9 In case firmware upgrade option was checked in Step 7 you will be notified if a firmware upgrade is required Step 10 Click Finish to complete the ins...

Страница 50: ...pon installation completion the inf files can be located at ProgramFiles Mellanox MLNX_WinOF2 Drivers OS To see the Mellanox network adapters display the Device Manager and pull down the Net work adap...

Страница 51: ...ning Installation To extract the files without running installation perform the following steps Step 1 Open a CMD console Click Start Task Manager File Run new task and enter CMD Step 2 Extract the dr...

Страница 52: ...Rev 1 0 52 Mellanox Technologies Step 3 Click Next to create a server image Step 4 Click Change and specify the location in which the files are extracted to...

Страница 53: ...folder or click Change to install to a different folder Step 6 To complete the extraction click Finish 4 2 6 Uninstalling Mellanox WinOF 2 Driver 4 2 6 1 Attended Uninstallation To uninstall MLNX_WinO...

Страница 54: ...n a Nano Server 4 2 8 1 Offline Installation To deploy the Driver in Nano Server Step 1 Go to the Mellanox WinOF web page at http www mellanox com Products InfiniBand VPI Drivers Windows SW Drivers St...

Страница 55: ...l directory see Section 4 2 5 Extracting Files With out Running Installation on page 51 Step 4 Open a remote connection to the Nano server Step 5 Copy all the driver files to the Nano server Step 6 In...

Страница 56: ...bug fixes To check that your card is programmed with the latest available firmware version down load the mlxup firmware update and query utility The utility can query for available Mellanox adapters...

Страница 57: ...Status Up to date Device Type ConnectX4 Part Number MCX456M ECAT Description ConnectX 4 VPI adapter card with Multi Host supporting dual socket server EDR IB 100Gb s and 100GbE dual port QSFP28 dual P...

Страница 58: ...ecessary Try using another cable Reinstall the drivers for the network driver files may be damaged or deleted Reboot the server Adapters stopped working after installing another adapter Try removing a...

Страница 59: ...grep i Mellanox Mellanox Firmware Tool MFT Download and install MFT http www mellanox com con tent pages php pg management_tools menu_section 34 Refer to the User Manual for installation instructions...

Страница 60: ...ction 34 Refer to the User Manual for installation instructions Once installed open a CMD window and run cd C Program Files Mellanox WinMFT mst start mst status flint d mst_device q Ports Information...

Страница 61: ...E R4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASESR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASEKR2 1000BASE CX 1000BASE KX 10GBASE SR 10GBASE LR 10GBASE ER 10GBASE CX4 10GBASE KX4 1...

Страница 62: ...rked with are not applicable Air Flow Direction Heat Sink to Port Port to Heat Sink Cable Type Passive Active 3 5W Passive Active 3 5W MCX456M ECAT 400 700 300 300 LED Color and State Description Off...

Страница 63: ...Figure 3 Mechanical Drawing of the Dual port Adapter Card Figure 4 Mechanical Drawing of the Auxiliary PCIe Connection Card Figure 5 Mechanical Drawing of the Slim Line SAS Cable All dimensions are i...

Страница 64: ...Rev 1 0 64 Mellanox Technologies 7 5 Bracket Mechanical Drawings Figure 6 Adapter Card Tall Bracket Figure 7 Auxiliary PCIe Connection Card Tall Bracket Port 1 Bi Color LED Bi Color LED Port 2...

Страница 65: ...r and Cage Views Table 9 Connector Pin Number and Name to Signal Name Map Connector Pin Number Connector Pin Name Port A Signal Name 1 GND GND 2 TXN_2 Tx2n 3 TXP_2 Tx2p 4 GND GND 5 TXN_4 Tx4n 6 TXP_4...

Страница 66: ...tan dards 15 RXN_3 Rx3n 16 GND GND 17 RXP_1 Rx1p 18 RXN_1 Rx1n 19 GND GND 20 GND GND 21 RXN_2 Rx2n 22 RXP_2 Rx2p 23 GND GND 24 RXN_4 Rx4n 25 RXP_4 Rx4p 26 GND GND 27 ModPrsl_Port0 Mod PrsL 28 IntL Int...

Страница 67: ...Rev 1 0 67 Mellanox Technologies Figure 9 PCIe x8 Edge Connector Pinout Applies to both to the adapter card and the auxiliary PCIe connection card...

Страница 68: ...7 GND A8 NC A9 S_WAKE1_CONN_L A10 GND A11 PCIE_REFCLK1_P A12 PCEI_REFCLK1_N A13 GND A14 PCIE_CX_CPU_1CP A15 PCIE_CX_CPU_1CN A16 GND A17 PCIE_CX_CPU_1DP A18 PCIE_CX_CPU_1DN A19 GND A20 PCIE_CX_CPU_1EP...

Страница 69: ...CX_1CP B15 PCIE_CPU_CX_1CN B16 GND B17 PCIE_CPU_CX_1D B18 PCIE_CPU_CX_1DN B19 GND B20 PCIE_CPU_CX_1EP B21 PCIE_CPU_CX_1EN B22 GND B23 PCIE_CPU_CX_1FP B24 PCIE_CPU_CX_1FN B25 GND B26 NC B27 NC B28 GND...

Страница 70: ...I2C_SMDAT_H1 A10 GND A11 S_PERST1_CONN_L A12 S_PRSNT1_L A13 GND A14 PCIE_CPU_CX_1CP A15 PCIE_CPU_CX_1CN A16 GND A17 PCIE_CPU_CX_1D A18 PCIE_CPU_CX_1DN A19 GND A20 PCIE_CPU_CX_1EP A21 PCIE_CPU_CX_1EN A...

Страница 71: ...B15 PCIE_CX_CPU_1CN B16 GND B17 PCIE_CX_CPU_1DP B18 PCIE_CX_CPU_1DN B19 GND B20 PCIE_CX_CPU_1EP B21 PCIE_CX_CPU_1EN B22 GND B23 PCIE_CX_CPU_1FP B24 PCIE_CX_CPU_1FN B25 GND B26 NC B27 NC B28 GND B29 NC...

Страница 72: ...er printed on the label serial number and the card MAC for the Ethernet protocol and the card GUID for the InfiniBand protocol VPI cards have both a GUID and a MAC derived from the GUID Figure 10 MCX4...

Страница 73: ...1 F To guarantee proper air flow allow at least 8cm 3 inches of clearance around the ven tilation openings During periods of lightning activity do not work on the equipment or connect or dis connect c...

Страница 74: ...se of controls or adjustment or performance of procedures other than those specified herein may result in hazardous radiation exposure CLASS 1 LASER PRODUCT and reference to the most recent laser stan...

Страница 75: ...maximale est n cessaire En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouver tures de ventilation Pendant un orage il ne faut pas utiliser l...

Страница 76: ...e en garde l utilisation de commandes ou de r glages ou l ex cution de proc dures autres que ce qui est sp cifi dans les pr sentes peut engendrer une exposition au rayonnement grave PRODUIT LASER DE C...

Страница 77: ...gstemperatur erforderlich Au erdem sollten mindestens 8 cm 3 in Freiraum um die Bel ftungs ffnungen sein um einen einwandfreien Luftstrom zu gew hrleisten Arbeiten Sie w hrend eines Gewitters und Blit...

Страница 78: ...ak Achtung Nutzung von Steuerungen oder Einstellungen oder Ausf hrung von Prozeduren die hier nicht spezifiziert sind kann zu gef hrlichem Strahlenkon takt f hren Klasse 1 Laserprodukt und Referenzen...

Страница 79: ...ar una circulaci n de aire adecuada se debe dejar como m nimo un espacio de 8 cm 3 pulgadas alrededor de las aberturas de ventilaci n No utilizar el equipo ni conectar o desconectar cables durante per...

Страница 80: ...ligrosos Precauci n el uso de controles o ajustes o la realizaci n de procedimientos distintos de los que aqu se especifican podr an causar exposici n a niveles de radiaci n peligrosos PRODUCTO L SER...

Отзывы:

Нет отзывов

Бренды по названию

Популярные бренды

Загрузить еще бренды