Mellanox Technologies MCX453A-FCAT Скачать руководство пользователя страница 4

Страница: 4 / 64

ConnectX-4 VPI Single and Dual Port QSFP28 Adapter Card User Manual

Rev 1.9

Mellanox Technologies

4.1.1 Hardware and Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.1.2 Downloading Mellanox OFED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.1.3 Installing Mellanox OFED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

4.1.3.1

Installation Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

4.1.3.2

Installation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.1.3.3

Installation Logging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.1.3.4

openibd Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.1.3.5

Driver Load Upon System Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.1.3.6

mlnxofedinstall Return Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.1.4 Uninstalling Mellanox OFED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.1.5 Installing MLNX_OFED using YUM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.1.5.1

Setting up MLNX_OFED YUM Repository. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.1.5.2

Installing MLNX_OFED using the YUM Tool . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.1.5.3

Uninstalling Mellanox OFED using the YUM Tool . . . . . . . . . . . . . . . . . . . . . . . 32

4.1.6 Installing MLNX_OFED using apt-get. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.1.6.1

Setting up MLNX_OFED apt-get Repository . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.1.6.2

Installing MLNX_OFED using the apt-get Tool. . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.1.6.3

Uninstalling Mellanox OFED using the apt-get Tool . . . . . . . . . . . . . . . . . . . . . . 33

4.1.7 Updating Firmware After Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.1.7.1

Updating the Device Online . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4.1.7.2

Updating the Device Manually . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4.1.7.3

Updating the Device Firmware Automatically upon System Boot . . . . . . . . . . . . 34

4.1.8 UEFI Secure Boot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.1.8.1

Enrolling Mellanox's x.509 Public Key On your Systems . . . . . . . . . . . . . . . . . . 35

4.1.8.2

Removing Signature from Kernel Modules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.1.9 Performance Tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Chapter 5 Updating Adapter Card Firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5.1 Firmware Update Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Chapter 6 Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

6.1 General . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

6.2 Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Chapter 7 Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

7.1 MCX453A-FCAT Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

7.2 MCX454A-FCAT Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

7.3 MCX455A-FCAT Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

7.4 MCX456A-FCAT Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

7.5 MCX455A-ECAT Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

7.6 MCX456A-ECAT Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

7.7 Airflow Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

7.8 Adapter Card LED Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

7.9 Board Mechanical Drawing and Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

7.10 Bracket Mechanical Drawings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

Appendix A Interface Connectors Pinout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

A.1 QSFP28 Connector Pinout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

7.11 PCI Express x8 orx16 Edge Connector Pinout . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Appendix B Finding the GUID/MAC and Serial Number on the Adapter Card . . . . 54

Содержание MCX453A-FCAT

Страница 1: ...www mellanox com ConnectX 4 VPI Single and Dual QSFP28 Adapter Card User Manual P N MCX453A FCAT MCX454A FCAT MCX455A FCAT MCX456A FCAT MCX455A ECAT MCX456A ECAT Rev 1 9...

Страница 2: ...OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLI...

Страница 3: ...net QSFP28 Interface 16 2 3 PCI Express Interface 16 2 4 LED Interface 16 Chapter 3 Hardware Installation 17 3 1 System Requirements 17 3 1 1 Hardware 17 3 2 Operating Systems Distributions 17 3 2 1 S...

Страница 4: ...ting Firmware After Installation 33 4 1 7 1 Updating the Device Online 34 4 1 7 2 Updating the Device Manually 34 4 1 7 3 Updating the Device Firmware Automatically upon System Boot 34 4 1 8 UEFI Secu...

Страница 5: ...Rev 1 9 Mellanox Technologies 5 Appendix C Safety Warnings 57 Appendix D Avertissements de s curit d installation Warnings in French 59 Appendix E Sicherheitshinweise Warnings in German 61 Appendix F...

Страница 6: ...Single and Dual port EDR Adapter Cards 13 Table 5 Features 14 Table 6 Hardware and Software Requirements 24 Table 7 MCX453A FCAT Specifications Table 40 Table 8 MCX454A FCAT Specifications Table 41 T...

Страница 7: ...he Single port x16 Adapter Cards 48 Figure 5 Mechanical Drawing of the Dual port x16 Adapter Card 48 Figure 6 Single port Bracket 49 Figure 7 Dual port Bracket 49 Figure 8 Connector and Cage Views 50...

Страница 8: ...ions Product Overview on page 12 Features on page 14 MCX453A FCAT Specifications on page 40 MCX454A FCAT Specifications on page 41 MCX455A FCAT Specifications on page 42 MCX456A FCAT Specifications on...

Страница 9: ...ons on page 42 MCX456A FCAT Specifications on page 43 MCX455A ECAT Specifications on page 44 MCX456A ECAT Specifications on page 45 Updated Features and Benefits on page 16 Updated Section 3 9 1 On Wi...

Страница 10: ...de See http www mellanox com Products Software Firm ware Tools Mellanox Firmware Utility mlxup User Manual and Release Notes Mellanox firmware update and query utility used to update the firmware See...

Страница 11: ...l Support Services please see your contract for details regarding Technical Support Customers who purchased Mellanox products through a Mellanox approved reseller should first seek assistance through...

Страница 12: ...ributions on page 15 Section OpenFabrics Enterprise Distribution OFED on page 15 1 1 Product Overview The following section provides the ordering part number port speed number of ports and PCI Express...

Страница 13: ...455A ECAT Single port card MCX456A ECAT Dual port card Data Transmission Rate InfiniBand SDR DDR QDR FDR EDR Ethernet 1 10 25 40 50 100 Gb s Network Connector Types Single or Dual port QSFP28 PCI Expr...

Страница 14: ...net IEEE 802 3ap based auto negotiation and KRstartup IEEE 802 3ad 802 1AX Link Aggregation IEEE 802 1Q 802 1P VLAN tags and priority IEEE 802 1Qau QCN Congestion Notification IEEE 802 1Qaz ETS IEEE 8...

Страница 15: ...xample from GPU to CPU and therefore significantly reduces application run time ConnectX 4 advanced acceleration technology enables higher cluster effi ciency and scalability to tens of thousands of n...

Страница 16: ...cards QSFP28 connec tors 2 2 Ethernet QSFP28 Interface The network ports of the ConnectX 4 adapter card are compliant with the IEEE 802 3 Ethernet standards listed in Table 5 Features on page 14 Ethe...

Страница 17: ...nts stated above 2 Shut down your system if active 3 After shutting down the system turn off power and unplug the cord 4 Remove the card from its package Please note that the card must be placed on an...

Страница 18: ...racket comes loose from the card 2 Save the two screws and the two fiber washers 3 5 2 Installing the New Bracket 1 Place the bracket onto the card until the screw holes line up 2 Screw on the bracket...

Страница 19: ...is off and the power cord is not con nected to the server Please follow proper electrical grounding procedures 2 Open the system case 3 Locate an available PCI Express slot for the adapter card 4 Appl...

Страница 20: ...able press the connector into the port receptacle until the connector is firmly seated a Support the weight of the cable before connecting the cable to the adapter card Do this by using a cable holder...

Страница 21: ...downward in the rack 6 To remove a cable disengage the locks and slowly pull the connector away from the port receptacle LED indicator will turn off when the cable is unseated 3 8 Adapter Card Un inst...

Страница 22: ...ull the adapter card in parallel to the motherboard 3 9 Identify the Card in Your System 3 9 1 On Windows 1 Open Device Manager on the server Click start Run and then enter devmgmt msc 2 Expand System...

Страница 23: ...ox Technologies and DEV is equal to 1003 this is a valid Mellanox Technologies PCI Device ID 3 9 2 On Linux Get the device location on the PCI bus by running lspci and locating lines with the string M...

Страница 24: ...You can download it from http www mellanox com Products Soft ware InfiniBand Drivers Table 6 Hardware and Software Requirements Requirements Description Platforms A server platform with an adapter car...

Страница 25: ...and then copy it to a shared file system such as NFS To install on all the cluster nodes use cluster aware tools such as pdsh If your kernel version does not match with any of the offered pre built RP...

Страница 26: ...as root Step 2 Mount the ISO image on your machine Step 3 Run the installation script MLNX_OFED_LINUX x x x rhel6 3 x86_64 mlnx_add_kernel_support sh m tmp MLNX_OFED_ LINUX x x x rhel6 3 x86_64 make...

Страница 27: ...n such as prefix kernel version and installation parameters can be retrieved by running the command etc infiniband info Most of the Mellanox OFED components can be configured or reconfigured after the...

Страница 28: ...directory openshmem bupc fca and ibutils The kernel modules are installed under lib modules uname r updates on SLES and Fedora Distributions lib modules uname r extra mlnx ofa_kernel on RHEL and othe...

Страница 29: ...machine as root Step 2 Mount the ISO image on your machine and copy its content to a shared location in your net work You can download it from http www mellanox com Products Software Infini Band Driv...

Страница 30: ...1 354 K s in 0s 2014 04 20 13 52 30 247 MB s RPM GPG KEY Mellanox saved 1354 1354 sudo rpm import RPM GPG KEY Mellanox warning rpmts_HdrFromFdno Header V3 DSA SHA1 Signature key ID 6224c050 NOKEY Ret...

Страница 31: ...nstaller package with KMP support mlnx ofed all Installs all available packages in MLNX_OFED mlnx ofed basic Installs basic packages required for running Mellanox cards mlnx ofed guest Installs packag...

Страница 32: ...tallation is applicable to Debian and Ubuntu Operating Systems 4 1 6 1 Setting up MLNX_OFED apt get Repository Step 1 Log into the installation machine as root Step 2 Extract the MLNX_OFED pacakge on...

Страница 33: ...C269 2013 08 11 sudo apt get update apt cache search mlnx ofed mlnx ofed vma eth MLNX_OFED vma eth installer package with DKMS support mlnx ofed hpc MLNX_OFED hpc installer package with DKMS support m...

Страница 34: ...are Step 4 Reboot your machine after the firmware burning is completed 4 1 7 3 Updating the Device Firmware Automatically upon System Boot As of MLNX_OFED v3 1 x x x firmware can be automatically upda...

Страница 35: ...he system key ring by the kernel Follow these steps below to add the Mellanox s x 509 public key to your system Step 1 Download the x 509 public key Step 2 Add the public key to the MOK list using the...

Страница 36: ...oot for the first module that either has no signature or whose key is not in the kernel key ring So it s much easier to miss this mes sage You won t see it on repeated tests where you unload and reloa...

Страница 37: ...e from http www mellanox com Products Software Firmware Tools 5 1 Firmware Update Example server1 mlxup Querying Mellanox devices firmware Device Type ConnectX4 Part Number MCX453A FCAT Description Co...

Страница 38: ...r in its slot or a different slot if necessary Try using another cable Reinstall the drivers for the network driver files may be damaged or deleted Reboot the server Adapters stopped working after ins...

Страница 39: ...etdev Card Detection lspci grep i Mellanox Mellanox Firmware Tool MFT Download and install MFT http www mellanox com content pages php pg management_tools menu_section 34 Refer to the User Manual for...

Страница 40: ...56GBASE R4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE KX 10GBASE SR 10GBASE LR 10GBASE ER 10GBASE CX4 10GB...

Страница 41: ...R4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE KX 10GBASE SR 10GBASE LR 10GBASE ER 10GBASE CX4 10GBASE KX4 1...

Страница 42: ...uct Ethernet 56GBASE R4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE KX 10GBASE SR 10GBASE LR 10GBASE ER 10GB...

Страница 43: ...R4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE KX 10GBASE SR 10GBASE LR 10GBASE ER 10GBASE CX4 10GBASE KX4 1...

Страница 44: ...oduct Ethernet 100GBASE CR4 100GBASE KR4 100GBASE SR4 56GBASE R4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE...

Страница 45: ...duct Ethernet 100GBASE CR4 100GBASE KR4 100GBASE SR4 56GBASE R4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE...

Страница 46: ...Port to Heat Sink Cable Type Passive Active 1 5W Active 3 5W Passive Active 1 5W Active 3 5W MCX453A FCAT 400 450 400 400 MCX454A FCAT 400 450 400 400 MCX455A FCAT 400 450 400 400 MCX456A FCAT 400 50...

Страница 47: ...er Card User Manual Rev 1 9 Mellanox Technologies 47 7 9 Board Mechanical Drawing and Dimensions Figure 2 Mechanical Drawing of the Single port x8 Adapter Cards Figure 3 Mechanical Drawing of the Dual...

Страница 48: ...ual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 48 Figure 4 Mechanical Drawing of the Single port x16 Adapter Cards Figure 5 Mechanical Drawing of the Dual port x16 Adapter Card...

Страница 49: ...ConnectX 4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 49 7 10 Bracket Mechanical Drawings Figure 6 Single port Bracket Figure 7 Dual port Bracket...

Страница 50: ...or Pinout Figure 8 Connector and Cage Views Table 14 Connector Pin Number and Name to Signal Name Map Connector Pin Number Connector Pin Name Port A Signal Name 1 GND GND 2 TXN_2 Tx2n 3 TXP_2 Tx2p 4 G...

Страница 51: ...21 RXN_2 Rx2n 22 RXP_2 Rx2p 23 GND GND 24 RXN_4 Rx4n 25 RXP_4 Rx4p 26 GND GND 27 ModPrsl_Port0 Mod PrsL 28 IntL IntL 29 VccTx 30 Vcc1 31 LPMode_Port0 LPMode 32 GND GND 33 TXP_3 Tx3p 34 TXN_3 Tx3n 35 G...

Страница 52: ...1 9 Mellanox Technologies 52 7 11 PCI Express x8 orx16 Edge Connector Pinout The adapter cards support PCI Express 3 0 x8 or x16 edge connector and the PCI Expressx8 or x16 standard pinout according...

Страница 53: ...ConnectX 4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 53 Figure 10 PCIe x16 Connector Pinout...

Страница 54: ...a different identifier printed on the label serial number and the card MAC for the Ethernet protocol and the card GUID for the InfiniBand protocol VPI cards have both a GUID and a MAC derived from th...

Страница 55: ...nectX 4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 55 Figure 13 MCX455A FCAT Board Label Figure 14 MCX456A FCAT Board Label Figure 15 MCX455A ECAT Board Lab...

Страница 56: ...ConnectX 4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 56 Figure 16 MCX456A ECAT Board Label...

Страница 57: ...air flow allow at least 8cm 3 inches of clearance around the ven tilation openings During periods of lightning activity do not work on the equipment or connect or dis connect cables Some copper cable...

Страница 58: ...e and Dual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 58 CLASS 1 LASER PRODUCT and reference to the most recent laser standards IEC 60 825 1 1993 A1 1997 A2 2001 and EN 60825 1...

Страница 59: ...200LFM cette temp rature ambiante maximale est n cessaire En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouver tures de ventilation Pendant...

Страница 60: ...xposition au rayonnement grave Mise en garde l utilisation de commandes ou de r glages ou l ex cution de proc dures autres que ce qui est sp cifi dans les pr sentes peut engendrer une exposition au ra...

Страница 61: ...erforderlich Au erdem sollten mindestens 8 cm 3 in Freiraum um die Bel ftungs ffnungen sein um einen einwandfreien Luftstrom zu gew hrleisten Arbeiten Sie w hrend eines Gewitters und Blitzschlag nich...

Страница 62: ...nox Technologies 62 8 Strahlenkontak Achtung Nutzung von Steuerungen oder Einstellungen oder Ausf hrung von Prozeduren die hier nicht spezifiziert sind kann zu gef hrlichem Strahlenkon takt f hren Kla...

Страница 63: ...cuada se debe dejar como m nimo un espacio de 8 cm 3 pulgadas alrededor de las aberturas de ventilaci n No utilizar el equipo ni conectar o desconectar cables durante per odos de actividad de rayos Da...

Страница 64: ...ConnectX 4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 64 PRODUCTO L SER DE CLASE 1 y referencia a las normas de l ser m s recientes IEC 60825 1...

Результаты поиска

Содержание MCX453A-FCAT

Отзывы:

Похожие инструкции для MCX453A-FCAT

Бренды по названию

Популярные бренды

Mellanox Technologies MCX453A-FCAT, Руководство пользователя

Результаты поиска

Содержание MCX453A-FCAT

Отзывы:

Похожие инструкции для MCX453A-FCAT

Бренды по названию

Популярные бренды