Mellanox Technologies MCX453A-FCAT User Manual Download Page 1

Page: 1 / 64

www.mellanox.com

ConnectX

-4 VPI Single and Dual QSFP28

Adapter Card User Manual

P/N:

MCX453A-FCAT, MCX454A-FCAT, MCX455A-FCAT, MCX456A-FCAT, MCX455A-ECAT, MCX456A-

ECAT

Rev 1.9

Summary of Contents for MCX453A-FCAT

Page 1: ...www mellanox com ConnectX 4 VPI Single and Dual QSFP28 Adapter Card User Manual P N MCX453A FCAT MCX454A FCAT MCX455A FCAT MCX456A FCAT MCX455A ECAT MCX456A ECAT Rev 1 9...

Page 2: ...OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLI...

Page 3: ...net QSFP28 Interface 16 2 3 PCI Express Interface 16 2 4 LED Interface 16 Chapter 3 Hardware Installation 17 3 1 System Requirements 17 3 1 1 Hardware 17 3 2 Operating Systems Distributions 17 3 2 1 S...

Page 4: ...ting Firmware After Installation 33 4 1 7 1 Updating the Device Online 34 4 1 7 2 Updating the Device Manually 34 4 1 7 3 Updating the Device Firmware Automatically upon System Boot 34 4 1 8 UEFI Secu...

Page 5: ...Rev 1 9 Mellanox Technologies 5 Appendix C Safety Warnings 57 Appendix D Avertissements de s curit d installation Warnings in French 59 Appendix E Sicherheitshinweise Warnings in German 61 Appendix F...

Page 6: ...Single and Dual port EDR Adapter Cards 13 Table 5 Features 14 Table 6 Hardware and Software Requirements 24 Table 7 MCX453A FCAT Specifications Table 40 Table 8 MCX454A FCAT Specifications Table 41 T...

Page 7: ...he Single port x16 Adapter Cards 48 Figure 5 Mechanical Drawing of the Dual port x16 Adapter Card 48 Figure 6 Single port Bracket 49 Figure 7 Dual port Bracket 49 Figure 8 Connector and Cage Views 50...

Page 8: ...ions Product Overview on page 12 Features on page 14 MCX453A FCAT Specifications on page 40 MCX454A FCAT Specifications on page 41 MCX455A FCAT Specifications on page 42 MCX456A FCAT Specifications on...

Page 9: ...ons on page 42 MCX456A FCAT Specifications on page 43 MCX455A ECAT Specifications on page 44 MCX456A ECAT Specifications on page 45 Updated Features and Benefits on page 16 Updated Section 3 9 1 On Wi...

Page 10: ...de See http www mellanox com Products Software Firm ware Tools Mellanox Firmware Utility mlxup User Manual and Release Notes Mellanox firmware update and query utility used to update the firmware See...

Page 11: ...l Support Services please see your contract for details regarding Technical Support Customers who purchased Mellanox products through a Mellanox approved reseller should first seek assistance through...

Page 12: ...ributions on page 15 Section OpenFabrics Enterprise Distribution OFED on page 15 1 1 Product Overview The following section provides the ordering part number port speed number of ports and PCI Express...

Page 13: ...455A ECAT Single port card MCX456A ECAT Dual port card Data Transmission Rate InfiniBand SDR DDR QDR FDR EDR Ethernet 1 10 25 40 50 100 Gb s Network Connector Types Single or Dual port QSFP28 PCI Expr...

Page 14: ...net IEEE 802 3ap based auto negotiation and KRstartup IEEE 802 3ad 802 1AX Link Aggregation IEEE 802 1Q 802 1P VLAN tags and priority IEEE 802 1Qau QCN Congestion Notification IEEE 802 1Qaz ETS IEEE 8...

Page 15: ...xample from GPU to CPU and therefore significantly reduces application run time ConnectX 4 advanced acceleration technology enables higher cluster effi ciency and scalability to tens of thousands of n...

Page 16: ...cards QSFP28 connec tors 2 2 Ethernet QSFP28 Interface The network ports of the ConnectX 4 adapter card are compliant with the IEEE 802 3 Ethernet standards listed in Table 5 Features on page 14 Ethe...

Page 17: ...nts stated above 2 Shut down your system if active 3 After shutting down the system turn off power and unplug the cord 4 Remove the card from its package Please note that the card must be placed on an...

Page 18: ...racket comes loose from the card 2 Save the two screws and the two fiber washers 3 5 2 Installing the New Bracket 1 Place the bracket onto the card until the screw holes line up 2 Screw on the bracket...

Page 19: ...is off and the power cord is not con nected to the server Please follow proper electrical grounding procedures 2 Open the system case 3 Locate an available PCI Express slot for the adapter card 4 Appl...

Page 20: ...able press the connector into the port receptacle until the connector is firmly seated a Support the weight of the cable before connecting the cable to the adapter card Do this by using a cable holder...

Page 21: ...downward in the rack 6 To remove a cable disengage the locks and slowly pull the connector away from the port receptacle LED indicator will turn off when the cable is unseated 3 8 Adapter Card Un inst...

Page 22: ...ull the adapter card in parallel to the motherboard 3 9 Identify the Card in Your System 3 9 1 On Windows 1 Open Device Manager on the server Click start Run and then enter devmgmt msc 2 Expand System...

Page 23: ...ox Technologies and DEV is equal to 1003 this is a valid Mellanox Technologies PCI Device ID 3 9 2 On Linux Get the device location on the PCI bus by running lspci and locating lines with the string M...

Page 24: ...You can download it from http www mellanox com Products Soft ware InfiniBand Drivers Table 6 Hardware and Software Requirements Requirements Description Platforms A server platform with an adapter car...

Page 25: ...and then copy it to a shared file system such as NFS To install on all the cluster nodes use cluster aware tools such as pdsh If your kernel version does not match with any of the offered pre built RP...

Page 26: ...as root Step 2 Mount the ISO image on your machine Step 3 Run the installation script MLNX_OFED_LINUX x x x rhel6 3 x86_64 mlnx_add_kernel_support sh m tmp MLNX_OFED_ LINUX x x x rhel6 3 x86_64 make...

Page 27: ...n such as prefix kernel version and installation parameters can be retrieved by running the command etc infiniband info Most of the Mellanox OFED components can be configured or reconfigured after the...

Page 28: ...directory openshmem bupc fca and ibutils The kernel modules are installed under lib modules uname r updates on SLES and Fedora Distributions lib modules uname r extra mlnx ofa_kernel on RHEL and othe...

Page 29: ...machine as root Step 2 Mount the ISO image on your machine and copy its content to a shared location in your net work You can download it from http www mellanox com Products Software Infini Band Driv...

Page 30: ...1 354 K s in 0s 2014 04 20 13 52 30 247 MB s RPM GPG KEY Mellanox saved 1354 1354 sudo rpm import RPM GPG KEY Mellanox warning rpmts_HdrFromFdno Header V3 DSA SHA1 Signature key ID 6224c050 NOKEY Ret...

Page 31: ...nstaller package with KMP support mlnx ofed all Installs all available packages in MLNX_OFED mlnx ofed basic Installs basic packages required for running Mellanox cards mlnx ofed guest Installs packag...

Page 32: ...tallation is applicable to Debian and Ubuntu Operating Systems 4 1 6 1 Setting up MLNX_OFED apt get Repository Step 1 Log into the installation machine as root Step 2 Extract the MLNX_OFED pacakge on...

Page 33: ...C269 2013 08 11 sudo apt get update apt cache search mlnx ofed mlnx ofed vma eth MLNX_OFED vma eth installer package with DKMS support mlnx ofed hpc MLNX_OFED hpc installer package with DKMS support m...

Page 34: ...are Step 4 Reboot your machine after the firmware burning is completed 4 1 7 3 Updating the Device Firmware Automatically upon System Boot As of MLNX_OFED v3 1 x x x firmware can be automatically upda...

Page 35: ...he system key ring by the kernel Follow these steps below to add the Mellanox s x 509 public key to your system Step 1 Download the x 509 public key Step 2 Add the public key to the MOK list using the...

Page 36: ...oot for the first module that either has no signature or whose key is not in the kernel key ring So it s much easier to miss this mes sage You won t see it on repeated tests where you unload and reloa...

Page 37: ...e from http www mellanox com Products Software Firmware Tools 5 1 Firmware Update Example server1 mlxup Querying Mellanox devices firmware Device Type ConnectX4 Part Number MCX453A FCAT Description Co...

Page 38: ...r in its slot or a different slot if necessary Try using another cable Reinstall the drivers for the network driver files may be damaged or deleted Reboot the server Adapters stopped working after ins...

Page 39: ...etdev Card Detection lspci grep i Mellanox Mellanox Firmware Tool MFT Download and install MFT http www mellanox com content pages php pg management_tools menu_section 34 Refer to the User Manual for...

Page 40: ...56GBASE R4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE KX 10GBASE SR 10GBASE LR 10GBASE ER 10GBASE CX4 10GB...

Page 41: ...R4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE KX 10GBASE SR 10GBASE LR 10GBASE ER 10GBASE CX4 10GBASE KX4 1...

Page 42: ...uct Ethernet 56GBASE R4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE KX 10GBASE SR 10GBASE LR 10GBASE ER 10GB...

Page 43: ...R4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE KX 10GBASE SR 10GBASE LR 10GBASE ER 10GBASE CX4 10GBASE KX4 1...

Page 44: ...oduct Ethernet 100GBASE CR4 100GBASE KR4 100GBASE SR4 56GBASE R4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE...

Page 45: ...duct Ethernet 100GBASE CR4 100GBASE KR4 100GBASE SR4 56GBASE R4 50GBASE R2 50GBASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE...

Page 46: ...Port to Heat Sink Cable Type Passive Active 1 5W Active 3 5W Passive Active 1 5W Active 3 5W MCX453A FCAT 400 450 400 400 MCX454A FCAT 400 450 400 400 MCX455A FCAT 400 450 400 400 MCX456A FCAT 400 50...

Page 47: ...er Card User Manual Rev 1 9 Mellanox Technologies 47 7 9 Board Mechanical Drawing and Dimensions Figure 2 Mechanical Drawing of the Single port x8 Adapter Cards Figure 3 Mechanical Drawing of the Dual...

Page 48: ...ual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 48 Figure 4 Mechanical Drawing of the Single port x16 Adapter Cards Figure 5 Mechanical Drawing of the Dual port x16 Adapter Card...

Page 49: ...ConnectX 4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 49 7 10 Bracket Mechanical Drawings Figure 6 Single port Bracket Figure 7 Dual port Bracket...

Page 50: ...or Pinout Figure 8 Connector and Cage Views Table 14 Connector Pin Number and Name to Signal Name Map Connector Pin Number Connector Pin Name Port A Signal Name 1 GND GND 2 TXN_2 Tx2n 3 TXP_2 Tx2p 4 G...

Page 51: ...21 RXN_2 Rx2n 22 RXP_2 Rx2p 23 GND GND 24 RXN_4 Rx4n 25 RXP_4 Rx4p 26 GND GND 27 ModPrsl_Port0 Mod PrsL 28 IntL IntL 29 VccTx 30 Vcc1 31 LPMode_Port0 LPMode 32 GND GND 33 TXP_3 Tx3p 34 TXN_3 Tx3n 35 G...

Page 52: ...1 9 Mellanox Technologies 52 7 11 PCI Express x8 orx16 Edge Connector Pinout The adapter cards support PCI Express 3 0 x8 or x16 edge connector and the PCI Expressx8 or x16 standard pinout according...

Page 53: ...ConnectX 4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 53 Figure 10 PCIe x16 Connector Pinout...

Page 54: ...a different identifier printed on the label serial number and the card MAC for the Ethernet protocol and the card GUID for the InfiniBand protocol VPI cards have both a GUID and a MAC derived from th...

Page 55: ...nectX 4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 55 Figure 13 MCX455A FCAT Board Label Figure 14 MCX456A FCAT Board Label Figure 15 MCX455A ECAT Board Lab...

Page 56: ...ConnectX 4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 56 Figure 16 MCX456A ECAT Board Label...

Page 57: ...air flow allow at least 8cm 3 inches of clearance around the ven tilation openings During periods of lightning activity do not work on the equipment or connect or dis connect cables Some copper cable...

Page 58: ...e and Dual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 58 CLASS 1 LASER PRODUCT and reference to the most recent laser standards IEC 60 825 1 1993 A1 1997 A2 2001 and EN 60825 1...

Page 59: ...200LFM cette temp rature ambiante maximale est n cessaire En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouver tures de ventilation Pendant...

Page 60: ...xposition au rayonnement grave Mise en garde l utilisation de commandes ou de r glages ou l ex cution de proc dures autres que ce qui est sp cifi dans les pr sentes peut engendrer une exposition au ra...

Page 61: ...erforderlich Au erdem sollten mindestens 8 cm 3 in Freiraum um die Bel ftungs ffnungen sein um einen einwandfreien Luftstrom zu gew hrleisten Arbeiten Sie w hrend eines Gewitters und Blitzschlag nich...

Page 62: ...nox Technologies 62 8 Strahlenkontak Achtung Nutzung von Steuerungen oder Einstellungen oder Ausf hrung von Prozeduren die hier nicht spezifiziert sind kann zu gef hrlichem Strahlenkon takt f hren Kla...

Page 63: ...cuada se debe dejar como m nimo un espacio de 8 cm 3 pulgadas alrededor de las aberturas de ventilaci n No utilizar el equipo ni conectar o desconectar cables durante per odos de actividad de rayos Da...

Page 64: ...ConnectX 4 VPI Single and Dual Port QSFP28 Adapter Card User Manual Rev 1 9 Mellanox Technologies 64 PRODUCTO L SER DE CLASE 1 y referencia a las normas de l ser m s recientes IEC 60825 1...

Search results

Summary of Contents for MCX453A-FCAT

Reviews:

Related manuals for MCX453A-FCAT

Brands by name

Popular brands

Mellanox Technologies MCX453A-FCAT, User Manual

Search results

Summary of Contents for MCX453A-FCAT

Reviews:

Related manuals for MCX453A-FCAT

Brands by name

Popular brands