Mellanox Technologies ConnectX-4 MCX413A-BCAT User Manual Download Page 10

Page: 10 / 83

Rev 2.6

Mellanox Technologies

November

2015

1.1

• Updated power numbers in:

•

MCX413A-BCAT Specifications on page 58

•

MCX414A-BCAT Specifications on page 59

• MCX415A-BCAT Specifications on page 60
•

MCX413A-GCAT Specifications on page 62

• MCX416A-GCAT Specifications on page 65

• Added

Airflow Specifications on page 58

• Added

VMware Driver on page 50

• Updated

Updating Adapter Card Firmware on page 54

November

2015

1.0

First Release

Table 1 - Revision History Table

Date

Rev

Comments/Changes

Summary of Contents for ConnectX-4 MCX413A-BCAT

Page 1: ...ellanox com ConnectX 4 Ethernet Single and Dual QSFP28 Port Adapter Card User Manual P N MCX413A BCAT MCX414A BCAT MCX415A BCAT MCX416A BCAT MCX413A GCAT MCX414A GCAT MCX415A GCAT MCX416A GCAT MCX415A...

Page 2: ...ww mellanox com page trademarks NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES AS IS WITH ALL FAULTS OF ANY KIND AND SO...

Page 3: ...ter 2 Interfaces 16 2 1 Ethernet QSFP28 Interface 16 2 2 PCI Express Interface 16 2 3 LED Interface 16 Chapter 3 Hardware Installation 17 3 1 System Requirements 17 3 1 1 Hardware 17 3 2 Operating Sys...

Page 4: ...lling MLNX_OFED using apt get 33 4 1 6 1 Setting up MLNX_OFED apt get Repository 33 4 1 6 2 Installing MLNX_OFED using the apt get Tool 33 4 1 6 3 Uninstalling Mellanox OFED using the apt get Tool 34...

Page 5: ...s 58 7 1 MCX413A BCAT Specifications 58 7 2 MCX414A BCAT Specifications 59 7 3 MCX415A BCAT Specifications 60 7 4 MCX416A BCAT Specifications 61 7 5 MCX413A GCAT Specifications 62 7 6 MCX414A GCAT Spe...

Page 6: ...A BCAT Specifications Table 58 Table 9 MCX414A BCAT Specifications Table 59 Table 10 MCX415A BCAT Specifications Table 60 Table 11 MCX416A BCAT Specifications Table 61 Table 12 MCX413A GCAT Specificat...

Page 7: ...e Dual port x16 Adapter Card 70 Figure 7 Single port Bracket 71 Figure 8 Dual port Bracket 71 Figure 9 MCX413A BCAT Board Label Example 72 Figure 10 MCX414A BCAT Board Label Example 72 Figure 11 MCX41...

Page 8: ...ications on page 58 Updated Cable Installation on page 20 July 2016 2 2 Updated Adapter Card Installation Instructions on page 18 Added the following mechanical drawings Mechanical Drawing of the Sing...

Page 9: ...rd Label Example on page 73 December 2015 1 5 Updated Airflow Specifications on page 58 December 2015 1 4 Updated MCX414A GCAT Specifications on page 63 Updated MCX416A GCAT Specifications on page 65...

Page 10: ...ns on page 59 MCX415A BCAT Specifications on page 60 MCX413A GCAT Specifications on page 62 MCX416A GCAT Specifications on page 65 Added Airflow Specifications on page 58 Added VMware Driver on page 5...

Page 11: ...ility Mellanox OFED for Linux User Manual Document no 2877 User Manual describing OFED features performance InfiniBand diagnostic tools content and configuration See http www mellanox com Products Sof...

Page 12: ...pproved reseller should first seek assistance through their reseller Firmware Updates The Mellanox support downloader contains software firmware and knowledge database infor mation for Mellanox produc...

Page 13: ...mber of ports and PCI Express speed Each adapter comes with two bracket heights short and tall Table 3 Single and Dual port EDR Adapter Cards Ordering Part Number OPN Single port cards MCX413A BCAT MC...

Page 14: ...er operators often create overlay networks that carry traffic from individual virtual machines over logical tunnels in encapsulated formats such as NVGRE and VXLAN While this solves network scalabilit...

Page 15: ...cation require ments for latency and SLA Hardware based I O Virtualization ConnectX 4 provides dedicated adapter resources and guaranteed isolation and protection for virtual machines within the serve...

Page 16: ...ansmitted through the cards QSFP28 connectors 2 2 PCI Express Interface The ConnectX 4 adapter card supports PCI Express 3 0 1 1 and 2 0 compatible through an x8 or x16 edge connector The device can b...

Page 17: ...er antistatic devices 3 4 Pre installation Checklist 1 Verify that your system meets the hardware and software requirements stated above 2 Shut down your system if active 3 After shutting down the sys...

Page 18: ...s 3 5 2 Installing the New Bracket 1 Place the bracket onto the adapter card until the screw holes line up 2 Screw on the bracket using the screws saved from the bracket removal procedure above 3 Make...

Page 19: ...on Rev 2 6 19 Mellanox Technologies 3 Locate an available PCI Express slot for the adapter card 4 Applying even pressure at both corners of the card insert the adapter card into the PCI Express slot u...

Page 20: ...to the port receptacle until the connector is firmly seated a Support the weight of the cable before connecting the cable to the adapter card Do this by using a cable holder or tying the cable to the...

Page 21: ...before bending upward or downward in the rack 6 To remove a cable disengage the locks and slowly pull the connector away from the port receptacle LED indicator will turn off when the cable is unseate...

Page 22: ...parallel to the motherboard 3 9 Identify the Card in Your System 3 9 1 On Windows 1 Open Device Manager on the server Click start Run and then enter devmgmt msc 2 Expand System Devices and locate you...

Page 23: ...DEV is equal to 1003 this is a valid Mellanox Technologies PCI Device ID 3 9 2 On Linux Get the device location on the PCI bus by running lspci and locating lines with the string Mella nox Technologi...

Page 24: ...rom http www mellanox com Products Soft ware Ethernet Drivers Table 5 Hardware and Software Requirements Requirements Description Platforms A server platform with an adapter card based on one of the f...

Page 25: ...prompted to acknowledge the deletion of the old packages If you need to install Mellanox OFED on an entire homogeneous cluster a common strategy is to mount the ISO image on one of the cluster nodes a...

Page 26: ...Linux has already installed the OpenSM Subnet Manager on your machine 4 1 3 1 Installation Procedure Step 1 Login to the installation machine as root Step 2 Mount the ISO image on your machine On Ubun...

Page 27: ...stall the MLNX_OFED_LINUX package on your machine Note that all other Mellanox OEM OFED or Distribution IB packages will be removed Uninstalling the previous version of MLNX_OFED_LINUX Starting MLNX_O...

Page 28: ...e openibd script supports pre post start stop scripts Software Most of MLNX_OFED packages are installed under the usr directory except for the following packages which are installed under the opt dire...

Page 29: ...nstall the Mellanox OFED package The script is part of the ofed scripts RPM 4 1 5 Installing MLNX_OFED using YUM This type of installation is applicable to RedHat OEL Fedora XenServer Operating System...

Page 30: ...0 http www mellanox com downloads ofed RPM GPG KEY Mellanox Resolving www mellanox com 72 3 194 0 Connecting to www mellanox com 72 3 194 0 80 connected HTTP request sent awaiting response 200 OK Leng...

Page 31: ...hpc installer package with KMP support mlnx ofed hypervisor noarch MLNX_OFED hypervisor installer package with KMP support mlnx ofed vma noarch MLNX_OFED vma installer package with KMP support mlnx of...

Page 32: ...stalling Mellanox OFED using the YUM Tool Use the script usr sbin ofed_uninstall sh to uninstall the Mellanox OFED package The script is part of the ofed scripts RPM mlnx ofed hypervisor 3 17 4 301 fc...

Page 33: ...w the available package groups by invoking Where deb file path to extracted MLNX_OFED package wget qO http www mellanox com downloads ofed RPM GPG KEY Mellanox sudo apt key add apt key list pub 1024D...

Page 34: ...t get install mlnx ofed all Installing MLNX_OFED using the apt get tool does not automatically update the firmware To update the firmware to the version included in MLNX_OFED package run apt get insta...

Page 35: ...nibd service script can invoke the firmware update process if requested on boot If the firmware is updated the following message is printed to the system s standard logging file Otherwise the followin...

Page 36: ...the system key ring and subsequent will be booted when the UEFI Secure Boot is enabled 4 1 8 2 Removing Signature from Kernel Modules The signature can be removed from a signed kernel module using the...

Page 37: ...via the Mellanox web site at http www mellanox com Products Software Ethernet Drivers Windows SW Driver Download Follow the installation instructions included in the download package also available f...

Page 38: ...ntervention Unattended Installation An automated installation procedure that requires no user intervention 4 2 3 1 Attended Installation The following is an example of an installation session Step 1 D...

Page 39: ...Driver Installation Rev 2 6 39 Mellanox Technologies Step 4 Click Next in the Welcome screen Step 5 Read then accept the license agreement and click Next...

Page 40: ...Driver Installation Rev 2 6 40 Mellanox Technologies Step 6 Select the target folder for the installation...

Page 41: ...the following cases If the user has an OEM card In this case the firmware will not be displayed If the user has a standard Mellanox card with an older firmware version the firmware will be updated acc...

Page 42: ...net from a host OpenSM is part of the driver and installed automatically Performances tools install the performance tools that are used to measure performance in user environment Analyze tools install...

Page 43: ...Driver Installation Rev 2 6 43 Mellanox Technologies Management tools installation tools used for management such as mlxstat b Click Install to start the installation...

Page 44: ...llation Rev 2 6 44 Mellanox Technologies Step 9 In case firmware upgrade option was checked in step Step 7 you will be notified if a firm ware upgrade is required Step 10 Click Finish to complete the...

Page 45: ...Manager Upon installation completion the inf files can be located at ProgramFiles Mellanox MLNX_WinOF2 Drivers OS To see the Mellanox network adapters display the Device Manager and pull down the Net...

Page 46: ...ning Installation To extract the files without running installation perform the following steps Step 1 Open a CMD console Click Start Task Manager File Run new task and enter CMD Step 2 Extract the dr...

Page 47: ...Driver Installation Rev 2 6 47 Mellanox Technologies Step 3 Click Next to create a server image Step 4 Click Change and specify the location in which the files are extracted to...

Page 48: ...folder or click Change to install to a different folder Step 6 To complete the extraction click Finish 4 2 6 Uninstalling Mellanox WinOF 2 Driver 4 2 6 1 Attended Uninstallation To uninstall MLNX_WinO...

Page 49: ...the Driver on a Nano Server 4 2 8 1 Offline Installation To deploy the Driver in Nano Server Step 1 Go to the Mellanox WinOF web page at http www mellanox com Products Ethernet Drivers Windows SW Driv...

Page 50: ...rectory see Section 4 2 5 Extracting Files With out Running Installation on page 46 Step 4 Open a remote connection to the Nano server Step 5 Copy all the driver files to the Nano server Step 6 Instal...

Page 51: ...ble 7 Hardware and Software Requirements Requirements Description Platforms A server platform with an adapter card based on one of the following Mellanox Technologies HCA devices MT27700 ConnectX 4 EN...

Page 52: ...Tools MFT site ESX 5 5 File mft 4 3 0 25 10EM 550 0 0 1391871 x86_64 vib MD5SUM 62f953b77f4ea49c420ff4e1ce8b925b ESX 6 0 File mft 4 3 0 25 10EM 600 0 0 2768847 x86_64 vib MD5SUM 6a3bd63882f2ca3f696c1...

Page 53: ...tion Rev 2 6 53 Mellanox Technologies Install the image according to the steps described in the MFT User Manual The following procedure requires custom boot image downloading mounting and boot ing fro...

Page 54: ...er1 mlxup Querying Mellanox devices firmware Device Type ConnectX4 Part Number MCX413A BCAT Description ConnectX 4 EN network interface card 40 56 GbE single port QSFP28 PCIe3 0 x8 tall bracket ROHS R...

Page 55: ...slot if necessary Try using another cable Reinstall the drivers for the network driver files may be damaged or deleted Reboot the server Adapters stopped working after installing another adapter Try...

Page 56: ...rep i Mellanox Mellanox Firmware Tool MFT Download and install MFT http www mellanox com con tent pages php pg management_tools menu_section 34 Refer to the User Manual for installation instructions O...

Page 57: ...on tent pages php pg management_tools menu_section 34 Refer to the User Manual for installation instructions Once installed open a CMD window and run WinMFT mst start mst status flint d mst_device q P...

Page 58: ...0GBASE KX4 10GBASE CR 10GBASE KR SGMII Data Rate 1 10 25 40 56 Gb s Ethernet PCI Express Gen3 SERDES 8 0GT s 8 lanes 2 0 and 1 1 compatible Power and Environmental Voltage 12V 3 3V Power Cable 40GbE 5...

Page 59: ...10GBASE CR 10GBASE KR SGMII Data Rate 1 10 25 40 56 Gb s Ethernet PCI Express Gen3 SERDES 8 0GT s 8 lanes 2 0 and 1 1 compatible Power and Environmental Voltage 12V 3 3V Power Cable 40GbE 56GbE Typic...

Page 60: ...40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE KX 10GBASE SR 10GBASE LR 10GBASE ER 10GBASE CX4 10GBASE KX4 10GBASE CR 10GBASE KR SGMII Data Rate Ethernet 1 10 25 40 56 G...

Page 61: ...10GBASE CR 10GBASE KR SGMII Data Rate 1 10 25 40 56 Gb s Ethernet PCI Express Gen3 SERDES 8 0GT s 16 lanes 2 0 and 1 1 compatible Power and Environmental Voltage 12V 3 3V Power Cable 40GbE 56GbE Typi...

Page 62: ...E KX4 10GBASE CR 10GBASE KR SGMII Data Rate Ethernet 1 10 25 40 50 Gb s PCI Express Gen3 SERDES 8 0GT s 8 lanes 2 0 and 1 1 compatible Power and Environmental Voltage 12V 3 3V Typical Powera a Typical...

Page 63: ...KX4 10GBASE CR 10GBASE KR SGMII Data Rate 1 10 25 40 50 Gb s Ethernet PCI Express Gen3 SERDES 8 0GT s 8 lanes 2 0 and 1 1 compatible Power and Environmental Voltage 12V 3 3V Typical Powera a Typical...

Page 64: ...E LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 1000BASE CX 1000BASE KX 10GBASE SR 10GBASE LR 10GBASE ER 10GBASE CX4 10GBASE KX4 10GBASE CR 10GBASE KR SGMII Data Rate Ethernet 1 10 25 40 50 Gb s PC...

Page 65: ...KX4 10GBASE CR 10GBASE KR SGMII Data Rate Ethernet 1 10 25 40 50 Gb s PCI Express Gen3 SERDES 8 0GT s 16 lanes 2 0 and 1 1 compatible Power and Environmental Voltage 12V 3 3V Typical Powera a Typical...

Page 66: ...ER 10GBASE CX4 10GBASE KX4 10GBASE CR 10GBASE KR SGMII Data Rate Ethernet 1 10 25 40 50 100 Gb s PCI Express Gen3 SERDES 8 0GT s 16 lanes 2 0 and 1 1 compatible Power and Environmental Voltage 12V 3...

Page 67: ...ER 10GBASE CX4 10GBASE KX4 10GBASE CR 10GBASE KR SGMII Data Rate Ethernet 1 10 25 40 50 100 Gb s PCI Express Gen3 SERDES 8 0GT s 16 lanes 2 0 and 1 1 compatible Power and Environmental Voltage 12V 3...

Page 68: ...ctive 3 5W Passive Active 1 5W Active 3 5W MCX413A BCAT 400 450 400 400 MCX414A BCAT 400 450 400 400 MCX416A BCAT 400 500 400 400 MCX413A GCAT 400 400 400 400 MCX414A GCAT 400 450 700 400 400 600 MCX4...

Page 69: ...Mechanical Drawing and Dimensions Figure 3 Mechanical Drawing of the Single port x8 Adapter Cards Figure 4 Mechanical Drawing of the Dual port x8 Adapter Cards All dimensions are in millimeters All th...

Page 70: ...Specifications Rev 2 6 70 Mellanox Technologies Figure 5 Mechanical Drawing of the Single port x16 Adapter Card Figure 6 Mechanical Drawing of the Dual port x16 Adapter Card 142 24 68 90 142 24 68 90...

Page 71: ...Specifications Rev 2 6 71 Mellanox Technologies 7 14 Bracket Mechanical Drawing Figure 7 Single port Bracket Figure 8 Dual port Bracket...

Page 72: ...card has a different identifier printed on the label serial number and the card MAC for the Ethernet protocol Figure 9 MCX413A BCAT Board Label Example Figure 10 MCX414A BCAT Board Label Example The r...

Page 73: ...Rev 2 6 73 Mellanox Technologies Figure 11 MCX415A BCAT Board Label Example Figure 12 MCX416A BCAT Board Label Example Figure 13 MCX413A GCAT Board Label Example...

Page 74: ...Rev 2 6 74 Mellanox Technologies Figure 14 MCX414A GCAT Board Label Example Figure 15 MCX415A GCAT Board Label Example Figure 16 MCX415A CCAT Board Label Example...

Page 75: ...Rev 2 6 75 Mellanox Technologies Figure 17 MCX416A GCAT Board Label Example Figure 18 MCX416A CCAT Board Label Example...

Page 76: ...1 F To guarantee proper air flow allow at least 8cm 3 inches of clearance around the ven tilation openings During periods of lightning activity do not work on the equipment or connect or dis connect c...

Page 77: ...se of controls or adjustment or performance of procedures other than those specified herein may result in hazardous radiation exposure CLASS 1 LASER PRODUCT and reference to the most recent laser stan...

Page 78: ...maximale est n cessaire En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouver tures de ventilation Pendant un orage il ne faut pas utiliser l...

Page 79: ...e en garde l utilisation de commandes ou de r glages ou l ex cution de proc dures autres que ce qui est sp cifi dans les pr sentes peut engendrer une exposi tion au rayonnement grave PRODUIT LASER DE...

Page 80: ...zar una circulaci n de aire adecuada se debe dejar como m nimo un espacio de 8 cm 3 pulgadas alrededor de las aberturas de ventilaci n No utilizar el equipo ni conectar o desconectar cables durante pe...

Page 81: ...ligrosos Precauci n el uso de controles o ajustes o la realizaci n de procedimientos distintos de los que aqu se especifican podr an causar exposici n a niveles de radiaci n peligrosos PRODUCTO L SER...

Page 82: ...gstemperatur erforderlich Au erdem sollten mindestens 8 cm 3 in Freiraum um die Bel ftungs ffnungen sein um einen einwandfreien Luftstrom zu gew hrleisten Arbeiten Sie w hrend eines Gewitters und Blit...

Page 83: ...ak Achtung Nutzung von Steuerungen oder Einstellungen oder Ausf hrung von Prozeduren die hier nicht spezifiziert sind kann zu gef hrlichem Strahlenkon takt f hren Klasse 1 Laserprodukt und Referenzen...

Search results

Summary of Contents for ConnectX-4 MCX413A-BCAT

Reviews:

Related manuals for ConnectX-4 MCX413A-BCAT

Brands by name

Popular brands

Mellanox Technologies ConnectX-4 MCX413A-BCAT, User Manual

Search results

Summary of Contents for ConnectX-4 MCX413A-BCAT

Reviews:

Related manuals for ConnectX-4 MCX413A-BCAT

Brands by name

Popular brands