Mellanox Technologies ConnectX-3 Pro User Manual Download

Page: 1 / 66

www.mellanox.com

ConnectX

-3 Pro Ethernet Single and Dual

SFP+ Port Adapter Card User Manual

P/N:

MCX312B-XCCT, MCX312B-XCCS, MCX311A-XCCT, MCX312C-XCCT

Rev 1.5

Summary of Contents for ConnectX-3 Pro

Page 1: ...www mellanox com ConnectX 3 Pro Ethernet Single and Dual SFP Port Adapter Card User Manual P N MCX312B XCCT MCX312B XCCS MCX311A XCCT MCX312C XCCT Rev 1 5...

Page 2: ...he most updated list of Mellanox trademarks visit http www mellanox com page trademarks NOTE THIS HARDWARE SOFTWARE OR TEST SUITE PRODUCT PRODUCT S AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLAN...

Page 3: ...ress Interface 13 2 3 I2C compatible Interface 14 2 4 LED Interface 14 Chapter 3 Hardware Installation 15 3 1 System Requirements 15 3 1 1 Hardware 15 3 1 2 Operating Systems Distributions 15 3 1 3 So...

Page 4: ...2 Installing and Running the offline_bundle Driver on ESXi 5 x 40 4 4 3 Removing the VIB offline_bundle Driver 40 4 5 FlexBoot 41 Chapter 5 Updating Adapter Card Firmware 42 Chapter 6 Troubleshooting...

Page 5: ...l port 10 Gigabit Ethernet Adapter Cards 10 Table 4 Features 11 Table 5 Software and Hardware Requirements 22 Table 6 Software and Hardware Requirements 24 Table 7 MCX312B XCC S T Specifications Table...

Page 6: ...Figure 7 Module With Locking Mechanism Open 19 Figure 8 PCI Device 20 Figure 9 Support Download Assistant 43 Figure 10 Mechanical Drawing of the Dual port MCX312B XCCT Adapter Card 50 Figure 11 Mechan...

Page 7: ...ecifications on page 47 Section 7 2 MCX312C XCCT Specifications on page 48 Section 7 3 MCX311A XCCT Specifications on page 49 September 2015 1 2 Updated bracket installation instructions See Section 3...

Page 8: ...ware Firm ware Tools MLNX_EN for Linux README Driver Kit for Mellanox Adapter Cards with 10GigE Support Document no 2950 User Manual providing information on the MLNX_EN Linux driver and instructions...

Page 9: ...Services please see your contract for details regarding Technical Support Customers who purchased Mellanox products through a Mellanox approved reseller should first seek assistance through their res...

Page 10: ...age 11 Section 1 3 Operating Systems Distributions on page 12 Section 1 4 Connectivity on page 12 1 1 Product Overview The following section provides the ordering part number port speed number of port...

Page 11: ...RDMA services over Layer 2 Ethernet CPU offload Adapter functionality enabling reduced CPU overhead allowing more available CPU GPUDirect RDMA Using GPUDirect RDMA adapters can directly read and write...

Page 12: ...SLES 11 SP2 OEL Fedora 14 15 17 Ubuntu 12 04 Windows Server 2008 2012 FreeBSD OpenFabrics Enterprise Distribution OFED OpenFabrics Windows Distribution WinOF VMware ESXi 4 x and 5 x 1 4 Connectivity...

Page 13: ...SFP Interface The network ports of the ConnectX 3 Pro adapter cards are compliant with the IEEE 802 3 Ethernet standards listed in Table 4 Features on page 11 The SFP port has one Tx Rx pair of SerDe...

Page 14: ...ink rate x8 Auto negotiates to x8 x4 x2 or x1 Support for MSI MSI X mechanisms 2 3 I2C compatible Interface A three pin header on the adapter cards is provided as the I2C compatible interface 2 4 LED...

Page 15: ...bove 2 Shut down your system if active 3 After shutting down the system turn off power and unplug the cord 4 Remove the card from its package Please note that the card must be placed on an antistatic...

Page 16: ...1 Removing the Existing Bracket Figure 2 Bracket Screws Figure 3 Bracket Screws 1 Remove the two screws holding the bracket in place The bracket comes loose from the card 2 Save the two screws and th...

Page 17: ...EMI Fingers 2 Screw on the bracket using the screws and washers saved from the bracket removal procedure above Figure 5 Placing the Bracket on the Card MCX312B XCCT Do not force the bracket onto the...

Page 18: ...he adapter faceplate is visible against the system chassis 5 Secure the adapter with the adapter clip or screw 6 Close the system case 3 6 Cables and Modules To obtain the list of supported cables for...

Page 19: ...gged into the port with the other end of the connector plugged into a functioning port See Section 7 4 Adapter Led Operations on page 50 4 After plugging in a cable lock the connector using the latchi...

Page 20: ...Hardware Ids Windows 2008 R2 from the Properties pull down menu Note Figure 8 is an example using ConnectX 3 device This figure will be updated in future releases Figure 8 PCI Device 1 In the Value di...

Page 21: ...anual Rev 1 5 Mellanox Technologies 21 3 7 2 On Linux Get the device location on the PCI bus by running lspci and locating lines with the string Mella nox Technologies lspci grep i Mellanox 27 00 0 Ne...

Page 22: ...om Products Software Ethernet Driver Linux Driver Download Step 2 Install Driver The below sections are examples using ConnectX 3 device and will be updated to include ConnectX 3 Pro in future release...

Page 23: ...e the sources are rebuilt with the running kernel This mode is used for vanilla kernels Note If the Vanilla kernel is installed as rpm please use the disable kmp flag when installing the driver The ke...

Page 24: ...ailable from the download page Hardware and Software Requirements 4 2 1 Downloading MLNX_WinOF Follow these steps to download the exe according to your Operating System Step 1 Verify the machine archi...

Page 25: ...out Running Installation To extract the files without running installation perform the following steps Step 1 Open a CMD console Click Start Run and enter CMD Step 2 Enter the following command Step 3...

Page 26: ...SFP Port Adapter Card User Manual Rev 1 5 Mellanox Technologies 26 Step 4 Click Change and specify the location in which the files are extracted to Step 5 Click Install to extract this folder or click...

Page 27: ...ion 4 2 3 1 Attended Installation The following is an example of a MLNX_WinOF_win8 x64 installation session Step 1 Double click the exe and follow the GUI instructions to install MLNX_WinOF To configu...

Page 28: ...ConnectX 3 Pro Ethernet Single and Dual SFP Port Adapter Card User Manual Rev 1 5 Mellanox Technologies 28 Click Next in the Welcome screen Step 2 Read then accept the license agreement and click Next...

Page 29: ...e screen will be displayed in the following cases If the user has an OEM card in this case the firmware will not be updated If the user has a standard Mellanox card and the firmware version is older t...

Page 30: ...Rev 1 5 Mellanox Technologies 30 Step 5 Configure your system for maximum performance by checking the maximum performance box Step 6 Select a Complete or Custom installation follow Step a on page 31...

Page 31: ...utomatically Performances tools install the performance tools that are used to measure the InfiniBand performance in user environment Analyze tools install the tools that can be used either to diagnos...

Page 32: ...ConnectX 3 Pro Ethernet Single and Dual SFP Port Adapter Card User Manual Rev 1 5 Mellanox Technologies 32 Click Install to start the installation Step 7 Click Finish to complete the installation...

Page 33: ...e version run the following command For further help please run 4 2 4 Upgrading MLNX_WinOF The MLNX_WinOF driver upgrades automatically MLNX_WinOF Windows 2008R2 driver by uninstalling the previous ve...

Page 34: ...environments and must be activated in one of the machines running on the subnet otherwise the interface link will not come up If the cards are connected to a managed network there is no need to run Op...

Page 35: ...LNX_WinOF 4 2 7 2 Unattended Uninstall To uninstall MLNX_WinOF in unattended mode perform the following Step 1 Open a CMD console Step 2 Uninstall the driver Run 4 2 8 Assigning Port IP After Installa...

Page 36: ...SFP Port Adapter Card User Manual Rev 1 5 Mellanox Technologies 36 Step 2 Right click a Mellanox Local Area Connection and left click Properties Step 3 Select Internet Protocol Version 4 TCP IPv4 fro...

Page 37: ...ify the IP configuration by running ipconfig from a CMD console 4 2 9 Port Type Management on Windows After installing Mellanox WinOF VPI for Windows on your machine you can change a port s pro tocol...

Page 38: ...nologies 38 Step 2 Right click on the Mellanox ConnectX VPI network adapter and left click Properties Select the Port Protocol tab from the Properties sheet Note The Port Protocol tab is displayed onl...

Page 39: ...ver For VMware download and install the latest Mellanox OFED Driver for VMware ESXi Server software package available via the Mellanox web site at http www mellanox com Products Software VMware Driver...

Page 40: ...ove the VIB offline_bundle driver package from the ESXi server machine run In Non Multifunction Mode port 2 is identified as a pseudo device Therefore devices are not seen by vSphere when added as upl...

Page 41: ...exBoot FlexBoot supports remote Boot over Ethernet This technology is based on the Preboot Execution Environment PXE standard specification and FlexBoot software is based on the open source iPXE proje...

Page 42: ...t http www mellanox com Products Software Firmware Tools Make sure to download the package corresponding to your computer s operating system b Enter mst start c Get the Mellanox mst device name using...

Page 43: ...date the firmware as follows a Download the firmware image zip file from the Support Downloader see Step 2a above b Unzip the firmware image c Burn the firmware image Enter a Reboot the computer b Ent...

Page 44: ...ing another cable Reinstall the drivers for the network driver files may be damaged or deleted Reboot the server Adapters stopped working after installing another adapter Try removing and re installin...

Page 45: ...2netdev Card Detection lspci grep i Mellanox Mellanox Firmware Tool MFT Download and install MFT http www mellanox com con tent pages php pg management_tools menu_section 34 Refer to the User Manual f...

Page 46: ...pages php pg management_tools menu_section 34 Refer to the User Manual for installation instructions Once installed open a CMD window and run cd C Program Files Mellanox WinMFT mst start mst status f...

Page 47: ...and Environmental Voltage 12V 3 3V Typ Power Passive Cables 5 92W Max Power Passive Cables 6 42W Temperature Operational 0 C to 55 C Non operational 20 C to 70 C Humidity 90 relative humiditya a For...

Page 48: ...mental Voltage 12V 3 3V Typ Power Passive Cables 6 0W Max Power Passive Cables 6 73W Temperature Operational 0 C to 55 C Non operational 20 C to 70 C Humidity 90 relative humiditya a For both operatio...

Page 49: ...mental Voltage 12V 3 3V Typ Power Passive Cables 5 02W Max Power Passive Cables 6 17W Temperature Operational 0 C to 55 C Non operational 20 C to 70 C Humidity 90 relative humiditya a For both operati...

Page 50: ...dapter Card Table 10 Physical and Logical Link Indication Ethernet Mode LED Function Green physical link Constant on indicates a good physical link If neither LED is lit then the physical link has not...

Page 51: ...Adapter Card User Manual Rev 1 5 Mellanox Technologies 51 Figure 11 Mechanical Drawing of the Dual port MCX312C XCCT Adapter Card Figure 12 Mechanical Drawing of the Single port MCX311A XCCT Adapter C...

Page 52: ...Rev 1 5 Mellanox Technologies 52 7 6 Bracket Mechanical Drawing Figure 13 Dual port Bracket 7 7 Regulatory Statements For regulatory statements for all ConnectX 3 Pro cards please refer to http www m...

Page 53: ...Pro Ethernet Single and Dual SFP Port Adapter Card User Manual Rev 1 5 Mellanox Technologies 53 Appendix A Interface Connectors Pinout A 1 SFP Connector Pinout Figure 14 Rear View of Module With Pin...

Page 54: ...Data Line d d Should be pulled up with 4 7k 10k on host board to a voltage between 2 0V and 3 6V MOD_ABS pulls line low to indicate module is plugged in 5 SCL 2 wire Serial Interface Clock Line d 6 M...

Page 55: ...User Manual Rev 1 5 Mellanox Technologies 55 A 2 PCI Express x8 Connector Pinout The adapter cards use a standard PCI Express x8 edge connector and the PCI Express x8 standard pinout according to the...

Page 56: ...le and Dual SFP Port Adapter Card User Manual Rev 1 5 Mellanox Technologies 56 A 3 I2 C compatible Connector Pinout Figure 16 Compatible Connector Plug and Pinout Connector Pin Number Signal Name 1 SP...

Page 57: ...et adapter cards there is a mismatch between the GUID value returned by firmware management tools and that returned by fabric driver utilities that read the GUID via device firmware e g using ibstat M...

Page 58: ...pability mask 0x00010000 Port GUID 0x0000000000000000 Link layer Ethernet Port 2 State Down Physical state Disabled Rate 10 Base lid 0 LMC 0 SM lid 0 Capability mask 0x00010000 Port GUID 0x00000000000...

Page 59: ...oper air flow allow at least 8cm 3 inches of clearance around the ven tilation openings During periods of lightning activity do not work on the equipment or connect or dis connect cables Some copper c...

Page 60: ...Single and Dual QSFP Port Adapter Card User Manual Rev 1 7 Mellanox Technologies 60 CLASS 1 LASER PRODUCT and reference to the most recent laser standards IEC 60 825 1 1993 A1 1997 A2 2001 and EN 608...

Page 61: ...de 200LFM cette temp rature ambiante maximale est n cessaire En outre pour garantir un bon coulement de l air laissez au moins 8 cm 3 pouces d espace libre autour des ouver tures de ventilation Penda...

Page 62: ...grave Mise en garde l utilisation de commandes ou de r glages ou l ex cution de proc dures autres que ce qui est sp cifi dans les pr sentes peut engendrer une exposition au rayonnement grave PRODUIT...

Page 63: ...atur erforderlich Au erdem sollten mindestens 8 cm 3 in Freiraum um die Bel ftungs ffnungen sein um einen einwandfreien Luftstrom zu gew hrleisten Arbeiten Sie w hrend eines Gewitters und Blitzschlag...

Page 64: ...ahlenkontak Achtung Nutzung von Steuerungen oder Einstellungen oder Ausf hrung von Prozeduren die hier nicht spezifiziert sind kann zu gef hrlichem Strahlenkon takt f hren Klasse 1 Laserprodukt und Re...

Page 65: ...adecuada se debe dejar como m nimo un espacio de 8 cm 3 pulgadas alrededor de las aberturas de ventilaci n No utilizar el equipo ni conectar o desconectar cables durante per odos de actividad de rayo...

Page 66: ...Pro Ethernet Single and Dual QSFP Port Adapter Card User Manual Rev 1 7 Mellanox Technologies 66 PRODUCTO L SER DE CLASE 1 y referencia a las normas de l ser m s recientes IEC 60825 1 2007 03 y EN 60...

Search results

Summary of Contents for ConnectX-3 Pro

Reviews:

Related manuals for ConnectX-3 Pro

Brands by name

Popular brands

Mellanox Technologies ConnectX-3 Pro, User Manual

Search results

Summary of Contents for ConnectX-3 Pro

Reviews:

Related manuals for ConnectX-3 Pro

Brands by name

Popular brands