background image

 

33 

Mellanox Technologies Confidential 

Rev 1.1

 

 

4

 

Driver Installation 

Please use the relevant driver installation section. 

 

Linux Driver Installation 

 

Windows Driver Installation 

4.1

 

Linux Driver Installation 

This section describes how to install and test the Mellanox OFED for Linux package on a 
single server with a Mellanox ConnectX-6 adapter card installed. 

4.1.1

 

Pr

erequisit

es 

Requirements 

Description 

Platforms 

A server platform with a ConnectX-6 InfiniBand/VPI adapter card 
installed. 

Required Disk Space for 
Installation 

1GB 

Device ID 

For the latest list of device IDs, please visit the Mellanox website 
at 

http://www.mellanox.com/page/firmware_HCA_FW_identification

. 

Operating System 

Linux operating system. 
For the list of supported operating system distributions and kernels, 
please refer to the 

Mellanox OFED Release Notes

 file. 

Installer Privileges 

The installation requires administrator (root) privileges on the target 
machine. 

4.1.2

 

Downloading Mellanox OFED 

 

Verify that the system has a Mellanox network adapter installed. 
The following example shows a system with an installed Mellanox adapter card: 

lspci -v |grep Mellanox 
af:00.0 Infiniband controller: Mellanox Technologies MT28908 Family [ConnectX-6] 
Subsystem: Mellanox Technologies Device 0018 

 

Download the ISO image to your host. 

The image’s name has the format 

MLNX_OFED_LINUX-<ver>-<OS label><CPU 

arch>.iso. 
You can download and install the latest OpenFabrics Enterprise Distribution (OFED) 
software package available via the Mellanox web site 
at 

http://www.mellanox.com

 > Products > Software > Ethernet Drivers > Linux 

SW/Drivers > Download.. 

a.

 

Scroll down to the Download wizard, and click the Download tab. 

b.

 

Choose your relevant package depending on your host operating system. 

c.

 

Click the desired ISO/tgz package. 

d.

 

To obtain the download link, accept the End User License Agreement 
(EULA). 

Содержание 1GK7G

Страница 1: ...www mellanox com Mellanox Technologies Confidential Mellanox ConnectX 6 VPI Adapter Cards User Manual for Dell EMC PowerEdge Servers OPNs Y1T43 7TKND 1GK7G CY7GD Rev 1 1...

Страница 2: ...DING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QU...

Страница 3: ...Heat Sink Interface 14 2 6 SMBus Interface 15 2 7 Voltage Regulators 15 3 Hardware Installation 16 3 1 Safety Warnings 16 3 2 Installation Procedure Overview 17 3 3 System Requirements 17 3 3 1 Hardw...

Страница 4: ...lanox WinOF 2 Driver 47 4 2 4 Uninstalling Mellanox WinOF 2 Driver 52 4 2 5 Extracting Files Without Running Installation 52 4 2 6 Firmware Upgrade 54 5 Updating Adapter Firmware 55 6 Troubleshooting...

Страница 5: ...2 Renamed Single slot Configuration to SNAPI Single slot Configuration Updated ConnectX 6 SNAPI Single slot PCIe 2 x8 in a Row Renamed section title to Installation Instructions for HDR100 Card Aug 2...

Страница 6: ...100 QSFP Adapter Tall Bracket 7TKND Mellanox ConnectX 6 Single Port VPI HDR100 QSFP Adapter Short Bracket 1GK7G Mellanox ConnectX 6 Single Port VPI HDR QSFP Adapter Tall Bracket CY7GD Mellanox Connect...

Страница 7: ...used to update the firmware See http www mellanox com Products Software Firmware Tools mlxup Firmware Utility Mellanox Firmware Tools MFT User Manual User Manual describing the set of MFT firmware ma...

Страница 8: ...s ConnectX 6 is a groundbreaking addition to the Mellanox ConnectX series of industry leading adapter cards In addition to all the existing innovative features of past ConnectX versions ConnectX 6 off...

Страница 9: ...low figure Part Number 1GK7G CY7GD Form Factor Dimensions Adapter Card PCIe Half Height Half Length 167 65mm x 68 90mm Auxiliary PCIe Connection Card 5 09 in x 2 32 in 129 30mm x 59 00mm Harnesses two...

Страница 10: ...tilization is improved as each CPU handles only its own traffic and not traffic from the other CPU A system with a custom PCI Express x16 slot that includes special signals is required for installing...

Страница 11: ...standard InfiniBand data rate where each lane of a 2X port runs a bit rate of 53 125Gb s with a 64b 66b encoding resulting in an effective bandwidth of 100Gb s Memory Components SPI Quad includes 256...

Страница 12: ...offloads for target machine Erasure Coding T10 DIF Signature Handover SR IOV ConnectX 6 SR IOV technology provides dedicated adapter resources and guaranteed isolation and protection for virtual mach...

Страница 13: ...ernet standards listed in Features and Benefits Ethernet traffic is transmitted through the QSFP56 connector on the adapter card 2 3 PCI Express Interface Initial release of ConnectX 6 cards is Infini...

Страница 14: ...Status Indications Physical and Logical InfiniBand Protocol LED Color and State Description Off A physical link has not been established Solid Amber Indicates an active physical link Blinking Amber a...

Страница 15: ...er PCIe protocols as if it is a standard Mellanox PCIe stand up adapter For configuring the adapter for the specific manageability solution in use by the server please contact Mellanox Support 2 7 Vol...

Страница 16: ...m recommended 55 C 131 F An air flow of 200LFM at this maximum ambient temperature is required for HCA cards and NICs To guarantee proper airflow allow at least 8cm 3 inches of clearance around the ve...

Страница 17: ...ements Unless otherwise specified Mellanox products are designed to work in an environmentally controlled data center with low levels of gaseous and dust particulate contamination The operation enviro...

Страница 18: ...d in a system that operates with voltages that can be lethal Before opening the case of the system observe the following precautions to avoid injury and prevent damage to system components Remove any...

Страница 19: ...PCIe auxiliary card and server OPNs Server PCIe Auxiliary Card and cable Installation Instructions IGK7G R740 FTR4J Mellanox PCIe Auxiliary card and cable Error Reference source not found CY7GD C6400...

Страница 20: ...labeled for easier installation Cables labeled Main White and Main Black should be connected to the adapter card 1 Open the plastic connector lock and insert both main and secondary cable through the...

Страница 21: ...en in the below figure 4 Repeat steps 1 3 to connect the Main Black cable to the Black Cable connector on the card 5 Slide the connector lock until the BOTTOM side is under card and tabs are through c...

Страница 22: ...e and lock the blue card latch 3 Connect the cables and connector lock to the PCIe Auxiliary card as specified in the above steps 4 Install the PCIe Auxiliary card into the riser per riser installatio...

Страница 23: ...labeled for easier installation Cables labeled Main White and Main Black should be connected to the adapter card 2 Open the plastic connector lock and insert both main and secondary cable through the...

Страница 24: ...en in the below figure 5 Repeat steps 1 3 to connect the Main Black cable to the Black Cable connector on the card 6 Slide the connector lock until the BOTTOM side is under card and tabs are through c...

Страница 25: ...4682 to secure the card to the bracket 9 Align and press down to seat the riser Use a 4 in lb torque tool to tighten 4 screws PN 267JK to secure the riser 10 Insert and press down to seat the PCIe Aux...

Страница 26: ...dapter card with the Auxiliary connection card using the supplied Cabline CA II Plus harnesses The cable and card connectors are labeled for easier installation Cables labeled Main White and Main Blac...

Страница 27: ...open the connector cover on the Main Cable connector White cable connector as seen in the below figure 4 Align and insert the cable connector into the White Cable connector on the card Close the conn...

Страница 28: ...nd tabs are through card slots Press the top section of the connector lock down and lock in place 7 Physically touch and verify that both connector lock tabs are completely seated 8 Install the Mellan...

Страница 29: ...nds of cables to the Mellanox Auxiliary Card 10 Install the Mellanox Auxiliary card into the riser per slot Matrix and card installation instructions 11 With cables routed under the riser install the...

Страница 30: ...p 1 Locate an available PCI Express x16 slot and insert the adapter card to the chassis Step 2 Applying even pressure at both corners of the card insert the adapter card in a PCI Express slot until fi...

Страница 31: ...make sure that the latches engage WARNING Always install and remove cables by pushing or pulling the cable and connector in a straight line with the card After inserting a cable into a port the Green...

Страница 32: ...b and select Hardware Ids Windows 2016 from the Property pull down menu PCI Device Example In the Value display box check the fields VEN and DEV fields are separated by In the display example above no...

Страница 33: ...Release Notes file Installer Privileges The installation requires administrator root privileges on the target machine 4 1 2 Downloading Mellanox OFED Verify that the system has a Mellanox network adap...

Страница 34: ...ou wish to perform a firmware upgrade using customized FW binaries you can provide a path to the folder that contains the FW binary files by running fw image dir Using this option the FW version embed...

Страница 35: ...t to continue y N y See log file tmp mlnx_ofed_iso 21642 log Building OFED RPMs Please wait Removing OFED RPMs Created tmp MLNX_OFED_LINUX x x x rhel6 3 x86_64 ext tgz The script adds the following li...

Страница 36: ...nt mlnxofedinstall without dkms add kernel support kernel kernel version in chroot without fw update force For example mlnxofedinstall without dkms add kernel support kernel 3 13 0 85 generic without...

Страница 37: ...at is without the option without fw update o The firmware version of the adapter device is older than the firmware version included with the Mellanox OFED ISO image Note If an adapter s flash was orig...

Страница 38: ...ists from step 1 and will prevent the kernel from loading the modules automatically 4 1 3 7 mlnxofedinstall Return Codes The table below lists the mlnxofedinstall script return codes and their meaning...

Страница 39: ...lanox Importing GPG key 0x6224C050 Userid Mellanox Technologies Mellanox Technologies Signing Key v2 support mellanox com From repos MLNX_OFED MLNX_OFED file RPM GPG KEY Mellanox Is this ok y N Check...

Страница 40: ...fc21 x86_64 noarch MLNX_OFED all installer package for kernel 3 17 4 301 fc21 x86_64 without KMP support mlnx ofed basic 3 17 4 301 fc21 x86_64 noarch MLNX_OFED basic installer package for kernel 3 17...

Страница 41: ...Create an apt get repository configuration file called etc apt sources list d mlnx_ofed list with the following content deb file path to extracted MLNX_OFED package DEBS Download and install Mellanox...

Страница 42: ...script usr sbin ofed_uninstall sh to uninstall the Mellanox OFED package The script is part of the ofed scripts package 4 1 6 Updating Firmware After Installation The firmware can be updated either m...

Страница 43: ...ollowing message is printed to the system s standard logging file fw_updater Firmware was updated Please reboot your system for the changes to take effect Otherwise the following message is printed fw...

Страница 44: ...Signature from kernel Modules The signature can be removed from a signed kernel module using the strip utility which is provided by the binutils package The strip utility will change the given file wi...

Страница 45: ...plication of the user s system it may be necessary to modify the default configuration of network adapters based on the ConnectX adapters In case that tuning is required please refer to the Performanc...

Страница 46: ...To download the exe file according to your Operating System please follow the steps below Obtain the machine architecture a To go to the Start menu position your mouse in the bottom right corner of t...

Страница 47: ...edure that requires no user intervention Both Attended and Unattended installations require administrator privileges 4 2 3 1 Attended Installation The following is an example of an installation sessio...

Страница 48: ...be displayed in the following cases If the user has an OEM card In this case the firmware will not be displayed If the user has a standard Mellanox card with an older firmware version the firmware wil...

Страница 49: ...erformances tools install the performance tools that are used to measure performance in user environment Documentation contains the User Manual and Release Notes Management tools installation tools us...

Страница 50: ...hnologies Confidential 50 Click Install to start the installation In case firmware upgrade option was checked in Step 7 you will be notified if a firmware upgrade is required see Click Finish to compl...

Страница 51: ...to contain the logs option _All_Arch exe S v qn v l vx Log File v shapes _x0000_s1026 Optional if you wish to control whether to install ND provider or not i e MT_NDPROPERTY default value is True MLN...

Страница 52: ...d mode Open a CMD console Click Task Manager File Run new task and enter CMD To uninstall the driver run MLNX_WinOF2 2_0_All_x64 exe S x v qn 4 2 5 Extracting Files Without Running Installation To ext...

Страница 53: ...Mellanox Technologies Confidential Rev 1 1 Click Change and specify the location in which the files are extracted to Click Install to extract this folder or click Change to install to a different fold...

Страница 54: ...f the machine has a standard Mellanox card with an older firmware version the firmware will be automatically updated as part of the WinOF 2 package installation For information on how to upgrade firmw...

Страница 55: ...entation are available from http www mellanox com Products Software Firmware Tools Firmware Update Example server1 mlxup Querying Mellanox devices firmware Device Type ConnectX 6 Part Number Y1T43 Des...

Страница 56: ...g another cable Reinstall the drivers for the network driver files may be damaged or deleted Reboot the server Adapters stopped working after installing another adapter Try removing and re installing...

Страница 57: ...Firmware Tool MFT Download and install MFT http www mellanox com content pages php pg management _tools menu_section 34 Refer to the User Manual for installation instructions Once installed run mst s...

Страница 58: ...x com content pages php pg management_t ools menu_section 34 Refer to the User Manual for installation instructions Once installed open a CMD window and run WinMFT mst start mst status flint d mst_dev...

Страница 59: ...SR Data Rate InfiniBand SDR DDR QDR FDR EDR HDR100 Ethernet 1 10 25 40 50 100 Gb s Gen3 0 4 0 SERDES 8 0GT s 16GT s x16 lanes SNAPI Single slot PCIe 2x8 in a row 2 0 and 1 1 compatible Adapter Card Po...

Страница 60: ...KR SGMII 1000BASE CX 1000BASE KX 10GBASE SR Data Rate InfiniBand SDR DDR QDR FDR EDR HDR100 HDR Ethernet 1 10 25 40 50 100 200 Gb s ConnectX 6 Card Gen 3 0 4 0 SERDES 8 0GT s 16 GT s Auxiliary PCIe Co...

Страница 61: ...al Drawings and Dimensions All dimensions are in millimeters The PCB mechanical tolerance is 0 13mm For the 3D Model of the card please refer to http www mellanox com page 3d_models 7 3 1 ConnectX 6 A...

Страница 62: ...nox Technologies Confidential 62 7 3 3 ConnectX 6 Brackets Mechanical Drawing and Dimensions All dimensions are in millimeters The bracket mechanical tolerance is 0 25mm 7 3 3 1 Tall Bracket 7 3 3 2 S...

Страница 63: ...ly its own traffic and not traffic from the other CPU In order to allow this capability a system with a special PCI Express x16 slot is required The below table provides the pin definitions of the req...

Страница 64: ...has a different identifier printed on the label serial number and the card MAC for the Ethernet protocol The product revisions indicated on the labels in the following figures do not necessarily repre...

Отзывы: