Mellanox Technologies 1GK7G User Manual Download

Page: 1 / 64

www.mellanox.com

Mellanox Technologies Confidential

Mellanox ConnectX®-6 VPI Adapter Cards User Manual for Dell

EMC PowerEdge Servers

OPNs: Y1T43, 7TKND, 1GK7G, CY7GD

Rev 1.1

Summary of Contents for 1GK7G

Page 1: ...www mellanox com Mellanox Technologies Confidential Mellanox ConnectX 6 VPI Adapter Cards User Manual for Dell EMC PowerEdge Servers OPNs Y1T43 7TKND 1GK7G CY7GD Rev 1 1...

Page 2: ...DING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS THE CUSTOMER S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QU...

Page 3: ...Heat Sink Interface 14 2 6 SMBus Interface 15 2 7 Voltage Regulators 15 3 Hardware Installation 16 3 1 Safety Warnings 16 3 2 Installation Procedure Overview 17 3 3 System Requirements 17 3 3 1 Hardw...

Page 4: ...lanox WinOF 2 Driver 47 4 2 4 Uninstalling Mellanox WinOF 2 Driver 52 4 2 5 Extracting Files Without Running Installation 52 4 2 6 Firmware Upgrade 54 5 Updating Adapter Firmware 55 6 Troubleshooting...

Page 5: ...2 Renamed Single slot Configuration to SNAPI Single slot Configuration Updated ConnectX 6 SNAPI Single slot PCIe 2 x8 in a Row Renamed section title to Installation Instructions for HDR100 Card Aug 2...

Page 6: ...100 QSFP Adapter Tall Bracket 7TKND Mellanox ConnectX 6 Single Port VPI HDR100 QSFP Adapter Short Bracket 1GK7G Mellanox ConnectX 6 Single Port VPI HDR QSFP Adapter Tall Bracket CY7GD Mellanox Connect...

Page 7: ...used to update the firmware See http www mellanox com Products Software Firmware Tools mlxup Firmware Utility Mellanox Firmware Tools MFT User Manual User Manual describing the set of MFT firmware ma...

Page 8: ...s ConnectX 6 is a groundbreaking addition to the Mellanox ConnectX series of industry leading adapter cards In addition to all the existing innovative features of past ConnectX versions ConnectX 6 off...

Page 9: ...low figure Part Number 1GK7G CY7GD Form Factor Dimensions Adapter Card PCIe Half Height Half Length 167 65mm x 68 90mm Auxiliary PCIe Connection Card 5 09 in x 2 32 in 129 30mm x 59 00mm Harnesses two...

Page 10: ...tilization is improved as each CPU handles only its own traffic and not traffic from the other CPU A system with a custom PCI Express x16 slot that includes special signals is required for installing...

Page 11: ...standard InfiniBand data rate where each lane of a 2X port runs a bit rate of 53 125Gb s with a 64b 66b encoding resulting in an effective bandwidth of 100Gb s Memory Components SPI Quad includes 256...

Page 12: ...offloads for target machine Erasure Coding T10 DIF Signature Handover SR IOV ConnectX 6 SR IOV technology provides dedicated adapter resources and guaranteed isolation and protection for virtual mach...

Page 13: ...ernet standards listed in Features and Benefits Ethernet traffic is transmitted through the QSFP56 connector on the adapter card 2 3 PCI Express Interface Initial release of ConnectX 6 cards is Infini...

Page 14: ...Status Indications Physical and Logical InfiniBand Protocol LED Color and State Description Off A physical link has not been established Solid Amber Indicates an active physical link Blinking Amber a...

Page 15: ...er PCIe protocols as if it is a standard Mellanox PCIe stand up adapter For configuring the adapter for the specific manageability solution in use by the server please contact Mellanox Support 2 7 Vol...

Page 16: ...m recommended 55 C 131 F An air flow of 200LFM at this maximum ambient temperature is required for HCA cards and NICs To guarantee proper airflow allow at least 8cm 3 inches of clearance around the ve...

Page 17: ...ements Unless otherwise specified Mellanox products are designed to work in an environmentally controlled data center with low levels of gaseous and dust particulate contamination The operation enviro...

Page 18: ...d in a system that operates with voltages that can be lethal Before opening the case of the system observe the following precautions to avoid injury and prevent damage to system components Remove any...

Page 19: ...PCIe auxiliary card and server OPNs Server PCIe Auxiliary Card and cable Installation Instructions IGK7G R740 FTR4J Mellanox PCIe Auxiliary card and cable Error Reference source not found CY7GD C6400...

Page 20: ...labeled for easier installation Cables labeled Main White and Main Black should be connected to the adapter card 1 Open the plastic connector lock and insert both main and secondary cable through the...

Page 21: ...en in the below figure 4 Repeat steps 1 3 to connect the Main Black cable to the Black Cable connector on the card 5 Slide the connector lock until the BOTTOM side is under card and tabs are through c...

Page 22: ...e and lock the blue card latch 3 Connect the cables and connector lock to the PCIe Auxiliary card as specified in the above steps 4 Install the PCIe Auxiliary card into the riser per riser installatio...

Page 23: ...labeled for easier installation Cables labeled Main White and Main Black should be connected to the adapter card 2 Open the plastic connector lock and insert both main and secondary cable through the...

Page 24: ...en in the below figure 5 Repeat steps 1 3 to connect the Main Black cable to the Black Cable connector on the card 6 Slide the connector lock until the BOTTOM side is under card and tabs are through c...

Page 25: ...4682 to secure the card to the bracket 9 Align and press down to seat the riser Use a 4 in lb torque tool to tighten 4 screws PN 267JK to secure the riser 10 Insert and press down to seat the PCIe Aux...

Page 26: ...dapter card with the Auxiliary connection card using the supplied Cabline CA II Plus harnesses The cable and card connectors are labeled for easier installation Cables labeled Main White and Main Blac...

Page 27: ...open the connector cover on the Main Cable connector White cable connector as seen in the below figure 4 Align and insert the cable connector into the White Cable connector on the card Close the conn...

Page 28: ...nd tabs are through card slots Press the top section of the connector lock down and lock in place 7 Physically touch and verify that both connector lock tabs are completely seated 8 Install the Mellan...

Page 29: ...nds of cables to the Mellanox Auxiliary Card 10 Install the Mellanox Auxiliary card into the riser per slot Matrix and card installation instructions 11 With cables routed under the riser install the...

Page 30: ...p 1 Locate an available PCI Express x16 slot and insert the adapter card to the chassis Step 2 Applying even pressure at both corners of the card insert the adapter card in a PCI Express slot until fi...

Page 31: ...make sure that the latches engage WARNING Always install and remove cables by pushing or pulling the cable and connector in a straight line with the card After inserting a cable into a port the Green...

Page 32: ...b and select Hardware Ids Windows 2016 from the Property pull down menu PCI Device Example In the Value display box check the fields VEN and DEV fields are separated by In the display example above no...

Page 33: ...Release Notes file Installer Privileges The installation requires administrator root privileges on the target machine 4 1 2 Downloading Mellanox OFED Verify that the system has a Mellanox network adap...

Page 34: ...ou wish to perform a firmware upgrade using customized FW binaries you can provide a path to the folder that contains the FW binary files by running fw image dir Using this option the FW version embed...

Page 35: ...t to continue y N y See log file tmp mlnx_ofed_iso 21642 log Building OFED RPMs Please wait Removing OFED RPMs Created tmp MLNX_OFED_LINUX x x x rhel6 3 x86_64 ext tgz The script adds the following li...

Page 36: ...nt mlnxofedinstall without dkms add kernel support kernel kernel version in chroot without fw update force For example mlnxofedinstall without dkms add kernel support kernel 3 13 0 85 generic without...

Page 37: ...at is without the option without fw update o The firmware version of the adapter device is older than the firmware version included with the Mellanox OFED ISO image Note If an adapter s flash was orig...

Page 38: ...ists from step 1 and will prevent the kernel from loading the modules automatically 4 1 3 7 mlnxofedinstall Return Codes The table below lists the mlnxofedinstall script return codes and their meaning...

Page 39: ...lanox Importing GPG key 0x6224C050 Userid Mellanox Technologies Mellanox Technologies Signing Key v2 support mellanox com From repos MLNX_OFED MLNX_OFED file RPM GPG KEY Mellanox Is this ok y N Check...

Page 40: ...fc21 x86_64 noarch MLNX_OFED all installer package for kernel 3 17 4 301 fc21 x86_64 without KMP support mlnx ofed basic 3 17 4 301 fc21 x86_64 noarch MLNX_OFED basic installer package for kernel 3 17...

Page 41: ...Create an apt get repository configuration file called etc apt sources list d mlnx_ofed list with the following content deb file path to extracted MLNX_OFED package DEBS Download and install Mellanox...

Page 42: ...script usr sbin ofed_uninstall sh to uninstall the Mellanox OFED package The script is part of the ofed scripts package 4 1 6 Updating Firmware After Installation The firmware can be updated either m...

Page 43: ...ollowing message is printed to the system s standard logging file fw_updater Firmware was updated Please reboot your system for the changes to take effect Otherwise the following message is printed fw...

Page 44: ...Signature from kernel Modules The signature can be removed from a signed kernel module using the strip utility which is provided by the binutils package The strip utility will change the given file wi...

Page 45: ...plication of the user s system it may be necessary to modify the default configuration of network adapters based on the ConnectX adapters In case that tuning is required please refer to the Performanc...

Page 46: ...To download the exe file according to your Operating System please follow the steps below Obtain the machine architecture a To go to the Start menu position your mouse in the bottom right corner of t...

Page 47: ...edure that requires no user intervention Both Attended and Unattended installations require administrator privileges 4 2 3 1 Attended Installation The following is an example of an installation sessio...

Page 48: ...be displayed in the following cases If the user has an OEM card In this case the firmware will not be displayed If the user has a standard Mellanox card with an older firmware version the firmware wil...

Page 49: ...erformances tools install the performance tools that are used to measure performance in user environment Documentation contains the User Manual and Release Notes Management tools installation tools us...

Page 50: ...hnologies Confidential 50 Click Install to start the installation In case firmware upgrade option was checked in Step 7 you will be notified if a firmware upgrade is required see Click Finish to compl...

Page 51: ...to contain the logs option _All_Arch exe S v qn v l vx Log File v shapes _x0000_s1026 Optional if you wish to control whether to install ND provider or not i e MT_NDPROPERTY default value is True MLN...

Page 52: ...d mode Open a CMD console Click Task Manager File Run new task and enter CMD To uninstall the driver run MLNX_WinOF2 2_0_All_x64 exe S x v qn 4 2 5 Extracting Files Without Running Installation To ext...

Page 53: ...Mellanox Technologies Confidential Rev 1 1 Click Change and specify the location in which the files are extracted to Click Install to extract this folder or click Change to install to a different fold...

Page 54: ...f the machine has a standard Mellanox card with an older firmware version the firmware will be automatically updated as part of the WinOF 2 package installation For information on how to upgrade firmw...

Page 55: ...entation are available from http www mellanox com Products Software Firmware Tools Firmware Update Example server1 mlxup Querying Mellanox devices firmware Device Type ConnectX 6 Part Number Y1T43 Des...

Page 56: ...g another cable Reinstall the drivers for the network driver files may be damaged or deleted Reboot the server Adapters stopped working after installing another adapter Try removing and re installing...

Page 57: ...Firmware Tool MFT Download and install MFT http www mellanox com content pages php pg management _tools menu_section 34 Refer to the User Manual for installation instructions Once installed run mst s...

Page 58: ...x com content pages php pg management_t ools menu_section 34 Refer to the User Manual for installation instructions Once installed open a CMD window and run WinMFT mst start mst status flint d mst_dev...

Page 59: ...SR Data Rate InfiniBand SDR DDR QDR FDR EDR HDR100 Ethernet 1 10 25 40 50 100 Gb s Gen3 0 4 0 SERDES 8 0GT s 16GT s x16 lanes SNAPI Single slot PCIe 2x8 in a row 2 0 and 1 1 compatible Adapter Card Po...

Page 60: ...KR SGMII 1000BASE CX 1000BASE KX 10GBASE SR Data Rate InfiniBand SDR DDR QDR FDR EDR HDR100 HDR Ethernet 1 10 25 40 50 100 200 Gb s ConnectX 6 Card Gen 3 0 4 0 SERDES 8 0GT s 16 GT s Auxiliary PCIe Co...

Page 61: ...al Drawings and Dimensions All dimensions are in millimeters The PCB mechanical tolerance is 0 13mm For the 3D Model of the card please refer to http www mellanox com page 3d_models 7 3 1 ConnectX 6 A...

Page 62: ...nox Technologies Confidential 62 7 3 3 ConnectX 6 Brackets Mechanical Drawing and Dimensions All dimensions are in millimeters The bracket mechanical tolerance is 0 25mm 7 3 3 1 Tall Bracket 7 3 3 2 S...

Page 63: ...ly its own traffic and not traffic from the other CPU In order to allow this capability a system with a special PCI Express x16 slot is required The below table provides the pin definitions of the req...

Page 64: ...has a different identifier printed on the label serial number and the card MAC for the Ethernet protocol The product revisions indicated on the labels in the following figures do not necessarily repre...

Search results

Summary of Contents for 1GK7G

Reviews:

Related manuals for 1GK7G

Brands by name

Popular brands

Mellanox Technologies 1GK7G, User Manual

Search results

Summary of Contents for 1GK7G

Reviews:

Related manuals for 1GK7G

Brands by name

Popular brands