manualshive.com logo in svg

Nvidia Mellanox, Руководство пользователя, Страница: 1 / 64

Поделиться
Скачать
Nvidia Mellanox Скачать руководство пользователя страница 1

Exported on Oct/23/2020 12:05 PM

https://docs.mellanox.com/x/TwvTAQ

NVIDIA Mellanox ConnectX-6 InfiniBand/VPI 

Adapter Cards for OCP Spec 3.0 User Manual

Содержание Mellanox

Страница 1: ...Exported on Oct 23 2020 12 05 PM https docs mellanox com x TwvTAQ NVIDIA Mellanox ConnectX 6 InfiniBand VPI Adapter Cards for OCP Spec 3 0 User Manual...

Страница 2: ...allation 15 Safety Warnings 15 Installation Procedure Overview 15 System Requirements 15 Hardware Requirements 16 Airflow Requirements 16 Software Requirements 16 Safety Precautions 17 Pre Installatio...

Страница 3: ...ation Results 31 Extracting Files Without Running Installation 31 Uninstalling Mellanox WinOF 2 Driver 33 Attended Uninstallation 33 Unattended Uninstallation 33 Firmware Upgrade 33 Deploying the Driv...

Страница 4: ...m kernel Modules 46 Performance Tuning 47 VMware Driver Installation 47 Hardware and Software Requirements 47 Installing Mellanox NATIVE ESXi Driver for VMware vSphere 48 Removing Earlier Mellanox Dri...

Страница 5: ...CX653435A HDAI Board Labels Example TBD 61 MCX653436A HDAI Board Labels Example TBD 61 MCX653435A EDAI Board Labels Example TBD 61 MCX653435A HDAE Board Labels Example TBD 61 Document Revision History...

Страница 6: ...his manual is intended for the installer and user of these cards The manual assumes basic familiarity with InfiniBand VPI network and architecture specifications Technical Support Customers who purcha...

Страница 7: ...ables and transceivers are designed to maximize the performance of High Performance Computing networks requiring high bandwidth low latency connections between compute nodes and switch nodes Mellanox...

Страница 8: ...dst of retesting these cards to comply with OCP spec 3 0 Products Overview Model ConnectX 6 VPI Cards for OCP Spec 3 0 Part Number MCX653435A HDAI MCX653436A HDAI MCX653435A HDAE a MCX653435A EDAI Hos...

Страница 9: ...ncoding resulting in an effective bandwidth of 100Gb s Up to 200 Gigabit Ethernet Mellanox adapters comply with the following IEEE 802 3 standards 200GbE 100GbE 50GbE 40GbE 25GbE 10GbE 1GbE IEEE 802 3...

Страница 10: ...nds of nodes CPU Offload Adapter functionality enabling reduced CPU overhead allowing more available CPU for computation tasks Open vSwitch OVS offload using ASAP2 TM Flexible match action flow tables...

Страница 11: ...VMware OpenFabrics Enterprise Distribution OFED OpenFabrics Windows Distribution WinOF 2 Connectivity Interoperable with 1 10 25 40 50 100 200 Gb s Ethernet switches Passive copper cable with ESD prot...

Страница 12: ...ce The table below describes the supported PCIe interface in ConnectX 6 adapter cards Supported PCIe Interface Features PCIe Gen 3 0 4 0 1 1 and 3 0 compatible through x16 edge connectors Link Rates 2...

Страница 13: ..._ID0 and SLOT_ID1 and its capacity is 4Kb Heat Sink Interface A heatsink is attached to the ConnectX 6 IC in order to dissipate the heat from the ConnectX 6 IC It is attached either by using four spri...

Страница 14: ...14 CPLD Interface The adapter card incorporates a CPLD device that controls the networking port LEDs and the scan chain It draws its power supply from 3 3V_EDGE...

Страница 15: ...Pre Installation Checklist 5 Optional Replace the assembled OCP 3 0 bracket with the desired form factor bracket Refer to OCP 3 0 Bracket Replacement Instructions 6 Install ConnectX 6 adapter card for...

Страница 16: ...rection Cold Aisle Cooling Port to heatsink Airflow Direction Software Requirements See Operating Systems Distributions section under the Introduction section Software Stacks Mellanox OpenFabric softw...

Страница 17: ...ation Instructions This section provides detailed instructions on how to install your adapter card in a system The below table lists the different ConnectX 6 OCP 3 0 retention mechanisms and provides...

Страница 18: ...re installing the card make sure that the system is off and the power cord is not connected to the server Please follow proper electrical grounding procedures Open the system case Align the card with...

Страница 19: ...er Please follow proper electrical grounding procedures Open the system case Align the card with the system rails while making sure the ejector latch is open Push the card until connectors are in full...

Страница 20: ...p Cable Installation All cables can be inserted or removed with the unit powered on To insert a cable press the connector into the port receptacle until the connector is firmly seated Support the weig...

Страница 21: ...he rack To remove a cable disengage the locks and slowly pull the connector away from the port receptacle LED indicator will turn off when the cable is unseated Identifying the Card in Your System On...

Страница 22: ...nstalled in a system that operates with voltages that can be lethal Before uninstalling the adapter card please observe the following precautions to avoid injury and prevent damage to system component...

Страница 23: ...t the adapter card Cards with Internal Lock Bracket Extraction Instructions Verify that the system is powered off and unplugged Wait 30 seconds Gently extract the adapter card from the server Cards wi...

Страница 24: ...24 1 2 3 4 Verify that the system is powered off and unplugged Wait 30 seconds Open the ejector latch Gently extract the adapter card from the server...

Страница 25: ...d above must run with administrator privileges Downloading Mellanox WinOF 2 Driver To download the executable file according to your operating system please follow the steps below Obtain the machine a...

Страница 26: ...you do not want to upgrade your firmware version Note MT_SKIPFWUPGRD default value is False MLNX_WinOF2 1_10_ revision_version _All_x64 exe v MT_SKIPFWUPGRD 1 Installing the incorrect exe file is pro...

Страница 27: ...the installation The firmware upgrade screen will be displayed in the following cases If the user has an OEM card In this case the firmware will not be displayed If the user has a standard Mellanox ca...

Страница 28: ...ion follow Step a and on Select the desired feature to install Performance tools install the performance tools that are used to measure performance in user environment Documentation contains the User...

Страница 29: ...ools used for diagnostics such as mlx5cmd Click Next to install the desired tools Click Install to start the installation In case that firmware upgrade option was checked in Step 7 you will be notifie...

Страница 30: ...ant to control whether to install ND provider or not Note MT_NDPROPERTY default value is True MLNX_WinOF2 1_10_All_x64 exe vMT_NDPROPERTY 1 Optional If you do not wish to upgrade your firmware version...

Страница 31: ...ers OS To see the Mellanox network adapters display the Device Manager and pull down the Network adapters menu Extracting Files Without Running Installation To extract the files without running instal...

Страница 32: ...3 4 5 Click Next to create a server image Click Change and specify the location in which the files are extracted to Click Install to extract this folder or click Change to install to a different fold...

Страница 33: ...CMD console Click Task Manager File Run new task and enter CMD To uninstall the driver run MLNX_WinOF2 1_10_All_x64 exe S x v qn Firmware Upgrade If the machine has a standard Mellanox card with an ol...

Страница 34: ...tPath InjectingDrivers vhdx DriversPath C WS2016TP5_Drivers Create the Nano server image Follow the instructions in the TechNet article for Getting Started with Nano Server Online Update Go to the Mel...

Страница 35: ...The following example shows a system with an installed Mellanox adapter card lspci v grep Mellanox 86 00 0 Network controller 0207 Mellanox Technologies TBD Family Subsystem Mellanox Technologies Devi...

Страница 36: ...e the deletion of the old packages If you need to install Mellanox OFED on an entire homogeneous cluster a common strategy is to mount the ISO image on one of the cluster nodes and then copy it to a s...

Страница 37: ...stalled the OpenSM Subnet Manager on your machine For the list of installation options run mlnxofedinstall h Installation Procedure Login to the installation machine as root Mount the ISO image on you...

Страница 38: ...installation a message similar to the following Device 1 Device Type ConnectX 6 Part Number MCX653436A HDAI Description ConnectX 6 VPI adapter card 200Gb s HDR IB and 200GbE for OCP 3 0 with host mana...

Страница 39: ...tallation script is run in default mode that is without the option without fw update The firmware version of the adapter device is older than the firmware version included with the Mellanox OFED ISO i...

Страница 40: ...ocess of updating the initramfs will add the blacklists from step 1 and will prevent the kernel from loading the modules automatically mlnxofedinstall Return Codes The table below lists the mlnxofedin...

Страница 41: ...cted HTTP request sent awaiting response 200 OK Length 1354 1 3K text plain Saving to RPM GPG KEY Mellanox 100 1 354 K s in 0s 2014 04 20 13 52 30 247 MB s RPM GPG KEY Mellanox saved 1354 1354 Install...

Страница 42: ...port for RHEL and SLES For other operating systems kernel module RPM packages are provided only for the operating systems default kernel In this case the group RPM packages have the supported kernel v...

Страница 43: ...rating systems Setting up MLNX_OFED apt get Repository Log into the installation machine as root Extract the MLNX_OFED pacakge on a shared location in your network You can download it from http www me...

Страница 44: ...pi installer package Install the desired group apt get install group name Example apt get install mlnx ofed all Uninstalling MLNX_OFED Using the apt get Tool Use the script usr sbin ofed_uninstall sh...

Страница 45: ...Firmware Download Get the device s PSID mlxfwmanager_pci grep PSID PSID MT_1210110019 Download the firmware BIN file from the Mellanox website or the OEM website Burn the firmware mlxfwmanager_pci i...

Страница 46: ...ill launch MokManager efi to allow you to complete the enrollment from the UEFI console You will need to enter the password you previously associated with this request and confirm the enrollment Once...

Страница 47: ...ssage will appear Note that this message cannot be eliminated Update the initramfs on RHEL systems with the stripped modules mkinitrd boot initramfs uname r img uname r force Performance Tuning Depend...

Страница 48: ...7 01 31 Removing Earlier Mellanox Drivers To remove all the drivers Log into the ESXi server with root permissions List all the existing NATIVE ESXi driver modules See Step 4 in Installing Mellanox NA...

Страница 49: ...Firmware Tools MFT site ESXi 6 5 File mft 4 6 0 48 10EM 650 0 0 4598673 x86_64 vib MD5SUM 0804cffe30913a7b4017445a0f0adbe1 Install the image according to the steps described in the MFT User Manual Th...

Страница 50: ...server1 mlxup Querying Mellanox devices firmware Device Type ConnectX 6 Part Number MCX653436A HDAI Description ConnectX 6 VPI adapter card 200Gb s HDR IB and 200GbE for OCP 3 0 with host management...

Страница 51: ...ng another adapter Try removing and re installing all adapters Check that cables are connected properly Make sure your motherboard has the latest BIOS Link indicator light is off Try another port on t...

Страница 52: ...ows desktop choose the Start menu and run msinfo32 To export system information to a text file choose the Export option from the File menu Assign a file name and save Mellanox Firmware Tool MFT Downlo...

Страница 53: ...0BASE CX 1000BASE KX 10GBASE SR InfiniBand IBTA v1 3 and 1 4 Auto Negotiation a 1X 2X 4X SDR 2 5Gb s per lane DDR 5Gb s per lane QDR 10Gb s per lane FDR 14 0625Gb s per lane EDR 25Gb s per lane port H...

Страница 54: ...DAI Specifications Physical Size 2 99 in x 4 52 in 76 00mm x 115 00mm Connector Dual QSFP56 InfiniBand and Ethernet copper and optical Retention Mechanism Internal Lock Protocol Support Ethernet 200GB...

Страница 55: ...de Passive Cable 700LFM 100LFM Not Supported Not Supported Active 4 55W Cable 700LFM 45C 500LFM Not Supported Not Supported Regulator y Safety CB cTUVus CE EMC CE FCC VCCI ICES RCM RoHS RoHS compliant...

Страница 56: ...lane port HDR100 2 lane x 50Gb s per lane HDR 50Gb s per lane port Data Rate Ethernet 1 10 25 40 100 Gb s InfiniBand SDR DDR QDR FDR EDR HDR100 PCI Express Gen 3 0 4 0 SERDES 8 0GT s 16 0GT s 16 lanes...

Страница 57: ...BASE R4 40GBASE CR4 40GBASE KR4 40GBASE SR4 40GBASE LR4 40GBASE ER4 40GBASE R2 25GBASE R 20GBASE KR2 10GBASE LR 10GBASE ER 10GBASE CX4 10GBASE CR 10GBASE KR SGMII 1000BASE CX 1000BASE KX 10GBASE SR In...

Страница 58: ...ConnectX 6 adapters supplement the IBTA auto negotiation specification to get better bit error rates and longer cable reaches This supplemental feature only initiates when connected to another Mellano...

Страница 59: ...59 Brackets Mechanical Drawings and Dimensions Cards with Ejector Latch Bracket All dimensions are in millimeters The brackets mechanical tolerance is 0 25mm...

Страница 60: ...60 Cards with Internal Lock Bracket...

Страница 61: ...UID and a MAC derived from the GUID MCX653435A HDAI Board Labels Example TBD MCX653436A HDAI Board Labels Example TBD MCX653435A EDAI Board Labels Example TBD MCX653435A HDAE Board Labels Example TBD...

Страница 62: ...fications Updated networking ports LEDs indications in Interfaces Sep 2020 1 3 Added OCP 3 0 bracket replacement instructions Nov 2020 1 2 Updated Specifications with airflow numbers Oct 2019 1 1 Adde...

Страница 63: ...arily performed by NVIDIA It is customer s sole responsibility to evaluate and determine the applicability of any information contained in this document ensure the product is suitable and fit for the...

Страница 64: ...A the NVIDIA logo and Mellanox are trademarks and or registered trademarks of Mellanox Technologies Ltd and or NVIDIA Corporation in the U S and in other countries Other company and product names may...

Отзывы:

Нет отзывов