Nvidia Mellanox ConnectX-5 Скачать руководство пользователя страница 1

Страница: 1 / 71

Exported on Sep/19/2022 09:36 AM

NVIDIA ConnectX-5 Ethernet Adapter Cards

for OCP Spec 3.0 User Manual

Содержание Mellanox ConnectX-5

Страница 1: ...Exported on Sep 19 2022 09 36 AM NVIDIA ConnectX 5 Ethernet Adapter Cards for OCP Spec 3 0 User Manual...

Страница 2: ...14 Hardware Installation 15 Safety Warnings 15 Installation Procedure Overview 16 System Requirements 17 Hardware Requirements 17 Airflow Requirements 17 Software Requirements 17 Safety Precautions 18...

Страница 3: ...tware Requirements 49 Installing NVIDIA NATIVE ESXi Driver for VMware vSphere 50 Removing Earlier NVIDIA Drivers 50 Firmware Programming 51 Updating Adapter Firmware 52 Troubleshooting 53 General Trou...

Страница 4: ...Lock bracket MCX565A CCAB ConnectX 5 EN network interface card for OCP 3 0 with host management 100GbE Single port QSFP28 PCIe3 0 x16 Thumbscrew Pull Tab bracket ConnectX 5 Ex MCX566M GDAI ConnectX 5...

Страница 5: ...Firmware Utility mlxup User Manual and Release Notes NVIDIA firmware update and query utility used to update the firmware See NVIDIA Firmware Utility mlxup Documentation NVIDIA Firmware Tools MFT Use...

Страница 6: ...ventions When discussing memory sizes MB and MBytes are used in this document to mean size in mega Bytes The use of Mb or Mbits small b indicates size in mega bits In this document PCIe is used to mea...

Страница 7: ...ar PCI connector may exceed the maximum insertion force that is allowed by the connector s spec NVIDIA is updating the OCP 3 0 cards to ensure the proper insertion process The following tables provide...

Страница 8: ...ock Bracket Pull tab Bracket Internal Lock Bracket Pull tab Bracket Host Management IC Part Number MT27808A0 FCCF AE MT27808A0 FCCF AE MT27808A0 FCCF CE MT27808A0 FCCF CE Device ID 4119 for Physical F...

Страница 9: ...from individual virtual machines over logical tunnels in encapsulated formats such as NVGRE and VXLAN While this solves network scalability issues it hides the TCP packet from the hardware offloading...

Страница 10: ...s optimizes board management and power performance and firmware update management via NC SI MCTP over SMBus and MCTP over PCIe as well as PLDM for Monitor and Control DSP0248 and PLDM for Firmware Upd...

Страница 11: ...EL CentOS Windows FreeBSD VMware OpenFabrics Enterprise Distribution OFED OpenFabrics Windows Distribution WinOF 2 Connectivity Interoperable with 1 10 25 40 50 100 Gb s Ethernet switches Passive copp...

Страница 12: ...s a bicolor Yellow and Green LED and LED1 is a single color Green LED Link Indications LED and State Description 1Hz blinking Yellow Beacon command for locating the adapter card 4Hz blinking Yellow In...

Страница 13: ...or x1 Support for MSI MSI X mechanisms ConnectX 5 Ex PCIe Gen 3 0 4 0 1 1 and 2 0 compatible through x16 edge connectors Link Rates 2 5 5 0 8 0 or 16 0GT s Auto Negotiation to x16 x8 x4 x2 or x1 Supp...

Страница 14: ...the server please contact NVIDIA Support Voltage Regulators The voltage regulator power is derived from the OCP 3 0 edge connector 12V and 3 3V supply pins These voltage supply pins feed onboard regul...

Страница 15: ...this maximum ambient temperature is required for HCA cards and NICs To guarantee proper airflow allow at least 8cm 3 inches of clearance around the ventilation openings During Lightning Electrical Haz...

Страница 16: ...software requirements Refer to System Requirements 2 Pay attention to the airflow consideration within the host system Refer to Airflow Requirements 3 Follow the safety precautions Refer to Safety Pre...

Страница 17: ...g Systems Distributions section under the Introduction section Unless otherwise specified NVIDIA products are designed to work in an environmentally controlled data center with low levels of gaseous a...

Страница 18: ...ssembled either with a thumbscrew pull tab bracket an internal lock bracket or an ejector latch bracket If this form factor is suitable for your requirements you can skip the remainder of this section...

Страница 19: ...ng the torx tool type listed in the below table remove the screws according to the instructions per OCP 3 0 bracket type Internal Lock Bracket Pull tab Thumbscrew Bracket Ejector Latch Bracket Gently...

Страница 20: ...d screws that came with the new bracket kit Use the specified torx tool type and apply the specified torque on the screws per bracket form factor Internal Lock Bracket Pull tab Bracket Ejector Latch B...

Страница 21: ...cord is not connected to the server Please follow proper electrical grounding procedures Open the system case Align the card with the system rails Internal Lock Bracket Pull tab Thumbscrew Bracket Eje...

Страница 22: ...tor will light when the physical connection is established that is when the unit is powered on and a cable is plugged into the port with the other end of the connector plugged into a functioning port...

Страница 23: ...s tab and select Hardware Ids Windows 2012 R2 2016 from the Property pull down menu PCI Device Example In the Value display box check the fields VEN and DEV fields are separated by In the display exam...

Страница 24: ...24 Cards Extraction Instructions Unable to render include or excerpt include Could not retrieve page...

Страница 25: ...012 Windows Server 2016 Windows Server 2019 Windows 8 1 Client 64 bit only Windows 10 Client 64 bit only Note The Operating System listed above must run with administrator privileges Downloading NVIDI...

Страница 26: ...Manually configure your setup to contain the logs option replace LogFile with the relevant directory MLNX_WinOF2 Driver Version _ revision_version _All_Arch exe v l vx LogFile Optional If you do not w...

Страница 27: ...27 4 5 6 Click Next in the Welcome screen Read and accept the license agreement and click Next Select the target folder for the installation...

Страница 28: ...he user has an OEM card In this case the firmware will not be displayed If the user has a standard NVIDIA card with an older firmware version the firmware will be updated accordingly However if the us...

Страница 29: ...Performances tools install the performance tools that are used to measure performance in user environment Documentation contains the User Manual and Release Notes Management tools installation tools u...

Страница 30: ...30 b Click Next to install the desired tools 9 Click Install to start the installation...

Страница 31: ...31 10 In case firmware upgrade option was checked in Step 7 you will be notified if a firmware upgrade is required see 11 Click Finish to complete the installation 1...

Страница 32: ...l_Arch exe vMT_NDPROPERTY 1 Optional If you do not wish to upgrade your firmware version i e MT_SKIPFWUPGRD default value is False MLNX_WinOF2 Driver Version _ revision_version _All_Arch exe vMT_SKIPF...

Страница 33: ...rt Control Panel Programs and Features MLNX_WinOF2 Uninstall NOTE This requires elevated administrator privileges Unattended Uninstallation To uninstall MLNX_WinOF2 in unattended mode Open a CMD conso...

Страница 34: ...onsole Click Start Task Manager File Run new task and enter CMD Extract the driver and the tools MLNX_WinOF2 2_0_ revision_version _All_x64 a To extract only the driver file MLNX_WinOF2 2_0_ revision_...

Страница 35: ...35 5 Click Install to extract this folder or click Change to install to a different folder...

Страница 36: ...e refer to the MFT User Manual at https docs nvidia com category mft Linux Driver Installation This section describes how to install and test the NVIDIA OFED for Linux package on a single server with...

Страница 37: ...MLNX_OFED_LINUX ver OS label CPU arch iso You can download and install the latest OpenFabrics Enterprise Distribution OFED software package available via https docs nvidia com category mlnxofedib Scr...

Страница 38: ...upport sh script can be executed directly from the mlnxofedinstall script For further information please see add kernel support option below Example The following command will create a MLNX_OFED_LINUX...

Страница 39: ...l other Mellanox OEM OFED RDMA or Distribution IB packages will be removed Those packages are removed due to conflicts with MLNX_OFED_LINUX do not reinstall them Starting MLNX_OFED_LINUX x x x install...

Страница 40: ...ll occur and the installation script will print at the end of installation a message similar to the following Device 1 Device Type ConnectX 5 Ex Part Number MCX546A CDAN Description ConnectX 5 Ex netw...

Страница 41: ...originally programmed with an Expansion ROM image the automatic firmware update will also burn an Expansion ROM image In case that your machine has an unsupported network adapter device no firmware u...

Страница 42: ...the blacklists from step 1 and will prevent the kernel from loading the modules automatically mlnxofedinstall Return Codes The table below lists the mlnxofedinstall script return codes and their mean...

Страница 43: ...TP request sent awaiting response 200 OK Length 1354 1 3K text plain Saving to RPM GPG KEY Mellanox 100 1 354 K s in 0s 2014 04 20 13 52 30 247 MB s RPM GPG KEY Mellanox saved 1354 1354 Install the ke...

Страница 44: ...with KMP support for RHEL and SLES For other operating systems kernel module RPM packages are provided only for the operating systems default kernel In this case the group RPM packages have the suppor...

Страница 45: ...installation is applicable to Debian and Ubuntu operating systems Setting up MLNX_OFED apt get Repository Log into the installation machine as root Extract the MLNX_OFED pacakge on a shared location i...

Страница 46: ...NX_OFED vma vpi installer package Install the desired group apt get install group name Example apt get install mlnx ofed all Uninstalling MLNX_OFED Using the apt get Tool Use the script usr sbin ofed_...

Страница 47: ...aded from the NVIDIA web site http www nvidia com Support Firmware Download Get the device s PSID mlxfwmanager_pci grep PSID PSID MT_1210110019 Download the firmware BIN file from the NVIDIA website o...

Страница 48: ...er Add the public key to the MOK list using the mokutil utility mokutil import mlnx_signing_key_pub der Reboot the system The pending MOK key enrollment request will be noticed by shim efi and it will...

Страница 49: ...ose key is not in the kernel key ring Therefore this message may go unnoticed Once the system is rebooted after unloading and reloading a kernel module the message will appear Note that this message c...

Страница 50: ...ore 4 16 8 8 1OEM 650 0 0 4240417 MEL PartnerSupported 2017 01 31 nmlx5 rdma 4 16 8 8 1OEM 650 0 0 4240417 MEL PartnerSupported 2017 01 31 Removing Earlier NVIDIA Drivers To remove all the drivers Log...

Страница 51: ...mft 4 6 0 48 10EM 650 0 0 4598673 x86_64 vib MD5SUM 0804cffe30913a7b4017445a0f0adbe1 Install the image according to the steps described in the MFT User Manual To remove the modules you must run the co...

Страница 52: ...pdate Example server1 mlxup Querying Mellanox devices firmware Device Type ConnectX 5 Part Number MCX562A ACAI Description ConnectX 5 EN network interface card for OCP 3 0 with host management 25GbE D...

Страница 53: ...ng another adapter Try removing and re installing all adapters Check that cables are connected properly Make sure your motherboard has the latest BIOS Link indicator light is off Try another port on t...

Страница 54: ...ows desktop choose the Start menu and run msinfo32 To export system information to a text file choose the Export option from the File menu Assign a file name and save NVIDIA Firmware Tool MFT Download...

Страница 55: ...E 3 3V_E DGE Typical Power a Passive Cables 12 9W 0 33W TBD 1 5W Active Cables 16 35W 0 33W TBD Maximum Power Passive Cables 15 3W 0 33W TBD 0 75W Active Cables 18 82W 0 33W TBD Maximum power availabl...

Страница 56: ...ble 400LFM 100LFM TBD TBD Temperature Operational 0 C to 55 C Non operational 40 C to 70 C Humidity 90 relative humidity b Altitude Operational 3050m Regulato ry Safety CB cTUVus CE EMC CE FCC VCCI IC...

Страница 57: ...Cable Type Active Mode Standby Mode 12V_EDGE 3 3V_EDGE Typical Power a Passive Cables 16 5W 0 33W TBD 2 2W Active Cables 21 7W 0 33W TBD Maximum Power Passive Cables 19 3W 0 33W TBD 2 2W Active Cables...

Страница 58: ...58 Altitude Operational 3050m Regulatory Safety CB cTUVus CE EMC CE FCC VCCI ICES RCM RoHS RoHS compliant a Typical power for ATIS traffic load b For both operational and non operational states...

Страница 59: ...16 lanes 2 0 and 1 1 compatible Power Voltage 3 3V_EDGE 12V_EDGE Power Cable Type Active Mode Standby Mode 12V_EDGE 3 3V_E DGE Typical Power a Passive Cables 14 5W 0 33W TBD 2 2W Active Cables 17W 0 3...

Страница 60: ...60 Regulatory Safety CB cTUVus CE EMC CE FCC VCCI ICES RCM RoHS RoHS compliant a Typical power for ATIS traffic load b For both operational and non operational states...

Страница 61: ...6 0GT s x16 lanes 2 0 and 1 1 compatible Power Voltage 3 3V_EDGE 12V_EDGE Power Cable Type Active Mode Standby Mode 12V_EDGE 3 3V_EDGE Typical Power a Passive Cables 19 65W 0 33W TBD 2 2W Active Cable...

Страница 62: ...70 C Humidity 90 relative humidity b Altitude Operational 3050m Regulatory Safety CB cTUVus CE EMC CE FCC VCCI ICES RCM RoHS RoHS compliant a Typical power for ATIS traffic load b For both operational...

Страница 63: ...0 and 1 1 compatible Power Voltage 3 3V_EDGE 12V_EDGE Power Cable Type Active Mode Standby Mode 12V_EDGE 3 3V_EDG E Typical Power a Passive Cables 19W 0 33W TBD 2 2W Active Cables 21 5W 0 33W TBD Max...

Страница 64: ...64 EMC CE FCC VCCI ICES RCM RoHS RoHS compliant a Typical power for ATIS traffic load b For both operational and non operational states...

Страница 65: ...Dimensions MCX562A ACAI Drawings MCX562A ACAB Drawings MCX566A CCAI Drawings MCX566A CDAB Drawings MCX566A CDAI Drawings MCX565M CDAI MCX565M CDAB Drawings All dimensions are in millimeters All the m...

Страница 66: ...ons MCX562A ACAI Bracket MCX562A ACAB Bracket MCX566A CCAI MCX566M GDAI MCX566A CDAI Bracket MCX565A CCAB MCX565M CDAB Bracket MCX565M CDAI Bracket MCX566A CDAB Bracket All dimensions are in millimete...

Страница 67: ...entifier printed on the label serial number and the card MAC for the Ethernet protocol MCX562A ACAB Board Label MCX566A CCAI Board Label MCX565M CDAB Board Label The product revisions indicated on the...

Страница 68: ...68 MCX566A CDAB Board Label MCX566A CDAI Board Label MCX565M CDAI Board Label MCX562A ACAI Board Label...

Страница 69: ...airflow numbers May 2020 2 0 Updated board labels Apr 2020 1 9 Updated Specifications Feb 2020 1 8 Added altitude criteria to Specifications Nov 2019 1 7 Added a note to Introduction Oct 2019 1 6 Add...

Страница 70: ...by customer and perform the necessary testing for the application in order to avoid a default of the application or the product Weaknesses in customer s product designs may affect the quality and rel...

Результаты поиска

Содержание Mellanox ConnectX-5

Отзывы:

Похожие инструкции для Mellanox ConnectX-5

Бренды по названию

Популярные бренды

Nvidia Mellanox ConnectX-5, Руководство пользователя

Результаты поиска

Содержание Mellanox ConnectX-5

Отзывы:

Похожие инструкции для Mellanox ConnectX-5

Бренды по названию

Популярные бренды