Nvidia MCX542A-ACAN User Manual Download Page 1

Page: 1 / 66

Exported on Dec/16/2020 07:33 PM

https://docs.mellanox.com/x/V4eO

NVIDIA Mellanox ConnectX-5 Ethernet Adapter

Cards for OCP Spec 2.0 User Manual

25 and 100 Gb/s Ethernet Adapter Cards; Intelligent RDMA (RoCE) enabled NICs. Supporting OCP

Specification 2.0 type 1 and type 2.

Summary of Contents for MCX542A-ACAN

Page 1: ...ttps docs mellanox com x V4eO NVIDIA Mellanox ConnectX 5 Ethernet Adapter Cards for OCP Spec 2 0 User Manual 25 and 100 Gb s Ethernet Adapter Cards Intelligent RDMA RoCE enabled NICs Supporting OCP Sp...

Page 2: ...ernet SFP28 and QSFP28 Interfaces 14 PCI Express Interface 14 LED Interface 14 SMBus Interface 15 Voltage Regulators 15 Thermal Sensors 15 Hardware Installation 16 Safety Warnings 16 Installation Proc...

Page 3: ...Installation Script 35 Installation Procedure 37 Installation Results 39 Installation Logs 39 openibd Script 39 Driver Load Upon System Boot 40 mlnxofedinstall Return Codes 40 Uninstalling MLNX_OFED...

Page 4: ...llanox Drivers 48 Firmware Programming 48 Updating Adapter Firmware 49 Troubleshooting 50 General Troubleshooting 50 Linux Troubleshooting 50 Windows Troubleshooting 51 Specifications 52 MCX542A ACAN...

Page 5: ...t SFP28 PCIe3 0 x8 UEFI Enabled x86 ARM Halogen free no bracket MCX545B GCUN ConnectX 5 EN network interface card for OCP2 0 Type 1 with hostmanagement 50GbE single port QSFP28 PCIe3 0x16 UEFI Enabled...

Page 6: ...nox firmware update and query utility used to update the firmware See http www mellanox com Products Software Firmware Tools mlxup Firmware Utility Mellanox Firmware Tools MFT User Manual User Manual...

Page 7: ...hen discussing memory sizes MB and MBytes are used in this document to mean size in mega Bytes The use of Mb or Mbits small b indicates size in mega bits In this document PCIe is used to mean PCI Expr...

Page 8: ...SERDES 8 0GT s 16 0GT s Dimensions 3 07 in x 4 33 in 78 00mm x 110 05mm RoHS RoHS Compliant IC Part Number MT28808A0 FCCF EV Device ID 4121 for Physical Function PF 4122 for Virtual Function VF a PCIe...

Page 9: ...x 4 33 in 78 00mm x 110 05mm 3 07 in x 4 33 in 78 00mm x 110 05mm 3 07 in x 4 33 in 78 00mm x 110 05mm RoHS RoHS Compliant IC Part Number MT28808A0 FCCF EV Device ID 4119 for Physical Function PF 412...

Page 10: ...ort Card The single port 100Gb s adapter card follows OCP Spec 2 0 Type 2 with 12mm stacking height OCP Spec 2 0 Type 2 Stacking Height Dual port Card The dual port 100Gb s adapter card follows OCP Sp...

Page 11: ...tion Notification IEEE 802 1Qaz ETS IEEE 802 1Qbb PFC IEEE 802 1Qbg IEEE 1588v2 Jumbo frame support 9 6KB Memory SPI Quad includes 128Mbit SPI Quad Flash device W25Q128FVSIG device by ST Microelectron...

Page 12: ...tion decapsulation Quality of Service QoS Support for port based Quality of Service enabling various application requirements for latency and SLA Hardware based I O Virtualization ConnectX 5 provides...

Page 13: ...13 Connectivity Interoperable with 1 10 25 40 50 100 Gb s Ethernet switches Passive copper cable with ESD protection Powered connectors for optical and active cable support...

Page 14: ...ge connectors Link Rates 2 5 5 0 or 8 0GT s Auto Negotiation to x16 x8 x4 x2 or x1 Support for MSI MSI X mechanisms ConnectX 5 Ex PCIe Gen 3 0 4 0 1 1 and 2 0 compatible through x16 edge connectors Li...

Page 15: ...attached either by using four spring loaded push pins that insert into four mounting holes or by screws ConnectX 5 IC has a thermal shutdown safety mechanism that automatically shuts down the Connect...

Page 16: ...rdware Requirements 3 Unpack the product Unpacking the Product 4 Install the adapter card in the system Adapter Card Installation 5 Connect cables or modules to the card Cables and Modules 6 Identify...

Page 17: ...for visible damage that may have occurred during shipping For proper operation and performance please make sure to use a PCIe slot with a corresponding bus width and that can supply sufficient power t...

Page 18: ...stalling the card make sure that the system is off and the power cord is not connected to the server Please follow proper electrical grounding procedures Open the system case Make sure the adapter cli...

Page 19: ...til firmly seated Secure the adapter with the adapter clip or screw To uninstall the adapter card see Uninstalling the Card Cables and Modules To obtain the list of supported Mellanox cables for your...

Page 20: ...m particular to the cable vendor When a logical connection is made Green LED1 will light When data is being transferred Green LED1 will blink Care should be taken as not to impede the air exhaust flow...

Page 21: ...nstalled in a system that operates with voltages that can be lethal Before uninstalling the adapter card please observe the following precautions to avoid injury and prevent damage to system component...

Page 22: ...Wait 30 seconds To remove the card disengage clip 1 and 2 on connector A side To disconnect connector A gently pull the adapter card upwards Disengage clip 3 and clip 4 on the adapter card on Connecto...

Page 23: ...23 6 To remove the card gently pull the adapter card upwards...

Page 24: ...2012 Windows Server 2016 Windows Server 2019 Windows 8 1 Client 64 bit only Windows 10 Client 64 bit only Note The Operating System listed above must run with administrator privileges Downloading Mell...

Page 25: ...tructions to install MLNX_WinOF2 Optional Manually configure your setup to contain the logs option replace LogFile with the relevant directory MLNX_WinOF2 Driver Version _ revision_version _All_Arch e...

Page 26: ...26 4 5 6 Click Next in the Welcome screen Read and accept the license agreement and click Next Select the target folder for the installation...

Page 27: ...user has an OEM card In this case the firmware will not be displayed If the user has a standard Mellanox card with an older firmware version the firmware will be updated accordingly However if the use...

Page 28: ...install the performance tools that are used to measure performance in user environment Documentation contains the User Manual and Release Notes Management tools installation tools used for management...

Page 29: ...9 Click Install to start the installation 10 In case firmware upgrade option was checked in Step 7 you will be notified if a firmware upgrade is required see 11 Click Finish to complete the installati...

Page 30: ...ptional if you wish to control whether to install ND provider or not i e MT_NDPROPERTY default value is True MLNX_WinOF2 Driver Version _ revision_version _All_Arch exe vMT_NDPROPERTY 1 Optional If yo...

Page 31: ...Drivers To see the Mellanox network adapters display the Device Manager and pull down the Network adapters menu Uninstalling Mellanox WinOF 2 Driver Attended Uninstallation To uninstall MLNX_WinOF2 o...

Page 32: ...without running installation perform the following steps Open a CMD console Click Start Task Manager File Run new task and enter CMD Extract the driver and the tools MLNX_WinOF2 2_0_ revision_version...

Page 33: ...33 5 Click Install to extract this folder or click Change to install to a different folder...

Page 34: ...MFT User Manual at www m ellanox com Products Ethernet Drivers Firmware Tools Linux Driver Installation This section describes how to install and test the Mellanox OFED for Linux package on a single...

Page 35: ...oftware package available via the Mellanox web site at http www mellanox com Products Software Ethernet Drivers Linux SW Drivers Download Scroll down to the Download wizard and click the Download tab...

Page 36: ...NX_OFED_LINUX x x x rhel6 3 x86_64 mlnx_add_kernel_support sh m tmp MLNX_OFED_LINUX x x x rhel6 3 x86_64 make tgz Note This program will create MLNX_OFED_LINUX TGZ for rhel6 3 under tmp directory All...

Page 37: ...re update Querying Mellanox devices firmware The DKMS on Debian based OS and the weak modules RedHat OS mechanisms rebuild the initrd initramfs for the respective kernel in order to add the MLNX_OFED...

Page 38: ...the end of installation a message similar to the following Device 1 Device Type ConnectX 5 Ex Part Number MCX546A CDAN Description ConnectX 5 Ex network interface card for OCP 100GbE dual port QSFP28...

Page 39: ...ll also burn an Expansion ROM image In case that your machine has an unsupported network adapter device no firmware update will occur and the error message below will be printed The firmware for this...

Page 40: ...ofedinstall script return codes and their meanings Return Code Meaning 0 The installation ended successfully 1 The installation failed 2 No firmware was found for the adapter device 22 Invalid paramet...

Page 41: ...MLNX_OFED file RPM GPG KEY Mellanox Importing GPG key 0x6224C050 Userid Mellanox Technologies Mellanox Technologies Signing Key v2 support mellanox com From repos MLNX_OFED MLNX_OFED file RPM GPG KEY...

Page 42: ...3 17 4 301 fc2 1 x86_64 without KMP support mlnx ofed guest 3 17 4 301 fc21 x86_64 noarch MLNX_OFED guest installer package for kernel 3 17 4 301 fc2 1 x86_64 without KMP support mlnx ofed hpc 3 17 4...

Page 43: ...ed apt key list pub 1024D A9E4B643 2013 08 11 uid Mellanox Technologies support mellanox com sub 1024g 09FCC269 2013 08 11 Update the apt get cache sudo apt get update Installing MLNX_OFED Using the a...

Page 44: ...t Number MCX546A CDAN Description ConnectX 5 Ex network interface card for OCP 100GbE dual port QSFP28 PCIe4 0 x16 no bracket ROHS R6 PSID MT_1020120019 PCI Device Name 0000 09 00 0 Port1 GUID 0002c90...

Page 45: ...is updated the following message is printed to the system s standard logging file fw_updater Firmware was updated Please reboot your system for the changes to take effect Otherwise the following messa...

Page 46: ...by resigning the kernel module Hence we recommend backing up a copy prior to removing the signature To remove the signature from the MLNX_OFED kernel modules Remove the signature rpm qa grep E kernel...

Page 47: ...th an adapter card based on one of the following Mellanox Technologies devices ConnectX 5 EN firmware fw ConnectX5 ConnectX 5 Ex EN firmware fw ConnectX5 Device ID For the latest list of device IDs pl...

Page 48: ...remove n nmlx5 rdma esxcli software vib remove n nmlx5 core Reboot the server Firmware Programming Download the VMware bootable binary images v4 6 0 from the Mellanox Firmware Tools MFT site ESXi 6 5...

Page 49: ...mware Update Example server1 mlxup Querying Mellanox devices firmware Device Type ConnectX 5 Part Number MCX546A CDAN Description ConnectX 5 Ex network interface card for OCP with host management 100G...

Page 50: ...ng another adapter Try removing and re installing all adapters Check that cables are connected properly Make sure your motherboard has the latest BIOS Link indicator light is off Try another port on t...

Page 51: ...ows desktop choose the Start menu and run msinfo32 To export system information to a text file choose the Export option from the File menu Assign a file name and save Mellanox Firmware Tool MFT Downlo...

Page 52: ...r Cable Type Typical Power a Passive Cables 12 95W 1 5W Active Cables 15 95W Maximum Power Passive Cables 15 55W 1 5W Active Cables 18 55W Maximum power available through QSFP28 port 1 5W Environmenta...

Page 53: ...bles 14 98W Maximum power available through QSFP28 port 1 5W Environmental Temperature Operational 0 C to 55 C Non operational 40 C to 70 C Humidity 90 relative humidity b Altitude Operational 3050m A...

Page 54: ...ERDES 8 0GT s respectively 16 lanes 2 0 and 1 1 compatible Power Voltage 3 3VAUX 5VAUX 12V Power Cable Type Typical Power a Passive Cables 13 77W Maximum Power Passive Cables 16 35W 1 5W Active Cables...

Page 55: ...2 0 and 1 1 compatible Power Voltage 3 3VAUX 5VAUX 12V Power Cable Type Typical Power a Passive Cables 14 48W Maximum Power Passive Cables 17 57W 1 5W Active Cables 21 07W Maximum power available thr...

Page 56: ...ltage 3 3VAUX 5VAUX 12V Power Cable Type Typical Power a Passive Cables 14 48W Maximum Power Passive Cables 17 57W 1 5W Active Cables 21 07W Maximum power available through QSFP28 port 3 5W Environmen...

Page 57: ...e 3 3VAUX 5VAUX 12V Power Cable Type Typical Power a Passive Cables 15 4W 1 5W Active Cables 23 2W Maximum Power Passive Cables 18 1W 1 5W Active Cables 25 9W Maximum power available through QSFP28 po...

Page 58: ...ly a maximum bandwidth of 256Gb s only 16 16GT s including overhead Power Voltage 3 3VAUX 5VAUX 12V Power Cable Type Typical Power a Passive Cables 20 03W 3 5W Active Cables 27 8W Maximum Power Passiv...

Page 59: ...AN Mechanical Drawing and Dimensions MCX542B ACAN Mechanical Drawing and Dimensions All dimensions are in millimeters All the mechanical tolerances are 0 1mm For the 3D Model of the card please refer...

Page 60: ...60 MCX545B CCUN Mechanical Drawing and Dimensions MCX545A CCAN and MCX545A CCUN Mechanical Drawing and Dimensions...

Page 61: ...61 MCX546A CDAN and MCX546A BCAN Mechanical Drawing and Dimensions...

Page 62: ...62...

Page 63: ...r card has a different identifier printed on the label serial number and the card MAC for the Ethernet protocol MCX542A ACAN Board Label Example The product revisions indicated on the labels in the fo...

Page 64: ...ols in Specifications Oct 2019 1 8 Updated board labels in Finding the MAC and Serial Number on the Adapter Card May 2019 1 7 Added a note to MCX542B ACAN Specifications Apr 2019 1 6 Updated comments...

Page 65: ...arily performed by NVIDIA It is customer s sole responsibility to evaluate and determine the applicability of any information contained in this document ensure the product is suitable and fit for the...

Page 66: ...A the NVIDIA logo and Mellanox are trademarks and or registered trademarks of Mellanox Technologies Ltd and or NVIDIA Corporation in the U S and in other countries Other company and product names may...

Search results

Summary of Contents for MCX542A-ACAN

Reviews:

Related manuals for MCX542A-ACAN

Brands by name

Popular brands

Nvidia MCX542A-ACAN, User Manual

Search results

Summary of Contents for MCX542A-ACAN

Reviews:

Related manuals for MCX542A-ACAN

Brands by name

Popular brands