background image

64

Document Revision History

Date

Revision

Description of Changes

Oct. 2020

2.4

Added MCX545B-GCUN support across the document. 

Jul. 2020

2.3

Added MCX542B-ACUN support across the document.

Apr. 2020

2.2

Added MCX545A-CCUN support across the document.

Feb. 2020

2.1

Added altitude criteria to "Specifications".

Jan. 2020

2.0

Added MCX545B-CCUN support across the document.

Oct. 2019

1.9

Updated the supported Ethernet protocols in 
"Specifications".

Oct. 2019

1.8

Updated board labels in "Finding the MAC and Serial 
Number on the Adapter Card".

May. 2019

1.7

Added a note to "MCX542B-ACAN Specifications".

Apr. 2019

1.6

Updated comments in "Introduction", "Hardware 
Installation" and "Specifications".

Feb. 2019

1.5

Added a comment to "Introduction" and 
"Specifications".

Jan. 2019

1.4

Added MCX542B-ACAN and MCX546A-BCAN support 
across the document.

Dec. 2018

1.3

Migrated to on-line format; minor reorganization.

Oct. 2018

1.2

Updated MCX542A-ACAN Specification.

Sep. 2018

1.1

Added MCX546A-CDAN and MCX542A-ACAN support 
across the document.

Jul. 2017

1.0

First release

Содержание MCX542A-ACAN

Страница 1: ...ttps docs mellanox com x V4eO NVIDIA Mellanox ConnectX 5 Ethernet Adapter Cards for OCP Spec 2 0 User Manual 25 and 100 Gb s Ethernet Adapter Cards Intelligent RDMA RoCE enabled NICs Supporting OCP Sp...

Страница 2: ...ernet SFP28 and QSFP28 Interfaces 14 PCI Express Interface 14 LED Interface 14 SMBus Interface 15 Voltage Regulators 15 Thermal Sensors 15 Hardware Installation 16 Safety Warnings 16 Installation Proc...

Страница 3: ...Installation Script 35 Installation Procedure 37 Installation Results 39 Installation Logs 39 openibd Script 39 Driver Load Upon System Boot 40 mlnxofedinstall Return Codes 40 Uninstalling MLNX_OFED...

Страница 4: ...llanox Drivers 48 Firmware Programming 48 Updating Adapter Firmware 49 Troubleshooting 50 General Troubleshooting 50 Linux Troubleshooting 50 Windows Troubleshooting 51 Specifications 52 MCX542A ACAN...

Страница 5: ...t SFP28 PCIe3 0 x8 UEFI Enabled x86 ARM Halogen free no bracket MCX545B GCUN ConnectX 5 EN network interface card for OCP2 0 Type 1 with hostmanagement 50GbE single port QSFP28 PCIe3 0x16 UEFI Enabled...

Страница 6: ...nox firmware update and query utility used to update the firmware See http www mellanox com Products Software Firmware Tools mlxup Firmware Utility Mellanox Firmware Tools MFT User Manual User Manual...

Страница 7: ...hen discussing memory sizes MB and MBytes are used in this document to mean size in mega Bytes The use of Mb or Mbits small b indicates size in mega bits In this document PCIe is used to mean PCI Expr...

Страница 8: ...SERDES 8 0GT s 16 0GT s Dimensions 3 07 in x 4 33 in 78 00mm x 110 05mm RoHS RoHS Compliant IC Part Number MT28808A0 FCCF EV Device ID 4121 for Physical Function PF 4122 for Virtual Function VF a PCIe...

Страница 9: ...x 4 33 in 78 00mm x 110 05mm 3 07 in x 4 33 in 78 00mm x 110 05mm 3 07 in x 4 33 in 78 00mm x 110 05mm RoHS RoHS Compliant IC Part Number MT28808A0 FCCF EV Device ID 4119 for Physical Function PF 412...

Страница 10: ...ort Card The single port 100Gb s adapter card follows OCP Spec 2 0 Type 2 with 12mm stacking height OCP Spec 2 0 Type 2 Stacking Height Dual port Card The dual port 100Gb s adapter card follows OCP Sp...

Страница 11: ...tion Notification IEEE 802 1Qaz ETS IEEE 802 1Qbb PFC IEEE 802 1Qbg IEEE 1588v2 Jumbo frame support 9 6KB Memory SPI Quad includes 128Mbit SPI Quad Flash device W25Q128FVSIG device by ST Microelectron...

Страница 12: ...tion decapsulation Quality of Service QoS Support for port based Quality of Service enabling various application requirements for latency and SLA Hardware based I O Virtualization ConnectX 5 provides...

Страница 13: ...13 Connectivity Interoperable with 1 10 25 40 50 100 Gb s Ethernet switches Passive copper cable with ESD protection Powered connectors for optical and active cable support...

Страница 14: ...ge connectors Link Rates 2 5 5 0 or 8 0GT s Auto Negotiation to x16 x8 x4 x2 or x1 Support for MSI MSI X mechanisms ConnectX 5 Ex PCIe Gen 3 0 4 0 1 1 and 2 0 compatible through x16 edge connectors Li...

Страница 15: ...attached either by using four spring loaded push pins that insert into four mounting holes or by screws ConnectX 5 IC has a thermal shutdown safety mechanism that automatically shuts down the Connect...

Страница 16: ...rdware Requirements 3 Unpack the product Unpacking the Product 4 Install the adapter card in the system Adapter Card Installation 5 Connect cables or modules to the card Cables and Modules 6 Identify...

Страница 17: ...for visible damage that may have occurred during shipping For proper operation and performance please make sure to use a PCIe slot with a corresponding bus width and that can supply sufficient power t...

Страница 18: ...stalling the card make sure that the system is off and the power cord is not connected to the server Please follow proper electrical grounding procedures Open the system case Make sure the adapter cli...

Страница 19: ...til firmly seated Secure the adapter with the adapter clip or screw To uninstall the adapter card see Uninstalling the Card Cables and Modules To obtain the list of supported Mellanox cables for your...

Страница 20: ...m particular to the cable vendor When a logical connection is made Green LED1 will light When data is being transferred Green LED1 will blink Care should be taken as not to impede the air exhaust flow...

Страница 21: ...nstalled in a system that operates with voltages that can be lethal Before uninstalling the adapter card please observe the following precautions to avoid injury and prevent damage to system component...

Страница 22: ...Wait 30 seconds To remove the card disengage clip 1 and 2 on connector A side To disconnect connector A gently pull the adapter card upwards Disengage clip 3 and clip 4 on the adapter card on Connecto...

Страница 23: ...23 6 To remove the card gently pull the adapter card upwards...

Страница 24: ...2012 Windows Server 2016 Windows Server 2019 Windows 8 1 Client 64 bit only Windows 10 Client 64 bit only Note The Operating System listed above must run with administrator privileges Downloading Mell...

Страница 25: ...tructions to install MLNX_WinOF2 Optional Manually configure your setup to contain the logs option replace LogFile with the relevant directory MLNX_WinOF2 Driver Version _ revision_version _All_Arch e...

Страница 26: ...26 4 5 6 Click Next in the Welcome screen Read and accept the license agreement and click Next Select the target folder for the installation...

Страница 27: ...user has an OEM card In this case the firmware will not be displayed If the user has a standard Mellanox card with an older firmware version the firmware will be updated accordingly However if the use...

Страница 28: ...install the performance tools that are used to measure performance in user environment Documentation contains the User Manual and Release Notes Management tools installation tools used for management...

Страница 29: ...9 Click Install to start the installation 10 In case firmware upgrade option was checked in Step 7 you will be notified if a firmware upgrade is required see 11 Click Finish to complete the installati...

Страница 30: ...ptional if you wish to control whether to install ND provider or not i e MT_NDPROPERTY default value is True MLNX_WinOF2 Driver Version _ revision_version _All_Arch exe vMT_NDPROPERTY 1 Optional If yo...

Страница 31: ...Drivers To see the Mellanox network adapters display the Device Manager and pull down the Network adapters menu Uninstalling Mellanox WinOF 2 Driver Attended Uninstallation To uninstall MLNX_WinOF2 o...

Страница 32: ...without running installation perform the following steps Open a CMD console Click Start Task Manager File Run new task and enter CMD Extract the driver and the tools MLNX_WinOF2 2_0_ revision_version...

Страница 33: ...33 5 Click Install to extract this folder or click Change to install to a different folder...

Страница 34: ...MFT User Manual at www m ellanox com Products Ethernet Drivers Firmware Tools Linux Driver Installation This section describes how to install and test the Mellanox OFED for Linux package on a single...

Страница 35: ...oftware package available via the Mellanox web site at http www mellanox com Products Software Ethernet Drivers Linux SW Drivers Download Scroll down to the Download wizard and click the Download tab...

Страница 36: ...NX_OFED_LINUX x x x rhel6 3 x86_64 mlnx_add_kernel_support sh m tmp MLNX_OFED_LINUX x x x rhel6 3 x86_64 make tgz Note This program will create MLNX_OFED_LINUX TGZ for rhel6 3 under tmp directory All...

Страница 37: ...re update Querying Mellanox devices firmware The DKMS on Debian based OS and the weak modules RedHat OS mechanisms rebuild the initrd initramfs for the respective kernel in order to add the MLNX_OFED...

Страница 38: ...the end of installation a message similar to the following Device 1 Device Type ConnectX 5 Ex Part Number MCX546A CDAN Description ConnectX 5 Ex network interface card for OCP 100GbE dual port QSFP28...

Страница 39: ...ll also burn an Expansion ROM image In case that your machine has an unsupported network adapter device no firmware update will occur and the error message below will be printed The firmware for this...

Страница 40: ...ofedinstall script return codes and their meanings Return Code Meaning 0 The installation ended successfully 1 The installation failed 2 No firmware was found for the adapter device 22 Invalid paramet...

Страница 41: ...MLNX_OFED file RPM GPG KEY Mellanox Importing GPG key 0x6224C050 Userid Mellanox Technologies Mellanox Technologies Signing Key v2 support mellanox com From repos MLNX_OFED MLNX_OFED file RPM GPG KEY...

Страница 42: ...3 17 4 301 fc2 1 x86_64 without KMP support mlnx ofed guest 3 17 4 301 fc21 x86_64 noarch MLNX_OFED guest installer package for kernel 3 17 4 301 fc2 1 x86_64 without KMP support mlnx ofed hpc 3 17 4...

Страница 43: ...ed apt key list pub 1024D A9E4B643 2013 08 11 uid Mellanox Technologies support mellanox com sub 1024g 09FCC269 2013 08 11 Update the apt get cache sudo apt get update Installing MLNX_OFED Using the a...

Страница 44: ...t Number MCX546A CDAN Description ConnectX 5 Ex network interface card for OCP 100GbE dual port QSFP28 PCIe4 0 x16 no bracket ROHS R6 PSID MT_1020120019 PCI Device Name 0000 09 00 0 Port1 GUID 0002c90...

Страница 45: ...is updated the following message is printed to the system s standard logging file fw_updater Firmware was updated Please reboot your system for the changes to take effect Otherwise the following messa...

Страница 46: ...by resigning the kernel module Hence we recommend backing up a copy prior to removing the signature To remove the signature from the MLNX_OFED kernel modules Remove the signature rpm qa grep E kernel...

Страница 47: ...th an adapter card based on one of the following Mellanox Technologies devices ConnectX 5 EN firmware fw ConnectX5 ConnectX 5 Ex EN firmware fw ConnectX5 Device ID For the latest list of device IDs pl...

Страница 48: ...remove n nmlx5 rdma esxcli software vib remove n nmlx5 core Reboot the server Firmware Programming Download the VMware bootable binary images v4 6 0 from the Mellanox Firmware Tools MFT site ESXi 6 5...

Страница 49: ...mware Update Example server1 mlxup Querying Mellanox devices firmware Device Type ConnectX 5 Part Number MCX546A CDAN Description ConnectX 5 Ex network interface card for OCP with host management 100G...

Страница 50: ...ng another adapter Try removing and re installing all adapters Check that cables are connected properly Make sure your motherboard has the latest BIOS Link indicator light is off Try another port on t...

Страница 51: ...ows desktop choose the Start menu and run msinfo32 To export system information to a text file choose the Export option from the File menu Assign a file name and save Mellanox Firmware Tool MFT Downlo...

Страница 52: ...r Cable Type Typical Power a Passive Cables 12 95W 1 5W Active Cables 15 95W Maximum Power Passive Cables 15 55W 1 5W Active Cables 18 55W Maximum power available through QSFP28 port 1 5W Environmenta...

Страница 53: ...bles 14 98W Maximum power available through QSFP28 port 1 5W Environmental Temperature Operational 0 C to 55 C Non operational 40 C to 70 C Humidity 90 relative humidity b Altitude Operational 3050m A...

Страница 54: ...ERDES 8 0GT s respectively 16 lanes 2 0 and 1 1 compatible Power Voltage 3 3VAUX 5VAUX 12V Power Cable Type Typical Power a Passive Cables 13 77W Maximum Power Passive Cables 16 35W 1 5W Active Cables...

Страница 55: ...2 0 and 1 1 compatible Power Voltage 3 3VAUX 5VAUX 12V Power Cable Type Typical Power a Passive Cables 14 48W Maximum Power Passive Cables 17 57W 1 5W Active Cables 21 07W Maximum power available thr...

Страница 56: ...ltage 3 3VAUX 5VAUX 12V Power Cable Type Typical Power a Passive Cables 14 48W Maximum Power Passive Cables 17 57W 1 5W Active Cables 21 07W Maximum power available through QSFP28 port 3 5W Environmen...

Страница 57: ...e 3 3VAUX 5VAUX 12V Power Cable Type Typical Power a Passive Cables 15 4W 1 5W Active Cables 23 2W Maximum Power Passive Cables 18 1W 1 5W Active Cables 25 9W Maximum power available through QSFP28 po...

Страница 58: ...ly a maximum bandwidth of 256Gb s only 16 16GT s including overhead Power Voltage 3 3VAUX 5VAUX 12V Power Cable Type Typical Power a Passive Cables 20 03W 3 5W Active Cables 27 8W Maximum Power Passiv...

Страница 59: ...AN Mechanical Drawing and Dimensions MCX542B ACAN Mechanical Drawing and Dimensions All dimensions are in millimeters All the mechanical tolerances are 0 1mm For the 3D Model of the card please refer...

Страница 60: ...60 MCX545B CCUN Mechanical Drawing and Dimensions MCX545A CCAN and MCX545A CCUN Mechanical Drawing and Dimensions...

Страница 61: ...61 MCX546A CDAN and MCX546A BCAN Mechanical Drawing and Dimensions...

Страница 62: ...62...

Страница 63: ...r card has a different identifier printed on the label serial number and the card MAC for the Ethernet protocol MCX542A ACAN Board Label Example The product revisions indicated on the labels in the fo...

Страница 64: ...ols in Specifications Oct 2019 1 8 Updated board labels in Finding the MAC and Serial Number on the Adapter Card May 2019 1 7 Added a note to MCX542B ACAN Specifications Apr 2019 1 6 Updated comments...

Страница 65: ...arily performed by NVIDIA It is customer s sole responsibility to evaluate and determine the applicability of any information contained in this document ensure the product is suitable and fit for the...

Страница 66: ...A the NVIDIA logo and Mellanox are trademarks and or registered trademarks of Mellanox Technologies Ltd and or NVIDIA Corporation in the U S and in other countries Other company and product names may...

Отзывы: