Nvidia ConnectX-6 Dx MCX623430MS-CDAB Скачать руководство пользователя страница 1

Страница: 1 / 75

Exported on Sep/19/2022 12:36 PM

NVIDIA ConnectX-6 Dx Ethernet Adapter

Cards for OCP Spec 3.0 User Manual

Содержание ConnectX-6 Dx MCX623430MS-CDAB

Страница 1: ...Exported on Sep 19 2022 12 36 PM NVIDIA ConnectX 6 Dx Ethernet Adapter Cards for OCP Spec 3 0 User Manual...

Страница 2: ...t 16 Ethernet Networking Interfaces 18 PCI Express Interface 18 Networking Ports LEDs Interface 18 SMBus Interface 19 Voltage Regulators 19 CPLD Interface 19 Heatsink 20 Thermal Sensors 20 Hardware In...

Страница 3: ...ning Installation 38 Uninstalling NVIDIA WinOF 2 Driver 40 Firmware Upgrade 40 Deploying the Driver on a Nano Server 41 Linux Driver Installation 42 Prerequisites 42 Downloading NVIDIA OFED 42 Install...

Страница 4: ...DAB MCX623436MC CDAB MCX623436MN CDAB MCX623436MS CDAB MCX623436AS CDAI Specifications 64 MCX623430MS CDAB Specifications 65 MCX623435AC CDAB MCX623435AE CDAB MCX623435AN CDAB MCX623435AC VDAB MCX6234...

Страница 5: ...t management Single port QSFP56 PCIe 4 0 x16 No Crypto Thumbscrew Pull Tab Bracket MCX623435AC CDAB ConnectX 6 Dx EN adapter card 100GbE OCP3 0 With Host Management Single port QSFP56 PCIe 4 0 x16 Cry...

Страница 6: ...nt Single port QSFP56 Multi Host or Socket Direct PCIe 4 0 x16 Crypto and Secure Boot Thumbscrew Pull Tab Bracket MCX623432A C ADAB ConnectX 6 Dx EN adapter card 25GbE OCP3 0 With Host Management Dual...

Страница 7: ...e http www nvidia com Products Software Firmware Tools mlxup Firmware Utility NVIDIA Firmware Tools MFT User Manual User Manual describing the set of MFT firmware management tools for a single node Se...

Страница 8: ...en discussing memory sizes MB and MBytes are used in this document to mean size in MegaBytes The use of Mb or Mbits small b indicates the size in Megabits In this document PCIe is used to mean PCI Exp...

Страница 9: ...er port speed number of ports and PCI Express speed Products Overview ConnectX 6 Dx 25GbE Adapter Cards for OCP 3 0 OPN MCX623432AN ADAB MCX623432AC ADAB MCX623432AE ADAB MCX623432AS ADAB Network Conn...

Страница 10: ...Device ID 4125 for Physical Function PF 4126 for Virtual Function VF ConnectX 6 Dx 100GbE Adapter Cards for OCP 3 0 OPN MCX 6234 36AN CDAB MCX 6234 36AC CDAB MCX 6234 35M N CDAB MCX 6234 36M N CDA B...

Страница 11: ...ant RoHS Compl iant RoHS Compl iant RoHS Compl iant RoHS Compl iant RoHS Compl iant RoHS Compl iant RoHS Compli ant RoHS Compl iant IC Part Numb er MT289 28A0 XCCF CE MT289 28A0 XCCF CE MT289 28A0 XCC...

Страница 12: ...3bm 100 Gigabit Ethernet IEEE 802 3by Ethernet Consortium25 50 Gigabit Ethernet supporting all FEC modes IEEE 802 3ba 40 Gigabit Ethernet IEEE 802 3by 25 Gigabit Ethernet IEEE 802 3ae 10 Gigabit Ether...

Страница 13: ...y of Service QoS Support for port based Quality of Service enabling various application requirements for latency and SLA Hardware based I O Virtualization ConnectX 6 Dx provides dedicated adapter reso...

Страница 14: ...ompute and storage racks with direct connectivity between multiple hosts and the centralized network controller This enables direct data access with the lowest latency to significantly improve densiti...

Страница 15: ...0 through an x16 edge connector 3 Ethernet Networking Interface Ethernet traffic is transmitted through the adapter s SFP28 SFP56 QSFP56 connectors The networking connectors allow for the use of modul...

Страница 16: ...AES data at rest cryptographic operations Stateful firewall solution acceleration powered by Open vSwitch connection tracking and NVIDIA s ASAP2 technology Embedded hardware root of trust and support...

Страница 17: ...on through the BIF 2 0 pins which it drives to the adapter card The NVIDIA OCP3 0 card has an internal logic that uses the BIF 2 0 data and determines the correct operating mode to boot at The combina...

Страница 18: ...atures PCIe Gen 3 0 4 0 1 1 and 2 0 compatible through x16 edge connectors Link Rates 2 5 5 0 8 0 or 16GT s Auto Negotiation to x16 x8 x4 x2 or x1 Support for MSI MSI X mechanisms Networking Ports LED...

Страница 19: ...ferent types of Mezzanine cards FRU EEPROM is accessible through SMCLK and SMDATA FRU EEPROM address is defined according to SLOT_ID0 and SLOT_ID1 and its capacity is 4Kb SMBus Interface ConnectX 6 Dx...

Страница 20: ...the device crosses the 100 C threshold a Warning Threshold message will be issued by the management SW indicating to system administration that the card has crossed the Warning threshold Note that thi...

Страница 21: ...w of 200LFM at this maximum ambient temperature is required for HCA cards and NICs To guarantee proper airflow allow at least 8cm 3 inches of clearance around the ventilation openings During Lightning...

Страница 22: ...uirements 3 Follow the safety precautions Refer to Safety Precautions 4 Follow the pre installation checklist Refer to Pre Installation Checklist 5 Optional Replace the assembled OCP 3 0 bracket with...

Страница 23: ...ow Direction Cold Aisle Cooling Port to heatsink Airflow Direction Software Requirements See Operating Systems Distributions section under the Introduction section Software Stacks NVIDIA OpenFabric so...

Страница 24: ...Replacement Instructions Unable to render include or excerpt include Could not retrieve page OCP 3 0 Adapter Card Installation Instructions This section provides detailed instructions on installing y...

Страница 25: ...ds with Thumbscrew Pull tab Bracket Before installing the card ensure the system is off and the power cord is not connected to the server Please follow proper electrical grounding procedures Open the...

Страница 26: ...ard with the system rails Push the card until connectors are in full mate and a clicking sound is heard Cards with Ejector Latch Bracket Before installing the card ensure the system is off and the pow...

Страница 27: ...cable to the rack Determine the correct orientation of the connector to the card before inserting the connector Do not try and insert the connector upside down This may damage the adapter card Insert...

Страница 28: ...To remove a cable disengage the locks and slowly pull the connector away from the port receptacle The LED indicator will turn off when the cable is unseated Identifying the Card in Your System On Linu...

Страница 29: ...a system that operates with voltages that can be lethal Before uninstalling the adapter card please observe the following precautions to avoid injury and prevent damage to system components Remove any...

Страница 30: ...ut the adapter card Cards with Internal Lock Bracket Extraction Verify that the system is powered off and unplugged Wait 30 seconds Gently extract the adapter card from the server Cards with Ejector L...

Страница 31: ...31 1 2 3 4 Verify that the system is powered off and unplugged Wait 30 seconds Open the ejector latch Gently extract the adapter card from the server...

Страница 32: ...Windows Server 2012 Windows Server 2016 Windows 8 1 Client 64 bit only Windows 10 Client 64 bit only Note The Operating System listed above must run with administrator privileges Downloading NVIDIA Wi...

Страница 33: ...n the logs option replace LogFile with the relevant directory MLNX_WinOF2 1_10_ revision_version _All_x64 exe v l vx LogFile Optional If you do not want to upgrade your firmware version Note MT_SKIPFW...

Страница 34: ...Read then accept the license agreement and click Next Select the target folder for the installation The firmware upgrade screen will be displayed in the following cases If the user has an OEM card In...

Страница 35: ...DIA card only the NVIDIA card will be updated Select a Complete or Custom installation follow Step a and on Select the desired feature to install Performance tools install the performance tools that a...

Страница 36: ...ools used for diagnostics such as mlx5cmd Click Next to install the desired tools Click Install to start the installation In case that firmware upgrade option was checked in Step 7 you will be notifie...

Страница 37: ...ant to control whether to install ND provider or not Note MT_NDPROPERTY default value is True MLNX_WinOF2 1_10_All_x64 exe vMT_NDPROPERTY 1 Optional If you do not wish to upgrade your firmware version...

Страница 38: ...vers OS To see the NVIDIA network adapters display the Device Manager and pull down the Network adapters menu Extracting Files Without Running Installation To extract the files without running install...

Страница 39: ...3 4 5 Click Next to create a server image Click Change and specify the location in which the files are extracted to Click Install to extract this folder or click Change to install to a different fold...

Страница 40: ...o uninstall MLNX_WinOF2 in unattended mode Open a CMD console Click Task Manager File Run new task and enter CMD To uninstall the driver run MLNX_WinOF2 1_10_All_x64 exe S x v qn Firmware Upgrade If t...

Страница 41: ...lows This location should be specified for DriversPath property when injecting driver into the Nano server image New NanoServerImage MediaPath Path To Media en_us BasePath Base TargetPath InjectingDri...

Страница 42: ...OFED Verify that the system has a NVIDIA network adapter installed The following example shows a system with an installed NVIDIA adapter card lspci v grep Mellanox 86 00 0 Network controller 0207 Mell...

Страница 43: ...install script with the without fw update option Note If you wish to perform a firmware upgrade using customized FW binaries you can provide a path to the folder that contains the FW binary files by...

Страница 44: ...he Ethernet fabric a Subnet Manager must be running on one of the fabric nodes At this point NVIDIA OFED for Linux has already installed the OpenSM Subnet Manager on your machine For the list of insta...

Страница 45: ...edinstall force MLNX_OFED for Ubuntu should be installed with the following flags in the root environment mlnxofedinstall without dkms add kernel support kernel kernel version in chroot without fw upd...

Страница 46: ...directory except for the following packages which are installed under the opt directory fca and ibutils The kernel modules are installed under lib modules uname r updates on SLES and Fedora Distributi...

Страница 47: ...ontact your HW vendor Installation Logs While installing MLNX_OFED the install log for each selected package will be saved in a separate log file The path to the directory containing the log files wil...

Страница 48: ...anings Return Code Meaning 0 The installation ended successfully 1 The installation failed 2 No firmware was found for the adapter device 22 Invalid parameter 28 Not enough free space 171 Not applicab...

Страница 49: ...pg Mellanox Technologies support mellanox com Create a yum repository configuration file called etc yum repos d mlnx_ofed repo with the following content mlnx_ofed name MLNX_OFED Repository baseurl fi...

Страница 50: ...noarch MLNX_OFED hypervisor installer package for kernel 3 17 4 301 fc21 x86_64 without KMP support mlnx ofed vma 3 17 4 301 fc21 x86_64 noarch MLNX_OFED vma installer package for kernel 3 17 4 301 f...

Страница 51: ...2013 08 11 Update the apt get cache sudo apt get update Installing MLNX_OFED Using the apt get Tool After setting up the apt get repository for MLNX_OFED package perform the following View the availa...

Страница 52: ...0 x16 Crypto and Secure Boot Thumbscrew Pull Tab Bracket PSID MT_1020120019 PCI Device Name 0000 09 00 0 Port1 GUID 0002c9000100d051 Port2 MAC 0002c9000002 Versions Current Available FW 2 32 5000 2 3...

Страница 53: ...n boot If the firmware is updated the following message is printed to the system s standard logging file fw_updater Firmware was updated Please reboot your system for the changes to take effect Otherw...

Страница 54: ...nt Once a key is in the MOK list it will be automatically propagated to the system key ring and subsequent will be booted when the UEFI Secure Boot is enabled Removing Signature from kernel Modules Th...

Страница 55: ...lication of the user s system it may be necessary to modify the default configuration of network adapters based on the ConnectX adapters In case that tuning is required please refer to the Performance...

Страница 56: ...er with root permissions List all the existing NATIVE ESXi driver modules See Step 4 in Installing NVIDIA NATIVE ESXi Driver for VMware vSphere Remove each module esxcli software vib remove n nmlx5 rd...

Страница 57: ...Querying Mellanox devices firmware Device Type ConnectX 6 Dx Part Number MCX623436AC CDAB Description ConnectX 6 Dx EN adapter card 100GbE OCP3 0 With Host Management Dual port QSFP56 PCIe 4 0 x16 Cr...

Страница 58: ...ng another adapter Try removing and re installing all adapters Check that cables are connected properly Make sure your motherboard has the latest BIOS Link indicator light is off Try another port on t...

Страница 59: ...ows desktop choose the Start menu and run msinfo32 To export system information to a text file choose the Export option from the File menu Assign a file name and save NVIDIA Firmware Tool NFT Download...

Страница 60: ...6 0GT s 16 lanes 2 0 and 1 1 compatible Power and Environmen tal Voltage 3 3VAUX 12V Power Cable Type Active Mode Standby Mode Typical Power b Passive Cables 15 7W 3 8W Maximum Power Passive Cables 19...

Страница 61: ...SFP56 Ethernet copper and optical Retention Mechanism MCX623432A N C GDAB Thumbscrew Pull Tab Bracket MCX623432AS GDAI Internal Lock Protocol Support Ethernet 50GBASE R2 50GBASE R4 40GBASE CR4 40GBAS...

Страница 62: ...4 52 in x 2 99 in 115 00mm x 76 00mm Connector Single QSFP56 Ethernet copper and optical Retention Mechanism Thumbscrew Pull Tab Bracket Protoco l Support Ethernet 100GBASE CR4 100GBASE KR4 100GBASE...

Страница 63: ...er for ATIS traffic load c For both operational and non operational states MCX623439MC CDAB MCX623439MN CDAB Specifications Physi cal Form Factor Small Form Factor SFF OCP 3 0 Size 4 52 in x 2 99 in 1...

Страница 64: ...y CB cTUVus CE EMC CE FCC VCCI ICES RCM RoHS RoHS compliant MCX623436AC CDAB MCX623436AN CDAB MCX623436AS CDAB MCX623436MC CDAB MCX623436MN CDAB MCX623436MS CDAB MCX623436AS CDAI Specifications Physi...

Страница 65: ...CE FCC VCCI ICES RCM RoHS RoHS compliant a ConnectX 6 Dx adapters supplement the auto negotiation specification to get better bit error rates and longer cable reaches This supplemental feature only in...

Страница 66: ...ve Cable 600LFM 100LFM 300LFM 100LFM Regula tory Safety CB cTUVus CE EMC CE FCC VCCI ICES RCM RoHS RoHS compliant a ConnectX 6 Dx adapters supplement the auto negotiation specification to get better b...

Страница 67: ...9W 5 6W Maximum Power Passive Cables 24W 11 4W Maximum power available through QSFP56 port 5W Temperature Operation al 0 C to 55 C Non operation al 40 C to 70 C Humidity 90 relative humidity c Altitud...

Страница 68: ...56 OCP 3 0 Mechanical Drawing and Dimensions ConnectX 6 Dx Dual Port QSFP56 OCP 3 0 Mechanical Drawing and Dimensions ConnectX 6 Dx Single Port QSFP56 OCP 3 0 Mechanical Drawing and Dimensions All dim...

Страница 69: ...9 Bracket Mechanical Drawings and Dimensions Dual port SFP28 SFP56 Thumbscrew Bracket Dual port QSFP56 Thumbscrew Bracket All dimensions are in millimeters The bracket s mechanical tolerance is 0 25mm...

Страница 70: ...70 Single port QSFP56 Thumbscrew Bracket...

Страница 71: ...al number and the card MAC for the Ethernet protocol MCX623432AN ADAB Board Label Example MCX623432AC ADAB Board Label Example MCX623432AN GDAB Board Label Example MCX623432AC GDAB Board Label Example...

Страница 72: ...72 MCX623436AN CDAB Board Label Example MCX623436AC CDAB Board Label Example MCX623435AN VDAB Board Label Example MCX623435AC VDAB Board Label Example...

Страница 73: ...36MN CDAB and MCX623436MS CDAB Mar 2021 1 9 Added OPN MCX623432AS ADAB Feb 2021 1 8 Added OPN MCX623439MC N CDAB Jan 2021 1 7 Added OPNs MCX623435AE CDAB and MCX623435AN CDAB Oct 2020 1 6 Updated airf...

Страница 74: ...by customer and perform the necessary testing for the application in order to avoid a default of the application or the product Weaknesses in customer s product designs may affect the quality and rel...

Результаты поиска

Содержание ConnectX-6 Dx MCX623430MS-CDAB

Отзывы:

Похожие инструкции для ConnectX-6 Dx MCX623430MS-CDAB

Бренды по названию

Популярные бренды

Nvidia ConnectX-6 Dx MCX623430MS-CDAB, Руководство пользователя

Результаты поиска

Содержание ConnectX-6 Dx MCX623430MS-CDAB

Отзывы:

Похожие инструкции для ConnectX-6 Dx MCX623430MS-CDAB

Бренды по названию

Популярные бренды