Mellanox Technologies Grid Director 4700 Installation Manual Download Page 21

Page: 21 / 88

Mellanox Grid Director 4700 Installation Manual

cables wasted for inter-switch connectivity. HyperScale combines the best of Hypercube
(price, simplicity, cabling) and Fat-tree (performance).

HyperScale features:



Each 324 node scalable unit is fully independent (failures, upgrades)



Lowest latency of 400 nanoseconds (vs. 500/750 nanoseconds)



There are fewer cables to wire



Ultimate application performance (large non-blocking islands and minimize application
specific tuning/layout).

2.3.2

Switching Topologies

The following figure illustrates the Grid Director 4700 InfiniBand internal switching
topology.

Figure 5: Grid Director 4700 Switching Topology

Summary of Contents for Grid Director 4700

Page 1: ...www mellanox com Mellanox Grid Director 4700 Installation Manual ...

Page 2: ... notice and should not be construed as any commitment by Mellanox Except as permitted by the applicable license no part of this document may be reproduced stored in a retrieval system or transmitted in any form or by any means without the express written consent of Mellanox Names and logos identifying products of Mellanox in this document are registered trademarks or trademarks of Mellanox Voltair...

Page 3: ...3 2 Switching Topologies 21 2 3 3 HyperScale Configurations 23 3 4700 Specifications 24 3 1 4700 Architecture 24 3 2 4700 Fabric Board sFB 4700 Specifications 24 3 3 4700 Line Board sLB 4018 Specifications 24 3 4 4700 Management Board sMB CM Specifications 25 3 5 Chassis Slot Numbering 25 3 6 Torque Settings 26 3 7 Module Configuration Rules 27 3 8 Clearance Requirements 27 3 9 Power Data 27 3 9 1...

Page 4: ...37 3 21 Label s 38 3 22 Gost R Certification 38 3 23 KCC Korea Certification 38 3 24 Oscillator Speeds 38 3 25 Shock and Vibration Test Reports 39 3 25 1 4700 Test for Transportation without a Rack 39 3 26 Export Information 40 3 27 Shipping Restrictions 40 3 28 Center of Gravity Data 41 3 29 Statement of Volatility 41 4 Packing and Unpacking 42 4 1 Package Contents 42 4 2 Unpacking the Grid Direc...

Page 5: ... 7 7 2 Horizontal Fan Unit sFU 40H 74 8 Verifying Installation 75 8 1 Verifying Start up Operations 75 8 2 Verifying Network Connections 75 8 3 Where to Go Next 75 8 4 Troubleshooting 75 8 4 1 Solving Operation Problems 75 8 4 2 Identifying Startup Problems 76 9 System Cabling 77 9 1 Cabling Guide Brackets Installation on Either Sides of the Chassis Line Boards Cables 77 9 1 1 Installing Cabling G...

Page 6: ...r 4700 50 Figure 17 4700 Fully Configured N 1 Redundancy 53 Figure 18 Overview of Assembled Rail Kit 56 Figure 19 Positioning the Insert in Front of the Right Bracket 58 Figure 20 Assembling the Rail Bracket 58 Figure 21 Right Rail Assembly top and Left Rail Assembly bottom 58 Figure 22 U Alignment 59 Figure 23 Securing the 4700 Chassis for Transport Regular Configuration 62 Figure 24 Securing the...

Page 7: ...R Sample Configurations 29 Table 12 Maximum Heat Dissipation 30 Table 13 4700 Airflow 30 Table 14 4700 Physical Characteristics 30 Table 15 HyperScale Physical Characteristics 30 Table 16 Weights and Dimensions 30 Table 17 sLB 4018 Line Board LEDs 32 Table 18 Fabric Board sFB 4700 LEDs 32 Table 19 HyperScale Fabric Board sFB 4700 2X LEDs 33 Table 20 Management Board sMB CM LEDs 33 Table 21 sFU 40H...

Page 8: ...n and assistance go to http www mellanox com content pages php pg support_index Audience The manual is intended primarily for system administrators who are authorized to install a Grid Director 4700 switch It is assumed that the readers are familiar with the InfiniBand technology and terminology Related Documentation For additional information refer to the following documents Grid Director Family ...

Page 9: ...Notes File name pathname opt ufm conf gv cfg Console session code flashClear CR Complete sample line or block Comprises both input and output The code can also be shaded Linux shell prompt The character stands for the Linux shell prompt Mellanox CLI Guest Mode Switch Mellanox CLI Guest Mode Mellanox CLI admin mode Switch Mellanox CLI admin mode String or Strings in or are descriptions of what will...

Page 10: ...48 ports of 40 Gb s per port InfiniBand connectivity the Grid Director 4700 delivers an impressive 51 8 Tb s of non blocking bandwidth with between 100 and 300 nanoseconds of port to port latency Additionally the switch s 648 ports can be divided between two racks for weight distribution and greater ease of cabling The switch s HyperScale architecture provides a unique inter switch link capability...

Page 11: ...Mellanox Grid Director 4700 Installation Manual 11 Figure 1 Grid Director 4700 with Standard Fabric Boards Front Figure 2 Grid Director 4700 with HyperScale Fabric Boards Front ...

Page 12: ...Introduction 12 Figure 3 Grid Director 4700 Rear ...

Page 13: ...19U switch 648 port option can be divided between two racks for weight distribution and greater ease of cabling Unique HyperScale architecture allows scaling to thousands of nodes with a single tier of switches under 400 nanoseconds of latency Available bandwidth up to 51 8 Tbps of non blocking bandwidth Simple and fast device management Fully managed by Mellanox Unified Fabric Manager UFM ...

Page 14: ...iniBand links for connectivity between the Line cards and the internal Switch fabric Supports up to nine vertical fabric boards Supports one or two redundant hot swappable Management Boards sMB for fabric management and chassis management when two Management boards are installed they support failover capabilities in the event of failure of one of the boards Up to 6 power supply units sPSU S with N...

Page 15: ...Band interfaces for connectivity between the Line cards and the Switch fabric Line boards are installed at the rear of the chassis in a horizontal position Fabric Boards The Grid Director 4700 supports up to nine Fabric Boards sFB that are installed at the front of the chassis 324 port chassis The sFB 4700 QDR Fabric Board is installed in a standard 324 port chassis in a vertical position Each sFB...

Page 16: ...ine Board ports InfiniBand products are RoHS 6 compliant Management The Grid Director 4700 provides a command line interface CLI interface for flexible access the switch comprehensive management capabilities Device Management DM Control a device e g a switch or a device cluster Remote management upgrades logs alarms Management Board The Grid Director supports either one or two hot swappable Manage...

Page 17: ...700 includes smart device management that provides a simple interface for deploying troubleshooting maintaining and upgrading the switch With a simple to use CLI interface routine tasks such as monitoring the switch operation or upgrading software and firmware are made simple Chapter 2 ...

Page 18: ...des 36 internal ports Up to 9 HyperScale Fabric Board sFB 4700 2X Includes 36 internal and 12 external CXP 12X ports Up to 9 Power Supply sPSU S AC DC Power supply Up to 6 Vertical Fan Unit sFU 40V Vertical Fan Unit 1 Horizontal Fan Unit sFU 40H Horizontal Fan Unit 1 Management Board sMB CM Chassis Management Board Up to 2 Cable Management Right N A Right cabling guide brackets for cable managemen...

Page 19: ...idth 51 8 Tbps Line Board Slots 18 Fabric Board Slots 9 Max of InfiniBand Ports 324 40 Gbps InfiniBand Ports Optional 648 40 Gbps InfiniBand Ports Interface Type QSFP 40 Gbps QDR QSFP ports with support for optical adapters and cables Redundant Components Power Supplies sPSU S Management Boards sMB CM Embedded Chassis Management CLI Yes 2 2 4700 Configuration and Part Numbers The following table d...

Page 20: ...d to as Fat Tree or Constant Bisectional Bandwidth CBB networks to construct large node count non blocking configurations A Clos network is a switch topology in which integrated non blocking switch chips crossbars with a relatively low number of ports are used to build a non blocking switch topology supporting a much larger number of endpoints The Grid Director 4700 features an internal Clos netwo...

Page 21: ...de scalable unit is fully independent failures upgrades Lowest latency of 400 nanoseconds vs 500 750 nanoseconds There are fewer cables to wire Ultimate application performance large non blocking islands and minimize application specific tuning layout 2 3 2 Switching Topologies The following figure illustrates the Grid Director 4700 InfiniBand internal switching topology Figure 5 Grid Director 470...

Page 22: ...ighteen symmetrical Line Boards sLB and nine interconnect Fabric Boards sFB Each Line Board sLB 4018 includes one 36 port 40Gb s InfiniBand switch chip Each Fabric Board sFB 4700 includes one 36 port 40Gb s InfiniBand switch chip Each HyperScale Fabric Board sFB 4700 2X includes two 36 port 40Gb s InfiniBand switch chips The LEDs on the Fabric Board modules indicate proper connectivity and traffic...

Page 23: ...700 chassis can be connected in a mesh This configuration uses 4700 stacking fabric boards and 4700 switches Ideal for over 648 node counts CXP CXP cables are used between switches HyperScale Edge Configuration Figure 8 HyperScale Edge Configuration Combines 4700 and 4036 switches CXP 3QSFP cables are used between switches Ideal for 324 648 node counts Chapter 3 ...

Page 24: ...vel chassis management purposes EEPROMs to store the switch ASIC ERP Embedded RISC processor code Indicators Physical connectivity and logical connectivity LEDs per Line Board link port power Rdy and Info LED On board temperature monitoring reported to Management system Thermal shutdown for over temperature sensing 3 3 4700 Line Board sLB 4018 Specifications Up to 18 sLB 4018 hot swappable Line Bo...

Page 25: ...gement Boards each including on board CPU LEDs Refer to LED Indicators on page 31 Connectors 10 100 1000 Ethernet RJ 45 EIA TIA 232 Console RJ 45 Reset button Resets the Management Board or the entire chassis 3 5 Chassis Slot Numbering Slot numbering at the front of the chassis for Management Board and Fabric Board is from left to right indication on the horizontal fan unit sFU 40H Figure 9 sFU 40...

Page 26: ...s refer to System Cabling on page 77 3 6 Torque Settings Table 5 Torque Settings for Module Assembly Assembly Torque in lb sLB 4018 Line Board to chassis 2 or use manual screwdriver sMB Management Board to chassis 2 or use manual screwdriver Rail kit part A to part B 10 Rail to chassis 10 ...

Page 27: ...ce Front of the rack 36 90 cm for technician access and maintenance Rear side of the rack 24 60 cm for human accessibility Sides of the rack No limits Space below the chassis No limits Space on top of the chassis No limits 3 9 Power Data 3 9 1 Type of Power Cord on the Power Supply Mellanox provides a 15 Amp cord 2 meter jumper with a universal plug for a power distribution unit PDU for each power...

Page 28: ... Factor Power supply efficiency 89 typical at 230Vac full load rated power 85 typical at 115Vac full load rated power A measure of the ability of the power supply to convert source AC power into usable DC power 3 9 4 Power Data per Module Table 9 Power Data by Module Module Name Module Watts Module Configured Qty Total Watts Fan Tray sFU 40H 380 1 380 Fan Tray sFU 40V 250 1 250 Management Board sM...

Page 29: ...r Consumption Watt sLB 4018 18 QSFP ports per sLB 4018 sLB 4018 110 0 Add 1 5 Watts for each connected QSFP optical module Spare Parts for the Mellanox Grid Director 4700 Description Module Name DC Power Consumption Watt sFU 40H Horizontal Fan Unit sFU 40H 380 0 sFU 40V Vertical Fan Unit sFU 40V 250 0 3 10 Calculating Heat Output in BTUs To compute the approximate BTUs for a given configuration fi...

Page 30: ...M Cubic Feet per Minute 3 13 Physical Characteristics Table 14 4700 Physical Characteristics 4700 Features Characteristics 19 rack mountable chassis 19U height Dimensions Width 19 440 mm Depth 22 75 578 mm Height 25 6 650 mm Weight 128 to 163 lbs 58 to 74 kg depending on configuration Minimum rack depth required for the rail kit N A Maximum rack depth required for the rail kit N A Table 15 HyperSc...

Page 31: ...17 13 1 57 6 4 51 5 40 9 5 3 5 20 2 15 7 3 74 7 7 Blank sFB 66 32 4 1 9 30 12 6 1 57 4 2 78 40 7 2 6 30 7 15 75 2 75 5 7 Blank sLB 43 3 4 0 26 17 1 18 1 57 0 57 63 8 3 5 0 36 24 8 3 14 1 38 0 8 Blank sMB 4 22 4 0 12 1 57 8 66 1 57 0 26 63 8 3 5 0 22 24 8 3 14 1 38 0 48 Blank PS 4 15 5 0 09 1 57 6 2 0 12 63 8 3 5 0 19 24 8 3 14 1 38 0 42 Rail Kit 53 2 14 5 4 3 9 21 5 7 1 7 19 84 62 14 7 5 8 24 5 5 ...

Page 32: ...d Blinking Chassis attention required Card Ready Green OFF Card not Ready ON Card ready Blinking Slowly Card loading Blinking Fast Card problem 3 15 2 Fabric Board sFB 4700 LEDs Table 18 Fabric Board sFB 4700 LEDs Indication LED Description Remarks Link 18 Amber LEDs 18 Green LEDs HW only Each pair of LEDs indicates the state of ports of each Line Board The two LEDs in each pair are as follows Gre...

Page 33: ...6 Amber LEDs 36 Green LEDs HW only The two LEDs in each pair are as follows Green ON indicates physical link up Amber ON indicates logical link up The LED states are as follows OFF no link detected ON the port has a link Blinking Green indicates errors Amber indicates data activity Indicates the state of the 36 external QSFP port connectivity Power Green HW only ON indicates that all board voltage...

Page 34: ...ed PSU OK Green OFF N A ON System Power OK Blinking System Power problem AC DC power supply fault indication Indicates that at least one of the main AC DC power supplies has an AC or DC fault Eth 1 and 2 Two LEDs Link Green and Activity Orange Eth 1 and 2 3 15 5 sFU 40H Horizontal Fan Unit LEDs Table 21 sFU 40H Horizontal Fan Unit LEDs Indication LED Description Remarks Temp Orange ON Indicates an...

Page 35: ...n Time between Failures MTBF for individual switch components is shown in the following tables These figures indicate the mean elapsed time until a single component fails in the module whether or not that failure impacts the operation of the switch Note that in many cases redundant modules can be configured which dramatically increases the overall system Mean Time between Critical Failures MTBCF M...

Page 36: ...ns cMETus UL 60950 1 07 Second Ed CSA C22 2 No 60950 1 07 Second Ed cTUVus UL60950 1 07 Second Ed S Mark Argentina CB Certificate and report according to IEC 60950 1 Second Ed KCC Korea Gost R CE according to EN 60950 1 06 Second Ed S Mark EMC Certifications FCC Part 15 Subpart B Class A and testing to ANSI 63 4 Industry Canada ICES 003 CAN CSA CEI IEC CISPR 22 02 CE EN55022 1998 A1 2000 A2 2003 E...

Page 37: ...R 22 04 KCC Korea Restricted Hazardous Substances Certification RoHS 6 InfiniBand Certification IBTA 1 2 3 20 Declarations 3 20 1 Declaration of Conformity Mellanox products comply with the Radio Telecommunications Terminal Equipment Directive 99 5 EEC the EMC Directive 2004 108 EC and of the Low Voltage Directive 2006 95 EC 3 20 2 Hazardous Substances RoHS 6 Compliance Declaration Mellanox produc...

Page 38: ...lanox products comply with the Korean KCC regulations 3 24 Oscillator Speeds Oscillator speeds are shown in the following table Table 26 Oscillator Speeds Mellanox PN Description Frequency Card 141C00005 CRYSTAL PECL156 25MHZ JOE75 SMD H 1 4MM 156 25 MHz sFB 4700 sFB 4700 2X sLB 4018 OSC 00005 OSCILLATOR PECL 100MHZ 7 X 5 X 1 9MM 50PPM 100MHz sFB 4700 sFB 4700 2X sLB 4018 sMB CM 141C71200 OSCILLAT...

Page 39: ...tor 4700 to ship in a rack by carefully following the rail kit installation instructions as detailed in Assembling and Mounting the Rails The Grid Director 4700 is susceptible to shock and vibration if the rail kit installation instructions are not followed precisely The following tables detail the shock and vibration test reports compliant with shipping in a rack 3 25 1 4700 Test for Transportati...

Page 40: ...culation measured in MTOPS millions of theoretical operations per second The 460EX 460GT contain a 440H6 core not a 460 core it is called 460 to distinguish it as a 90 nm core The CTP in MTOPS for the PPC440 core 17 18 F where F is the CPU clock rate in MHz Refer to the following table accordingly Table 29 Export Information CTP value for the PPC440 core 460EX 460GT 600MHz 567 MTOPS CTP value for ...

Page 41: ...rner of the chassis Figure 12 Center of Gravity Data Center of gravity for minimum typical and maximum configurations are shown in the following table Table 30 Center of Gravity Data 4700 Width X mm Depth Y mm Height Z mm Empty Chassis 220 250 345 Basic Configuration 220 270 330 Full Configuration 220 300 320 3 29 Statement of Volatility Refer to the Grid Director 4700 Statement Of Volatility docu...

Page 42: ... supply units sPSU S The Grid Director 4700 chassis is shipped with a horizontal fan unit sFU 40H and a vertical fan unit sFU 40V installed at the front of the chassis and two power supplies at the rear of the chassis Additional components include Rail kit for rack installation Cabling guide brackets One power cord per power supply One console cable A grounding kit Mellanox Getting Started Short G...

Page 43: ...llation Manual 43 Part Number Description 1 Grid Director 4700 2 Top Foam 3 Cable Management 4 Mounting Rails Left and Right 5 Foam Plate 6 Carton Plate placed at the top of the rear of the chassis to protect the opening 7 Bottom Foam ...

Page 44: ... 1 Carefully cut the packing straps and open the carton box 2 Remove the box from around the chassis by sliding it upwards as shown The Grid Director 4700 chassis in its partially unpacked state is shown as follows The upper packing foam covering the Grid Director 4700 contains the following components Rail kit Cabling guide brackets cable management ...

Page 45: ... Packing the Grid Director 4700 Switch If you need to move the Grid Director 4700 switch to a different physical location pack the equipment according to the following procedure To pack the Grid Director 4700 chassis 1 Insert the foam grip on the pallet 2 Position the chassis on the foam Verify that the two sFU 40H horizontal fan unit handles at front of chassis point to the two niches in the bott...

Page 46: ...Packing and Unpacking 46 Figure 13 4700 Switch Packed Chapter 5 ...

Page 47: ...the system panels so that they are accessible and easy to reach system panels inaccessible and difficult to maintain Plan your site configuration and prepare your site before installation to ensure normal operation and avoid unnecessary maintenance Refer to Environmental Specifications for the operational and non operational environmental site requirements for the Grid Director 4700 The system can...

Page 48: ...tead of 19U 5 4 Clearance Requirements You should be certain to provide sufficient clearance around the racks to install and maintain the Grid Director 4700 Clearance specifications are detailed in Chapter 3 on page 27 5 5 Site Environment Specifications The system can continue to operate within specified environmental ranges however a measurement that approaches the minimum or maximum of a range ...

Page 49: ...The system must be permanently grounded using the grounding stud The grounding kit includes the following items A 10 32 5 8 screw 3 spring washers 2 nuts 10 Ground label Safety warning label This grounding procedure requires a 6 AWG Grounding Cable Figure 14 4700 Grounding Connection Figure 15 Ground Warning Label ...

Page 50: ...ctor 4700 provides two grounding options as shown For convenience grounding can be performed from any of these locations Figure 16 Grounding Holes for 4700 Install the grounding stud before you mount the Grid Director 4700 chassis onto the rack ...

Page 51: ... nut 10 4 Add another spring washer 5 Insert the 6 AWG grounding cable Connect the other end of the grounding cable to a grounding point at your site 6 Add another spring washer Make sure that the grounding cable is tightly secured between the two spring washers 7 Secure everything with a grounding nut 10 8 Apply the Ground label under the grounding nut 9 Apply the Safety Warning label on the side...

Page 52: ...rector 4700 includes hot swappable power supply units sPSU S with N 1 or N N redundancy A non redundant power configuration includes the minimum number of power supplies required by the system for operation N N redundancy implies that each non redundant power supply is fully backed up by a redundant power supply which is routed to a different power source This provides for full power backup in cas...

Page 53: ...QDR N 1 N N Redundancy Configuration Modules Included Non redundant power supplies N 1 N N or N M 18 ports QSFP copper 9 sFB 4700 1 sLB 4018 1 sFU 40H 1 sFU 40V 1 sMB CM 2 sPSU S 2 3 2 1 4 2 2 162 ports QSFP copper 9 sFB 4700 9 sLB 4018 1 sFU 40H 1 sFU 40V 1 sMB CM 2 sPSU S 2 3 2 1 4 2 2 324 ports QSFP copper 9 sFB 4700 18 sLB 4018 1 sFU 40H 1 sFU 40V 2 sMB CM 3 sPSU S 3 4 3 1 6 3 3 Chapter 6 ...

Page 54: ... on the rail in a rack Installing the cable management 2 HyperScale chassis installation up to 648 port configuration Assembling and mounting the rail Remove the standard 19 brackets keep the screws for installation of the 19 brackets Install the 19 long brackets part of P N KIT 000222 Slide and mount the chassis into the rack Installing the cable management Install the HyperScale cabling guide br...

Page 55: ...t kit includes the following items Left bracket left rail Right bracket right rail 2 inserts with elongated holes same for both left and right rails 8 32 screws 5 8 qty 2 x 8 8 32 screws 3 8 qty 2 x 3 to be used in case of chassis transportation in a rack 8 32 nuts qty 2 x 11 M4 flat washers qty 2 x 11 Spring washers qty 2 x 8 Tooth washers qty 2 x 3 Clip nuts are not provided 6 2 2 Rail Kit Assem...

Page 56: ...in front of the bracket 2 For each of left and right rails take a bracket and attach an insert the two inserts are identical At this stage position the two parts to fit the distance between the rack rails of each side Make sure that you are using the right bracket for the right rail right side when facing the front of the rack and the left bracket for the left rail left side when facing the front ...

Page 57: ...ly Procedure This section presents the procedure for assembling and mounting the rail kit Each step is accompanied by a figure showing the components as they appear aligned and then assembled one by one In addition eight screw sets are included comprising one screw one nut and two washers To assemble the rail 1 Use 8 32 5 8 screws and flat washers and spring provided with the kit to assemble the r...

Page 58: ...gure 19 Positioning the Insert in Front of the Right Bracket Figure 20 Assembling the Rail Bracket 2 Select the round holes in the bracket 3 Assemble the rail with flat washers spring washers and nuts 4 Do not tighten the screws at this time Figure 21 Right Rail Assembly top and Left Rail Assembly bottom ...

Page 59: ...ck Note you can install any equipment adjacent to the rail kit as long as you align the rail kit with the U mark Figure 22 U Alignment 2 Attach the front side of the rail to the rack s front vertical side rail with 2 screws and the rear side of the rail to the rack s rear vertical side rail with 2 screws The following figure shows the left rail completely assembled and what it looks like when it i...

Page 60: ...ction provides instructions on how to mount the 4700 in a rack Screws and clip nuts to connect the chassis to the rack and rail kit to rack are not supplied by Mellanox Make sure that the grounding stud is installed before performing the steps below see Chassis Grounding on page 48 Prerequisites Rail kit 19 bracket attached to the chassis To install the 4700 chassis 1 Lift the chassis using a hydr...

Page 61: ...k rail using 16 screws 8 for each side and adequate clip nuts as shown 5 To stabilize the chassis for transport secure the 4700 chassis onto the rail by using three 8 32 screws 3 8 tooth washers and flat washers insert them into the oval holes and the threaded holes in the chassis side walls see Figure 6 10 ...

Page 62: ... 4700 Chassis for Transport Regular Configuration Figure 24 Securing the 4700 Chassis for Transport HyperScale Configuration 6 4 Mounting the HyperScale Chassis in a Rack The installation kit includes Rack mount bracket for recessed installation supporting cabling from both sides HyperScale cabling guide bracket for CXP cables Screws you will need to save the screws from the original brackets use ...

Page 63: ...sis in the rack as described in Assembling and Mounting the Rails Note it is important to leave enough space in the rack for the second 4700 X2 chassis 5 In the back of the rack install the regular cable management as described in Chapter 9 6 Secure the chassis onto the rack 1 Align the holes in the front of the chassis with the mounting holes in the equipment rack s vertical front rail 2 Secure t...

Page 64: ...properly inserted 3 Connect the power supply cords to the AC connectors located at the rear of the chassis All LED indicators on the Fabric Board and Line Board briefly light up and then turn off just after power is turned on This is a normal part of the power on initialization sequence After initialization is completed the link LED indicators on the Fabric Board and Line Board are in a state repr...

Page 65: ... sFU 40V vertical fan unit The rear panel of a Grid Director 4700 includes 18 slots which can be filled with line boards 324 ports Two slots for management boards sMB Six bays for power supply modules the power supplies can be installed in any of the bays without restriction See Mellanox Grid Director 4700 Overview on page 10 for illustrations of front and rear panels of a fully populated 4700 cha...

Page 66: ...ne Board Pull the ejector outward on both ends of the Line Board Figure 27 Using Ejectors to remove a Line Board It is recommended to insert boards from top to bottom starting from slot 1 down to 18 Fabric Board Press the ejector inward at the top and at the bottom of the Fabric Board Figure 28 Using Ejectors to Install a Fabric Board ...

Page 67: ... the system Refer to Torque Settings for torque specifications When inserting the screws make sure that they are inserted directly into their groove and that they slide in easily It is recommended to start the insertion manually to ensure that the insertion is correct When using an electric screwdriver to fasten the board screws in place do not use more than Torque 3 force as this could damage the...

Page 68: ...the board into the side guide rails Slowly slide the board into the chassis until the module guide pins engage the chassis The ejectors begin to engage the chassis and start moving When sliding in the board push from the center of the board front panel and not by pushing the ejectors Make sure that the front panel of the Fabric board is completely vertical is pushed all the way in and is leveled w...

Page 69: ...red from 1 18 from left to right Figure 30 Front Panel of the sLB 4018 Line Board For more information see LED Indicators on page 31 and System Cabling on page 77 When installing the line boards start at the bottom of the chassis and work your way up To insert a Line Board into the chassis 1 Carefully seat the board into the side guide rails Slowly slide the board into the chassis until the ejecto...

Page 70: ...or to a fully open position 2 Carefully seat the board into the side guide rails Slowly slide the board into the chassis until the module guide engage the chassis then the ejectors begin to engage on the chassis edge 3 Push the ejector inwards 4 Tighten the two security screws 5 Verify the following LED conditions Power LED ON Info LED OFF To extract a Management Board from the chassis 1 Disconnec...

Page 71: ...l power supplies must be powered simultaneously If not powered simultaneously the system may not start properly To extract a power supply from the chassis 1 Disconnect power cable 2 Loosen the screws 3 Extract the power supply by pulling the handles This product may contain sealed lead acid nickel cadmium nickel metal hydride lithium or lithium ion battery The battery must be recycled or disposed ...

Page 72: ... to customers for the return recycling and treatment of batteries and accumulators Customer participation is important to minimize any potential effects of batteries and accumulators on the environment and human health due to the potential presence of hazardous substances 7 7 Fan Units sFUs The 4700 chassis is pre assembled at the factory with the fan units installed at the front of the chassis Th...

Page 73: ... the management boards It includes 5 fans with fan speed control fan speed is dynamically controlled as a function of temperature Figure 33 sFU 40V Vertical Fan Unit Front Panel Never remove the sFU 40V and sFU 40H at the same time at least one fan unit must be installed at all times To replace the sFU 40V 1 Loosen the four screws that secure the sFU 40V to the chassis 2 Remove the sFU 40V by slid...

Page 74: ...namically controlled as a function of temperature LED indications provide an onsite report regarding the operating conditions of the air cooling system modules and provide an alarm indicator in the event of a high temperature condition To replace the sFU 40H horizontal fan unit 1 Loosen the four screws that secure the sFU 40H horizontal fan unit to the chassis 2 Remove the sFU 40H horizontal fan u...

Page 75: ...fying Network Connections Verifying the network connections consists of making sure that the following network ports are operational InfiniBand 10 100 1G Ethernet management 8 3 Where to Go Next Once you have verified that the 4700 hardware is properly installed it is ready for optional system configuration To configure the system software refer to Mellanox Grid Director 4000 Family User Manual 8 ...

Page 76: ...are faulty contact a customer service representative If the 4700 fan does not function properly at initial startup there are no installation adjustments that you can make contact a customer service representative 2 Check the power supply LEDs on the rear panel The power LEDs illuminate immediately upon the connection of power to the 4700 If the LEDs are not on the power supplies may need to be rep...

Page 77: ...s on the overall cluster configuration and requires proper planning to produce orderly and maintainable cabling HyperScale cable guide requires an additional 1U of rack space making the total height 20U instead of 19U The 4700 space in the Rack should be free from obstructions such as power strips The rack mounted cabling guide bracket should be installed with its bend facing out otherwise it will...

Page 78: ...00 rack s rear rail To install the rack gripping cabling guide brackets 1 Install the cabling guide brackets at the rear of the rack itself on the rail and not on the 4700 chassis It is not important from the cable management perspective where on the rack the 4700 is installed 2 Install the cabling guide brackets in parallel with the 4700 chassis so that each Line Board lines up with its correspon...

Page 79: ... an InfiniBand cable to an sLB 4018 Line Board 1 Route the cable to the nearest cabling guide bracket hanger position 2 Position the InfiniBand cable connector with the pulling tab at top and push the connector in place 3 Verify that the cable is inserted properly into the connector by gently tugging the cable verify that the Link State green colored LED lights and does not flash indicating a good...

Page 80: ...dius The cabling guide bracket is part of a kit for setting up a 648 port configuration Selecting the optimal InfiniBand cable length depends on the overall cluster configuration and requires proper planning to produce orderly and maintainable cabling The 4700 space in the Rack should be free from obstructions such as power strips It is recommended that the rack rails that overlap with the 19U of ...

Page 81: ...guide bracket for HyperScale fabric boards Install the cabling guide bracket at the front of the rack under the rail kit on the rack itself between the two 4700 using four screws to secure the bracket in place Figure 36 Cable Management for Switches with HyperScale Fabric Boards 9 2 2 Connecting the InfiniBand Cable between Fabric Boards To connect a cable between Fabric Boards 1 In a Fabric Board...

Page 82: ...y that the Link State green colored LED lights and does not flash indicating a good physical connection 5 Gently fasten all cables in place with Velcro tie or tie wraps Avoid using tie wrap guns or similar tools do not fasten the InfiniBand cable too tightly as this may cause irreversible damage to the cables Tie wraps and zip ties can damage the cables Velcro tie is the recommended option Do not ...

Page 83: ...ines It is recommended to dress the cables connected to the InfiniBand connectors of each sLB into two groups left and right and fasten them down gently after connectivity has been verified The cables from the same half of the Line Boards are bundled on the corresponding hangers and will be routed down along the Rack side wall Cables should be arranged so that each Line board s cables don t interf...

Page 84: ... Velcro tie neatly placing the cables on the outside of cable guide bracket rails 2 Remove a Line Board as described in Inserting Extracting Boards 3 Insert a new Line Board as described in Inserting Extracting Boards 4 Replace the cable connections 9 5 Connecting to the Management Ports The Mellanox 4700 has management ports located at top back of the 4700 This includes two interfaces a serial in...

Page 85: ...end of the management cable with the single RJ 45 connector to the Management port on the 4700 2 Connect the end of the cable with the DB 9 serial connector to a PC or a console The PC console must support VT100 terminal emulation and should be equipped with terminal emulation software such as HyperTerminal or ProComm Plus 3 Make sure that your PC console terminal emulation software is configured ...

Page 86: ...er Fibre cable with two separate modules Active optic cable where cable and optic adaptors are delivered as a single unit permanently attached cable Data rate 2 5 Gbps SDR 5 Gbps DDR and up to 10 Gbps QDR per channel QSFP optic modules use standard MPO parallel 12 fibre jacketed ribbon optical connector Provide manageability discovery and identification capability such as vendor identification mod...

Page 87: ...cables You can install any combination of the cables in any of the QSFP cages A 2 IB Port Cable Specifications The following table details the cabling specifications for the 4036E 4X InfiniBand ports Table 33 4X InfiniBand Port Cabling Specifications Type Maximum Cable Length 4XQDR AWG Length ft Length m Mellanox P N 26 22 96 7 CBL 00158 26 16 4 5 CBL 00086 30 9 84 3 CBL 00185 30 6 57 2 CBL 00084 ...

Page 88: ...through UTP cables to connect the 10 100 1000 Fast Ethernet ports to external switches routers that relay management traffic Use modular RJ 45 cross connect cables to connect to end systems The following figures show straight through cables and cross connect cables Figure 39 Straight through Cables Figure 40 Cross connect Cables The 10 100 1000 Fast Ethernet port supports a RJ 45 connector ...

Search results

Summary of Contents for Grid Director 4700

Reviews:

Related manuals for Grid Director 4700

Brands by name

Popular brands

Mellanox Technologies Grid Director 4700, Installation Manual

Search results

Summary of Contents for Grid Director 4700

Reviews:

Related manuals for Grid Director 4700

Brands by name

Popular brands