manualshive.com logo in svg

Oracle InfiniBand Switch IS2-46, Installation Manual, Page: 1 / 110

Share
Download
Oracle InfiniBand Switch IS2-46 Installation Manual Download Page 1

Oracle

®

 InfiniBand Switch IS2-46

Installation Guide

Part No: E65868-01

September 2016

Summary of Contents for InfiniBand Switch IS2-46

Page 1: ...Oracle InfiniBand Switch IS2 46 Installation Guide Part No E65868 01 September 2016...

Page 2: ......

Page 3: ...angerous applications including applications that may create a risk of personal injury If you use this software or hardware in dangerous applications then you shall be responsible to take all appropri...

Page 4: ...risque notamment dans des applications pouvant causer des dommages corporels Si vous utilisez ce logiciel ou mat riel dans le cadre d applications dangereuses il est de votre responsabilit de prendre...

Page 5: ...ifications 17 Electrical Specifications 18 Environmental Specifications 18 Airflow Precautions 19 Rack Compatibility 20 Rack Precautions 20 Preparing for Installation 23 Shipping Kit 24 Record the MAC...

Page 6: ...e the Power Cords and SER MGT Cable 67 Connect the SER MGT Cable 71 Connect a Serial Management Device to the SER MGT Port 74 Connect the Power Cords 74 Install the Air Duct Assembly 76 Power On the S...

Page 7: ...Contents Oracle Fabric OS Configuration Wizard Parameters 98 Glossary 99 Index 107 7...

Page 8: ...8 Oracle InfiniBand Switch IS2 46 Installation Guide September 2016...

Page 9: ...nd authorized service providers Required knowledge Advanced experience installing network hardware Product Documentation Library Documentation and resources for this product and related products are a...

Page 10: ...10 Oracle InfiniBand Switch IS2 46 Installation Guide September 2016...

Page 11: ...connection to the SER MGT port and it is used for switch administration A temporary SER MGT connection is where the SER MGT port is used only for switch installation and configuration of network mana...

Page 12: ...58 Connect the SFP Cables on page 60 Secure Cables to the CMB on page 62 14 Retrieve the power cords and SER MGT cable Retrieve the Power Cords and SER MGT Cable on page 67 15 Connect the serial mana...

Page 13: ...ds The switch features up to 38 4x IB ports supporting EDR speeds Additionally 10 10GbE ports and 24 1GBASE T ports provide both network and gateway capabilities Separate dedicated and optimized on bo...

Page 14: ...r supply end of the switch This end of the switch is installed to the cold aisle No Description 1 Power supplies 2 Power cord receptacles 3 Fan modules 4 RFID tag 5 SER MGT port 6 SER MGT module SMM w...

Page 15: ...ack housing 2 SFP connectors Ports 1 to 8 left 10GbE Ethernet overlay network IB switch chip Ports 1 and 2 right 10GbE network management uplinks Ethernet switch chip Note The two right most connector...

Page 16: ...16 Oracle InfiniBand Switch IS2 46 Installation Guide September 2016...

Page 17: ...ge 20 Rack Precautions on page 20 Related Information Understanding the Switch on page 11 Preparing for Installation on page 23 Installing the Switch on page 29 Connecting Cables on page 49 Powering O...

Page 18: ...g power supplies Related Information Physical Specifications on page 17 Environmental Specifications on page 18 Airflow Precautions on page 19 Rack Compatibility on page 20 Rack Precautions on page 20...

Page 19: ...the switch Air flows through the power supplies from the sides of the switch to out the front Follow these guidelines to ensure unrestricted airflow in the switch Install the switch so that the front...

Page 20: ...the front of the switch Related Information Physical Specifications on page 17 Electrical Specifications on page 18 Environmental Specifications on page 18 Airflow Precautions on page 19 Rack Precauti...

Page 21: ...w the equipment nameplate power ratings and consider the effect that circuit overloading might have on overcurrent protection and supply wiring Caution Reliable Grounding Maintain reliable grounding o...

Page 22: ...22 Oracle InfiniBand Switch IS2 46 Installation Guide September 2016...

Page 23: ...the SP and SCP Through DHCP on page 26 Handling Precautions on page 26 ESD Precautions on page 27 Installation Tools on page 27 Related Information Understanding the Switch on page 11 Confirming Spec...

Page 24: ...4 C channel brackets shown mounted 5 Fasteners 6 Power cords 7 Documentation 8 Cable management extenders 9 Long rails 10 Cable management cover 11 Cable management bracket CMB 12 Attachment brackets...

Page 25: ...number to a document intended for this purpose 5 In the Network Interfaces table of the sheet find the primary MAC ID This is the MAC ID of the SP within the switch 6 Record the SP MAC ID to the docu...

Page 26: ...recorded Record this IP address and host name for future reference 3 Configure the DHCP server to assign a predetermined IP address and host name for the SCP MAC ID you recorded Record this IP addres...

Page 27: ...page 27 Installation Tools on page 27 ESD Precautions When installing the switch chassis follow antistatic precautions Use an antistatic mat as a work surface Wear an antistatic wrist strap that is a...

Page 28: ...aper clip PrizmMT ferrule cleaner USConec part number 16899 Antistatic mat Flashlight Gloves Related Information Shipping Kit on page 24 Record the MAC IDs on page 25 Assign IP Addresses to the SP and...

Page 29: ...on page 39 Install the Switch on page 40 Install the CMB on page 45 Related Information Understanding the Switch on page 11 Confirming Specifications on page 17 Preparing for Installation on page 23 C...

Page 30: ...age 30 Assemble the Cable Management Extenders on page 32 Prepare the Air Duct Assembly on page 33 Install the Cage Nuts on page 35 Install the Rackmount Hardware on page 37 Route the Power Cords on p...

Page 31: ...4 Assemble the cable management extenders See Assemble the Cable Management Extenders on page 32 Related Information Stabilize the Rack on page 29 Assemble the Cable Management Extenders on page 32 Pr...

Page 32: ...bracket is opposite the flange on the extender The open end of the tab is toward the flange The flat end of the tab is toward the rear of the extender 3 Place the attachment plate on the flange side o...

Page 33: ...ts on page 35 Install the Rackmount Hardware on page 37 Route the Power Cords on page 38 Route the SER MGT Cable on page 39 Install the Switch on page 40 Install the CMB on page 45 Prepare the Air Duc...

Page 34: ...e Cage Nuts on page 35 Related Information Stabilize the Rack on page 29 Install the Switch Brackets on page 30 Assemble the Cable Management Extenders on page 32 Install the Cage Nuts on page 35 Inst...

Page 35: ...lation Task Overview on page 11 2 Determine where you are installing the switch 3 Mark this location on each rack post with masking tape Counting the number of rack units from the bottom of the rack t...

Page 36: ...as a guide catch the inner flange and pry it into the square hole d Pull the cage nut installation tool or flat blade screwdriver from the flange e Repeat Step 4a to Step 4d for each cage nut 5 Instal...

Page 37: ...u must perform in conjunction with this task See Installation Task Overview on page 11 2 Place the long rail at the mounting location on the post at the rear of the rack 3 Butt the flange of the cable...

Page 38: ...in conjunction with this task See Installation Task Overview on page 11 2 Route the female end of the power cords up from the power distribution panel or receptacles at the rear of the rack 3 Route th...

Page 39: ...k to route the cable 1 Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task See Installation Task Overview on page 11 2 Route the SER MGT cab...

Page 40: ...t Extenders on page 32 Prepare the Air Duct Assembly on page 33 Install the Cage Nuts on page 35 Install the Rackmount Hardware on page 37 Route the Power Cords on page 38 Install the Switch on page 4...

Page 41: ...f the long rails slide into the C channel brackets at the rear of the switch chassis 3 Using a No 2 Philips screwdriver hold the front of the switch chassis to the front rack posts with two M6x12 scre...

Page 42: ...Install the Switch Note Do not fully tighten the screws at this time 42 Oracle InfiniBand Switch IS2 46 Installation Guide September 2016...

Page 43: ...Install the Switch 4 Slide the snap bracket under the loose screws with the tabs pointing into the rack Installing the Switch 43...

Page 44: ...Information Stabilize the Rack on page 29 Install the Switch Brackets on page 30 Assemble the Cable Management Extenders on page 32 Prepare the Air Duct Assembly on page 33 Install the Cage Nuts on pa...

Page 45: ...and subsequent installation tasks that you must perform in conjunction with this task See Installation Task Overview on page 11 2 If installed loosen the two captive thumbscrews that secure the cover...

Page 46: ...ble assembly is required route the cables to the switch See Route the Cables on page 56 Related Information Stabilize the Rack on page 29 Install the Switch Brackets on page 30 Assemble the Cable Mana...

Page 47: ...Install the CMB Install the Switch on page 40 Installing the Switch 47...

Page 48: ...48 Oracle InfiniBand Switch IS2 46 Installation Guide September 2016...

Page 49: ...he CMB Secure Cables to the CMB on page 62 Related Information Understanding the Switch on page 11 Confirming Specifications on page 17 Preparing for Installation on page 23 Installing the Switch on p...

Page 50: ...ables or optical fiber PrizmMT ports These 30 ports provide switched 100 Gbps EDR IB functionality with special optical fiber cables Related Information Standard MT and PrizmMT Cables on page 50 Cable...

Page 51: ...ack Black Pin or Hole Narrow Note Some supported cables have a PrizmMT connector on one end and a standard MT connector on the other Pay close attention when connecting this cable to identify which co...

Page 52: ...on page 62 Cable Precautions To prevent data cable damage you must follow these cautions Do not uncoil the cable as a kink might occur Hold the coil closed as you unroll the cable pausing to allow th...

Page 53: ...le surface Do not cinch the cable with hard fasteners or cable ties Use soft hook and loop fastener for bundling and securing cables Do not drag the cable or its connectors over any surface Carry the...

Page 54: ...nsceivers until instructed to do so 1 Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task See Installation Task Overview on page 11 2 Remove...

Page 55: ...Step 3 for the other end of the cable 8 Repeat from Step 2 for all SFP cables to be assembled 9 Route the data cables See Route the Cables on page 56 Related Information Cable Requirements on page 49...

Page 56: ...dy to accept fabric or network cabling 4 For each node or host begin data cabling a Attach the cable to the appropriate connector of the node or host b Route and bundle the data cable through the topo...

Page 57: ...to where it will connect to the switch The tab is up for the upper row of receptacles and the tab is down for the lower row of receptacles 3 Insert the connector into the receptacle until it clicks se...

Page 58: ...ed housing PrizmMT cables 1 Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task See Installation Task Overview on page 11 2 Remove any fille...

Page 59: ...f the PrizmMT cleaner b Insert the connector into the adapter c Press the connector against the adapter thereby pumping the cleaner and advancing the cleaning cloth inside d Remove the connector from...

Page 60: ...es on page 57 Connect the SFP Cables on page 60 Secure Cables to the CMB on page 62 Connect the SFP Cables 1 Identify the prerequisite and subsequent installation tasks that you must perform in conjun...

Page 61: ...he center of the SFP receptacle If the connector stops or binds after about 1 4 in 5 mm travel back out and repeat from Step 4 If the connector stops or binds with about 1 8 in 2 mm still to go back o...

Page 62: ...on page 54 Route the Cables on page 56 Connect the RJ 45 Ethernet Cables on page 57 Connect the PrizmMT Cables on page 58 Secure Cables to the CMB on page 62 Secure Cables to the CMB 1 Identify the p...

Page 63: ...Secure Cables to the CMB 2 While connecting the RJ 45 SFP and PrizmMT cables to the switch lay the cables into an available slot on the CMB Connecting Cables 63...

Page 64: ...t use cable zip ties to bundle or secure the cables because the ties damage the fibers inside the cables 5 Retrieve the power cords and SER MGT cable if previously routed for connection to the switch...

Page 65: ...Secure Cables to the CMB Connect the RJ 45 Ethernet Cables on page 57 Connect the PrizmMT Cables on page 58 Connect the SFP Cables on page 60 Connecting Cables 65...

Page 66: ...66 Oracle InfiniBand Switch IS2 46 Installation Guide September 2016...

Page 67: ...DHCP assigned IP addresses log in to the service processor SP through the SER MGT connection Log In to the SP SER MGT on page 79 Assign a static IP address to the SP Assign a Static IP Address to the...

Page 68: ...open the paper clip and fold the smaller loop over so that the paper clip now resembles the letter S 4 Open the larger loop approximately 45 degrees to form a hook 5 Holding the smaller loop insert th...

Page 69: ...until you can grasp it with your fingers 7 Pull the cord through so that the end of the cord is free 8 Repeat Step 5 through Step 7 for the other power cord 9 If you will have a full time SER MGT con...

Page 70: ...nect the power cords See Connect the Power Cords on page 74 Otherwise connect the SER MGT cable See Connect the SER MGT Cable on page 71 Related Information Connect the SER MGT Cable on page 71 Connec...

Page 71: ...ks that you must perform in conjunction with this task See Installation Task Overview on page 11 2 Consider your next step If the SER MGT cable will be a full time connection go to Step 3 If the SER M...

Page 72: ...g on the right side Oracle logo in to the center 4 Continue to pull the cable through the air duct assembly to take up the slack 5 Orient the RJ 45 connector to the SER MGT receptacle at the front of...

Page 73: ...n page 74 Related Information Retrieve the Power Cords and SER MGT Cable on page 67 Connect a Serial Management Device to the SER MGT Port on page 74 Connect the Power Cords on page 74 Install the Air...

Page 74: ...other cables with servicing the switch or with other systems 5 Connect the power cords See Connect the Power Cords on page 74 Related Information Retrieve the Power Cords and SER MGT Cable on page 67...

Page 75: ...of the switch chassis 4 Plug each power cord into the facility power receptacle or power distribution device Note To provide redundancy connect each power cord to a separate power source The switch ca...

Page 76: ...GT on page 79 Assign a Static IP Address to the SP on page 80 Assign a Static IP Address to the SCP on page 82 Log In to the SP Network Management on page 84 Install the Air Duct Assembly 1 Identify t...

Page 77: ...next steps If you have DHCP assigned IP addresses power on the switch See Power On the Switch on page 78 If you have a full time SER MGT connection power on the switch See Power On the Switch on page...

Page 78: ...of the IB fabric and hosts of the Ethernet network are operational and securely cabled to the switch The serial management console is operational configured with correct parameters and cabled to the s...

Page 79: ...and SER MGT Cable on page 67 Connect the SER MGT Cable on page 71 Connect a Serial Management Device to the SER MGT Port on page 74 Connect the Power Cords on page 74 Install the Air Duct Assembly on...

Page 80: ...to Assign a Static IP Address to the SP on page 80 Related Information Retrieve the Power Cords and SER MGT Cable on page 67 Connect the SER MGT Cable on page 71 Connect a Serial Management Device to...

Page 81: ...the netmask to be configured Note You can configure one several or all properties in one command line For example set SP network pendingipdiscovery static pendingipaddress 123 45 67 89 pendingipgatewa...

Page 82: ...itch ships with DHCP IP address assignment for the SCP 1 Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task See Installation Task Overview...

Page 83: ...start the network to use the new values Restart network now Y n Y Shutting down interface eth0 OK Shutting down interface eth1 OK Shutting down loopback interface OK Bringing up loopback interface OK...

Page 84: ...erform in conjunction with this task See Installation Task Overview on page 11 2 Open an SSH session and connect to the SP by specifying the IP address you previously configured For example type ssh l...

Page 85: ...formation Retrieve the Power Cords and SER MGT Cable on page 67 Connect the SER MGT Cable on page 71 Connect a Serial Management Device to the SER MGT Port on page 74 Connect the Power Cords on page 7...

Page 86: ...86 Oracle InfiniBand Switch IS2 46 Installation Guide September 2016...

Page 87: ...ration Wizard Parameters on page 98 Related Information Understanding the Switch on page 11 Confirming Specifications on page 17 Preparing for Installation on page 23 Installing the Switch on page 29...

Page 88: ...face on page 94 Log In to the Oracle Fabric OS Interface on page 96 Run the Oracle Fabric OS Configuration Wizard on page 97 Oracle Fabric OS Configuration Wizard Parameters on page 98 LED Status Glyp...

Page 89: ...C power present and good Off AC power not present Fan Module Status Attention Amber Indicates these conditions On Fault detected Off No faults detected Related Information Verify LED Status on page 87...

Page 90: ...Fans Fan_5 health OK System Cooling Fans Fan_6 health OK System Cooling Fans Fan_7 health OK System Networking health OK System Networking Switches Switch_0 health OK System Networking Switches Switc...

Page 91: ...s on page 88 for a description of Link LED states 3 Access the Oracle ILOM CLI See Log In to the SP SER MGT on page 79 or Log In to the SP Network Management on page 84 4 Verify Ethernet connectivity...

Page 92: ...s Gi0 25 and Gi0 26 are internal to the switch architecture Ports Ex0 1 and Ex0 2 are the two SFP uplink ports The Status is the state of the port The Duplex Speed and Negotiation are the parameters o...

Page 93: ...the state of the gateway ports OFOS show ioports name type state descr v resources 0 1 gwEthernet10GbPort up indeterminate 0 0 2 gwEthernet10GbPort up indeterminate 0 0 3 gwEthernet10GbPort up indeter...

Page 94: ...exit and press the Enter key then press the Esc key and type OFOS exit root nsn156 168 Serial console stopped 6 Administer the switch from Oracle Fabric Manager or Oracle Fabric OS See Log In to the...

Page 95: ...e the user name and password and click Submit The default username is admin and the password is admin The Discovery Devices pane updates and the green circle checkmark in the Manage Status column indi...

Page 96: ...nistrator The default is admin SCP_IP_address is the IP address you configured in Assign a Static IP Address to the SCP on page 82 hostname is the host name you configured in Assign a Static IP Addres...

Page 97: ...clear the configuration and restart the system confirm 2 Review the parameters that the Oracle Fabric OS configuration wizard requests See Oracle Fabric OS Configuration Wizard Parameters on page 98 3...

Page 98: ...t is the minimum number of special characters in the password 1 8 If yes what is the minimum number of upper case letters in the password 1 8 What is the password for the admin user Enter the admin pa...

Page 99: ...s A AD Abbreviation for Active Directory adapter Physical device such as an HCA or NIC that enables a host to communicate through a fabric or network ASR Automatic Service Request A feature of Oracle...

Page 100: ...anagement switch An Ethernet switch that enables single point management of multiple service processors SP or hosts through their respective network management interfaces See also SP F fabric card A c...

Page 101: ...ect I2 C nodes See also IPMI and Oracle ILOM IB InfiniBand A high bandwidth messaging technology used for very high performance computing IB SFM module I O module for the Oracle InfiniBand Switch IS2...

Page 102: ...iver used for long range Fibre Channel communication M MAC Machine Allocation Code A 12 digit hexadecimal number that uniquely identifies a network node MIB Management Information Base A plain text fi...

Page 103: ...web based GUI to administer the switch and fabric Oracle Fabric OS Oracle Fabric Operating System The operating system within the SCP that manages the fabric See also SCP Oracle ILOM Oracle Integrated...

Page 104: ...management interfaces the Subnet Manager and so on The SCP might also be the chassis host See also SP SEFOS Sun Ethernet Fabric Operating System The operating system used by the EMS module to adminis...

Page 105: ...hat faces the cold aisle when mounted in a rack Typically is the end with fan modules and power supplies switch rear The exhaust end of the switch chassis that faces the hot aisle when mounted in a ra...

Page 106: ...106 Oracle InfiniBand Switch IS2 46 Installation Guide September 2016...

Page 107: ...retrieving 67 routing 56 securing to CMB 62 SFP 54 standard MT 50 cage nuts 35 CMB installing 45 securing cables 62 components front panel 14 rear panel 15 configuration wizard Oracle Fabric OS parame...

Page 108: ...96 configuration wizard parameters 98 running configuration wizard 97 overview installation 11 switch 13 P physical specifications 17 power cord connecting 74 retrieving 67 routing 38 powering on swit...

Page 109: ...ddress 26 assigning static IP address 80 login network management 84 SER MGT 79 specifications 17 electrical 18 environmental 18 physical 17 stabilize the rack 29 standard MT cable 50 static IP addres...

Page 110: ...110 Oracle InfiniBand Switch IS2 46 Installation Guide September 2016...

Reviews:

No comments