Cisco InfiniBand 4x User Manual Download Page 3

Page: 3 / 58

Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide

78-17838-01

C O N T E N T S

Preface

Audience

Organization

Conventions

Summary of Contents for InfiniBand 4x

Page 1: ...co Systems Inc 170 West Tasman Drive San Jose CA 95134 1706 USA http www cisco com Tel 408 526 4000 800 553 NETS 6387 Fax 408 526 4100 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Text Part Number 78 17838 01 ...

Page 2: ...CO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES Any Internet Protocol IP addresses used in this document are not intended to be actual addresses Any examples command display output and figures included in the document are shown for illustrative purposes only Any use of actual IP addresses in illustrative content is unintentional and coincidental Cisco 4x InfiniBand Switch ...

Page 3: ...i Cisco Technical Support Documentation Website ix Submitting a Service Request ix Definitions of Service Request Severity ix IBM Statement of Limited Warranty Z125 4753 08 04 2004 xi Part 1 General Terms xi What this Warranty Covers xi What This Warranty Does Not Cover xii How to Obtain Warranty Service xii What IBM Will Do to Correct Problems xii Exchange of a Machine or Part xiii Your Additiona...

Page 4: ...MACAU S A R OF CHINA xvii INDIA xviii JAPAN xviii MALAYSIA xviii NEW ZEALAND xviii PEOPLE S REPUBLIC OF CHINA PRC xix PHILIPPINES xix SINGAPORE xix EUROPE MIDDLE EAST AFRICA EMEA xix EUROPEAN UNION EU xxi CONSUMERS xxii FRANCE AND BELGIUM xxii THE FOLLOWING TERMS APPLY TO THE COUNTRY SPECIFIED xxiii AUSTRIA xxiii EGYPT xxiii FRANCE xxiii GERMANY xxiv HUNGARY xxiv IRELAND xxiv SLOVAKIA xxv SOUTH AF...

Page 5: ...uropean Union EMC Directive conformance statement xxxiii Taiwanese Class A warning statement xxxiii Chinese Class A warning statement xxxiv Japanese Voluntary Control Council for Interference VCCI statement xxxiv WEEE Notice xxxv Safety xxxvii C H A P T E R 1 About the Server Switch Module 1 1 Introduction 1 1 Server Switch Module and HCA Expansion Cards 1 2 Topologies 1 3 Single Switch Topology 1...

Page 6: ...ter User Guide 78 17838 01 Changing the Default Password 2 3 Configuring the System Clock 2 3 Connecting Beyond BladeCenter 2 5 Connecting InfiniBand Cables 2 5 C H A P T E R 3 Specifications 3 1 Electrical Requirements 3 1 Environmental Requirements 3 1 I N D E X ...

Page 7: ...itch Module Describes the server switch module and HCA expansion cards including topology management the LEDs and external InfiniBand ports Chapter 2 Installing and Configuring the Server Switch Installation configuration and external connection of the server switch module are described Chapter 3 Specifications Electrical and environmental requirements are described in this chapter Convention Desc...

Page 8: ...ces and separated by vertical bars Braces can also be used to group keywords and or arguments for example interface interface type x y z Optional alternative keywords are grouped in brackets and separated by vertical bars string A nonquoted set of characters Do not use quotation marks around the string or the string will include the quotation marks screen font Terminal sessions and information the...

Page 9: ... the telephone numbers below To obtain warranty service contact IBM or your IBM reseller In Canada or the United States call 1 800 IBM SERV 426 7378 In the EU countries see the telephone numbers in Table 1 Table 1 EU Country Telephone List Phone numbers are subject to change without notice Note IBM and Cisco are both available for more information For support contact IBM before Cisco Austria 43 1 ...

Page 10: ...so be found in the output of the show backplane command Determining the Software Version If InfiniBand drivers are already installed on the host to determine the version of the Topspin Linux host drivers you are running enter the following command at the prompt rpm qa grep topspin topspin ib mod rhel3 2 4 21 32 ELsmp 3 2 0 82 topspin ib mpi rhel3 3 2 0 82 topspin ib rhel3 3 2 0 82 Downloading the ...

Page 11: ...te the serial number label on your product and record the information before placing a service call Submitting a Service Request Using the online TAC Service Request Tool is the fastest way to open S3 and S4 service requests S3 and S4 service requests are those in which your network is minimally impaired or for which you require product information After you describe your situation the TAC Service...

Page 12: ...oducts You and Cisco will commit full time resources during normal business hours to resolve the situation Severity 3 S3 Operational performance of your network is impaired but most business operations remain functional You and Cisco will commit resources during normal business hours to restore service to satisfactory levels Severity 4 S4 You require information or assistance with Cisco product ca...

Page 13: ...ecifications Specifications which are available on request The warranty period for the Machine starts on the original Date of Installation and is specified in Part 3 Warranty Information The date on your invoice or sales receipt is the Date of Installation unless IBM or your reseller informs you otherwise Many features conversions or upgrades involve the removal of parts and their return to IBM A ...

Page 14: ...idence of your entitlement to warranty service What IBM Will Do to Correct Problems When you contact IBM for service you must follow the problem determination and resolution procedures that IBM specifies An initial diagnosis of your problem can be made either by a technician over the telephone or electronically by access to an IBM website The type of warranty service applicable to your Machine is ...

Page 15: ...r of changes in the Machine s location 4 a ensure all information about identified or identifiable individuals Personal Data is deleted from the Machine to the extent technically possible b allow IBM your reseller or an IBM supplier to process on your behalf any remaining Personal Data as IBM or your reseller considers necessary to fulfill its obligations under this Statement of Limited Warranty w...

Page 16: ... EXCLUSION OR LIMITATION OF INCIDENTAL OR CONSEQUENTIAL DAMAGES SO THE ABOVE LIMITATION OR EXCLUSION MAY NOT APPLY TO YOU SOME STATES OR JURISDICTIONS DO NOT ALLOW LIMITATIONS ON HOW LONG AN IMPLIED WARRANTY LASTS SO THE ABOVE LIMITATION MAY NOT APPLY TO YOU Governing Law Both you and IBM consent to the application of the laws of the country in which you acquired the Machine to govern interpret an...

Page 17: ...atement of Limited Warranty will be settled exclusively by the Judges of the Republic of Colombia EQUADOR Jurisdiction The following is added after the first sentence Any litigation arising from this Statement of Limited Warranty will be settled exclusively by the Judges of Quito MEXICO Jurisdiction The following is added after the first sentence Any litigation arising from this Statement of Limit...

Page 18: ...from IBM in Canada or the United States call 1 800 IBM SERV 426 7378 CANADA Limitation of Liability The following replaces item 1 of this section 1 damages for bodily injury including death or physical harm to real property and tangible personal property caused by IBM s negligence and Governing Law The following replaces laws of the country in which you acquired the Machine in the first sentence l...

Page 19: ...in Singapore in accordance with the Arbitration Rules of Singapore International Arbitration Center SIAC Rules then in effect The arbitration award shall be final and binding for the parties without appeal and shall be in writing and set forth the findings of fact and the conclusions of law The number of arbitrators shall be three with each side to the dispute being entitled to appoint one arbitra...

Page 20: ...ed by the president of the Bar Council of India Other vacancies shall be filled by the respective nominating party Proceedings shall continue from the stage they were at when the vacancy occurred If one of the parties refuses or otherwise fails to appoint an arbitrator within 30 days of the date the other party appoints its the first appointed arbitrator shall be the sole arbitrator provided that ...

Page 21: ...Dispute Resolution Center Inc Other vacancies shall be filled by the respective nominating party Proceedings shall continue from the stage they were at when the vacancy occurred If one of the parties refuses or otherwise fails to appoint an arbitrator within 30 days of the date the other party appoints its the first appointed arbitrator shall be the sole arbitrator provided that the arbitrator was...

Page 22: ...vory Coast Lebanon Madagascar Mali Mauritania Mauritius Mayotte Morocco New Caledonia Niger Reunion Senegal Seychelles Togo Tunisia Vanuatu and Wallis Futuna 3 the laws of Finland in Estonia Latvia and Lithuania 4 the laws of England in Angola Bahrain Botswana Burundi Egypt Eritrea Ethiopia Ghana Jordan Kenya Kuwait Liberia Malawi Malta Mozambique Nigeria Oman Pakistan Qatar Rwanda Sao Tome Saudi ...

Page 23: ...tion termination or nullity will be finally settled under the Rules of Arbitration and Conciliation of the International Arbitral Center of the Federal Economic Chamber in Vienna Vienna Rules by three arbitrators appointed in accordance with these rules The arbitration will be held in Vienna Austria and the official language of the proceedings will be English The decision of the arbitrators will b...

Page 24: ...IERS OR RESELLERS LIABLE FOR ANY OF THE FOLLOWING EVEN IF INFORMED OF THEIR POSSIBILITY 1 LOSS OF OR DAMAGE TO DATA 2 INCIDENTAL OR INDIRECT DAMAGES OR FOR ANY ECONOMIC CONSEQUENTIAL DAMAGES 3 LOST PROFITS EVEN IF THEY ARISE AS AN IMMEDIATE CONSEQUENCE OF THE EVENT THAT GENERATED THE DAMAGES OR 4 LOSS OF BUSINESS REVENUE GOODWILL OR ANTICIPATED SAVINGS FRANCE AND BELGIUM Limitation of Liability Th...

Page 25: ... Will Do to Correct Problems The following is added to this section During the warranty period IBM will reimburse you for the transportation charges for the delivery of the failing Machine to IBM Limitation of Liability The following paragraph is added to this section The limitations and exclusions specified in the Statement of Limited Warranty will not apply to damages caused by IBM with fraud or...

Page 26: ...rdinary negligence HUNGARY Limitation of Liability The following is added at the end of this section The limitation and exclusion specified herein shall not apply to liability for a breach of contract damaging life physical well being or health that has been caused intentionally by gross negligence or by a criminal act The parties accept the limitations of liability as valid provisions and state t...

Page 27: ...AMIBIA BOTSWANA LESOTHO AND SWAZILAND Limitation of Liability The following is added to this section IBM s entire liability to you for actual damages arising in all situations involving nonperformance by IBM in respect of the subject matter of this Statement of Warranty will be limited to the charge paid by you for the individual Machine that is the subject of your claim from IBM UNITED KINGDOM Li...

Page 28: ... consequential loss or 3 loss of profits business revenue goodwill or anticipated savings Part 3 Warranty Information This Part 3 provides information regarding the warranty applicable to your Machine including the warranty period and type of warranty service IBM provides Warranty Period The warranty period may vary by country or region and is specified in the table below Note Region means either ...

Page 29: ...cement CRU and 2 you may be charged for the replacement CRU if IBM does not receive the defective CRU within 30 days of your receipt of the replacement 2 On site Service IBM or your reseller will either repair or exchange the failing Machine at your location and verify its operation You must provide suitable working area to allow disassembly and reassembly of the IBM Machine The area must be clean...

Page 30: ...BM reseller In Canada or the United States call 1 800 IBM SERV 426 7378 In the EU countries see the telephone numbers below EU Country Telephone List Phone numbers are subject to change without notice For the warranty service contact telephone number in a country subsequently added to the EU and not yet reflected in the list below contact IBM in that country or visit the website above for a curren...

Page 31: ...ensing IBM Corporation North Castle Drive Armonk NY 10504 1785 U S A INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND EITHER EXPRESS OR IMPLIED INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF NON INFRINGEMENT MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE SOME STATES DO NOT ALLOW DISCLAIMER OF EXPRESS OR IMPLIED WARRANTIES IN CERT...

Page 32: ... Group in the United States and other countries Java and all Java based trademarks and logos are trademarks or registered trademarks of Sun Microsystems Inc in the United States other countries or both Adaptec and HostRAID are trademarks of Adaptec Inc in the United States other countries or both Linux is a trademark of Linus Torvalds in the United States other countries or both Red Hat the Red Ha...

Page 33: ...d to the implied warranties of merchantability and fitness for a particular purpose These products are offered and warranted solely by third parties IBM makes no representations or warranties with respect to non IBM products Support if any for the non IBM products is provided by the third party not IBM Some software may differ from its retail version if available and may not include user manuals o...

Page 34: ...ed in a commercial environment This equipment generates uses and can radiate radio frequency energy and if not installed and used in accordance with the instruction manual may cause harmful interference to radio communications Operation of this equipment in a residential area is likely to cause harmful interference in which case the user will be required to correct the interference at his own expe...

Page 35: ...United Kingdom European Union EMC Directive conformance statement This product is in conformity with the protection requirements of EU Council Directive 89 336 EEC on the approximation of the laws of the Member States relating to electromagnetic compatibility IBM cannot accept responsibility for any failure to satisfy the protection requirements resulting from a nonrecommended modification of the ...

Page 36: ...co 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78 17838 01 Notices Electronic Emission Notices Chinese Class A warning statement Japanese Voluntary Control Council for Interference VCCI statement ...

Page 37: ...belled in accordance with European Directive 2002 96 EC concerning waste electrical and electronic equipment WEEE The Directive determines the framework for the return and recycling of used appliances as applicable throughout the European Union This label is applied to various products to indicate that the product is not to be thrown away but rather reclaimed upon end of life per this Directive ...

Page 38: ...xxxvi Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78 17838 01 WEEE Notice ...

Page 39: ...xxxvii Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78 17838 01 Safety ...

Page 40: ...xxxviii Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78 17838 01 Safety ...

Page 41: ...xxxix Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78 17838 01 Safety ...

Page 42: ...xl Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78 17838 01 Safety ...

Page 43: ...5 Introduction The Cisco 4x InfiniBand Switch module for IBM BladeCenter hereafter Server Switch Module adds InfiniBand IB switching capability to hosts in your IBM BladeCenter H chassis When you add one or two Server Switch Modules to your BladeCenter chassis and add HCA expansion cards to your BladeCenter hosts your hosts can communicate to one another over IB within the chassis When you connect...

Page 44: ...etup and management After you set up a Server Switch Module and bring it online the on board Cisco Subnet Manager brings distributed intelligence to the IB network Cisco Server Switch Modules allow you to perform a field upgrade to load the latest firmware on the Server Switch Module You can perform a firmware upgrade even following a failed upgrade CLI and GUI interfaces both support upgrades For...

Page 45: ... When you add a second Server Switch Module to the BladeCenter chassis each port of each HCA expansion card connects to a Server Switch Module Note The Server Switch Modules do not connect to each other within the BladeCenter chassis To connect the modules to enable features such as subnet manager failover connect the modules with an IB cable through the external connectors Before you connect the ...

Page 46: ...Status Server Switch Module Status LEDs Module Status LEDs provide an at a glance indication of the health of the Server Switch Module The tables list and describe the states of the Module Status LEDs LED Label Color Function Status OK Yellow Green Yellow OK Green Decode Off Off Module powered off or LED failure Blink Blink LED test first 1 3 seconds after power on Off Blink Test in progress On Of...

Page 47: ...pport three 4x connections using an octopus cable This cable begins as one 12x connector which connects to your Server Switch Module then branches into three 4x connectors that you can attach to 4x IB interfaces The external ports are numbered 17 24 Ports 19 20 21 and ports 22 23 24 are on 12x connectors and can be set up as 12x or three 4x connections LED Label Color Function port TX RX Green LED...

Page 48: ...1 6 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78 17838 01 Chapter 1 About the Server Switch Module External InfiniBand Ports ...

Page 49: ...hese steps Step 1 Ensure that you are properly grounded by attaching the ESD wrist strap to bare skin and attaching the clip on the other end to unpainted metal Step 2 Choose the I O slot into which you want to install the Server Switch Module You must install the module in I O slot 7 or slot 9 Step 3 Refer to your BladeCenter unit documentation for standard hardware module installation instructio...

Page 50: ...d navigation tree Step 4 Select Management under I O Module Tasks in the left pane An I O Module Management heading appears in the display Step 5 Click the link of the physical slot that holds the Server Switch Module that you want to configure A heading appears where X is the number of the slot Step 6 Enter an IP address for the Server Switch Module in the IP address field of the New Static IP Co...

Page 51: ...isco BladeCenterH Switch to Global Configuration mode by entering the configure terminal command Cisco BladeCenterH configure terminal Cisco BladeCenterH config Step 2 Change the password of the user ID by entering the username command Include the following User ID for which you need to change the password in this case USERID password keyword New password that you want to apply to the user ID Cisc...

Page 52: ...Exec mode by entering the enable command Cisco BladeCenterH enable Cisco BladeCenterH Step 2 Switch to Global Configuration mode by entering the configure terminal command Cisco BladeCenterH configure terminal Cisco BladeCenterH config Step 3 Assign a primary NTP server by entering the ntp command with this information server one keyword IP address of the NTP server Cisco BladeCenterH config ntp s...

Page 53: ...y entering the ib sm command with this information subnet prefix keyword Subnet prefix of the InfiniBand network to which the Server Switch belongs priority keyword Priority integer value of the subnet manager 1 is the lowest the priority Cisco BladeCenterH config ib sm subnet prefix fe 80 00 00 00 00 00 00 priority 1 Step 4 Exit Global Configuration mode Cisco BladeCenterH config exit Cisco Blade...

Page 54: ...Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78 17838 01 Chapter 2 Installing and Configuring the Server Switch Connecting Beyond BladeCenter Figure 2 1 Removing a Pull Connector 154590 ...

Page 55: ...C Supply Voltage 12 2 Volts nominal DC Supply Current 6 0 Amps maximum DC Supply Power without External Port Powered Devices 39 watts maximum DC Supply Power with External Port Powered Devices 60 watts maximum Area Requirement Module Operating Temperature 0 to 60 degrees C module inlet Forced Air Cooling Rate 6 5 CFM minimum Module Non Operating Temperature 40 to 70 degrees C Operating Altitude 0 ...

Page 56: ...ronmental Requirements Note In the case of non uniform air flow the velocity at any point must meet or exceed 150 LFM Operational Vibration 0 27G at 5Hz to 500Hz for 30 minutes Storage Shock 50G with a velocity change of 3454 mm sec squared Storage Vibration 0 5G at 2Hz to 200Hz for 15 minutes Area Requirement ...

Page 57: ...ation v H HCA expansion cards about 1 2 I I O module management 2 2 IB cable connection 2 5 initial configuration 2 2 installing the server switch module 2 1 IP address 2 2 L LEDs 1 4 port status 1 5 switch module status 1 4 M management interfaces 1 3 N new static IP configuration 2 2 P password changing 2 3 ports internal and external 1 2 port status LEDs 1 5 S server switch module initial confi...

Page 58: ...Index IN 2 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78 17838 01 T topology dual switch 1 3 single switch 1 3 ...

Search results

Summary of Contents for InfiniBand 4x

Reviews:

Related manuals for InfiniBand 4x

Brands by name

Popular brands

Cisco InfiniBand 4x, User Manual

Search results

Summary of Contents for InfiniBand 4x

Reviews:

Related manuals for InfiniBand 4x

Brands by name

Popular brands