Sun Microsystems Dual Port 4x IB Host Channel Adapter PCI-ExpressModule User Manual Download Page 1

Sun Microsystems, Inc.
www.sun.com

Submit comments about this document at:

http://www.sun.com/hwdocs/feedback

Sun

Dual Port 4x IB Host Channel

Adapter PCI-ExpressModule User’s

Guide

Part No. 819-6182-11
July 2006, Revision A

Summary of Contents for Dual Port 4x IB Host Channel Adapter PCI-ExpressModule

Page 1: ...systems Inc www sun com Submit comments about this document at http www sun com hwdocs feedback Sun Dual Port 4x IB Host Channel Adapter PCI ExpressModule User s Guide Part No 819 6182 11 July 2006 Re...

Page 2: ...O BE LEGALLY INVALID Copyright 2006 Sun Microsystems Inc 4150 Network Circle Santa Clara Californie 95054 tats Unis Tous droits r serv s Sun Microsystems Inc poss de les droits de propri t intellectue...

Page 3: ...Interface 3 PCI Express Interface 3 LED Displays 3 Hardware Software and Cable Requirements 4 2 Installing the Adapter 5 Installing the Hardware 5 To Install the Adapter 5 Verifying the Installation 6...

Page 4: ...work 15 InfiniBand Management Framework 15 Tavor HCA Driver 15 Related Software Documentation 16 Manuals 16 Man Pages 16 Downloading the Software for the Linux Operating System 17 A InfiniBand Cable P...

Page 5: ...2000 MHz IEC 61000 4 4 1 kV AC and DC Power Lines 0 5 kV Signal Lines IEC 61000 4 5 2 kV AC Line Gnd 1 kV AC Line Line and Outdoor Signal Lines 0 5 kV Indoor signal Lines 10m IEC 61000 4 6 3 V IEC 610...

Page 6: ...vi Sun IB Host Channel Adapter PCI ExpressModule User s Guide July 2006...

Page 7: ...ce that are not approved by Sun Microsystems Inc may void the authority granted to the user by the FCC to operate this equipment FCC Class B Notice This device complies with Part 15 of the FCC Rules O...

Page 8: ...Class A digital apparatus complies with Canadian ICES 003 Cet appareil num rique de la classe A est conforme la norme NMB 003 du Canada ICES 003 Class B Notice Avis NMB 003 Classe B This Class B digi...

Page 9: ...ement is applicable to products shipped to Taiwan and marked as Class A on the product compliance label CCC Class A Notice The following statement is applicable to products shipped to China and marked...

Page 10: ...x Sun IB Host Channel Adapter PCI ExpressModule User s Guide July 2006...

Page 11: ...reduce the risk of electric shock and danger to personal health follow the instructions Depending on the type of power switch your device has one of the following symbols may be used On Applies AC po...

Page 12: ...me manufacturer and type or equivalent type recommended by the manufacturer per the instructions provided in the product service manual Do not disassemble batteries or attempt to recharge them outside...

Page 13: ...ent de prendre pour l installation d un produit Sun Microsystems Mesures de s curit Pour votre s curit nous vous recommandons de suivre scrupuleusement les mesures de s curit ci dessous lorsque vous i...

Page 14: ...t d un produit Sun Attention Evitez d obstruer ou de recouvrir les orifices de votre produit Sun N installez jamais un produit Sun pr s d un radiateur ou d une source de chaleur Si vous ne respectez p...

Page 15: ...la l gislation locale en vigueur Notez que sur les cartes processeur de Sun une batterie au lithium a t moul e dans l horloge temps r el Les batteries ne sont pas des pi ces rempla ables par le client...

Page 16: ...Frequenz der Stromversorgung den Nennleistungen auf dem am Ger t angebrachten Etikett entsprechen F hren Sie niemals Fremdobjekte in die ffnungen am Ger t ein Es k nnen gef hrliche Spannungen anliegen...

Page 17: ...g gilt nur f r Ger te mit Standby Netzschalter Achtung Beim Netzschalter dieses Ger ts handelt es sich nur um einen Ein Standby Schalter Zum v lligen Abtrennen des Systems von der Stromversorgung dien...

Page 18: ...rte Luftzirkulation zu vermei den sollte die f r den sicheren Betrieb des Ger ts erforderliche Luftzirkulation eingesetzt werden Hinweis zur Laser Konformit t Sun Produkte die die Laser Technologie ve...

Page 19: ...recalentarse y la fiabilidad de su funcionamiento se ver a afectada Cumplimiento de la normativa para instalaciones SELV Las condiciones de seguridad de las conexiones de entrada y salida cumplen los...

Page 20: ...e en las placas CPU de Sun hay una bater a de litio incorporada en el reloj en tiempo real Los usuarios no deben reemplazar este tipo de bater as Cubierta de la unidad del sistema Debe extraer la cubi...

Page 21: ...e Ved utskifting benyttes kun batteri som anbefalt av apparatfabrikanten Brukt batteri returneres apparatleverand ren Sverige Varning Explosionsfara vid felaktigt batteribyte Anv nd samma batterityp e...

Page 22: ...xxii Sun IB Host Channel Adapter PCI ExpressModule User s Guide July 2006...

Page 23: ...ule card is an InfiniBand IB host channel adapter HCA card The adapter is referred to as IB EM in this manual How This Document Is Organized Chapter 1 provides a description of the adapter Chapter 2 d...

Page 24: ...s shutting down the system booting the system and configuring devices Refer to the following for this information Software documentation that you received with your system Solaris Operating System doc...

Page 25: ...ut Edit your login file Use ls a to list all files You have mail AaBbCc123 What you type when contrasted with on screen computer output su Password AaBbCc123 Book titles new words or terms words to be...

Page 26: ...6183 PDF HTML Online Service System Administration Guide Devices and Files System 817 5093 PDF HTML Online Network Services System Administration Guide Network Services 816 4555 PDF HTML Online Switc...

Page 27: ...or in connection with the use of or reliance on any such content goods or services that are available on or through such sites or resources Sun Welcomes Your Comments Sun is interested in improving it...

Page 28: ...xxviii Sun IB Host Channel Adapter PCI ExpressModule User s Guide July 2006...

Page 29: ...4 Product Features Sun IB Host Channel Adapter PCI ExpressModule card features include Dual 4X 10Gb s Infiniband ports Integrated serializer deserializer SerDes x8 PCI Express 16Gb s full duplex Infi...

Page 30: ...e x8 PCI Express connection which provides full duplex data transfers of up to 16 Gbps FIGURE 1 1 shows the IB EM card The board characteristics are Power approximately 12 watts total Adapter protecti...

Page 31: ...tiating the PCI bus operations or a slave responding to PCI bus operations The PCI Express bus can connect to a host CPU in an HCA application LED Displays Two InfiniBand ports connectors are located...

Page 32: ...other IB switch vendor ports may require 4x 4x IB cables Refer to the documentation for the IB switch you use TABLE 1 2 Hardware Software and Cable Requirements Operating System Servers Solaris 10 1 0...

Page 33: ...basic tasks required to install the adapter Refer to your system installation or service manual for detailed ExpressModule adapter installation instructions To Install the Adapter 1 Halt and power of...

Page 34: ...verify the installation you must install the adapter in the chassis power the server and cable it to an operational InfiniBand switch The InfiniBand switch should automatically recognize InfiniBand se...

Page 35: ...pci ID 370704 kern info PCI device pciex15b3 6278 0 tavor0 Feb 17 20 30 42 hostname genunix ID 936769 kern info tavor0 is pci 7c0 pci 0 pci 8 pciex15b3 6278 0 Feb 17 20 30 42 hostname tavor ID 548581...

Page 36: ...8 Sun IB Host Channel Adapter PCI ExpressModule User s Guide July 2006...

Page 37: ...re can be used when making changes to your configuration See System Administration Guide Basic Administration 817 1985 for Solaris setup information Configuring IPoIB 1 Confirm that the IB EM ports ar...

Page 38: ...pciex15b3 6278 0 Feb 17 20 30 42 hostname tavor ID 548581 kern info tavor0 FW ver 0004 0007 0190 HW rev a0 Feb 17 20 30 42 hostname tavor ID 827127 kern info tavor0 MT23108 Sun Microsystems InfiniBan...

Page 39: ...etween InfiniBand hosts For example where host ip address is the IP address of another IB host See ping 1M for details Repeat this procedure on all hosts that are connected to the IB switch ifconfig i...

Page 40: ...12 Sun IB Host Channel Adapter PCI ExpressModule User s Guide July 2006...

Page 41: ...on on page 18 Downloading the Software for the Linux Operating System on page 19 InfiniBand Software for the Solaris Operating System The InfiniBand software is included in the Solaris 10 Operating Sy...

Page 42: ...IP over InfiniBand Kernel The IPoIB kernel implements the IETF IPoIB protocol and provides IPoIB services for all IB EM ports present in the system This driver enables the standard Solaris TCP IP sta...

Page 43: ...kup related functionality See ibcm 7d for details InfiniBand Management Framework IBMF is a complementary component to IBTF that enables IB clients to provide and access management services over the I...

Page 44: ...555 Note You can view or print Sun documentation at the following web site http www sun com documentation Man Pages cfgadm_ib 1M InfiniBand hardware specific commands for cfgadm datadm 1M maintain DAT...

Page 45: ...Operating System If you are using the Linux operating system use the Mellanox software to operate the IB EM adapter The Mellanox software and documentation are available at the following web site htt...

Page 46: ...20 Sun IB Host Channel Adapter PCI ExpressModule User s Guide July 2006...

Page 47: ...appendix lists the cable pin assignments for the Sun IB Host Channel Adapter PCI ExpressModule It contains the following section Cable Pin Assignments on page 19 Cable Pin Assignments FIGURE A 1 shows...

Page 48: ...G1 G25 S47 S2 S48 S16 G9 S2 S15 S1 G1 P1 12X P2 4X G S1 S2 G S3 S4 G S5 S6 G S7 S8 G S41 S42 G S43 S44 G S45 S46 G S47 S48 G SHELL G1 S16 S15 G2 S14 S13 G3 S12 S11 G4 S10 S9 G5 S8 S7 G6 S6 S5 G7 S4 S...

Page 49: ...sed to verify the configuration and functionality of most hardware controllers and devices The SunVTS software operates primarily from a graphical user interface enabling test parameters to be set qui...

Page 50: ...following patch numbers SPARC patch 118962 01 or higher X86 patch 118961 01 or higher The adapter and Tavor device driver must be installed and the IB port interface must be configured offline for the...

Page 51: ...unVTS 6 0 PS1 Documentation Supplement 819 1804 for instructions on how to run ibhcatest Troubleshooting Tasks The following tasks can be useful when troubleshooting the IB HCA and the link Verify tha...

Page 52: ...and Other Useful Utilities These utilities can display status and other information about InfiniBand devices cfgadm snoop netstat kstat cfgadm The cfgadm utility displays status and other information...

Page 53: ...Who is 199 1 1 168 ib 1 168 ib 1 168 ib 1 167 ARP R 199 1 1 168 ib 1 168 is 0 2 4 7 0 0 0 0 a 4 7c 4f 0 2 c9 2 0 0 55 91 ib 1 167 ib 1 168 ICMP Echo request ID 35608 Sequence number 0 ib 1 168 ib 1 16...

Page 54: ...26 Sun IB Host Channel Adapter PCI ExpressModule User s Guide July 2006...

Page 55: ...5 InfiniBand cable 19 InfiniBand Management Framework 15 InfiniBand software overview 13 InfiniBand Transport Layer 15 installing the adapter 5 interface InfiniBand 3 PCI X 3 IPoIB 13 configuring 9 L...

Page 56: ...28 Sun IB Host Channel Adapter PCI ExpressModule User s Guide July 2006 tasks 23 utilities 24 U uDAPL 13 14 utilities 24 cfgadm 24 kstat 25 netstat 25 snoop 25...

Reviews: