Qlogic InfiniPath Install Manual Download Page 22 | Manualshive

Page: 22 / 74

background image

2 – Hardware Installation
Unpacking Information

2-6

IB0056101-00 C

Q

Figure 2-1. InfiniPath QLE7140 Card with Riser, Top View

PCI Express

Edge Connectors

I

nfiniBand

Connector

Face Plate

IBA6120 ASIC

PCI Express Riser Card. Not
supplied; shown for reference.

«
...
20
21
22
23
24
...
»

Summary of Contents for InfiniPath

Page 1: ...IB0056101 00 C Page i Q S i m p l i f y InfiniPath Install Guide Version 2 0...

Page 2: ...trademark of Linus Torvalds QLA QLogic SANsurfer the QLogic logo PathScale the PathScale logo InfiniPath and EKOPath are registered trademarks of QLogic Corporation Red Hat and all Red Hat based trad...

Page 3: ...iniPath Install Guide Version 2 0 Q IB0056101 00 C Page iii 2006 QLogic Corporation All rights reserved worldwide PathScale 2004 2005 2006 All rights reserved First Published August 2005 Printed in U...

Page 4: ...InfiniPath Install Guide Version 2 0 Q Page iv IB0056101 00 C...

Page 5: ...1 1 1 Form Factors 2 2 2 1 2 Cabling and Switches 2 3 2 1 3 Software 2 3 2 1 4 Configuring the BIOS 2 3 2 2 Safety with Electricity 2 4 2 3 Unpacking Information 2 5 2 3 1 Verify Package Contents 2 5...

Page 6: ...ing Rocks and InfiniPath RPMs 3 11 3 6 2 Installing Rocks and InfiniPath RPMs Details 3 11 3 6 3 Further Information on Rocks and kickstart 3 14 3 7 InfiniPath and OpenFabrics Driver Overview 3 15 3 8...

Page 7: ...rd 2 10 2 5 Assembled QLE7140 with Riser 2 11 2 6 HTX Slot 2 12 2 7 InfiniPath Adapter with Riser Card 2 13 2 8 Assembled QHT7140 with Riser 2 14 2 9 QHT7140 without Riser Installed in a 3U Chassis 2...

Page 8: ...InfiniPath Install Guide Version 2 0 Page viii IB0056101 00 C Q A 6 OpenFabrics RPMs A 4 A 7 OpenFabrics Devel RPMs A 5 A 8 OpenSM RPMs A 6 A 9 OtherHCAs RPMs A 7...

Page 9: ...40 QHT7140product on your Linux cluster More detailed installation information and instructions for administration of the InfiniPath cluster are provided in the InfiniPath Interconnect User Guide This...

Page 10: ...cs 1 4 Switches The InfiniPath interconnect is designed to work with all InfiniBand compliant switches Use of OpenSM as a subnet manager is now supported OpenSM is part of the OpenFabrics component of...

Page 11: ...ise the same Unless otherwise stated QHT7140 will refer to both the QHT7040 and QHT7140 in this documentation Expanded MPI scalability enhancements for PCI Express have been added The QHT7040 and QHT7...

Page 12: ...ith the distributions unless otherwise noted NOTE IBM Power systems run only with the SLES 10 distribution The SUSE10 release series is no longer supported as of this InfiniPath 2 0 release Fedora Cor...

Page 13: ...nfiniBand protocols are interoperable between InfiniPath 2 0 and OFED 1 1 This release provides support for the following protocols IPoIB TCP IP networking SDP Sockets Direct Protocol OpenSM UD Unreli...

Page 14: ...ommand lines concept Italic font is used for emphasis concepts user input Bold fixed space font is used for literal items in commands or constructs that you type in Indicates a command line prompt Ind...

Page 15: ...finiPath Install Guide The InfiniPath Interconnect User Guide Release Notes Quick Start Guide Readme file The Troubleshooting Appendix for installation InfiniPath and OpenFabrics administration and MP...

Page 16: ...1 Introduction Documentation and Technical Support 1 8 IB0056101 00 C Q...

Page 17: ...te of 8Gb s OpenFabrics is interoperable with other vendors InfiniBand HCAs running compatible OpenFabrics releases 2 1 Hardware Installation Requirements This section lists hardware and software envi...

Page 18: ...del number for the adapter card that ships in the PCIe low profile short form factor This card can be used with either a full height or low profile faceplate The QHT7040 is the model number for the ad...

Page 19: ...information Optical fibre with optical adapters such as the Emcore is also supported See section 2 6 2 There are two options for Subnet Management in your cluster Use the Subnet Manager on one or mor...

Page 20: ...djustment You can check and adjust these BIOS settings using the BIOS Setup Utility For specific instructions on how to do this follow the hardware documentation that came with your system 2 2 Safety...

Page 21: ...immediately 2 3 2 List of the Package Contents The package contents for the QMI7140 adapter are InfiniPath QMI7140 An illustration of the QMI7140 is not yet available The package contents for the QLE...

Page 22: ...npacking Information 2 6 IB0056101 00 C Q Figure 2 1 InfiniPath QLE7140 Card with Riser Top View PCI Express Edge Connectors InfiniBand Connector Face Plate IBA6120 ASIC PCI Express Riser Card Not sup...

Page 23: ...e plate pull the adapter out of the anti static bag Handle the adapter only by its edges or the IB connector Do not allow the InfiniPath Adapter or any adapter card components to touch any metal parts...

Page 24: ...InfiniPath QLE7140 in 1U or 2U chassis requires installation with a PCI Express riser card A taller riser card can be used if necessary The QLE7140 can connect to any of the standard compatible PCIe r...

Page 25: ...o your system vendor instructions for how to remove the blanking panel 7 Remove the InfiniPath QLE7140 from the anti static bag 8 Locate the face plate on the connector edge of the card 9 Connect the...

Page 26: ...ge is facing the PCIe slot on the mother board and the face plate is toward the front of the chassis 12 Holding this assembly above the motherboard at about a 45 degree angle slowly lower it so that t...

Page 27: ...by powering up and verifying link status See section 2 7 2 4 3 Hardware Installation for QHT7140 with HTX Riser Most installations will be in 1U and 2U chassis using the HTX riser card This results i...

Page 28: ...are documentation that came with your system 5 Locate the HTX slot on your motherboard Note that the HTX slot has two separate connectors corresponding to the connector edges of the adapter See figure...

Page 29: ...assembly of the HTX riser card and InfiniPath Adapter This assembly is what you ll insert into the HTX slot on the motherboard in the next step 11 Turn the assembly so that the riser card connector e...

Page 30: ...tem documentation for information about mounting details such as mounting holes screws to secure the card or other brackets The InfiniPath QHT7140 with HTX riser card is now installed Next install the...

Page 31: ...te the HTX slot on your motherboard See figure 2 6 above 6 Remove the InfiniPath Adapter from the anti static bag Hold the card by the top horizontal section of the bracket and the top rear corner of...

Page 32: ...should work The longest IB cable we have currently qualified is twenty meters To install the InfiniBand cables 1 Check that you have removed the protector plugs from the cable connector ends 2 Differ...

Page 33: ...ertors For more information on the Emcore convertor contact www emcore com 2 7 Completing the Installation This section applies to the QLE7140 QHT7040 and QHT7140 adapters NOTE If you have a QMI7140 a...

Page 34: ...2 Hardware Installation Completing the Installation 2 18 IB0056101 00 C Q...

Page 35: ...rks OpenFabrics protocols including Subnet Management Agent OpenFabrics libraries and utilities OpenFabricskernelmodulesupportisnowbuiltandinstalledaspartoftheInfiniPath RPM install The InfiniPath rel...

Page 36: ...ernel software is installed on each node in your cluster The required kernels and supported Linux distributions for both InfiniPath and OpenFabrics are defined below See section 3 2 3 For each release...

Page 37: ...ution as described below under section 3 3 Among the many optional packages that each distribution offers the InfiniPath software requires openssh and openssh server and if the MPD job launcher is to...

Page 38: ...ystems running SLES 10 InfiniPath specific kernels and patches for these problems are available for download on the QLogic web site Please go to http www qlogic com Follow the links to the download pa...

Page 39: ...a distribution identifier yyy The identifiers for this release are fc4 suse9 3 and sles10 Note that these identifiers are different than those for the InfiniPath RPMs The RPM distribution identifiers...

Page 40: ...te node front end node development machine or subnet manager see appendix A 3 5 1 Downloading the tar File Containing the RPMs There are multiple interdependent RPM packages that make up the InfiniPat...

Page 41: ...ms install the RPMs in the InfiniPath and OpenFabrics directories If you plan to run both InfiniPath and OpenFabrics install all three of these RPMs You should only install the OpenSM RPM if you do no...

Page 42: ...ilt See section 3 14 for more information NOTE RPMs contain config files Your current config files will not be overwritten when new RPMs are installed New config files will contain the suffix rpmnew a...

Page 43: ...ur Distribution You may be using a kernel which is compatible with one of the supported distributions but which may not be picked during infinipath kernel installation It may also happen when using ma...

Page 44: ...ing clusters from the San Diego Supercomputer center SDSC Rocks is a way to manage the kickstart automated installation method that was created by Red Hat By using the Rocks conventions the installati...

Page 45: ...iPath RPMs onto the front end node 5 Create a directory that contains the config kickstart and InfiniPath RPMs on the front end node 6 Create the xml file home install site profiles 4 0 0 nodes extend...

Page 46: ...end node this example includes both for convenience Then copy the RPMs to the same directory as in step 4 The directory home install contrib 4 0 0 x86_64 RPMS should now contain the files in this lis...

Page 47: ...e infinipath compilers libs package InfiniPath Drivers package mpi doc package package mpi frontend package package mpi libs package package mpi benchmark package package mpi devel package post Here w...

Page 48: ...th rpm etc init d infinipath start rm etc rc d rocksconfig d post 97 infinipath file Add InfiniPath compiler library path to ld so file name etc ld so conf mode append opt infinipath lib 2 5 file post...

Page 49: ...Ethernet emulation and general OpenFabrics protocols such as IPoIB and SDP IPoIB is the protocol that allows you to run Internet Protocol over an IB fabric The module also contains a Subnet Management...

Page 50: ...s except where it will be used as subnet manager If you are upgrading your existing configuration files will not be overwritten The device files are dev ipath dev ipath0 dev ipath1 The numbered device...

Page 51: ...ck for the number of Ethernet drivers you currently have by either one of the two following commands ifconfig a ls sys class net As mentioned above we assume that two Ethernet devices numbered 0 and 1...

Page 52: ...Path chips are present the configuration for eth3 eth4 and so on follow the same format as for adding eth2 in the examples below Similarly in step 2 add one to the unit number so replace 00 guid with...

Page 53: ...to your reseller or InfiniPath support organization 3 Skip to Step 4 if you are using SUSE 9 3 or SLES 9 This step is only done on SLES 10 systems Edit the file etc udev rules d 30 net_persistent_nam...

Page 54: ...orrect you will normally now be able to run the commands ifup eth2 ifconfig eth2 Note that it may be necessary to reboot the system before the configuration changes will work 3 9 OpenFabrics Configura...

Page 55: ...plied in a later release Type the following commands as root ifconfig ib0 10 1 17 3 netmask 0xffffff00 To verify the configuration type ifconfig ib0 The output from this command should be similar to t...

Page 56: ...ig opensmd on You can start opensmd without rebooting your machine as follows etc init d opensmd start and you can stop it again like this etc init d opensmd stop If you wish to pass any arguments to...

Page 57: ...NOTE This does not stop and unload the driver if it is already loaded You can start stop or restart as root the InfiniPath support with etc init d infinipath start stop restart This method will not r...

Page 58: ...smod grep ipath_ether If there is output you should look at the output from this command to determine if it is configured sbin ifconfig a Finally if you need to find which InfiniPath and OpenFabrics m...

Page 59: ...umber of other compilers These include PGI 5 2 and 6 0 Intel 9 x the GNU gcc 3 3 x 3 4 x and 4 x compiler suites and gfortran The IBM XL family of compilers are also supported on ppc64 Power systems T...

Page 60: ...le ipath_checkout options hostsfile wherehostsfiledesignatesafilelistingthehostnamesofthenodesofthecluster one hostname per line The format of hostsfile is as follows hostname1 hostname2 ipath_checkou...

Page 61: ...s When specified the tests continue after a failure with failing nodes excluded from subsequent tests workdir DIR Use DIR to hold intermediate files created while running tests DIR must not already ex...

Page 62: ...brics it is recommended that you remove all the packages at the same time 3 20 1 Downgrading If you want to downgrade please remove both the InfiniPath and OpenFabrics RPMs then install the older bits...

Page 63: ...tory 7 If not yet installed install the InfiniPath and OpenFabrics modules from your alternate set of RPMs 8 Reload all modules by using this command as root etc init d infinipath start An alternate m...

Page 64: ...3 Software Installation Installation Troubleshooting 3 30 IB0056101 00 C Q...

Page 65: ...nt or running jobs We define these nodes in the following way Front end node This node will launch jobs It is referred to as the front end node in this documentation Compute node These are the nodes t...

Page 66: ...nfinipath doc 2 0 xxx_yyy noarch rpm InfiniPath man pages and other documents Optional Optional Optional mpi doc 2 0 xxx_yyy noarch rpm Man pages for MPI functions and other MPI documents Optional Opt...

Page 67: ...nfiniPath configuration files Optional Required Optional infinipath kernel 2 0 xxx_yyy x86_64 rpm InfiniPath drivers OpenFabrics kernel modules Optional Required Optional infinipath libs 2 0 xxx_yyy i...

Page 68: ...for the IB diagnostic and management tools Required for OpenSM libibmad 2 0 xxx 1_0 yyy x86_64 rpm Low layer IB functions for use by the IB diagnostic and managementprograms TheseincludeMAD SA SMP and...

Page 69: ...out recompiling the application Required for OpenFabrics ofed scripts 2 0 xxx 1_1 yyy x86_64 rpm OpenFabrics scripts Optional for OpenFabrics openib diags 2 0 xxx 1_1_0 yyy x86_64 rpm Useful programs...

Page 70: ...0 xxx 2_0_0 yyy x86_64 rpm Static component libraries for OpenSM Optional for OpenFabrics libosmcomp devel 2 0 xxx 2_0_0 yyy x86_64 rpm Static component libraries for OpenSM Optional for OpenFabrics l...

Page 71: ...108 InfiniHost and MT25208 InfiniHost III Ex for use with the libibverbs library Optional mstflint 2 0 xxx 1_0 yyy x86_64 rpm This package contains a burning tool for Mellanox manufactured HCA cards I...

Page 72: ...A RPM Descriptions OpenFabrics RPMs A 8 IB0056101 00 C Q...

Page 73: ...finiPath scripts using to start stop or restart drivers 3 23 InfiniPath software installed layout 3 8 InfiniPath software list of 1 4 3 1 Installation for CFEE ppc64 hardware 2 8 for HTX adapter hardw...

Page 74: ...rsion 2 0 Index 2 IB0056101 00 C Q S Safety precautions 2 4 Software package requirements 3 3 Switch configuration 3 25 Switches supported 1 2 T Tarball layout 3 7 Technical support 1 7 Troubleshootin...

Reviews:

No comments

Related manuals for InfiniPath

Brand: Emulex Pages: 2

Brand: Black Box Pages: 107

Brand: Intellinet Pages: 3

Brand: TP-Link Pages: 30

GA511 - Gigabit Ethernet PC Card

Brand: NETGEAR Pages: 50

Brand: Abus Pages: 23

Brand: Avaya Pages: 2

Brand: DataCube Pages: 15

Brand: Uplift Desk Pages: 2

Brand: TRENDnet Pages: 9

Brand: Askey Pages: 5

trusst CT-CONSPK

Brand: Chauvet Pages: 8

Brand: CNET Pages: 1

Brand: Aukey Pages: 14

Brand: EnGenius Pages: 45

Brand: Schwaiger Pages: 2

Brand: Gossen MetraWatt Pages: 2

Brand: Gossen MetraWatt Pages: 7

Brands by name

0 1 2 3 4 5 6 7 8 9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Popular brands

Load more brands