manualshive.com logo in svg

Silicon Graphics SGI Altix 3700 Bx2, Руководство пользователя

Поделиться
Скачать
Silicon Graphics SGI Altix 3700 Bx2 Скачать руководство пользователя страница 5

Contents

007-4377-002

v

Contents

Figures

  .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     . xiii

Tables

   .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

xvii

About This Guide

.     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     . xix

Audience     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     . xix
Important Information     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     . xix
Chapter Descriptions  .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     . xx
Related Publications   .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     . xxi
Conventions     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

xxiii

Product Support    .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

xxiii

Reader Comments .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

xxiv

1.

Operation Procedures

     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

1

Precautions .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

1

ESD Precaution     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

1

Safety Precautions      .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

2

Connecting a System Console     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

3

Powering the System On and Off     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

3

Powering On the System .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

4

Preparing to Power On  .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

4

Powering On at the System Console    .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

7

Powering On in the L1 Mode    .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

7

Powering Off the System.     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

8

Powering Off at the System Console   .     .     .     .     .     .     .     .     .     .     .     .     .     .     .

9

Using Embedded Support Partner (ESP)     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     . 10
Monitoring Your Server   .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     .     . 11

Содержание SGI Altix 3700 Bx2

Страница 1: ...SGI Altix 3700 Bx2 User s Guide Document Number 007 4377 002 ...

Страница 2: ...and conventions This document is provided with limited rights as defined in 52 227 14 The electronic software version of this document was developed at private expense if acquired under an agreement with the USA government or any contractor thereto it is acquired as commercial computer software subject to the provisions of its applicable license agreement as specified in a 48 CFR 12 212 of the FAR...

Страница 3: ...007 4377 002 iii Record of Revision Version Description 001 September 2004 first release 002 November 2004 Updated IO9 information and L1 bezel color ...

Страница 4: ......

Страница 5: ...iii Reader Comments xxiv 1 Operation Procedures 1 Precautions 1 ESD Precaution 1 Safety Precautions 2 Connecting a System Console 3 Powering the System On and Off 3 Powering On the System 4 Preparing to Power On 4 Powering On at the System Console 7 Powering On in the L1 Mode 7 Powering Off the System 8 Powering Off at the System Console 9 Using Embedded Support Partner ESP 10 Monitoring Your Serv...

Страница 6: ...tive 27 Command Targeting 28 Viewing Information Warnings and Error Messages 28 Powering On Powering Off and Resetting the Brick 29 Console Mode from L1 30 L1 Console Selection 30 Viewing Information Warnings and Error Messages 32 L2 Emulator Operation 33 Configuring an Emulated L2 s IP Address 33 Viewing System Configuration 35 Setting Command Targeting 36 Viewing Information Warnings and Error M...

Страница 7: ...liability Availability and Serviceability RAS 53 System Components 55 Bay Unit Numbering 57 Rack Numbering 57 Optional System Components 58 4 Rack Information 59 Overview 59 Tall Rack 40U 60 Locking the Front Door 62 Technical Specifications 63 5 CR brick 65 Overview 65 External Components 69 Front Panel Components 69 Rear Panel Components 71 Internal Components 72 Processors 73 Memory DIMMs 73 Pi...

Страница 8: ...k 87 Product Overview 88 External Components 90 Front Panel Components 90 Rear Panel Components 92 PCI and PCI X Card Configuration Guidelines 94 Important Installation Considerations 95 Supported PCI and PCI X Cards 95 PCI Card Carrier 96 Technical Specifications 96 8 R brick 97 Overview 97 External Components 100 Front Panel Components 100 Rear Panel Components 101 Technical Specifications 103 9...

Страница 9: ...minator Plug 118 Technical Specifications 119 Product Options 119 11 D brick2 Storage Module 121 D brick2 Overview 122 Functional Description 123 D brick2 Features 123 D brick2 External Components 123 Drive Carrier Modules and Dummy Modules 124 Power Supply Cooling Modules 125 Operator s Panel 126 Loop Resiliency Circuit LRC Modules 127 Powering the D brick2 On or Off 128 Powering On the D brick2 ...

Страница 10: ...a Drive Carrier Module 158 13 Troubleshooting and Diagnostics 161 Troubleshooting Chart 162 L1 Controller Error Messages 164 SGI Electronic Support 166 A Technical Specifications and Pinouts 169 System level Specifications 169 Physical Specifications 170 Environmental Specifications 171 Power Specifications 172 I O Port Specifications 173 Ethernet Port 174 Serial Ports 175 B Safety Information and...

Страница 11: ...007 4377 002 xi Chinese Class A Regulatory Notice 181 Korean Class A Regulatory Notice 181 Shielded Cables 182 Electrostatic Discharge 182 Laser Compliance Statements 183 Lithium Battery Statements 184 Index 185 ...

Страница 12: ......

Страница 13: ... Figure 3 1 SGI Altix 3700 Bx2 Series Systems 46 Figure 3 2 SGI Altix 3700 Bx2 Server System Example Configuration 48 Figure 3 3 Functional Block Diagram of CR brick 50 Figure 3 4 Altix 3700 Bx2 System Components Example 56 Figure 4 1 Tall 40U Rack 61 Figure 4 2 Locking the Front Door of the 40U Rack 62 Figure 5 1 Front and Side View of CR brick 66 Figure 5 2 CR brick Block Diagram 68 Figure 5 3 F...

Страница 14: ...s 122 Figure 11 2 D brick2 Drive Carrier Module and Dummy Module 124 Figure 11 3 Anti Tamper Locking on the D brick2 Disk Drive 125 Figure 11 4 D brick2 Operators Panel and Power Supply Cooling Modules 126 Figure 11 5 D brick2 Loop Resiliency Circuit LRC Module 127 Figure 12 1 Removing a Card Carrier 137 Figure 12 2 Extracting the Carrier Metal Filler Plate 138 Figure 12 3 Adjusting the Carrier Gu...

Страница 15: ...ting the Disk Drive Module in a D brick2 156 Figure 12 19 Locking the Drive Carrier Module 157 Figure 12 20 Unlocking the Disk Drive Module 158 Figure 12 21 Removing the Drive Carrier Module 159 Figure 13 1 Full Support Sequence 166 Figure A 1 Ethernet Port 174 Figure A 2 Serial Port 175 Figure B 1 VCCI Notice Japan Only 181 Figure B 2 Chinese Class A Regulatory Notice 181 Figure B 3 Korean Class ...

Страница 16: ......

Страница 17: ...2 R brick Port Specifications 103 Table 9 1 Power Supply LED States 110 Table 9 2 Power Bay Technical Specifications 111 Table 9 3 Power Supply Technical Specifications 111 Table 10 1 Status LEDs 115 Table 10 2 TP900 Storage Module Technical Specifications 119 Table 10 3 Configurable Items 119 Table 11 1 D brick2 Weight Information 130 Table 11 2 D brick2 Enclosure Power Specifications 131 Table 1...

Страница 18: ... 4377 002 Tables Table A 2 Altix 3700 Bx2 Physical Specifications 170 Table A 3 Environmental Specifications 171 Table A 4 Power Specifications 172 Table A 5 Ethernet Pinouts 174 Table A 6 Serial Port Pinout 176 ...

Страница 19: ... To avoid problems you must ask your SGI system support engineer SSE to perform all the set up addition or replacement of parts cabling and service of your SGI Altix 3700 Bx2 system with the exception of the following items that you can perform yourself Using your system console and your L1 controller to enter commands and perform system functions such as powering on and powering off as described ...

Страница 20: ...front and rear of the R brick Chapter 9 Power Bay describes the function and physical components of the power bay Chapter 10 SGI TP900 Storage Module provides a brief overview and functional description of the TP900 storage module Chapter 11 D brick2 Storage Module provides a brief overview and functional description of the D brick2 and also provides powering on and powering off instructions Chapt...

Страница 21: ... User s Guide P N 007 4304 xxx This fully illustrated guide explains how to operate and maintain the 2Gb SGI TP9400 and 2Gb SGI TP9500 Fibre Channel storage systems SGIconsole Hardware Connectivity Guide P N 007 4340 xxx This fully illustrated guide explains how to connect the SGIconsole to the various SGI server and graphics system configurations SGIconsole is a multi server management system tha...

Страница 22: ...et of Linux man pages formatted in the standard UNIX man page style Important system configuration files and commands are documented on man pages These are found online on the internal system disk or CD ROM and are displayed using the man command For example to display the man page for the xscsidisktest command type the following on a command line man xscsidisktest References in the documentation ...

Страница 23: ... files routines path names signals messages and programming language structures variable The italic typeface denotes variable entries and words or concepts being defined Italic typeface is also used for book titles user input This fixed space font denotes literal items that the user enters in interactive sessions Output is shown in nonbold fixed space font Brackets enclose optional portions of a c...

Страница 24: ...ment number is located at the bottom of each page You can contact SGI in any of the following ways Send e mail to the following address techpubs sgi com Use the Feedback option on the Technical Publications Library Web page http docs sgi com Contact your customer service representative and ask that an incident be filed in the SGI incident tracking system Send mail to the following address Technica...

Страница 25: ...n page 11 Installing Optional Components on page 13 Precautions Before operating your system familiarize yourself with the safety information in the following sections ESD Precaution on page 1 Safety Precautions on page 2 ESD Precaution Caution Observe all ESD precautions Failure to do so can result in damage to the equipment Wear an SGI approved wrist strap when you handle an ESD sensitive device...

Страница 26: ...er off the system only after the system software has been shut down in an orderly manner If you power off the system before you halt the operating system data may be corrupted Note A lithium battery is installed on the system I O card located in the IX brick Warning If a lithium battery is a soldered part only qualified SGI service personnel should replace this lithium battery For a battery of ano...

Страница 27: ...ies Boundary scan capabilities enable an SGI system support engineer SSE to test the functionality of your system Powering the System On and Off This section explains how to power on and power off individual bricks or your entire Altix 3700 Bx2 system as follows Powering On the System on page 4 Powering Off the System on page 8 For servers with a system console you can power on and power off indiv...

Страница 28: ...e that the cabling between the power distribution unit PDU and the wall power plug receptacle is secure 2 For each individual brick that you want to power on make sure that the PWR power switch is set to the 1 on position as shown in Figure 1 1 for the IX brick This will start the L1 controller s for the brick s when the brick s are powered on The 12 VDC LED on an individual brick illuminates gree...

Страница 29: ...Power Switch The standard configuration of the TP900 storage module has one power supply as shown in Figure 1 3 You can add an optional second power supply to the TP900 system to provide n 1 redundant power The second power supply is identical to the first power supply and would be located in the lower right section of the TP900 storage module Both power switches must be in the 1 on position to pr...

Страница 30: ...light green and that your controllers display that the system is powered on for each segment of the procedure which indicates that the power on procedure is proceeding properly If you have a problem while powering on and an error message appears on the L1 controller or the system console see your online log files and the information in L1 Controller Error Messages on page 164 to learn what the err...

Страница 31: ...and the SGIconsole Hardware Connectivity Guide P N 007 4340 00x for additional information on the optional SGI console Powering On in the L1 Mode The following sections describe how to power on your system in L1 mode The L1 controller display located on the front of each brick should display L1 running once the power on procedure starts Note If you have a problem while powering on and an error mes...

Страница 32: ... only has information about its attached I O brick and if another CR brick is attached to the I O brick information about that CR brick and its attached I O brick An I O brick only has information about its attached CR brick and an R brick only has information about itself The L1 controller display located on the front of each brick should display L1 running once the power on procedure starts Powe...

Страница 33: ...er 2 The following instructions describe how to power off your system in the L1 mode Powering Off in the L1 Mode The L1 controller display for each brick should display L1 running once the power off procedure starts Note If you have a problem while powering off and an error message appears on your console display see L1 Controller Error Messages on page 164 to learn what the error message indicate...

Страница 34: ... indicate potential future problems and then notifies the appropriate personnel This enables you and SGI system support engineers SSEs to proactively support systems and resolve issues before they develop into actual failures ESP enables users to monitor one or more systems at a site from a local or remote connection ESP can perform the following functions Monitor the system configuration events p...

Страница 35: ...ontroller s display at the front of each brick except the TP900 and D brick2 as shown in Figure 1 5 you can monitor brick items For example you can see if the fans on a particular brick are operating properly Figure 1 5 L1 Controller Display Location Example Reset L1 controller display NMI Service required LED Failure LED Power button with LED ...

Страница 36: ... tools such as VACM Console Manager and PCP to manage and monitor your system See the SGIconsole Start Here guide for descriptions of these tools and for references to other documents with information about these tools You can connect a dumb terminal to the CR brick console port If you have multiple CR bricks connect the dumb terminal to the bottom most CR brick in the rack This connection enables...

Страница 37: ...SSE to install any hardware items not listed in this section Warning Before installing operating or servicing any part of this product read the Safety Information on page 177 Adding or Removing PCI Cards The PCI X based I O system an industry standard for connecting peripherals to a processor is the primary I O system for the Altix 3700 Bx2 series server The IX brick and PX brick subsystems provid...

Страница 38: ...storage module that provides JBOD just a bunch of disks mass storage and that has up to 8 disk drives in each storage module See Replacing a TP900 Drive Carrier Module on page 149 for detailed instructions on installing or removing disk drives The D brick2 is a Fibre Channel based disk storage enclosure that provides JBOD mass storage and that has up to 16 disk drives in each storage enclosure See...

Страница 39: ...on page 25 L2 Emulator Operation on page 33 Upgrading L1 Firmware on page 43 The control system for the SGI Altix 3700 Bx2 series servers manages power control and sequencing provides environmental control and monitoring initiates system resets stores identification and configuration information and provides console diagnostic and scan interface Figure 2 1 shows an example system control network u...

Страница 40: ...0 Bx2 Server System Control Network Example Remote workstation monitor SGIconsole IX brick CR brick CR brick CR brick CR brick Local Area Network LAN Type A USB to Enet adapter Local Area Network LAN NUMAlink cable USB signals used Xtown2 RS 422 signals used ...

Страница 41: ... is interconnected to a CR brick via an SGI Xtown2 interface using RS 422 signals Optional SGIconsole The L1 controllers in the system report and share status information via the NUMAlink 4 cables thus maintaining controller configuration and topology information between all controllers and the system console For additional information on the SGIconsole options see SGIconsole Hardware Connectivity...

Страница 42: ...g LVDS An L1 controller of an I O brick communicates with an L1 controller of a CR brick via the Crosstown2 connections using serial RS 422 signals If a USB to Ethernet adapter is connected to either the CR or NUMAlink 4 R brick the L1 system controller on that brick emulates an L2 controller Access to the L2 functionality is made by way of an Ethernet connection Figure 2 2 diagrams the paths for ...

Страница 43: ...els NUMAlink 4 channels R brick R brick R brick R brick USB to Enet adapter Type A USB connector Ethernet bus Console Ethernet Ethernet NUMAlink 4 lvds signals used Xtown2 RS 422 signals used CR brick CR brick CR brick CR brick CR brick CR brick CR brick CR brick CR brick CR brick CR brick CR brick CR brick CR brick CR brick CR brick ...

Страница 44: ...he L1 controller performs Many of the L1 controller functions are common across all brick types however some functions are applicable to a specific brick type Table 2 1 L1 Controller Functions Function CR brick R brick IX brick PX brick Controls voltage regulator modules VRMs X X X X Controls voltage margining within the brick X X X X Controls and monitors fan speed X X X X Monitors voltage and re...

Страница 45: ... 21 Reports the population of the PCI cards and the power levels of the PCI slots X X Powers on the PCI slots and their associated LEDs X X Table 2 1 L1 Controller Functions continued Function CR brick R brick IX brick PX brick ...

Страница 46: ...t panel display contains the following items 2 x 12 character liquid crystal display LCD The display identifies the brick shows system status warns of required service and identifies a failed component Power On Off switch with LED button with light emitting diode LED Reset L1 controller display NMI Service required LED Failure LED Power button with LED ...

Страница 47: ...Ethernet switch is the preferred method of connecting remote support hardware to multiple systems The optional Ethernet switch provides eight Ethernet connectors Figure 2 4 shows sample connections between the Ethernet switch an SGIconsole a CR brick and other local SGI systems Figure 2 4 Ethernet Switch System Controller Block Diagram Example Ethernet switch To Ethernet connectors on other system...

Страница 48: ... control RTS CTS You can also connect a USB to Ethernet adapter to the CR brick or R brick and place it on a network This starts an L2 level emulator on the brick s system controller Use the telnet command to connect to the CR brick or R brick Table 2 2 provides information on connection points for the USB to Ethernet adapter depending on the system size Table 2 2 Location Guidelines for Connectin...

Страница 49: ...ware Connectivity Guide P N 007 4340 00x SGIconsole 2 x Start Here P N 007 4356 00x Console Manager for SGIconsole Administrator s Guide P N 007 4477 00x For more details on connecting a console to an Altix 3700 Bx2 series server see Connecting a System Console on page 3 For more information on monitoring your server see Monitoring Your Server on page 11 Operating the L1 Each CR brick and NUMAlink...

Страница 50: ...ach of the CR or NUMA 4 R bricks is a complete and fully functional system controller All the bricks are interconnected by NUMAlink 4 connectors and each brick shares its system control information with the others Console Mode from L1 Output from the system is visible and all input is directed to the system console Note The console mode from L1 mode is supported only if the system console is conne...

Страница 51: ... of the emulated L2 via the use of a USB to ethernet adapter see L2 Emulator Operation on page 33 for further details You can view a brick s configuration information with the config command as in the following 001c07 L1 config 0 001c07 LOC 1 001i11 U F 2 001p15 U G 001c05 L1 This example is a system with one CR brick and two I O bricks The number that follows the colon 0 1 and 2 from top to botto...

Страница 52: ...4 14 59 42 2MB image 001c05 L1 Viewing Information Warnings and Error Messages All information warnings and error messages generated by any of the system controllers are in the following form 001c05 ERROR invalid arguments for ver command try help ver The general format of the message includes a brick identification this is not present if the command was to the local brick only type of message and...

Страница 53: ...to all bricks 001c05 L1 power up 001c05 L1 power down 001c05 L1 reset You can power on and power off a specific brick with the power command 001c05 L1 power up 001c05 L1 This will power on or power off only the locally attached brick You can target other specific bricks for the power on or power off commands by preceding the power up down command with a specific rack and slot destination such as 0...

Страница 54: ... T and then enter the l1 command Ctrl T escaping to L1 system controller 001c05 L1 l1 L1 command processor engaged CTRL D for console mode 001c05 L1 L1 Console Selection The brick with which the L1 communicates in console mode is the system console or global master and you can view and set it with the select command By default the CR brick attempts to communicate with its local CPUs when console m...

Страница 55: ...d CR brick use the select rack slot command 001c05 L1 select r 2 s 1 console input 001c05 console console output not filtered 001c05 L1 To change the subchannel used on the selected brick use the select command followed by the subchannel number or the word console 001c05 L1 select sub 0A console input 001c05 CPU 0A console output not filtered 001c05 L1 During the boot process on a multi rack syste...

Страница 56: ...stem is not partitioned R bricks are not associated with a partition In the example shown above 001c05 is a CR brick in rack 001 and slot position 05 Viewing Information Warnings and Error Messages All information warnings and error messages generated by any of the system controllers are in the following form 001c05 ERROR invalid arguments for ver command try help ver The general format includes a...

Страница 57: ...s on the R brick but this is also the procedure to set the IP address on the CR brick when using a USB ethernet adapter in non R brick configurations Setting the IP address of the L2 emulator on the target bricks should be done before connecting the bricks to the network as follows Connect a serial cable to the console port on the target NUMAlink 4 R brick and get the L1 prompt At the L1 prompt ty...

Страница 58: ... The rackid on the L2 emulator cannot be set with the L2 rackid command instead it will be inherited from the local L1 As an example the L2 emulator running on the system controller in 1r14 will have a rack id of 114 rack 100 slot of the local L1 Once the L2 emulator is running you can telnet to the L2 emulator or use an optional SGIConsole Functionally it is the same as the separate hardware L2 b...

Страница 59: ...k This is similar to the output from using the config command on the L1 with the addition of the emulated L2 IP address and USB port number The structure of the brick s address is as follows a b c d x y where a b c d is the IP address of the emulated L2 In the example above the IP address is 127 0 0 1 x is the USB port number In the example above the port number is 0 y is the L1 index as follows 0...

Страница 60: ...pecifies racks 2 3 4 and 7 You can specify the slot list using the same nomenclature The slot number sometimes referred to as a bay number is the unit position number located on the rack slightly above where the bottom of the brick sits Each rack unit position number is located toward the top of the two lines that mark the unit position that the number represents For example the rack numbering for...

Страница 61: ...the L2 emulator the command is sent to these bricks Note In the current implementation if you add a brick to either rack 2 or 3 it would not be automatically included in the default destination You would need to reset the default destination L2 destination 002c05 127 0 0 1 0 2 003c05 127 0 0 1 0 0 L2 The following command resets the default destination to all racks and all slots L2 destination res...

Страница 62: ...mmand try help ver The general format includes a brick identification and the type of message followed by the message A message may be the result of an invalid command as shown in the example or the result of tasks running on the L1 such as the environmental monitor Each L1 has a log of local events Use the L1 command log to view events on any of the L1s Powering On Powering Off and Resetting the ...

Страница 63: ...will cause the operating system to reboot and all memory will be lost Console Mode from the L2 Emulator In console mode all output from the system is visible and all input is directed to the system To enter console mode from L2 press Ctrl D at the L2 prompt and observe the response L2 Ctrl D entering system console mode 001c05 console0 CTRL_T to escape to L2 system output appears here To return to...

Страница 64: ...fault console in the following order of priority The CR brick in the lowest numbered rack and slot which has produced console output and has an attached IX brick The CR brick in the lowest numbered rack and slot which has an attached IX brick The CR brick in the lowest numbered rack and slot The select command by itself shows the current console mode settings L2 select known system consoles non pa...

Страница 65: ...f the brick to be the system console enter the following L2 select subchannel 1A console input 003c01 console CPU1A console output no filtered During the boot process on a multibrick system there is a window of time in which the CR bricks are all producing output This can result in a somewhat jumbled output from the L2 emulator However you can filter console output so that the L2 emulator will sho...

Страница 66: ... to L2 emulation mode press Ctrl T 001c05 L1 Ctrl T escaping to L2 system controller CTRL T to send escape to L1 L2 At this point you can enter any L2 command Once the command is executed the L2 emulator returns to L1 mode re entering L1 mode 002c01 CTRL T to escape to L2 001c05 L1 To permanently engage the L2 emulation mode press Ctrl T and enter the l2 command 002c01 L1 Ctrl T escaping to L2 sys...

Страница 67: ...equirements on page 24 and Figure 2 2 on page 19 for descriptions of the hardware connections The L1 firmware consists of three parts Boot image A image B image At boot time the boot image validates the A and B image and if it is not instructed otherwise it executes the newer of the two images Because the L1 is running one of the two images the image not in use is the image that will be overwritte...

Страница 68: ...2 System Control You can update individual bricks by replacing all with a rack and slot number usr cpu firmware sysco flashsc 12 10 1 1 1 usr cpu firmware sysco l1 bin 1 19 This updates only the brick in rack 1 slot 19 ...

Страница 69: ...t your SGI sales or service representative for the most current information on this topic In a DSM system each processor board contains memory that it shares with the other processors in the system Because the DSM system is modular it combines the advantages of low entry level cost with global scalability in processors memory and I O You can install and operate the Altix 3700 Bx2 series system in ...

Страница 70: ...46 007 4377 002 3 System Overview Figure 3 1 SGI Altix 3700 Bx2 Series Systems ...

Страница 71: ... expand up to 512 Intel Itanium 2 processors a minimum of one IX brick is required for every 256 processors The system requires a minimum of one 40U tall rack with at least one power bay and one single phase PDU per rack The single phase PDU has two openings with three cables that extend from each opening to connect to the power bay The three phase PDU has two openings with six cables that extend ...

Страница 72: ... 3 System Overview Figure 3 2 SGI Altix 3700 Bx2 Server System Example Configuration c ix IX brick CR brick CR brick Power bay CR brick CR brick Power bay CR brick CR brick CR brick CR brick Power bay R brick R brick ...

Страница 73: ...ct is the CR brick A CR brick contains up to four processor nodes each processor node consists of a Super Hub SHub ASIC and two 64 bit processors with three levels of on chip secondary caches The two Intel 64 bit processors are connected to the SHUB ASIC via a single high speed front side bus The SHub ASIC is the heart of the CR brick This specialized ASIC acts as a crossbar between the processors...

Страница 74: ...stem On Service ReqÕd System Fault A B Node 0 Node 3 Brick Front Top Side Nodes Bottom Side Nodes Top Side Router Bottom Side Router SHUB 1 2 IP57 Node Card System Control XIO NI1 Power Pod Power Pod NI0 Node 2 SHUB 1 2 SHUB 1 2 DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot D...

Страница 75: ...into custom 19 inch racks The air cooled system has redundant hot swap fans at the brick level and redundant hot swap power supplies at the rack level Distributed Shared Memory DSM In the Altix 3700 Bx2 series server memory is physically distributed among the CR bricks compute router nodes however it is accessible to and shared by all CR bricks within the system A CR brick with configured memory b...

Страница 76: ... coherence protocol each block of memory 128 bytes has an entry in a table that is referred to as a directory Like the blocks of memory that they represent the directories are distributed among the compute nodes A block of memory is also referred to as a cache line Each directory entry indicates the state of the memory block that it represents For example when the block is not cached it is in an u...

Страница 77: ...ple speeds in all bricks except the optional R brick Speed increases automatically when temperature increases or when a single fan fails System monitoring System controllers monitor the internal power and temperature of the bricks and automatically shut down bricks to prevent overheating Memory L2 cache L3 cache and all external bus transfers are protected by single bit error correction and double...

Страница 78: ...referred to as power on diagnostics or PODs Processors and memory are automatically de allocated when a self test failure occurs Boot times are minimized Further RAS features Systems support partitioning Systems have a local field replaceable unit FRU analyzer All system faults are logged in files Memory can be scrubbed when a single bit error occurs ...

Страница 79: ...U high brick provides the boot I O functions and 12 PCI X slots PX brick This 4U high brick provides 12 PCI X slots on 6 buses for PCI expansion R brick This is a 2U high 8 port metarouter brick Power bay The 3U high power bay holds a maximum of six power supplies that convert 200 240 VAC to 48 VDC The power bay has eight 48 VDC outputs D brick2 This is a 3U high disk storage enclosure that holds ...

Страница 80: ...power bay per I O rack 40U Rack CR brick 0 or 8 processors Up to 48 memory DIMMs PX brick 0 to 12 PCI cards IX brick 1 base I O PCI card 0 to 11 PCI cards 2U 2U 3U 4U 4U 0 to 4 per system high bandwidth config 0 to 24 per system high capacity config 0 to 2 per system high bandwidth config 0 to 62 per system high capacity config Minimum qty 1 per system 1 to 8 per compute rack qty 0 to 7 per system...

Страница 81: ...he bottom unit U in which the brick resides For example in a tall 40U rack the CR brick positioned in U05 U06 and U07 is identified as C05 Rack Numbering A rack is numbered with a three digit number Compute racks are numbered sequentially beginning with 001 A compute rack is a rack that contains CR bricks I O racks are numbered sequentially and by the physical quadrant in which the I O rack reside...

Страница 82: ...I TP9300 TP9300S TP9500 or TP9500S 2 Gbit Fibre Channel D brick2 2 Gbit Fibre Channel Data servers SGI InfiniteStorage NAS 2000 Fileserver SGI SAN Server 2000 San Gateway Tape libraries STK L20 L40 L80 L180 and L700 ADIC Scalar 100 Scalar 1000 Scalar 10000 and ADIC Scalar i2000 Tape drives STK 9840B 9940B LTO SDLT and DLT Availability of optional components for the SGI 3700 Bx2 systems may vary ba...

Страница 83: ...acteristics of the tall 40 U racks in the following sections Overview on page 59 Tall Rack 40U on page 60 Technical Specifications on page 63 Overview At the time this document was published only the tall 40U rack shown in Figure 4 1 was approved for use with the Altix 3700 Bx2 systems ...

Страница 84: ...d on four casters the two rear casters swivel The base of the rack has leveling pads a ground strap and seismic tie downs The tall rack also has cable entry exit areas at the top bottom and sides of the rack I O and power cables pass through the bottom of the rack NUMAlink cables pass through the top and sides of the rack Cable management occurs in the rear of the rack One two or three power bays ...

Страница 85: ...Tall Rack 40U 007 4377 002 61 Figure 4 1 Tall 40U Rack ...

Страница 86: ... tall rack follow these steps 1 Close the front door 2 In the rear of the rack press in the door lock actuator and rotate the lever to point downward see Figure 4 2 Figure 4 2 Locking the Front Door of the 40U Rack Front door latch in unlocked position Front door latch in locked position ...

Страница 87: ...pecifications Table 4 1 lists the technical specifications of the tall rack Table 4 1 Tall Rack Technical Specifications Characteristic Specification Height 74 25 in 1885 95 mm Width 30 875 in 784 23 mm Depth 53 in 1346 2 mm Weight empty 425 lb 192 78 kg ...

Страница 88: ......

Страница 89: ...a 4U high enclosure that contains the compute and memory functionality for the SGI Altix 3700 Bx2 system Figure 5 1 on page 66 shows a front and side view of the CR brick The CR brick is divided into separate nodes that are connected internally by high speed NUMAlink channels Each node has two 64 bit processors and 12 memory DIMM slots connected to a custom designed application specific integrated...

Страница 90: ...66 007 4377 002 5 CR brick Figure 5 1 Front and Side View of CR brick ...

Страница 91: ... to 48 memory DIMM slots 12 slots per node Up to eight internal NUMAlink channels two per node Up to eight external NUMAlink channels two per node Two 1 2 GB s each direction Xtown2 I O channels One USB port for system controller support One serial console port One L1 controller and LCD display Three hot pluggable fans Figure 5 2 shows a block diagram of the CR brick ...

Страница 92: ...rvice ReqÕd System Fault A B Node 0 Node 3 Brick Front Top Side Nodes Bottom Side Nodes Top Side Router Bottom Side Router SHUB 1 2 IP57 Node Card System Control XIO NI1 Power Pod Power Pod NI0 Node 2 SHUB 1 2 SHUB 1 2 DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot DIMM Slot D...

Страница 93: ...red LED This LED illuminates orange to indicate that an item is broken or not operating properly for example a fan is off but the CR brick is still operating Failure LED This LED illuminates red to indicate that a system failure has occurred and the CR brick system is down Reset switch Press this switch to reset the internal processors of the CR brick All register values will be reset to their def...

Страница 94: ...70 007 4377 002 5 CR brick Figure 5 3 Front View of CR brick Service required LED Reset Failure LED NMI Power button with LED L1 controller display ...

Страница 95: ...r heartbeat LEDs turn on and off at a preset rate when the Linux operating system is running The heartbeat LEDs indicate that the processor is functioning and can process an interrupt from the operating system The LEDs for each node are arranged in pairs The lower LED represents the processor 0 heartbeat and the upper LED represents the processor 1 heartbeat Node 1 and Node 3 XIO II I O Interface ...

Страница 96: ...ought out to the rear of the brick Each node board contains two processors and 4 8 or 12 memory DIMMs The processors memory I O and network connections on each node board are interconnected via a SHub ASIC This ASIC acts a a crossbar between the processor interface local memory interface network interface and I O interfaces Note The internal components of the CR brick can be serviced only by train...

Страница 97: ...systems use commodity off the shelf memory DIMMs The DIMM card is a JEDEC standard 184 pin card Pinouts See I O Port Specifications on page 173 for pinout specifications for the non proprietary connectors on the CR brick Product Options See Table 5 1 for configurable items in the CR brick Note A CR brick with 0 processors on the internal node boards is generally referred to as a memory only brick ...

Страница 98: ...eater than n 1 All memory DIMMs within a memory bank group must be the same speed capacity and use the same chip technology Different logical banks within a CR brick can have different DIMM capacities and chip technologies Memory DIMMs must be added in groups of four DIMMs per node board Technical Specifications Table 5 2 lists the technical specifications of the CR brick Table 5 2 CR brick Techni...

Страница 99: ...sk drives and DVD ROM that are required to run your operating system or install other software applications or upgrades This chapter describes the function and physical components of the IX brick and it describes how to install and replace PCI cards and disk drives Specifically it includes the following information Product Overview on page 76 External Components on page 79 PCI and PCI X Card Confi...

Страница 100: ...s required for the base I O functions including system disk s and DVDROM Ethernet interface and serial connections Figure 6 1 Front View of the IX brick The IX brick offers the following features The IX brick supports both PCI and PCI X cards The IX brick can have up to four serial ports supported by the base I O card Additional optional serial ports reside on a daughtercard that you can purchase ...

Страница 101: ...base I O performance leave the PCI slot bus 1 slot 2 next to the base I O card empty or install only a 66 MHz PCI card Figure 6 2 IX brick Block Diagram Bus 6 slot 2 Bus 6 slot 1 Bus 5 slot 2 Bus 5 slot 1 Bus 4 slot 2 Bus 4 slot 1 Bus 3 slot 2 Bus 3 slot 1 Bus 2 slot 2 Bus 2 slot 1 Bus 1 slot 2 Crosstown2 XIO port Crosstown2 XIO port PIC ASIC Port F Port D XIO PIC ASIC Port F Port A XIO Port B XIO...

Страница 102: ...s This autonegotiating 10BaseT 100BaseT 1000BaseT twisted pair Ethernet port connects the system to an Ethernet network External SCSI connector This 68 pin VHDCI external SCSI port enables you to connect your system to an external SCSI device For an updated list of supported SCSI devices see the SGI Supportfolio at http support sgi com Real time interrupt input and output The RT interrupt input an...

Страница 103: ...ays status and error messages that the L1 controller generates Note For more information about the L1 controller see the SGI L1 and L2 Controller Software User s Guide On Off switch with LED Press this button to turn on the internal components of the IX brick Alternatively you can turn on the internal components at a system console When the internal components are on the LED illuminates green LEDs...

Страница 104: ...ry or damage to the IX brick the hot swappable fans can be installed only by a trained SGI system support engineer SSE Figure 6 3 Front Panel of IX brick On Off switch with LED Display Service required LED Failure LED DVD ROM System disk drives Drive bay cover ...

Страница 105: ...ower to the IX brick PCI X slots These slots seat the PCI cards See Adding or Replacing a PCI or PCI X Card on page 136 The card slots are numbered bus 1 through bus 6 Each bus has two slots labeled 1 and 2 as shown in Figure 6 4 PCI X slot LEDs Each PCI X slot has the following LEDs PWR power LED This LED illuminates green when the PCI card is installed securely and is getting power Fault LED Thi...

Страница 106: ...rt sgi com Ethernet RJ45 connector This autonegotiating 10 100 1000BaseT Ethernet port connects the system to an Ethernet network RTO and RTI stereo jack connectors Not used The RTO output and RTI input connectors are not supported under the current SGI Linux ProPack release Figure 6 4 Rear Panel of IX brick 48VDC 12VDC XIO 10 XIO 11 Serial port connector 1 Serial port connector 0 Ethernet connect...

Страница 107: ...e two 66 MHz cards bus 3 can have two 33 MHz cards and so on The PCI X bus supports 32 bit and 64 bit PCI or PCI X cards at the same time For maximum bandwidth PCI cards are distributed across all six buses when they are integrated at the factory This section contains the following information Important Installation Considerations on page 84 Supported PCI Cards on page 84 PCI Card Carrier on page ...

Страница 108: ...33 MHz PCI X cards reside on a bus the cards operate at 100 MHz in PCI X mode When two 66 MHz PCI X cards reside on a bus the cards operate at 66 MHz in PCI X mode When two 66 MHz PCI cards reside on a bus the cards operate at 66 MHz in PCI mode When one 66 MHz PCI card and one 66 MHz PCI X card reside on a bus the cards operate at 66 MHz in PCI mode When two 33 MHz PCI cards reside on a bus the c...

Страница 109: ... shipped any card that was ordered is installed with a carrier and any unoccupied slot is populated with an empty carrier A carrier must be present in an unpopulated slot to maintain an even airflow through the brick and to protect against electromagnetic interference EMI Technical Specifications Table 6 1 lists the physical specifications of the IX brick Table 6 1 Physical Specifications of the I...

Страница 110: ...tity Connector Type Power entry One 21 pin Foxcon XIO Two Proprietary 100 pin dual row The following external connectors are located on the base I O card SCSI connector One external 68 pin VHDCI external Ethernet One RJ 45 RT interrupt input and output One input and one output not used Stereo jack not supported under SGI ProPack for Linux ...

Страница 111: ...frequency This chapter describes the function and physical components of the PX brick and it provides guidelines for configuring PCI cards in the brick Specifically it includes the following information Product Overview on page 88 External Components on page 90 PCI and PCI X Card Configuration Guidelines on page 94 Technical Specifications on page 96 Note Throughout this chapter the term PCI card ...

Страница 112: ... connect an InfiniteReality graphics pipe to a CR brick Figure 7 1 Front View of the PX brick Three PIC PCI interface chip ASICs are key components of the PX brick architecture The PIC ASICs support the following see Figure 7 2 Two 1200 or 800 MB s Xtown2 XIO ports You can select the MB s setting with the L1 controller command XIO For more information see the SGI L1 and L2 Controller Software User...

Страница 113: ...IC ASIC Port F Port F Port A XIO Port B XIO Port F Port D XIO Port D XIO Port C XIO Port D XIO Bus 6 slot 2 Bus 6 slot 1 Bus 5 slot 2 Bus 5 slot 1 Bus 4 slot 2 Bus 4 slot 1 Bus 3 slot 2 Bus 3 slot 1 Bus 2 slot 2 Bus 2 slot 1 Bus 1 slot 2 Bus 1 slot 1 Crosstown2 XIO port Crosstown2 XIO port ...

Страница 114: ...rn on the internal components at an L2 controller display or at a system console When the internal components are on the LED illuminates green LEDs On Off switch LED This green LED illuminates when the internal components of the PX brick are on and turns off when they are off Service required LED This LED illuminates yellow to indicate that a component is broken or is not operating properly for ex...

Страница 115: ...External Components 007 4377 002 91 Figure 7 3 Front Panel and Display on the PX brick On Off switch with LED Display Service required LED Failure LED ...

Страница 116: ...his connector connects to a power bay which provides power to the PX brick PCI PCI X slots These slots support PCI or PCI X cards See Adding or Replacing a PCI or PCI X Card on page 136 The card slots are numbered bus 1 through bus 6 Each bus has two slots labeled 1 and 2 as shown in Figure 5 4 PCI X Slot LEDs Each slot has the following LEDs PWR power LED This LED illuminates green when the PCI c...

Страница 117: ...4 shows the location of the rear panel components of the PX brick Figure 7 4 Rear Panel of the PX brick 48VDC 12VDC XIO 10 XIO 11 Power switch One LED lights yellow the other lights green XIO Connectors PCI X slot LEDs 48 12VDC LEDs PWR Connector ...

Страница 118: ...Hz and 33 MHz cards at the same time The PCI PCI X bus supports 32 bit and 64 bit PCI or PCI X cards at the same time For maximum bandwidth PCI PCI X cards are distributed across all six buses when they are integrated at the factory This section contains the following information Important Installation Considerations on page 95 Supported PCI and PCI X Cards on page 95 PCI Card Carrier on page 96 F...

Страница 119: ... 133 MHz PCI X cards reside on a bus the cards operate at 100 MHz in PCI X mode When two 66 MHz PCI X cards reside on a bus the cards operate at 66 MHz in PCI X mode When two 66 MHz PCI cards reside on a bus the cards operate at 66 MHz in PCI mode When one 66 MHz PCI card and one 66 MHz PCI X card reside on a bus the cards operate at 66 MHz in PCI mode When two 33 MHz PCI cards reside on a bus the...

Страница 120: ...h an empty carrier A carrier must be present in an unpopulated slot to maintain an even airflow through the brick and to protect against electromagnetic interference EMI Technical Specifications Table 7 1 lists the physical specifications of the PX brick Table 7 2 shows the port specifications of the PX brick Table 7 1 Physical Specifications of PX brick Characteristic Specification Height 6 64 in...

Страница 121: ... R brick router brick is an eight port optional router that functions as a high speed switch to route network packets from one CR brick to another throughout the NUMAlink interconnect fabric Figure 8 1 shows a front view of the R brick The key component within the R brick is the router chip an SGI custom designed ASIC The router chip is an eight port crossbar that connects any input link channel t...

Страница 122: ... the following features Eight NUMAlink channels One USB port for system controller support One L1 controller and LCD display Two hot pluggable cooling fans Figure 8 2 shows a block diagram of the R brick L1 controller display Service required LED Failure LED Power button with LED ...

Страница 123: ... 8 2 R brick Block Diagram LVDS LVDS LVDS LVDS LVDS LVDS LVDS LVDS Port 5 Port 6 Port 3 Port 2 Port 1 Port 4 Port 8 Port 7 NL4 Router ASIC G H D F E C A B L2 host Type A USB connector USB System control board L1 console DB9 connector ...

Страница 124: ... each line On Off switch with LED Press this button to turn on the R brick internal components You can also turn on the R brick internal components at a system console Three LEDs On Off switch LED This LED illuminates green when the R brick internal components are on and turns off when they are off Service required LED This LED illuminates orange to indicate that an item is broken or not operating...

Страница 125: ... the internal components by pressing the On Off switch on the L1 controller panel Links R TO R connectors 1 6 7 and 8 or A F G and H These link connectors connect the R brick to other R bricks in the network fabric Links R to R and CR to R connectors 2 3 4 and 5 or B C D and E These link connectors normally connect the R brick to CR bricks however in larger systems they are also used to connect R ...

Страница 126: ...02 8 R brick Figure 8 3 Rear View of R brick 48VDC 12VDC CONSOLE L1 USB ELEC RATING 48VDC 1 1A 12VDC 1 2A DC IN 1 2 3 4 5 6 7 8 A B C D H G F E Power connector Console Power switch L1 USB Port A Port D Port E Port H ...

Страница 127: ...the specifications of the R brick ports Table 8 1 R brick Technical Specifications Characteristic Specification Height 3 3 in 83 82 mm Width 17 38 in 441 45 mm Depth 27 5 in 698 50 mm Weight 20 lb 9 1 kg Input power 48 VDC 60 W Table 8 2 R brick Port Specifications Port Quantity Peak Transfer Rate link 8 3 2 GB s each direction L1 1 12 Mbits s ...

Страница 128: ......

Страница 129: ...ay monitors controls and supplies AC power to the DPSs Although the power bay can hold a maximum of six DPSs in this system the power bay contains three or five DPSs The compute racks require power bays with five DPSs and the I O racks require power bays with three DPSs Each DPS inputs single phase AC voltage and outputs 950 W at 48 VDC and 42 W at 12 VDC The outputs of the DPSs are bused together...

Страница 130: ...ure 9 1 Front and Rear Views of Power Bay The power bay has the following features 3U high enclosure Six AC input connectors Eight DC output connectors Fault reset button Front view Rear view DP S 1 DP S 2 DP S 3 DP S 4 DP S 6 DP S 5 ...

Страница 131: ...els of the power bay Front Panel Components The front of the power bay seats the distributed power supplies see Figure 9 2 When the power bay contains three power supplies the supplies reside in locations 4 5 and 6 Figure 9 2 Front View of Power Bay Power supply 1 Power supply 6 Power supply 2 Power supply 3 Power supply 4 Power supply 5 ...

Страница 132: ...put ports of the power bay The power bay module also has six AC input connectors one for each power supply location For example when the power bay contains three power supplies connectors 4 5 and 6 connect to the power distribution strip PDS When the power bay contains six power supplies connectors 1 2 and 3 also connect to the PDS The reset button is used to reset the power bay when a fault condi...

Страница 133: ...ve air from the front of the rack to the rear of the rack Figure 9 4 shows front and rear views of a DPS Each DPS has the following LEDs Power Predictive fan fail PFAIL Power supply fail FAIL Figure 9 4 Front and Rear Views of the DPS Front view Rear view Predictive fan fail LED Power LED Power supply fail LED Output power connector Fans Input power connector ...

Страница 134: ... identifies the model and serial number of the supply The L1 controller of a connecting module reads this information Table 9 1 Power Supply LED States LED States Power Supply Condition Power Green PFAIL Amber FAIL Amber AC voltage not applied to all power supplies Off Off Off AC voltage not applied to this power supply Off Off On AC voltage present standby voltage on Blinking Off Off Power supply...

Страница 135: ...l Specifications Characteristic Specification Height 5 12 in 130 04 mm Width 17 5 in 444 5 mm Depth 23 87 in 606 3 mm Weight with five power supplies 64 5 lb 29 26 kg Input voltage 220 VAC Output voltage 12 VDC and 48 VDC Table 9 3 Power Supply Technical Specifications Characteristic Specification Height 5 in 127 mm Width 2 8 in 71 12 mm Depth 13 in 330 2 mm Weight 7 5 lb 3 40 kg Input voltage 220...

Страница 136: ......

Страница 137: ...bunch of disks storage for the SGI Altix 3700 Bx2 system The drive backplane of the TP900 connects the 8 drives on one SCSI bus As an option the storage system can also be configured on two SCSI buses 2 strings of 4 drives Note For more information about the TP900 storage module see the SGI Total Performance 900 Storage System User s Guide The TP900 storage module has the following features Suppor...

Страница 138: ...ich are discussed in the subsections that follow The front of the TP900 is 4 drive bays wide by 2 drive bays high The bays are numbered 1 and 2 from top to bottom and 1 to 4 from left to right See Figure 10 1 Figure 10 1 Front View of TP900 Storage Module The TP900 houses the following front panel components which are discussed in the subsections that follow Drive carrier module Dummy carrier modu...

Страница 139: ...i tamper lock that locks the handle in place therefore the carrier cannot be removed from the drive bay When the handle is locked a red indicator is visible in the handle When carrier is unlocked a white indicator is visible Dummy Carrier Module Dummy carrier modules are provided for installation in all unused drive bays They are designed as integral drive module front caps with handles They must ...

Страница 140: ...m top to bottom and 1 to 3 from right to left Figure 10 3 Rear View of TP900 Storage Module The TP900 rear bays house the following components which are discussed in the subsections that follow Power supply module Input output I O module Loopback module Cooling module Blank module Terminator plug FAULT OUTPUT GOOD FAULT OUTPUT GOOD Loopback module 1 3 I O module 1 1 Cooling module 1 2 Power supply...

Страница 141: ...l of the power supply The green LED indicates power output The amber LED indicates a power supply failure Input Output I O Module The I O module contains two VHDCI connectors and a switch to set the SCSI ID range The SGI TP900 storage module supports one I O module for a 1 x 8 configuration and 2 I O modules for a 2 x 4 configuration The 1 x 8 configuration creates a SCSI bus structure with 1 stri...

Страница 142: ...iguration the cooling modules operate together if one module fails the other module adequately cools the system until the faulty unit is replaced The front panel of the cooling module has two LEDs The green LED indicates that the cooling module is functioning properly The amber LED indicates a fan failure Blank Module Blank modules are placed in all vacant power supply and cooling module bays to m...

Страница 143: ...10 2 TP900 Storage Module Technical Specifications Characteristic Specification Height 3 37 in 85 7 mm Width 17 6 in 447 mm Depth 21 46 in 545 mm Weight Maximum configuration Empty enclosure 48 5 lb 22 kg 14 3 lb 6 5 kg Input power 100 254 VAC 175 W Table 10 3 Configurable Items Configurable Item Options Disk drives 1 minimum 8 maximum I O module 1 minimum 2 maximum Cooling module 1 minimum 2 maxi...

Страница 144: ......

Страница 145: ...User s Guide P N 007 4522 00x which is available online at http docs sgi com Note If you require RAID storage ask your SGI sales representative about SGI RAID storage products such as the SGI TP9400 and SGI TP9500 or TP9500S This chapter is divided into the following five major sections D brick2 Overview on page 122 D brick2 External Components on page 123 Powering the D brick2 On or Off on page 1...

Страница 146: ...f 16 disk drives and the component modules that handle I O power and cooling and operations Optional RAID storage systems are available for your system through your SGI sales representative Figure 11 1 shows front and rear views of the D brick2 Figure 11 1 D brick2 Front and Rear Views ID RS232 FC AL Loops RS232 FC AL Loops LRC I O module B LRC I O module A Front view Rear view Operator s panel PS...

Страница 147: ...ximum configuration of up to 96 drives six D brick2 units 1x16 more storage and 2x8 more bandwidth disk topologies in each brick Dual power feeds with dual power supplies Redundant cooling Non disruptive component replacement Enclosure services interface ESI for SCSI enclosure services SES D brick2 External Components This section discusses the external D brick2 components that you can access add ...

Страница 148: ...n and physical damage It also provides a means for thermal conduction Dummy drive carrier modules must be installed in all unused drive bays They are designed as integral drive module front caps with handles and must be fitted to all unused drive bays to maintain a balanced airflow Figure 11 2 shows examples Figure 11 2 D brick2 Drive Carrier Module and Dummy Module Note Ensure that the handle alw...

Страница 149: ...2 These modules supply redundant cooling and power to the enclosure and are auto ranging The supplies should be plugged into your SGI rack s power distribution strip PDS Four LEDs mounted on the rear panel of the D brick2 s PSU cooling module indicate the status of the power supply and the fans see Figure 11 4 on page 126 Note If a power supply fails do not remove it from the enclosure until you h...

Страница 150: ...ds for one second and the power on LED illuminates Figure 11 4 identifies all controls and indicators on the ops panel Note that the operator s panel configuration switches slide left for on and right for off For detailed descriptions of the LEDs and configuration switch information see Chapter 5 in the SGI Total Performance 9100 2Gb TP9100 Storage System User s Guide P N 007 4522 00x Power on off...

Страница 151: ...monitor internal functions These processors operate in a master slave configuration to allow failover See the SGI Total Performance 9100 2Gb TP9100 Storage System User s Guide P N 007 4522 00x for details Note The JBOD LRC I O module can address up to 96 disk drives a maximum of six enclosures can be cabled together The enclosure may be configured with either one or two LRC I O modules If only one...

Страница 152: ...rick2 follow these steps 1 Connect an AC power cord to each PSU cooling module 2 Connect the AC power cords to the system PDS 3 Turn the power switch on each PSU cooling module to the on position I on O off See Figure 11 4 on page 126 for switch locations The green PSU good LED illuminates Also the power on LED on the ESI ops panel see Figure 11 4 on page 126 of each module turns green when AC pow...

Страница 153: ...itch on the rear of each PSU cooling module to the off position position I on O off See Figure 11 4 on page 126 for switch locations 2 Unplug the power cable from the PSU cooling module s as appropriate For example you would follow this step if you are replacing an individual module 3 If you are shutting down multiple bricks in the system you may want to switch the PDS or PDU breaker switch to the...

Страница 154: ...llowing tables Note You should always follow system level technical operational and environmental specifications regardless of the tolerances of individual system bricks Table 11 1 shows the weights of the brick and the various D brick2 components Table 11 1 D brick2 Weight Information Component Weight D brick2 enclosure fully populated 32 3 kg 71 lb D brick2 enclosure empty 17 9 kg 39 4 lb Power ...

Страница 155: ...Voltage range for rack 200 240 VAC Voltage range selection Automatic Frequency 50 60 Hz Power factor 0 98 Maximum power consumption 700 VA Typical power consumption 400 VA or less Inrush current 25 C 77 F cold start 1 PSU 100 A maximum peak for 4 ms 25 A thereafter at maximum voltage Harmonics Meets EN61000 3 2 Power cord Cord type Plug Socket SV or SVT 18 WG minimum 3 conductor 250 V 10 A IEC 320...

Страница 156: ... C 80 F Storage temperature 1 C to 60 C 34 F to 140 F 8 to 80 noncondensing 29 C 84 F Shipping temperature 40 C to 60 C 40 F to 140 F 5 to 100 nonprecipitating 29 C 84 F Table 11 4 Additional Environmental Requirements for D brick2 Environmental Factor Requirement Altitude operating 0 to 3047 m 0 to 10 000 ft Altitude non operating 305 to 12 192 m 1000 to 40 000 ft Shock operating Vertical axis 5 ...

Страница 157: ... IX brick on page 144 Replacing a TP900 Drive Carrier Module on page 149 Installing or Replacing a D brick2 Drive Carrier Module on page 153 Maintenance Precautions and Procedures This section describes how to open the system for maintenance and upgrade protect the components from static damage and return the system to operation The following topics are covered Preparing the System for Maintenance...

Страница 158: ...n off the circuit breaker switches on each PDU Returning the System to Operation When you finish installing or removing components return the system to operation as follows 1 Turn each of the circuit breaker switches to the on position 2 At your system console enter the following command L1 power up 3 Verify that the LEDs turn on and illuminate green and that your controllers display that the syst...

Страница 159: ...l part of the chassis Caution Do not attempt to install or remove components that are not listed in Table 12 1 Components not listed must be installed or removed by a qualified SGI field engineer Table 12 1 lists the customer replaceable components and the page on which you can find the instructions for installing or removing the component Table 12 1 Customer replaceable Components and Maintenance...

Страница 160: ...tion provides instructions for adding or replacing a PCI or PCI X card in the IX brick or PX brick To maximize the operating efficiency of your cards be sure to read all the introductory matter in the PCI and PCI X Card Configuration Guidelines on page 94 before beginning the installation Caution To protect the PCI cards from ESD damage SGI recommends that you use a grounding wrist strap while ins...

Страница 161: ...ating system See your software guide if you need instructions to do this 2 Power off the I O brick by following the power off instructions in the Powering Off the System on page 8 3 To extract it from the slot pull the handle on the selected card carrier straight out as shown in Figure 12 1 Figure 12 1 Removing a Card Carrier ...

Страница 162: ...isting card from the carrier If you are adding a card extract the carrier metal filler plate by pushing down on it as shown in Figure 12 2 This filler plate covers the area where your card connectors will protrude out to the rear panel of the I O brick Figure 12 2 Extracting the Carrier Metal Filler Plate ...

Страница 163: ...that you want to install does not fit in the carrier follow these steps a Loosen the three screws on the guide bar with a Phillips screwdriver if your carrier uses T8 Torx screws you will need a Torx screw driver b Adjust the carrier guide bar also known as the alignment rail up or down as necessary see Figure 12 3 Figure 12 3 Adjusting the Carrier Guide Bar Carrier guide bar ...

Страница 164: ...d connectors protrude through the bulkhead where the metal filler plate was located The edge connectors fit between the bottom guides of the carrier The top of the card fits under the clips on the guide bar d Tighten the three screws on the guide bar Then skip step 6 and proceed to step 7 Figure 12 4 Mounting Card in Carrier Guides Bulkhead Clips ...

Страница 165: ... of the carrier guide bar c Place the half height PCI card in the carrier so that the connector edge of the card rests against the lower guide tab of the carrier see Figure 12 5 Figure 12 5 Mounting Half height PCI Card into Carrier d Move the carrier guide bar adjustment rail so that it holds the card firmly in place see Figure 12 6 Figure 12 6 Moving Carrier Guide Bar to Secure Half height PCI C...

Страница 166: ... screw holes of the bracket align with the empty screw holes of the carrier guide bar as shown in Figure 12 7 f Secure the bracket to the carrier guide bar with two T8 Torx screws g Tighten the screw that you loosened in step 6b Figure 12 7 Installing the Bracket to Secure a Half height PCI Card ...

Страница 167: ...slot guide as shown in Figure 12 8 Figure 12 8 Installing a Card in a Slot 8 Push horizontally on the carrier seating bar as shown in Figure 12 9 to seat the card securely in the slot Figure 12 9 Seating the Card in the Slot 9 Power on the I O brick by following the power on instructions in the Powering On the System on page 4 ...

Страница 168: ...ing or Replacing a Disk Drive in the IX brick The IX brick has two system disk drive bays as shown in Figure 12 10 This section describes how to install or remove the drives Figure 12 10 Location of SCSI Disk Drive Bays Installing a System Disk Drive Before you install a disk drive you must power off the IX brick according to the instructions in the Powering Off the System on page 8 Caution To pro...

Страница 169: ... drive install it in the rightmost bay 2 Position the drive assembly so that it engages the bay guide rails and then gently push the drive into the bay Do not use the locking handle to push the drive into the bay 3 Swing the locking handle towards the chassis until the locking handle engages the latch 4 Snap on the drive bay cover After you finish installing the disk drive power on the IX brick ac...

Страница 170: ...146 007 4377 002 12 Maintenance and Upgrade Procedures Figure 12 11 Installing a Disk Drive Locking handle 1 2 3 4 1 Drive bay cover ...

Страница 171: ...2 2 Remove the disk drive by depressing its handle lock with your thumb and pulling the handle away from the chassis until the handle disengages the disk drive connector from the backplane connector 3 Carefully slide the disk drive out of the bay and place it on an ESD safe surface Do not use the handle to pull the disk drive out of the bay Note When the IX brick has only one disk drive installed ...

Страница 172: ...148 007 4377 002 12 Maintenance and Upgrade Procedures Figure 12 12 Removing a Disk Drive 4 3 1 1 Drive bay cover Locking handle 2 ...

Страница 173: ...ace Any unused drive bays must be fitted with a dummy carrier module 1 Spin down the drive prior to removal using the operating system Caution Damage can occur to a drive if it is removed while still spinning If you cannot use the operating system to spin down the drives prior to removal perform all steps of the following procedure to ensure that the drive has stopped prior to removal Note The ant...

Страница 174: ...amper Lock 2 Release the carrier handle by pressing the latch in the handle towards the right 3 Gently withdraw the drive carrier module approximately 1 inch 25 mm and then wait 30 seconds for the drive to spin down 4 After 30 seconds withdraw the module from the drive bay Torx screwdriver ...

Страница 175: ...placement carrier by pressing the latch in the handle towards the right as shown in Figure 12 14 Insert the carrier into the enclosure Figure 12 14 Releasing the Carrier Handle Important Ensure that the carrier is oriented so that the drive is uppermost and the handle opens from the left ...

Страница 176: ...on the right of the carrier as shown in Figure 12 15 Figure 12 15 Carrier Camming Lever 7 Push the carrier handle into place the camming foot on the base of the carrier will engage in a slot in the enclosure Continue to push firmly until the handle fully engages You should hear a click as the latch engages and holds the handle closed TP900 Storage Module ...

Страница 177: ...tem must not be run without all modules in place Any unused drive bays must be fitted with a dummy carrier module Installing a Drive Carrier Module Note the following All disk drive bays must be filled with either a drive carrier module or a dummy drive module no bay should be left completely empty The drives in bays 1 1 and 4 4 of the D brick2 storage module are required for enclosure management ...

Страница 178: ...d if its anti tamper lock is activated locked outside the enclosure Drives are shipped with their locks set in the unlocked position However if a drive is locked insert the Torx screwdriver included with the disk drive into the socket in the lower part of the handle trim and turn it 90 degrees counterclockwise until the indicator visible in the center aperture of the handle shows black See Figure ...

Страница 179: ...arrier Module 007 4377 002 155 4 Open the handle of the replacement carrier by pressing the latch handle towards the right see Figure 12 17 Figure 12 17 Opening the Module Handle 5 Remove the dummy drive carrier module from the target drive bay ...

Страница 180: ...nserting the Disk Drive Module in a D brick2 7 Swing the drive handle shut and press it to seat the drive carrier module The camming lever on the right of the module will engage with a slot in the chassis Continue to push firmly until the handle fully engages with the module cap You should hear a click as the latch engages and holds the handle closed 8 Repeat steps 2 through 7 for all drive module...

Страница 181: ... Torx screwdriver in the drive module and turning it 90 degrees clockwise The indicator in the drive carrier module turns red when the drive is locked See Figure 12 19 Figure 12 19 Locking the Drive Carrier Module 10 Install dummy drive carrier modules in all empty drive bays The drive handle and camming mechanisms operate the same as those in a standard drive carrier module Torx screwdriver ...

Страница 182: ...ne at a time To remove a drive carrier module from the D brick2 enclosure follow these steps 1 Make sure that enough disk drives and dummy drives are available to occupy all bays 2 Ensure that users are logged off of the affected systems back up data if necessary 3 If the drive module is locked insert the Torx screwdriver into the anti tamper lock and turn it 90 degrees counterclockwise The indica...

Страница 183: ...5 mm and wait 30 seconds for the drive to stop spinning See Figure 12 21 Caution Damage can occur to a drive if it is removed while still spinning Figure 12 21 Removing the Drive Carrier Module 5 After 30 seconds withdraw the module from the drive bay Replace it immediately follow the instructions in Installing a Drive Carrier Module on page 153 sgi TP9100 sgi sgi sgi sgi sgi sgi sgi sgi TP9100 sg...

Страница 184: ......

Страница 185: ...3 13 Troubleshooting and Diagnostics This chapter provides the following sections to help you troubleshoot your system Troubleshooting Chart on page 162 L1 Controller Error Messages on page 164 SGI Electronic Support on page 166 ...

Страница 186: ...ick is on 1 position View the L1 display see Table 13 2 if an error message is present If the L1 controller is not running contact your SSE Check the connection between the brick and its power source The system will not boot the operating system Contact your SSE The Service Required LED illuminates on a CR brick an R brick an IX brick or a PX brick View the L1 display of the failing brick see Tabl...

Страница 187: ...r supply module The power supply module also has an amber LED that indicates a fault The Cooling Status LED of the TP900 is amber Contact your SSE to replace the cooling module The cooling module also has an amber LED that indicates a fault The amber LED of a disk drive is on Replace the disk drive Table 13 1 Troubleshooting Chart continued Problem Description Recommended Action ...

Страница 188: ...er off sequence for the brick ATTN power VRM description low fault limit reached x xxV 30 second power off sequence for the brick ATTN power VRM description high warning limit reached x xxV A higher than nominal voltage condition is detected ATTN power VRM description low warning limit reached x xxV A lower than nominal voltage condition is detected ATTN power VRM description level stabilized x xx...

Страница 189: ...et has exceeded 27 C ATTN temp sensor description critical temperature reached xxC xxF The ambient temperature at the brick s air inlet has exceeded 31 C ATTN temp sensor description fault temperature reached xxC xxF The ambient temperature at the brick s air inlet has exceeded 35 C Temperature stable message ATTN temp sensor description stabilized xxC xxF The ambient temperature at the brick s ai...

Страница 190: ...port integrates several services so they work together to monitor your system notify you if a problem exists and search for solutions to problems Figure 13 1 shows the sequence of events that occurs if you use all of the SGI Electronic Support capabilities Figure 13 1 Full Support Sequence Page or e mail alert 5 Supportfolio Online View the case solutions 2 e mail 6 Implement solution SGI customer...

Страница 191: ... tested solutions for possible fixes to the problem Solutions that are located in SGI Knowledgebase are attached to the service case 5 You and the SGI support engineers can view and manage the case by using Supportfolio Online as well as search for additional solutions or schedule maintenance 6 Implement the solution Most of these actions occur automatically and you may receive solutions to proble...

Страница 192: ...answers to questions that can be searched by sophisticated knowledge management tools You can log on to SGI Knowledgebase at any time to describe a problem or ask a question Knowledgebase searches thousands of possible causes problem descriptions fixes and how to instructions for the solutions that best match your description or question Supportfolio Online is a customer support resource that incl...

Страница 193: ...page 171 Power Specifications on page 172 I O Port Specifications on page 173 System level Specifications Table A 1 summarizes the Altix 3700 Bx2 system configuration ranges Table A 1 Altix 3700 Bx2 System Configuration Ranges Category Minimum Maximum Processors 8 512 CR brick DIMM capacity 16 48 System main memory capacity 8 GB 4 node bricks 6 TB 64 node bricks Available XIO I O channels 2 128 Nu...

Страница 194: ... rack including doors and side panels Height 74 25 in 1886 mm Width 30 88 in 784 mm Depth 53 27 in 1353 mm Shipping dimensions Height 80 in 2032 mm Width 41 5 in 1054 mm Depth 62 75 in 1594 mm Weight maximum Compute rack I O rack Disk rack 1400 lb 635 kg 1225 lb 556 kg 1400 lb 635 kg Shipping weight maximum Compute rack I O rack Disk rack 1785 lb 810 kg 1588 lb 720 kg 1785 lb 810 kg Access require...

Страница 195: ...86 F 1500 m to 3000 m 5000 ft to 10 000 ft Temperature tolerance non operating 40 C 40 F to 60 C 140 F Relative humidity 10 to 80 operating no condensation 10 to 95 non operating no condensation Heat dissipation to air Altix 3700 Bx2 rack 37 2 kBTU hr maximum based on 10 9 kW Cooling requirement Ambient air Air flow intake front exhaust rear Approximately 3200 CFM Maximum altitude 10 000 ft 3 049 ...

Страница 196: ...A 4 shows the power specifications for the system Table A 4 Power Specifications Feature Specification Single phase power requirements Voltage Frequency Power 200 240V 180 254 VAC min max 50 60 Hz 11 2 kW per rack Hold up time 20 ms Total harmonic distortion Less than 14 at full load ...

Страница 197: ...fications 007 4377 002 173 I O Port Specifications This section contains specifications and port pinout information for the base I O ports of your system as follows Ethernet Port on page 174 Serial Ports on page 175 ...

Страница 198: ...signments for the Ethernet port operating in 10 100 Base T mode and also operating in 1000Base T mode NU Not used Table A 5 Ethernet Pinouts Ethernet 10 100Base T Pinouts Gigabit Ethernet Pinouts Pins Assignment Pins Assignment 1 Transmit 1 Transmit Receive 0 2 Transmit 2 Transmit Receive 0 3 Receive 3 Transmit Receive 1 4 NU 4 Transmit Receive 2 5 NU 5 Transmit Receive 2 6 Receive 6 Transmit Rece...

Страница 199: ...res of the ports include the following Programmable data parity and stop bits Programmable baud rate and modem control Figure A 2 shows a serial port Figure A 2 Serial Port Pin 7 Request to Send RTS Pin 3 Transmit Data TXD Pin 2 Receive Data RXD Pin 4 Data Terminal Ready DTR not used Pin 5 Ground Pin 9 Ringing Indicator RI not used Pin 8 Clear to Send CTS Pin 6 Data Set Ready DSR not used Pin 1 Da...

Страница 200: ...ents for the 9 pin male DB 9 connector Table A 6 Serial Port Pinout Pin Assignment Description 1 DCD Data carrier detect 2 RXD Receive data 3 TXD Transmit data 4 DTR Data terminal ready 5 GND Signal ground 6 DSR Data set ready 7 RTS Request to send 8 CTS Clear to send 9 RI Ring indicator ...

Страница 201: ... may fall causing serious damage to the product 5 Slots and openings in the system are provided for ventilation To ensure reliable operation of the product and to protect it from overheating these openings must not be blocked or covered This product should never be placed near or over a radiator or heat register or in a built in installation unless proper ventilation is provided 6 This product sho...

Страница 202: ...per adjustment of other controls may result in damage and will often require extensive work by a qualified technician to restore the product to normal condition If the product has been dropped or the cabinet has been damaged If the product exhibits a distinct change in performance indicating a need for service 11 If a lithium battery is a soldered part only qualified SGI service personnel should r...

Страница 203: ...signia displayed on each device is an indication of conformity to the European requirements Caution This product has several governmental and third party approvals licenses and permits Do not modify this product in any way that is not expressly approved by SGI If you do you may lose these approvals and your governmental agency authority to operate this device CMN Number The model number or CMN num...

Страница 204: ...e radio frequency energy and if not installed and used in accordance with the instruction manual may cause harmful interference to radio communications Operation of this equipment in a residential area is likely to cause harmful interference in which case you will be required to correct the interference at your own expense If this equipment does cause harmful interference to radio or television re...

Страница 205: ...turbations radioélectriques dépassant les normes applicables aux appareils numériques de Classe A préscrites dans le Règlement sur les interferences radioélectriques établi par le Ministère des Communications du Canada VCCI Notice Japan Only Figure B 1 VCCI Notice Japan Only Chinese Class A Regulatory Notice Figure B 2 Chinese Class A Regulatory Notice Korean Class A Regulatory Notice Figure B 3 K...

Страница 206: ...he cable supplied with your system If your monitor cable becomes damaged obtain a replacement cable from SGI Electrostatic Discharge SGI designs and tests its products to be immune to the effects of electrostatic discharge ESD ESD is a source of electromagnetic interference and can cause problems ranging from data errors and lockups to permanent component damage It is important that you keep all t...

Страница 207: ...e exposition aux rayons Warning Vorsicht Unsichtbare Laserstrahlung Wenn Abdeckung geöffnet nicht dem Strahl aussetzen Warning Advertencia Radiación láser invisible al ser abierto Evite exponerse a los rayos Warning Advarsel Laserstråling vedåbning se ikke ind i strålen Warning Varo Lavattaessa Olet Alttina Lasersåteilylle Warning Varning Laserstrålning når denna del år öppnad ålå tuijota såteesee...

Страница 208: ...ning Advarsel Eksplosjonsfare ved feilaktig skifte av batteri Benytt samme batteritype eller en tilsvarende type anbefalt av apparatfabrikanten Brukte batterier kasseres i henhold til fabrikantens instruksjoner Warning Varning Explosionsfara vid felaktigt batteribyte Anvãnd samma batterityp eller en ekvivalent typ som rekommenderas av apparattillverkaren Kassera anvãnt batteri enligt fabrikantens ...

Страница 209: ...C cache coherency 52 carrier module replacement procedure 153 CMN number 179 command targeting L1 mode 28 L2 mode 36 console L1 mode power off 9 L1 mode power on 7 powering off bricks 9 powering on bricks 7 console communications 40 subchannels 40 console mode from L1 mode 30 from L2 emulator mode 39 console selection 40 controller L1 17 cooling module LED TP900 storage module 118 CR brick block d...

Страница 210: ...uted power supplies 109 front view 109 LEDs 109 rear view 109 technical specifications 111 documentation available via the World Wide Web xxii conventions xxiii man pages xxii drive carrier module antitamper lock disengaging 158 E embedded support partner ESP 168 system support 10 environmental specifications 171 error messages 164 ESI ops panel module 17 ESP embedded support partner system suppor...

Страница 211: ...f at console 9 powering on 29 powering on at console 7 resetting the brick 29 to console mode 30 viewing information warning and error messages 28 32 viewing system configuration 27 L1 software modes of operation 25 to L1 mode 26 L2 emulation mode to L1 mode 42 viewing system configuration 35 L2 emulator mode powering off 38 powering on 38 resetting system 38 setting command targeting 36 to consol...

Страница 212: ...170 pinouts Ethernet connector 174 serial connector 175 power connector TP900 storage module 117 power bay features 106 front panel components 107 front view 106 107 overview 105 rear panel components 108 rear view 106 108 technical specifications 111 power specifications 172 power supply LED TP900 storage module 117 powering off at system console 9 bricks and server 8 in L1 mode 9 L1 mode 29 L2 m...

Страница 213: ...with console communications 40 support embedded support partner ESP 10 Supportfolio Online 168 system architecture 49 system block diagram 50 system components Altix 3700 Bx2 server 56 list of 55 optional 58 system controller 15 system features 51 system overview 45 T tall rack features 60 locking front door 62 technical specifications CR brick 74 distributed power supplies 111 IX brick 85 power b...

Страница 214: ...190 007 4377 002 Index W warning and error messages viewing in L1 mode 28 warning messages viewing in L2 mode 32 38 X XIO 11 and XIO 10 IX brick 81 PX brick 92 ...

Отзывы:

Нет отзывов

Бренды по названию

Популярные бренды

Загрузить еще бренды