manualshive.com logo in svg

Myricom Myrinet-2000, Руководство пользователя

Поделиться
Скачать
Myricom Myrinet-2000 Скачать руководство пользователя страница 20

© 2001, Myricom, Inc.

-- 20 --

Revision: 27 August 2001

All configurations larger than 16 hosts and up to 128 hosts escape the (page 12) ‘special case’
optimization of the M3-E16 with an M3-SPINE line card, and use all M3-SW16 line cards.
Indeed, the configuration of Myrinet-2000 switch networks for 16 < 

N

 

≤

 128, where 

N

 is the

number of hosts, is particularly simple.

First, count the number of hosts you wish to connect using each Myrinet PHY
(Fiber, Serial, SAN).  You’ll need one M3-SW16-8{F|S|M} line card for each 8
hosts of each PHY.  Then, pick an enclosure that will accommodate all of these
port line cards, plus as many spare slots as you may expect to add line cards when
you expand the cluster

3

.

Example 1:

 You are under contract to build a cluster with 20 hosts within a cabinet, for

which you have decided to use SAN links, and 6 external hosts connected by Fiber links.
You’ll need 3 M3-SW16-8M line cards, and 1 M3-SW16-8F line card, in an enclosure that
will accommodate at least 4 port line cards (M3-E32).  The cluster is for a fixed, embedded
application, and will not grow, and rack space is at a premium.  So, you order the following
components for the switch network:

Qty

Modular component

1

M3-E32 (enclosure)

1

M3-M monitoring card (highly recommended) or M3-BLANK

3

M3-SW16-8M (port line card with 8 SAN ports)

1

M3-SW16-8F (port line card with 8 Fiber ports)

Example 2:

 You plan to build a 64-host cluster that you expect to expand soon to 128 hosts, and

perhaps to an even larger size eventually.  Accordingly, you order the following components for
the switch network:

Qty

Modular component

1

M3-E128 (enclosure)

1

M3-M monitoring card (highly recommended) or M3-BLANK

8

M3-SW16-8F (port line card with 8 Fiber ports)

8

M3-BLANK (blank front panels)

Hosts and line cards can then be added incrementally up to 128 hosts.

Scenario 2a:

 The management (or sponsor) is so happy with the results from the 64-host and

then the 128-host cluster that they want you to expand it to 256 hosts.  Every 128-host switch
component, as well as interfaces and cables, can be re-used effectively in a 256-host
configuration (see the following sections).

                                    

3

 

We’re not trying to promote the sale of empty enclosure slots or of M3-BLANK panels.  Myricom sales records

show that 

most

 Myrinet clusters will be expanded during their lifetime, and often within months of their initial

installation.

Содержание Myrinet-2000

Страница 1: ...ack mounting holes conform to the EIA 310 standard for 19 inch racks Line cards and the fan tray may be hot swapped inserted or removed with the power on However Insert and remove line cards gently according to the instructions that start on page 4 Do not operate a switch for extended periods with missing line cards Use M3 BLANK panels to fill in any empty line card slots The front of the enclosur...

Страница 2: ... Myrinet 2000 switch products The M3 E16 M3 E32 M3 E64 and M3 E128 were certified in accordance with the IEC System for Conformity Testing and Certification of Electrical Equipment IECEE CB Scheme In addition TUV certified these products to the US and Canadian safety standards Electromagnetic Compatibility EMC Subject to the limitations listed below Myrinet 2000 switch products based on the M3 E16...

Страница 3: ... cables and connectors exhibit exceptionally high reliability Myrinet Fiber cables may be up to 200m in length Myrinet Fiber components and cables operate within Class A limits for the emission of electromagnetic interference EMI Myrinet Fiber links carry packet data at a 2 2 Gb s data rate on industry standard 50 125 multimode fiber pair cables with LC connectorization Myricom ships Fiber compone...

Страница 4: ... include a locking mechanism These locking springs should both be depressed when inserting and when removing a SAN cable end connector SAN links are for in cabinet applications only Switch configurations with any SAN line cards M3 SW16 8M or M3 SW16 4DM exceed Class A limits for EMI and should be used only within a shielding enclosure Also it is best to restrict the use of SAN cables to within an ...

Страница 5: ... the middle like this You ll notice from the illuminated LEDs that this switch is powered That s OK and why it s called hot swapping To remove the line card first press the red tabs to unlock its handles Then turn both handles outward together You should be able to see and feel the lever action as the front panel of the line card is ejected toward you It is normal that the ejection is a little sti...

Страница 6: ...arefully into the card guides The signal pins and ground blades on the high density connectors on the line card align themselves by means of alignment grounding posts on the backplane Conical depressions on the line card connectors center themselves on the posts to align the connectors with great precision Nevertheless it is best to insert the line cards slowly and gently When you use the lever ha...

Страница 7: ... be removed by loosening the two locking screws and pulling the fan tray out with the handle When a fan try is removed it should be replaced within approximately one minute or else line cards may power themselves off in response to an over temperature condition ...

Страница 8: ... 1 Introduction 9 2 The Family of Enclosures 11 3 Other Features of the Enclosures and Line Cards 12 4 Port Line Cards 14 5 Monitoring Line Card 17 6 Configurations up to 128 Hosts 19 7 Topology Concepts 21 8 Clos Networks 22 9 Clos Networks for more than 128 Hosts 23 ...

Страница 9: ... hosts 8 hosts 8 hosts 8 hosts 8 hosts 8 hosts 8 hosts 8 hosts 8 hosts 8 hosts 8 hosts 8 hosts 8 hosts 8 hosts Clos spreader network Ports to up to 128 hosts line cards The network pictured above provides routes from any host to any other host There is a unique shortest route between hosts connected to the same XBar16 The eight minimal routes between hosts connected to different XBar16s traverse t...

Страница 10: ...part of the backplane A port line card with connectors to the backplane on the back and connectors to external links on the front panel The topology is a full bisection Clos network with any combination of port line cards inserted or omitted For example you could plug 10 port line cards into the M3 E128 enclosure to support up to 80 hosts Ten of the ports on each of the 8 spine XBar16s would be us...

Страница 11: ...switches for the spine An XBar16 can serve the purpose of two 8 port switches with some additional communication provided free Thus a suitable topology for a 64 host Clos network of 16 port switches is 8 hosts 8 hosts 8 hosts 8 hosts 8 hosts 8 hosts 8 hosts 8 hosts 2 links each in which each of the thicker lines represents two links If we wanted to get theoretical we d point out that the paired li...

Страница 12: ...e powered off Other features of the enclosure products include a slot for an optional line card for monitoring and control of the switch This monitoring line card product code M3 M is the same size as the port line cards but has different backplane connectors such that it can be inserted only into the top line card slot The M3 M includes a big microcontroller that communicates externally via eithe...

Страница 13: ... flexibility manufacturing economies and most importantly easy expansion or reconfiguration of switches and switch networks already installed and the ability to repair switches by replacing just a line card or the fan tray Although the MTBF of Myrinet components is exceptionally high field failures do occur The monitoring line card allows nearly any fault to be identified unambiguously A line card...

Страница 14: ...ne cards or 4 legacy LAN ports M3 SW12 4L line card depending upon the Physical level conversion circuits The port numbers 8 15 8 11 for the M3 SW12 4L line card appear on the front panel silkscreen and correspond to the port numbering of the XBar16 Inasmuch as the native Physical level PHY of the XBar16 ports is SAN there are no conversion circuits for the M3 SW16 8M or the M3 SW16 4DM The only d...

Страница 15: ...ch port The port LEDs become green when a port is connected through a cable to an active port and will blink when traffic is flowing The M3 SW16 8S M3 SW16 8F and M3 SW12 4L also include a Status LED for the line card When a line card is inserted the Status LED shows green within 0 5 seconds if the line card has passed its self test and all voltages temperatures and internal status bits are at nom...

Страница 16: ...ny indicator LEDs The M3 SPINE 8F and M3 SPINE 8S port line cards are similar to their M3 SW16 counterparts except for the absence of the XBar16 µC Clocks Backplane Interface 0 1 2 3 4 5 6 7 Front panel Ports Serial link Sense Control Power hot swap circuits Physical Level Conversion Circuits dual 12V 0 1 2 3 4 5 6 7 The front panel ports are labeled 0 7 corresponding to the ports of the backplane...

Страница 17: ...nclosure via 26 separate and independent serial links 16 serial links to µCs on port line cards 8 serial links to µCs associated with backplane XBars 1 serial link to the fan monitoring µC and 1 serial link to its own small µC The front panel includes the usual Status LED green for operating and yellow for fault There are two LEDs for the 12V A and B internal power buses Each of the RJ 45 10 Base ...

Страница 18: ...page The following collection of screen snapshots from a web browser will give you some impression of the capabilities of the monitoring and of the web browser interface Home page of an M3 E32 Beginning of page for first port line card End of page for first port line card A part of the fan monitor page ...

Страница 19: ...odular assembly M3F SW16M Fiber switch M3 E16 M3 M M3 SPINE 8F M3 SW16 8F M3S SW16M Serial switch M3 E16 M3 M M3 SPINE 8S M3 SW16 8S M3M SW16M SAN switch M3 E16 M3 M 2 M3 SW16 8M There is no M3 SPINE 8M product due to signal integrity limitations on carrying SAN signals across backplanes and then across line cards to cables Hence the 16 port all SAN switch has a redundant layer of switching see pa...

Страница 20: ... space is at a premium So you order the following components for the switch network Qty Modular component 1 M3 E32 enclosure 1 M3 M monitoring card highly recommended or M3 BLANK 3 M3 SW16 8M port line card with 8 SAN ports 1 M3 SW16 8F port line card with 8 Fiber ports Example 2 You plan to build a 64 host cluster that you expect to expand soon to 128 hosts and perhaps to an even larger size even...

Страница 21: ...mum bisection suppose that we wished to connect 8 hosts and the largest crossbar switches available had 5 ports One way to connect the 8 hosts would be 4 hosts 4 hosts bisecting cut 1 link The bisecting cut that exhibits the minimum bisection is evident If the 4 hosts on the left were sending messages to the 4 hosts on the right and vice versa the total traffic handling capacity of the network wou...

Страница 22: ...s the path formation latency of Myrinet switches is much smaller than software latency the maximal or average diameter measured in switches traversed is a relatively unimportant metric of the network topology 8 Clos Networks Clos networks are named for Charles Clos who introduced them in a paper titled A Study of Non Blocking Switching Networks published in the Bell System Technical Journal in Mar...

Страница 23: ...k at this necessity is to consider a bisecting cut that crosses the 8 host links There is another adjacent bisecting cut across the 8 upper links Thus any structure used as a building block for larger networks must have this property of preserving link count and data rate The proof that the 128 host Clos network of 16 port switches is a rearrangeable network a stronger statement than that it exhib...

Страница 24: ...rk preserves 64 link data rate between the hosts and the deeper parts of the network fabric so that if these 64 hosts are communicating entirely with hosts outside their own group there are enough links to carry all the traffic This network is referred to as a Clos64 64 and it can be implemented with an M3 E128 enclosure with 8 M3 SW16 line cards and 8 M3 SPINE line cards In analogy to the structu...

Страница 25: ... enclosures and 40 M3 SPINE line cards If you refer to the port layout and numbering linked from http www myri com myrinet m3switch guide you will see that each M3 SPINE line card in an M3 E128 connects to the same port of all 8 backplane switches Thus if an M3 E128 is populated with 15 M3 SPINE line cards it provides 8 15 port switches which can be used as 24 5 port switches Two such units and an...

Отзывы:

Нет отзывов

Бренды по названию

Популярные бренды

Загрузить еще бренды