background image

SGI

®

10-Gigabit Ethernet Network

Adapter User’s Guide

007–4669–003

Summary of Contents for Ethernet Card

Page 1: ...SGI 10 Gigabit Ethernet Network Adapter User s Guide 007 4669 003 ...

Page 2: ... and or other countries worldwide Intel is a registered trademark of Intel Corporation Linux is a registered trademark of Linus Torvalds used with permission by Silicon Graphics Inc Neterion is a trademark and Xframe is a registered trademark of Neterion Inc UNIX is a registered trademark of The Open Group in the United States and other countries FCC WARNING This equipment has been tested and foun...

Page 3: ... Class A limits for radio noise emissions from digital apparatus as set out in the Radio Interference Regulations of the Canadian Department of Communications Attention Cet appareil numérique n émet pas de perturbations radioélectriques dépassant les normes applicables aux appareils numériques de Classe A préscrites dans le Règlement sur les interferences radioélectriques etabli par le Ministère d...

Page 4: ......

Page 5: ...New Features This revision adds support for the Release 2 card which is supported with SGI ProPack 4 Service Pack 3 and later releases 007 4669 003 v ...

Page 6: ......

Page 7: ...Record of Revision Version Description 001 May 2004 Initial publication 002 August 2005 Revision 003 July 2006 Revision 007 4669 003 vii ...

Page 8: ......

Page 9: ...res and Capabilities of the SGI 10 Gbit Ethernet Network Adapter 1 Features 2 Cabling 2 Configuration Limits 3 Tools 3 2 Connecting the Adapter to a Network 5 3 Operating the Adapter 9 Verifying Functionality 9 Using LEDs to Determine Functionality 9 Verifying Adapter Recognition 10 IRIX Adapter Recognition Verification 10 Linux Adapter Recognition Verification 11 Enabling the Adapter 12 007 4669 ...

Page 10: ...nfiguration 14 MTU Size Settings 14 4 Performance Tuning 17 IRIX Performance Tuning 17 Jumbo Frames and IRIX 17 Read Write Size and IRIX 18 Socket Buffer Size and IRIX 18 Multibuffer Mode and IRIX 18 SGI ProPack Performance Tuning 19 Socket Read and Write Buffer Sizes for SGI ProPack 19 Jumbo Frames and SGI ProPack 19 Read Write Size and SGI ProPack 19 TCP IP Socket Buffer Size and SGI ProPack 19 ...

Page 11: ...This guide assumes that you have general knowledge of Ethernet networks and the system in which the adapter is installed Important Information Warning Never look into the end of a fiber optic cable to confirm that light is being emitted or for any other reason Do not use any type of magnifying device such as a microscope eye loupe or magnifying glass Such activity causes cause a permanent burn on ...

Page 12: ... discusses performance tuning topics for IRIX and SGI ProPack for Linux Related Publications This guide is part of a document set that fully supports the installation operation and service of the adapter For more information about installing and servicing the adapter see the user s guide for the system in which the adapter is installed Also see the following IRIX Admin Networking and Mail Linux Co...

Page 13: ...system enter infosearch at a command line or select Help InfoSearch from the Toolchest On IRIX systems you can view release notes by entering either grelnotes or relnotes at a command line On Linux systems you can view release notes on your system by accessing the README txt file for the product This is usually located in the usr share doc productname directory although file locations may vary You...

Page 14: ...About this Guide user input This bold fixed space font denotes literal items that the user enters in interactive sessions Output is shown in nonbold fixed space font xiv 007 4669 003 ...

Page 15: ...our country Reader Comments If you have comments about the technical accuracy content or organization of this publication contact SGI Be sure to include the title and document number of the publication with your comments Online the document number is located in the front matter of the publication In printed publications the document number is located at the bottom of each page You can contact SGI ...

Page 16: ...s Guide Send mail to the following address Technical Publications SGI 1500 Crittenden Lane M S 535 Mountain View California 94043 1351 SGI values your comments and will respond to them promptly xvi 007 4669 003 ...

Page 17: ...ps The 10 Gbit Ethernet technology is targeted at backbone networks and interserver connectivity It provides an upgrade path for high end workstations that require more bandwidth than Gigabit Ethernet can provide The adapter is supported in the following systems SGI Altix XE SGI Altix 4700 SGI Altix 450 Silicon Graphics Prism SGI Altix 3000 SGI Altix 350 SGI Origin 350 Silicon Graphics Onyx 4 Sili...

Page 18: ...l voltage signaling 3 3 V and 5 V Compliance with PCI Local Bus Specification Revision 2 3 and PCI X Addendum to the PCI Local Bus Specification Revision 1 0a standards Optical cabling 850 nm 10GBASE SR TCP large send offload TCP large receive offload Release 2 only UDP large send offload Release 2 only Cabling The adapter is an LC connector and uses a 10GBASE SR transceiver at 850 nm It is connec...

Page 19: ...ult with your SGI representative to determine the currently supported maximum for your configuration Tools The SGI 10 Gbit Ethernet network adapter product contains the following IRIX tools xg 7F driver xgcmd 1M diagnostic tool to interrogate and control both the driver itself and the interfaces under its control The following standard Linux commands are useful with any Ethernet device ethtool 8 i...

Page 20: ......

Page 21: ...not be cleaned 2 Insert the LC connector on one end of the fiber optic cable into the adapter as shown in Figure 2 1 Ensure that the connector is inserted completely into the jack Note If the network connects to an Ethernet switch consult the operating manual for the switch to ensure that the switch port is enabled and configured correctly and as the correct adapter type 10GBASE SR 3 Insert the co...

Page 22: ...2 Connecting the Adapter to a Network Fiber optic connector Figure 2 1 Release 1 Card Fibre Optic Connections 6 007 4669 003 ...

Page 23: ...pter User s Guide Fiber optic connector Rx Tx Link Figure 2 2 Release 2 Card Fibre Optic Connections For information about configuring the interfaces see IRIX Admin Networking and Mail and The Network Administrators Guide 007 4669 003 7 ...

Page 24: ......

Page 25: ...14 Verifying Functionality This section explains the following Using LEDs to Determine Functionality on page 9 Verifying Adapter Recognition on page 10 Enabling the Adapter on page 12 Verifying that the Adapter is Properly Configured and Enabled on page 12 Using LEDs to Determine Functionality The SGI 10 Gbit Ethernet network adapter has light emitting diodes LEDs that indicate whether the adapter...

Page 26: ...r recognition for IRIX and Linux IRIX Adapter Recognition Verification Under IRIX the network interface name for the SGI 10 Gbit Ethernet network adapter is xgN where N is the number of the adapter 1 for the first adapter 2 for the second adapter if installed and so on Use the commands in the examples below to display the network interface names Note IRIX uses etc ioconfig conf to keep adapters an...

Page 27: ...root linux root sbin lspci 0011 01 01 0 Ethernet controller S2io Inc Xframe 10 Gigabit Ethernet PCI X rev 03 Note At this point before the module is loaded the proc net dev file will not display the device 2 Load the s2io module root linux root insmod s2io The device will now appear in the proc net dev file root linux root cat proc net dev grep eth1 eth1 0 0 0 0 0 0 0 0 root linux root The device ...

Page 28: ...abled IRIX Adapter Configuration Verification To verify that the network interface is configured properly and is enabled on enter the following on an IRIX system irix usr etc netstat ia Columns with the following headings should appear Name Mtu Network Address In the Name column the xg number should appear If it is followed by an asterisk the interface is disabled In the Mtu column the size of the...

Page 29: ...that the MAC address is 00 0C FC 00 01 24 the IP address is 10 2 2 2 the card is up and the MTU is 9600 root linux root ifconfig a eth1 Link encap Ethernet HWaddr 00 0C FC 00 01 24 MAC address inet addr 10 2 2 2 Bcast 10 2 2 255 Mask 255 255 255 0 IP address UP BROADCAST RUNNING MULTICAST MTU 9600 Metric 1 the card is up MTU RX packets 5 errors 0 dropped 0 overruns 0 frame 0 TX packets 5 errors 0 ...

Page 30: ...ation of your adapter use the ethtool 8 command For more information see the ethtool 8 man page MTU Size Settings The maximum transmission unit MTU size is controlled by the mtu mtu_size switch of the ifconfig command The most common MTU sizes are 1500 bytes standard size Ethernet frames and 9000 jumbo Ethernet frames The adapter supports an MTU size of up to 9600 bytes Configuring the adapter to ...

Page 31: ...mmand where N is the number of the adapter ifconfig xgN down 3 To change the MTU size of the 10 Gbit Ethernet adapter enter the following command where Nis the number of the adapter ifconfig xgN mtu 9600 4 To restart the adapter enter the following ifconfig xgN up 5 Enter the following command to verify that the MTU size has been changed netstat i The output should look similar to the following Na...

Page 32: ......

Page 33: ...X on page 17 Read Write Size and IRIX on page 18 Socket Buffer Size and IRIX on page 18 Multibuffer Mode and IRIX on page 18 There is no one tuning configuration that is best for all environments This section gives guidelines for the following cases A point to point configuration in which IRIX is transmitting to or receiving from IRIX A multiclient configuration in which an IRIX system is connecte...

Page 34: ...ize 1500 bytes 2048 KB 9000 bytes 4096 KB In a multiclient configuration use a socket buffer size of 2048 KB regardless of the MTU size tcp_delwake_count is a tunable parameter used to delay the wakeup of the receiving process for TCP input segments until a certain amount of data has been placed on the socket queue This count is the number of bytes to be accumulated in the socket receive buffer be...

Page 35: ...ximum transmission unit MTU is necessary for the best 10 Gbit Ethernet performance Generally the bigger the MTU the better The driver supports MTUs as large as 9600 bytes Read Write Size and SGI ProPack Applications should read large buffers from and write large buffers to the network for the best throughput and to reduce CPU utilization For example an application that uses recv 2 calls with 32 KB...

Page 36: ...e for each socket s buffers and can result in degraded network throughput Unless it is required to limit memory usage SGI recommends that you do not reduce these below the default SGI ProPack values If you to adjust the socket buffers use sysctl 8 command 20 007 4669 003 ...

Page 37: ... of 2 30 bits per second host Any system connected to the network hostname The name that uniquely identifies each host system on the network IP address A number that uniquely identifies each host system on a TCP IP network LED Light emitting diode a light on a piece of hardware that indicates status or error conditions MAC Medium access control also called the physical layer MAC address The physic...

Page 38: ...h that fans out to multiple clients via 1 Gbit Ethernet NIS Network Information Service a distributed database mechanism for user accounts host names mail aliases and so on PCI Peripheral Component Interconnect a bus specification The PCI bus is a high performance local bus used to connect peripherals to memory and a microprocessor Many vendors offer devices that plug into the PCI bus point to poi...

Page 39: ...onf 10 ethtool 3 F features 1 2 fiber connection 2 fibre optic connections 6 fibre type 3 frame sizes 2 frames 14 functionality verification 9 H hinv 11 I IEEE standards 2 ifconfig 3 12 14 installation 5 interface names 10 ip 3 IP address 12 IRIX performance tuning 17 J jumbo frames IRIX performance tuning 17 MTU sizes supported 14 SGI ProPack performance tuning 19 sizes supported 2 L LC connector...

Page 40: ...net core wmem_max 21 R read write size IRIX performance tuning 18 SGI ProPack performance tuning 19 Release 1 and Release 2 cards 5 resetting the adapter 13 rmem_max 21 S SGI ProPack performance tuning 19 signaling 2 socket buffer size IRIX performance tuning 18 SGI ProPack performance tuning 19 standard size Ethernet frames 14 supported systems 1 sysctl 20 T TCP IP and tuning 19 Tezro systems 1 t...

Page 41: ...SGI 10 Gigabit Ethernet Network Adapter User s Guide X xg driver 3 xgcmd diagnostic tool 3 007 4669 003 25 ...

Reviews: