background image

Sun Microsystems, Inc.
www.sun.com

Submit comments about this document at:

http://www.sun.com/hwdocs/feedback

Sun

Dual 10GbE SFP+ PCIe

ExpressModule User’s Guide

Part No. 820-4979-10
August 2008, Revision A

Summary of Contents for 5945532

Page 1: ...Sun Microsystems Inc www sun com Submit comments about this document at http www sun com hwdocs feedback Sun Dual 10GbE SFP PCIe ExpressModule User s Guide Part No 820 4979 10 August 2008 Revision A ...

Page 2: ... Microsystems Inc 4150 Network Circle Santa Clara California 95054 Etats Unis Tous droits réservés Cette distribution peut comprendre des composants développés par des tierces parties Sun Sun Microsystems le logo Sun SunSolve Sun Fire et Solaris sont des marques de fabrique ou des marques déposées de Sun Microsystems Inc ou ses filiales aux Etats Unis et dans d autres pays UNIX est une marque dépo...

Page 3: ...stalling and Setting Up the Device Driver Software 7 Downloading and Installing the Driver on a Linux Platform 7 To Download the Driver for a Linux Platform 7 To Install the Driver for a Linux Platform 8 To Remove the Driver From a Linux Platform 9 Downloading and Installing the Driver on a Microsoft Windows Platform 10 To Download and Install the Driver on a Microsoft Windows Platform 10 To Remov...

Page 4: ...abit Ethernet Network for Linux Systems 21 To Boot Over the Network on Linux Systems 21 5 Configuring the Driver Parameters 23 Driver Overview 23 Driver Parameters for Linux Systems 24 Setting ixgbe Driver Parameters in Linux Systems 25 To Configure Jumbo Frames 25 6 Configuring VLANs 27 VLAN Overview 27 Configuring VLANs 30 To Configure VLANs in a Linux Environment 30 To Configure VLANs in a Micr...

Page 5: ...Contents v Physical Characteristics 37 Power and Environmental Requirements 38 Index 39 ...

Page 6: ...vi Sun Dual 10GbE SFP PCIe ExpressModule User s Guide August 2008 ...

Page 7: ...comply with the limits for a Class A digital device pursuant to Part 15 of the FCC Rules These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment This equipment generates uses and can radiate radio frequency energy and if it is not installed and used in accordance with the instruction manual it may cause harm...

Page 8: ...following statement is applicable to products shipped to Taiwan and marked as Class A on the product compliance label CCC Class A Notice The following statement is applicable to products shipped to China and marked with Class A on the product s compliance label GOST R Certification Mark T33012 ...

Page 9: ...injury if touched Caution Hazardous voltages are present To reduce the risk of electric shock and danger to personal health follow the instructions Depending on the type of power switch your device has one of the following symbols may be used On Applies AC power to the system Off Removes AC power from the system Standby The On Standby switch is in the standby position Modifications to Equipment Do...

Page 10: ...es Evitez tout contact Les surfaces sont brûlantes Vous risquez de vous blesser si vous les touchez Attention Tensions dangereuses Pour réduire les risques de décharge électrique et de danger physique observez les consignes indiquées Selon le type d interrupteur marche arrêt dont votre appareil est équipé l un des symboles suivants sera utilisé Marche Met le système sous tension alternative Arret ...

Page 11: ...ng Achtung Gefahr von Verletzung und Geräteschaden Befolgen Sie die Anwei sungen Achtung Heiße Oberfläche Nicht berühren da Verletzungsgefahr durch heiße Oberfläche besteht Achtung Gefährliche Spannungen Befolgen Sie die Anweisungen um Stromschläge und Verletzungen zu vermeiden Je nach Netzschaltertyp an Ihrem Gerät kann eines der folgenden Symbole verwendet werden Ein Versorgt das System mit Wech...

Page 12: ...de que se produzcan lesiones personales y daños en el equipo Siga las instrucciones Precaución Superficie caliente Evite todo contacto Las superficies están calientes y pueden causar lesiones personales si se tocan Precaución Voltaje peligroso Para reducir el riesgo de descargas eléctricas y lesiones personales siga las instrucciones En función del tipo de interruptor de alimentación del que dispo...

Page 13: ...directive 2002 95 EC Compliance Model Number 5945532 Product Family Name Sun Dual 10GbE SFP PCIe ExpressModule As Information Technology Equipment ITE Class A per as applicable EN 55022 2006 Class A EN 61000 3 2 2000 A2 2005 Pass EN 61000 3 3 1995 A1 2001 Pass EN 55024 1998 A1 2001 A2 2003 Required Limits IEC61000 4 2 4 kV Direct 8 kV Air IEC61000 4 3 3 V m IEC61000 4 4 1 kV AC Power Lines 0 5 kV ...

Page 14: ...xiv Sun Dual 10GbE SFP PCIe ExpressModule User s Guide August 2008 ...

Page 15: ...it and 32 bit systems manufactured using processors compatible with the AMD64 Intel Xeon or Intel Pentium product families How This Document Is Organized Chapter 1 describes the Sun 10GbE XFP SR PCI Express Card hardware and software Chapter 2 explains how to download install and verify the ixgbe device driver software on Linux and Microsoft Windows systems Chapter 3 describes how to install the E...

Page 16: ...ames of commands files and directories on screen computer output Edit your login file Use ls a to list all files You have mail AaBbCc123 What you type when contrasted with on screen computer output su Password AaBbCc123 Book titles new words or terms words to be emphasized Replace command line variables with real names or values Read Chapter 6 in the User s Guide These are called class options You...

Page 17: ... or alleged damage or loss caused by or in connection with the use of or reliance on any such content goods or services that are available on or through such sites or resources Application Title Part Number Format Location Release Notes Sun 10GbE XFP SR PCI Express Card Sun Dual 10GbE XFP 2 SR PCI Express Card and Sun Dual 10GbE SFP PCIe ExpressModule Release Notes 820 4505 PDF HTML Online Getting...

Page 18: ...erested in improving its documentation and welcomes your comments and suggestions You can submit your comments by going to http www sun com hwdocs feedback Please include the title and part number of your document with your feedback Sun Dual 10GbE SFP PCIe ExpressModule User s Guide part number 820 4979 10 ...

Page 19: ...page 5 Patches and Updates on page 6 Shipping Kit Contents The carton in which your Sun 10GbE XFP SR PCI Express Card was shipped should contain the following items Sun 10GbE XFP SR PCI Express Card Sun Dual 10GbE SFP PCIe ExpressModule Getting Started Guide Safety and Compliance Manual ExpressModule Hardware Overview The Sun 10GbE XFP SR PCI Express Card is a 10 Gigabit Ethernet 10GbE fiber netwo...

Page 20: ...ules The product conforms to the IEEE 802 3 standard and supports standards for system manageability and power management FIGURE 1 1 Sun Dual 10GbE SFP PCIe ExpressModule ExpressModule Features The Sun 10GbE XFP SR PCI Express Card provides the following features and benefits Intel 82598EB 10 Gigabit Ethernet controller Load balancing on multiple CPUs Intel I O Acceleration Technology I OAT iSCSI ...

Page 21: ...for SFP form factor transceivers RoHS compliant lead free technology Intel PROSet Utility for Windows Device Manager Indicator Lights on the ExpressModule On the front panel of the ExpressModule shown in FIGURE 1 2 next to each port there are viewing holes for two lights dedicated to that port FIGURE 1 2 Front Panel Lights and Attention Switch TABLE 1 1 explains the meaning of the lights for each ...

Page 22: ...r 1GbE link TABLE 1 2 LED Indicator Lights on the ExpressModule LED Color LED State Meaning Action Green Off Power off Insertion or removal of add in cards is permitted All supply voltages except Vaux have been removed from the slot if required for add in card removal Green On Power on The slot is powered on Insertion or removal of add in cards is not permitted Green Blinking Power transition The ...

Page 23: ... Requirements You must install the latest version of the patch from the following web site http sunsolve sun com If the patch is not available on the SunSolveSM web site contact your local sales or service representative TABLE 1 3 Hardware and Software Requirements Requirements Supported Products Hardware Sun Blade X6220 Server Module Sun Blade X6250 Server Module Sun Blade X6450 Server Module Sun...

Page 24: ...atches and Updates Check the Sun Update Connection to ensure that you have the latest recommended patch clusters and security patches You can download the latest recommended patch clusters and security patches at http sunsolve sun com pub cgi show pl target patchpage ...

Page 25: ...and Installing the Driver on a Microsoft Windows Platform on page 10 Downloading and Installing the Driver on a Linux Platform If your system uses the RedHat or SuSe Linux operating system you must download the ixgbe device driver to install it To Download the Driver for a Linux Platform 1 Log in to your system 2 With a browser go to this location http support intel com support network adapter 10g...

Page 26: ... for 10 GbE PCI E Based Network Connections for Linux The download begins The file named ixgbe 1 3 16 1 tar gz is saved in the Desktop directory of your system To Install the Driver for a Linux Platform 1 Copy the file containing the driver from Desktop to temp The file is named ixgbe 1 3 16 1 tar gz 2 Uncompress and untar the file with this command 3 Go to the newly created src directory 4 Compil...

Page 27: ...with this modinfo command For example the output might be the following To Remove the Driver From a Linux Platform Use the rmmod command lsmod grep ixgbe ixgbe 118052 0 modinfo ixgbe grep ver filename lib modules 2 6 18 53 el5 kernel drivers net ixgbe ixgbe ko version 1 3 16 1 lro description Intel R 10 Gigabit PCI Express Network Driver srcversion 5CFF6AEBA251050F8A4B746 vermagic 2 6 18 53 el5 SM...

Page 28: ...ad drivers and software 5 Select one of the following as the operating system For a 64 bit driver Windows Server 2003 Standard x64 Edition For a 32 bit driver Windows Server 2003 Standard Edition 6 Locate one of the following and select Download next to it For a 64 bit driver Network Adapter Driver for Windows XP Professional x64 Edition or Windows Server 2003 x64 Edition For a 32 bit driver Netwo...

Page 29: ... instructions in the installation wizard 11 If the Found New Hardware Wizard screen is displayed click Cancel The autorun automatically runs after you have extracted the files To Remove the Driver From a Microsoft Windows Platform 1 From the Control Panel double click Add Remove Programs 2 Select Intel PRO Network Connections Drivers 3 Click Add Remove 4 When the confirmation dialog displays click...

Page 30: ...12 Sun Dual 10GbE SFP PCIe ExpressModule User s Guide August 2008 ...

Page 31: ...9 Installing an Optical Transceiver The Sun 10GbE XFP SR PCI Express Card requires a short range optical transceiver in at least one port to create an Ethernet connection The short range optical transceiver part number X5561A Z is available from Sun Microsystems Note Install the optical transceivers into the ExpressModule before installing the ExpressModule into the system To Install an Optical Tr...

Page 32: ...r by the edges align the transceiver with the slot in the ExpressModule and slide the transceiver into the opening 3 Applying even pressure at both corners of the transceiver push the transceiver until it is firmly seated in the slot 4 Push the handle closed to lock the optical transceiver in place ...

Page 33: ...al transceiver Caution If you pull the locking handle down when the optical transceiver is installed remove the optical transceiver entirely and reinstall it The handle operates an internal lock Pulling the handle down can disconnect the optical transceiver even though it might appear to be connected ...

Page 34: ...essModule in your chassis all ExpressModule slots must be filled with either operating ExpressModules or filler panels To Install the ExpressModule With the Power Off 1 Halt and power off your system 2 Power off all peripherals connected to your system 3 Attach the adhesive copper strip of the antistatic wrist strap to the metal casing of the power supply Wrap the other end twice around your wrist...

Page 35: ...Align the ExpressModule with the vacant ExpressModule slot 1 in the following figure Ensure that the ExpressModule s indicator lights on the front panel are facing toward you and that the ExpressModule ejector lever on the bottom is fully opened ...

Page 36: ...he ExpressModule s internal connection to the chassis midplane 8 Complete the installation by closing the ejector lever until the latch snaps into place 3 in the preceding figure Caution Do not use excessive force when installing the ExpressModule into the slot You might damage the ExpressModule s connector If the ExpressModule does not seat properly when you apply even pressure remove and careful...

Page 37: ...rify the installation To Verify the Installation in a Linux System Verify the new network interface instances corresponding to the Sun 10GbE XFP SR PCI Express Card To Verify the Installation in a Microsoft Windows System 1 Click on Control Panel ifconfig a grep eth eth3 Link encap Ethernet HWaddr 00 1B 21 17 67 B0 eth4 Link encap Ethernet HWaddr 00 1B 21 17 67 9B ...

Page 38: ...98EB 10 Gigabit AF Dual Port Network Connection will be displayed at the Network Connection window screen if the driver is installed successfully 3 To check the driver version use the Administration Tool The minimum Windows Server 2003 driver version is 1 2 22 0 4 In the Administration Tool click Computer Management Device Manager and Network Adapter ...

Page 39: ...port That port is the logical Port 0 and it has the lowest MAC address Configuring the Network Host Files for Booting Over the Gigabit Ethernet Network for Linux Systems To Boot Over the Network on Linux Systems 1 Obtain the MAC address of the first Sun 10GbE XFP SR PCI Express Card port by checking the label of the card The MAC address of the first port should be the MAC address from the label pl...

Page 40: ...et adapter 11 After the Linux OS install completes use the BIOS to change the boot device priority to Boot from Hard Disk in order to boot up the newly installed OS Unless the boot device priority is changed the OS installation process will repeat image pxe mac addr PXE E61 Media test failure check cable PXE MOF Exiting Intel Boot Agent NVIDIA Boot Agent 217 0513 Copyright C 2001 2005 NVIDIA Corpo...

Page 41: ...vice in your system This chapter lists the available device driver parameters and describes how you can set these parameters Driver Overview on page 23 Driver Parameters for Linux Systems on page 24 Setting ixgbe Driver Parameters in Linux Systems on page 25 Driver Overview Each ixgbe channel provides 10000BASE T networking interfaces The ixgbe driver is capable of supporting 10000 Mbit sec full d...

Page 42: ...ng on the MTU setting When the MTU size is 1500 or less the receive buffer size is 2048 bytes When the MTU is greater than 1500 the receive buffer size will be either 4056 8192 or 16384 bytes The maximum MTU size is 16114 RxIntDelay 0 to 65535 0 off 72 This value delays the generation of receive interrupts in units of 0 8192 microseconds Receive interrupt reduction can improve CPU efficiency if pr...

Page 43: ...Jumbo Frames Jumbo Frames can support up to 15000 MTU The default value is 1500 MTU Use the ifconfig command to increase MTUs to allow transmission of Jumbo Frames For example where the IP address for eth7 is 192 1 1 200 the following command increases MTUs to the maximum ifconfig eth7 192 1 1 200 mtu 15000 up ...

Page 44: ...26 Sun Dual 10GbE SFP PCIe ExpressModule User s Guide August 2008 ...

Page 45: ...boot you might experience configuration problems VLAN Overview With multiple VLANs on a card a server with a single card can have a logical presence on multiple IP subnets By default 128 VLANs can be defined for each VLAN aware card on your server However this number can be increased by changing the system parameters If your network does not require multiple VLANs you can use the default configura...

Page 46: ...l broadcast domains or separate IP subnets it can be useful for a server to have a presence on more than one VLAN simultaneously Several Sun products support multiple VLANs on a per port or per interface basis allowing very flexible network configurations FIGURE 6 1 shows an example network that uses VLANs FIGURE 6 1 Example of Servers Supporting Multiple VLANs With Tagging Adapters The example ne...

Page 47: ...in broadcast separation between all of those subnets The following list describes the components of this network The Accounting Server is available to only VLAN 3 The Accounting Server is isolated from all traffic on VLANs 1 and 2 The switch port connected to the server has tagging turned off Clients 1 and 2 are attached to a shared media hub that is then connected to the switch Clients 1 and 2 be...

Page 48: ... address The tag header consists of two bytes of Ethernet Tag Protocol identifier TPID 0x8100 and two bytes of tag control information TCI FIGURE 6 2 shows the Ethernet tag header format FIGURE 6 2 Ethernet Tag Header Format By default a single VLAN is configured for every port which groups all ports into the same broadcast domain just as if there were no VLANs at all This means that VLAN tagging ...

Page 49: ...xx xx xxx the IP address of the interface To Configure VLANs in a Microsoft Windows 2003 Environment 1 Click Control Panel 2 Click Network Connection 3 Click the folder icon from the sub manual bar 4 Right click the Sun 10GbE XFP SR PCI Express Card port then select Properties 5 Click Configure 6 Click VLAN then click on New 7 Type VLAN with ID for example type Vlan10 8 Click Internet Protocol TCP...

Page 50: ...Bonding for Multiple Interfaces To Configure Bonding for Multiple ixgbe Interfaces 1 Use the modprobe command to configure the mode In this command max_bonds is the number of bond interfaces to be created mode specifies the bonding policy This example uses balance rr 2 Use the ifconfig command to create the bond In this command bond0 is the bonding device 3 Configure the bondn interfaces modprobe ...

Page 51: ...formation To Remove Bonding Use the rmmod command to remove bonding modprobe bonding mode balance rr miimon 100 max_bonds 2 ifconfig bond0 192 2 2 15 netmask 255 255 255 0 broadcast 192 2 2 255 ifenslave bond0 eth4 eth5 ifconfig bond1 193 2 2 15 netmask 255 255 255 0 broadcast 193 2 2 255 ifenslave bond1 eth6 eth7 rmmod bonding ...

Page 52: ...34 Sun Dual 10GbE SFP PCIe ExpressModule User s Guide August 2008 ...

Page 53: ...ns This appendix lists the specifications for the Sun 10GbE XFP SR PCI Express Card This appendix contains the following sections Connectors on page 36 Technical Features on page 37 Physical Characteristics on page 37 Power and Environmental Requirements on page 38 ...

Page 54: ...un Dual 10GbE SFP PCIe ExpressModule User s Guide August 2008 Connectors FIGURE A 1 shows the connectors for the Sun 10GbE XFP SR PCI Express Card FIGURE A 1 Sun Dual 10GbE SFP PCIe ExpressModule Connectors ...

Page 55: ... 10 Gigabit Gb Bus type PCI Express 2 0 Bus width x8 lane PCI Express Conforms to Ethernet Standard 802 3 Boot ROM 2 Mbit SPI Flash Electromagnetic Interference EMI FCC Class A TABLE A 2 Physical Characteristics Dimension Measurement Length 6 62 in 168 2 mm Width 4 25 in 108 mm Power LED green Attention LED yellow Attention button recessed Activity LED each port Green 10 GbE link Yellow 1 GbE link...

Page 56: ...ower Requirements Specification Measurement Typical power consumption 14W 1 17A at 12V dual port Main host power supply 12 V 15 Operating temperature 35 to 70 C 95 to 158 F module inlet temperature Storage temperature 40 to 70 C 40 to 158 F Storage humidity 90 noncondensing relative humidity at 35 C Airflow 2 to 12 CFM ...

Page 57: ...ware components 1 I installing the ExpressModule 16 installing the ixgbe driver on Linux platforms 7 on Windows platforms 10 J Jumbo Frames 25 N networking interfaces 23 P patches recommended patch clusters 6 recommended security 6 patches and updates 6 power requirements 38 R removing the ixgbe driver from Linux platforms 9 from Windows platforms 11 S setting ixgbe driver parameters 23 shipping k...

Page 58: ...40 Sun Dual 10GbE SFP PCIe ExpressModule User s Guide August 2008 ...

Reviews: