background image

40

Sun Dual 10GbE XFP PCI Express Card User’s Guide • May 2010

b. The following parameters operate on a per port basis and can be set using

the

/platform/sun4u/kernel/drv/nxge.conf

file.

c. In the following example, the ports of

all

the Sun x8 Express Dual 10 Gigabit

Ethernet Fiber XFP Low Profile Adapter are being set for load balancing Rx
traffic based on IP source address. The default value is F80 indicating Rx
load balancing based on IP 5-tuple. Notice the semi-colon at the end of the
last parameter.

#

# ------- How to set parameters for a particular interface --------

# The example below shows how to locate the device path and set a

# parameter for a particular nxge interface. (Using jumbo support as

# an example)

#

# Use the following command to find out the device paths for nxge,

#       more /etc/path_to_inst | grep nxge

#

# For example, if you see,

#       "/pci@7c0/pci@0/pci@8/network@0" 0 "nxge"

#       "/pci@7c0/pci@0/pci@8/network@0,1" 1 "nxge"

#       "/pci@7c0/pci@0/pci@8/network@0,2" 2 "nxge"

#       "/pci@7c0/pci@0/pci@8/network@0,3" 3 "nxge"

#

# then you can enable jumbo for ports 0 and 1 and disable jumbo for ports 2

# and 3 as follows,

#

# name = "pciex108e,abcd" parent = "/pci@7c0/pci@0/pci@8/" unit-address

= "0"

# accept_jumbo = 1;

# name = "pciex108e,abcd" parent = "/pci@7c0/pci@0/pci@8/" unit-address

= "0,1"

# accept_jumbo = 1;

# name = "pciex108e,abcd" parent = "/pci@7c0/pci@0/pci@8/" unit-address

= "0,2"

# accept_jumbo = 0;

# name = "pciex108e,abcd" parent = "/pci@7c0/pci@0/pci@8/" unit-address

= "0,3"

# accept_jumbo = 0;

class-opt-ipv4-tcp = 100;

class-opt-ipv4-udp = 100;

Содержание ATLS2XGF

Страница 1: ...Sun Dual 10GbE XFP PCI Express Card User s Guide Part No 820 0184 14 May 2010 Revision A...

Страница 2: ...or any loss costs or damages incurred due to your access to or use of third party content products or services Copyright 2009 2010 Oracle et ou ses affili s Tous droits r serv s Ce logiciel et la docu...

Страница 3: ...ding and Installing the Driver on a Solaris SPARC or x86 Platform 7 To Download the Driver on a Solaris Platform 7 To Remove the Driver from a Solaris Platform 9 Downloading and Installing the Driver...

Страница 4: ...oot Over the Network on Solaris x86 and Linux Systems 30 5 Configuring the nxge Device Driver Parameters 33 nxge Hardware and Software Overview 33 Setting nxge Driver Parameters on a Solaris Platform...

Страница 5: ...er 2 Configuration 53 To Check Layer 3 Configuration 54 Enabling Jumbo Frames in a Linux Environment 54 To Enable Jumbo Frames in a Linux Environment 54 To Show the Driver Statistics in a Linux Enviro...

Страница 6: ...67 Performance Specifications 68 Physical Characteristics 69 Power Requirements 69 B Diagnostic Software 71 SunVTS Diagnostic Software 71 Updating SunVTS to Recognize the Adapter 72 To Update SunVTS...

Страница 7: ...In this document the term x86 refers to 64 bit and 32 bit systems manufactured using processors compatible with the AMD64 or Intel Xeon Pentium product families For supported systems see the Solaris H...

Страница 8: ...Sun Dual 10GbE XFP PCI Express Card User s Guide May 2010 Shell Prompts Shell Prompt C shell machine name C shell superuser machine name Bourne shell and Korn shell Bourne shell and Korn shell superu...

Страница 9: ...Include the title and part number of your document with your feedback Sun Dual 10GbE XFP PCI Express Card User s Guide part number 820 0184 14 Application Title Part Number Format Location Release Not...

Страница 10: ...x Sun Dual 10GbE XFP PCI Express Card User s Guide May 2010...

Страница 11: ...should contain the following items Sun x8 Express Dual 10 Gigabit Ethernet Fiber XFP Low Profile Adapter with a low profile bracket attached Standard height bracket Sun x8 Express Dual 10 Gigabit Ethe...

Страница 12: ...ware TABLE 1 1 Hardware and Software Requirements for Sun SPARC and X64 Servers Requirements Hardware or Software Hardware Sun Fire T1000 Sun Fire T2000 Sun Fire V445 Sun Fire V245 Sun Fire V215 Sun F...

Страница 13: ...tworking I O virtualization supporting Solaris LDOM 1 0 VMware support planned for mid 07 Hardware based flow classification for extending parallelism and virtualization to networking Up to 16 Receive...

Страница 14: ...azardous Substances RoHS directive 2002 95 EC Compliance Model Number ATLS2XGF Product Family Name Sun x8 Express Dual 10 Gigabit Ethernet Fiber XFP Low Profile Adapter As Information Technology Equip...

Страница 15: ...s Inc 4150 Network Circle MPK15 102 Santa Clara CA 95054 U S A Tel 650 786 3255 Fax 650 786 3723 Donald Cameron DATE Program Manager Customer Quality Sun Microsystems Scotland Limited Blackness Road P...

Страница 16: ...6 Sun Dual 10GbE XFP PCI Express Card User s Guide May 2010...

Страница 17: ...e data link interfaces and link aggregations This chapter contains the following sections Downloading and Installing the Driver on a Solaris SPARC or x86 Platform on page 7 Downloading and Installing...

Страница 18: ...yping the following at the command line 5 For x86 systems install the software packages by typing the following at the command line A menu similar to the following displays gunzip nxge tar gz tar xvf...

Страница 19: ...ms Executing checkinstall script Using as the package base directory Processing package information Processing system information 4 package pathnames are already properly installed Verifying package d...

Страница 20: ...the network interfaces before adding the package by using the ifconfig a command 4 Use the rpm tool to install the driver on SuSe and RedHat Linux nxge 1 0 1 x86_64 rpm ifconfig a grep eth eth0 Link...

Страница 21: ...fter the rpm installation enter the following 5 Verify the new network interface instances corresponding to the Sun x8 Express Dual 10 Gigabit Ethernet Fiber XFP Low Profile Adapter The Sun x8 Express...

Страница 22: ...Driver From a Linux Platform To remove the driver packages from a Linux Platform use the rpm e command For 10G ethtool i eth4 driver nxge version 2 0 1 firmware version 2XGF PXE1 47 FCode 3 9 07 04 24...

Страница 23: ...achine running either the Solaris x86 Operating System or the Linux x86 Operating System Dynamic Reconfiguration DR is not supported Installing the Adapter If you are installing the Sun x8 Express Dua...

Страница 24: ...f a Sun Fire X4600 system because the heatsink on the adapter will sit against the daughter card of the Sun Fire X4600 system Note When you install the adapter ensure that the optical transceiver has...

Страница 25: ...dard height bracket for the low profile adapter 6 Open the retaining clip at the edge of the PCI E opening 7 Holding the adapter by the edges align the adapter edge connector with the PCI E slot Slide...

Страница 26: ...ution Do not use excessive force when installing the adapter into the PCI E slot You might damage the adapter s PCI connector If the adapter does not seat properly when you apply even pressure remove...

Страница 27: ...installing the optical transceiver and shows the most complicated adapter installation Installing an Optical Transceiver The Sun x8 Express Dual 10 Gigabit Ethernet Fiber XFP Low Profile Adapter requ...

Страница 28: ...f you are installing an optical transceiver in a Sun Fire T1000 ensure that the label and the edge connector is visible to you 2 Holding the optical transceiver by the edges align the transceiver with...

Страница 29: ...the second optical transceiver 6 Detach the wrist strap Caution If you pull the locking handle down when the optical transceiver is installed remove the optical transceiver entirely and reinstall it...

Страница 30: ...banner appears press the Stop A key sequence to interrupt the boot process and display the OpenBoot ok prompt 2 List the network devices on your system Checking the properties output for each device...

Страница 31: ...03600000 00000000 00100000 local mac address 00 14 4f 6b ce 88 phy type xgf reg 00080000 00000000 00000000 00000000 00000000 03080010 00000000 00000000 00000000 01000000 03080018 00000000 00000000 00...

Страница 32: ...s Guide May 2010 5 Type the following when you finish looking at the properties values Rebooting the System After verifying the adapter installation use the boot r command to perform a reconfiguration...

Страница 33: ...rk on a Diskless Client System on page 25 Installing the Solaris Operating System Over a 10 Gigabit Ethernet Network on page 27 Booting Over the 10 Gigabit Ethernet Network for Solaris x86 and Linux S...

Страница 34: ...ip address with the adapter s IP address Refer to the ifconfig 1M man page and the Solaris documentation for more information If you want a setup that remains the same after you reboot create an etc h...

Страница 35: ...diskless client system across a 10 Gigabit Ethernet network you must first install the 10 Gigabit Ethernet software packages into the root directory of the diskless client You can find the 10 Gigabit...

Страница 36: ...e server Install the software packages to the client s root directory 4 Create a hostname nxgenumber file in the diskless client s root directory Create an export root client name etc hostname nxgenum...

Страница 37: ...Gigabit Ethernet software packages to the install server These software packages are on Sun 10 Gigabit Ethernet Driver CD Note Refer to the Solaris Advanced Installation Guide for more information ab...

Страница 38: ...s hard drive The package is a folder SUNWnxge v or SUNWnxge u which you can download from the following web site http www sun com download products xml id 44eb1efd 4 On the install server install the...

Страница 39: ...install the Sun x8 Dual 10 Gigabit Ethernet software on the client system The software installed in Step 4 is required to boot the client system over the 10 Gigabit Ethernet interface You now need to...

Страница 40: ...information about editing these files 11 Use the dladm show dev command to show configuration information for all data links or the specified data link By default the system is configured to have one...

Страница 41: ...t order of Hard Drive is higher than network devices 8 Refer the boot device order image The reconfiguration boot attaches the driver to the adapter You can now configure the driver parameters for you...

Страница 42: ...heck cable PXE MOF Exiting Intel Boot Agent NVIDIA Boot Agent 217 0513 Copyright C 2001 2005 NVIDIA Corporation Copyright C 1997 2000 NVIDIA Corporation PXE E61 Media test failure check cable PXE MOF...

Страница 43: ...Express Dual 10 Gigabit Ethernet Fiber XFP Low Profile Adapter provides two 10 Gigabit Full Duplex networking interfaces The device driver automatically sets the link speed to 10000 Mbit sec and confo...

Страница 44: ...t the system create a platform sun4u kernel drv nxge conf file and add parameter values to this file when you need to set a particular parameter for a device in the system Setting Parameters Using the...

Страница 45: ...value which must be specified down to the named dev nxgedriver_instance and assigns the value to the parameter Where X is the driver instance for example dev nxge0 dev nxge1 2 To display the value of...

Страница 46: ...g parameters in a nxge conf file To access any man page type the man command plus the name of the man page For example to access man pages for prtconf 1M type ndd dev nxge1 name to get set read only f...

Страница 47: ...The last part in double quotes is the driver name To identify a PCI E device unambiguously in the nxge conf file use the name parent name and the unit address for the device Refer to the pci 4 man pa...

Страница 48: ...lt is 1 adv_1000fdx_cap 1 adv_100fdx_cap Advertise 100mbps Full duplex capability default is 1 adv_100fdx_cap 1 adv_10fdx_cap Advertise 10mbps Full duplex capability default is 1 adv_10fdx_cap 1 adv_a...

Страница 49: ...fine how each IP class is configured Configuration options range from whether TCAM lookup ie is enabled to flow hash generation This parameters also control how the flow template is constructed and ho...

Страница 50: ...set a parameter for a particular nxge interface Using jumbo support as an example Use the following command to find out the device paths for nxge more etc path_to_inst grep nxge For example if you se...

Страница 51: ...atform or an AMD CPU based platform To Improve Performance on an UltraSPARC CPU Based Sun Platform 1 Improve performance by adding the following etc system file Increasing the MSI improves the Rx perf...

Страница 52: ...an AMD CPU Based Sun Platform 1 Enable soft rings and change to a higher value than the default of 2 by adding the following to the etc system file Bulk throughput for Rx can be significantly improved...

Страница 53: ...he Solaris TCP IP Stack To obtain higher throughput add the following to a startup script ndd set dev tcp tcp_conn_req_max_q 8192 ndd set dev tcp tcp_conn_req_max_q0 8192 ndd set dev tcp tcp_max_buf 4...

Страница 54: ...AME ethtool A DEVNAME autoneg on off rx on off tx on off ethtool c DEVNAME ethtool C DEVNAME adaptive rx on off adaptive tx on off rx usecs N rx frames N rx usecs irq N rx frames irq N tx usecs N tx f...

Страница 55: ...VNAME magic N offset N value N ethtool k DEVNAME ethtool K DEVNAME rx on off tx on off sg on off tso on off ethtool r DEVNAME ethtool p DEVNAME d ethtool t DEVNAME online offline ethtool s DEVNAME spe...

Страница 56: ...set rx usecs to smaller value For bulk traffic it is recommended to use larger values of rx usecs and control the rate with rx frames rx frames irq controls the maximum number of rx packets processed...

Страница 57: ...usecs 0 sample interval 0 pkt rate low 0 pkt rate high 0 rx usecs 20 rx frames 512 rx usecs irq 0 rx frames irq 512 tx usecs 0 tx frames 0 tx usecs irq 0 tx frames irq 0 rx usecs low 0 rx frame low 0...

Страница 58: ...er 2 To get a particular variable use the nxge_config if_name get param_name usr local bin nxge_config eth4 get The tunable parameters exported by this device are class_opt_ipv4_tcp Read Write class_o...

Страница 59: ...s the big gainer net ipv4 tcp_sack 0 sets min default max TCP read buffer default 4096 87380 174760 net ipv4 tcp_rmem 10000000 10000000 10000000 sets min pressure max TCP write buffer default 4096 163...

Страница 60: ...50 Sun Dual 10GbE XFP PCI Express Card User s Guide May 2010 2 Set up the sysctl utility sysctl p etc sysctl_nxge conf...

Страница 61: ...ronment on page 54 Jumbo Frames Overview Configuring Jumbo Frames enables the Ethernet interfaces to send and receive packets larger than the standard 1500 bytes However the actual transfer size depen...

Страница 62: ...statistics of a VLAN interface for example kstat nxge 1 grep rdc_packets rdc_packets 798982054 rdc_packets 792546171 rdc_packets 803941759 rdc_packets 805674872 rdc_packets 798714912 rdc_packets 79929...

Страница 63: ...3 Set the maximum MTU for maximum performance To Check Layer 2 Configuration View the maximum transmission unit MTU configuration of an nxge instance at any time with the kstat command The kstat mac_m...

Страница 64: ...s Dual 10 Gigabit Ethernet Fiber XFP Low Profile Adapter interface where xxx xxx xx xxx is the IP address of the interface 3 Set the MTU for maximum performance Where x is the instance number of the i...

Страница 65: ...HWaddr 00 14 4F 6C 78 E9 inet addr 38 38 9 42 Bcast 38 38 9 255 Mask 255 255 255 0 inet6 addr fe80 214 4fff fe6c 78e9 64 Scope Link UP BROADCAST RUNNING MULTICAST MTU 9194 Metric 1 RX packets 748274 e...

Страница 66: ...atistics use the grep command on the output of ethtool S ethtool S eth9 grep tx_pac tx_packets 8645339 tx_packets 3418334 tx_packets 26 tx_packets 0 tx_packets 22 tx_packets 401791 tx_packets 1466540...

Страница 67: ...on allows one or more network links to be aggregated together to form a link aggregation group This link aggregation group appears to MAC clients as a regular link Link aggregation is defined by IEEE...

Страница 68: ...ey 33 without specifying mode As the command returns one line appears in etc aggregation conf file and indicates that the default mode is off as shown in the following example ifconfig down unplumb nx...

Страница 69: ...dcast up ifconfig aggr33 aggr33 flags 1000843 UP BROADCAST RUNNING MULTICAST IPv4 mtu 1500 index 8 inet 192 168 1 1 netmask ffffff00 broadcast 192 168 1 255 ether 0 3 ba d8 9d e8 dladm show aggr key 3...

Страница 70: ...e bonding device max_bonds is the number of bond interfaces to be created mode specifies the bonding policies miimon is the frequency in milliseconds that MII link monitoring will occur Refer to Linux...

Страница 71: ...rther configuration is necessary Overview of VLANs VLANs enable you to split your physical LAN into logical subparts providing an essential tool for increasing the efficiency and flexibility of your n...

Страница 72: ...sisting of the Main Server Client 3 and Client 5 This represents an engineering group VLAN 2 includes the Main Server Clients 1 and 2 by means of a shared media segment and Client 5 This is a software...

Страница 73: ...10 Gigabit Ethernet Fiber XFP Low Profile Adapter installed It is connected to switch port 10 Both the adapter and the switch port are configured for VLANs 1 and 2 and have tagging enabled VLAN taggi...

Страница 74: ...mber file for each VLAN that will be configured for each adapter on the server Use the following naming format which includes both the VID and the physical point of attachment PPA VLAN logical PPA 100...

Страница 75: ...ure VLANs in a Linux Environment 1 Ensure that the nxge module is loaded ifconfig nxge123000 plumb up ifconfig nxge224000 plumb up ifconfig a lo0 flags 1000849 UP LOOPBACK RUNNING MULTICAST IPv4 mtu 8...

Страница 76: ...le Adapter interface where xxx xxx xx xxx the IP address of the interface 3 Insert the VLAN module 4 Add the VLAN instance VID 5 Configure the nxge VLAN eth2 in this example where xxx xxx xx xxx the I...

Страница 77: ...s the following sections Connectors on page 67 Performance Specifications on page 68 Physical Characteristics on page 69 Power Requirements on page 69 Connectors FIGURE A 1 shows the connector for the...

Страница 78: ...10GBase SR Depending on fiber quality up to 300 meters for high bandwidth multimode fiber 10GBase LR Depending on fiber quality 10km maximum Fiber type 10GBase SR multimode 10GBase LR single mode TAB...

Страница 79: ...cs Dimension Measurement Length 167 65 mm 6 6 inches Width 68 90 mm 2 713 inches Height Component side 14 48 mm 0 57 inches maximum Solder side 2 67 mm 0 105 inches maximum TABLE A 4 Low Profile Power...

Страница 80: ...70 Sun Dual 10GbE XFP PCI Express Card User s Guide May 2010...

Страница 81: ...and devices The SunVTS software operates primarily from a graphical user interface enabling test parameters to be set quickly and easily while a diagnostic test operation is being performed Note SunV...

Страница 82: ...VTS to Recognize the Adapter 1 Plug in a loopback cable 2 Ensure that the SunVTS software and the nxge driver are installed on your system 3 Add the following lines to the opt SUNWvts lib conf netlbte...

Страница 83: ...evice be configured offline before running the test Use the ifconfig 1M command to bring the Ethernet device down before running netlbtest To Use the netlbtest 1 Ensure that the SunVTS software and th...

Страница 84: ...74 Sun Dual 10GbE XFP PCI Express Card User s Guide May 2010...

Страница 85: ...ifconfig command 24 installing the adapter without DR 13 installing the Solaris operating environment over the network 27 N ndd utility 35 P parameter values how to modify and display 35 S setting nx...

Страница 86: ...76 Sun Dual 10GbE XFP PCI Express Card User s Guide May 2010...

Отзывы: