NetXen Intelligent NIC Installation And Tuning Manual Download

Page: 5 / 76

1. Overview

NetXen, Inc.

Intelligent NIC Installation and Tuning Guide

Overview

This guide describes the system requirements, installation, and setup required for the
NetXen Intelligent NIC® network interface card product release.

1.1

Related Documents

Release notes included in the software release.

Linux ‘man’ pages:

- man nx_nic

- man nx_lsa

Check NetXen web site for updates.

1.2

NIC Versions

The NIC products support Windows, Linux, and other operating systems. See the release
notes for an updated list.

1.3

Currently Supported Features

Note: Check with NetXen periodically for software updates and additional features.

Product Number

Product Definition

NXB-10GSXR

Single 10-GE port (optical fiber)

NXB-10CX4

Single 10-GE port (copper)

Feature

Windows

Linux

TCP, UDP, and IP checksum offload

Yes

TCP segmentation offload (TSO / LSO)

Yes

Interrupt coalescing

Yes

Interrupt distribution

Yes

VLAN

Yes

SNMP Ethernet MIB

Yes

Jumbo frames

Yes

Teaming and Failover

Yes

Linux Sockets Acceleration (LSA)

N/A

Yes

Receive Side Scaling (RSS)

Yes

Summary of Contents for Intelligent NIC

Page 1: ...ent NIC Installation and Tuning Guide Version 3 7 November 14 2008 This document is applicable to releases 3 4 337 NetXen Inc 18922 Forge Drive Cupertino CA 95014 0701 Telephone 408 861 4500 Web Site...

Page 2: ...NetXen and its customer Except as may be explicitly set forth in such agreement NetXen does not make and expressly disclaims any representations or warranties as to the completeness accuracy or useful...

Page 3: ...Guidelines 14 5 Detailed Installation and Configuration 16 5 1 Installing and Connecting the Intelligent NIC Adapter 16 5 2 Windows Driver Installation 18 5 2 1 NIC Driver Installation 18 5 2 2 Confi...

Page 4: ...d Switch Side 58 9 4 2 NIC VLAN as Send Side 59 9 4 3 NIC VLAN as Receive Side 60 10 Troubleshooting Guidelines 61 10 1 Unable to Boot the Host System with the NIC Card Installed 61 10 2 Unable to pin...

Page 5: ...ux and other operating systems See the release notes for an updated list 1 3 Currently Supported Features Note Check with NetXen periodically for software updates and additional features Product Numbe...

Page 6: ...rcept Driver version lsa tar gz version lsa driver nx_intercept ko or modprobe for nx_intercept LSA Driver version lsa tar gz version lsa driver nx_lsa ko or modprobe for nx_lsa Firmware cx4_romimage...

Page 7: ...Multi rate 10 Gb s 850 nm Optical XFP Transceiver Module PL XXL SC S45 22 Table 4 System and Software Requirements Hardware Systems Validated Platforms Validated OS 32bit and x64 Firmware1 1 NIC and...

Page 8: ...lspci vvv d 4040 For Windows based systems execute the following sequence 1 Right click on My Computer 2 Select Manage 3 Select Device Manager 4 Check under Other devices for an unknown device 5 Righ...

Page 9: ...lash i eth n all3 Unload and Remove the Existing Driver 1 Unload the existing driver ifconfig eth n down rmmod nx_nic 2 Remove the existing driver module rm f lib modules kernel version kernel drivers...

Page 10: ...ing Linux LSA Driver Enabling Linux LSA Driver Note the Linux NIC driver must be installed before executing the procedure below 1 Untar the file build and load the drivers tar xzf version lsa tar gz c...

Page 11: ...the firmware update process The flash utility will use the backup image to recover if required 3 The flash tool will not start the firmware update process if the NIC driver is not loaded 4 The flash...

Page 12: ...Y key to continue c Download the NetXen LiveCD ISO image and burn that image onto a CD and boot the system with the CD Use option 2 from the LiveCD menu shown below to update the flash on the card Re...

Page 13: ...ing the package manager 4 2 2 Linux NIC GPL Inbox Kernel and Commercial Drivers The NetXen Linux NIC driver is available in GPL version and it is included in the Linux kernels available from kernel or...

Page 14: ...s 32 GB or more of RAM on Linux kernels 2 6 15 and above Support for GFP_DMA32 is required to properly map virtual address to the first 4 GB of physical memory The LSA driver will report that LSA is d...

Page 15: ...pgrading the flash please refer to the Quick Start Section of the Installation and Tuning Guide and rebooting the system MSI X has now been enabled as shown below by running cat proc interrupts cat pr...

Page 16: ...ems Added settings iommu soft swiotlb 131072 For AMD based systems Added settings iommu memaper 4 In some cases the kernel may fail to allocate requested aperture there will be error messages on the s...

Page 17: ...additions made to the boot entry Example modified menu lst entry for Intel system Method 3 Reduce the amount of memory on the system by physically removing the DIMMs or switching to lower density DIMM...

Page 18: ...ad Firmware Location Firmware ROM image file nxromimg bin will be placed in the C WINDOWS system32 drivers directory 4 4 VMware Guidelines The following material should be read before starting a VMwar...

Page 19: ...nux console by pressing ALT F1 from that server and log onto that system as root a Locate the correct vmnic number by running ifconfig a Look for the 00 0E 1E that indicates this is a NetXen card s MA...

Page 20: ...Locate the x8 PCIe slot if one is available 4 Once the bracket is removed insert the adapter into the PCI Express slot1 Apply uniform pressure on top of the board until it is firmly seated in the bus...

Page 21: ...nstallations once the board is inserted into the slot connect the XFP module Make sure the optical module clicks when inserted into the cage 6 Secure the bracket 7 Connect the appropriate cables to th...

Page 22: ...Linux driver installation procedure refer to Section 5 3 5 2 1 NIC Driver Installation Note that the following procedure is for a new driver installation 1 Copy the NIC release package to a directory...

Page 23: ...ocations Select Include this location in the search Type the appropriate path to the NIC driver directory in the accompanying field and click Next An example is shown below If the exact directory path...

Page 24: ...etXen Inc 5 Detailed Installation and Configuration Select the appropriate directory x86 or x64 and click OK Then click Next in the Hardware Update Wizard window to install the driver 5 The following...

Page 25: ...allation is complete the following window will appear Click Finish to exit 7 To verify that the driver was properly installed Right click on My Computer then select Device Manager A listing of devices...

Page 26: ...nd Configuration 8 In the above window right click on the NetXen 10Gb Ethernet adapter and select Properties The General tab will be displayed as shown 9 There are four tabs General Advanced Driver an...

Page 27: ...Hex Options Value a value is required enter the 12 hex bytes of the MAC address to be used or not present Max Ethernet Frame Size Ethernet frame size for packet transmission and receive does not inclu...

Page 28: ...sabled Status Ring Size Receive status ring size from hardware Registry Key StsRingSize Default 8192 Values 1024 2048 4096 8192 16384 32768 TCPv4 Stateful Offload TCPv4 stateful offload Registry Key E...

Page 29: ...information on configuring the NIC 5 2 2 Configuring the Network Interface Card 1 Configure the network connections by opening the Network Connections window in the control panel or directly from the...

Page 30: ...e the network cable is connected properly at both ends Is a known good cable being used Check the XFP module and replace if necessary Has an IP address been assigned to the interface Refer to Section...

Page 31: ...en Inc Intelligent NIC Installation and Tuning Guide 27 3 In the Internet Protocol TCP IP Properties window assign the appropriate IP subnet mask and default gateway values 4 Using the ping utility ve...

Page 32: ...r rename this file then reboot 2 Or to uninstall the driver perform a rmmod netxen nic before continuing with the following steps in this section The following sections describe the procedure for a ne...

Page 33: ...ace ifconfig eth n down ifconfig eth n IP Address netmask Netmask Value up 3 Verify network services are running and ping service network restart ping IP address Table 9 insmod and modprobe Command Op...

Page 34: ...nstalling the NetXen NIC Driver in Linux Note that if the LSA driver is installed it must be uninstalled before the NetXen Linux NIC driver is uninstalled Refer to Section 5 4 3 below for information...

Page 35: ...ommand for Linux OS In this command line the i indicates the interface command and the NX_NIC specifies the actual interface for Windows The n indicates NetXen s Ethernet interface Along with the inte...

Page 36: ...takes a few minutes to complete and MUST NOT BE INTERRUPTED 2 Get Flash related information The info option may be used to get information about the Flash e g nxflash exe i NX_NIC info for Windows nxf...

Page 37: ...firmware from the bin directory cd new release version directory bin nxflash i eth n all 3 Bring down the Ethernet interface ifconfig eth n down 4 Remove the existing or old driver module rmmod nx_ni...

Page 38: ...rmware Updates 2 Updates can be done simultaneously from two different directory trees Note that the following two commands are being executed from two separate directory trees c version a utils nxfla...

Page 39: ...running at all times for the LSA driver to work Whenever the server is rebooted verify the NIC and intercept drivers are running by using the lsmod command before using the LSA driver The LSA intercep...

Page 40: ...an be obtained by typing nxoffload help usage nxoffload options The options are listed below 7 2 2 LSA Configuration Examples To offload the iperf application type the following nxoffload a n iperf To...

Page 41: ...7 3 Running LSA Once the appropriate rule has been entered into nxoffload no further user intervention is required The offload occurs automatically The offload statistics can be monitored by reading t...

Page 42: ...TcpOutCollapsed The total number of times two outgoing packets were collapsed to form a single packet during retransmission Supported TcpTimeWaitConns The total number of connections currently in TIME...

Page 43: ...ote that this counter includes any packets counted in ipForwDatagrams which meet this no route criterion Note that this includes any datagrams which a host cannot route because all of its default rout...

Page 44: ...TcpMaxConn 0 0 0 0 TcpActiveOpens 180 0 0 0 TcpPassiveOpens 0 0 0 0 TcpAttemptFails 0 0 0 0 TcpEstabResets 0 0 0 0 TcpCurrEstab 0 0 0 0 TcpInSegs 54593130 0 0 0 TcpOutSegs 173931132 0 0 0 TcpSlowOutSe...

Page 45: ...ux Sockets Acceleration LSA NetXen Inc Intelligent NIC Installation and Tuning Guide 41 IpOutNoRoutes 0 IpReasmTimeout 0 IpReasmReqds 0 IpReasmOKs 0 IpReasmFails 0 IpFragOKs 0 IpFragFails 0 IpFragCrea...

Page 46: ...e in the bond is active A different slave becomes active if and only if the active slave fails By setting mode 1 when ifenslave is executed the bonding driver calls into bond_enslave and sets the slav...

Page 47: ...p MII Polling Interval ms 100 Up Delay ms 50000 Down Delay ms 0 Slave Interface eth16 MII Status down down state Link Failure Count 0 Permanent HW addr 00 0e 1e 00 11 d2 Slave Interface eth15 MII Stat...

Page 48: ...link failures Very few devices do not support at least miimon so there is really no reason not to use it Options with textual values will accept either the text name or for backwards compatibility th...

Page 49: ...artner to transmit LACPDUs every 1 second The default is slow 8 2 Configuring VLAN under Linux The VLAN function can be configured as described below 1 Ensure the 8021q ko driver has been loaded Check...

Page 50: ...included in the NetXen driver release package or it can be downloaded directly from Microsoft Microsoft Visual C Redistributable Package The Microsoft Visual C 2005 Redistributable Package x86 instal...

Page 51: ...etwork Component Type window is displayed 5 Select Protocol and click the Add button This causes the Select Network Protocol window to be displayed as shown below 6 Click on the Have Disk button in th...

Page 52: ...click OK After the driver is installed Windows should be rebooted to complete the installation of the NetXen Network Configuration Utility 9 2 Configuration 1 The icon for Launch NetXen Network is loc...

Page 53: ...nd Tuning Guide 49 2 Single click on the NetXen Network Configuration Utility 7 entry then click Properties to display the NetXen Network Configuration 7 Properties window shown below The following wi...

Page 54: ...50 Intelligent NIC Installation and Tuning Guide NetXen Inc 9 Windows Teaming Driver and Virtual LAN VLAN...

Page 55: ...lick the Team button above to team the adapters together An example is shown below 3 The NetXen Network Configuration Utility 7 Properties window contains all the configuration options to setup the te...

Page 56: ...ndow click on the Information tab to display general information about the board as shown below 5 After clicking OK in the Team Properties window above and OK in the NetXen Configuration Utility 7 Pro...

Page 57: ...inue Anyway as shown in the window below to permit the completion of the teaming setup 6 During the teaming setup the following window is displayed The time required to configure the adapters varies d...

Page 58: ...ed NIC This team member is always used as long as it is enabled However if the preferred NIC fails and then some time later is brought back on line the system automatically switches back to it since i...

Page 59: ...switch of primary adapter by clicking Switch Now 3 Finally click on OK to complete the manual switch process Fail on Fault The Fail on Fault mode causes the system to automatically switch to the secon...

Page 60: ...team member is always used as long as it is enabled However if the preferred NIC fails and then some time later is brought back on line the system automatically switches back to it since it is the pre...

Page 61: ...Virtual LAN VLAN NetXen Inc Intelligent NIC Installation and Tuning Guide 57 2 In the Team Properties window click on Smart Switch as shown below then click OK 3 Click OK on the NetXen Network Config...

Page 62: ...default when the VLAN ID is set on the NIC driver property tagging is enabled Ensure that the switch port connected to the NIC has tagging enabled 9 4 1 Optional Configuration on the NIC and Switch Si...

Page 63: ...B data VLAN ID in the packet s tag header If the priority value that the miniport driver retrieves from the packet is zero the miniport driver should not mark the packet with a priority value before t...

Page 64: ...Packet s OOB Data Packet Tag Header VLAN ID Miniport Driver Action Nonzero VLAN ID does not match configured VLAN ID for NIC Any value or not present Fail the send Nonzero Not present Does not match t...

Page 65: ...nable to ping the NIC Installed on the Host Type lspci v and look for vendor ID 4040 Is the NIC visible on the PCI bus Is the NIC is placed in the PCIe x8 slot Check if the driver is loaded correctly...

Page 66: ...ription How many servers clients etc Model of the host system Network setup back to back router switch backplane etc Check for error messages from the kernel Type dmesg Type cat var log messages Check...

Page 67: ...optimal PCIe performance first determine the PCI vendor ID and device ID codes using the following command lspci n A message similar to the following should be displayed 06 00 0 0200 4040 0001 rev 24...

Page 68: ...oards the PCI e slots are all the same physical size required by a x16 slot but electrically they are different widths Check the motherboard specification for the supported width of each of the PCI e...

Page 69: ...hown in the table below Note that while running these tests the network traffic will be interrupted Table 18 NetXen Windows Diagnostic Utility Commands Command Description i interface Specifies the in...

Page 70: ...o LED status test combine D or a E ExtLB External full duplex loop back test h help Prints the help message Table 19 NetXen Linux Diagnostic Utility Commands Command Description i interface Specifies...

Page 71: ...L noIntLB No internal loopback test combine D or a nH noHw No hardware test combine D or a nS noLinkSt No Link status test combine D or a nl noLED No LED status test combine D or a E ExtLB External fu...

Page 72: ...For transmit a should be a value less than 8 The more threads the lower the number should be Note The send and receive side NTTTCP commands must use the receive side s IP address Steps for NIC Perfor...

Page 73: ...tttcpr exe m 1 1 192 168 20 2 n 300000 a 16 rb 512000 p 5001 ntttcpr exe m 1 1 192 168 20 2 n 300000 a 16 rb 512000 p 5002 A 1 2 nttcps and ntttcpr Command Line Options C NTTTCP_NEW ntttcps exe parame...

Page 74: ...re efficient use of high bandwidth networks a larger TCP window size may be used The TCP window size field controls the flow of data and is limited to 2 bytes or a window size of 65 535 bytes Since th...

Page 75: ...Kernel versions 2 6 15 or later provide better performance optimization for the NIC Check CPU utilization on the host for CPU bound bottlenecks The tunable performance parameters are system and appli...

Page 76: ...for card settings See man ethtool in Linux iperf Linux performance tool LKM Linux Kernel Module Load balancing Allows the load to be balanced between 2 or more ports in a system LSA Linux Sockets Acc...

Search results

Summary of Contents for Intelligent NIC

Reviews:

Related manuals for Intelligent NIC

Brands by name

Popular brands

NetXen Intelligent NIC, Installation And Tuning Manual

Search results

Summary of Contents for Intelligent NIC

Reviews:

Related manuals for Intelligent NIC

Brands by name

Popular brands