manualshive.com logo in svg

Solarflare Solarflare XtremeScale series, Руководство пользователя, Страница: 340 / 400

Поделиться
Скачать
Solarflare Solarflare XtremeScale series Скачать руководство пользователя страница 340

Solarflare

 

Server

 

Adapter

 

User

 

Guide

SR

‐

IOV

 

Virtualization

 

Using

 

KVM

Issue

 

20

©

 

Solarflare

 

Communications

 

2017

333

03:00.2

 

Ethernet

 

controller:

 

Solarflare

 

Communications

 

Device

 

1903

 

(rev

 

01)

03:00.3

 

Ethernet

 

controller:

 

Solarflare

 

Communications

 

Device

 

1903

 

(rev

 

01)

03:00.4

 

Ethernet

 

controller:

 

Solarflare

 

Communications

 

Device

 

1903

 

(rev

 

01)

03:00.5

 

Ethernet

 

controller:

 

Solarflare

 

Communications

 

Device

 

1903

 

(rev

 

01)

4

To

 

identify

 

which

 

physical

 

port

 

a

 

given

 

network

 

interface

 

is

 

using:

#

 

cat

 

/sys/class/net/eth<N>/device/physical_port

5

To

 

identify

 

which

 

PF

 

a

 

given

 

VF

 

is

 

associated

 

with

 

use

 

the

 

following

 

command

 

(in

 

this

 

example

 

there

 

are

 

4

 

VFs

 

assigned

 

to

 

PF

 

eth4):

#

 

ip

 

link

 

show

19:

 

eth4:

 

<NO

‐

CARRIER,BROADCAST,MULTICAST,UP>

 

mtu

 

1500

 

qdisc

 

mq

 

state

 

DOWN

 

qlen

 

1000

    

link/ether

 

00:0f:53:21:00:61

 

brd

 

ff:ff:ff:ff:ff:ff

    

vf

 

0

 

MAC

 

76:c1:36:0a:be:2b

    

vf

 

1

 

MAC

 

1e:b8:a8:ea:c7:fb

    

vf

 

2

 

MAC

 

52:6e:32:3d:50:85

    

vf

 

3

 

MAC

 

b6:ad:a0:56:39:94

MAC

 

addresses

 

beginning

 

00:0f:53

 

are

 

Solarflare

 

designated

 

hardware

 

addresses.

 

MAC

 

addresses

 

assigned

 

to

 

VFs

 

in

 

the

 

above

 

example

 

output

 

have

 

been

 

randomly

 

generated

 

by

 

the

 

host.

 

MAC

 

addresses

 

visible

 

to

 

the

 

host

 

will

 

be

 

replaced

 

by

 

libvirt

‐

generated

 

MAC

 

addresses

 

in

 

a

 

VM.

7.3

 

KVM

 

Network

 

Architectures

This

 

section

 

identifies

 

SR

‐

IOV

 

and

 

the

 

Linux

 

KVM

 

virtualization

 

infrastructure

 

configurations

 

to

 

consume

 

adapter

 

port

 

Physical

 

Functions

 

(PF)

 

and

 

Virtual

 

Functions

 

(VF).

•

KVM

 

libvirt

 

Bridged

 

on

 

page

 

333

•

KVM

 

Direct

 

Bridged

 

on

 

page

 

337

•

KVM

 

Libvirt

 

Direct

 

Passthrough

 

on

 

page

 

340

•

KVM

 

Libvirt

 

Network

 

Hostdev

 

on

 

page

 

343

•

General

 

Configuration

 

on

 

page

 

348

•

Enabling

 

Virtual

 

Functions

 

on

 

page

 

348

When

 

migration

 

is

 

not

 

a

 

consideration,

 

Solarflare

 

recommends

 

the

 

network

‐

hostdev

 

configuration

 

for

 

highest

 

throughput

 

and

 

lowest

 

latency

 

performance

KVM

 

libvirt

 

Bridged

The

 

traditional

 

method

 

of

 

configuring

 

networking

 

in

 

KVM

 

virtualized

 

environments

 

uses

 

the

 

para

‐

virtualized

 

(PV)

 

driver,

 

virtio

‐

net

,

 

in

 

the

 

virtual

 

machine

 

and

 

the

 

standard

 

Linux

 

bridge

 

in

 

the

 

host.

The

 

bridge

 

emulates

 

a

 

layer

 

2

 

learning

 

switch

 

to

 

replicate

 

multicast

 

and

 

broadcast

 

packets

 

in

 

software

 

and

 

supports

 

the

 

transport

 

of

 

network

 

traffic

 

between

 

VMs

 

and

 

the

 

physical

 

port.

Содержание Solarflare XtremeScale series

Страница 1: ...e part of SOLARFLARE Communications Inc or its affiliates The only warranties granted by SOLARFLARE Communications Inc or its affiliates in connection with the Product described in this document are t...

Страница 2: ...pter User Guide Issue 20 Solarflare Communications 2017 ii Trademarks OpenOnload EnterpriseOnload XtremeScale and Flareon are registered trademarks of Solarflare Communications Inc in the United State...

Страница 3: ...ttaching a Cable RJ 45 35 2 5 Attaching a Cable SFP 36 2 6 Supported SFP Cables 38 2 7 Supported SFP 10G SR Optical Transceivers 40 2 8 Supported SFP 10G LR Optical Transceivers 41 2 9 QSFP Transceive...

Страница 4: ...SE Linux Enterprise Server 69 3 11 Configuring the Solarflare Adapter 70 3 12 Setting Up VLANs 73 3 13 Setting Up Teams 73 3 14 NIC Partitioning 74 3 15 NIC Partitioning with SR IOV 78 3 16 Receive Si...

Страница 5: ...ostics 177 4 16 Using SAM for Boot ROM Configuration 181 4 17 Managing Firmware with SAM 183 4 18 Configuring Network Adapter Properties in Windows 184 4 19 Windows Command Line Tools 189 4 20 Sfboot...

Страница 6: ...Kernel and Network Adapter Statistics 317 7 SR IOV Virtualization Using KVM 326 7 1 Introduction 326 7 2 SR IOV 331 7 3 KVM Network Architectures 333 7 4 PF IOV 346 7 5 General Configuration 348 7 6...

Страница 7: ...2017 vii 9 Solarflare Boot Manager 374 9 1 Introduction 374 9 2 Solarflare Boot Manager 375 9 3 iPXE Support 376 9 4 sfupdate Options for PXE upgrade downgrade 376 9 5 Starting PXE Boot 378 9 6 iPXE...

Страница 8: ...Onload to achieve minimum latency and maximum throughput 1 1 Virtual NIC Interface Solarflare s VNIC architecture provides the key to efficient server I O and is flexible enough to be applied to mult...

Страница 9: ...s can be instantiated for each running guest operating system or application giving it a direct pipeline to the Ethernet network This architecture provides the most efficient way to maximize network a...

Страница 10: ...ployment of 1024 channels between Virtual and Physical Functions Support Alternate Routing ID ARI SR IOV is not supported for Solarflare adapters on IBM System p servers 10 Gigabit Ethernet Supports t...

Страница 11: ...ardware to steer packets based on the MAC address to a VNIC Hardware timestamps The Solarflare Flareon SFN7000 and XtremeScale SFN8000 series adapters can support hardware timestamping for all packets...

Страница 12: ...bled for the Plus version PTP and hardware timestamps Yes factory enabled for the Plus version 1PPS Optional bracket and cable assembly not factory installed SR IOV Yes Network ports 2 x QSFP 40G 10G...

Страница 13: ...240 VF 2048 VI 2048 MSI X Interrupts Supports OpenOnload Yes factory enabled for the Onload and Plus versions PTP and hardware timestamps Yes factory enabled for the Plus version 1PPS Optional bracke...

Страница 14: ...16 PF 240 VF 2048 VI 2048 MSI X Interrupts Supports OpenOnload Yes factory enabled PTP and hardware timestamps Yes factory enabled 1PPS Optional bracket and cable assembly not factory installed SR IO...

Страница 15: ...2048 MSI X Interrupts Supports OpenOnload Yes factory enabled PTP and hardware timestamps Enabled by installing AppFlex license 1PPS Optional bracket and cable assembly not factory installed SR IOV Y...

Страница 16: ...VI 2048 MSI X Interrupts Supports OpenOnload Enabled by installing AppFlex license PTP and hardware timestamps Enabled by installing AppFlex license 1PPS Optional bracket and cable assembly not factor...

Страница 17: ...VI 2048 MSI X Interrupts Supports OpenOnload Enabled by installing AppFlex license PTP and hardware timestamps Enabled by installing AppFlex license 1PPS Optional bracket and cable assembly not factor...

Страница 18: ...slots PCIe features support Per adapter 16 PF 240 VF 2048 VI 2048 MSI X Interrupts Supports OpenOnload Enabled by installing AppFlex license PTP and hardware timestamps Enabled by installing AppFlex l...

Страница 19: ...SR IOV virtual functions per port Virtual NIC support 1024 vNIC interfaces per port Supports OpenOnload Yes SR IOV Yes Network ports 2 x SFP 10G 1G Part number SFN6122F Controller silicon SFC9020 Pow...

Страница 20: ...edge connector usable in x8 and x16 slots 127 SR IOV virtual functions per port Virtual NIC support 1024 vNIC interfaces per port Supports OpenOnload Yes SR IOV Yes Network ports 2 x SFP 10G 1G Part...

Страница 21: ...lanes Gen2 5 0GT s x8 edge connector usable in x8 and x16 slots Virtual NIC support 1024 vNIC interfaces per port Supports OpenOnload No SR IOV Yes1 1 SR IOV is not supported for Solarflare adapters o...

Страница 22: ...con SFC9020 Power 5 9W typical PCI Express 8 lanes Gen2 5 0GT s x8 edge connector usable in x8 and x16 slots 127 SR IOV virtual functions per port Virtual NIC support 1024 vNIC interfaces per port Sup...

Страница 23: ...edge connector usable in x8 and x16 slots 127 SR IOV virtual functions per port Virtual NIC support 1024 vNIC interfaces per port Supports OpenOnload Yes SR IOV Yes Ports 4 x 10GBASE KX4 backplane tra...

Страница 24: ...are ESX 5 0 ESXi 5 1 5 5 and 6 0 Linux KVM Support includes all minor updates releases service packs of the above major releases for which the distributor has not yet declared end of life support Sola...

Страница 25: ...page at https support solarflare com myappflex The sfkey utility application is used to install the generated license key file on selected adapters For detailed instructions for sfkey and license ins...

Страница 26: ...butors may be used to endorse or promote products derived from this software without specific prior written permission THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS AS IS AND ANY...

Страница 27: ...ision reception which can be determined by turning the equipment off and on the user is encouraged to try to correct the interference by one or more of the following measures Reorient or relocate the...

Страница 28: ...r example equipment may use one of a multiple of different LDMs depending on path length of the laser communication signal This equipment is not basic consumer ITE The equipment is installed and maint...

Страница 29: ...tory Approval Additional Regulatory Information for SFN8722 SFN8542 SFN8522M and SFN8042 adapters VCCI A A A Pb Hg Cd Cr VI PB B PBDE RC0402FR series resistor x o o o o o ERJ 2GE series resistor x o o...

Страница 30: ...an CNS 13438 2006 Class A Japan VCCI Regulations V 3 2014 04 Class A South Korea KCC KN 32 KN 35 Australia AS NZS CISPR 22 2009 A1 2010 Safety1 Europe BS EN 60950 1 2006 A11 2009 A1 2010 A12 2011 A2 2...

Страница 31: ...ations 2017 24 Additional Regulatory Information for SFN8522 adapter B VCCI B B B B Pb Hg Cd Cr VI PB B PBDE RC0402FR series resistor x o o o o o ERJ 2GE series resistor x o o o o o CRCW0402 series re...

Страница 32: ...an CNS 13438 2006 Class B Japan VCCI Regulations V 3 2014 04 Class B South Korea KCC KN 32 KN 35 Australia AS NZS CISPR 22 2009 A1 2010 Safety1 Europe BS EN 60950 1 2006 A11 2009 A1 2010 A12 2011 A2 2...

Страница 33: ...1998 A1 2001 A2 2003 US FCC Part 15 Class B Canada ICES 003 NMB 003 Class B Taiwan CNS 13438 2006 Class B Japan VCCI Regulations V 3 2010 Class B South Korea KCC KN 22 KN 24 Australia AS NZS CISPR 22...

Страница 34: ...ass B Canada ICES 003 NMB 003 Class B Taiwan CNS 13438 2006 Class B Japan VCCI Regulations V 3 2014 04 Class B South Korea KCC KN 32 KN 24 Australia AS NZS CISPR 22 2009 A1 2010 Safety1 1 The safety a...

Страница 35: ...ss B Canada ICES 003 NMB 003 Class B Taiwan CNS 13438 2006 Class B Japan VCCI Regulations V 3 2010 Class B South Korea KCC KN 22 KN 24 Australia AS NZS CISPR 22 2009 Safety1 1 The safety assessment ha...

Страница 36: ...55024 1998 A1 2001 A2 2003 US FCC Part 15 Class B Canada ICES 003 NMB 003 Class B Taiwan CNS 13438 2006 Class A Japan VCCI Regulations V 3 2010 Class A Australia AS NZS CISPR 22 2009 Safety1 1 The sa...

Страница 37: ...Details EMC Europe BS EN 55022 2006 BS EN 55024 1998 A1 2001 A2 2003 US FCC Part 15 Class B Canada ICES 003 NMB 003 Class B Safety1 1 The safety assessment has been concluded on this product as a com...

Страница 38: ...d Mode on page 46 LED States on page 48 Configure QSFP Adapter on page 49 Single Optical Fiber RX Configuration on page 51 Installing the OCP Mezzanine AdapterSolarflare Mezzanine Adapters SFN5814H an...

Страница 39: ...ort 40GbE PCIe 3 0 QSFP Server Adapter Solarflare Flareon Ultra SFN7124F Quad Port 10GbE PCIe 3 0 SFP Server Adapter Solarflare Flareon Ultra SFN7122F Dual Port 10GbE PCIe 3 0 Server I O Adapter Solar...

Страница 40: ...rEdge C6200 series servers Solarflare SFN6822F Dual Port 10GbE SFP FlexibleLOM Onload Server Adapter Solarflare SFN5814H Quad Port 10G Ethernet Mezzanine Adapter for IBM BladeCenter Solarflare SFN5812...

Страница 41: ...dapter remove the screws securing the bracket 2 Slide the bracket away from the adapter 3 Taking care not the overtighten the screws attach the full height bracket to the adapter 2 3 Inserting the Ada...

Страница 42: ...to install drivers for the new hardware Click Cancel or abort the installation and refer to the relevant chapter in this manual for how to install the Solarflare adapter drivers for your operating sys...

Страница 43: ...re SFP Server Adapters can be connected to the network using either an SFP Direct Attach cable or a fiber optic cable Attaching an SFP Direct Attach Cable To attach an SFP Direct Attach cable 1 Turn t...

Страница 44: ...e from the adapter socket Attaching a fiber optic cable WARNING Do not look directly into the fiber transceiver or cables as the laser beams can damage your eyesight To attach a fiber optic cable 1 Re...

Страница 45: ...ody of the adapter bracket and remove it from the adapter 2 6 Supported SFP Cables Table 2 is a list of supported SFP cables that have been tested by Solarflare Solarflare is not aware of any issues p...

Страница 46: ...SFP 10GE DAC 3m 3m Molex 74752 1101 1m Molex 74752 2301 3m Molex 74752 3501 5m Molex 74752 9093 1m 37 0960 01 0K585N Molex 74752 9094 3m 37 0961 01 0J564N Molex 74752 9096 5m 37 0962 01 0H603N Panduit...

Страница 47: ...e supported Table 3 Supported SFP 10G Optical SR Transceivers Manufacturer Product Code Notes SFN5xxx SFN6xxx SFN7xxx SFN8xxx Arista SFP 10G SR 10G Arista XVR 00002 02 10G Avago AFBR 703SDZ 10G Avago...

Страница 48: ...rflare Solarflare is not aware of any issues preventing the use of other brands of 10G LR transceivers with Solarflare network adapters However only transceivers in the table below have been fully ver...

Страница 49: ...adapters However only products listed in the tables below have been fully verified and are therefore supported Supported QSFP 40GBASE SR4 Transceivers Solarflare Flareon Ultra SFN7000 series and Xtrem...

Страница 50: ...ing QSFP Direct Attach Cables DAC QSFP cables may not work with all switches Table 6 Supported QSFP Active Optical Cables Manufacturer Product Code Notes SFN5xxx SFN6xxx SFN7xxx SFN8x42 Avago AFBR 7QE...

Страница 51: ...R QSFP2SFP 1M 3M copper DAC cables are fully tested and compatible with the Solarflare SFN7142Q and SFN7042Q server I O adapters These cables are compliant with the SFF 8431 SFF 8432 SFF 8436 SFF 8472...

Страница 52: ...sted by Solarflare Solarflare is not aware of any issues preventing the use of other brands of 1000BASE T transceivers with the Solarflare network adapters However only transceivers in the table below...

Страница 53: ...tiation to automatically select the highest speed supported in common with the link partner On SFP adapters the SFP module transceiver determines the supported speeds typically SFP modules only suppor...

Страница 54: ...SFP optical module 10G No 10G SFPopticalmodule 1G No 1G SFP optical module 10G 1G No 10G or 1G Dual speed modules run at the maximum speed 10G unless explicitly configured to the lower speed 1G SFP 10...

Страница 55: ...LED states are as follows Table 13 LED States Adapter Type LED Description State QSFP SFP SFP Speed Green solid at all speeds Activity Flashing green when network traffic is present LEDs are OFF when...

Страница 56: ...re 40G breakout cables have only 2 physical cables see Supported QSFP to SFP Breakout Cables on page 44 Breakout cables from other suppliers may have 4 physical cables When connecting a third party br...

Страница 57: ...tion for SFN8x42 The sfboot utility from the Solarflare Linux Utilities package SF 107601 LS is used to configure a Solarflare QSFP adapter for 10G or 40G operation For example sfboot port mode 2x40G...

Страница 58: ...e light signal to achieve a ratio that will deliver sufficient signal strength to all endpoints Solarflare adapters do not support a receive only configuration on 1G links 2 16 Solarflare Mezzanine Ad...

Страница 59: ...chassis HS22 HS22V and HX5 servers The IBM BladeCenter blade supports a single Solarflare mezzanine adapter 1 The blade should be extracted from the BladeCenter in order to install the mezzanine adap...

Страница 60: ...adapter See Figure 5 Figure 5 In position mezzanine adapter 5 Press the port connector gently into the connector block ensuring that the adapter is firmly and correctly seated in the connector block...

Страница 61: ...Dual Port SFP are 10GbE Mezzanine Adapters for the DELL PowerEdge C6100 series rack server Each DELL PowerEdge node supports a single Solarflare mezzanine adapter 1 The node should be extracted from...

Страница 62: ...PCB card into the slot as shown in Figure 5 top diagram Note that the riser card only fits one way 4 Offer the adapter to the node and ensure it lies underneath the chassis cover 5 Lower the adapter...

Страница 63: ...s rack server Each DELL PowerEdge node supports a single Solarflare mezzanine adapter 1 The node should be extracted from the rack server in order to install the mezzanine adapter Refer to the PowerEd...

Страница 64: ...er to the Solarflare Enhanced PTP User Guide SF 109110 CD 2 21 Solarflare ApplicationOnload Engine The ApplicationOnload Engine AOE SFA7942Q is a half length full height PCIe form factor adapter combi...

Страница 65: ...ed Installation Red Hat Enterprise Linux on page 68 Unattended Installation SUSE Linux Enterprise Server on page 69 Configuring the Solarflare Adapter on page 70 Setting Up VLANs on page 73 Setting Up...

Страница 66: ...Solarflare adapters on Red Hat and SUSE Linux distributions Table 14 Linux Feature Set Fault diagnostics Support for comprehensive adapter and cable fault diagnostics and system reports See Linux Util...

Страница 67: ...ed RFS See Solarflare Accelerated RFS SARFS on page 84 Transmit Packet Steering XPS Supported on Linux 2 6 38 and later kernels Selects the transmit queue when transmitting on multi queue devices See...

Страница 68: ...DKMS is installed To install the Solarflare driver DKMS package execute the following command rpm i sfc dkms version noarch rpm Load the driver modprobe sfc Teaming Improve server reliability and ban...

Страница 69: ...e full path to the source RPM 3 To build for a different kernel to the running system enter the following command rpmbuild define kernel kernel version rebuild package_name 4 Install the resulting RPM...

Страница 70: ...Enterprise Linux Distributions These instructions cover installation and configuration of the Solarflare network adapter drivers on Red Hat Enterprise Linux Server Refer to Software Driver Support on...

Страница 71: ...to yes For SLES 11 edit the last line in etc modprobe d unsupported modules to allow_unsupported_modules 1 2 Install the RPMs rpm ivh kernel module sfc 2 6 5 7 244 smp 2 1 0111 0 sf 1 SLES9 i586 rpm...

Страница 72: ...driver sfc version 4 0 this might also be the 4 1 driver 2 To remove the in tree driver and rebuild the initramfs so that the in tree driver does not automatically reload following reboot find lib mo...

Страница 73: ...nux unattended installation requires building two drivers A minimal installation Solarflare driver that only provides networking support This driver is used for network access during the installation...

Страница 74: ...ended Installations Table 15 below identifies the various stages of an unattended installation process Table 15 Installation Stages In Control Stages of Boot Setup needed BIOS PXE code on the adapter...

Страница 75: ...t contains the kickstart file or a network location where the kickstart file can be accessed A Solarflare driver disk Unattended Red Hat Enterprise Linux installations are configured with Kickstart Th...

Страница 76: ...RPM mnt bin rpm Uvh mnt filename of Solarflare RPM bin umount mnt 3 10 Unattended Installation SUSE Linux Enterprise Server Unattended SUSE Linux Enterprise Server installations are configured with Au...

Страница 77: ...boot_ pxe 3 11 Configuring the Solarflare Adapter Ethtool is a standard Linux tool that you can use to query and change Ethernet adapter settings Ethtool can be downloaded from http sourceforge net pr...

Страница 78: ...onfigured to operate at a lower speed 1G The following commands demonstrate ethtool to configure the network adapter Ethernet settings Identify interface configuration settings ethtool ethX Set link s...

Страница 79: ...e ethtool g ethX To set the new transmit or receive ring size to value N ethtool G ethX rx N tx N The ring buffer size must be a value between 128 and 4096 On the SFN7000 and SFN8000 series adapters t...

Страница 80: ...d VLAN traffic only on Red Hat Enterprise Linux see http kbase redhat com faq docs DOC 8062 To configure mixed VLAN tagged and untagged traffic on Red Hat Enterprise Linux see http kbase redhat com fa...

Страница 81: ...twork port can be exposed to the host as multiple PCIe Physical Functions PF with each having a unique interface name and unique MAC address When the Solarflare NET driver sfc ko is loaded in the host...

Страница 82: ...re delivered to each subscriber Received unicast packets are delivered to the PF with a matching MAC address Because the TCP IP stack has multiple network interfaces on the same broadcast domain there...

Страница 83: ...nt to the OS and driver VLAN tags can be assigned when PFs are enabled using the sfboot command A single PF can be assigned VLAN tag 0 allowing it to receive untagged traffic sfboot switch mode partit...

Страница 84: ...24 07 00 0 Ethernet controller Solarflare Communications Device 0903 rev 01 07 00 1 Ethernet controller Solarflare Communications Device 0903 rev 01 07 00 2 Ethernet controller Solarflare Communicatio...

Страница 85: ...nd confirmed with rx0 tx0 appearing after the version number in the output from ethtool as shown above The firmware update utility sfupdate and boot ROM configuration tool sfboot are available in the...

Страница 86: ...ilable via eth4 00 0F 53 21 00 60 eth13 Interface specific boot options are not available Adapter wide options are available via eth4 00 0F 53 21 00 60 eth14 Interface specific boot options are not av...

Страница 87: ...k encap Ethernet HWaddr 00 0F 53 21 00 63 eth14 Link encap Ethernet HWaddr 00 0F 53 21 00 62 eth15 Link encap Ethernet HWaddr 00 0F 53 21 00 67 3 The output from steps 1 and 2 above identifies a serve...

Страница 88: ...queue has its own dedicated MSI X interrupt which ideally should be tied to a dedicated CPU core This allows the receive side TCP processing to be distributed amongst the available CPU cores providing...

Страница 89: ...of RSS receive queues to a CPU is governed by the receive queue s MSI X interrupt affinity See Interrupt Affinity on page 113 for more details rmmod sfc modprobe sfc NOTE RSS also works for UDP packe...

Страница 90: ...ature supported since Linux 2 6 35 that attempts to schedule protocol processing of incoming packets on the same processor as the user thread that will consume the packets Accelerated RFS requires Lin...

Страница 91: ...le rps_sock_flow_entries 32768 and there are 16 queues then rps_flow_cnt for each queue will be configured as 2048 echo 2048 sys class net eth3 queues rx 0 rps_flow_cnt echo 2048 sys class net eth3 qu...

Страница 92: ...Linux 2 6 38 and later XPS is a mechanism for selecting which transmit queue to use when transmitting a packet on a multi queue device XPS is configured on a per transmit queue basis where a bitmap of...

Страница 93: ...to CPU2 and so on Map TX queue to CPU Hyperthreaded cores are included with the associated physical core echo 110011 sys class net eth3 queues tx 0 xps_cpus echo 11001100 sys class net eth3 queues tx...

Страница 94: ...M package is supplied as 64bit and 32bit binaries compiled to be compatible with GLIBC versions for all supported distributions The Solarflare utilities RPM file can be downloaded from the following l...

Страница 95: ...agent during server startup See Solarflare Boot Manager on page 374 for more information on the Boot Rom agent PXE and UEFI network boot is not supported for Solarflare adapters on IBM System p serve...

Страница 96: ...mpletion code to determine the outcome of commands when operating silently l list Lists all available Solarflare adapters This option shows the ifname and MAC address Note this option may not be used...

Страница 97: ...adapter dependent SFN8722 2x10G SFN8x42 4x10G 2x40G default SFN8522 M 2x10G default SFN7xx4F 2x10G 4x10G default SFN7xx2Q 2x10G 4x10G 2x40G default SFN7xx2F 1x10G 2x10G default Changes to this settin...

Страница 98: ...rts on the NIC rx dc size 8 16 32 64 Specifies the size of the descriptor cache for each receive queue This is for SFN7000 and SFN8000 series adapters only The default is 16 if the port mode supports...

Страница 99: ...capabilities of the link partner which may result in an inability to establish the link auto Auto negotiate link speed default 10G 10G bit sec 1G 1G bit sec 100M 100M bit sec linkup delay delay time...

Страница 100: ...eries adapters vf count vf count The number of virtual functions VF advertised to the operating system for each Physical Function on this physical network port SFN5000 6000 series adapters support 102...

Страница 101: ...Setting pf vlans none will clear all VLAN tags on the port pf vlans should be included after the pf count option on the sfboot command line If the number of PFs is changed then the VLAN tags will be...

Страница 102: ...s 2002 2005 eth4 Boot image Option ROM only Link speed Negotiated automatically Link up delay time 7 seconds Banner delay time 3 seconds Boot skip delay time 6 seconds Boot type PXE MSI X interrupt li...

Страница 103: ...he general usage for sfupdate is as follows as root sfupdate adapter eth N options where ethN is the interface name ifname of the Solarflare adapter to be upgraded option is one of the command options...

Страница 104: ...unbind done 2 Identify the bus device function for all Solarflare interfaces Using ifconfig a will not discover any Solarflare interfaces Use lspci lspci D d 1924 Output similar to the following will...

Страница 105: ...ivers sfc bind done Alternatively reload the sfc driver onload_tool reload or modprobe r sfc modprobe sfc 8 Run ifconfig a again to find that all the interfaces are reported and all have been firmware...

Страница 106: ...n is the same as or more recent then the firmware embedded in sfupdate backup Backup existing firmware image before updating This option may be used with write and force image filename Update the firm...

Страница 107: ...ed to install Solarflare AppFlex licenses and enable selected on board services for Solarflare adapters For more information about license requirements see Solarflare AppFlex Technology Licensing on p...

Страница 108: ...arSystem License Inventory Use the combined inventory and keys options to identify the licenses installed on an adapter sfkey adapter eth2 inventory keys eth2 714100101282140148200014 Flareon ONL PTP...

Страница 109: ...the keys installed sfc driver reload is required after sfkey installs certain types of license e g a PTP license To reload the sfc driver modprobe r sfc modprode sfc or when Onload is installed onloa...

Страница 110: ...tilization Different applications may be more or less affected by improvements in these three metrics For example transactional request response network applications can be very sensitive to latency w...

Страница 111: ...igured potentially improving performance Adapter MTU Maximum Transmission Unit The default MTU of 1500 bytes ensures that the adapter is compatible with legacy 10 100Mbps Ethernet endpoints However if...

Страница 112: ...mmediately A new moderation interval then starts during which no interrupt is raised Solarflare adapters by default use an adaptive algorithm where the interrupt moderation delay is automatically adju...

Страница 113: ...the transmit interrupt moderation interval unless the driver module parameter separate_tx_channels is enabled Normally packet RX and TX completions will share interrupts so RX and TX interrupt modera...

Страница 114: ...ecommend that you do not disable this setting TCP Large Receive Offload LRO TCP Large Receive Offload LRO is a feature whereby the adapter coalesces multiple packets received on a TCP connection into...

Страница 115: ...only the Solarflare interfaces and re enable LRO sysctl w net ipv4 conf ethX forwarding 0 ethtool K ethX lro on where X is the id of the Solarflare interface Disabling the libvirtd service is a perma...

Страница 116: ...t throughput To change buffer settings adjust the tcp_rmem and tcp_wmem using the sysctl command Receive buffering sysctl net ipv4 tcp_rmem min default max Transmit buffering sysctl net ipv4 tcp_wmem...

Страница 117: ...le option piobuf_size 3 25 Web Server Driver Optimization Introduction The Solarflare net driver from version 4 4 1 1017 on Solarflare SFN7000 and SFN8000 series adapters includes optimizations aimed...

Страница 118: ...MSI X interrupt is created for every core rx_recycle_ring_size The default value for the maximum number of receive buffers to recycle pages for has been changed to 512 and in newer drivers will be fur...

Страница 119: ...d to have all interrupts on the NUMA node local to the Solarflare adapter rss_numa local 1 and pin nginx threads to the free CPUs even when these are on the non local node When this is not possible CP...

Страница 120: ...isable irqbalance permanently run sbin chkconfig level 12345 irqbalance off To see whether irqbalance is currently running run sbin service irqbalance status To disable irqbalance temporarily run sbin...

Страница 121: ...Set all network interface interrupts to a single CPU in this case CPU 0 echo 1 proc irq 123 smp_affinity echo 1 proc irq 131 smp_affinity echo 1 proc irq 139 smp_affinity echo 1 proc irq 147 smp_affi...

Страница 122: ...0 Other Considerations PCI Express Lane Configurations The PCI Express PCIe interface used to connect the adapter to the server can function at different speeds and widths This is independent of the p...

Страница 123: ...ation CPU Speed Service Most Linux distributions will have the cpuspeed service running by default This service controls the CPU clock speed dynamically according to current processing demand For late...

Страница 124: ...the number of memory modules DIMMs to populate in the server For optimal memory bandwidth in the system it is likely that all DIMM slots should be populated all NUMA nodes should have memory installed...

Страница 125: ...m Offload Leave at default Enabled TCP Segmentation Offload Leave at default Enabled TCP Large Receive Offload Leave at default Enabled TCP Protocol Tuning Leave at default for 2 6 16 and later kernel...

Страница 126: ...probe ioatdma Table 25 Latency Tuning Settings Tuning Parameter How MTU Size Configure to maximum supported by network sbin ifconfig ethX mtu size Interrupt moderation Disable with ethtool C ethX rx u...

Страница 127: ...ported speed reads back as x8 and 5GT s or x8 and 8GT s or x8 and Unknown CPU Speed Service cpuspeed Disable with sbin service cpuspeed stop CPU Power Service cpupower Disable with systemctl stop cpup...

Страница 128: ...s class net ethX device lro TCP Protocol Tuning Leave at default for 2 6 16 and later kernels For earlier kernels sysctl net core tcp_rmem 4096 87380 524288 sysctl net core tcp_wmem 4096 87380 524288...

Страница 129: ...option see Driver Tuning on page 111 tx_push_max_fill 0 Direct copying Disable copying directly from the network stack for transmits by setting the following driver option see Driver Tuning on page 11...

Страница 130: ...rsordecreasedtoincrease responsiveness uint 10ms sarfs_sample_rate The frequency at which TCP packets are inspected by the SARFS feature This can be increased on heavily loaded servers to reduce the C...

Страница 131: ...e separate channels for TX and RX uint 0 rss_cpus Number of CPUs to use for Receive Side Scaling or packages cores or hyperthreads uint or string empty irq_adapt_enable Enable adaptive interrupt moder...

Страница 132: ...ailable to reset stats is to cold reboot power OFF ON the server or reload the firmware image Per port statistics port_ are from the physical adapter port Other statistics are from the specified PCIe...

Страница 133: ...than 64 bytes port_tx_64 Number of frames transmitted where the length is exactly 64 bytes port_tx_65_to_127 Number of frames transmitted where the length is between 65 and 127 bytes port_tx_128_to_25...

Страница 134: ...eived port_rx_broadcast Number of broadcasted packets received port_rx_lt64 Number of packets received where the length is less than 64 bytes port_rx_64 Number of packets received where the length is...

Страница 135: ...hen there are insufficient RX descriptors in the RX queue to allocate to the packet This problem occurs if the receive rate is very high and the network adapter receive cycle process has insufficient...

Страница 136: ...or performance for the adapter port_rx_pm_discard_vfifo_full Count of the number of packets dropped because of a lack of main packet memory on the adapter to receive the packet into port_rx_pm_trunc_q...

Страница 137: ...bytes Number of unicast bytes received rx_multicast Number of multicast packets received rx_multicast_bytes Number of multicast bytes received rx_broadcast Number of broadcast packets received rx_broa...

Страница 138: ...rs tx_tso_packets Number of physical packets produced by TSO tx_pushes Number of times a packet descriptor is pushed to the adapter from the network adapter driver tx_pio_packets Number of packets sen...

Страница 139: ...rames truncated because an internal FIFO is full As a packet is received it is fed by the MAC into a 128K FIFO If for any reason the PCI interface cannot keep pace and is unable to empty the FIFO at a...

Страница 140: ...ounter is incremented otherwise the bad_syncs counter is incremented If sfptpd is unable to synchronize the clocks at this event the sync_timeout counter is incremented sfptpd will synchronize clocks...

Страница 141: ...tion of the values in proc sys kernel printk The netif message level provides additional logging control for a specified interface These message levels are documented in Documentation networking netif...

Страница 142: ...e 29 Adapter Diagnostic Tests Diagnostic Test Purpose core nvram Verifies the flash memory board configuration area by parsing and examining checksums core registers Verifies the adapter registers by...

Страница 143: ...gth 9 phy cable pairA status 1 phy cable pairB status 1 phy cable pairC status 1 phy cable pairD status 1 Cable length is the estimated length in metres A length value of 65535 indicates length not es...

Страница 144: ...mmand Prompt on page 146 Unattended Installation on page 150 Managing Adapters with SAM on page 154 Managing Adapters Remotely with SAM on page 156 Using SAM on page 156 Configuring Network Adapter Pr...

Страница 145: ...loads Solarflare adapters support Large Segmentation Offload LSO Receive Segment Coalescing RSC and TCP UDP IP checksum offload for improved adapter performance and reduced CPU processing requirements...

Страница 146: ...to Configure Teams and VLANs on page 168 See Sfteam Adapter Teaming and VLAN Tool on page 200 See Teaming and VLANs on page 216 PXE and UEFI booting Solarflare adapters support PXE and UEFI booting en...

Страница 147: ...follow the instructions below Solarflare does not recommend installing drivers via Remote Desktop Protocol RDP For example via Terminal Services The drivers install package is named after the Solarfl...

Страница 148: ...indows To install the drivers from the command line see Install Drivers and Options From a Windows Command Prompt on page 146 1 Double click the supplied Setup exe to start the Solarflare Driver Packa...

Страница 149: ...143 Solarflare Package Installation Procedure on page 143 Repair Remove and Change Drivers and Utilities on page 145 Prerequisites The Solarflare Adapter Manager Utility SAM requires Microsoft NET Fra...

Страница 150: ...ge To install this from the command line see Install Drivers and Options From a Windows Command Prompt on page 146 1 Double click the supplied Setup exe The Solarflare Driver Package Setup wizard star...

Страница 151: ...Installs Solarflare drivers for Windows The Solarflare drivers are installed by default Install Solarflare command line tools Installs the following Solarflare Windows command line tools sfboot exe B...

Страница 152: ...anel Programs Programs and Features select the Solarflare Driver Package then select Uninstall Change or Repair from the menu bar above the program list Install Solarflare Adapter Manager Installs Sol...

Страница 153: ...on of the core Solarflare drivers and package utilities The drivers install package is named after the Solarflare document part number e g SF 107785 LS 2_Solarflare_Windows_x64_64 bit_Driver_Package e...

Страница 154: ...drivers and utilities rebooting the host to complete the installation as required setup exe Install Quiet Performs a silent installation of the drivers and utilities rebooting the host without prompt...

Страница 155: ...installed if either NetworkAdapterManager or CommandLineTools are not installed or are still being installed ADDLOCAL examples Install the package interactively with the default installation options...

Страница 156: ...ADDLOCAL NetworkAdapterManager REBOOT Suppress Extract Solarflare Drivers If it is necessary to extract the Solarflare Windows drivers e g before WDS installs this can be done from the Windows command...

Страница 157: ...using WDS Windows Deployment Services WDS enables the deployment of Windows over a network from a WDS server avoiding the need to install each operating system directly from a CD or DVD This guide ass...

Страница 158: ...lare drivers into the Boot Image Modifying the Boot Image You next need to modify the boot image to include the Solarflare Drivers extracted from the setup package Table 33 identifies drivers required...

Страница 159: ...nto the Custom Install Image These are the same Solarflare drivers added to the boot image Preparing the Custom Install Image 1 From WDS locate the install image from the Install Images folder on your...

Страница 160: ...ger Part of the Windows Automated Installation Kit to create the WDSClientUnattend xml file To associate your WDSClientUnattend xml file with your modified boot image 1 Copy the WDSClientUnattend xml...

Страница 161: ...microsoft com en us library cc749082 28WS 10 29 aspx Performing Unattended Installations http technet microsoft com en us library cc771830 28WS 10 29 aspx How to add network driver to WDS boot image h...

Страница 162: ...Ethernet and task offloading settings set up teams and VLANs configure the Boot ROM for PXE or UEFI booting and upgrade the adapter firmware Figure 17 SAM Main Screen Windows Server 2012 SAM s diagno...

Страница 163: ...Using SAM Starting SAM There are various ways of starting SAM To manage a local computer do one of the following If the Solarflare notification area icon is installed right click the icon and choose...

Страница 164: ...te computer On Windows Server 2008 R2 choose Start All Programs Solarflare Network Adapters Manage network adapters on a remote computer On Windows Server 2012 or later click the Start button followed...

Страница 165: ...pter and select Show graphs from the menu By default SAM shows the load transmitted packets and received packets graphs only To view other available graphs Select Graphs from the right click menu or f...

Страница 166: ...r the value 2 300 000 Enabled by default This can be useful when dealing with the large Tx Rx numbers that can accumulate with 10Gb networking Note The Transmit and Receive bytes columns ignore this s...

Страница 167: ...tures on Solarflare adapters Accessing Adapter Feature Settings on page 160 Checksum Offload on page 163 RSS and Interrupts on page 163 Segmentation Offload Ethernet Link Speed on page 166 Ethernet fl...

Страница 168: ...Solarflare Adapters on Windows Issue 20 Solarflare Communications 2017 161 From SAM select an adapter and from the Action menu select Configure Offload tasks Ethernet and other features The Adapter F...

Страница 169: ...y or OK when changes to Adapter Features are modified Note that the Receive legend in the Segmentation Offload field differs depending on the version of Windows that is installed for Windows Server 20...

Страница 170: ...SS Receive Side Scaling and interrupt moderation Both are enabled by default and can significantly improve the performance of the host CPU when handling large amounts of network data RSS attempts to d...

Страница 171: ...A node but there is no dynamic load balancing NUMA Scaling CPUs are assigned on a round robin basis across every NUMA node NUMA Scaling Static As for NUMA Scaling but without dynamic load balancing Co...

Страница 172: ...received packets from one interface to another then Windows will automatically disable RSC RSC is enabled by default Large Receive Offload LRO is a Solarflare proprietary mechanism similar to RSC It...

Страница 173: ...ach other when they are being overloaded by received data This prevents one device from overwhelming the other device with network packets For instance when a switch is unable to keep up with forwardi...

Страница 174: ...spread across the specified cores however the static configuration does not provide best performance when the network load varies over time Dynamic VMQ supported in Windows Server 2012 and later will...

Страница 175: ...lerant teams Teaming allows the user to configure teams consisting of all Solarflare adapter ports on all installed Solarflare adapters or might consist of only selected adapter ports e g from a dual...

Страница 176: ...am The Solarflare Create a team Wizard starts 4 Team Create Wizard 5 The wizard will guide you through the process of creating a team and optionally adding VLANs to your team see Table 40 on page 175...

Страница 177: ...n the server Solarflare recommend only changing network settings when disruption to the services can be tolerated 1 Start SAM and from the Network Adapter list select the team you want to reconfigure...

Страница 178: ...the key button Note Before physically removing an adapter from a server first check it is not the key adapter You must reassign the key adapter if you want to remove it from the team to avoid duplica...

Страница 179: ...tions list click Add one or more adapters or choose Actions Add one or more adapters The Available Network Adapters dialog box is displayed To specify a new active adapter For Fault Tolerant Teams onl...

Страница 180: ...onfiguration settings and become available on the server once again Any VLANs set up for the team will be deleted when the team is deleted To delete a team NOTE Changing team settings can disrupt curr...

Страница 181: ...anager through which the operating system is able to receive data tagged with the correct VLAN ID VID You may assign one VLAN to accept VLAN 0 or untagged traffic which allows the interface to communi...

Страница 182: ...when remotely administering the server Supports the handling of priority traffic Enables the handling of traffic that is tagged as priority Supports untagged and VLAN 0 traffic Restricts the VLAN to h...

Страница 183: ...2 In the Network adapter list select the VLAN to delete If necessary expand the team if the VLAN is attached to a team then select the VLAN 3 From the Actions list click Delete this VLAN or choose Ac...

Страница 184: ...nstalling of the adapter drivers If necessary you can reset the adapter statistics to restart the accumulated data values at any time 1 Start SAM and select a Solarflare adapter from the Network Adapt...

Страница 185: ...e selected by default See Table 41 for a description of the tests that are available 4 To stop as soon as a failure is detected select Stop on first test failure 5 To run all the tests more than once...

Страница 186: ...its in a selection of registers Interrupts Examines the available hardware interrupts by requesting the controller to generate an interrupt and verifying that the interrupt has been processed by the n...

Страница 187: ...ce more detailed results but at the expense of disrupting the connection while tests are running CAUTION The offline tests will cause the network link to momentarily drop and disrupt data flow Solarfl...

Страница 188: ...ager To configure PXE or UEFI booting on the Solarflare Boot ROM 1 Start SAM and select a Solarflare adapter from the Network Adapter list From the Action menu select the Configure Boot ROM option The...

Страница 189: ...m the Link tab change the Link Speed option depending on your link requirement Note that Auto negotiated is correct for most links and should not be changed unless advised The Link Speed options will...

Страница 190: ...K or Apply to save your settings to the Boot ROM 4 17 Managing Firmware with SAM SAM allows you to monitor the firmware PHY Boot ROM and Adapter for your Solarflare adapters Either select Manage firmw...

Страница 191: ...not be reflected in SAM until you close the Advanced Properties page To configure network adapter properties 1 From the Control Panel select System 2 Select Device Manager from the left hand menu 3 Ex...

Страница 192: ...hat it is overloaded such as when a device is receiving data faster than it can process it The adapter does this by generating a pause frame to request the sending device to temporarily stop transmitt...

Страница 193: ...end Offload Version 2 IPv4 and IPv6 Enabled Disabled Large Send Offload LSO is an offload technology for reducing the load on a CPU by processing TCP segmentation for transmitted packets in the adapte...

Страница 194: ...ter driver Default is 8 Preferred Numa Node All 0 to 15 The adapter attempts to use only the CPUs from the specified NUMA node for RSS If this is set to All or is greater than or equal to the number o...

Страница 195: ...abled Rx Tx Enabled Rx Enabled Tx Enabled UDP checksum offload is a hardware offload technology for reducing the load on a CPU by processing UDP checksums in the adapter hardware Default setting Rx Tx...

Страница 196: ...ws console to run Solarflare command line tools Table 44 List Available Command Line Utilities Utility Description sfboot exe A tool for configuring Solarflare adapter Boot ROM options including PXE a...

Страница 197: ...o using Ctrl B to access the Boot Rom agent during server startup See Solarflare Boot Manager on page 374 for more information on the Boot Rom agent Sfboot Command Usage 1 Login with an administrator...

Страница 198: ...ename Logs output to the specified file in the current folder or an existing folder Specify Silent to suppress simultaneous output to screen if required Computer ComputerName Performs the operation on...

Страница 199: ...tors available after using any splitter cables The usable values are adapter dependent SFN8722 2x10G SFN8x42 4x10G 2x40G default SFN8522 M 2x10G default SFN7xx4F 2x10G 4x10G default SFN7xx2Q 2x10G 4x1...

Страница 200: ...cache for each receive queue This is for SFN7000 and SFN8000 series adapters only The default is 16 if the port mode supports the maximum number of connectors for the adapter 32 if the port mode suppo...

Страница 201: ...e capabilities of the link partner which may result in an inability to establish the link auto Auto negotiate link speed default 10G 10G bit sec 1G 1G bit sec 100M 100M bit sec linkup delay delay time...

Страница 202: ...00 series adapters vf count vf count The number of virtual functions VF advertised to the operating system for each Physical Function on this physical network port SFN7000 and SFN8000 series adapters...

Страница 203: ...h PF in the range 0 4094 see sfboot help for details Setting pf vlans none will clear all VLAN tags on the port pf vlans should be included after the pf count option on the sfboot command line If the...

Страница 204: ...Solarflare adapters installed on the localhost sfboot List Sample console output Solarflare boot ROM configuration utility v4 1 4 Copyright Solarflare Communications 2006 2014 Level 5 Networks 2002 2...

Страница 205: ...owing syntax sfupdate Adapter Identifier options where Identifier is the name or ID of the adapter that you want to manage Specifying the adapter is optional if it is not included the command is appli...

Страница 206: ...required Adapter Identifier Performs the action on the identified Solarflare network adapter The identifier can be the adapter ID number name or MAC address Force Forces a firmware update Can be used...

Страница 207: ...le output Solarflare firmware update utility v4 1 4 Copyright Solarflare Communications 2006 2014 Level 5 Networks 2002 2005 1 Solarflare SFN7122F SFP Server Adapter MAC address 00 0F 53 21 9B B1 Firm...

Страница 208: ...variable SFTOOLS to set the path to the Solarflare tools For example SET PATH PATH SFTOOLS or refer to sfteam as SFTOOLS sfteam Sfteam Command Line Options Table 49 lists the command line options sfte...

Страница 209: ...created sfteam does not allow you to change its adapters VLANs or team name Either delete the team and set it up again or use SAM instead to configure the team Configure TeamIdentifier Configures the...

Страница 210: ...g priority name DHCP to be explicit Formats tag e g 2 assumes no priority tag priority e g 2 p tag priority name e g 2 p my name tag priority name DHCP e g 2 p my name DHCP tag priority name addr mask...

Страница 211: ...s either dynamic or static The options are auto default faulttolerant bandwidth key adapter See Teaming and VLANs on page 216 for an explanation of the different teaming modes Distribution Distributio...

Страница 212: ...dapter 2 with VLAN tag 4 and priority traffic handling enabled sfteam Create Adapter 2 Vlan 4 P Sample output Solarflare teaming configuration utility v4 1 4 Copyright Solarflare Communications 2006 2...

Страница 213: ...ne options are case insensitive and may be abbreviated NOTE Abbreviations in scripts should be avoided since future updates to the application may render your abbreviated scripts invalid Table 50 Sfca...

Страница 214: ...9m 2 Solarflare SFN5121T 10GBASE T Server Adapter 2 MAC address 00 0F 53 01 40 8D Link state Up Link speed 10 Gbps Pair 1 OK length 9m Pair 2 OK length 9m Pair 3 OK length 9m Pair 4 OK length 9m Adapt...

Страница 215: ...Command Prompt window enter the following command sfkey Adapter Identifier options where Identifier is the name or ID of the adapter that you want to manage Specifying the adapter is optional if it i...

Страница 216: ...number name and MAC address Backup Filename Output a report of the installed keys in all adapters The report can be saved to file and later used with the Install option or can be output to stdout by...

Страница 217: ...fkey Inventory Keys Solarflare Key Management Application v4 8 2 Copyright Solarflare Communications 2006 2016 Level 5 Networks 2002 2005 42 43 000F53430560 8xxx Onload Precision Time TCP Direct Low L...

Страница 218: ...e tools instead 3 In the Command Prompt window enter your command using the following syntax sfnet Adapter Identifier options where Identifier is the name or ID of the adapter that you want to manage...

Страница 219: ...he operation on the identified remote host Administrator rights on the remote host computer is required Adapter Identifier Perform the action on the identified Solarflare physical or virtual network a...

Страница 220: ...ue is given it must formated as group number where group is in the range 0 9 and number in the range 0 to 63 rssmaxprocessor group number The maximum number of processors available for RSS If a value...

Страница 221: ...bled disabled Allows the adapter to vary interrupt moderation automatically if interruptmoderation is enabled wake enabled disabled Specify whether Wake on LAN is enabled sleep enabled disabled Specif...

Страница 222: ...apters were not able to be added 32 Application failed initialization 33 Access denied Either the remote host refused a connection on the basis of account privileges or a file could not be opened 34 C...

Страница 223: ...the same team This section is only relevant to teams of Solarflare adapters Solarflare adapters can be used in multi vendor teams when teamed using another vendor s teaming driver NOTE Windows Server...

Страница 224: ...nected to the same switch In case of link failover traffic on the failed link will be re distributed to the remaining links Link aggregation offers the following functionality Teams can be built from...

Страница 225: ...of a link aggregation until a link within the aggregation fails VLANs are supported within 802 1AX teams In the event of failover the load on the failed link is redistributed over the remaining links...

Страница 226: ...mode that requires manual configuring of the ports at both ends of the link Static link aggregation is protocol independent and typically inter operates with common link aggregation schemes such as In...

Страница 227: ...ch It can also be used with each network link connected to separate switches A fault tolerant team is a set of one or more network adapters bound together by the adapter driver A fault tolerant team i...

Страница 228: ...n Figure 39 Fault Tolerant Team Failover The teaming driver monitors the state of the active adapter and in the event that its physical link is lost down or that it fails in service swaps to one of th...

Страница 229: ...7 222 Note that in this example Adapter 3 is now active The order in which the adapters are used is determined by a number of factors including user definable rank VLANs VLANs offer a method of dividi...

Страница 230: ...on Windows Issue 20 Solarflare Communications 2017 223 VLANs and Teaming VLANs are supported on all Solarflare adapter teaming configurations VLANs with Fault Tolerant Teams Figure 42 shows a fault to...

Страница 231: ...gardless of the number of VLANs as show in Figure 43 Figure 43 Failover in Fault Tolerant Team VLAN VLANs with Dynamic or Static Link Aggregation Teams VLANs work in the same way with either Dynamic o...

Страница 232: ...of link failure all traffic is distributed over the remaining links as in Figure 45 Figure 45 VLAN with Failed Dynamic or Static Team Link Key Adapter Every team must have a key adapter Figure 46 show...

Страница 233: ...e Receive Side Scaling RSS on page 233 and provides the MAC Address that will be used for all traffic sent and received by the team When a link failure occurs in the active adapter for example the phy...

Страница 234: ...tions 2017 227 Dynamic and Static Link Aggregation Teams The assignment of key adapters is supported in both dynamic and static link aggregated teams and works in the same way for both Any link failur...

Страница 235: ...apter driver settings that affect the performance metrics described This section covers the tuning of all Solarflare adapters Latency will be affected by the type of physical medium used 10GBase T twi...

Страница 236: ...rver 2008 R2 Supported on Windows Server 2012 Windows Server 2012 R2 Adaptive Interrupt Moderation Yes Yes Interrupt Moderation Yes Yes Interrupt Moderation Time Yes Yes Large Receive Offload IPv4 Yes...

Страница 237: ...es 802 1Q tag control information and 2 bytes EtherType The default maximum frame size is therefore 1518 bytes Since the maximum frame size should ideally be matched across all endpoints in the same L...

Страница 238: ...duce peak throughput Turning off interrupt moderation will generate the most interrupts give the highest CPU utilization give the lowest latency give the biggest reduction in peak throughput For many...

Страница 239: ...ling LSO will reduce CPU utilization on the transmit side of a TCP connection and improve peak throughput if the CPU is fully utilized Since LSO has no effect on latency it can be enabled at all times...

Страница 240: ...use LRO instead The Solarflare network adapter driver disables LRO by default NOTE LRO should NOT be enabled when using the host to forward packets from one interface to another For example if the ho...

Страница 241: ...increased to spread the load over more cores This will also increase the amount of receive buffering due to a larger number of RX queues NOTE If hyper threading is enabled RSS will only select one thr...

Страница 242: ...e AutoSize Id Name PreferredNumaNode On Windows Server 2012 and later you can instead run the PowerShell Get NetAdapterRss cmdlet and look at lines starting RssProcessorArray Group Number NUMA Distanc...

Страница 243: ...rRss Other Considerations PCI Express Lane Configurations The PCI Express PCIe interface used to connect the adapter to the server can function at different speeds and widths This is independent of th...

Страница 244: ...when selecting the number of memory modules DIMMs to populate in the server For optimal memory bandwidth in the system it is likely that all DIMM slots should be populated all NUMA nodes should have m...

Страница 245: ...t hyper threading is generally beneficial on systems fitted with Core i5 Core i7 and Xeon Nehalem or later CPUs TCP IP Options TCP timestamps window scaling and selective acknowledgments are enabled b...

Страница 246: ...ce idle policy to favor performance over power savings Disables PCIe ASPM Active State Power Management The installer does not enable this by default as on certain systems it might significantly incre...

Страница 247: ...or any third party Firewall should be disabled with caution The network administrator should be consulted before making any changes Disable the Windows Firewall a Set the Startup Type of the Windows F...

Страница 248: ...nterface 2 On client run ntttcps test ntttcps exe rb 500000 a 24 n 100000 l 524288 m 1 1 server_adapter_IP_interface C ntttcps exe rb 500000 a 24 n 100000 l 524288 m 1 1 server adapter IP interface Co...

Страница 249: ...Receive Offloads Enable in Network Adapter Advanced Properties Large Send Offloads Leave at default Enabled Max Frame Size Configure to maximum supported by network in Network Adapter s Advanced Prope...

Страница 250: ...Max Frame Size Configure to maximum supported by network in Network Adapter s Advanced Properties Memory bandwidth Ensure memory utilizes all memory channels on system motherboard Offload Checksums L...

Страница 251: ...al 0xA0000004L The driver can t add a device due to the system being started in safe mode SAFEMODE_MINIMAL Warning 0xA0000005L The driver could not allocate memory on a specific NUMA node Warning For...

Страница 252: ...g The adapter or server maybe faulty 0xA0010017L Hardware MCDI communication suffered an error Warning None required 0xA0010019L Hardware MCDI boot from non primary flash Possible flash corruption War...

Страница 253: ...e has reported an error condition Error 0xE0010018L Hardware MCDI communication timed out Error None required Table 60 NDIS Driver Status Codes Value Description Severity Notes 0x60030001L The driver...

Страница 254: ...pecified in the registry is out of range Using default value Warning 0xA0010007L The parameter IfType is missing from the registry Using default value Warning 0xA0010008L The parameter IfType specifie...

Страница 255: ...missing from the registry Using default value Warning 0xA0010010L The parameter LROIPv4 specified in the registry is out of range Using default value Warning 0xA0010011L Theparameter LSOv2IPv4 is mis...

Страница 256: ...ecified in the registry is invalid Using default value Warning 0xA0010019L The parameter PhysicalMediaType is missing from the registry Using default value Warning 0xA001001AL The parameter PhysicalMe...

Страница 257: ...L The parameter SpeedDuplex is missing from the registry Using default value Warning 0xA0010022L The parameter SpeedDuplex specified in the registry is out of range Using default value Warning 0xA0010...

Страница 258: ...er VlanId specified in the registry is out of range Using default value Warning 0xA001002BL The parameter WarnLevel is missing from the registry Using default value Warning 0xA001002CL The parameter W...

Страница 259: ...fault value Warning 0xA0010033L The parameter UDPChecksum OffloadIPv4ismissingfrom the registry Using default value Warning 0xA0010034L The parameter UDPChecksum OffloadIPv4 specified in the registry...

Страница 260: ...t value Warning 0xA001003BL The parameter TCPChecksum OffloadIPv6ismissingfrom the registry Using default value Warning 0xA001003CL The parameter TCPChecksumOffload IPv6 specified in the registry is o...

Страница 261: ...leepOn Disconnect is missing from the registry Using default value Warning 0xA0010044L The parameter DeviceSleepOn Disconnect specified in the registry is out of range Using default value Warning 0xA0...

Страница 262: ...f range Using default value Warning 0xA001004DL The parameter FlowControlAuto Negotiation is missing from the registry Using default value Warning 0xA001004EL The parameter FlowControlAuto Negotiation...

Страница 263: ...issing from the registry Using default value Warning 0xA0010056L The parameter VMQLookaheadSplit specified in the registry is out of range Using default value Warning 0xA0010057L The parameter VMQVlan...

Страница 264: ...specified in the registry is out of range Using default value Warning 0xA001005FL The parameter RssProfile is missing from the registry Using default value Warning 0xA0010060L The parameter RssProfile...

Страница 265: ...parameter RssMaxProcNumber is missing from the registry Using default value Warning 0xA0010068L The parameter RssMaxProcNumber specified in the registry is out of range Using default value Warning 0xA...

Страница 266: ...he LSOv2IPv4 and LSOv2IPv6 registry parameters Warning 0xA0020003L The driver has disabled Large Receive Offload overriding the LROIPv4 registry parameter as it is incompatible with IEEE 802 1p 802 1Q...

Страница 267: ...support TCP Connection Offload and therefore has overridden the TCPConnection OffloadIPv4 registry parameter Warning 0xA0020009L ThedriverhasdisabledTCP Connection Offload overriding the TCPConnectio...

Страница 268: ...t support IPv6 checksum offloads and therefore has overridden the TCPChksumOffv6 and UDPChksumOffv6 registry parameters Warning 0xA002000DL The driver has determined that the network adapter does not...

Страница 269: ...upport will be unavailable whilst IPv6 TCP transmit checksum offload is disabled Warning 0xA0020012L The driver has determined that the network adapter does not support Virtual Machine Queues and ther...

Страница 270: ...cessary for operation Error 0xE0000002L The driver has determined thatthenetworkadapteris not functioning properly Error 0xE0000003L The driver has encountered an internal error and has failed Error 0...

Страница 271: ...em Requirements on page 264 VMware Feature Set on page 265 Installing Solarflare Drivers and Utilities on VMware on page 266 Configuring Teams on page 267 Configuring VLANs on page 268 Running Adapter...

Страница 272: ...ter performance and reduced CPU processing requirements See TCP IP Checksum Offload on page 285 See TCP Segmentation Offload TSO on page 285 NetQueue Support for NetQueue a performance technology that...

Страница 273: ...their configurations Installing on VMware ESX 5 0 ESXi 5 1 5 5 and 6 0 To install or update the VIB through the CLI esxcli software vib install v absolute PATH to the vib To install or update the off...

Страница 274: ...on type and click Next 7 Choose Create a Virtual Switch or Use vSwitchX as desired 8 Follow the remaining on screen instructions 5 4 Configuring Teams A team allows two or more network adapters to be...

Страница 275: ...tch must have an appropriate VLAN ID specified NOTE VMware recommend that you create or amend VLAN details from the physical console of the server not via the Infrastructure Client to prevent potentia...

Страница 276: ...nvram Verifies the flash memory board configuration area by parsing and examining checksums core registers Verifies the adapter registers by attempting to modify the writable bits in a selection of r...

Страница 277: ...Wrote line at the end of the build process identifies the location of the binary RPM Wrote root rpmbuild RPMS x86_64 sfutils version x86_64 rpm 3 Identify where the binary tools would be installed bu...

Страница 278: ...ng the Boot ROM with Sfboot Sfboot Command Usage on page 271 Sfboot Command Line Options on page 272 Sfboot Examples on page 278 Sfboot is a command line utility for configuring Solarflare adapter Boo...

Страница 279: ...iases silent Suppresses all output including warnings and errors no user interaction You should query the completion code to determine the outcome of commands when operating silently see Performance T...

Страница 280: ...s Table 63 Sfboot Options Option Description Table 64 Sfboot Global Parameters Parameter Description boot image all optionrom uefi disabled Specifies which boot firmware images are served up to the BI...

Страница 281: ...security on non privileged functions This is for SFN7000 and SFN8000 series adapters only This reduces security in virtualized environments The default is disabled When enabled a function PF or VF ca...

Страница 282: ...rts a reduced number of connectors change mac default enabled disabled This is for SFN7000 and SFN8000 series adapters only Change the unicast MAC address for non privileged functions on this port Thi...

Страница 283: ...apabilities of the link partner which may result in an inability to establish the link auto Auto negotiate link speed default 10G 10G bit sec 1G 1G bit sec 100M 100M bit sec linkup delay delay time in...

Страница 284: ...0 series adapters vf count vf count The number of virtual functions VF advertised to the operating system for each Physical Function on this physical network port SFN7000 and SFN8000 series adapters h...

Страница 285: ...pf vlans none will clear all VLAN tags on the port pf vlans should be included after the pf count option on the sfboot command line If the number of PFs is changed then the VLAN tags will be cleared...

Страница 286: ...ice Console as root and enter the following command sfupdate adapter vmnicX options where vmnicX is the interface name of the Solarflare adapter you want to upgrade Specifying the adapter is optional...

Страница 287: ...pdate To use an external image specify image filename in the command write fails if the embedded image is the same or a previous version to that in the adapter To force a write in this case specify fo...

Страница 288: ...The PHY firmware is up to date 5 10 Performance Tuning on VMware Introduction on page 281 Tuning Settings on page 282 Other Considerations on page 286 Introduction The Solarflare family of network ad...

Страница 289: ...virtual adapter of type vmxnet To check that VMware Tools are installed 1 From the VMware Infrastructure Client power on the virtual machine and click the Summary tab 2 In the General panel check the...

Страница 290: ...pt vector in hex and CPU is the CPU number in decimal To determine the value for IRQVEC enter cat proc vmware interrupts Locate the interrupts associated with the Solarflare adapter e g vmnic2 Interru...

Страница 291: ...ver software adapts the interrupt moderation setting according to the traffic and workloads it sees Alternatively you can set the moderation interval manually You would normally only do this if you ar...

Страница 292: ...olarflare recommend you do not disable checksum offload For advice on configuring checksum offload in the guest consult the relevant Solarflare section for that guest or the documentation for the gues...

Страница 293: ...with the number of available CPU cores RSS requires a platform that supports MSI X interrupts RSS is enabled by default When RSS is enabled the controller uses multiple receive queues into which to de...

Страница 294: ...ould be taken into account when selecting the number of DIMMs to populate in the server Consult your motherboard documentation for details Intel QuickData Intel QuickData Technology allows VMware ESX...

Страница 295: ...7 288 Server Motherboard Server BIOS Chipset Drivers Tuning or enabling other system capabilities may further enhance adapter performance Readers should consult their server user guide Possible opport...

Страница 296: ...s on page 295 FreeBSD Utilities Package on page 296 Configuring the Boot ROM with sfboot on page 297 Upgrading Adapter Firmware with sfupdate on page 303 Performance Tuning on FreeBSD on page 305 Modu...

Страница 297: ...eBSD Feature Set Jumbo frames Support for MTUs Maximum Transmission Units to 9000 bytes See Configuring Jumbo Frames on page 295 Task offloads Support for TCP Segmentation Offload TSO Large Receive Of...

Страница 298: ...add or pkg_add The following instructions assume that the source package has been downloaded to the tmp directory 1 Ensure you are the root user If not su 2 To avoid using the previous driver that is...

Страница 299: ...Ensure that DHCP is available with PXE boot options Ensure that a TFTP server is available Ensure that a FreeBSD server is available This is required only to generate the FreeBSD PXE boot image Downlo...

Страница 300: ...e for an unattended installation Table 68 Installation Stages In Control Stages of Boot Setup needed BIOS PXE code on the adapter runs Adapter must be in PXE boot mode See Solarflare Boot Manager on p...

Страница 301: ...tings permanent create entries in the configuration file as described below Using IPv6 To configure using IPv6 create an IPv6 interface sfxge n interface with a link local IPv6 address by entering ifc...

Страница 302: ...fconfig sfxge n sfxge0 flags 8843 UP BROADCAST RUNNING SIMPLEX MULTICAST metric 0 mtu 1500 If you want to have an MTU configured when the interface is brought up add an mtu parameter to the single lin...

Страница 303: ...cd source package dir 3 Build and install a binary package from the source make install Alternatively to build and install in separate steps a Build a binary package from the source make package b In...

Страница 304: ...e target host The format for the parameters are parameter value Sfboot Command Line Options Table 70 lists the options for sfboot Table 71 lists the available global parameters and Table 72 lists the...

Страница 305: ...t values If an adapter is specified options for the given adapter are reset but global options shown in Table 71 are not reset Note that clear can also be used with parameters allowing you to reset to...

Страница 306: ...ilter security on non privileged functions This is for SFN7000 and SFN8000 series adapters only This reduces security in virtualized environments The default is disabled When enabled a function PF or...

Страница 307: ...meout timeout in nanoseconds Specifies the timeout in nanoseconds for RX event merging A timeout of 0 means that event merging is disabled Table 71 Sfboot Global Parameters Parameter Description Table...

Страница 308: ...apter booting delay time in seconds 0 256 boot type pxe uefi disabled Sets the adapter boot type effective on next boot pxe PXE Preboot eXecution Environment booting disabled Disable adapter booting p...

Страница 309: ...greater than zero Changes to this setting with sfboot require a cold reboot to become effective vf msix limit 1 2 4 8 16 32 64 128 256 The maximum number of interrupts a virtual function may use pf vl...

Страница 310: ...rading Adapter Firmware with sfupdate To Update Adapter Firmware Reinstall the sfutils package as described in FreeBSD Utilities Package on page 296 Sfupdate Command Usage The general usage for sfupda...

Страница 311: ...or scripts V version Display version information and exit i adapter sfxge n Specifies the target adapter when more than one adapter is installed in the machine sfxge n Adapter ifname or MAC address as...

Страница 312: ...page 306 Other Considerations on page 311 Introduction The Solarflare family of network adapters are designed for high performance network applications The adapter driver is pre configured with defau...

Страница 313: ...pter MTU Maximum Transmission Unit The default MTU of 1500 bytes ensures that the adapter is compatible with legacy 10 100Mbps Ethernet endpoints However if a larger MTU is used adapter throughput and...

Страница 314: ...val then starts during which no interrupt is raised Interrupt moderation settings are critical for tuning adapter latency Increasing the interrupt moderation interval will generate less interrupts red...

Страница 315: ...Checksum offload is changed dynamically using ifconfig with the following parameters rxcsum txcsum rxcsum6 txcsum6 Enable Rx and Tx checksum offload for IPv4 and IPv6 rxcsum txcsum rxcsum6 txcsum6 Dis...

Страница 316: ...config_sfxge n inet IPv4 address netmask netmask tso TCP and IP checksum offloads must be enabled for TSO to work NOTE Solarflare recommend that you do not disable this setting TCP Large Receive Offlo...

Страница 317: ...e size all influence the affect of the max buffer size values The minimum and default values can be left at their defaults minimum 4096 and default 87380 See https wiki freebsd org NetworkPerformanceT...

Страница 318: ...widths This is independent of the physical slot size used to connect the adapter The possible widths are multiples x1 x2 x4 x8 and x16 lanes of 2 5Gbps for PCIe Gen 1 5 0 Gbps for PCIe Gen 2 and 8 0G...

Страница 319: ...e channels to access main system memory Maximum memory performance is only achieved when the chipset can make use of all channels simultaneously This should be taken into account when selecting the nu...

Страница 320: ...t default Enabled TCP Protocol Tuning Leave at default Receive Side Scaling RSS Application dependent Buffer Allocation Method Leave at default Some applications may benefit from specific setting PCI...

Страница 321: ...nt speed not the supported speed reads back as x8 and 5GT s or x8 and 8GT s or x8 and Unknown CPU Power Management Disable with etc rc d powerd stop Memory bandwidth Ensure memory utilizes all memory...

Страница 322: ...bandwidth Ensure memory utilizes all memory channels on system motherboard Table 76 Forwarding Tuning Settings Tuning Parameter How Table 77 Driver Module Parameters Parameter Description Possible Va...

Страница 323: ...w counter grows uint 65536 tx_dpl_get_non_tcp_max Maximum number of non TCP packets queued in the software get list for a transmit queue This parameter can restrict utilizing the queue for non TCP e g...

Страница 324: ...cified PCIe function NOTE sysctl dev sfxge n stats output depends on the features supported by the adapter type tso_fw_assisted Whether to assist TSO using the firmware Applicable to SFN7000 and SFN80...

Страница 325: ...following mismatched unicast address and unicast promiscuous mode is not enabled packet is a pause frame packet has length discrepancy internal FIFO overflow condition length 60 bytes ev_rx_pause_frm...

Страница 326: ...ber of transmitted packets dropped because of FIFO overrun ev_tx_pkt_err Number of transmitted packets dropped because of driver error ev_tx_pkt_too_big Number of transmitted packets dropped because o...

Страница 327: ...many streams lro_new_stream Number of distinct streams the driver has tracked lro_drop_idle Number of streams discarded because they went idle lro_drop_closed Number of streams that have seen a FIN o...

Страница 328: ...max in Table 77 on page 315 tx_netdown_drops Number of packets early dropped by the driver because of link is down tso_pdrop_too_many Number of TSO packets partially dropped by the driver because TSO...

Страница 329: ...s 1522 with VLAN tag rx_ge_15xx_pkts Number of packets received where the length is between 1518 bytes 1522 with VLAN tag and 9000 bytes rx_errors Number of packets received with errors rx_fcs_errors...

Страница 330: ...o refill the queue with new descriptors A number of different steps can be tried to resolve this issue Disable the irqbalance daemon in the OS Distribute the traffic load across the available CPU core...

Страница 331: ...ngth is between 512 and 1023 bytes tx_1024_to_15xx_pkts Number of frames transmitted where the length is between 1024 and 1518 bytes 1522 with VLAN tag tx_ge_15xx_pkts Number of frames transmitted whe...

Страница 332: ...hey are periodically updated from the port and driver statistics See Table 81 Table 81 Netstat statistics Field Value Ipkts rx_pkts Ierrs rx_errors Idrop 0 Ibytes rx_octets Opkts tx_pkts Oerrs tx_erro...

Страница 333: ...number of features to support SR IOV These features can be summarized as follows PCIe Virtual Functions VF A PCIe physical function PF can support a configurable number of PCIe virtual functions In t...

Страница 334: ...ethtool as shown above The firmware update utility sfupdate and boot ROM configuration tool sfboot are available in the Solarflare Linux Utilities package SF 107601 LS issue 28 or later Platform suppo...

Страница 335: ...unsafe_assigned_interrupts 1 RHEL 7 options vfio_iommu_type1 allow_unsafe_assigned_interrupts 1 Alternative Routing ID Interpretation ARI The ARI extension to the PCI Express Base Specification extend...

Страница 336: ...p delay time 5 seconds Boot type Disabled Physical Functions per port 1 MSI X interrupt limit 32 Number of Virtual Functions 2 VF MSI X interrupt limit 8 Firmware variant full feature virtualization I...

Страница 337: ...for a PF must be a power of 2 switch mode mode default Specifies the mode of operation that the port will be used in default single PF created zero VFs created sriov SR IOV enabled single PF created...

Страница 338: ...e the virtual adapters and virtual ports as required Adapter firmware will also configure the firmware switching functions allowing packets to pass between PF and VFs or from VF to VF Figure 49 SR IOV...

Страница 339: ...ver kernel command line Refer to Platform support SR IOV on page 327 1 The example configures 1 PF per port default 2 VFs per PF sfboot switch mode sriov pf count 1 vf count 2 Solarflare boot configur...

Страница 340: ...ware addresses MAC addresses assigned to VFs in the above example output have been randomly generated by the host MAC addresses visible to the host will be replaced by libvirt generated MAC addresses...

Страница 341: ...count 1 The sfboot settings shown above are the default shipping state settings for the SFN7000 series adapter A cold reboot of the server is only required when changes are made using sfboot 3 Create...

Страница 342: ...6 errors 0 dropped 0 overruns 0 carrier 0 collisions 0 txqueuelen 0 RX bytes 55760 54 4 KiB TX bytes 468 468 0 b 8 Define a network in an XML file i e host network xml network name host network name...

Страница 343: ...address 52 54 00 96 0a 8a 3 source bridge br1 4 model type virtio 5 address type pci domain 0x0000 bus 0x00 slot 0x07 function 0x0 6 interface 1 Interface type must be specified by the user as bridge...

Страница 344: ...ion and increasing performance When using macvtap there is no link state propagation to the guest which is unable to identify if a physical link is up or down Macvtap does not currently forward multic...

Страница 345: ...start the GUI interface A VM can also be created from an existing VM XML file The following procedure assumes the VM is created The example procedure will create an interface configuration file and c...

Страница 346: ...n 0x0 6 interface 1 Interface type must be specified by the user as direct 2 The MAC address If not specified by the user this will be automatically assigned a random MAC address by libvirt 3 The sour...

Страница 347: ...F network interface and is not visible to the host OS Each macvtap instance builds over a different network interface so there is no implicit macvtap bridge Macvtap does not currently forward multicas...

Страница 348: ...flare Communications Device 1903 0000 03 00 4 Ethernet controller Solarflare Communications Device 1903 0000 03 00 5 Ethernet controller Solarflare Communications Device 1903 VFs will also be listed u...

Страница 349: ...type component is required for each VF 9 Restart the VM after editing the XML file virsh start vmname The passed through VF interface is visible when viewed from the GUI Virtual Machine Manager Figur...

Страница 350: ...automatically assigned by libvirt when the guest is started 5 The VF PCIe address as known by the guest will be added automatically by libvirt For further information about the direct passthrough conf...

Страница 351: ...the Ethernet interfaces and PCIe addresses VFs shown below in bold text lspci D d1924 0000 03 00 0 Ethernet controller Solarflare Communications SFC9120 rev 01 0000 03 00 1 Ethernet controller Solarfl...

Страница 352: ...utomatically reattached to the host after the guest exits If managed no the user must call virNodeDeviceDetach or use the command virsh nodedev detach before starting the guest or hot plugging the dev...

Страница 353: ...o a different Virtual Machine VM Each VM supports a TCP IP stack and Solarflare adapter driver sfc ko This mode allows switching between PFs via the Layer 2 switch function configured in firmware Figu...

Страница 354: ...sical Functions per port 4 MSI X interrupt limit 32 Number of Virtual Functions 0 VF MSI X interrupt limit 8 Firmware variant full feature virtualization Insecure filters Disabled VLAN tags None Switc...

Страница 355: ...l via sysfs the Solarflare net driver module option max_vfs can be used to enable VFs The max_vfs value applies to all adapters and can be set to a single integer i e all adapter physical functions wi...

Страница 356: ...e driver after setting changing this value onload_tool reload 7 6 Feature Summary Table 83 Feature Summary Default SRIOV Partitioning Partitioning SRIOV PFIOV Number of PFs per adapter num ports num p...

Страница 357: ...ort have the same expansion ROM configuration where PXE UEFI settings are stored This means that all PFs will PXE boot or none will attempt to PXE boot Users should ensure that a DHCP server responds...

Страница 358: ...ers provides accelerated cut through performance and is compatible with hypervisor based services and management tools The advanced design of Solarflare SFN7000 and SFN8000 series adapters incorporate...

Страница 359: ...hp Ensure the BIOS has all SR IOV Virtualization options enabled On a server with SR IOV correctly configured identify if Virtual Functions VF can be exposed to the host OS Refer to sfboot options bel...

Страница 360: ...rt SR IOV Features described in this chapter are not supported by Solarflare SFN5000 or SFN6000 series adapters Solarflare Driver Firmware Features described in the chapter require the following minim...

Страница 361: ...tilities on the ESXi host on page 355 Install Solarflare Drivers in the Guest on page 354 Partition the Adapter on page 370 Virtual Machine on page 358 Make PF Passthrough Devices available to the Gue...

Страница 362: ...mber VFs exposed per port to host and or Virtual Machine VM The number of MSI X interrupts assigned to each PF or VF Firmware Variant and switch mode To check the current adapter configuration run the...

Страница 363: ...or each physical port sfboot switch mode sriov pf count 1 vf count 4 When used without the adapter option the command applies to all adapters DirectPath I O PF Passthrough sfboot Settings The followin...

Страница 364: ...Following restart list VFs exposed in the host lspci grep Solarflare 0000 04 00 0 Network controller Solarflare SFC9120 vmnic6 0000 04 00 1 Network controller Solarflare SFC9120 vmnic7 0000 04 00 2 N...

Страница 365: ...later VM Compatibility The VM must be compatible with ESXi 5 5 or later When the VM is not compatible the following procedure via the vSphere Web Client will upgrade compatibility Locate the VM from t...

Страница 366: ...on Using ESXi Issue 20 Solarflare Communications 2017 359 8 9 List Adapters Web Client To list available adapters Navigate to the Host Networking Physical Adapters Selecting an adapter and clicking th...

Страница 367: ...Port Group Configuration Using the vSphere Web Client navigate to the host Right click and select All vCenter Actions Add Networking to display the Add Networking wizard Select Virtual Machine Port G...

Страница 368: ...V Virtualization Using ESXi Issue 20 Solarflare Communications 2017 361 When creating a new standard switch click the icon under Assigned adapters Select the required physical adapter s which as uplin...

Страница 369: ...e 20 Solarflare Communications 2017 362 Label the portgroup and assign a network label and VLAN ID if required VFs will later be assigned to the same portgroup and will be able to send receive traffic...

Страница 370: ...er User Guide SR IOV Virtualization Using ESXi Issue 20 Solarflare Communications 2017 363 The vSwitch and associated uplink adapter s topology can be viewed as follows Select the host Manage tab Netw...

Страница 371: ...OS installed A Solarflare SFN7000 or SFN8000 series adapter is physically installed on the host The Solarflare VMware net driver package is installed on the host A Solarflare net driver is installed i...

Страница 372: ...Settings dialog enable SRIOV and specify the number of VFs which can be used by the VM NOTE The number of virtual functions should not exceed the value set by the max vfs Solarflare driver option 5 T...

Страница 373: ...the Manage tab Settings VM Hardware then click the Edit button to display the VM Edit Settings window 4 To add a VF select Network from the New device drop down list and click the Add button 5 Expand...

Страница 374: ...Issue 20 Solarflare Communications 2017 367 7 Select Adapter Type as SR IOV passthrough 8 Select the physical adapter associated with the port group and which will back the passthrough virtual machin...

Страница 375: ...indow expand the Memory section and check the Reserve all guest memory All locked check box Reserving all memory is required for both SR IOV and DirectPath configurations because the adapter PF VF mus...

Страница 376: ...selected click the Manage tab and the VM Hardware option to view hardware configuration The Solarflare adapter PF and VF are listed in this window Listing Passthrough Devices in a Linux Guest When th...

Страница 377: ...partitioning The server must be cold power cycled When the server restarts PFs will be visible in the host lspci vvv grep Solarflare 0000 07 00 0 Ethernet controller Network controller Solarflare SFC9...

Страница 378: ...t and follows the procedure from the vmware documentation for DirectPath I O VMware Networking User Guide for ESXi 5 5 1 Navigate to the host Select the Manage tab Settings option 2 Under the Hardware...

Страница 379: ...as Available pending The host must be restarted to effect the changes Assign PF Passthrough Devices to the VM 1 Locate and select the VM in the vSphere Web Client 2 Power OFF the VM 3 Select the Manag...

Страница 380: ...dd for each PF to be passed through 9 Click the OK button when done 10 Power ON the VM Listing Devices in a Windows Guest Start the Solarflare Adapter Manager GUI by selecting Start select arrow icon...

Страница 381: ...so be invoked using the Solarflare supplied sfboot utility For instructions on the sfboot method refer to the sfboot commands in the relevant OS specific sections of this user guide Using the sfboot u...

Страница 382: ...upport on page 376 sfupdate Options for PXE upgrade downgrade on page 376 Starting PXE Boot on page 378 iPXE Image Create on page 382 Multiple PF PXE Boot on page 384 9 2 Solarflare Boot Manager The s...

Страница 383: ...evelopment mailing lists and participate to ensure the iPXE driver for Solarflare adapters operates correctly NOTE It is recommended that customers having support questions on the iPXE feature set wor...

Страница 384: ...ite img image dat Use the force option when downgrading Use the backup option to create a backup image dat file of the current firmware and Solarflare Boot Manager image sfupdate iPXE image To install...

Страница 385: ...ation Tool on page 190 From a Linux environment you can use the sfboot utility See Configuring the Boot Manager with sfboot on page 88 From a VMware environment you can use the sfboot utility See Sola...

Страница 386: ...enable ANSI terminal emulation on both the BIOS and your terminal Some BIOSs are known to not render the Solarflare Boot Manager properly when using vt100 terminal emulation 1 On starting or re start...

Страница 387: ...The Global Adapter Options menu is displayed 4 Select the required boot image a Use the arrow keys to highlight the Boot Image b Use the space bar to choose the required image c Press the Esc key to e...

Страница 388: ...the arrow keys to highlight the Boot Type b Use the space bar to select PXE Solarflare recommend leaving other settings at their default values For details on the default values for the various adapt...

Страница 389: ...Communications 2017 382 9 6 iPXE Image Create Solarflare do not provide pre built iPXE images The Solarflare iPXE boot ROM image can be generated from the rom o matic iPXE web builder wizard available...

Страница 390: ...vendor identifier 1924 b Enter the SFN8000 PCI Device Code 0a03 c Select the GIT version master is recommended 4 Generate the image file Click the Proceed button to start image generation then downloa...

Страница 391: ...ical interface sfboot Solarflare boot configuration utility v4 5 0 eth2 Boot image Option ROM only Link speed Negotiated automatically Link up delay time 5 seconds Banner delay time 2 seconds Boot ski...

Страница 392: ...s are not available Adapter wide options are available via eth2 00 0F 53 25 39 90 eth5 Interface specific boot options are not available Adapter wide options are available via eth2 00 0F 53 25 39 90 U...

Страница 393: ...e a server is moved to a different part of the network PXE booting will then fail To correct these settings you must use the Solarflare Boot Manger GUI utility You cannot use the sfboot command line u...

Страница 394: ...Settings Resetting an adapter does not change the boot ROM image To reset an adapter to its default settings 1 Start or re start the server and when prompted press Ctrl B The Solarflare Boot Manager...

Страница 395: ...s Enter The settings are reset to the defaults NOTE Under Windows you can also reset the settings by clicking Default from SAM Table 9 8 lists the various adapter settings and their default values Tab...

Страница 396: ...Solarflare Server Adapter User Guide Solarflare Boot Manager Issue 20 Solarflare Communications 2017 389 DHCP Vendor MPIO attempts MSIX Limit 32 Table 85 Default Adapter Settings Setting Default Value...

Страница 397: ...ith SAM 166 Ethtool Configure Interrupt moderation on Linux 105 Configure Interrupt moderation on VMware 284 Configure segmentation offload 107 Configuring adapter 70 Running adapter diagnostics on Li...

Страница 398: ...ies Configuration 184 NIC Partitioning 74 O OpenOnload 2 P PCI Express Lane Configuration On FreeBSD 311 On Linux 115 On VMware 286 On Windows 236 PF IOV 346 Powerd Service Tuning on FreeBSD 312 PXE C...

Страница 399: ...T TCP Protocol Tuning On Linux 109 On VMware 286 On Windows 238 Teaming see also sfteam on Windows 200 Adding adapters with SAM 172 Configure on VMware 267 Deleting from SAM 173 Key adapter 225 Reconf...

Страница 400: ...Solarflare Server Adapter User Guide Index Issue 20 Solarflare Communications 2017 393...

Отзывы:

Нет отзывов