Broadcom BCM5706C Скачать руководство пользователя страница 49 | Manualshive

Страница: 49 / 247

background image

User Guide

NetXtreme II

January 2010

Bro a d c o m Co rp o r a ti o n

Document

ENGSRVT52-CDUM100-R

Application Considerations

Page 49

Oracle

In our Oracle Solution Stacks, we support adapter teaming in both the private network (interconnect between RAC nodes)
and public network with clients or the application layer above the database layer.

Figure 8: Clustering With Teaming Across Two Switches

«
...
47
48
49
50
51
...
»

Содержание BCM5706C

Страница 1: ...notice 2010 Broadcom Corporation All rights reserved This document is protected by copyright and is distributed under licenses restricting its use copying distribution and decompilation No part of th...

Страница 2: ...ancing are among the trademarks of Broadcom Corporation and or its affiliates in the United States certain other countries and or the EU Microsoft and Windows are trademarks of Microsoft Corporation L...

Страница 3: ...ired for all offloading technologies Enterprise networks that use multiple protocols and multiple network fabrics benefit from the C NICs ability to combine data communications storage and clustering...

Страница 4: ...configuration software suite Supports PXE 2 0 specification Linux Red Hat PXE Server SUSE Linux Enterprise Server Windows 2000 Server Windows Server 2003 Windows Server 2008 Windows Server 2008 R2 In...

Страница 5: ...ork adapters and 1880 fully offloaded TCP connections for 10 Gbps network adapters The TOE support on the adapter significantly reduces the host CPU utilization while preserving the implementation of...

Страница 6: ...ycles ASIC WITH EMBEDDED RISC PROCESSOR The core control for Broadcom NetXtreme II adapters resides in a tightly integrated high performance ASIC The ASIC includes a RISC processor This functionality...

Страница 7: ...Page 7 SUPPORTED OPERATING ENVIRONMENTS The Broadcom NetXtreme II adapter has software support for the following operating systems Microsoft Windows 32 bit and 64 bit extended Microsoft Windows Vista...

Страница 8: ...ngle LED located adjacent to the port connector as described in Table 2 Broadcom Advanced Control Suite 3 also provides information about the status of the network link and activity see Viewing Vital...

Страница 9: ...you are using teaming software For more information on network adapter teaming concepts see Broadcom Gigabit Ethernet Teaming Services LOAD BALANCING AND FAULT TOLERANCE Teaming provides traffic load...

Страница 10: ...orts balancing IP traffic across multiple adapters team members in a bidirectional manner In this type of team all adapters in the team have separate MAC addresses This type of team provides automatic...

Страница 11: ...ype of team is very similar to the Link Aggregation 802 3ad type of team in that all adapters in the team are configured to receive packets for the same MAC address The Generic Trunking FEC GEC 802 3a...

Страница 12: ...ving a portion of the total traffic Standby interfaces take over in the event that all primary interfaces have lost their links Failover teaming provides redundant adapter operation fault tolerance in...

Страница 13: ...lt tolerant NOTE If you do not enable LiveLink when configuring teams disabling Spanning Tree Protocol STP or enabling Port Fast at the switch is recommended This minimizes the downtime due to the spa...

Страница 14: ...SUMMARY Glossary Teaming Concepts Software Components Hardware Requirements Teaming Support by Processor Configuring Teaming Supported Features by Team Type Selecting a Team Type This section describ...

Страница 15: ...heavily offloaded to the hardware where much of the Layer 3 IP and Layer 4 TCP processing is done in the hardware to improve performance LACP Link Aggregation Control Protocol Link Aggregation 802 3ad...

Страница 16: ...eme II User Guide January 2010 Broadcom Corporation Page 16 Executive Summary Document ENGSRVT52 CDUM100 R WINS Windows name service WLBS Windows Load Balancing Service Table 1 Glossary Cont Item Defi...

Страница 17: ...affic flows across applications Layer 4 protocols such as TCP or UDP are most commonly used in today s networks The combination of the IP address and the TCP port number is called a socket Ethernet de...

Страница 18: ...he Smart Load Balancing and Failover type of team provides both load balancing and failover when configured for load balancing and only failover when configured for fault tolerance This type of team w...

Страница 19: ...otocol address This G ARP is unicast with the MAC and IP Address of a client machine in the target physical and protocol address respectively This causes the target client to update its ARP cache with...

Страница 20: ...he link to be in a link up state Because manual aggregation provides for the activation of a member link without performing the LACP message exchanges it should not be considered as reliable and robus...

Страница 21: ...tion with the SLB modes of teaming Other teaming modes Generic Trunking or Link Aggregation can still be used on TOE capable devices but if those other modes are enabled the TOE feature is disabled Si...

Страница 22: ...Teaming Software Component Software Component Broadcom Name Network Adapter Operating System System Architecture Windows File Name Virtual Bus Driver VBD BCM5706 BCM5708 BCM5709 32 bit bxvbdx sys BCM...

Страница 23: ...ision domain operate in half duplex mode Although half duplex mode is supported for Gigabit Ethernet adapters in the IEEE 802 3 specification half duplex mode is not supported by the majority of Gigab...

Страница 24: ...rison across the team types Use this table to determine the best type of team for your application The teaming software supports up to eight ports in a single team and up to four teams in a single sys...

Страница 25: ...Yes Yes No must be the same speed Yes Load balances TCP IP No Yes Yes Yes Mixed vendor teaming Yesb Yesb Yesb Yesb Load balances non IP No Yes IPX outbound traffic only Yes Yes Same MAC address for a...

Страница 26: ...s the need for additional network bandwidth and fault tolerance Teaming offers link aggregation and fault tolerance to meet both of these requirements Preference teaming should be selected in the foll...

Страница 27: ...TCP IP and IPX and appears as a virtual adapter This virtual adapter inherits the MAC Address of the first port initialized in the team A Layer 3 address must also be configured for the virtual adapte...

Страница 28: ...to the selected physical adapter for transmission The flow classification involves an efficient hash computation over known protocol fields The resulting hash value is used to index into an Outbound F...

Страница 29: ...operating system protocol stack will always associate the logical IP address with the MAC address of the primary physical adapter Like the outbound case there is an Inbound Flow Head Hash Table Each e...

Страница 30: ...react to its contents before sending it out through a particular physical interface Though the BASP driver can process each outgoing packet in near constant time some applications that may already be...

Страница 31: ...4 flows Outbound Load Balancing using MAC Address No Outbound Load Balancing using IP Address Yes Multivendor Teaming Supported must include at least one Broadcom Ethernet adapter as a team member App...

Страница 32: ...ation Both the BASP and the switch continually monitor their ports for link loss In the event of link loss on any port traffic is automatically diverted to other ports in the team Network Communicatio...

Страница 33: ...switches that support IEEE 802 3ad Link Aggregation dynamic mode using LACP Inbound load balancing is switch dependent In general the switch traffic is load balanced based on L2 addresses In this cas...

Страница 34: ...BASP Outgoing packet management BASP LiveLink support Yes Failover event Loss of link Failover time 500 ms Fallback time 1 5 sb approximate MAC address Different Multivendor teaming Yes Generic Trunki...

Страница 35: ...ing at different link speeds Hot replace Yes Hot add Yes Hot remove Yes Link speed support Different speeds Frame protocol All Incoming packet management Switch Outgoing packet management BASP Failove...

Страница 36: ...ay not support this feature if ASF IPMI is also enabled b Must be supported by all adapters in the team c Only for Broadcom adapters d As a PXE sever only not as a client A team does not necessarily i...

Страница 37: ...tly sequenced header based on the single header originally provided TCP OFFLOAD ENGINE TOE The TCP IP protocol suite is used to provide transport services for a wide range of applications for the Inte...

Страница 38: ...and not in a multivendor teaming environment The BASP intermediate driver supports VLAN tagging One or more VLANs may be bound to a single instance of the intermediate driver WAKE ON LAN Wake on LAN...

Страница 39: ...neric Trunking and Link Aggregation do not work across switches because each of these implementations requires that all physical adapters in a team share the same Ethernet MAC address It is important...

Страница 40: ...ar scenario occurs when Gray attempts to ping Blue An ICMP Request goes out on 5E C9 destined for Blue 82 82 but cannot get there Top Switch does not have an entry for 82 82 in its CAM table because t...

Страница 41: ...to Blue and Gray to reach each other without any problems Note the additional entries in the CAM table for both switches The link interconnect is critical for the proper operation of the team As a res...

Страница 42: ...ral Network Considerations Document ENGSRVT52 CDUM100 R Figure 5 represents a failover event in which the cable is unplugged on the Top Switch port 4 This is a successful failover with all stations pi...

Страница 43: ...an take up to 50 seconds During this time end to end communications are interrupted The use of Spanning Tree is not recommended for ports that are connected to end stations because by definition an en...

Страница 44: ...Network Configurations SLB Teams SLB Team Connected to a Single Hub Generic and Dynamic Trunking FEC GEC IEEE 802 3ad SLB teaming can be used with 10 100 hubs but it is only recommended for troublesh...

Страница 45: ...however would be degraded Figure 6 Team Connected to a Single Hub Generic and Dynamic Trunking FEC GEC IEEE 802 3ad FEC GEC and IEEE 802 3ad teams cannot be connected to any hub configuration These te...

Страница 46: ...te private subnetwork The other adapter is used for client communications and is referred to as the public adapter Multiple adapters may be used for each of these purposes private intracluster communi...

Страница 47: ...r cluster node one private and two public On each node the two public adapters are teamed and the private adapter is not Teaming is supported across the same switch or across two switches Figure 8 sho...

Страница 48: ...A and in band OMSA management of the nodes in the cluster It can also be used for job scheduling and monitoring In our current HPCC offerings only one of the on board adapters is used If Myrinet or IB...

Страница 49: ...lication Considerations Page 49 Oracle In our Oracle Solution Stacks we support adapter teaming in both the private network interconnect between RAC nodes and public network with clients or the applic...

Страница 50: ...sume a high percentage of the network link bandwidth thus impacting production data and tape backup performance Network backups usually consist of a dedicated backup server running with tape backup so...

Страница 51: ...One possible path that Client Server Red can use to send data to the backup server is Example Path Client Server Red sends data through Adapter A Switch 1 Backup Server Adapter A The designated path i...

Страница 52: ...without interruption during the link failure All failover processes within the network are transparent to tape backup software applications To understand how backup data streams are directed during n...

Страница 53: ...User Guide NetXtreme II January 2010 Broadcom Corporation Document ENGSRVT52 CDUM100 R Application Considerations Page 53...

Страница 54: ...revision and software revision to be supported 4 To be teamed adapters should be members of the same VLAN In the event that multiple teams are configured each team should be on a separate network 5 D...

Страница 55: ...for Generic Trunking FEC GEC 802 3ad Draft Static type of teaming and that it matches the adapter teaming type If the system is configured for an SLB type of team make sure the corresponding switch p...

Страница 56: ...o eight ports can be assigned to a team Question What is the maximum number of teams that can be configured on the same server Answer Up to eight teams can be configured on the same server Question Wh...

Страница 57: ...river revision is to physically locate the driver file and check the properties Question Can SLB detect a switch failure in a Switch Fault Tolerance configuration Answer No SLB can only detect the los...

Страница 58: ...able 8 Base Driver Event Log Messages Message Number Severity Message Cause Corrective Action 1 Error Failed to allocate memory for the device block Check system memory resource usage The driver canno...

Страница 59: ...ion is required 13 Error Medium not supported The operating system does not support the IEEE 802 3 medium Reboot the operating system run a virus check run a disk check chkdsk and reinstall the operat...

Страница 60: ...en manually configured for the selected line speed and duplex settings No action is required 22 Error Network controller failed initialization because it cannot allocate system memory Insufficient sys...

Страница 61: ...s stopped No action is required 7 Error Could not allocate memory for internal data structures The driver cannot allocate memory from the operating system Close running applications to free memory 8 W...

Страница 62: ...Informational Network adapter is activated and is participating in network traffic A physical adapter has been added to or activated in a team No action is required 18 Informational Network adapter is...

Страница 63: ...ated from the others increasing bandwidth efficiency within each logical group Up to 64 VLANs 63 tagged and 1 untagged can be defined for each Broadcom adapter on your server depending on the amount o...

Страница 64: ...om adapter installed All three IP subnets are accessed via the single physical adapter interface The server is attached to one of the switch ports which is configured for VLANs 1 2 and 3 Both the adap...

Страница 65: ...ver with a single adapter can have a logical presence on multiple IP subnets With multiple VLANs in a team a server can have a logical presence on multiple IP subnets and benefit from load balancing a...

Страница 66: ...m Current implementation will provide team information and information of physical network adapters in the team Broadcom Advanced Server Program provides events through event logs Users can use the Ev...

Страница 67: ...cipating in a team and the virtual NIC adapters created as the result of teaming Non teamed NIC adapters are not monitored at this time The BASP configuration data includes information such as team ID...

Страница 68: ...dy provided by the standard SNMP Management Network Interface information The extensible agent provides in depth information about a Broadcom NetXtreme II adapter such as MAC address Bound IP address...

Страница 69: ...work interface cards SYSTEM REQUIREMENTS Before you install a Broadcom NetXtreme II adapter verify that your system meets the following hardware and operating system requirements HARDWARE REQUIREMENTS...

Страница 70: ...precautions to protect yourself and to prevent damage to the system components Remove any metallic objects or jewelry from your hands and wrists Make sure to use only insulated or nonconducting tools...

Страница 71: ...th the PCI Express connector slot in the system 5 Applying even pressure at both corners of the card push the adapter card into the slot until it is firmly seated When the adapter is properly seated t...

Страница 72: ...m 328 ft 1 1000BASE T signaling requires four twisted pairs of Category 5 balanced cabling as specified in ISO IEC 11801 2002 and ANSI EIA TIA 568 B 2 Category 5 is the minimum requirement Category 5e...

Страница 73: ...ace UNDI and provides a network interface in the Linux remotely booted client environment Intel APITEST The Broadcom PXE driver passes all API compliance test suites MS DOS UNDI The MS DOS Universal N...

Страница 74: ...u can use the MBA Configuration Menu to configure the MBA driver one adapter at a time as described below or you can use the Broadcom NetXtreme II User Diagnostics MS DOS based application to simultan...

Страница 75: ...linuz and initial ram disk initrd which are located on the Red Hat disk 1 images pxeboot vmlinuz images pxeboot initrd img Refer to the Red Hat documentation for instructions on how to install PXE Ser...

Страница 76: ...option located on the General Parameters screen of the iSCSI Configuration utility See Table 1 for more information on all General Parameters screen configuration options SUPPORTED OPERATING SYSTEMS F...

Страница 77: ...oot host software will configure a stateless link local address and or stateful address if DHCPv6 is present and used Enabled Router Solicit packets are sent out up to three times with 4 second interv...

Страница 78: ...HAP Authentication Disabled DHCP Vendor ID Controls how the iSCSI boot host software interprets the Vendor Class ID field used during DHCP If the Vendor Class ID field in the DHCP Offer packet matches...

Страница 79: ...IP Address TCP Port Boot LUN iSCSI Name 10 Select ESC to return to the Main menu 11 A second iSCSI boot adapter can be configured for redundancy in the event the primary adapter fails to boot To conf...

Страница 80: ...string 11 22 33 44 55 66 corresponds to the controller s MAC address If DHCP option 43 IPv4 only is used then any settings on the Initiator Parameters 1st Target Parameters or 2nd Target Parameters sc...

Страница 81: ...ntry by changing Group to base and Start to 0 When the software initiator mode operating system installation is used the Windows operating system install must be performed in two stages In the first s...

Страница 82: ...lace the iSCSI boot adapter before the hard drive in the boot menu 5 Reboot the host and boot into the OS in the local hard drive 6 Launch Windows compatible cloning software such as Ghost32 and write...

Страница 83: ...t provide configuration information to the DHCP client For iSCSI boot Broadcom adapters support the following DHCP configurations DHCP Option 17 Root Path DHCP Option 43 Vendor Specific Information DH...

Страница 84: ...option 43 requires more configuration than DHCP option 17 but it provides a richer environment and provides more configuration options Broadcom recommends that customers use DHCP option 43 when perfor...

Страница 85: ...value is BRCM ISAN as shown in General Parameters of the iSCSI Boot Configuration menu The content of Option 16 should be 2 byte length DHCP Vendor ID DHCPv6 Option 17 Vendor Specific Information DHCP...

Страница 86: ...arameters See Configuring the iSCSI Boot Parameters Changing the BIOS Boot Order Change the BIOS boot order to the following iSCSI Option ROM CDROM DVD Installing to Target 1 Update the iSCSI boot Opt...

Страница 87: ...eating and installing a Microsoft OS directly to the target through Broadcom iSCSI solution Windows Server 2003 OIS Windows Server 2008 Non OIS Windows Server 2008 OIS Windows Server 2003 OIS Installa...

Страница 88: ...s a session with the remote iSCSI target 11 Continue to boot with the Windows Server installation CD 12 Press F6 when prompted to install additional drivers 13 Press S to install additional drivers In...

Страница 89: ...by selecting CTRL S when prompted 11 Enter the iSCSI configuration menu by selecting CTRL K 12 Enter General Parameters and set Boot to iSCSI target to Enable 13 Exit and save the iSCSI configuration...

Страница 90: ...igured as DHCP in other words it will not occur if the IP addresses are statically configured in the iSCSI configuration program Solution Perform the first boot with both cables attached and ensure th...

Страница 91: ...owing process enables your system to take advantage of Broadcom s iSCSI offload feature Installing Broadcom Drivers and Management Applications Installing the Microsoft iSCSI Initiator Configuring Bro...

Страница 92: ...used to manage all of Broadcom s network adapters and advanced features For more information see Using Broadcom Advanced Control Suite 3 1 Open BACS 3 2 Select the Broadcom NetXtreme II C NIC iSCSI ad...

Страница 93: ...fload Page 93 3 Select the Configuration tab 4 DHCP is the default for IP address assignment but this can be changed to static IP address assignment if this is the preferred method of IP address assig...

Страница 94: ...NetXtreme II User Guide January 2010 Broadcom Corporation Page 94 iSCSI Offload Document ENGSRVT52 CDUM100 R 5 Select Apply and close BACS...

Страница 95: ...IP address has been configured for the iSCSI adapter you need to use Microsoft Initiator to configure and add a connection to the iSCSI target using Broadcom iSCSI adapter See Microsoft s user guide...

Страница 96: ...ser Guide January 2010 Broadcom Corporation Page 96 iSCSI Offload Document ENGSRVT52 CDUM100 R 4 Select the Discovery tab and click Add to add a target portal 5 Enter the IP address of the target and...

Страница 97: ...User Guide NetXtreme II January 2010 Broadcom Corporation Document ENGSRVT52 CDUM100 R iSCSI Offload Page 97 6 From the General tab select Broadcom NetXtreme II C NIC iSCSI Adapter from Local adapter...

Страница 98: ...NetXtreme II User Guide January 2010 Broadcom Corporation Page 98 iSCSI Offload Document ENGSRVT52 CDUM100 R 7 Select the IP address for the adapter from Source IP...

Страница 99: ...User Guide NetXtreme II January 2010 Broadcom Corporation Document ENGSRVT52 CDUM100 R iSCSI Offload Page 99 8 Click OK to close Advanced setting and then OK to add the target portal...

Страница 100: ...nuary 2010 Broadcom Corporation Page 100 iSCSI Offload Document ENGSRVT52 CDUM100 R 9 From the Targets tab select the target and click Log On to log into your iSCSI target using the Broadcom iSCSI ada...

Страница 101: ...e Disk Manager NOTES No teaming support for iSCSI enabled adapters is available at this point MPIO handles all failover and link aggregation for iSCSI traffic Network teaming and MPIO are supported co...

Страница 102: ...te the connection to the target A Use Microsoft iSCSI Software Initiator version 2 08 or later Q How do I know that the connection is offloaded A Use Microsoft iSCSI Software Initiator From a command...

Страница 103: ...llowing are installed when running the installer Control Suite Broadcom Advanced Control Suite 3 BACS 3 BASP Installs Broadcom Advanced Server Program SNMP Installs the Simple Network Management Proto...

Страница 104: ...OTE Before starting the installation verify that NET Framework 2 0 or above is installed This ensures optimal performance of BACS 3 1 Insert the installation CD into the CD or DVD drive 2 On the insta...

Страница 105: ...og file Type the following setup s v qn L f ia32 1testlog txt The 1testlog txt log file will be created at f ia32 To perform a silent uninstall from any folder on the hard drive msiexec x 26E1BFB0 E87...

Страница 106: ...roadcom Management Programs and then click Change 3 Click Next to continue 4 Click Modify to change program features 5 Click Next to continue 6 Click on an icon to change how a feature is installed 7...

Страница 107: ...Document ENGSRVT52 CDUM100 R Removing Management Applications Page 107 1 In Control panel double click Add or Remove Programs 2 Click Broadcom Management Programs and then click Remove 3 Reboot your...

Страница 108: ...STALLATION REQUIREMENTS Before you can successfully install the NDIS2 driver software the Broadcom network adapter must be physically installed in the server Networking software that is appropriate to...

Страница 109: ...s ENTER 5 Close the command prompt window by typing exit and then pressing ENTER 6 Start Windows Explorer 7 Open the NCADMIN folder and double click ncadmin exe 8 Follow the on screen instructions to...

Страница 110: ...otman DriverName PROTMAN PRIORITY MS NDISHLP MS NDISHLP DriverName ndishlp BINDINGS ms ne2clone ms nwlink DriverName nwlink FRAME Ethernet_802 2 BINDINGS MS NE2CLONE LANABASE 0 Example Protocol ini fi...

Страница 111: ...rt basic maxconnections 8 network drivers netcard BXND20X dos transport ndishlp sys netbeui devdir A NET LoadRMDrivers yes 3 Copy BXND20X dos to A Net 4 Create the appropriate Autoexec bat file in dri...

Страница 112: ...C LANMAN PROTMAN DOS DEVICE C LANMAN BXND20X DOS DEVICE C LANMAN NETBEUI DOS 4 Edit the Autoexec bat file by adding the following lines C LANMAN NETBIND EXE C LANMAN NET START WORKSTATION C LANMAN NE...

Страница 113: ...miliar with how to configure PCI devices A PCI device scan utility is needed to find this information LineSpeed Specifies the speed of the network connection in Mbit s Requires the decimal number 10 1...

Страница 114: ...eme II User Guide January 2010 Broadcom Corporation Page 114 Using Keywords for the Drivers Document ENGSRVT52 CDUM100 R AcceptAllMC Informs the driver to deliver all multicast packets to the upper pr...

Страница 115: ...ware and or the protocol stacks of communicating systems An intervening layer called the link support layer LSL is responsible for routing different packet types between local area network LAN drivers...

Страница 116: ...stem configuration and the Broadcom banner the adapter name the MAC address the IRQ number the detected line speed and the bus number BusNum and device number DevNum of the adapter are displayed If th...

Страница 117: ...me BIND BX2 BIND BX2 LINK DRIVER BX2 BUSNUM 1 DEVNUM 9 FRAME frame 1 name LINK DRIVER BX2 BUSNUM 1 DEVNUM A FRAME frame 2 name The LineSpeed keyword is used to force the speed of the network connectio...

Страница 118: ...llation Requirements 1 Verify that the Broadcom Ethernet adapter driver software has been installed on your computer and install it if it has not see Installing the Driver Software Otherwise you canno...

Страница 119: ...em and each adapter must be loaded in a specific order BUSNUM specifies the PCI bus number and DEVNUM specifies the device number that is assigned to the board when it is configured by the PCI BIOS Th...

Страница 120: ...SUBVID cannot be used with BUSNUM DEVNUM or FUNNUM The MODEL keyword is for loading the driver only on the first network interface card found that has a subvendor ID SVID that matches the one set in...

Страница 121: ...eaming with Channel Bonding Statistics Linux iSCSI Offload INTRODUCTION This section discusses the Linux drivers for the Broadcom NetXtreme II network adapters Table 1 Broadcom NetXtreme II Linux Driv...

Страница 122: ...licon revisions of B2 or later the bnx2 driver reports support for Magic Packet WOL via ethtool Enabling support via ethtool is mandatory to successfully wake the system To determine the silicon revis...

Страница 123: ...TAR source packages The supplemental tar file contains additional utilities such as patches and driver diskette images for network installation Updated open iSCSI components are released in source RP...

Страница 124: ...ECS netxtreme2 spec for RPM version 4 x x NOTE During your attempt to install a source RPM package the following message may be displayed error cannot create sourcedir usr src redhat SOURCES The most...

Страница 125: ...2 version tar gz 2 Build the driver bnx2 ko or bnx2 o as a loadable module for the running kernel cd netxtreme2 version make 3 Test the driver by loading it first unload the existing driver if necessa...

Страница 126: ...tware Components Document ENGSRVT52 CDUM100 R 1 Unload existing driver and kill user daemon if necessary Manual rmmod bnx2i pkill 9 bnx2id Using service service bnx2id stop 2 Load the iSCSI driver and...

Страница 127: ...ng to remove it rpm e netxtreme2 REMOVING THE DRIVER FROM A TAR INSTALLATION NOTE The examples used in this procedure refer to the bnx2 driver but also apply to the bnx2x and bnx2i drivers If the driv...

Страница 128: ...nx2 Driver bnx2x Driver bnx2i Driver BNX2 DRIVER disable_msi The disable_msi optional property can be supplied as a command line argument to the insmod or modprobe command The property can also be set...

Страница 129: ...the system insmod bnx2x ko int_mode 1 or modprobe bnx2x int_mode 1 Set the int_mode parameter to 2 to force using MSI mode on all NetXtreme II adapters in the system insmod bnx2x ko int_mode 2 or modp...

Страница 130: ...roper knowledge of iSCSI protocol details users are advised not to experiment with these parameters en_tcp_dack Enable TCP Delayed ACK enables disables TCP delayed ACK feature on offloaded iSCSI conne...

Страница 131: ...UT and iSCSI CMD pdus are intermixed Newer iSCSI target systems are immune to this condition and this parameter is turned ON for quite some time Defaults 1 Values Binary parameter 0 1 The parameters c...

Страница 132: ...ntrol Autonegotiation with RX and TX advertised MTU 1500 range is 46 9000 RX Ring Size 4078 range is 0 4078 TX Ring Size 4078 range is MAX_SKB_FRAGS 4 4078 MAX_SKB_FRAGS varies on different kernels an...

Страница 133: ...BCM5708 1000Base T B0 PCI X 64 bit 133MHz found at mem f6000000 IRQ 16 node addr 0010180476ae cnic Added CNIC device eth0 Link Up and Speed Indication bnx2 eth NIC Link is Up 1000 Mbps full duplex Lin...

Страница 134: ...A to which target node is bound using iscsi transport name Target cannot be reached on any of the CNIC devices bnx2i check route cannot connect using cnic Network route is assigned to network interfac...

Страница 135: ...si_error reject pdu len error bnx2i iscsi_error async pdu len error bnx2i iscsi_error nopin pdu len error bnx2i iscsi_error pend r2t in cleanup bnx2i iscsi_error IP fragments rcvd bnx2i iscsi_error IP...

Страница 136: ...h the Linux drivers you can team adapters together using the bonding kernel module and a channel bonding interface For more information see the Channel Bonding information in your operating system doc...

Страница 137: ...APPLICATIONS Install and run the open iscsi programs iscsid and iscsiadm from the Broadcom distributed open iscsi packages Refer to Packaging for details All pre installed open iscsi packages need to...

Страница 138: ...sport name binding can be obtained by executing the following dmesg grep bnx2i netif Sample output in a system with two NetXtreme II devices bnx2i netif eth1 iscsi bcm570x 050000 bnx2i netif eth0 iscs...

Страница 139: ...on on sq_size and rq_size Driver logs the following message to syslog when the maximum allowed connection offload limit is reached bnx2i unable to allocate iSCSI context resources LINUX ISCSI OFFLOAD...

Страница 140: ...be installed before the Broadcom NetXtreme II Gigabit Ethernet adapter can be used with your Novell NetWare system Before you can successfully install the adapter driver for Novell NetWare the adapter...

Страница 141: ...have the CD mounted as a NetWare volume press F3 and type Volume Name NetWare Driver as the source path where Volume Name is the name of the NetWare volume for the CD and NetWare Driver is the directo...

Страница 142: ...or automatic link setup auto negotiation AUTO 10FD 10HD 100FD 100HD The default setting is AUTO Note that 1000 Mbit s speed is auto detected and cannot be forced Link This keyword is set to AUTO to al...

Страница 143: ...Disabled Max 100 The default value is 75 CheckSum Enables or disables the transmit and receive checksum offload feature Checksum offload support is only for TCP IP packets therefore the default setti...

Страница 144: ...HIN that may be the physical slot number on a slot based bus such as PCI SLOT n Jumbo Enables jumbo frame support When enabled jumbo packets of up to 9000 bytes are supported Jumbo 1536 9000 The defau...

Страница 145: ...nk and data integrity even if the adapter is attached to a questionable cable and or switch For example an adapter that is trying to connect at 1000 Mbit s on a Category 3 cable ordinarily would not l...

Страница 146: ...TOBER SUNDAY LAST 2 00 00 AM SET Time Zone TPST8TPDT Note The Time zone information mentioned above should always precede the SERVER name WARNING FILE SERVER NAME DL760 WARNING If you change the name...

Страница 147: ...nning of each command line or by deleting the statement Example LOAD BX2 SLOT 2 FRAME Ethernet_802 2 NAME BX2_1_E82 BIND IPX BX2_1_E82 NET FAFD3D25 LOAD BX2 SLOT 2 FRAME Ethernet_802 3 NAME BX2_1_E83...

Страница 148: ...to install the driver software for that device Both a graphical interactive installation mode see Using the Installer and a command line silent mode for unattended installation see Using Silent Insta...

Страница 149: ...nformation about unattended installs refer to the Silent txt file in the DrvInst folder To perform a silent install from within the installer source folder Type the following setup s v qn or msiexec i...

Страница 150: ...txt REMOVING THE DEVICE DRIVERS NOTE Windows Server 2003 Windows Server 2008 and Windows Server 2008 R2 provide the Device Driver Rollback feature to replace a device driver with one that was previous...

Страница 151: ...required in Windows PE the only supported driver is the monolithic driver for the NetXtreme II adapter in this environment as well Place the b06nd inf file in the INF directory within the Windows PE i...

Страница 152: ...copype cmd arch destination For example copype x86 c VistaPEx86 NOTE The directory structure c VistaPEx86 is used throughout this procedure 3 Mount the base image to a local directory so that you can...

Страница 153: ...table CD ROM 1 On your technician computer at the command prompt create an iso file by typing oscdimg n bc VistaPEx86 etfsboot com c VistaPEx86 ISO C VistaPEx86 VistaPEx86 iso 2 Burn the iso image to...

Страница 154: ...network adapter 1 In Control Panel click Broadcom Control Suite 3 2 Click the Advanced section of the Configurations tab 3 See Setting Adapter Properties for a detailed description of the available pr...

Страница 155: ...ower management To enable Wake on LAN WOL when the computer is on standby click Allow the device to bring the computer out of standby box If you select Only allow management stations to bring the comp...

Страница 156: ...NCED CONTROL SUITE 3 OVERVIEW Broadcom Advanced Control Suite 3 BACS 3 is an integrated utility that provides useful information about each network adapter that is installed in your system BACS 3 also...

Страница 157: ...UITE 3 Start BACS 3 Click the tab that provides the information of interest or from which to perform a desired test diagnostic analysis or set adapter properties To create a team see Configuring Teami...

Страница 158: ...d in the Device Management pane the tabs showing the information and options that are available for the item appear Because the BACS 3 interface is context sensitive only the information that applies...

Страница 159: ...hows its status An icon next to a device name that appears normal means the device is connected and working X A red X that appears on the device s icon indicates the device is currently not connected...

Страница 160: ...Windows taskbar when the program is installed Use the Options window to turn this icon on or off To enable or disable the BACS tray icon 1 From the Tools menu select Options 2 In the Options window se...

Страница 161: ...dapters that are installed in your system such as the link status of the adapter and general network connectivity To view Vital Signs information for any installed network adapter click the name of th...

Страница 162: ...rk adapter is loaded onto the system this parameter will display the iSCSI MAC address IPv4 DHCP The IP address is from a DHCP server if the value is Enable IP Address The network address associated w...

Страница 163: ...er hardware rather than by the host CPU LiveLink IP Address The network address of the LiveLink enabled adapter Local Connection Identifies the module to which the blade server is attached Chassis SW...

Страница 164: ...adapter driver Driver Version The current version of the adapter driver Driver Date The creation date of the adapter driver VIEWING RESOURCE INFORMATION The Resources section of the Information tab d...

Страница 165: ...dapters Bridge Lanes The number of PCI E lanes connected to the bridge This information is only available for Broadcom NetXtreme II adapters Bridge Speed The clock speed on PCI E bus This information...

Страница 166: ...for the first port is 0 and the function number for the second port is 1 Interrupt Request The interrupt line number that is associated with the adapter Valid numbers range from 2 to 25 Memory Address...

Страница 167: ...select Navigate then Device Management and then Information If the Hardware section within the Information tab is not visible then from the Context View tab on the right side of the window select Info...

Страница 168: ...that system If a gateway is not configured or if the gateway is unreachable the test prompts for a destination IP address NOTE The network test option is not available on adapters that are grouped int...

Страница 169: ...User Guide NetXtreme II January 2010 Broadcom Corporation Document ENGSRVT52 CDUM100 R Using Broadcom Advanced Control Suite 3 Page 169...

Страница 170: ...the Select a test to run list select Diagnostic Tests If the Diagnostic Tests option is not available then from the Context View tab on the right side of the window select Diagnostics and then select...

Страница 171: ...tent of the electrically erasable programmable read only memory EEPROM by reading a portion of the EEPROM and computing the checksum The test fails if the computed checksum is different from the check...

Страница 172: ...ters This option is not available for all Broadcom network adapters This option is available for Broadcom NetXtreme II VBD drivers To run a cable analysis 1 Connect the cable to a port on a switch whe...

Страница 173: ...d pair Short Two pins from the same twisted pair are shorted together Noise Persistent noise present most likely caused by Forced 10 100 GB Link Gigabit link is up and running N A Algorithm failed to...

Страница 174: ...he adapter in the Device Management pane 2 Click the Configurations tab If the Configurations tab is not visible then from the View menu select Navigate then Device Management and then Configurations...

Страница 175: ...ferent types of network traffic diversely to ensure required levels or reliability and latency according to the type of traffic This property is disabled by default Unless the network infrastructure s...

Страница 176: ...x Enabled Enables transmit TCP IP UDP checksum offload Tx Rx Enabled default Enables transmit and receive TCP IP UDP checksum offload None Disables checksum offload IPv4 Large Send Offload Normally th...

Страница 177: ...d at 100 Mbit s and the mode to Half Duplex 1 Gb Full Auto Sets the speed to Auto Negotiate at 1000 Mb Full Duplex mode only 2 5 Gb Full Sets the speed at 2 5 10 GB Full Sets the speed to 10 Gbit s an...

Страница 178: ...receive packets to memory For 10 Gbps adapters the range of valid receive buffers is 0 to 3000 in increments of 1 with 0 receive buffers as the default value Transmit Buffers The number of transmit b...

Страница 179: ...the adapter listed in the Device Management pane then click the Statistics tab If the Statistics tab is not visible then from the View menu select Navigate then Device Management and then Statistics...

Страница 180: ...lost because of internal MAC sublayer errors This counter is incremented as indicated by the Receive OK status Broadcast Frames Rx A count of frames that were successfully received and are directed t...

Страница 181: ...target Session Id The identifier used for the session between the initiator and the target Bytes sent The number of bytes sent for the named session Bytes received The number of bytes received for the...

Страница 182: ...with the pause command and a length equal to 0 MAC Rx w Pause Command and Length greater than 0 MAC control frames with the pause command and a length greater than 0 MAC Rx w no Pause Command MAC con...

Страница 183: ...e then from the View menu select Navigate then Device Management and then Configurations 3 From the Resource Reservations section select the property you want to set If the Resource Reservations secti...

Страница 184: ...etXtreme II User Guide January 2010 Broadcom Corporation Page 184 Using Broadcom Advanced Control Suite 3 Document ENGSRVT52 CDUM100 R The Configurations tab for 10 Gbps network adapters is shown belo...

Страница 185: ...ssing from the CPU to the Broadcom network adapter 1 Click the name of the Broadcom NetXtreme II iSCSI device in the SCSI controller section of the Device Management pane 2 Depending on the protocol y...

Страница 186: ...apters and VBD drivers Not all offload technologies are available with all adapters The Licenses section shows the number of connections available for TOE and iSCSI offload technologies To view licens...

Страница 187: ...recommended This minimizes the downtime due to spanning tree loop determination when failing over LiveLink mitigates such issues BASP is available only if a system has one or more Broadcom network ad...

Страница 188: ...d by the BASP driver The link partner of the team determines the load balancing scheme for inbound packets In this mode at least one of the link partners must be in active mode NOTE TOE is not applica...

Страница 189: ...new standby member LIVELINK LiveLink is a feature of BASP that is available for the Smart Load Balancing SLB and SLB Auto Fallback Disable type of teaming The purpose of LiveLink is to detect link los...

Страница 190: ...ing Teaming Document ENGSRVT52 CDUM100 R NOTE If you prefer to work without the wizard for now click Expert Mode If you want to always use Expert Mode to create a team select Default to Expert Mode on...

Страница 191: ...of the following characters 6 Select the type of team you want to create If the team type is an SLB type team click Next If the team type is not an SLB type team then a dialog box appears Verify that...

Страница 192: ...NetXtreme II User Guide January 2010 Broadcom Corporation Page 192 Configuring Teaming Document ENGSRVT52 CDUM100 R...

Страница 193: ...rs support and are configured for the feature If this is the case then the team offload capabilities appear on the bottom of the screen NOTES Adding a network adapter to a team where its driver is dis...

Страница 194: ...robe retries the number of consecutively missed responses from a probe target before a failover is triggered 12 Set the Probe VLAN ID to allow for connectivity with probe targets residing on a tagged...

Страница 195: ...e or all probe targets and then click OK Click Next NOTE Only the first probe target is required You can specify up to three additional probe targets to serve as backups by assigning IP addresses to t...

Страница 196: ...VLAN then click Next If you do not want to create or edit a VLAN select Skip Manage VLAN then click Next and continue with the wizard from the Finish screen see Step 20 of this procedure VLANs enable...

Страница 197: ...eam name cannot exceed 39 characters cannot begin with spaces and cannot contain any of the following characters 17 To tag the VLAN select Tagged and then click Next Otherwise click Untagged click Nex...

Страница 198: ...Repeat until you do not want to add or manage any additional VLANs NOTE You can define up to 64 VLANs per team 63 VLANs that are tagged and 1 VLAN that is not tagged Adding several VLANS may slow down...

Страница 199: ...g the wizard select Save changes and continue to manage more teams Click Finish NOTE At any point in the Broadcom Teaming Wizard procedure click Preview to get a visual representation of what the team...

Страница 200: ...ard Mode the default is Wizard Mode Creating a Team NOTE Enabling Dynamic Host Configuration Protocol DHCP is not recommended for members of an SLB type of team 1 Click the Team Management button at t...

Страница 201: ...u want to always use Expert Mode to create a team click Default to Expert Mode on next start 5 Click the Create Team tab NOTE The Create Team tab appears only if there are teamable adapters available...

Страница 202: ...een NOTES Adding a network adapter to a team where its driver is disabled may negatively affect the offloading capabilities of the team This may have an impact on the team s performance Therefore it i...

Страница 203: ...ure the team IP address a From Control Panel double click Network Connections b Right click the name of the team to be configured and then click Properties c On the General tab click Internet Protocol...

Страница 204: ...this is that your system can have one network adapter that can belong to multiple subnets With a VLAN you can couple the functionality of load balancing for the load balance members and you can employ...

Страница 205: ...stics and Running VLAN Tests To view VLAN properties and statistics and to run VLAN tests 1 Click the Team Management button at the bottom of the BACS 3 window or from the View menu select Navigate th...

Страница 206: ...ast octet To configure LiveLink 1 Click the Team Management button at the bottom of the BACS 3 window or from the View menu select Navigate then Team Management 2 Select the Smart Load Balance and Fai...

Страница 207: ...ion 1 From the File menu select Team Save As 2 Type the path and file name of the new configuration file and then click Save a bcg extension is added The configuration file is a text file that can be...

Страница 208: ...nformation about the network adapters that are on a team To view BASP Statistics information for any team member adapter or the team as a whole click the name of the adapter or team listed in the Team...

Страница 209: ...formation about each network adapter and enables you to perform detailed tests run diagnostics view statistics and modify property values BACSCLI also allows you the ability to team network adapters t...

Страница 210: ...ecifications Port Type Connector Media Maximum Distance 10BASE T RJ 45 Category 3 4 or 5 unshielded twisted pairs UTP 100m 328 feet 100 1000BASE T1 RJ 45 Category 52 UTP 100m 328 feet 1 1000BASE T sig...

Страница 211: ...raded index fiber 550m 1804 feet 2500BASE X1 Small form factor pluggable SFP transceiver with LC connection system Finisar p n FTLF8542E2KNV Multimode fiber MMF System optimized for 62 5 50 m graded i...

Страница 212: ...57711 PCI Express x8 low profile 16 8 cm 6 6 inches 5 1 cm 2 0 inches Table 7 BCM5708C NIC Power Requirements Link NIC 3 3V Current Draw A NIC Power W Idle no link 1 44 4 75 1 Gbit 1 97 6 50 100 Mbit...

Страница 213: ...tion Storage Specification Temperature 0 C to 55 C 32 F to 131 F 40 C to 85 C 40 F to 185 F Relative humidity 5 to 85 noncondensing 40 C 16 hour dwells at extremes 5 to 95 noncondensing 10 C hour Alti...

Страница 214: ...These limits are designed to provide reasonable protection against harmful interference in a residential installation The equipment generates uses and can radiate radio frequency energy and if not in...

Страница 215: ...e with radio communications Operation of this product in a residential area is likely to cause harmful interference in which case you will be required to correct the interference at your own expense T...

Страница 216: ...gy Equipment VCCI If used near a radio or television receiver in a domestic environment it may cause radio interference Install and use the equipment according to the instruction manual CAUTION The po...

Страница 217: ...A This equipment is a Class A product based on the standard of the Voluntary Control Council for interference by Information Technology Equipment VCCI If used in a domestic environment radio disturban...

Страница 218: ...NetXtreme II User Guide January 2010 Broadcom Corporation Page 218 CE Notice Document ENGSRVT52 CDUM100 R...

Страница 219: ...User Guide NetXtreme II January 2010 Broadcom Corporation Document ENGSRVT52 CDUM100 R CE Notice Page 219...

Страница 220: ...NetXtreme II User Guide January 2010 Broadcom Corporation Page 220 CE Notice Document ENGSRVT52 CDUM100 R...

Страница 221: ...User Guide NetXtreme II January 2010 Broadcom Corporation Document ENGSRVT52 CDUM100 R CE Notice Page 221...

Страница 222: ...NetXtreme II User Guide January 2010 Broadcom Corporation Page 222 CE Notice Document ENGSRVT52 CDUM100 R...

Страница 223: ...3 Notice The Industry Canada regulations provide that changes or modifications not expressly approved by Broadcom could void your authority to operate this equipment INDUSTRY CANADA CLASS A Broadcom N...

Страница 224: ...s d utilisation de cet quipement peuvent tre annul s si des changements ou modifications non express ment approuv s par Broadcom y sont apport s INDUSTRY CANADA CLASSE A Broadcom NetXtreme II Gigabit...

Страница 225: ...a Only Page 225 KOREA COMMUNICATIONS COMMISSION KCC NOTICE REPUBLIC OF KOREA ONLY B CLASS DEVICE Broadcom NetXtreme II Gigabit Ethernet Controller Broadcom Corporation 190 Mathilda Place Sunnyvale Cal...

Страница 226: ...mmunications Commission KCC Notice Republic of Korea Only Document ENGSRVT52 CDUM100 R A CLASS DEVICE Broadcom NetXtreme II Gigabit Ethernet Controller Broadcom NetXtreme II 10 Gigabit Ethernet Contro...

Страница 227: ...om NetXtreme II User Diagnostics also allows you to update device firmware and to view and change settings for available adapter properties There are two versions of the Broadcom NetXtreme II User Dia...

Страница 228: ...t checking the version uxdiag fbc bc_image Specifies the bin file to update the bootcode uxdiag fib ib_image Specifies the bin file for iSCSI boot uxdiag fibc Programs the iSCSI configuration block Us...

Страница 229: ...ble uxdiag mbavval n Sets MBA VLAN 65536 uxdiag mfw 1 0 Enables disables management firmware 1 Enable 0 Disable uxdiag t groups tests Disables certain groups tests uxdiag T groups tests Enables certai...

Страница 230: ...p Programs iSCSI configuration software Used only with fib ib_image uediag fipmi ipmi_image Specifies the bin file to update IPMI firmware uediag fmba mba_image Specifies the bin file to update the MB...

Страница 231: ...anuary 2010 Broadcom Corporation Document ENGSRVT52 CDUM100 R Performing Diagnostics Page 231 uediag wol 1 0 Enables disable Magic Packet WOL 1 Enable 0 Disable Table 2 uediag Command Options Cont Com...

Страница 232: ...es the correct ISR A negative test is also performed to verify that a masked interrupt does not invoke the ISR A5 MSI Verifies that a Message Signaled Interrupt MSI causes an MSI message to be DMA d t...

Страница 233: ...resetting the chip multiple times checking that the bootcode and the internal uxdiag driver loads unloads correctly C8 DMA Engine Verifies the functionality of the DMA engine block by performing numer...

Страница 234: ...that the basic statistics information maintained by the chip is correct by enabling MAC loopback mode and sending Layer 2 packets of various sizes The adapter should not be connected to a network D6...

Страница 235: ...2000 Server to Windows Server 2003 Upgrading Windows Operating Systems Preparing an Answer File for Unattended Installation Broadcom Boot Agent Broadcom Advanced Server Program BASP Linux Miscellaneou...

Страница 236: ...er is using the latest BIOS Try inserting the adapter in another slot If the new position works the original slot in your system may be defective Replace the failed adapter with one that is known to w...

Страница 237: ...le analysis is not available for NetXtreme II 10 GbE network adapters TESTING NETWORK CONNECTIVITY NOTE When using forced link speeds verify that both the adapter and the switch are forced to the same...

Страница 238: ...e network connection is working LINUX To verify that the Ethernet interface is up and running run ifconfig to check the status of the Ethernet interface It is possible to use netstat i to check the st...

Страница 239: ...all inclusive list of Hyper V features NOTE Ensure that Integrated Services which is a component of Hyper V is installed in the guest operating system child partition for full functionality Table 1 C...

Страница 240: ...In an IPv6 network a team that supports CO and or LSO and is bound to a Hyper V virtual network will report CO and LSO as an offload capability in BACS however CO and LSO will not work This is a limi...

Страница 241: ...ith latest BASP6 version Link Aggregation IEEE 802 3ad LACP team type Yes Yes Generic Trunking FEC GEC 802 3ad Draft Static team type Yes Yes Failover Yes Yes LiveLink Yes Yes Large Send Offload LSO L...

Страница 242: ...aming software you must configure the team for tagging Unlike VLAN tagging with a single adapter tagging cannot be managed by Hyper V when using BASP software When making changes to a team or removing...

Страница 243: ...drivers for Windows 2000 and Windows Server 2003 are not the same and require the installation of the Windows 2003 drivers to be performed after the OS upgrade is completed UPGRADING WINDOWS OPERATING...

Страница 244: ...InfId b06bdrv l2nd pci_16ac14e4 For a BCM5706C Netxtreme II GigE InfId b06bdrv l2nd pci_164a14e4 For a BCM5706S Netxtreme II GigE InfId b06bdrv l2nd pci_16aa14e4 Currently the PCI location PCI bus dev...

Страница 245: ...s a third party issue It is seen only when configuring an 802 3ad team with greater than two members on the server and connecting an HP2524 switch with LACP enabled as passive or active The HP switch...

Страница 246: ...cking causes the on chip firmware to discard packets inside the on chip receive buffers in the case a particular host queue is depleted the on chip receive buffers will rarely be depleted therefore it...

Страница 247: ...installing SNP from the system ensure that NDIS is enabled via the checkbox on the Resource Configuration screen available from the Resource Reservations section of the Configurations tab see Viewing...

Отзывы:

Нет отзывов

Похожие инструкции для BCM5706C

Бренд: D-Link Страницы: 28

Бренд: D-Link Страницы: 2

DIR-655 - Xtreme N Gigabit Router Wireless

Бренд: D-Link Страницы: 3

Бренд: D-Link Страницы: 11

Бренд: D-Link Страницы: 12

Бренд: D-Link Страницы: 12

Бренд: D-Link Страницы: 24

Бренд: D-Link Страницы: 20

Бренд: D-Link Страницы: 44

Бренд: D-Link Страницы: 52

Бренд: D-Link Страницы: 103

LightSpeed Appliance QLS250

Бренд: Quantum Страницы: 12

Бренд: APM Страницы: 3

Бренд: Patton electronics Страницы: 23

Бренд: UHP Страницы: 15

Бренд: Ubiquiti Страницы: 21

Бренд: Airlink101 Страницы: 46

Бренд: Compro Technology Страницы: 2

Бренды по названию

0 1 2 3 4 5 6 7 8 9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Популярные бренды

Загрузить еще бренды