manualshive.com logo in svg

H3C s5820x series, Руководство по настройке высокой доступности, Страница: 196 / 233

Поделиться
Скачать
H3C s5820x series Скачать руководство пользователя страница 196

 

184 

Operation of BFD 

Figure 51

 

BFD session establishment (on OSPF routers) 

 

 

BFD session establishment: 

1.

 

A protocol sends Hello messages to discover neighbors and establish neighborships. 

2.

 

After establishing neighborships, the protocol notifies BFD of the neighbor information, including 
destination and source addresses. 

3.

 

BFD uses the information to establish BFD sessions. 

Figure 52

 

BFD fault detection (on OSPF routers) 

 

 

BFD fault detection: 

1.

 

BFD detects a link failure. 

2.

 

BFD clears the neighbor session. 

3.

 

BFD notifies the protocol of the failure. 

4.

 

The protocol terminates the neighborship on the link. 

5.

 

If a backup link is available, the protocol will use it to forward packets. 

 

 

NOTE: 

No detection time resolution is defined in the BFD draft. Most devices supporting BFD provide detection 
measured in milliseconds.  

 

Содержание s5820x series

Страница 1: ...H3C S5820X S5800 Switch Series High Availability Configuration Guide Hangzhou H3C Technologies Co Ltd http www h3c com Software version Release 1211 Document version 6W100 20110415...

Страница 2: ...re Secware Storware NQA VVG V2 G Vn G PSPT XGbus N Bus TiGem InnoVision and HUASAN are trademarks of Hangzhou H3C Technologies Co Ltd All other trademarks that may be mentioned in this manual are the...

Страница 3: ...network recovery This preface includes Audience Added and modified features Conventions About the H3C S5800 S5820X documentation set Obtaining documentation Technical support Documentation feedback Au...

Страница 4: ...cription Boldface Bold text represents commands and keywords that you enter literally as shown Italic Italic text represents arguments that you replace with actual values Square brackets enclose synta...

Страница 5: ...le device such as a router or Layer 3 switch Represents a generic switch such as a Layer 2 or Layer 3 switch or a router that supports Layer 2 forwarding and other Layer 2 features About the H3C S5800...

Страница 6: ...s of the pluggable modules available for the products S5800 60C PWR Ethernet Switch Hot Swappable Power Module Ordering Guide Guides you through ordering the hot swappable power modules available for...

Страница 7: ...including the version history hardware and software compatibility matrix version upgrade information technical support information and software upgrading Obtaining documentation You can access the mo...

Страница 8: ...detection 11 Configuring errored frame seconds event detection 12 Configuring OAM remote loopback 12 Enabling Ethernet OAM remote loopback 12 Rejecting the Ethernet OAM remote loopback request from a...

Страница 9: ...RRPPDUs 58 RRPP timers 59 How RRPP works 59 Typical RRPP networking 61 Protocols and standards 64 RRPP configuration task list 64 Creating an RRPP domain 65 Configuring control VLANs 65 Configuring p...

Страница 10: ...group member ports 110 Displaying and maintaining Monitor Link 111 Monitor Link configuration example 111 VRRP configuration 115 VRRP overview 115 VRRP standard protocol mode 116 Introduction to VRRP...

Страница 11: ...uring the backup VLAN 180 Displaying and maintaining stateful failover 180 Stateful failover configuration example 180 Configuration guidelines 182 BFD configuration 183 Introduction to BFD 183 How BF...

Страница 12: ...P master to monitor the uplinks 205 Static routing track NQA collaboration configuration example 209 Static routing track BFD collaboration configuration example 213 VRRP track interface management co...

Страница 13: ...from being affected by failures Device and link redundancy and switchover 3 Enable the system to recover as fast as possible Fault detection diagnosis isolation and recovery technologies The level 1 a...

Страница 14: ...nistration and Maintenance OAM mechanism used for link connectivity detection fault verification and fault location CFD configuration in the High Availability Configuration Guide DLDP The Device link...

Страница 15: ...to ensure continuity of network services For more information about protection switchover technologies see Table 3 Table 3 Protection switchover technologies Technology Introduction Reference Ethernet...

Страница 16: ...thout requiring the cooperation of other devices IS IS supports this feature IS IS configuration in the Layer 3 IP Routing Configuration Guide Stateful Failover Two devices back up the services of eac...

Страница 17: ...As a tool monitoring Layer 2 link status Ethernet OAM mainly addresses common link related issues on the last mile When you enable Ethernet OAM on two devices connected by a point to point link you c...

Страница 18: ...s Status information of an Ethernet OAM entity Code Type of the Ethernet OAMPDU NOTE Throughout this document a port with Ethernet OAM enabled is an Ethernet OAM entity or an OAM entity Table 5 Functi...

Страница 19: ...e Available Transmitting Information OAMPDUs Available Available Transmitting Event Notification OAMPDUs Available Available Transmitting Information OAMPDUs without any TLV Available Available Transm...

Страница 20: ...stem transforms the period of detecting errored frame period events into the maximum number of 64 byte frames that a port can send in the specific period The system takes the maximum number of frames...

Страница 21: ...check the link status and locate link failures Performing remote loopback periodically helps to detect network faults in time Furthermore performing remote loopback by network segments helps to locate...

Страница 22: ...whether the Ethernet OAM connection is normal If an Ethernet OAM entity receives no Information OAMPDU within the Ethernet OAM connection timeout time the Ethernet OAM connection is considered discon...

Страница 23: ...frame event detection An errored frame event occurs when the number of detected error frames over a specific interval exceeds the configured threshold Follow these steps to configure errored frame ev...

Страница 24: ...seconds detection interval Otherwise no errored frame seconds event can be generated Configuring OAM remote loopback Enabling Ethernet OAM remote loopback When you enable Ethernet OAM remote loopback...

Страница 25: ...l shut down and then come up Ethernet OAM remote loopback is disabled when you execute the undo oam enable command to disable Ethernet OAM when you execute the undo oam loopback interface or undo oam...

Страница 26: ...ink error events after an Ethernet OAM connection is established display oam link event local remote interface interface type interface number begin exclude include regular expression Display the info...

Страница 27: ...erify the configuration Use the display oam configuration command to display the Ethernet OAM configuration For example Display the Ethernet OAM configuration on Switch A SwitchA display oam configura...

Страница 28: ...net OAM link error events For example Display Ethernet OAM link event statistics of the remote end of Switch B SwitchB display oam link event remote Port GigabitEthernet1 0 1 Link Status Up OAMRemoteE...

Страница 29: ...MEPs configured on the ports An MD is identified by an MD name To accurately locate faults CFD assigns eight levels from 0 to 7 to MDs The bigger the number the higher the level and the larger the ar...

Страница 30: ...ts MIPs MEP Each MEP is identified by an integer called a MEP ID The MEPs of an MD define the range and boundary of the MD The MA and MD that a MEP belongs to define the VLAN attribute and level of th...

Страница 31: ...covered In this example Port 1 of device B is configured with the following MPs a level 5 MIP a level 3 inward facing MEP a level 2 inward facing MEP and a level 0 outward facing MEP Figure 6 Levels...

Страница 32: ...ames pass send back linktrace reply messages LTRs to the source MEP Based on the reply messages the source MEP can identify the path to the target MEP LTM frames are multicast frames and LTRs are unic...

Страница 33: ...cast frames Protocols and standards IEEE 802 1ag Virtual Bridged Local Area Networks Amendment 5 Connectivity Fault Management ITU T Y 1731 OAM functions and mechanisms for Ethernet based networks CFD...

Страница 34: ...tings Enabling CFD Enable CFD on all concerned devices Follow these steps to enable CFD on a device To do Use the command Remarks Enter system view system view Enable CFD cfd enable Required Disabled...

Страница 35: ...w system view Create an MD cfd md md name level level value Required Not created by default Create an MA cfd ma ma name md md name vlan vlan id Required Not created by default Create a service instanc...

Страница 36: ...abled by default NOTE You cannot create a MEP if the MEP ID is not included in the MEP list of the service instance Configuring MIP generation rules As functional entities in a service instance MIPs r...

Страница 37: ...efore configuring CFD functions you need to complete basic CFD configurations first Configuring CC on MEPs After the CC function is configured MEPs can send CCM frames to one another to check the conn...

Страница 38: ...er number Required Disabled by default Available in any view Configuring LT on MEPs LT can trace the path between the source and target MEPs and can also locate link faults by sending LT messages auto...

Страница 39: ...eriod value service instance instance id Optional 1 second by default NOTE To make an MEP in the service instance send AIS frames you must configure the AIS frame transmission level to be higher than...

Страница 40: ...r clock synchronization can be relaxed To view the test result use the display cfd dm one way history command on the target MEP Configuring two way DM The two way DM function measures the two way fram...

Страница 41: ...e regular expression Available in any view Display MEP list in a service instance display cfd meplist service instance instance id begin exclude include regular expression Available in any view Displa...

Страница 42: ...s in the two MDs all serve VLAN 100 MD_A has three edge ports GigabitEthernet 1 0 1 on Device A GigabitEthernet 1 0 3 on Device D and GigabitEthernet 1 0 4 on Device E and they are all inward facing M...

Страница 43: ...cfd ma MA_A md MD_A vlan 100 DeviceA cfd service instance 1 md MD_A ma MA_A Configure Device E as you configure Device A Create MD_A level 5 on Device B create MA_A which serves VLAN 100 in MD_A and t...

Страница 44: ...ce 2 on GigabitEthernet 1 0 1 and then create and enable inward facing MEP 4002 in service instance 1 on GigabitEthernet 1 0 3 DeviceD cfd meplist 1001 4002 5001 service instance 1 DeviceD cfd meplist...

Страница 45: ...et 1 0 4 DeviceE interface gigabitethernet 1 0 4 DeviceE GigabitEthernet1 0 4 cfd cc service instance 1 mep 5001 enable DeviceE GigabitEthernet1 0 4 quit 7 Configure AIS Enable AIS on Device B and con...

Страница 46: ...20 Near end frame loss 25 Far end frame loss rate 25 Near end frame loss rate 32 Send LMMs 5 Received 5 Lost 0 4 Verify the one way DM function After the CC function obtains the status information of...

Страница 47: ...est the bit errors on the link from MEP 1001 to MEP 4002 in service instance 1 on Device A DeviceA cfd tst service instance 1 mep 1001 target mep 4002 Info TST process is done Please check the result...

Страница 48: ...e the link between two switches Switch A and Switch B is a bidirectional link when they are connected via a fiber pair with one fiber used for sending packets from A to B and the other for sending pac...

Страница 49: ...ation mechanism and DLDP work together to ensure that physical logical unidirectional links can be detected and shut down and to prevent failure of other protocols such as STP If both ends of a link a...

Страница 50: ...can be sent successively is 5 Advertisement timer Determines the interval for sending common advertisement packets which defaults to 5 seconds Probe timer Determines the interval for sending Probe pac...

Страница 51: ...ng neighbor entry and sends an Advertisement packet with the RSY tag In enhanced DLDP mode when an entry timer expires the Enhanced timer is triggered and the switch tests the neighbor by sending up t...

Страница 52: ...lain text authentication Before sending a DLDP packet the sending side sets the Authentication field to the password configured in plain text and sets the Authentication type field to 1 The receiving...

Страница 53: ...and transits to Probe state If the corresponding neighbor entry does not exist creates the neighbor entry triggers the Entry timer and transits to Probe state Normal Advertisement packet Retrieves the...

Страница 54: ...down the local port or prompts users to shut down the port and removes the corresponding neighbor entry Link auto recovery mechanism If the port shutdown mode upon detection of a unidirectional link i...

Страница 55: ...yDown timer Optional Setting the port shutdown mode Optional Configuring DLDP authentication Optional Resetting DLDP state Optional CAUTION To ensure that DLDP works properly on a link you must config...

Страница 56: ...n identify only one type of unidirectional links cross connected fibers In enhanced mode DLDP actively detects neighbors when the corresponding neighbor entries age out so the system can identify two...

Страница 57: ...o set the DelayDown timer To do Use the command Remarks Enter system view system view Set the DelayDown timer dldp delaydown timer time Optional 1 second by default NOTE DelayDown timer setting applie...

Страница 58: ...ssword Required none by default NOTE To enable DLDP to operate properly make sure that DLDP authentication modes and passwords on both sides of a link are the same Resetting DLDP state After DLDP dete...

Страница 59: ...y to the current port only configurations performed in port group view apply to all the ports in the port group Reset DLDP state dldp reset Required Displaying and maintaining DLDP To do Use the comma...

Страница 60: ...bps and enable DLDP on the port DeviceA interface gigabitethernet 1 0 49 DeviceA GigabitEthernet1 0 49 duplex full DeviceA GigabitEthernet1 0 49 speed 1000 DeviceA GigabitEthernet1 0 49 dldp enable De...

Страница 61: ...the port shutdown mode to auto DeviceB dldp unidirectional shutdown auto 3 Verifying the configurations After the configurations are complete you can use the display dldp command to display the DLDP...

Страница 62: ...ink status is DOWN Jan 18 17 36 20 190 2010 DeviceA DLDP 3 DLDP_UNIDIRECTION_AUTO Slot 1 DLDP detects a unidirectional link on port GigabitEthernet1 0 50 The transceiver has malfunction in the Tx dire...

Страница 63: ...E1 0 50 GE1 0 49 GE1 0 50 GE1 0 49 GE1 0 50 GE1 0 49 GE1 0 50 Configuration procedure 1 Configuration on Device A Enable DLDP globally DeviceA system view DeviceA dldp enable Configure GigabitEthernet...

Страница 64: ...l DeviceB GigabitEthernet1 0 50 speed 1000 DeviceB GigabitEthernet1 0 50 dldp enable DeviceB GigabitEthernet1 0 50 quit Set the DLDP mode to enhanced DeviceB dldp work mode enhance Set the port shutdo...

Страница 65: ...shutdown mode is MANUAL The port needs to be shut down by the user Jan 18 18 10 38 618 2010 DeviceA DLDP 1 TrapOfUnidirectional Slot 1 h3cDLDPUnidirectionalPort DLDP detects a unidirectional link in...

Страница 66: ...igabitEthernet1 0 49 Jan 18 18 22 46 065 2010 DeviceA IFNET 3 LINK_UPDOWN GigabitEthernet1 0 49 link status is UP The output indicates that the link status of both GigabitEthernet 1 0 49 and GigabitEt...

Страница 67: ...enterprise networks usually use the ring structure to improve reliability However services will be interrupted if any node in the ring network fails A ring network usually uses Resilient Packet Ring...

Страница 68: ...llowing states Health state All the physical links on the Ethernet ring are connected Disconnect state Some physical links on the Ethernet ring are broken As shown in Figure 13 Domain 1 contains two R...

Страница 69: ...imary ring and Ring 2 is a subring Device A is the master node of Ring 1 Device B Device C and Device D are the transit nodes of Ring 1 Device E is the master node of Ring 2 Device B is the edge node...

Страница 70: ...master node initiates Hello packets to detect the integrity of a ring in a network Fast Hello The master node initiates Fast Hello packets to fast detect the integrity of a ring in a network Link Dow...

Страница 71: ...ello packets ensuring that all nodes in the ring network are consistent in the two timer settings A transit node however cannot learn the Fast Hello timer value and the Fast Fail timer value set on th...

Страница 72: ...figured with different protected VLANs Device A is the master node of Ring 1 in Domain 1 Device B is the master node of Ring 1 in Domain 2 With such configurations traffic of different VLANs can be tr...

Страница 73: ...master node of Ring 1 sends out Fast Hello packets periodically and determines the ring status according to whether Fast Hello packets are received before the Fast Fail timer expires implementing link...

Страница 74: ...efine an RRPP domain and configure one ring as the primary ring and the other rings as subrings Figure 16 Schematic diagram for an intersecting ring network Dual homed rings As shown in Figure 17 two...

Страница 75: ...balancing network Domain 1 Ring 1 Device A Device B Device D Device C Domain 2 Intersecting ring load balancing In an intersecting ring network you can also achieve load balancing by configuring mult...

Страница 76: ...ntrol VLANs and data VLANs for each RRPP domain and then determine the ring roles and node roles based on the traffic paths in each RRPP domain Complete the following tasks to configure RRPP Task Rema...

Страница 77: ...vices in the same RRPP domain must be configured with the same domain ID Make this configuration on devices you want to configure as nodes in the RRPP domain Follow these steps to create an RRPP domai...

Страница 78: ...ould be protected by the RRPP domains Perform this configuration on all nodes in the RRPP domain to be configured Follow these steps to configure protected VLANs To do Use the command Remarks Enter sy...

Страница 79: ...unk permit vlan commands see the Layer 2 LAN Switching Command Reference For more information about the undo stp enable command see the Layer 2 LAN Switching Command Reference The 802 1p priority of t...

Страница 80: ...t node of the ring and specify the primary port and the secondary port ring ring id node mode transit primary port interface type interface number secondary port interface type interface number level...

Страница 81: ...ype interface number Required Activating an RRPP domain To activate an RRPP domain on the current device enable the RRPP protocol and RRPP rings for the RRPP domain on the current device Perform this...

Страница 82: ...ling fast detection Perform this configuration on the master node edge node and assistant edge node in the RRPP domain to be configured Follow these steps to enable fast detection To do Use the comman...

Страница 83: ...ge node and can only be configured on these two types of nodes Perform this configuration on both the edge node and the assistant edge node in an RRPP domain Follow these steps to configure an RRPP ri...

Страница 84: ...et rrpp statistics domain domain id ring ring id Available in user view RRPP configuration examples Single ring configuration example Networking requirements Device A Device B Device C and Device D fo...

Страница 85: ...DeviceA GigabitEthernet1 0 2 undo stp enable DeviceA GigabitEthernet1 0 2 port link type trunk DeviceA GigabitEthernet1 0 2 port trunk permit vlan 1 to 30 DeviceA GigabitEthernet1 0 2 qos trust dot1p...

Страница 86: ...nd configure the VLANs mapped to MSTI 1 through 32 as the protected VLANs of RRPP domain 1 DeviceB rrpp domain 1 DeviceB rrpp domain1 control vlan 4092 DeviceB rrpp domain1 protected vlan reference in...

Страница 87: ...ort Figure 21 Network diagram for intersecting rings configuration Configuration procedure 1 Configuration on Device A Create VLANs 1 through 30 map these VLANs to MSTI 1 and activate the MST region c...

Страница 88: ...map these VLANs to MSTI 1 and activate the MST region configuration DeviceB system view DeviceB vlan 1 to 30 DeviceB stp region configuration DeviceB mst region instance 1 vlan 1 to 30 DeviceB mst reg...

Страница 89: ...ring 2 enable DeviceB rrpp domain1 quit Enable RRPP DeviceB rrpp enable 3 Configuration on Device C Create VLANs 1 through 30 map these VLANs to MSTI 1 and activate the MST region configuration Devic...

Страница 90: ...tEthernet 1 0 3 as the edge port and enable ring 2 DeviceC rrpp domain1 ring 2 node mode assistant edge edge port gigabitethernet 1 0 3 DeviceC rrpp domain1 ring 2 enable DeviceC rrpp domain1 quit Ena...

Страница 91: ...configuration DeviceE mst region instance 1 vlan 1 to 30 DeviceE mst region active region configuration DeviceE mst region quit Disable physical state change suppression and STP on GigabitEthernet 1...

Страница 92: ...the primary ring Ring 1 Device D is the transit node of the primary ring Ring 1 Device F is the master node of the subring Ring 3 Device C is the edge node of the subring Ring 3 Device B is the assist...

Страница 93: ...sion and STP on GigabitEthernet 1 0 1 and GigabitEthernet 1 0 2 configure the two ports as trunk ports remove them from VLAN 1 and assign them to VLAN 10 and VLAN 20 and configure them to trust the 80...

Страница 94: ...PP domain 2 DeviceA rrpp domain 2 DeviceA rrpp domain2 control vlan 105 DeviceA rrpp domain2 protected vlan reference instance 2 Configure Device A as the master node of primary ring 1 with GigabitEth...

Страница 95: ...it vlan 1 DeviceB GigabitEthernet1 0 3 port trunk permit vlan 20 DeviceB GigabitEthernet1 0 3 qos trust dot1p DeviceB GigabitEthernet1 0 3 quit Disable physical state change suppression and STP on Gig...

Страница 96: ...ain2 ring 2 node mode assistant edge edge port gigabitethernet 1 0 3 DeviceB rrpp domain2 ring 2 enable DeviceB rrpp domain2 quit Enable RRPP DeviceB rrpp enable 3 Configuration on Device C Create VLA...

Страница 97: ...viceC GigabitEthernet1 0 4 undo link delay DeviceC GigabitEthernet1 0 4 undo stp enable DeviceC GigabitEthernet1 0 4 port link type trunk DeviceC GigabitEthernet1 0 4 undo port trunk permit vlan 1 Dev...

Страница 98: ...configuration DeviceD mst region instance 1 vlan 10 DeviceD mst region instance 2 vlan 20 DeviceD mst region active region configuration DeviceD mst region quit Disable physical state change suppressi...

Страница 99: ...and enable ring 1 DeviceD rrpp domain2 ring 1 node mode transit primary port gigabitethernet 1 0 1 secondary port gigabitethernet 1 0 2 level 0 DeviceD rrpp domain2 ring 1 enable DeviceD rrpp domain2...

Страница 100: ...system view DeviceF vlan 10 DeviceF vlan10 quit DeviceF stp region configuration DeviceF mst region instance 1 vlan 10 DeviceF mst region active region configuration DeviceF mst region quit Disable p...

Страница 101: ...1 DeviceB rrpp ring group1 domain 2 ring 2 DeviceB rrpp ring group1 domain 1 ring 3 Create RRPP ring group 1 on Device C and add subrings 2 and 3 to the RRPP ring group DeviceC rrpp ring group 1 Devi...

Страница 102: ...ernet 1 0 2 DeviceA GigabitEthernet1 0 2 undo link delay DeviceA GigabitEthernet1 0 2 undo stp enable DeviceA GigabitEthernet1 0 2 port link type trunk DeviceA GigabitEthernet1 0 2 port trunk permit v...

Страница 103: ...DeviceB GigabitEthernet1 0 1 qos trust dot1p DeviceB GigabitEthernet1 0 1 quit DeviceB interface gigabitethernet 1 0 2 DeviceB GigabitEthernet1 0 2 undo link delay DeviceB GigabitEthernet1 0 2 undo st...

Страница 104: ...le 5 Verification Use the display command to view RRPP configuration and operational information on Device A and Device D Troubleshooting Symptom When the link state is normal the master node cannot r...

Страница 105: ...d to upstream devices As shown in Figure 24 a downstream device connects to two different upstream devices Figure 24 Diagram for a dual uplink network Device A Device E Device D Device C Device B Core...

Страница 106: ...vice C and Port1 and Port2 of Device D each form a smart link group with Port1 being active and Port2 being standby Master port slave port Master port and slave port are two port roles in a smart link...

Страница 107: ...including devices of other vendors Flush update where a Smart Link enabled device updates its information by transmitting flush messages over the backup link to its upstream devices This mechanism re...

Страница 108: ...nk status Smart Link ports need to use link detection protocols When a fault is detected or cleared the link detection protocols inform Smart Link to switch over the links With the collaboration betwe...

Страница 109: ...when STP is disabled but Smart Link has not yet taken effect on a port Configuring protected VLANs for a smart link group Follow these steps to configure the protected VLANs for a smart link group To...

Страница 110: ...r slave Required Configuring role preemption for a smart link group Follow these steps to configure role preemption for a smart link group To do Use the command Remarks Enter system view system view C...

Страница 111: ...member port To do Use the command Remarks Enter system view system view Enter Ethernet port view interface interface type interface number Configure the collaboration between Smart Link and the CC fun...

Страница 112: ...ANs Otherwise flush messages cannot be sent properly Make sure that the control VLANs are existing VLANs and assign the ports capable of receiving flush messages to the control VLANs Displaying and ma...

Страница 113: ...0 DeviceC interface gigabitethernet 1 0 1 DeviceC GigabitEthernet1 0 1 shutdown DeviceC GigabitEthernet1 0 1 undo stp enable DeviceC GigabitEthernet1 0 1 port link type trunk DeviceC GigabitEthernet1...

Страница 114: ...n DeviceD GigabitEthernet1 0 1 undo stp enable DeviceD GigabitEthernet1 0 1 port link type trunk DeviceD GigabitEthernet1 0 1 port trunk permit vlan 1 to 30 DeviceD GigabitEthernet1 0 1 quit DeviceD i...

Страница 115: ...eB GigabitEthernet1 0 3 port trunk permit vlan 1 to 30 DeviceB GigabitEthernet1 0 3 smart link flush enable control vlan 10 20 DeviceB GigabitEthernet1 0 3 quit 4 Configuration on Device E Create VLAN...

Страница 116: ...tions Use the display smart link group command to display the smart link group configuration on each device For example Display the smart link group configuration on Device C DeviceC display smart lin...

Страница 117: ...region configuration DeviceC system view DeviceC vlan 1 to 200 DeviceC stp region configuration DeviceC mst region instance 1 vlan 1 to 100 DeviceC mst region instance 2 vlan 101 to 200 DeviceC mst re...

Страница 118: ...C smlk group2 port gigabitethernet 1 0 1 slave Enable role preemption in smart link group 2 enable flush message sending and configure VLAN 101 as the transmit control VLAN DeviceC smlk group2 preempt...

Страница 119: ...control vlan 10 101 DeviceD GigabitEthernet1 0 2 quit 4 Configuration on Device A Create VLAN 1 through VLAN 200 DeviceA system view DeviceA vlan 1 to 200 Configure GigabitEthernet 1 0 1 and GigabitEt...

Страница 120: ...ember Role State Flush count Last flush time GigabitEthernet1 0 2 MASTER ACTVIE 5 16 37 20 2010 02 21 GigabitEthernet1 0 1 SLAVE STANDBY 1 17 45 20 2010 02 21 Use the display smart link flush command...

Страница 121: ...orts and adapt the up down state of downlink ports to the up down state of uplink ports triggering link switchover on the downstream device in time as shown in Figure 27 Figure 27 Monitor Link applica...

Страница 122: ...roup contains no uplink port or all its uplink ports are down the monitor link group goes down which forces all its downlink ports down at the same time When any uplink port goes up the monitor link g...

Страница 123: ...can be assigned to only one monitor link group Configure uplink ports prior to downlink ports to avoid undesired down up state changes on the downlink ports Displaying and maintaining Monitor Link To...

Страница 124: ...gabitEthernet1 0 1 port link type trunk DeviceC GigabitEthernet1 0 1 port trunk permit vlan 1 to 30 DeviceC GigabitEthernet1 0 1 quit DeviceC interface gigabitethernet 1 0 2 DeviceC GigabitEthernet1 0...

Страница 125: ...ethernet 1 0 1 DeviceB GigabitEthernet1 0 1 port link type trunk DeviceB GigabitEthernet1 0 1 port trunk permit vlan 1 to 30 DeviceB GigabitEthernet1 0 1 smart link flush enable DeviceB GigabitEtherne...

Страница 126: ...k DeviceD mtlk group1 quit 5 Verify the configurations Use the display monitor link group command display the monitor link group information on devices For example when GigabitEthernet 1 0 2 on Device...

Страница 127: ...3 Ethernet interface is an Ethernet interface operating in route mode For more information about the operating mode of the Ethernet interface see the Layer 2 LAN Switching Configuration Guide VRRP ov...

Страница 128: ...ther of the following modes Standard protocol mode Includes two versions VRRPv2 and VRRPv3 based on RFCs VRRPv2 is based on IPv4 and VRRPv3 is based on IPv6 The two versions implement the same functio...

Страница 129: ...P determines the role master or backup of each router in a VRRP group by priority A router with a higher priority is more likely to become the master VRRP priority is in the range of 0 to 255 The grea...

Страница 130: ...de VRRP timers VRRP timers include VRRP advertisement interval timer and VRRP preemption delay timer VRRP advertisement interval timer The master in a VRRP group periodically sends VRRP advertisements...

Страница 131: ...outer that is ID of the VRRP group It ranges from 1 to 255 Priority Priority of the router in the VRRP group in the range 0 to 255 A greater value represents a higher priority Count IP Addrs Count IPv...

Страница 132: ...master and sends VRRP advertisements to start a new master election NOTE The VRRP group configuration might be different on routers and network problems might exist so multiple master routers might ex...

Страница 133: ...n master backup mode only the master forwards packets When the master fails a new master is elected from the backups This mode requires only one VRRP group in which each router holds a different prior...

Страница 134: ...the expected role in the group VRRP load balancing mode Overview When VRRP works in standard protocol mode only the master can forward packets and the backups are in the state of listening You can cr...

Страница 135: ...MAC addresses to the routers including the master itself and the backups in the VRRP group For example as shown in Figure 35 the virtual IP address of the VRRP group is 10 1 1 1 24 Router A is the mas...

Страница 136: ...example as shown in Figure 37 Host A regards the virtual MAC address of Router A as the gateway MAC address so it sends packets to Router A for forwarding Host B regards the virtual MAC address of Rou...

Страница 137: ...n the weight is lower than the lower limit of failure the router cannot be capable of forwarding packets for the hosts The priority of a VF determines the VF state Among the VFs that correspond to the...

Страница 138: ...which is lower than that of VF 1 on Router A In this case VF 1 on both Router B and Router C acts as the LVF to listen to the status of VF 1 on Router A When VF 1 on Router A fails VF 1 on Router B an...

Страница 139: ...ol mode defines only VRRP advertisement Only the master in a VRRP group periodically sends VRRP advertisements and the backups do not send VRRP advertisements VRRP load balancing mode defines the foll...

Страница 140: ...RRP groups on the router work in the specified working mode Follow these steps to configure a VRRP working mode To do Use the command Remarks Enter system view system view Configure VRRP to work in st...

Страница 141: ...ltiple interfaces of a device and the VRRP advertisements of these VRRP groups are to be sent through QinQ networks H3C recommends you to map the real MAC addresses of the interfaces to the virtual IP...

Страница 142: ...d for it In addition configurations on that VRRP group do not take effect any longer Removal of the VRRP group on the IP address owner causes IP address collision To solve the collision modify the IP...

Страница 143: ...need to configure it An IP address owner always works in preemptive mode Do not configure VRRP tracking for an interface or a track entry on an IP address owner The tracked interface can be a Layer 3...

Страница 144: ...the VF owner decreases by a properly specified value and becomes lower than the lower limit of failure In other words the weight of the VF owner decreases by more than 245 Configuring VRRP packet attr...

Страница 145: ...o the information center of the device where you can configure whether to output the trap information and the output destination For how to configure the information center see the Network Management...

Страница 146: ...nswering ND requests from hosts so that the hosts in the internal network can learn the mapping between the IPv6 address and the MAC address The following types of MAC addresses are available to be ma...

Страница 147: ...group later the virtual IPv6 address is added to the virtual IPv6 address list of the VRRP group NOTE H3C does not recommend you to create VRRP groups on the VLAN interface of a super VLAN because ne...

Страница 148: ...roup on the IP address owner causes IP address collision To resolve the collision change the IPv6 address of the interface on the IP address owner first and then remove the VRRP group from the interfa...

Страница 149: ...id to positive the priority of the router where the track entry is configured is automatically restored Configuring VF tracking Configuration prerequisites Before you configure the VF tracking functio...

Страница 150: ...Layer 3 Ethernet interface view interface interface type interface number Configure the authentication mode and authentication key when the VRRP groups send or receive VRRP packets vrrp ipv6 vrid vir...

Страница 151: ...in user view IPv4 based VRRP configuration examples This section provides these configuration examples Single VRRP group configuration example VRRP interface tracking configuration example VRRP with...

Страница 152: ...5 2 Configure Switch B Configure VLAN 2 SwitchB system view SwitchB vlan 2 SwitchB Vlan2 port gigabitethernet 1 0 5 SwitchB vlan2 quit SwitchB interface vlan interface 2 SwitchB Vlan interface2 ip add...

Страница 153: ...information of the VRRP group on Switch B When Switch A fails the detailed information of VRRP group 1 on Switch B is displayed SwitchB Vlan interface2 display vrrp verbose IPv4 Standby Information R...

Страница 154: ...packets sent from Host A to Host B are forwarded by Switch A If VLAN interface 3 through which Switch A connects to the Internet is not available packets sent from Host A to Host B are forwarded by S...

Страница 155: ...e the master SwitchA Vlan interface2 vrrp vrid 1 track interface vlan interface 3 reduced 30 2 Configure Switch B Configure VLAN 2 SwitchB system view SwitchB vlan 2 SwitchB vlan2 port gigabitethernet...

Страница 156: ...2 38 160 1 The output shows that in VRRP group 1 Switch A is the master Switch B is the backup and packets sent from Host A to Host B are forwarded by Switch A If interface VLAN interface 3 through wh...

Страница 157: ...8 160 100 25 as their default gateway and hosts in VLAN 3 use 202 38 160 200 25 as their default gateway Switch A and Switch B belong to both VRRP group 1 and VRRP group 2 The virtual IP address of VR...

Страница 158: ...5 128 Create a VRRP group 2 and set its virtual IP address to 202 38 160 200 SwitchA Vlan interface3 vrrp vrid 2 virtual ip 202 38 160 200 2 Configure Switch B Configure VLAN 2 SwitchB system view Swi...

Страница 159: ...rface Vlan interface3 VRID 2 Adver Timer 1 Admin Status Up State Backup Config Pri 100 Running Pri 100 Preempt Mode Yes Delay Time 0 Auth Type None Virtual IP 202 38 160 200 Master IP 202 38 160 131 D...

Страница 160: ...n the LAN can access external networks through another gateway VRRP group 1 works in load balancing mode to make good use of network resources Configure a track entry on Switch A Switch B and Switch C...

Страница 161: ...ry 1 making the weight of Switch A decrease by more than 245 250 in this example when track entry 1 turns to negative In such a case another router with a higher weight can take over SwitchA interface...

Страница 162: ...quit Create track entry 1 to associate with the physical status of VLAN interface 3 on Switch C When the track entry becomes negative it means that the interface fails SwitchC track 1 interface vlan...

Страница 163: ...Information Track Object 1 State Positive Weight Reduced 250 Display the detailed information of VRRP group 1 on Switch B SwitchB Vlan interface2 display vrrp verbose IPv4 Standby Information Run Mod...

Страница 164: ...y Information Run Mode Load Balance Run Method Virtual MAC Total number of virtual routers 1 Interface Vlan interface2 VRID 1 Adver Timer 1 Admin Status Up State Backup Config Pri 100 Running Pri 100...

Страница 165: ...n Mode Load Balance Run Method Virtual MAC Total number of virtual routers 1 Interface Vlan interface2 VRID 1 Adver Timer 1 Admin Status Up State Master Config Pri 120 Running Pri 120 Preempt Mode Yes...

Страница 166: ...01 State Active Virtual MAC 000f e2ff 0011 Take Over Owner ID 0000 5e01 1101 Priority 85 Active local Redirect Time 93 secs Time out Time 1293 secs Forwarder 02 State Listening Virtual MAC 000f e2ff 0...

Страница 167: ...Owner ID 0000 5e01 1103 Priority 127 Active 10 1 1 3 Forwarder 03 State Active Virtual MAC 000f e2ff 0013 Owner Owner ID 0000 5e01 1105 Priority 255 Active local Forwarder Weight Track Information Tr...

Страница 168: ...rity is higher than that of Switch C IPv6 based VRRP configuration examples This section provides these configuration examples Single VRRP group configuration example VRRP interface tracking configura...

Страница 169: ...6 vrid 1 virtual ip 1 10 Set the priority of Switch A in VRRP group 1 to 1 10 which is higher than that of Switch B so that Switch A can become the master SwitchA Vlan interface2 vrrp ipv6 vrid 1 prio...

Страница 170: ...6 verbose command Display the detailed information of VRRP group 1 on Switch A SwitchA Vlan interface2 display vrrp ipv6 verbose IPv6 Standby Information Run Mode Standard Run Method Virtual MAC Total...

Страница 171: ...fter Switch A resumes normal operation use the display vrrp ipv6 verbose command to display the detailed information of VRRP group 1 on Switch A SwitchA Vlan interface2 display vrrp ipv6 verbose IPv6...

Страница 172: ...0 5 SwitchA vlan2 quit SwitchA interface vlan interface 2 SwitchA Vlan interface2 ipv6 address fe80 1 link local SwitchA Vlan interface2 ipv6 address 1 1 64 Create a VRRP group 1 and set its virtual...

Страница 173: ...p 1 10 Set the authentication mode for VRRP group 1 to simple and authentication key to hello SwitchB Vlan interface2 vrrp ipv6 vrid 1 authentication mode simple hello Set the VRRP advertisement inter...

Страница 174: ...interface 3 on Switch A is not available you can still ping Host B on Host A To view the detailed information of the VRRP group use the display vrrp ipv6 verbose command When interface VLAN interface...

Страница 175: ...nts Switch A and Switch B belong to both VRRP group 1 and VRRP group 2 The virtual IPv6 addresses of VRRP group 1 are 1 10 64 and FE80 10 and those of VRRP group 2 are 2 10 64 and FE90 10 Hosts in VLA...

Страница 176: ...Vlan interface2 quit Configure VLAN 3 SwitchA vlan 3 SwitchA vlan3 port gigabitethernet 1 0 6 SwitchA vlan3 quit SwitchA interface vlan interface 3 SwitchA Vlan interface3 ipv6 address fe90 1 link loc...

Страница 177: ...VRRP group 2 SwitchB Vlan interface3 vrrp ipv6 vrid 2 priority 110 Enable Switch B to send RA messages so that hosts in VLAN 3 can learn the default gateway address SwitchB Vlan interface3 undo ipv6...

Страница 178: ...e Internet through Switch A in VRRP group 2 Switch A is the backup Switch B is the master and hosts with the default gateway of 2 10 64 accesses the Internet through Switch B VRRP load balancing mode...

Страница 179: ...l IPv6 addresses as FE80 10 and 1 10 SwitchA interface vlan interface 2 SwitchA Vlan interface2 ipv6 address fe80 1 link local SwitchA Vlan interface2 ipv6 address 1 1 64 SwitchA Vlan interface2 vrrp...

Страница 180: ...interface2 vrrp ipv6 vrid 1 virtual ip fe80 10 link local SwitchB Vlan interface2 vrrp ipv6 vrid 1 virtual ip 1 10 Set the priority of Switch B in VRRP group 1 to 1 10 which is higher than that of Swi...

Страница 181: ...e 3 on Switch C When the track entry becomes negative it means that the interface fails SwitchC track 1 interface vlan interface 3 Configure the VFs to monitor track entry 1 making the weight of Switc...

Страница 182: ...d 250 Display the detailed information of VRRP group 1 on Switch B SwitchB Vlan interface2 display vrrp ipv6 verbose IPv6 Standby Information Run Mode Load Balance Run Method Virtual MAC Total number...

Страница 183: ...Mode Load Balance Run Method Virtual MAC Total number of virtual routers 1 Interface Vlan interface2 VRID 1 Adver Timer 100 Admin Status Up State Backup Config Pri 100 Running Pri 100 Preempt Mode Yes...

Страница 184: ...un Mode Load Balance Run Method Virtual MAC Total number of virtual routers 1 Interface Vlan interface2 VRID 1 Adver Timer 100 Admin Status Up State Master Config Pri 120 Running Pri 120 Preempt Mode...

Страница 185: ...arder 01 State Active Virtual MAC 000f e2ff 4011 Take Over Owner ID 0000 5e01 1101 Priority 85 Active local Redirect Time 93 secs Time out Time 1293 secs Forwarder 02 State Listening Virtual MAC 000f...

Страница 186: ...4012 Learnt Owner ID 0000 5e01 1103 Priority 127 Active FE80 2 Forwarder 03 State Active Virtual MAC 000f e2ff 4013 Owner Owner ID 0000 5e01 1105 Priority 255 Active local Forwarder Weight Track Info...

Страница 187: ...s priority is higher than that of Switch C Troubleshooting VRRP The screen frequently displays error prompts Analysis This error is probably caused by Inconsistent configuration of the devices in the...

Страница 188: ...ms of number of virtual IP addresses virtual IP addresses advertisement interval and authentication Frequent VRRP state transition Analysis The VRRP advertisement interval is set too short Solution In...

Страница 189: ...ission Deploying only one device even with high reliability in such a network risks a single point of failure and cannot meet the requirement as shown in Figure 47 Figure 47 Network with one device de...

Страница 190: ...stateful failover Introduction to stateful failover states The stateful failover states include Silence Indicates that the device has just started or is transiting from synchronization state to indepe...

Страница 191: ...after the configurations take effect For a device providing portal services you need to perform further configurations on the device before it can automatically back up portal service information to...

Страница 192: ...er services for the backup VLAN such as MAC VLAN or Voice VLAN otherwise the operation of stateful failover may be affected The interfaces assigned to a backup VLAN can forward other packets besides s...

Страница 193: ...figure Device B Create VLAN 100 DeviceB system view DeviceB vlan 100 Assign GigabitEthernet 1 0 1 to VLAN 100 DeviceB vlan100 port gigabitethernet 1 0 1 DeviceB vlan100 quit Assign GigabitEthernet 1 0...

Страница 194: ...t 1 0 1 and GigabitEthernet 1 0 3 Do not configure other functions or parameters on a failover interface To run NAT on two failover devices you need to configure two identical NAT address pools for ea...

Страница 195: ...ause a large quantity of data to be dropped The hello mechanism is unacceptable for delay sensitive services such as voice service Moreover this detection method largely relies on the routing protocol...

Страница 196: ...ce addresses 3 BFD uses the information to establish BFD sessions Figure 52 BFD fault detection on OSPF routers BFD fault detection 1 BFD detects a link failure 2 BFD clears the neighbor session 3 BFD...

Страница 197: ...rate in one of the following BFD operating modes Asynchronous mode both endpoints periodically send BFD control packets to each other BFD considers that the session is down if it receives no BFD contr...

Страница 198: ...eserved for future use State Sta Current BFD session state Its value can be 0 for AdminDown 1 for Down 2 for Init and 3 for Up Poll P If set the transmitting system is requesting verification of conne...

Страница 199: ...ween received BFD echo packets that this system is capable of supporting If this value is zero the transmitting system does not support receipt of BFD echo packets Auth Type The authentication type in...

Страница 200: ...the destination port number for multi hop BFD control packets bfd multi hop destination port port number Optional 4784 by default Configure the source IP address of echo packets bfd echo source ip ip...

Страница 201: ...nimum interval for transmitting BFD control packets on Router A and the minimum interval for receiving BFD control packets on Router B The actual transmitting interval on Router B is 300 milleseconds...

Страница 202: ...lay information about BFD enabled interfaces display bfd interface verbose begin exclude include regular expression Available in any view Display information about enabled BFD debugging display bfd de...

Страница 203: ...ance and informs the track module of detection results The track module sends the detection results to the associated application module and the application module takes actions when the tracked objec...

Страница 204: ...y can preempt as the master to forward packets When the uplink interface recovers if the track module immediately notifies the original master to restore its priority the master will immediately forwa...

Страница 205: ...the track module that the tracked object is unreachable Then the track module sets the track entry to the Negative state If the specified threshold is not reached the NQA module tells the track module...

Страница 206: ...track track entry number bfd echo interface interface type interface number remote ip remote ip local ip local ip delay negative negative time positive positive time Required No track entry is created...

Страница 207: ...responsibility of the failed master When VRRP works in standard protocol mode or load balancing mode associate the track module with the VRRP group to implement the following objects Change the priori...

Страница 208: ...d virtual router id virtual ip virtual address Required No VRRP group is created by default Associate a track entry with the VRRP VF vrrp ipv6 vrid virtual router id weight track track entry number re...

Страница 209: ...the track entry shows that the reachability of the next hop of the static route is unknown and the static route is valid Follow these steps to associate track with static routing To do Use the command...

Страница 210: ...state of the track entry shows that the object is not available and the apply clause is invalid The Invalid state of the track entry shows that the apply clause is valid The following objects can be...

Страница 211: ...ded through Switch A When VRRP finds that there is a fault on the uplink of Switch A through NQA packets from Host A to Host B are forwarded through Switch B Figure 54 Network diagram for VRRP track N...

Страница 212: ...to 1 10 SwitchA Vlan interface2 vrrp vrid 1 priority 110 Set the authentication mode of VRRP group 1 to simple and the authentication key to hello SwitchA Vlan interface2 vrrp vrid 1 authentication m...

Страница 213: ...chB Vlan interface2 display vrrp verbose IPv4 Standby Information Run Mode Standard Run Method Virtual MAC Total number of virtual routers 1 Interface Vlan interface2 VRID 1 Adver Timer 5 Admin Status...

Страница 214: ...ween Switch A and Switch C the priority of Switch A decreases to 80 Switch A becomes the backup and Switch B becomes the master Packets from Host A to Host B are forwarded through Switch B Configuring...

Страница 215: ...RRP group 1 to 1 10 SwitchA Vlan interface2 vrrp vrid 1 virtual ip 192 168 0 10 SwitchA Vlan interface2 vrrp vrid 1 priority 110 SwitchA Vlan interface2 return 3 Configure BFD on Switch B Configure th...

Страница 216: ...0000 5e00 0101 Master IP 192 168 0 101 Display the detailed information of VRRP group 1 on Switch B SwitchB display vrrp verbose IPv4 Standby Information Run Mode Standard Run Method Virtual MAC Tota...

Страница 217: ...interface2 VRID 1 Adver Timer 1 Admin Status Up State Master Config Pri 100 Running Pri 100 Preempt Mode Yes Delay Time 0 Auth Type None Virtual IP 192 168 0 10 Virtual MAC 0000 5e00 0101 Master IP 19...

Страница 218: ...sion on Switch A Create track entry 1 to be associated with the BFD session to check whether the uplink device with the IP address 1 1 1 2 is reachable SwitchA track 1 bfd echo interface vlan interfac...

Страница 219: ...101 VRRP Track Information Track Object 1 State Positive Pri Reduced 20 Display the information of track entry 1 on Switch A SwitchA display track 1 Track ID 1 Status Positive Duration 0 days 0 hours...

Страница 220: ...s 1 Interface Vlan interface2 VRID 1 Adver Timer 1 Admin Status Up State Backup Config Pri 110 Running Pri 90 Preempt Mode Yes Delay Time 0 Auth Type None Virtual IP 192 168 0 10 Master IP 192 168 0 1...

Страница 221: ...real time If the master route is unavailable the backup route takes effect and Switch A forwards packets to 30 1 1 0 24 through Switch C Similarly Switch D is the default gateway of the hosts in segme...

Страница 222: ...try 1 specifying that five consecutive probe failures trigger the static routing track NQA collaboration SwitchA nqa admin test icmp echo reaction 1 checked element probe fail threshold type consecuti...

Страница 223: ...ncy 100 Configure reaction entry 1 specifying that five consecutive probe failures trigger the static routing track NQA collaboration SwitchD nqa admin test icmp echo reaction 1 checked element probe...

Страница 224: ...eference object NQA entry admin test Reaction 1 Display the routing table of Switch A SwitchA display ip routing table Routing Tables Public Destinations 10 Routes 10 Destination Mask Proto Pre Cost N...

Страница 225: ...30 1 1 0 24 Configure static routes on these routers so that the two segments can communicate with each other and configure route backup to improve reliability of the network Switch A is the default...

Страница 226: ...30 1 1 0 24 10 3 1 3 preference 80 Configure the source address of BFD echo packets as 10 10 10 10 SwitchA bfd echo source ip 10 10 10 10 Configure track entry 1 and associate it with the BFD session...

Страница 227: ...extHop Interface 10 2 1 0 24 Direct 0 0 10 2 1 1 Vlan2 10 2 1 1 32 Direct 0 0 127 0 0 1 InLoop0 10 3 1 0 24 Direct 0 0 10 3 1 1 Vlan3 10 3 1 1 32 Direct 0 0 127 0 0 1 InLoop0 20 1 1 0 24 Direct 0 0 20...

Страница 228: ...till communicate with the hosts in 30 1 1 0 24 SwitchA ping a 20 1 1 1 30 1 1 1 PING 30 1 1 1 56 data bytes press CTRL_C to break Reply from 30 1 1 1 bytes 56 Sequence 1 ttl 254 time 2 ms Reply from 3...

Страница 229: ...1 1 10 24 Vlan int2 10 1 1 1 24 Vlan int2 10 1 1 2 24 Host B 10 1 1 3 24 20 1 1 1 24 Internet Vlan int3 10 1 2 1 24 Vlan int3 10 1 3 1 24 Vlan int3 10 1 3 2 24 Vlan int3 10 1 2 2 24 Switch C Switch D...

Страница 230: ...0 Virtual MAC 0000 5e00 0101 Master IP 10 1 1 1 VRRP Track Information Track Object 1 State Positive Pri Reduced 30 Display detailed information about VRRP group 1 on Switch B SwitchB Vlan interface2...

Страница 231: ...d 30 After shutting down the uplink interface on Switch A display detailed information about VRRP group 1 on Switch B SwitchB Vlan interface2 display vrrp verbose IPv4 Standby Information Run Mode Sta...

Страница 232: ...AN 180 Configuring the Ethernet OAM connection detection timers 10 Configuring VRRP for IPv4 127 Configuring VRRP for IPv6 134 Creating an RRPP domain 65 D Displaying and maintaining BFD 190 Displayin...

Страница 233: ...4 Setting the port shutdown mode 45 Smart Link configuration examples 100 Smart Link configuration task list 96 Smart Link overview 93 Stateful failover configuration example 180 T Track configuration...

Отзывы:

Нет отзывов