Cisco Dial NMS Скачать руководство пользователя страница 64 | Manualshive

Страница: 64 / 132

background image

Task 3Using MRTG to Monitor and Graph Traffic Loads

How to Inspect and Interpret Data

$"

Basic Dial NMS Implementation Guide

"/tmp/crontabmMaqZd" 14 lines, 610 characters

#ident "@(#)root 1.19 98/07/06 SMI" /* SVr4.0 1.1.3.1 */

#

# The root crontab should be used to perform accounting data collection.

#

# The rtc command is run to adjust the real time clock if and when

# daylight savings time changes.

#

10 3 * * 0,4 /etc/cron.d/logchecker

10 3 * * 0 /usr/lib/newsyslog

15 3 * * 0 /usr/lib/fs/nfs/nfsfind

1 2 * * * [ -x /usr/sbin/rtc ] && /usr/sbin/rtc -c > /dev/null 2>&1

30 3 * * * [ -x /usr/lib/gss/gsscred_clean ] && /usr/lib/gss/gsscred_clean

0,5,10,15,20,25,30,35,40,45,50,55 * * * * /opt/downloads/mrtg/mrtg-2.8.8/run/mrtg

/opt/downloads/mrtg/mrtg-2.8.8/run/travis-nas-01.cfg

On a 5-minute time interval, MRTG will start up, read the configuration file, and re-generate
performance graphs.

Sending MRTG Graphs to a Web Server

MRTG builds all the graphs and web pages.

To browse and view the graphs produced by MRTG, make sure the web server is running. For
information on how to set up a web server, go to http://www.apache.org/

To send MRTG graphs to a web server, follow these steps:

Step 1

Verify that the configuration file points to the correct working directory (WorkDir:) on your web server
by entering the more command. See WorkDir: in the following example.

igloo:/opt/downloads/mrtg/mrtg-2.8.8/run ->more travis-nas-01.cfg

WorkDir: /export/home/www/mrtg/travis-nas-01/dial

# set defaults

Options[_]: growright

# make legends reflect these are call counters

YLegend[_]: Active Calls

ShortLegend[_]: calls

LegendI[_]:  calls:

LegendO[_]:  calls:

.

.

.

Step 2

To send the web pages and graphs to the web-server directory, enter the following command:

igloo:/opt/downloads/mrtg/mrtg-2.8.8/run ->./mrtg travis-nas-01.cfg

igloo:/opt/downloads/mrtg/mrtg-2.8.8/run ->

Now, the crontab will automatically perform this function every five minutes.

Ignore any Rateup WARNING errors, which means that crontab is working in the background.

Rateup WARNING: .//rateup The backup log file for 172.21.101.20.178 was invalidl

Rateup WARNING: .//rateup Can't remove 172.21.101.20.178.old updating log file

Rateup WARNING: .//rateup Can't rename 172.21.101.20.178.log to 172.21.101.20.1e

Rateup WARNING: .//rateup could not read the primary log file for 172.21.101.209

«
...
62
63
64
65
66
...
»

Содержание Dial NMS

Страница 1: ... 95134 1706 USA http www cisco com Cisco Systems Inc Corporate Headquarters Tel 800 553 NETS 6387 408 526 4000 Fax 408 526 4100 Basic Dial NMS Implementation Guide Internetworking Solutions Guide August 2000 Text Part Number OL 0556 01 ...

Страница 2: ...dy AtmDirector Browse with Me CCDA CCDE CCDP CCIE CCNA CCNP CCSI CD PAC the Cisco logo Cisco Certified Internetwork Expert logo CiscoLink the Cisco Management Connection logo the Cisco NetWorks logo the Cisco Powered Network logo Cisco Systems Capital the Cisco Systems Capital logo Cisco Systems Networking Academy the Cisco Systems Networking Academy logo the Cisco Technologies logo Fast Step Fire...

Страница 3: ...13 What are the Basic Components of SNMP 14 About Basic SNMP Message Types and Commands 15 What are SNMP MIBs 16 What is SNMPv1 18 What is SNMPv2 19 About SNMP Management 20 About SNMP Security 21 Network Design for a Dial NMS Case Study 23 Introduction to the Case Study 23 Benefits of a Dial NMS 24 Dial NMS Planning Questionnaire 25 Dial NMS Service Definition 27 Network Topology 30 Hardware Requ...

Страница 4: ...How to Inspect and Interpret Data 56 Creating and Editing a Configuration File 59 Sending MRTG Graphs to a Web Server 64 Task 4 Using Syslog NTP and Modem Call Records to Isolate and Troubleshoot Fault s 67 About Syslog 67 About NTP 69 About Modem Call Records 69 Enabling NTP on a Cisco IOS Device 71 Setting Up an NTP Client 72 Troubleshooting the NTP Client 74 Enabling Syslog and Modem Call Recor...

Страница 5: ...guration 102 About SNMP Demand Polls 105 Performing an SNMP Demand Poll 105 Testing SNMP Get Requests 107 Troubleshooting SNMP and a Demand Poll 108 Verifying that SNMP Traps Are Received 108 Unmanaging the Dial Ports 110 Creating and Adjusting Maps 111 About Discovery Filters 112 Setting Up and Editing a Discovery Filter 113 Using the HPOV CLI to Enter a Device into the Database 115 Task 8 Using ...

Страница 6: ...Contents LE Basic Dial NMS Implementation Guide ...

Страница 7: ...as Simple Network Management Protocol SNMP Network Time Protocol NTP and syslog Hands on experience working with Cisco routers IOS technologies and UNIX Success configuring a Cisco network access server NAS for basic IP modem services A Cisco Certified Network Associate CCNA certificate or equivalent level of experience Scope This guide provides guidelines and a case study for Designing a dial NMS...

Страница 8: ... access server configurations http www cisco com pcgi bin Support PSP index pl i Products Access_Products Information about integrating high end NMS systems in to a dial access environment http www cisco com univercd cc td doc cisintwk intsolns index htm Conventions Caution Means reader be careful In this situation you might do something that could result in equipment damage or loss Convention Des...

Страница 9: ...al endeavor of the University of Texas at Austin and Cisco Systems Inc http mccain ots utexas edu index html Wholesale Dial Resources Provides links to technical documents related to wholesale dial Internet access services http mccain ots utexas edu coe wholesaledial index html Technical Assistance Center Provides technical support information about Cisco technologies Locate your technology of int...

Страница 10: ...shoot access VPN solutions See also Access VPDN Dial in Using L2TP http www cisco com univercd cc td doc cisintwk intsolns index htm Freeware Sunfreeware com A repository of freeware programs and news for Solaris http www sunfreeware com The UCD SNMP Home Page Provides an overview of UCD SNMP links to the FTP site recent news documentation bug reports mailing lists and where to go for more informa...

Страница 11: ...figuration notes brochures descriptions of service offerings and download access to public and authorized files CCO serves a wide variety of users through two interfaces that are updated and enhanced simultaneously a character based version and a multimedia version that resides on the World Wide Web WWW The character based CCO supports Zmodem Kermit Xmodem FTP and Internet e mail and it is excelle...

Страница 12: ...or http www europe cisco com Documentation Feedback If you are reading Cisco product documentation on the World Wide Web you can submit comments electronically Click Feedback in the toolbar and select Documentation After you complete the form click Submit to send it to Cisco You can also submit feedback on Cisco documentation by sending an e mail to bug doc cisco com or sending a fax to 408 527 80...

Страница 13: ...ch is described in RFC 1157 http www ietf org rfc rfc1157 SNMP Version 2 SNMPv2 An improved version of SNMPv1 that includes additional protocol operations For the SNMPv2 Structure of Management Information SMI see RFC 1902 http www ietf org rfc rfc1902 SNMP Version 3 SNMPv3 SNMPv3 has yet to be standardized The case study in this guide describes how to create a dial NMS environment To successfully...

Страница 14: ...ation and makes that information available by using SNMP Network Management Systems NMS Run applications that monitor and control managed devices NMS provide resources required for network management In the case study the NMS applications are UCD SNMP MRTG HPOV CW2000 RME Table 1 Related SNMP Documentation and Sites Site Description URL SNMP Technology TAC Page Network design tips implementation a...

Страница 15: ...MS initiated commands used by an NMS to control managed devices The NMS changes the values of variables stored within managed devices Trap Agent initiated messages sent from a managed device which reports events to the NMS The Cisco IOS generates SNMP traps for many distinct network conditions Through SNMP traps the Network Operations Center NOC is notified of network events such as Link up down c...

Страница 16: ...racteristic of a managed device Managed objects reference one or more object instances variables Two types of managed objects exist Scalar objects Define a single object instance Tabular objects Define multiple related object instances that are grouped together in MIB tables Object identifier or object ID Identifies a managed object in the MIB hierarchy The MIB hierarchy is depicted as a tree with...

Страница 17: ...in the experimental branch A managed object has these unique identities The object name For example iso identified organization dod internet private enterprise cisco temporary variables AppleTalk atInput or The equivalent object descriptor For example 1 3 6 1 4 1 9 3 3 1 ccitt 0 iso 1 iso ccitt 2 registration authority 1 standard 0 dod 6 member body 2 identified organization 3 mgmt 2 directory 1 e...

Страница 18: ...stract Syntax Notation One ASN 1 The SNMPv1 SMI is defined in RFC 1155 http www ietf org rfc rfc1155 The SMI makes three specifications ASN 1 data types SMI specific data types SNMP MIB tables SNMPv1 and ASN1 Data Types The SNMPv1 SMI specifies that all managed objects must have a subset of associated ASN 1 data types Three ASN 1 data types are required Name Serves as the object identifier object ...

Страница 19: ...Pv1 SMI defines structured tables that are used to group the instances of a tabular object an object that contains multiple variables Tables contain zero or more rows that are indexed to allow SNMP to retrieve or alter an entire row with a single Get GetNext or Set command SNMPv1 Protocol Operations SNMP is a simple request response protocol The NMS issues a request and managed devices return resp...

Страница 20: ...te the precise level of support that an agent claims with respect to a MIB group An NMS can adjust its behavior towards agents according to the capability statements associated with each agent SNMPv2 Protocol Operations The Get GetNext and Set operations used in SNMPv1 are exactly the same as those used in SNMPv2 SNMPv2 however adds and enhances protocol operations The SNMPv2 trap operation for ex...

Страница 21: ...enerated by an authorized entity so the message results in unauthorized accounting management or configuration management operations Message sequence and timing modifications Occurs when an unauthorized entity reorders delays or copies and later replays a message generated by an authorized entity Disclosure Results when an unauthorized entity extracts values stored in managed objects The entity ca...

Страница 22: ...Overview of Basic SNMP Building Blocks About SNMP Security Basic Dial NMS Implementation Guide ...

Страница 23: ...HEnet is an ISP in Austin Texas that wants to develop a dial NMS and integrate it with its existing Network Operations Center NOC THEnet has two dial point of presences POPs that provide dial up services for the following types of customers Residential subscribers Corporations who outsource their dial up services and want to avoid the overhead of operating their own dial POP Figure 4 THEnet Operat...

Страница 24: ...rmance management S Security management A dial NMS provides the FCAPS management functions for a DIAS Benefits of a Dial NMS A dial NMS Increases network availability Improves end user satisfaction by improving service performance Provides fault isolation capabilities which improves fault analysis information Reduces network support costs Enables capacity plannning Enables security improvements Pr...

Страница 25: ...orate outsourcing services What is the user growth projection for the next 5 years 3 months Current deployment requirement 1 year Current design plan requirement 5 years Future scalability plan requirement 3 months 50 000 users 1 year 100 000 users 5 years 1 million users What is the user to line ratio during busy hours 10 1 What level of service must you guarantee to your customers Guaranteed up ...

Страница 26: ...orts in HTML web pages and online graphs Who will monitor the management systems The network operations staff How will network operators be notified of network problems By sending e mail to their pagers For fault and performance management purposes do you need to provide call detail records Yes Disconnect cause codes and retrain counters must be inspected What security protocols do you use for aut...

Страница 27: ...the SNMP Management Information Bases MIBs and create the SNMP framework for the dial NMS The Cisco IOS command line interface CLI Troubleshoot network connectivity problems by collecting robust network statistics For example use the following commands show controller t1 show isdn status debug ppp negotiation show isdn service debug ppp error debug isdn events debug isdn q921 debug isdn q931 Syslo...

Страница 28: ...configuration changes are made to the routers and identify who is making the changes Authenticated usernames also appear in syslog Effective IP address management Manage all assigned IP subnets by using a DNS server and the application Cisco Network Registrar Web based management Navigate devices and enable HTTP access to the CLI Accounting management Send accounting information to a database that...

Страница 29: ...igate devices and enable HTTP access to the CLI War Dialer Test remote client PCs by using a free client simulator Security management Authenticate authorize and account for dial access clients modem users in each POP by using RADIUS Authenticate authorize and account for router administrators in the NOC by using TACACS Review the AAA service security logs Review the AAA server database by using S...

Страница 30: ...POPs Figure 6 Network Topology for the NOC An important design issue to consider is where to send syslog data If syslog data is sent back to a central site NOC the syslog data must travel across WAN links Estimate and monitor how much syslog data is generated by each POP and the impact on the WAN links Modem call records can add a significant amount of traffic to syslog data 38197 Cisco AS5800 acc...

Страница 31: ...to provide access in to the Internet from the PSTN Cisco IOS Release 12 0 7 T is installed in each access server 2 backbone gateways Enables management data streams to enter the NOC Routes traffic to the intranet WAN and the Internet 2 Cisco 2511 OOB console servers Accesses the console ports in the Cisco AS5800s by using out of band OOB management lines 3 AAA servers One server in each POP to aut...

Страница 32: ...RTG version 2 8 12 Monitors and graphs the traffic load on the network Web based management Manages a network by using light weight NMS tools LWT A LWT is light on Budget Staff support Course requirements GUI requirements THEnet requires the following LWTs Device Navigator A web page that links network devices together Cisco IOS Command Center A web page that provides HTTP access to the CLI Log Fi...

Страница 33: ...implify IP address management each POP uses a similar IP subnetting plan CiscoSecure Unix version 2 3 3 Authenticates authorizes and accounts for dial access clients in each POP by using RADIUS Authenticates authorizes and accounts for router administrators in the NOC by using TACACS Uses AAA accounting records to collect performance data fault data and track router configuration changes War Diale...

Страница 34: ...nets DeviceID 172 21 10 0 24 172 22 10 0 24 Identifies each Cisco IOS device with a unique fixed and stable loopback IP address for network management purposes One IP address is assigned to the loopback 0 interface of each Cisco IOS device One IP address block is used to simplify IP security filtering at the NOC This technique protects the NOC from devices that should not access management service...

Страница 35: ...to Isolate and Troubleshoot Faults Task 5 Setting Up a Web Portal for the Dial NMS Phase B Monitoring and maintaining basic dial NMS functions by using commercially available management systems Task 6 Managing IP Addresses by Using DNS Task 7 Using HP OpenView to Create the SNMP Framework Task 8 Using CiscoWorks 2000 Resource Manager Essentials Note Providing information for integrating high end m...

Страница 36: ...Network Design for a Dial NMS Case Study Implementation and Operation Tasks Basic Dial NMS Implementation Guide ...

Страница 37: ...e CPU problems on a Cisco IOS device For example do not get the ARP table walk large portions of a MIB tree poll the wrong OID too frequently or get statistics that have an entry for every interface For example a Cisco 7200 may have 10 interfaces whereas a Cisco AS5800 may have 3 000 interfaces For a complete list of available Cisco MIBs go to http www cisco com public sw center netmgmt cmtk mibs ...

Страница 38: ...GMT MIB http www cisco com univercd cc td doc cisintwk intsolns dialnms popmgt txt CISCO MODEM MGMT MIB http www cisco com univercd cc td doc cisintwk intsolns dialnms modemmgt txt Table 11 Description of CISCO POP MGMT MIB Description OID Equivalent Cisco IOS Command Number of analog calls connected cpmISDNCfgBChanInUseForAn alog 1 3 6 1 4 1 9 10 19 1 1 2 show modem summary Number of active DS0s ...

Страница 39: ...n OID Equivalent Cisco IOS Command Modems available to take calls cmSystemModemsAvailable 1 3 6 1 4 1 9 9 47 1 1 7 show modem summary Average call duration for each modem cmCallDuration 1 3 6 1 4 1 9 9 47 1 3 1 1 9 show modem Number of times each modem failed to answer cmRingNoAnswers 1 3 6 1 4 1 9 9 47 1 3 3 1 1 show modem Number of times each modem failed to train up successfully cmIncomingConne...

Страница 40: ... in use cpmActiveTTYNumber 1 3 6 1 4 1 9 10 19 1 3 1 1 14 show caller ip List of which user is using which modem slot cpmActiveModemSlot 1 3 6 1 4 1 9 10 19 1 3 1 1 6 show caller user List of which user is using which modem port cpmActiveModemPort 1 3 6 1 4 1 9 10 19 1 3 1 1 7 show caller user List of which IP addresses are currently in use cpmActiveUserIpAddr 1 3 6 1 4 1 9 10 19 1 3 1 1 4 show ca...

Страница 41: ...protects the NOC from devices that should not access management services such as TACACS RADIUS syslog and SNMP The dial NMS environment interfaces with SNMP through these applications UCD SNMP SNMP Commander Multi Router Traffic Grapher MRTG HP OpenView HPOV Cisco Works 2000 Resource Manager Essentials CW2000 RME Caution Avoid using well known community strings such as public private or cisco Thes...

Страница 42: ...p server location Lake Travis Austin Dial POP snmp server contact net admin aurora the net snmp server enable traps snmp host 172 23 10 1 traps SNMPv1 Table 13 SNMP Command Descriptions Command Purpose snmp server community 5urf5h0p RO Assigns a read only RO community string Only get requests queries can be performed The RO community string in this example 5urf5h0p allows Get requests but no Set o...

Страница 43: ...tal variables and critical device conditions This command enables 14 other commands for distinct types of SNMP traps Edit this command list to include only the traps that are used by your network environment snmp host 172 23 10 1 traps SNMPv1 Identifies the host destination for the traps Traps are sent in the SNMP v1 format in this case study Table 13 SNMP Command Descriptions continued Command Pu...

Страница 44: ...Task 1 Enabling SNMP in a Cisco IOS Device About Enabling SNMP Basic Dial NMS Implementation Guide ...

Страница 45: ... applications Learn how to use a low level troubleshooting capability in the event that other SNMP applications produce questionable results Poll any OID and verify SNMP agent responses Use stable and reliable CLI commands UCD SNMP is unobstructed by GUI functionality Explore and research MIB content Discover what functions are available to manage a Cisco IOS device Create customized scripts and t...

Страница 46: ...y gzip d v1 tar gz tar xvf v1 tar Note There are many MIBs in the tar file that you may not use Regardless Cisco recommends you keep all the MIBs on file to support your evolving network needs Exploring SNMP MIBs for Dial Netwo rks To explore the MIBs for a Cisco IOS device by using SNMP CLI commands follow the steps in this section Poll OID variables by using the commands snmpget snmpwalk and snm...

Страница 47: ...cai system sysObjectID 0 OID enterprises 9 1 188 system sysUpTime 0 Timeticks 45492606 5 days 6 22 06 06 system sysContact 0 net admin aurora the net system sysName 0 travis nas 01 the net system sysLocation 0 Lake Travis Austin Dial POP system sysServices 0 78 system 8 0 Timeticks 0 0 00 00 00 onionring Step 4 Change the OID environmental prefix by entering the commands prefix and export prefix T...

Страница 48: ...g cat export home www travis nas 01_ifTable txt SNMP table interfaces ifTable ifEntry ifIndex ifDescr ifType ifMtu ifSpeed 1 Async1 2 00 other 1500 9000 2 Async1 2 01 other 1500 9000 3 Async1 2 02 other 1500 9000 289 FastEthernet0 0 0 ethernetCsmacd 1500 100000000 290 Null0 other 1500 4294967295 291 T1 1 0 0 ds1 292 T1 1 0 1 ds1 301 T1 1 0 10 ds1 302 T1 1 0 11 ds1 303 Serial1 0 0 0 propPointToPoin...

Страница 49: ...ommands This file is read by the snmpcmds pl script For the source code go to http www cisco com univercd cc td doc cisintwk intsolns dialnms snmpdat txt snmpcmds pl A script that loads and reads a data file You can use additional data files by creating multiple instances of the original script and altering the data file descriptor For the source code go to http www cisco com univercd cc td doc ci...

Страница 50: ...Task 2 Exploring SNMP Capabilities by Using UCD SNMP About SNMP Commander Basic Dial NMS Implementation Guide Figure 8 SNMP Commander Tool ...

Страница 51: ...Task 2 Exploring SNMP Capabilities by Using UCD SNMP About SNMP Commander Basic Dial NMS Implementation Guide Figure 9 Polling Results from the table cpmActiveCallSummaryTable Command ...

Страница 52: ...Task 2 Exploring SNMP Capabilities by Using UCD SNMP About SNMP Commander Basic Dial NMS Implementation Guide ...

Страница 53: ...minutes a cron job runs MRTG to query a user configured list of OIDs and network devices After each data collection cycle the MRTG perl script posts updated graphs to a web page Efficiently compresses and archives data samples to create graphs Enables you to determine if trending data is useful for monitoring your environment before you invest in costly network performance software If trending dat...

Страница 54: ...oblems on a Cisco IOS device For example do not get the ARP table walk large portions of a MIB tree poll the wrong OID too frequently or get statistics that have an entry for every interface For example a Cisco 7200 may have 10 interfaces whereas a Cisco AS5800 may have 3 000 interfaces In this case study the tools UCD SNMP and SNMP Commander were used to inspect and understand the MIBs Based on t...

Страница 55: ... 1 3 6 1 4 1 9 9 47 1 3 1 1 9 The average call duration for each modem in the NAS No answers CISCO MODEM MGMT MIB 1 3 6 1 4 1 9 9 47 1 3 3 1 1 The number of calls not answered by a modem Failed Train CISCO MODEM MGMT MIB 1 3 6 1 4 1 9 9 47 1 3 3 1 2 The number of modem calls that failed to train up It s normal behavior for most modems to not have a 100 percent success rate Successful train CISCO M...

Страница 56: ...ary data path and jumps in the backup path The utilization of the IP backbone such as a Frame Relay link or Ethernet campus Table 16 User Information OIDs Variable Base MIB and OID Description Active user ID CISCO MODEM MGMT MIB 1 3 6 1 4 1 9 10 19 1 3 1 1 3 List of users currently connected and authenticated Active call duration CISCO MODEM MGMT MIB 1 3 6 1 4 1 9 10 19 1 3 1 1 8 Call durations fo...

Страница 57: ...g modem OIDs or the show modem Cisco IOS command SNMP rather than the Cisco IOS CLI is the preferred method to collect these counters SNMP can scale to support large numbers of access servers The following graphs show the DS0s and PPP sessions in use for 70 000 modem users calling in to a dial up service at a large university The graphs are taken from one Cisco AS5300 in a large dial up modem pool...

Страница 58: ...ently compresses and archives data to create graphs For example you can keep information for an entire year on a server without using much disk space Figure 14 Yearly Graph DS0s and PPP Sessions in Use The configuration file used to create these graphs is posted at http www cisco com univercd cc td doc cisintwk intsolns dialnms mrtg53 txt Note the numeric OIDs in the configuration file ...

Страница 59: ...ris workstation For the source code go to http ee staff ethz ch oetiker webtools mrtg pub For the documentation see the section Getting and Installing MRTG on a UNIX System at http ee staff ethz ch oetiker webtools mrtg mrtg html Step 3 Create a configuration file There are two basic ways to create the file Manually create it by using the MRTG files config text and sample mrtg config These files a...

Страница 60: ... community string work directory and device name The following definitions are used in the example The RO community string is 5urf5h0p The work directory is WorkDir export home www mrtg travis nas 01 dial The device name is travis nas 01 An electronic copy of this template is available at http www cisco com univercd cc td doc cisintwk intsolns dialnms dialmrtg txt WorkDir export home www mrtg trav...

Страница 61: ...RIAL H2 DS0s and SerialX Y in Use H2 TABLE TR TD Device TD TD travis nas 01 TD TR TR TD a href mrtg mrtg html HOME a TD TR TABLE Options travis nas 01_DS0SERIAL gauge purpose DS0s and Sw56 Target travis nas 01_DS0Sw56 1 3 6 1 4 1 9 10 19 1 1 4 0 1 3 6 1 4 1 9 10 19 1 1 10 0 5urf5h0p travis nas 01 MaxBytes1 travis nas 01_DS0Sw56 200 MaxBytes2 travis nas 01_DS0Sw56 200 Title travis nas 01_DS0Sw56 DS...

Страница 62: ...01 MaxBytes1 travis nas 01_callNoResource 200 MaxBytes2 travis nas 01_callNoResource 200 Title travis nas 01_callNoResource travis nas 01 cpmISDNNoResource and cpmModemNoResource PageTop travis nas 01_callNoResource H2 cpmISDNNoResource and cpmModemNoResource H2 TABLE TR TD Device TD TD travis nas 01 TD TR TR TD a href mrtg mrtg html HOME a TD TR TABLE purpose cmSystemModemsInUse and cmSystemModem...

Страница 63: ...LE Options travis nas 01_activeDS0vpdnSession gauge Step 5 Open the crontab file in your system by entering crontab e The e enables edit mode You can run crontab from any directory igloo crontab e tmp crontabmMaqZd 14 lines 610 characters ident root 1 19 98 07 06 SMI SVr4 0 1 1 3 1 The root crontab should be used to perform accounting data collection The rtc command is run to adjust the real time ...

Страница 64: ...erver go to http www apache org To send MRTG graphs to a web server follow these steps Step 1 Verify that the configuration file points to the correct working directory WorkDir on your web server by entering the more command See WorkDir in the following example igloo opt downloads mrtg mrtg 2 8 8 run more travis nas 01 cfg WorkDir export home www mrtg travis nas 01 dial set defaults Options _ grow...

Страница 65: ...Data Basic Dial NMS Implementation Guide Step 3 Use a web browser to view the MRTG output files in the web page directory Note If the domain name server DNS is not working a hostname cannot be used by MRTG Use the IP address instead Figure 15 MRTG Graphs Viewed by Using a Web Browser ...

Страница 66: ...Task 3 Using MRTG to Monitor and Graph Traffic Loads How to Inspect and Interpret Data Basic Dial NMS Implementation Guide ...

Страница 67: ...al conditions trace backs and CPU process overloads Capture client debug output sessions in a real time scenario Reserve telnet sessions for making configurations changes and using show commands Telnet sessions that are cluttered with debug output interfere with troubleshooting procedures Reduce network downtime by knowing when the network has quality problems Figure 16 Cisco IOS Sending Syslog Me...

Страница 68: ...tical conditions severity 2 debugging Debugging messages severity 7 emergencies System is unusable severity 0 errors Error conditions severity 3 informational Informational messages severity 6 notifications Normal but significant conditions severity 5 warnings Warning conditions severity 4 cr In this case study syslog is enabled on all Cisco access servers and backbone routers Each device sends sy...

Страница 69: ... in the network you cannot make time comparisons For a list of NTP clients go to http www eecis udel edu ntp software html About Modem Call Records A modem call record MCR is a type of syslog message that is Created when a user dials in and hangs up but it is not generated until the end of the call Used to gather statistics and modem performance logs on a per call basis such as Modulation trends V...

Страница 70: ...eb Based MCR Viewer You can view entire log files or portions of logs in the MCR viewer In addition you can parse for specific users and other call attributes for a modem call for example modulation error correction compression disconnect causes and retrains Note Modem call records are available in syslog starting with Cisco IOS Releases 11 3AA and 12 0T ...

Страница 71: ... 4 Specify the IP address for the NTP server and enable automatic calendar updates by entering the following commands ntp update calendar ntp server 172 22 255 1 Note By default the ntp clock period command is enabled in some Cisco IOS releases The Cisco IOS software appends an arbitrary number to the end of the command Step 5 Verify that the clock is synchronized with the NTP server by entering t...

Страница 72: ... many available NTP servers on the Internet If your workstation cannot reach the Internet locate an NTP server within your network Note A common practice is to configure an area border router as an NTP server for a particular subnet The area border router then points to an external NTP server Other equipment on that subnet uses the loopback 0 IP address on the area border router as an NTP server S...

Страница 73: ...net ls in dhcpd xntpd onionring usr lib inet xntpd onionring usr lib inet Step 6 Verify that the NTP daemon is running by entering the ntpq p command onionring usr lib inet ntpq p remote refid st t when poll reach delay offset disp maui rtr 01 mau CHU 1 8 u 49 64 377 1 08 0 131 0 08 onionring usr lib inet The following information appears The remote NTP server to which the workstation is connected...

Страница 74: ... descriptions logging buffered 10000 debugging no logging console guaranteed logging console informational logging trap debugging logging facility local0 logging 172 21 100 100 Table 19 NTP Problems and Solutions Problem Solution The ntp client file or the xntpd daemon cannot be found in the directories shown in the examples Verify that the workstation is running Solaris v2 6 or a later version of...

Страница 75: ...cting logs on a NAS when you are at a remote location For example turn on debugs and start logging them in the history buffer Make your test call then re connect in shell mode and inspect the debugs logging console informational no logging console guaranteed Sends the most urgent informational logs to the console port in the event the IP network or syslog server fails Alternatively send messages t...

Страница 76: ...ingle log file The syslog messages from the backbone routers are sent to a different log file To configure the syslog daemon on a Solaris syslog server follow these steps Step 1 On the syslog server edit the file syslog conf in the etc directory by using a text editor To get syslog working you must add the following line to the file local0 debug var log router log The local facility number is loca...

Страница 77: ...3 Verify the syslog daemon is running by entering the ps elf grep syslog command from the etc directory If the daemon is running a process ID is returned by the system for example 169 If the daemon is not running no ID is returned aurora etc ps elf grep syslog 8 S root 169 1 0 41 20 60756cc8 187 604e3156 Jun 19 d aurora etc Step 4 Activate the configuration changes you made in syslog conf by resta...

Страница 78: ...og file Enter the tail f command to monitor the last 10 lines of an active log file To exit tail f mode press Ctrl C aurora etc tail f var log router log May 26 17 43 12 172 21 101 20 6 122 629 May 26 20 35 23 551 CST SYS 5 CONFIG_I Configured from console by vty0 172 22 61 200 May 26 17 51 15 172 21 101 20 6 122 630 May 26 20 43 27 068 CST SYS 5 CONFIG_I Configured from console by console May 26 ...

Страница 79: ...sure to replace the variables with your own information The FTP server automatically prompts you for a login password Table 21 URL Syntax Descriptions and Examples Generic URL Syntax Description Example ftp username host directory path Uses FTP to view logs from a remote location ftp sam 172 23 84 22 var log router l og file directory path Views logs on a local host file var log router log ...

Страница 80: ...Task 4 Using Syslog NTP and Modem Call Records to Isolate and Troubleshoot Faults About Modem Call Records Basic Dial NMS Implementation Guide ...

Страница 81: ... and scripts Network policies procedures and reports Periodic and just in time reporting The help desk can access operational information for example current connected caller status The operations staff can report on current service levels Tips For more information on building a management intranet go to http www cisco com warp public cc serv mkt nmps ent tech bmi_wi htm Table 22 Utilities Provide...

Страница 82: ...rk statistics by using OID bookmarks and a web browser No keyboard is required See the About SNMP Commander section on page 49 Syslog Viewer A utility that uses FTP to access a syslog server and a web browser to view syslog messages Migration to HTTP is straightforward after security issues are addressed The use of non wrapping text is useful when viewing debug messages and modem call records See ...

Страница 83: ... 2001 To build a device linker web page follow these steps Step 1 Collect the IP addresses for the Cisco IOS devices Step 2 Collect the device console out of band OOB paths for the terminal server and the lines connected to Cisco IOS devices Step 3 Create a basic HTML table and enter the information for each device The telnet and HTTP information is in bold in the following HTML code fragment Step...

Страница 84: ...ob 01 2001 a td td 5800 td td Dial POP 1 td tr table body html Step 4 Post the device linker web page to a WWW server in the NOC Figure 20 A Device Linker Management Page Table 23 Functions and Parameters for Designing a Device Linker Web Page Function Formula Example OOB console access telnet termserver ip 20XX telnet 172 21 101 250 2001 Basic IP access telnet ip address telnet 172 21 10 1 IOS HT...

Страница 85: ... configuration fragment shows you how to configure 16 TTY lines on a Cisco 2511 terminal server line 1 16 no exec transport input telnet Step 2 If the console port is blocked you may need to telnet to the terminal server and clear the line Enter the show users EXEC command followed by the clear line type number command c2511 oob show users Line User Host s Idle Location 0 con 0 admin idle 4 tty 4 ...

Страница 86: ...securing a router is to use access control lists on all VTY lines Enable only devices in the NOC to access the VTY lines Not recommended for service providers If used you should weigh the perceived ease of use versus the additional security issues involved with HTTP access to a network device The Cisco IOS CLI Command Center is a web page utility that provides HTTP access to CLI commands on a rout...

Страница 87: ...nds useful for managing a dial Internet access service including System commands Table 26 Interface commands Table 27 Call state commands Table 28 Debug commands Table 29 Table 25 Formula and Example for Linking a CLI Command Formula Example http ip address exec ios key word cr http 172 23 84 20 exec sh caller cr Table 26 System Commands show running configuration show file systems show ip route s...

Страница 88: ...how users show modem call stats show caller ip show dialer show modem show caller timeout show dialer map show caller Table 29 Debugging Commands show logging debug isdn q931 debug aaa authentication clear counters debug modem debug aaa authorization clear logging debug ppp negotiation debug aaa accounting show debug debug ppp authentication debug aaa per user undebug all debug dialer debug vtempl...

Страница 89: ...out HTTP Access to the CLI Basic Dial NMS Implementation Guide Figure 22 Cisco IOS CLI Commander For the source code that created the Cisco IOS CLI Commander in Figure 22 go to http www cisco com univercd cc td doc cisintwk intsolns dialnms httpcli txt ...

Страница 90: ... Up a Web Portal for the Dial NMS About HTTP Access to the CLI Basic Dial NMS Implementation Guide Step 4 Click on a CLI command and view the command output in a web page Figure 23 Output for the Show Caller Command ...

Страница 91: ...rom the NOC with DNS administrative rights and instructions for setting up IP address space Track IP address assignments by using DNS reverse lookup zones within the existing DNS service Use the application Cisco Network Registrar CNR and its CLI to manage the IP address database CNR is a full featured IP address management solution for both enterprise and service provider networks It includes adv...

Страница 92: ...tive tasks by using CNR CLI commands follow these steps Step 1 Log in to the Cisco Network Registrar application by entering the following directory path opt nwreg2 usrbin nrcmd nrcmd After logging in the command mode is accessed and the prompt nrcmd appears Step 2 To create an account for an administrator enter the admin command and an associated password nrcmd admin bob create password xyz In th...

Страница 93: ...mmand and pass it to the shell opt nwreg2 usrbin nrcmd zone 101 21 172 in addr arpa listRR Note The NRCMD command mode is not used Step 8 Sort the records and parse the output by entering the following CNR command from the Unix command line opt nwreg2 usrbin nrcmd zone 101 21 172 in addr arpa listRR sort n more username password 0 IN PTR broadcast 0 the net IN NS onionring the net IN SOA onionring...

Страница 94: ...e nas 01 IN A 172 21 10 10 bobslake nas 02 IN A 172 21 10 18 doc 2610 01 IN A 172 21 10 13 doc 3810a 01 IN A 172 21 10 14 doc 3810d 01 IN A 172 21 10 15 doc AS5850 01 IN A 172 21 10 11 doc core 01 IN A 172 21 10 5 doc core 02 IN A 172 21 10 6 doc core 03 IN A 172 21 10 7 truncated for brevity In the previous example the zone command Creates an A record for the net Assigns the IP address 172 21 10 ...

Страница 95: ...two new device IP addresses In addition to adding two A records lines 1 and 2 remove the unused PTR records from the reverse zone lines 3 and 4 before adding the new PTR records in place of the unused records to the reverse zone lines 5 and 6 See line 7 to reload the DNS server Step 2 Run the script by using the b option nrcmd b 172 21 10 batch The following output appears nrcmd zone the net addRR...

Страница 96: ...IN NS onionring the net doc rtr58 01 IN A 172 21 101 20 doc rtr54 01 IN A 172 21 101 21 doc rtr53 01 IN A 172 21 101 22 doc rtr53 05 IN A 172 21 101 23 doc 3810a 01 IN A 172 21 10 14 doc 3810d 01 IN A 172 21 10 15 doc ubr7246 01 IN A 172 21 10 16 doc switch 02 IN A 172 21 10 17 Step 2 Verify that the primary zone was created by entering the zone command with the listRR option nrcmd zone the net li...

Страница 97: ...mplementation Guide To create an IP tracker web page follow these steps Step 1 Become familiar with the layout of an IP tracker web page In Figure 24 the subnet column shows a list of all managed zones The assignment column describes the purpose of each zone Figure 24 IP Tracker Web Page ...

Страница 98: ...t of a ls ZONE command to the dnszone pl script The zone list returned to the requesting web based management browser appears ls 10 21 172 in addr arpa www the net 0 host broadcast 0 the net 1 host unused 1 the net 2 host unused 2 the net 3 host unused 3 the net 4 host unused 4 the net 5 host doc core 01 the net 6 host doc core 02 the net 7 host doc core 03 the net 8 host doc ls1010 01 the net 9 h...

Страница 99: ...in the net 2000071600 serial number 3600 Refresh 1 hours 1800 Retry 30 minutes 86400 Expire 24 hours 86400 TTL 24 hours IN NS onionring the net 0 IN PTR broadcast 0 the net 1 IN PTR unused 1 the net 2 IN PTR unused 2 the net 3 IN PTR unused 3 the net 4 IN PTR unused 4 the net 5 IN PTR unused 5 the net 6 IN PTR unused 6 the net 7 IN PTR unused 7 the net 8 IN PTR unused 8 the net 9 IN PTR unused 9 t...

Страница 100: ...Task 6 Managing IP Addresses by Using DNS How to Create a Reverse DNS Zone Basic Dial NMS Implementation Guide ...

Страница 101: ...managers EM After HPOV is installed the remaining components of the network management architecture are built around HPOV Resides on the same Unix workstation as CiscoWorks 2000 Resource Manager Essentials which gathers the following database information from HPOV Device names and IP addresses Community strings Figure 25 Other Element Managers Start from HPOV Note This section assumes that HP Netw...

Страница 102: ...ory aurora opt OV bin ovw 1 5079 Step 2 Verify that all the HPOV daemons are running by entering the ovstatus command from the root directory aurora ovstatus object manager name OVsPMD state RUNNING PID 430 exit status object manager name ovwdb state RUNNING PID 431 last message Initialization complete exit status object manager name ovtrapd state RUNNING PID 433 last message Initialization comple...

Страница 103: ...om Step 3 From HPOV enter the SNMP community strings and target loopback IP addresses for each Cisco IOS device From the Options menu select SNMP Configuration In the SNMP Configuration screen enter the following information Target field The target loopback IP address for example 172 21 10 1 Community field The Read Only RO community string for example 5urf5h0p Set Community field The Read Write R...

Страница 104: ...HP OpenView to Create the SNMP Framework About HP OpenView Basic Dial NMS Implementation Guide Figure 26 SNMP Configuration Loopback IP Address and Community Strings Step 4 Click Add and Apply to submit the entries ...

Страница 105: ...er 2 or Layer 3 information Choosing discovery options is outside the scope of this document Depending on the number of devices that need to be discovered it could take hours or even days for HPOV to discover a device If HPOV cannot find a device enter the device manually in to the database See the Using the HPOV CLI to Enter a Device into the Database section on page 115 To organize and adjust th...

Страница 106: ...t the sysobjectID vendor ID for each Cisco device Associate MIBs with each device Collect interface information Table 31 Important Fields to Inspect In the Polling Results Field Description Changing SNMP sysobjectID to 1 3 6 1 4 1 9 1 162 Indicates SNMP is working and the system identifier for the device was found This field appears only the first time a device is successfully polled HPOV changes ...

Страница 107: ... Table 32 describes the important fields in Figure 29 Interface Confirms the interfaces were successfully pinged Get system description Verifies that the system description information was collected so you can identify the software version running on the device Table 31 Important Fields to Inspect In the Polling Results continued Field Description Table 32 Test IP TCP SNMP Field Descriptions Field...

Страница 108: ... 2c c community community string p port remote port t timeout retransmission timeout 1 10th seconds r retries maximum retransmission attempts Caution Overpolling the wrong OIDs overloads CPUs and crashes network devices Verifying that SNMP Traps Are Received Traps appear in the All Events Browser which reports what is happening in the network The events are updated every few seconds Understanding ...

Страница 109: ...receiving traps from devices in the network follow these steps Step 1 Open the All Events Browser From the Fault menu select Events Figure 30 Traps in the All Events Browser Step 2 Force a trap to be sent into the browser by manually causing a fault Pull out a card on a Cisco device or shut down an interface Caution Do not shut down a communication link that can cause a service outage Step 3 Look ...

Страница 110: ... three alerts At approximately 6 alerts per hour one modem can produce up 144 events each day One Cisco AS5800 fitted with 1296 modems can produce up to 186 624 modem events per day To unmanage the asynchronous and serial interfaces for a Cisco access server follow these steps Step 1 From the top level map double click on an access server icon The available interfaces and ports appear Figure 31 Av...

Страница 111: ...us ports which appear tan Tips When the status of an object changes to managed or unmanaged HPOV switches to synchronization mode Creating and Adjusting Maps Maps provide a view of the network topology and they enable you to quickly troubleshoot faults in the network HPOV automatically polls devices and builds maps for you however devices often get stacked in the map which is undesirable The follo...

Страница 112: ...el Map Adjustments Step 2 Go to View Step 3 Select Automatic Layout Step 4 Choose Off For This Submap About Discovery Filters A discovery filter is an ASCII file that HPOV reads to limit the discovery of devices on the network Use a discovery filter to Define the subnets and devices you want to monitor Avoid managing PCs and other non SNMP devices on the network Sometimes HPOV discovers too many d...

Страница 113: ...ntsolns dialnms filter txt To set up and edit a discovery filter follow these steps Step 1 Find the filters file on your Unix workstation aurora etc opt OV share conf C ls filters oid_to_sym trapd conf Step 2 Edit the filters file by using a text editor to include a node list and a filter list for your network environment aurora etc opt OV share conf C vi filters OV_CONF LANG filters HP OpenView N...

Страница 114: ...he more specific form to exclude only the segments in the engineering lan This could have been specified directly as a negation in the filter part but this form works well if you have several networks to manipulate in this manner EverythingButEngr Everything but the engineering LAN EngrLan2 Of course the above filter expressions when used as map filters pass all networks and segments You may wish ...

Страница 115: ... exit status Exit 0 Step 2 To load new devices in to the database enter the loadhosts m command from the root directory followed by a single netmask for the devices Include an end of file statement EOF to enter multiple lines with one return aurora loadhosts m 255 255 255 0 EOF 10 10 10 104 hostname 14 14 14 14 host2name EOF aurora Note Enter devices by using a DNS format IP address then hostname ...

Страница 116: ...Task 7 Using HP OpenView to Create the SNMP Framework About Discovery Filters Basic Dial NMS Implementation Guide ...

Страница 117: ...n files The following installation assumptions are made in this case study CW2000 maintenance release 2 has been installed on a Solaris workstation RME version 2 2 is available CW2000 is installed on the same Unix workstation as HP OpenView HPOV CiscoView uses HPOV as a starting point Table 33 Related References and Documents Reference URL CiscoWorks 2000 TAC Support Page Provides links to technic...

Страница 118: ...and SNMP community strings from HPOV into CW2000 RME follow these steps Step 1 Verify that the basic setup for HPOV is working correctly Incorrect SNMP community strings prevent polling cycles For basic verification steps see the Task 7 Using HP OpenView to Create the SNMP Framework section on page 101 Step 2 From the root directory verify that the HPOV database daemon is running in the background...

Страница 119: ...se Cisco devices only The SNMP community strings are automatically set during the import operation Figure 33 Devices Imported from HP OpenView Step 7 Click Next The devices are imported and a status summary appears Step 8 Click Update until you see all the devices classified as managed devices A constant pending or conflicting state indicates a problem that requires resolution Inspect the details ...

Страница 120: ...ing is Turned On To verify that polling is enabled or to alter any polling settings follow these steps Step 1 From the Admin menu click on Availability Change Polling Options Step 2 In the Select Polled Views screen select All Views and All Polled Views Figure 34 Polling Setup Step 3 Click Next Step 4 To accept the default settings click Finish ...

Страница 121: ...s Step 1 From the Tasks menu click on Availability Reachability Dashboard Figure 35 The Status of the Devices Step 2 Click a device to become familiar with the different management elements Green arrows indicate devices that are up Red arrows indicate devices that are down Step 3 To turn on continuous availability monitoring and reporting go to the Tasks menu Click on Availability Availability Mon...

Страница 122: ...ut CiscoWorks 2000RME Basic Dial NMS Implementation Guide Step 4 Select All in the Views window Figure 36 Devices Listed in the Availability Monitor Step 5 Select one or more devices Step 6 Click Finish Step 7 Inspect the available elements for the devices 38630 ...

Страница 123: ... Note You can only back up managed devices To back up the Cisco IOS start up configuration files for devices within the network follow these steps Step 1 From the Tasks menu select Device Configuration Update Archive Figure 37 The Update Archive Screen Step 2 Select All Devices Step 3 Select one or more devices from the list that appears Step 4 Click Finish The Cisco IOS start up configuration fil...

Страница 124: ...tion that lets you access dynamic status statistics and comprehensive configuration information for Cisco products To inspect device specific characteristics on different Cisco devices follow these steps Step 1 From the top level map in HPOV select a device Step 2 Go to Monitor CiscoView Step 3 Select and view different system components Figure 38 Card Positions in the Cisco AS5800 Dial Shelf ...

Страница 125: ...Task 8 Using CiscoWorks 2000 Resource Manager Essentials About CiscoWorks 2000RME Basic Dial NMS Implementation Guide Figure 39 Available Modems in the Cisco AS5800 Dial Shelf ...

Страница 126: ...Task 8 Using CiscoWorks 2000 Resource Manager Essentials About CiscoWorks 2000RME Basic Dial NMS Implementation Guide ...

Страница 127: ...ns backing up 123 Cisco Network Registrar about 91 batch files using 95 CLI commands using 92 forward zone creating 96 reverse zone creating 99 Cisco PIX 31 CiscoSecure Unix 33 Cisco TAC online ix CiscoView 124 CiscoWorks 2000 RME about 117 configurations backing up 123 design 32 devices importing 118 devices polling 120 clear line command 85 CLI commands for dial operations 27 38 87 clocking NTP ...

Страница 128: ...SNMP Commander 49 UCD SNMP 45 FTP MIBs 46 syslog messages 78 H hardware for a dial NMS 30 HP OpenView about 101 basic setup 102 CLI entering devices 115 color legend 106 110 design 32 devices entering 115 dial ports unmanaging 110 discovery filters about 112 setting up 113 filters setting up 112 get requests testing 107 maps adjusting 111 polling devices about 105 demand polls performing 105 SNMP ...

Страница 129: ...IDs to poll 55 MRTG configuration files editing 59 60 design 32 dial counters 54 59 electronic template 60 functions 53 installing 59 OIDs to poll 54 web site 59 N network topology dial NMS 30 NTP about 69 client setting up 72 client troubleshooting 74 enabling on a Cisco IOS device 71 verifying 71 ntp clock period command 71 ntpq p command 73 ntp server command 71 ntp update calendar command 71 O...

Страница 130: ... show modem call stats command 88 show modem command 39 88 show modem connect speeds command 39 show modem summary command 38 show modem version command 87 show ntp association command 72 show ntp status command 71 show users command 85 88 SNMP about 13 agent 14 community strings 42 enabling in a Cisco IOS device 41 FAQ 14 managed devices 14 management 20 message types and commands 15 MIBs 16 NMS ...

Страница 131: ...sign 26 29 tail f command 78 terminal server setting up 83 topology NOC 30 POP 30 touch command 77 troubleshooting HP OpenView 108 terminal server 85 using modem call records 69 75 using NTP 69 using syslog 69 U UCD SNMP about 45 design 32 downloading MIBs 46 installing 46 MIBs for dial exploring 46 web based access setting up 49 web site 46 Unix workstations 31 user growth projections 25 31 user ...

Страница 132: ...Index Basic Dial NMS Implementation Guide ...

Отзывы:

Нет отзывов

Похожие инструкции для Dial NMS

BroadLink SP3353

Бренд: MicroNet Страницы: 66

ION 1200 Series

Бренд: PaloAlto Networks Страницы: 90

Бренд: IDT Страницы: 292

Бренд: DIGIWEB Страницы: 16

Бренд: 2N Telekomunikace Страницы: 67

Бренд: DHD Power Cruiser Страницы: 60

Бренд: Aaeon Страницы: 100

ibaBM-FOX-i-3o-D

Бренд: IBA Страницы: 13

Бренд: Hameg Страницы: 46

Бренд: B&B Electronics Страницы: 74

Бренд: Radionics Страницы: 100

Бренд: Johnson Controls Страницы: 72

Бренд: D-Link Страницы: 141

Бренд: D-Link Страницы: 72

Бренд: Huawei Страницы: 52

Бренд: Huawei Страницы: 389

Бренд: Huawei Страницы: 24

Бренд: Huawei Страницы: 49

Бренды по названию

0 1 2 3 4 5 6 7 8 9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Популярные бренды

Загрузить еще бренды