manualshive.com logo in svg

Cisco BTS 10200 Softswitch, Troubleshooting Manual

Share
Download
Cisco BTS 10200 Softswitch Troubleshooting Manual Download Page 735

 

15-109

Cisco BTS 10200 Softswitch Troubleshooting Guide, Release 5.0.x

OL-8723-19

Chapter 15      Diagnostic Tests

Pre-Manual Switchover Switch Integrity Diagnostic Utility

Switchover Impact Alarms Check

The Cisco BTS 10200 Softswitch Pre-Manual Switchover Switch Integrity Diagnostic Utility feature 
queries the database for the following alarms to see if they are being raised by the switchover target side. 
A complete listing of these outstanding alarms is stored in a log file.

Type and Number

Description

Severity

CALLP (12)

Feature Server Both Links Down

CRITICAL

MAINTENANCE (50)

Index Table Usage Exceeded Critical Usage Threshold Level (IDX Table Usage 
Exceeded Critical Usage Threshold Level)

CRITICAL

MAINTENANCE (53)

The Central Processing Unit Usage is Over 90% Busy (The CPU Usage is Over 90% 
Busy)

CRITICAL

MAINTENANCE (55)

The Five Minute Load Average is Abnormally High

MAJOR

MAINTENANCE (57)

Memory and Swap are Consumed at Critical Levels

CRITICAL

MAINTENANCE (61)

No Heartbeat Messages Received Through the Interface (No HB Messages Received 
Through the Interface)

CRITICAL

MAINTENANCE (62)

Link Monitor: Interface Lost Communication

MAJOR

MAINTENANCE (63)

Outgoing Heartbeat Period Exceeded Limit (Outgoing HB Period Exceeded Limit)

MAJOR

MAINTENANCE (64)

Average Outgoing Heartbeat Period Exceeds Major Alarm Limit (Average Outgoing 
HB Period Exceeds Maj Alarm Limit)

MAJOR

MAINTENANCE (65)

Disk Partition Critically Consumed

CRITICAL

MAINTENANCE (66)

Disk Partition Significantly Consumed

MAJOR

MAINTENANCE (68)

The Free Inter-Process Communication Pool Buffers Below Major Threshold (The 
Free IPC Pool Buffers Below Major Threshold)

MAJOR

MAINTENANCE (69)

The Free Inter-Process Communication Pool Buffers Below Critical Threshold (The 
Free IPC Pool Buffers Below Critical Threshold)

CRITICAL

MAINTENANCE (70)

The Free Inter-Process Communication Pool Buffer Count Below Minimum Required 
(The Free IPC Pool Buffer Count Below Minimum Required)

CRITICAL

MAINTENANCE (82)

Average Outgoing Heartbeat Period Exceeds Critical Limit (Average Outgoing HB 
Period Exceeds Critical Limit)

CRITICAL

MAINTENANCE (84)

Swap Space Below Major Threshold

MAJOR

MAINTENANCE (85)

Swap Space Below Critical Threshold

CRITICAL

MAINTENANCE (107)

No Heartbeat Messages Received Through Interface From Router (No HB Messages 
Received Through Interface From Router)

CRITICAL

SIGNALING (109)

Stream Control Transmission Protocol Association Failure (SCTP Association 
Failure)

MAJOR

SIGNALING (113)

Signaling Gateway Failure

MAJOR

SIGNALING (114)

Signaling Gateway Process is Out-of-Service

MAJOR

SIGNALING (121)

Message Transfer Part 3 User Adapter Cannot Go Standby (M3UA/SUA Cannot Go 
Standby)

MAJOR

Summary of Contents for BTS 10200 Softswitch

Page 1: ... Inc 170 West Tasman Drive San Jose CA 95134 1706 USA http www cisco com Tel 408 526 4000 800 553 NETS 6387 Fax 408 527 0883 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x Revised July 22 2009 Text Part Number OL 8723 19 ...

Page 2: ... of UCB s public domain version of the UNIX operating system All rights reserved Copyright 1981 Regents of the University of California NOTWITHSTANDING ANY OTHER WARRANTY HEREIN ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED AS IS WITH ALL FAULTS CISCO AND THE ABOVE NAMED SUPPLIERS DISCLAIM ALL WARRANTIES EXPRESSED OR IMPLIED INCLUDING WITHOUT LIMITATION THOSE OF MERCHANTABILITY F...

Page 3: ...i C H A P T E R 1 Troubleshooting Overview 1 1 Introduction 1 1 Interoperability 1 1 Symptoms Problems and Solutions 1 2 General Problem Solving Model 1 2 Resolving Network Problems 1 5 Resolving System Problems 1 6 Managing Events and Alarms 1 9 Managing Event and Alarm Reports 1 10 Show Alarm Command 1 10 Report Alarm Command 1 11 Ack Alarm Command 1 12 Clear Alarm Command 1 12 Format of Alarm R...

Page 4: ...DIT 7 2 5 AUDIT 8 2 5 AUDIT 9 2 5 AUDIT 10 2 6 AUDIT 11 2 6 AUDIT 12 2 7 AUDIT 13 2 7 AUDIT 14 2 8 AUDIT 15 2 8 AUDIT 16 2 9 AUDIT 17 2 10 AUDIT 18 2 11 AUDIT 19 2 12 AUDIT 20 2 13 AUDIT 21 2 13 AUDIT 22 2 14 AUDIT 23 2 14 AUDIT 24 2 15 AUDIT 25 2 16 Monitoring Audit Events 2 17 Test Report Audit 1 2 17 Start or Stop of Signaling System 7 Circuit Identification Code Audit Audit 2 2 18 Signaling Sy...

Page 5: ... Audit Complete Audit 21 2 21 Quality of Service Gate Status Audit Complete Audit 22 2 21 Recover Memory of Dangling Gate Audit 23 2 22 No Gate in the Cable Modem Termination System for Active Connection Audit 24 2 22 Core File Present Audit 25 2 22 Troubleshooting Audit Alarms 2 23 Critical Internal Audit Failure Audit 5 2 23 Major Internal Audit Failure Audit 6 2 23 Minor Internal Audit Failure ...

Page 6: ...15 3 10 BILLING 16 3 10 BILLING 17 3 10 BILLING 18 3 10 BILLING 19 3 10 BILLING 20 3 11 BILLING 21 3 11 BILLING 22 3 11 BILLING 23 3 11 BILLING 24 3 11 BILLING 25 3 11 BILLING 26 3 11 BILLING 27 3 11 BILLING 28 3 11 BILLING 29 3 12 BILLING 30 3 12 BILLING 31 3 13 BILLING 32 3 13 BILLING 33 3 14 BILLING 34 3 14 BILLING 35 3 14 BILLING 36 3 15 BILLING 37 3 15 BILLING 38 3 16 BILLING 39 3 16 BILLING ...

Page 7: ...Secure File Transfer Protocol Transfer Failed Billing 6 3 28 Transmission Control Protocol Connection Error Billing 7 3 28 Transmission Control Protocol Packet Receive Failure Billing 8 3 28 Database Connection Error Billing 13 3 28 File Open Error Billing 14 3 28 File Write Error Billing 15 3 29 Call Data Block Send Failed Billing 29 3 29 Domain Name Mapping Failed Billing 30 3 29 Port not Specif...

Page 8: ...eded Billing 3 3 35 Billing Partition Disk Usage Critical Threshold Exceeded Billing 4 3 36 File Transfer Protocol Secure File Transfer Protocol Transfer Failed Billing 6 3 36 Transmission Control Protocol Connection Error Billing 7 3 37 Transmission Control Protocol Packet Receive Failure Billing 8 3 38 Database Connection Error Billing 13 3 38 File Write Error Billing 15 3 38 Call Data Block Sen...

Page 9: ...er Inaccessible Billing 58 3 43 C H A P T E R 4 Call Processing Troubleshooting 4 1 Introduction 4 1 Call Processing Events and Alarms 4 2 CALLP 1 4 3 CALLP 2 4 3 CALLP 3 4 3 CALLP 4 4 3 CALLP 5 4 3 CALLP 6 4 3 CALLP 7 4 3 CALLP 8 4 4 CALLP 9 4 5 CALLP 10 4 5 CALLP 11 4 6 CALLP 12 4 6 CALLP 13 4 7 CALLP 14 4 7 CALLP 15 4 7 CALLP 16 4 8 CALLP 17 4 8 CALLP 18 4 9 CALLP 19 4 9 CALLP 20 4 10 CALLP 21 ...

Page 10: ...cation Not Found in Media Gateway Table Call Processing 15 4 25 Route Index has No Trunk Group Assigned Call Processing 16 4 25 Invalid Route Index Used Call Processing 17 4 25 Unable to Play Announcement Call Processing 18 4 26 Call Routed to Unprovisioned Subscriber Call Processing 19 4 26 No Route or Trunk Group Available to Route Call Call Processing 20 4 26 Call Released Due to Maximum Hop Co...

Page 11: ...ber Call Processing 42 4 30 Call Processing Session Initiation Protocol Trigger Provisioning Error Call Processing 43 4 30 Call Processing No Session Initiation Protocol Trigger Context Found Call Processing 44 4 31 Context in call from Application Server not Found Call Processing 45 4 31 Troubleshooting Call Processing Alarms 4 32 Feature Server One Link Down Call Processing 11 4 33 Feature Serve...

Page 12: ...se and Command Line Host Mismatch Configuration 5 5 8 C H A P T E R 6 Database Troubleshooting 6 1 Introduction 6 1 Database Events and Alarms 6 2 DATABASE 1 6 2 DATABASE 2 6 3 DATABASE 3 6 4 DATABASE 4 6 5 DATABASE 5 6 6 DATABASE 6 6 6 DATABASE 7 6 7 DATABASE 8 6 8 DATABASE 9 6 8 DATABASE 10 6 9 DATABASE 11 6 10 DATABASE 12 6 11 DATABASE 13 6 12 DATABASE 14 6 13 DATABASE 15 6 13 DATABASE 16 6 14 ...

Page 13: ...ld Limit Database 14 6 24 Table Size Exceeds Major Threshold Limit Database 15 6 24 Table Size Exceeds Critical Threshold Limit Database 16 6 24 Data Replication Failed Database 17 6 25 Unexpected Runtime Data Interaction Database 18 6 25 Daily Database Backup Completed Successfully Database 19 6 25 Replication Data Flush Timeout During Switchover Database 20 6 25 Database Statistics Collection Ex...

Page 14: ... Flush Timeout During Switchover Database 20 6 33 Database Statistics Collection Exception Database 21 6 34 Unprovisioned Language Database 22 6 35 Element Management System Oracle Database Minor Error Database 23 6 36 Element Management System Oracle Database Major Error Database 24 6 37 Secure File Transfer Protocol Transfer Failed Database 25 6 37 File Write Error Database 26 6 37 Failure Setti...

Page 15: ... 20 MAINTENANCE 32 7 20 MAINTENANCE 33 7 20 MAINTENANCE 34 7 21 MAINTENANCE 35 7 21 MAINTENANCE 36 7 22 MAINTENANCE 37 7 22 MAINTENANCE 38 7 23 MAINTENANCE 39 7 23 MAINTENANCE 40 7 23 MAINTENANCE 41 7 24 MAINTENANCE 42 7 24 MAINTENANCE 43 7 25 MAINTENANCE 44 7 25 MAINTENANCE 45 7 26 MAINTENANCE 46 7 26 MAINTENANCE 47 7 27 MAINTENANCE 48 7 27 MAINTENANCE 49 7 28 MAINTENANCE 50 7 29 MAINTENANCE 51 7...

Page 16: ...7 39 MAINTENANCE 72 7 39 MAINTENANCE 73 7 40 MAINTENANCE 74 7 40 MAINTENANCE 75 7 41 MAINTENANCE 76 7 41 MAINTENANCE 77 7 41 MAINTENANCE 78 7 42 MAINTENANCE 79 7 42 MAINTENANCE 80 7 43 MAINTENANCE 81 7 43 MAINTENANCE 82 7 44 MAINTENANCE 83 7 44 MAINTENANCE 84 7 45 MAINTENANCE 85 7 45 MAINTENANCE 86 7 46 MAINTENANCE 87 7 46 MAINTENANCE 88 7 47 MAINTENANCE 89 7 47 MAINTENANCE 90 7 48 MAINTENANCE 91 ...

Page 17: ...NANCE 120 7 64 MAINTENANCE 121 7 64 MAINTENANCE 122 7 65 MAINTENANCE 123 7 65 Monitoring Maintenance Events 7 66 Test Report Maintenance 1 7 69 Report Threshold Exceeded Maintenance 2 7 70 Local Side has Become Faulty Maintenance 3 7 70 Mate Side has Become Faulty Maintenance 4 7 70 Changeover Failure Maintenance 5 7 70 Changeover Timeout Maintenance 6 7 70 Mate Rejected Changeover Maintenance 7 7...

Page 18: ...d Initialization Maintenance 35 7 75 Process Manager Sending All Processes Initialization Complete to Process Maintenance 36 7 75 Process Manager Killing Process Maintenance 37 7 75 Process Manager Clearing the Database Maintenance 38 7 75 Process Manager Cleared the Database Maintenance 39 7 75 Process Manager Binary Does not Exist for Process Maintenance 40 7 75 Administrative State Change Succe...

Page 19: ...ystem Server Response Too Slow Maintenance 71 7 81 External Domain Name System Server Response Too Slow Maintenance 72 7 81 External Domain Name System Server not Responsive Maintenance 73 7 81 Local Domain Name System Service not Responsive Maintenance 74 7 82 Mismatch of Internet Protocol Address Local Server and Domain Name System Maintenance 75 7 82 Mate Time Differs Beyond Tolerance Maintenan...

Page 20: ...ive Successive Log Files Cannot be Transferred Maintenance 109 7 87 Access to Log Archive Facility Configuration File Failed or File Corrupted Maintenance 110 7 87 Cannot Login to External Archive Server Maintenance 111 7 88 Congestion Status Maintenance 112 7 88 Central Processing Unit Load of Critical Processes Maintenance 113 7 88 Queue Length of Critical Processes Maintenance 114 7 88 Inter Pr...

Page 21: ...able Usage Exceeded Critical Usage Threshold Level Maintenance 50 7 101 A Process Exceeds 70 of Central Processing Unit Usage Maintenance 51 7 101 The Central Processing Unit Usage is Over 90 Busy Maintenance 53 7 101 The Five Minute Load Average is Abnormally High Maintenance 55 7 102 Memory and Swap are Consumed at Critical Levels Maintenance 57 7 102 No Heartbeat Messages Received Through the I...

Page 22: ... at 25 of Its Hi Watermark Maintenance 98 7 108 Inter Process Communication Input Queue Depth at 50 of Its Hi Watermark Maintenance 99 7 108 Inter Process Communication Input Queue Depth at 75 of Its Hi Watermark Maintenance 100 7 109 Switchover in Progress Maintenance 101 7 109 Thread Watchdog Counter Close to Expiry for a Thread Maintenance 102 7 109 Central Processing Unit is Offline Maintenanc...

Page 23: ...d with Error Operations Support System 10 8 11 Queue Processing Module Database Management Index Mismatch During Add or Delete Operations Support System 11 8 11 User Session Count is Approaching Threshold Limit Operation Support System 12 8 11 User Session Count Exceeds Major Threshold Limit Operation Support System 14 8 12 Session Has Been Removed by Session Control Policy Operation Support Syste...

Page 24: ... Terminal Adapter Key Establish Error Security 4 9 5 Internet Protocol Security Outgoing Security Association Not Found Security 5 9 6 Secure Session Initiation Protocol Endpoint Validation Failure Security 6 9 6 Troubleshooting Security Alarms 9 7 Internet Protocol Security Connection Down Security 3 9 7 C H A P T E R 10 Signaling Troubleshooting 10 1 Introduction 10 1 Signaling Events and Alarms...

Page 25: ...NG 30 10 19 SIGNALING 31 10 20 SIGNALING 32 10 21 SIGNALING 33 10 22 SIGNALING 34 10 23 SIGNALING 35 10 23 SIGNALING 36 10 24 SIGNALING 37 10 24 SIGNALING 38 10 24 SIGNALING 39 10 24 SIGNALING 40 10 25 SIGNALING 41 10 25 SIGNALING 42 10 25 SIGNALING 43 10 26 SIGNALING 44 10 26 SIGNALING 45 10 27 SIGNALING 46 10 27 SIGNALING 47 10 27 SIGNALING 48 10 27 SIGNALING 49 10 28 SIGNALING 50 10 28 SIGNALIN...

Page 26: ...NG 70 10 38 SIGNALING 71 10 38 SIGNALING 72 10 39 SIGNALING 73 10 39 SIGNALING 74 10 40 SIGNALING 75 10 40 SIGNALING 76 10 41 SIGNALING 77 10 41 SIGNALING 78 10 42 SIGNALING 79 10 42 SIGNALING 80 10 43 SIGNALING 81 10 44 SIGNALING 82 10 44 SIGNALING 83 10 45 SIGNALING 84 10 45 SIGNALING 85 10 46 SIGNALING 86 10 46 SIGNALING 87 10 47 SIGNALING 88 10 47 SIGNALING 89 10 48 SIGNALING 90 10 48 SIGNALIN...

Page 27: ...0 60 SIGNALING 111 10 60 SIGNALING 112 10 61 SIGNALING 113 10 62 SIGNALING 114 10 63 SIGNALING 115 10 63 SIGNALING 116 10 64 SIGNALING 117 10 64 SIGNALING 118 10 65 SIGNALING 119 10 65 SIGNALING 120 10 66 SIGNALING 121 10 66 SIGNALING 122 10 67 SIGNALING 123 10 67 SIGNALING 124 10 67 SIGNALING 125 10 68 SIGNALING 126 10 68 SIGNALING 127 10 68 SIGNALING 128 10 69 SIGNALING 129 10 69 SIGNALING 130 1...

Page 28: ...10 78 SIGNALING 151 10 79 SIGNALING 152 10 79 SIGNALING 153 10 80 SIGNALING 154 10 80 SIGNALING 155 10 81 SIGNALING 156 10 81 SIGNALING 157 10 82 SIGNALING 158 10 82 SIGNALING 159 10 83 SIGNALING 160 10 83 SIGNALING 161 10 84 SIGNALING 162 10 84 SIGNALING 163 10 85 SIGNALING 164 10 85 SIGNALING 165 10 86 SIGNALING 166 10 86 SIGNALING 167 10 87 SIGNALING 168 10 87 SIGNALING 169 10 88 SIGNALING 170 ...

Page 29: ...0 100 Route Set Failure Signaling 21 10 100 Route Set Congested Signaling 22 10 101 Destination Point Code Unavailable Signaling 23 10 101 Destination Point Code Congested Signaling 24 10 101 Unanswered Blocking Message Signaling 25 10 101 Unanswered Unblocking Message Signaling 26 10 101 Unanswered Circuit Group Blocking Message Signaling 27 10 101 Unanswered Circuit Group Unblocking Message Sign...

Page 30: ...ling 64 10 107 Media Gateway Adapter Running out of Heap Memory Signaling 65 10 107 Call Agent Internal Error Because of Which Media Gateway Adapter has to Start Automatically Signaling 66 10 107 Trunking Gateway Endpoints are out of Service at the Gateway Signaling 68 10 107 Call Agent is not Up or is not Responding to the Feature Server Signaling 69 10 107 Integrated Services Digital Network Una...

Page 31: ... Security Violation Signaling 101 10 113 H323 Network Congested Signaling 102 10 113 Aggregation Connection Down Signaling 103 10 113 Aggregation Unable To Establish Connection Signaling 104 10 113 Aggregation Gate Set Failed Signaling 105 10 113 Enhanced Subscriber Authentication BTS 10200 Delivery Function Connection Down Signaling 106 10 114 Logical Internet Protocol Addresses not Mapped Correc...

Page 32: ...eous State Signaling 139 SS7 10 119 Unanswered Information Message Signaling 140 10 119 Address not Resolved by Domain Name System Server Signaling 141 10 119 Session Initiation Protocol Trunk Operationally Out of Service Signaling 142 10 120 Internet Protocol Interface Link to the Signaling System 7 Signaling Gateway is Down Signaling 143 10 120 All Internet Interface Links to Signaling System 7 ...

Page 33: ...Effective Aggr Id Becomes Unavailable Due to Its IP Address Signaling 172 10 125 ENUM Server Domain Cannot be Resolved Into Any IP Address Signaling 173 10 125 ENUM Server Unavailable Signaling 174 10 126 ENUM Server Farm Unavailable Signaling 175 10 126 No Resources Available to Launch ENUM Query Signaling 176 10 126 ISDN Unable to Restore D Channel Into In Service Active State Signaling 177 10 1...

Page 34: ...rvice at the Gateway Signaling 68 10 138 Call Agent is not Up or is not Responding to the Feature Server Signaling 69 10 139 Signaling System 7 Stack not Ready Signaling 75 10 139 Integrated Services Digital Network Single D channel Down for Non Facility Associated Signaling Signaling 78 10 139 Trunking Gateway Unreachable Signaling 79 10 139 Out of Bounds Memory Socket Error Signaling 80 10 139 I...

Page 35: ...ion Protocol Association Configuration Error Signaling 112 10 151 Message Transfer Part 3 User Adapter Troubleshooting Procedure 10 151 Signaling Connection Control Part User Adapter Troubleshooting Procedure 10 151 Signaling Gateway Failure Signaling 113 10 152 Signaling Gateway Process Is Out of Service Signaling 114 10 152 Destination Point Code User Part Unavailable Signaling 116 10 153 Circui...

Page 36: ...ignaling Gateway Traffic Mode Mismatch Signaling 164 10 158 Residential Gateway Endpoints are out of Service at the Gateway Signaling 170 10 158 Residential Gateway Unreachable Signaling 171 10 158 Multimedia Terminal Adapter Effective Aggr Id Becomes Unavailable Due to Its IP Address Signaling 172 10 158 ENUM Server Domain Cannot be Resolved Into Any IP Address Signaling 173 10 159 ENUM Server Un...

Page 37: ... Table Structured Query Language Read Error Statistics 9 11 14 Measurement Table Structured Query Language Write Error Statistics 10 11 15 Measurement Collection Application Programming Interface Failure Statistics 11 11 15 Measurement Handshake Error Schema Inconsistency Statistics 12 11 15 Traffic and Measurements Module Application Programming Interface Failure Statistics 13 11 16 MDII Trunk St...

Page 38: ...ximum Q Depth Reached System 13 12 13 Multiple Readers and Multiple Writers Queue Reached Low Queue Depth System 14 12 13 Multiple Readers and Multiple Writers Throttle Queue Depth Reached System 15 12 13 Troubleshooting System Alarms 12 14 Inter Process Communication Queue Read Failure System 2 12 14 Inter Process Communication Message Allocate Failure System 3 12 15 Inter Process Communication M...

Page 39: ...quests 14 4 Report Requests 14 4 Troubleshooting Alerting Notification Problems 14 5 Command Responses 14 6 Success and Failure Responses 14 7 Termination Reason Responses 14 9 Trunk Reason Responses 14 11 Trunk Termination Reason Responses Signaling System 7 Only 14 12 Fault Reason Responses 14 14 Protocol Troubleshooting 14 15 Troubleshooting Signaling System 7 SIGTRAN Problems 14 15 Cisco Inter...

Page 40: ...eway Provisioning 14 38 Additional Command Line Interface Verification 14 41 Maintenance of a Call Agent Connected to an Integrated Services Digital Network Trunk Group 14 42 File Configuration bts properties 14 43 Editing bts properties 14 44 Example Edit bts properties 14 44 Privacy Screening Troubleshooting 14 44 Symptom One 14 44 Solution One 14 45 Symptom Two 14 45 Solution Two 14 45 Symptom ...

Page 41: ...ing Snoop 15 15 Query Verification Tool and Translation Verification Tool 15 19 Tool Requirements 15 19 Query Verification Tool 15 20 Overview 15 20 Command Format 15 20 Response Format 15 21 Query Errors 15 22 Query Verification Tool Measurements 15 26 Translation Verification Tool 15 27 Overview 15 27 Command Format 15 27 Response Format 15 28 Translation Verification Tool Measurements 15 29 Usi...

Page 42: ...4 Command Responses 15 54 Database Out of Synchronization 15 54 ISDN Network Loopback Test 15 55 Configuring 15 55 Originating Trunk Group 15 56 Call Agent Configuration Table 15 56 Dial Plan 15 56 Sample Configurations 15 57 Line Loopback Tests over an ISDN Trunks 15 57 Trunk Loopback Tests over an ISDN Trunk 15 65 Enhanced Traffic Measurement 15 67 Measurement Data Transport and Access 15 67 Mea...

Page 43: ... 106 Operating 15 107 CLI Commands 15 107 Pre Manual Switchover Switch Integrity Diagnostic Utility 15 107 Application Status Check 15 108 Database Check 15 108 System Time Check 15 108 Switchover Impact Alarms Check 15 109 Inter Node Communication Check 15 110 Process Configuration Check 15 110 Operating System Issues in var adm messages Check 15 110 Software Configuration Check 15 110 Installing...

Page 44: ...g Test Line 15 119 Function 15 119 Test Equipment 15 119 Test Line 15 119 TTL Dial Plan 15 119 1xx Test Lines 15 120 1xx Test Line Support 15 120 100 Test Balance 15 121 101 Test Communications and Test 15 121 102 Test Milliwatt 15 121 103 Test Signaling and Supervisory 15 121 104 Test 2 Way Test 15 121 105 Test ROTL Responder 15 121 107 Test Line Data Transmission 15 122 108 Test Digital Loopback...

Page 45: ... 16 17 Recovery Procedures 16 18 Fresh Download 16 26 Call Agent Database Download and Recovery 16 27 Recovering Shared Memory Data 16 29 Recovering Shared Memory 16 30 Restoring Subscriber and Trunk Terminations to Service 16 32 Controlling Trunks and Trunk Groups 16 33 Using the cs control Tool to Bring Subscribers In Service 16 33 Disaster Recovery Using the Automatic Shared Memory Backup 16 34...

Page 46: ...4 Initial Transmission Waiting Period mgcp t tran B 4 Scenarios with AUEP Message Retransmissions and ACK Received B 5 Scenarios with AUEP Message Retransmissions and No ACK B 7 MGCP RTO MAX B 7 MGCP MAX2 RETRIES MGCP T MAX and 2 MGCP T HIST B 8 Keepalive Process B 9 Scenario 1 MGW Reachable B 9 Scenario 2 Special Case Reachable RGW with 16 or Fewer Endpoints and No Stable Calls In Progress B 12 S...

Page 47: ...e Handling C 5 SIP Message Types C 5 Message Rejection Logic C 6 Operating C 6 Viewing MCL C 7 Editing the OLM CFG File C 7 Sample olm cfg C 8 Measurements C 10 Call Processing Measurements C 10 Service Interaction Manager Measurements C 11 Traffic Measurements Monitor Counters C 11 Miscellaneous Measurements C 12 Troubleshooting C 12 Events and Alarms C 12 Congestion Status Maintenance 112 C 12 C...

Page 48: ...Contents xlviii Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 ...

Page 49: ...ubscriber billing operations and maintenance system administrative problems It also includes details for aiding troubleshooting by utilizing diagnostic and trace procedures Objective The maintenance personnel or troubleshooters of a BTS 10200 can use this document to better understand how to troubleshoot the BTS 10200 and quickly clear network problems Audience This document is designed for engine...

Page 50: ...the information needed to monitor and troubleshoot Signaling events and alarms Chapter 11 Statistics Troubleshooting Provides the information needed to monitor and troubleshoot Statistics events and alarms Chapter 12 System Troubleshooting Provides the information needed to monitor and troubleshoot System events and alarms Chapter 13 Network Troubleshooting Provides the information needed to condu...

Page 51: ... document was updated from Version OL 8723 05 to Version OL 8723 06 to incorporate updated alarm information This document was updated from Version OL 8723 06 to Version OL 8723 08 to incorporate updated keepalive information This document was updated from Version OL 8723 08 to Version OL 8723 09 to incorporate additional Signaling 151 troubleshooting information and the Viewing Trace Logs for Thr...

Page 52: ...e Modem Termination System for Active Connection No Gate in CMTS for Active Connection AUDIT 25 Core File Present BILLING 59 Billing File Name Type Change in the Command Line Interface is Inconsistent CALLP 42 Call Failed after Local Number Portability Query with Location Routing Number of this BTS 10200 and the Directory Number Call Failed after LNP Query with LRN of this switch and the DN CALLP ...

Page 53: ...ited Gate Delete PCMM Unsolicited Gate Delete SIGNALING 156 Integrated Services Digital Network Signaling Gateway Down ISDN Signaling Gateway Down SIGNALING 157 Integrated Services Digital Network Signaling Gateway Inactive ISDN Signaling Gateway Inactive SIGNALING 158 Invalid Integrated Services Digital Network Interface Identification Invalid ISDN Interface ID SIGNALING 159 Integrated Services D...

Page 54: ...imedia Terminal Adapter Effective Aggr Id Becomes Unavailable Due to its IP Address MTA Effective Aggr Id Becomes Unavailable Due to its IP Address SIGNALING 173 ENUM Server Domain Cannot be Resolved into Any IP Address SIGNALING 174 ENUM Server Unavailable SIGNALING 175 ENUM Server Farm Unavailable SIGNALING 176 No Resources Available to Launch ENUM Query SIGNALING 177 ISDN Unable to Restore D Ch...

Page 55: ...e DefError Queue Contact DBA DATABASE 11 Element Management System Database Alert log Alerts EMS Database Alert log Alerts MAINTENANCE 63 Outgoing Heartbeat Period Exceeded Limit Outgoing HB Period Exceeded Limit MAINTENANCE 74 Local Domain Name System Service not Responsive Local DNS Service not Responsive SIGNALING 111 Stream Control Transmission Protocol Association Degraded One of Two Internet...

Page 56: ...n see the monthly What s New in Cisco Product Documentation which also lists all new and revised Cisco technical documentation at http www cisco com en US docs general whatsnew whatsnew html Subscribe to the What s New in Cisco Product Documentation as a Really Simple Syndication RSS feed and set content to be delivered directly to your desktop using a reader application The RSS feeds are a free s...

Page 57: ...t arise in the day to day operations of a telephony solution that employs the Cisco BTS 10200 Softswitch Interoperability The BTS 10200 inter works with a wide range of network elements NEs but there are certain limitations Cisco recommends that you keep the following caution in mind as you prepare to purchase and use NEs for your network Caution Some features involve the use of other NEs deployed...

Page 58: ... provisioning problems or by problems with initial status or control settings If this is the case your system should display one or more status or error messages These messages are described in detail in the following chapters along with the probable cause and recommended action s General information about the managing the alarm and error messages is provided in the Managing Events and Alarms sect...

Page 59: ...at you gathered Using the facts you can eliminate some of the potential problems from your list Depending on the data for example you might be able to eliminate hardware as a problem so that you can focus on software problems At every opportunity try to narrow the number of potential problems so that you can create an efficient plan of action Step 4 Create an action plan based on the remaining pot...

Page 60: ...yze the results to determine whether the problem has been resolved If it has then the process is complete Step 7 If the problem has not been resolved you must create an action plan based on the next most likely problem in your list Return to Step 4 change one variable at a time and repeat the process until the problem is solved Note If you exhaust all the common causes and actions either those out...

Page 61: ...hat outlines the physical location of all the devices on the network and how they are connected as well as a logical map of network addresses network numbers subnetworks and so forth Do you have a list of all network protocols implemented in your network For each of the protocols implemented do you have a list of the network numbers subnetworks zones areas and so on that are associated with them D...

Page 62: ...ntial problem If restoral of database application or operating system OS is required or if hardware repair is required use the procedures shown in Figure 1 3 The restoral procedures shown in Figure 1 3 are used only on BTS 10200 systems that are not carrying live traffic Caution These are not upgrade procedures Performing the steps in these restoral procedures brings the platform down and stops ca...

Page 63: ...as become corrupted A problem has occurred in the operating system OS on a machine in the softswitch system A hardware problem has occurred on a machine in the softswitch system The problem is external to the softswitch system Cisco TAC will provide detailed guidance until problem is resolved Cisco TAC will provide general recommend ations Problem needs to be resolved with applicable supplier s Pr...

Page 64: ...ed and system function is normal A hardware problem has occurred on a machine in the softswitch system D Locate the hardware problem in power or signal cabling frame PROM or firmware Reinstall or replace HW as necessary Confirm HW is repaired and system function is normal Did the problem clear An application file in the softswitch system has become corrupted B Reload the application from CD ROM co...

Page 65: ...occurrence through the operator interfaces without any state change being required An informational event indicates that a condition worthy of note has occurred An invalid protocol call state transition is an example of an informational event This section contains the following Managing Event and Alarm Reports page 1 10 Show Alarm Command page 1 10 Report Alarm Command page 1 11 Ack Alarm Command ...

Page 66: ...severity allows you to monitor the system for the specified type and severity of events and alarms The default type and severity is all Show Alarm Command Use the show alarm command to view all real time alarms All of the following tokens are optional show alarm id sn type type number num severity sev component id comp origin process start time yyyy mm dd hh mm ss end time yyyy mm dd hh mm ss Note...

Page 67: ...0 to 59 Report Alarm Command Use the report alarm command to view all real time alarms All of the following tokens are optional report alarm id sn type type number num severity sev component id comp origin process output file name output type file type start time yyyy mm dd hh mm ss end time yyyy mm dd hh mm ss Note If the report alarm command is issued without any tokens parameters all alarms of ...

Page 68: ...rs start time or end time Timestamp indicating the time the monitoring of the specified alarm states should start or end in the format yyyy mm dd hh mm ss where yyyy year 4 digit number mm month 01 to 12 dd day 01 to 31 hh hour 00 to 23 mm minute 00 to 59 ss second 00 to 59 Ack Alarm Command Use the ack alarm command to acknowledge an alarm i e to turn off the alarm bell ack alarm id sn The ID tok...

Page 69: ... the report AUDIT BILLING call processing CALLP CONFIG DATABASE MAINTENANCE operations support system OSS SECURITY SIGNALING STATISTICS or SYSTEM Event or Alarm Number Event or alarm numbers are preset in the BTS 10200 They are not provisionable and cannot be changed Event or Alarm Description Up to an 80 character description of the event or alarm Alarm Status Status for an alarm can be ALARM_ON ...

Page 70: ... alarm log id sn type type number num severity severity component id component id origin process id start time yyyy mm dd hh mm ss end time yyyy mm dd hh mm ss Note If the show event log or show alarm log commands are issued without any tokens parameters all events or alarms of all types and all severities for all components are displayed Issuing the show event log or show alarm log commands with ...

Page 71: ... severity The severity level of the event or alarm which can be any one of the following critical major minor warning events only info events only origin The internal designation of the process generating the event or alarm 1 to 64 ASCII characters component id The ID of the component reporting the event or alarm 1 to 32 ASCII characters Specifying the component ID setting type to a value and spec...

Page 72: ...their respective logs Show Report Properties Command Use the following show report properties command to view the event or alarm properties currently used to specify which event levels events and alarms are saved to the event or alarm logs CLI show report properties Reply Success Entries 1 3 of 3 returned TYPE EVENT_LOGSIZE VALUE 30000 TYPE ALARM_LOGSIZE VALUE 30000 TYPE EVENT_LEVEL VALUE INFO Not...

Page 73: ...values used in event and alarm reporting are user provisionable You can use the following show event prov command to display the current threshold and throttle values for any event or alarm message CLI show event prov type callp number 9 Reply Success Entry 1 of 1 returned REPORTTYPE 2 REPORTNUMBER 9 REPORTDESCRIPTION No Route Available for Carrier Dialed THRESHLIM 100 THROTTLELIM 20 DW1NAME Orig ...

Page 74: ...scribe command see the Managing Event and Alarm Reports section on page 1 10 Step 3 Set the alarm logsize and event level parameters as desired using the report properties command see the Changing Report Properties section on page 1 16 Step 4 Subscribe to alarms and request alarm summary reports as needed using the subscribe command see the Managing Event and Alarm Reports section on page 1 10 Ste...

Page 75: ...d in the text in the right column adjacent to each of the headings are explained in the sections that follow SIGNALING 20 Example DESCRIPTION Link Set Congestion SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Link Set No ONE_BYTE Link Set Name STRING 8 Congestion Level ONE_BYTE PRIMARY CAUSE Issued when the specified SS7 link set is experiencing congestion PRIMARY ACTION Monitor event reports a...

Page 76: ...d by signaling protocols or interfaces Refer to Chapter 10 Signaling Troubleshooting STATISTICS Events or alarms that provide information about system statistics Refer to Chapter 11 Statistics Troubleshooting SYSTEM Events or alarms that convey information about system status or trouble Refer to Chapter 12 System Troubleshooting Event Level The event level designates the severity levels of the eve...

Page 77: ...ng threshold count is reset back to 0 Throttle describes the number of events or alarms suppressed for every X number of events alarms For example if the throttle is 10 for a particular event or alarm then the 1st through the 9th event are discarded and the 10th event is transmitted In other words every X throttle will be transmitted and every event in between will be discarded The only exception ...

Page 78: ...1 22 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Chapter 1 Troubleshooting Overview Managing Events and Alarms ...

Page 79: ... Audit Alarms Provides the information needed to troubleshoot and correct the audit alarms Audit Events and Alarms This section provides a brief overview of the audit events and alarms for the Cisco BTS 10200 Softswitch in numerical order Table 2 1 lists all of the audit events and alarms by severity Note Click the Audit message number in Table 2 1 to display information about the event or alarm T...

Page 80: ... section on page 2 18 DESCRIPTION Test Report SEVERITY Information INFO THRESHOLD 100 THROTTLE 0 PRIMARY CAUSE This event is used for testing the new audit category PRIMARY ACTION No action is necessary DESCRIPTION Start or Stop of Signaling System 7 Circuit Identification Code Audit Start or Stop of SS7 CIC Audit SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Type of Audit STRING 64 PRIMARY CAU...

Page 81: ...etion SS7 CIC Audit Terminated Before Successful Completion SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Type of Audit STRING 64 PRIMARY CAUSE A higher priority SS7 CIC audit interrupted and terminated a lower priority SS7 CIC audit PRIMARY ACTION Do not schedule an SS7 remote termination audit to occur while a periodic SS7 local termination audit is executing DESCRIPTION Call Exceeds a Lon...

Page 82: ...ajor Internal Audit Failure Audit 6 section on page 2 23 DESCRIPTION Critical Internal Audit Failure SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Failure Details STRING 220 Probable Causes STRING 80 Corrective Actions STRING 80 PRIMARY CAUSE See the data field PRIMARY ACTION See the data field DESCRIPTION Major Internal Audit Failure SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Failur...

Page 83: ...ernal Audit Audit 8 section on page 2 19 AUDIT 9 AUDIT 9 is not used DESCRIPTION Minor Internal Audit Failure SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Failure Details STRING 220 Probable Causes STRING 80 Corrective Actions STRING 80 PRIMARY CAUSE See the data field PRIMARY ACTION See the data field DESCRIPTION Warning From Internal Audit SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS...

Page 84: ...4 DESCRIPTION Call Data Audit Complete SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Audit Information STRING 256 PRIMARY CAUSE A memory audit has been completed PRIMARY ACTION Check if any call blocks are freed as a result of the audit An investigation to determine the root cause may be useful DESCRIPTION Critical Network Time Protocol Service Failure Critical NTP Service Failure SEVERITY CRIT...

Page 85: ...ure Audit 13 section on page 2 25 DESCRIPTION Major Network Time Protocol Service Failure Major NTP Service Failure SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Failure Details STRING 220 Probable Causes STRING 80 Corrective Actions STRING 80 PRIMARY CAUSE See the data field PRIMARY ACTION See the data field DESCRIPTION Minor Network Time Protocol Service Failure Minor NTP Service Failure SEV...

Page 86: ...udit 15 section on page 2 25 DESCRIPTION Network Time Protocol Service Warning NTP Service Warning SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Failure Details STRING 220 Probable Causes STRING 80 Corrective Actions STRING 80 PRIMARY CAUSE See the data field PRIMARY ACTION See the data field DESCRIPTION Critical Index Shared Memory Error Critical IDX Shared Memory Error SEVERITY CRITICAL TH...

Page 87: ...eap tool contact Cisco TAC Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Process Heap Memory Usage Exceeds Minor Threshold Level SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 10 Heap Size in KB FOUR_BYTES Heap Limit in KB FOUR_BYTES He...

Page 88: ...ain the show_heap tool contact Cisco TAC Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Process Heap Memory Usage Exceeds Major Threshold Level SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 10 Heap Size in KB FOUR_BYTES Heap Limit in KB...

Page 89: ...w_heap tool contact Cisco TAC Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Process Heap Memory Usage Exceeds Critical Threshold Level SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 10 Heap Size in KB FOUR_BYTES Heap Limit in KB FOUR...

Page 90: ...ng Cisco TAC collect a trace log corresponding to the time of the alarm DESCRIPTION Recovered Memory of Stale Call SEVERITY WARNING THRESHOLD 20 THROTTLE 0 DATAWORDS Stale Memory Release Info STRING 128 PRIMARY CAUSE A loss of communication with originating or terminating side has occurred PRIMARY ACTION Check to see if adjacent network element is up and has a proper communication link with the BT...

Page 91: ...cause of the event refer to the Quality of Service Gate Memory Audit Complete Audit 21 section on page 2 21 DESCRIPTION Audit Found Lost Call Data Record SEVERITY MAJOR THRESHOLD 20 THROTTLE 0 DATAWORDS Error Text STRING 200 PRIMARY CAUSE An software error has occurred However the orphaned records are recovered on detection 2d PRIMARY ACTION Contact Cisco Support Contact Cisco TAC DESCRIPTION Qual...

Page 92: ...ollect a BTS 10200 trace log corresponding to the time of the event and collect the following additional information From the CMTS show packetcable gate summary show packetcable gate gate id DESCRIPTION Quality of Service Gate Status Audit Complete QoS Gate Status Audit Complete SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Num Records Audited FOUR_BYTES Audit Start Time STRING 64 PRIMARY CAUSE...

Page 93: ...ening a service request Prior to contacting Cisco TAC collect a BTS 10200 trace log corresponding to the time of the event and collect the following additional information From the CMTS show packetcable global show packetcable gate summary DESCRIPTION No Gate in the Cable Modem Termination System for Active Connection No Gate in CMTS for Active Connection SEVERITY WARNING THRESHOLD 100 THROTTLE 0 ...

Page 94: ...ent occurred To move the core file to an FTP server run pstack corefile ftp core file to ftp sj cisco com incoming Once the core files is moved to the file server FTP server contact Cisco TAC Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Core File Present SEV...

Page 95: ...6 MAJOR AUDIT 7 Minor Internal Audit Failure Audit 7 MINOR AUDIT 8 Warning From Internal Audit Audit 8 WARNING AUDIT 10 Call Data Audit Complete Audit 10 INFO AUDIT 11 Critical Network Time Protocol Service Failure Audit 11 CRITICAL AUDIT 12 Major Network Time Protocol Service Failure Audit 12 MAJOR AUDIT 13 Minor Network Time Protocol Service Failure Audit 13 MINOR AUDIT 14 Network Time Protocol ...

Page 96: ...ger valid the associated trunk facility should be released To check the current threshold setting for Long Duration calls proceed as follows Step 1 As root user from the Call Agent CA execute the following command grep LongDuration opt OptiCall ls opt OptiCall grep CA bin platform cfg Step 2 Review the command results Example results Args ems_pri_dn blg asys07EMS mssol cisco com ems_sec_dn blg asy...

Page 97: ...e Call Data Audit Complete event serves as an informational alert that a call data memory audit has been completed The call data memory audit information should be checked to see if any call blocks have been cleared as a result of the audit Critical Network Time Protocol Service Failure Audit 11 The Critical Network Time Protocol Service Failure alarm critical indicates that a critical Network Tim...

Page 98: ...ess Heap Memory Usage Exceeds Minor Threshold Level Audit 16 The Process Heap Memory Usage Exceeds Minor Threshold Level alarm minor indicates that a process heap memory usage minor threshold level crossing has occurred To troubleshoot and correct the cause of the Process Heap Memory Usage Exceeds Minor Threshold Level alarm refer to the Process Heap Memory Usage Exceeds Minor Threshold Level Audi...

Page 99: ...ice Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Prior to contacting Cisco TAC collect a trace log corresponding to the time of the alarm Audit Found Lost Call Data Record Audit 20 The Audit Found Lost Call Data Record alarm major indicates that an audit process has found a lost call data record To troubleshoot and correct the cause of...

Page 100: ...heCableModemTerminationSystemforActiveConnection Audit 24 The No Gate in the Cable Modem Termination System for Active Connection event serves as a warning that no gate in the CMTS is available for the active connection The primary cause of the warning is that a communication error has occurred between packet cable network components If the situation persists contact Cisco to obtain technical assi...

Page 101: ...he Corrective Actions dataword has been reviewed proceed with the corrective actions listed Minor Internal Audit Failure Audit 7 The Minor Internal Audit Failure alarm minor indicates that a minor internal audit failure has occurred To find the probable causes of the alarm review the data field Failure Details and Probable Causes datawords For the corrective actions for the alarm review the data f...

Page 102: ...From EMS show ems From UNIX prompt opt BTSxntp bin ntpq c peers opt BTSxntp bin ntpq c lpeers opt BTSxntp bin ntpq c lopeers opt BTSxntp bin ntpq c opeers cat etc inet ntp conf Major Network Time Protocol Service Failure Audit 12 The Major Network Time Protocol Service Failure alarm major indicates that a major NTP service failure has occurred To find the probable causes of the alarm review the da...

Page 103: ...hared memory index error has occurred To find the probable causes of the alarm review the data field Failure Details and Probable Causes datawords For the corrective actions for the alarm review the data field Corrective Actions dataword Once the Corrective Actions dataword has been reviewed proceed with the corrective actions listed Process Heap Memory Usage Exceeds Minor Threshold Level Audit 16...

Page 104: ...taining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Process Heap Memory Usage Exceeds Critical Threshold Level Audit 18 The Process Heap Memory Usage Exceeds Critical Threshold Level alarm critical indicates that a process heap memory usage critical threshold level crossing has occurred The prima...

Page 105: ...e Present alarm major indicates that a network element process has crashed and that the BTS 10200 system has created a core file to assist in determining the root cause of the network element process crash To correct the primary cause of the alarm move the core file to a file server Note While the BTS 10200 software directs its core files to the directory bin core files generated by non BTS softwa...

Page 106: ...2 28 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Chapter 2 Audit Troubleshooting Troubleshooting Audit Alarms ...

Page 107: ...larms Provides a brief overview of each billing event and alarm Monitoring Billing Events Provides the information needed to monitor and correct the billing events Troubleshooting Billing Alarms Provides the information needed to troubleshoot and correct the billing alarms Note The following billing records are created when a call is rejected due to overload conditions SS7 termination cause code 4...

Page 108: ...y Severity Critical Major Minor Warning Info Not Used BILLING 4 BILLING 3 BILLING 2 BILLING 14 BILLING 1 BILLING 9 BILLING 7 BILLING 6 BILLING 40 BILLING 42 BILLING 5 BILLING 10 BILLING 13 BILLING 8 BILLING 41 BILLING 36 BILLING 11 BILLING 35 BILLING 15 BILLING 45 BILLING 57 BILLING 12 BILLING 49 BILLING 29 BILLING 46 BILLING 59 BILLING 16 BILLING 52 BILLING 30 BILLING 47 BILLING 17 BILLING 55 BIL...

Page 109: ...k associated with the billing database in the Element Management System EMS This is because data is being written into the database faster than it is being read out of the database The minor threshold default value 70 has been exceeded PRIMARY ACTION Monitor this alarm The read process should catch up to write process within a few minutes and the alarm should not remain active SECONDARY CAUSE Some...

Page 110: ... database in the EMS This is because data is being written into the database faster than it is being read out of the database The major threshold default value 80 has been exceeded PRIMARY ACTION Monitor this alarm The read process should catch up to write process within a few minutes and the alarm should not remain active SECONDARY CAUSE Some fluctuation in disk usage is to be expected as the cal...

Page 111: ... the billing database in the EMS This is because data is being written into the database faster than it is being read out of the database The major threshold default value 90 has been exceeded PRIMARY ACTION Monitor this alarm The read process should catch up to write process within a few minutes and the alarm should not remain active SECONDARY CAUSE Some fluctuation in disk usage is to be expecte...

Page 112: ... using the change billing acct addr command SECONDARY CAUSE Unable to login to the remote host SECONDARY ACTION Use show billing acct addr command to verify that the user name is a valid user for the host specified in the billing server addr If user name is correct and the TransferType dataword shows File Transfer Protocol FTP re enter the password via the change billing acct addr command If user ...

Page 113: ... To troubleshoot and correct the cause of the alarm refer to the Transmission Control Protocol Packet Receive Failure Billing 8 section on page 3 38 DESCRIPTION Transmission Control Protocol Connection Error TCP Connection Error SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Hostname STRING 100 PRIMARY CAUSE A system call error has occurred PRIMARY ACTION Check the address in platform config...

Page 114: ...on page 3 38 Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Prior to contacting Cisco TAC collect the following additional information From the EMS UNIX prompt ps ef nodestat From the CLI prompt status system DESCRIPTION Database Connection Error SEVERITY CRITICAL THRESHO...

Page 115: ...taining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Prior to contacting Cisco TAC collect the following information From the UNIX prompt sysdef i df k DESCRIPTION File Open Error SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Path Name STRING 100 PRIMARY CAUSE System error may be out of file...

Page 116: ...n page lvi for detailed instructions on contacting Cisco TAC and opening a service request Prior to contacting Cisco TAC collect the following information From the UNIX prompt sysdef i df k BILLING 16 BILLING 16 is not used BILLING 17 BILLING 17 is not used BILLING 18 BILLING 18 is not used BILLING 19 BILLING 19 is not used DESCRIPTION File Write Error SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAW...

Page 117: ...Events and Alarms BILLING 20 BILLING 20 is not used BILLING 21 BILLING 21 is not used BILLING 22 BILLING 22 is not used BILLING 23 BILLING 23 is not used BILLING 24 BILLING 24 is not used BILLING 25 BILLING 25 is not used BILLING 26 BILLING 26 is not used BILLING 27 BILLING 27 is not used BILLING 28 BILLING 28 is not used ...

Page 118: ...end Failed CDB Send Failed SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 PRIMARY CAUSE Transmission Control Protocol TCP send call failure PRIMARY ACTION Check the port number and address of the blg and bmg processes in the platform cfg file SECONDARY CAUSE Both the EMS servers are down SECONDARY ACTION Check if both EMS servers are down If so bring at least one up DESCRIPTION Domain Name Mapping Failed...

Page 119: ...ified Billing 32 section on page 3 39 DESCRIPTION Port not Specified SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 PRIMARY CAUSE Port not specified in the platform cfg file PRIMARY ACTION Check platform cfg file and add the argument to blg port 15260 DESCRIPTION Element Management System Address not Specified EMS Address not Specified SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 PRIMARY CAUSE Either the prim...

Page 120: ...rAddr STRING 100 User Name STRING 100 PRIMARY CAUSE The billing acct addr table is not fully provisioned with the information needed to perform file transfers PRIMARY ACTION Check billing acct addr fields using the show billing acct addr command For FTP file transfer ensure that the billing server addr billing server directory user name and password not displayed are provisioned Also ensure that s...

Page 121: ...ection on page 3 40 DESCRIPTION Billing Link Restored SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Interface Name STRING 50 PRIMARY CAUSE The cable connection has been restored PRIMARY ACTION None DESCRIPTION Billing Link Failure SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Interface Name STRING 5 PRIMARY CAUSE The cable connection may have been pulled PRIMARY ACTION Restore the cable con...

Page 122: ...ructions on contacting Cisco TAC and opening a service request BILLING 39 BILLING 39 is not used DESCRIPTION Event Message Log File Access Error EM Log File Access Error SEVERITY MAJOR THRESHOLD 50 THROTTLE 0 DATAWORDS Type of Access Error STRING 25 Reason for Error STRING 40 Sequence Number OFF FOUR_BYTES Index of Event Messa FOUR_BYTES Location Tag STRING 30 PRIMARY CAUSE System error may be out...

Page 123: ...tation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Event Message Encode Failure EM Encode Failure SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Location Tag STRING 30 PRIMARY CAUSE There is a problem with the format of the data to be sent to the record keeping system RKS PRIMARY ACTION If probl...

Page 124: ...ontacting Cisco TAC and opening a service request BILLING 43 BILLING 43 is not used DESCRIPTION Error Reading Provisioned Data Using Default SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Error Code FOUR_BYTES Table Name STRING 20 Field Name STRING 20 Default Value Decim FOUR_BYTES Default Value Text STRING 20 Location Tag STRING 30 PRIMARY CAUSE An application was unable to read the provisio...

Page 125: ...witch Occurred SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Type of RKS Switch STRING 20 Location Tag STRING 30 PRIMARY CAUSE Billing changed the destination RKS to which event messages are transmitted The change could have been triggered by a communication problem with an RKS or by an attempt to reestablish RKS communication PRIMARY ACTION No action is necessary DESCRIPTION Event Message Log...

Page 126: ...THRESHOLD 50 THROTTLE 0 DATAWORDS Element Type STRING 5 File Name STRING 60 Location Tag STRING 30 PRIMARY CAUSE An open event message log file has been closed PRIMARY ACTION No action is necessary DESCRIPTION Record Keeping System Unreachable for One Hour RKSs Unreachable for 1 Hour SEVERITY MINOR THRESHOLD 25 THROTTLE 0 DATAWORDS Location Tag STRING 30 PRIMARY CAUSE Billing has not been able to ...

Page 127: ...S Location Tag STRING 30 PRIMARY CAUSE Billing has not been able to communicate with any RKS for the past three hours PRIMARY ACTION Check status of the primary and secondary RKS servers attempt to bring them into service Verify that the radius profile table and call agent profile table are provisioned such that communication with the RKS servers is possible DESCRIPTION Record Keeping System Unrea...

Page 128: ...n Server The FTP to the Billing Mediation Server may not be working The maximum disk partition for billing records has been exceeded or the maximum number of files has been exceeded PRIMARY ACTION Check Billing Mediation Server node name user name and password specified in BILLING_ACCT_ADDR table and log files Correct any error to let FTP start again If billing_server_directory dev null as in the ...

Page 129: ...UR_BYTES Directory Containing Event Messa STRING 30 Location Tag STRING 30 PRIMARY CAUSE The event message storage has reached 70 of the maximum storage allowed PRIMARY ACTION Move the event message files out of the specified directory Store them in another location or discard them DESCRIPTION Event Message Disk Space 100 Percent Full SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Number of ...

Page 130: ...NG 32 Low End of the Range of Records FOUR_BYTES High End of the Range of Records FOUR_BYTES Error Code FOUR_BYTES Location Tag STRING 32 PRIMARY CAUSE The billing data stored on the disk may have become corrupted due to a power outage ungraceful shutdown or disk failure PRIMARY ACTION The BDMS that detected the problem should have gone out of service leave it in the out of service state and conta...

Page 131: ...ROTTLE 0 DATAWORDS Instance Name STRING 65 Pop ID STRING 32 Radius Profile ID STRING 32 PRIMARY CAUSE All of the prepaid servers are down PRIMARY ACTION Check the operating status of the prepaid servers SECONDARY CAUSE The Internet Protocol IP network between the BTS 10200 plain old telephone service POTS Feature Server FS and the prepaid servers is down SECONDARY ACTION Check and correct any prob...

Page 132: ...15 File Write Error Billing 15 MAJOR BILLING 29 Call Data Block Send Failed Billing 29 MAJOR BILLING 30 Domain Name Mapping Failed Billing 30 MAJOR BILLING 31 Port not Specified Billing 31 MAJOR BILLING 32 Element Management System Address not Specified Billing 32 MAJOR BILLING 33 File Transfer Protocol Secure File Transfer Protocol Parameters Invalid Billing 33 MAJOR BILLING 35 All Billing Links ...

Page 133: ...er to the Billing Partition Disk Usage Major Threshold Exceeded Billing 3 section on page 3 35 Billing Partition Disk Usage Critical Threshold Exceeded Billing 4 The Billing Partition Disk Usage Critical Threshold Exceeded alarm critical indicates that a billing partition disk usage critical threshold crossing has occurred To troubleshoot and correct the cause of the Billing Partition Disk Usage C...

Page 134: ...n page 3 37 Transmission Control Protocol Packet Receive Failure Billing 8 The Transmission Control Protocol Packet Receive Failure alarm major indicates that a TCP packet receive failure has occurred To troubleshoot and correct the cause of the Transmission Control Protocol Packet Receive Failure alarm refer to the Transmission Control Protocol Packet Receive Failure Billing 8 section on page 3 3...

Page 135: ...rt not Specified Billing 31 The Port not Specified alarm major indicates that a port has not been specified or configured To troubleshoot and correct the cause of the Port not Specified alarm refer to the Port not Specified Billing 31 section on page 3 39 Element Management System Address not Specified Billing 32 The Element Management System Address not Specified alarm major indicates that an EMS...

Page 136: ... the billing server has failed To troubleshoot and correct the cause of the Billing Link Failure alarm refer to the Billing Link Failure Billing 37 section on page 3 40 Event Message Log File Access Error Billing 38 The Event Message Log File Access Error alarm major indicates that an event message EM log file access error has occurred To troubleshoot and correct the cause of the Event Message Log...

Page 137: ...ent Message Log File Opened Billing 45 The Event Message Log File Opened alarm minor indicates that a event message log file has been opened To troubleshoot and correct the cause of the Event Message Log File Opened alarm refer to the Event Message Log File Opened Billing 45 section on page 3 41 Event Message Log File Closed Billing 46 The Event Message Log File Closed alarm minor indicates that a...

Page 138: ...rm minor indicates that the event message disk space is 50 percent full To troubleshoot and correct the cause of the Event Message Disk Space 50 Percent Full alarm refer to the Event Message Disk Space 50 Percent Full Billing 53 section on page 3 42 Event Message Disk Space 70 Percent Full Billing 54 The Event Message Disk Space 70 Percent Full alarm major indicates that the event message disk spa...

Page 139: ...formation on the prepaid server To correct the secondary cause of the event verify the billing information on the prepaid server Signaling Prepaid Server Inaccessible Billing 58 The Signaling Prepaid Server Inaccessible alarm major indicates that the signaling prepaid server has become inaccessible To troubleshoot and correct the cause of the Signaling Prepaid Server Inaccessible alarm refer to th...

Page 140: ...JOR BILLING 31 Port not Specified Billing 31 MAJOR BILLING 32 Element Management System Address not Specified Billing 32 MAJOR BILLING 33 File Transfer Protocol Secure File Transfer Protocol Parameters Invalid Billing 33 MAJOR BILLING 35 All Billing Links at Billing Server Down Billing 35 CRITICAL BILLING 37 Billing Link Failure Billing 37 MAJOR BILLING 38 Event Message Log File Access Error Billi...

Page 141: ...on from the UNIX prompt collect df k Billing Partition Disk Usage Major Threshold Exceeded Billing 3 The Billing Partition Disk Usage Major Threshold Exceeded alarm major indicates that a billing partition disk usage major threshold crossing has occurred The primary cause of the alarm is that call detail records are accumulating on the disk associated with the billing database in the EMS This is b...

Page 142: ...df k File Transfer Protocol Secure File Transfer Protocol Transfer Failed Billing 6 The File Transfer Protocol Secure File Transfer Protocol Transfer Failed alarm major indicates that the billing information FTP SFTP transfer has failed The primary cause of the alarm is that the BTS 10200 is unable to connect to remote host To correct the primary cause of the alarm verify the remote host is reacha...

Page 143: ...if 1 The FTP information has not been initialized 2 The information in the Softswitch database does not match the remote billing server a The login username for the billing server is missing or incorrect in the database b The username is correct but does not have the privilege level to write to the specified directory c The password for the remote billing server is missing or incorrect in the data...

Page 144: ...quest Prior to contacting Cisco TAC collect the following additional information From the EMS UNIX prompt ps ef nodestat From the CLI prompt status system File Write Error Billing 15 The File Write Error alarm major indicates that a file write error has occurred The primary cause of the alarm is that a system error has occurred The BTS 10200 system may be out of file descriptors To correct the pri...

Page 145: ...ssing addresses to the file File Transfer Protocol Secure File Transfer Protocol Parameters Invalid Billing 33 The File Transfer Protocol Secure File Transfer Protocol Parameters Invalid alarm major indicates that the FTP SFTP parameters configuration is not valid or has not been fully configured The primary cause of the alarm is that the billing acct addr table is not fully provisioned with the i...

Page 146: ... opening a service request Event Message Encode Failure Billing 40 The Event Message Encode Failure alarm minor indicates that an EM encode failure has occurred The primary cause of the alarm is that there is a problem with the format of the data being sent to the RKS To correct the primary cause of the alarm contact Cisco TAC Note Refer to the Obtaining Documentation and Submitting a Service Requ...

Page 147: ...d secondary RKS servers and attempt to bring them into service Verify that the radius profile table and call agent profile table are provisioned such that communication with the RKS servers is possible Record Keeping System Unreachable for Three Hours Billing 48 The Record Keeping System Unreachable for Three Hours alarm major indicates that the RKS servers have been unreachable for three hours Th...

Page 148: ...it will start generating new billing files Event Message Disk Space 50 Percent Full Billing 53 The Event Message Disk Space 50 Percent Full alarm minor indicates that the event message disk space is 50 percent full The primary cause of the alarm is that the event message storage has reached 50 of the maximum storage space allowed To correct the primary cause of the alarm move the event message fil...

Page 149: ...rvice state and contact Cisco TAC for assistance Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Signaling Prepaid Server Inaccessible Billing 58 The Signaling Prepaid Server Inaccessible alarm major indicates that the signaling prepaid server has become inaccessible The p...

Page 150: ...3 44 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Chapter 3 Billing Troubleshooting Troubleshooting Billing Alarms ...

Page 151: ... Call Processing Events Provides the information needed to monitor and correct the call processing events Troubleshooting Call Processing Alarms Provides the information needed to troubleshoot and correct the call processing alarms For additional call processing routing and translations information refer to the Cisco BTS 10200 Softswitch Routing Translations and Dial Plan Guide Note The following ...

Page 152: ...lick the call processing message number in Table 4 1 to display information about the event Table 4 1 Call Processing Events and Alarms by Severity Critical Major Minor Warning Info Not Used CALLP 12 CALLP 11 CALLP 8 CALLP 1 CALLP 2 CALLP 38 CALLP 16 CALLP 9 CALLP 3 CALLP 41 CALLP 17 CALLP 13 CALLP 4 CALLP 18 CALLP 14 CALLP 5 CALLP 19 CALLP 15 CALLP 6 CALLP 20 CALLP 42 CALLP 7 CALLP 21 CALLP 10 CA...

Page 153: ...larms CALLP 1 For additional information refer to the Test Report Call Processing 1 section on page 4 24 CALLP 2 CALLP 2 is not used CALLP 3 CALLP 3 is not used CALLP 4 CALLP 4 is not used CALLP 5 CALLP 5 is not used CALLP 6 CALLP 6 is not used CALLP 7 CALLP 7 is not used DESCRIPTION Test Report SEVERITY Information INFO THRESHOLD 10000 THROTTLE 0 ...

Page 154: ...o route available PRIMARY ACTION The data words in the event report indicate what parameters that need to be corrected Refer to the office records for the subscriber SECONDARY CAUSE Parameter s in the subscriber and or dial plan table are missing or incorrect for the dialed number SECONDARY ACTION Determine whether the routing parameters such as digit string were entered correctly in the subscribe...

Page 155: ...lled Party Number STRING 20 Carrier Code Dialed STRING 20 PRIMARY CAUSE No route is available for the interexchange carrier IXC dialed PRIMARY ACTION The data words in the event report indicate the parameters that need to be corrected Refer to the office records for the carrier SECONDARY CAUSE Parameter s in the carrier and or route grp table are missing or incorrect for the carrier SECONDARY ACTI...

Page 156: ...re Server Both Links Down Call Processing 12 section on page 4 35 DESCRIPTION Feature Server One Link Down SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Interface Name STRING 65 Interface IP Address STRING 65 PRIMARY CAUSE The hardware is broken PRIMARY ACTION Check the link interfaces DESCRIPTION Feature Server Both Links Down SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Interface Nam...

Page 157: ...ss Server Create Connection Error NAS CRCX Error SEVERITY INFO THRESHOLD 100 THROTTLE 0 PRIMARY CAUSE A pre authentication failure has occurred PRIMARY ACTION None DESCRIPTION Network Access Server Authentication Failure NAS Authentication Failure SEVERITY INFO THRESHOLD 100 THROTTLE 0 PRIMARY CAUSE The authentication authorization and accounting AAA server denied the request PRIMARY ACTION Check ...

Page 158: ...r to the Invalid Route Index Used Call Processing 17 section on page 4 25 DESCRIPTION Route Index has No Trunk Group Assigned SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Route Index FOUR_BYTES PRIMARY CAUSE A trunk group was not assigned to the given route PRIMARY ACTION Provision a trunk group for the associated route index DESCRIPTION Invalid Route Index Used SEVERITY WARNING THRESHOLD 1...

Page 159: ...ll Routed to Unprovisioned Subscriber Call Processing 19 section on page 4 26 DESCRIPTION Unable to Play Announcement SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Announcement Index FOUR_BYTES PRIMARY CAUSE An announcement was not provisioned correctly PRIMARY ACTION Provision the announcement DESCRIPTION Call Routed to Unprovisioned Subscriber SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATA...

Page 160: ...page 4 26 DESCRIPTION No Route or Trunk Group Available to Route Call SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Calling Number STRING 20 Called Number STRING 20 Route Index FOUR_BYTES Trunk Group ID FOUR_BYTES PRIMARY CAUSE A Trunk Group was not provisioned correctly in the route PRIMARY ACTION Verify the route and trunk group provisioning DESCRIPTION Call Released Due to Maximum Hop Cou...

Page 161: ...t a Subscriber Call Processing 23 section on page 4 26 DESCRIPTION Trunk Group Index Read Failure SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Trunk Group Index FOUR_BYTES Call Index FOUR_BYTES PRIMARY CAUSE The Trunk Group Index could not be retrieved from call data PRIMARY ACTION Check the provisioning DESCRIPTION Routing Error Termination is Not a Subscriber SEVERITY WARNING THRESHOLD 10...

Page 162: ...scriber Routing Call Processing 25 section on page 4 27 DESCRIPTION Invalid Route for Subscriber Index SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Route STRING 20 Subscriber Index FOUR_BYTES PRIMARY CAUSE The route is not provisioned correctly for the specified subscriber PRIMARY ACTION The check provisioning DESCRIPTION Invalid Route Group for Subscriber Routing SEVERITY WARNING THRESHOLD...

Page 163: ...r Status Call Processing 27 section on page 4 27 DESCRIPTION Invalid Trunk Group for Subscriber Routing SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Trunk Group Index FOUR_BYTES Subscriber Index FOUR_BYTES PRIMARY CAUSE The trunk group is not provisioned correctly for the specified subscriber PRIMARY ACTION Check the provisioning DESCRIPTION Unable to Route Blocked by Destination Subscriber...

Page 164: ...event refer to the Routing Selection Failure Call Processing 29 section on page 4 27 DESCRIPTION Route Name Does Not Exist SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Route Name STRING 40 PRIMARY CAUSE The route is not correctly provisioned PRIMARY ACTION Check the provisioning DESCRIPTION Routing Selection Failure SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Route Index FOUR_BYTES ...

Page 165: ... page 4 28 Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Customer Originated Trace Test Failed COT Test Failed SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Termination Index FOUR_BYTES PRIMARY CAUSE A customer originated trace COT test has failed PRIMA...

Page 166: ...alid Call Call Processing 33 section on page 4 28 DESCRIPTION Country Code Dialing Plan Error SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Called Number STRING 20 Dial Plan Index FOUR_BYTES PRIMARY CAUSE The country code was not found in the dial plan PRIMARY ACTION Check the provisioning DESCRIPTION Invalid Call SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Calling Number STRING 20 C...

Page 167: ...SCRIPTION Dial Plan Information Not Found for Digits Received SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Calling Number STRING 20 Called Number STRING 20 Dial Plan Index FOUR_BYTES PRIMARY CAUSE The call could not be completed because the number entered could not be located in the dial plan PRIMARY ACTION Check the provisioning and the number dialed DESCRIPTION Dial Plan Information for T...

Page 168: ...ce request Prior to contacting Cisco TAC collect a billing record for the failed call using the following report billing record orignumber string DESCRIPTION Invalid or Unknown Nature of Address Invalid or Unknown NOA SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS NOA Received FOUR_BYTES Calling Number STRING 20 Called Number STRING 20 PRIMARY CAUSE The nature of address NOA was incorrect in ...

Page 169: ... 4 29 DESCRIPTION Release Cause 25 Exchange Routing Error Received SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS CIC FOUR_BYTES TGN ID FOUR_BYTES DPC STRING 64 OPC STRING 64 PRIMARY CAUSE Received release REL with cause number 25 PRIMARY ACTION Log and map the cause to number 31 DESCRIPTION Test Call Blocked Due to Congestion or Isolation SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS CIC...

Page 170: ...DATAWORDS Route Guide ID STRING 17 Trunk Group Index FOUR_BYTES PRIMARY CAUSE The interactive voice response IVR server is not ready or the connection failed PRIMARY ACTION Check IVR server The related route guide ID and or trunk group index are provided if known at the time the event report is issued DESCRIPTION INVITE Message From Unauthorized Call Agent INVITE Message From Unauthorized CA SEVER...

Page 171: ...er the given DN has been provisioned correctly in the reporting BTS If there is an error in the LNP database SCP or other switch then notify the appropriate administrators Note that porting of the subscriber and DN from the donor switch to the recipient switch and the associated updates of the Location Routing Number LRN in all the SCP databases may not all occur at exactly the same time Therefore...

Page 172: ...NG THRESHOLD 100 THROTTLE 0 DATAWORDS Calling Party Number STRING 64 Called Party Number STRING 64 Call ID FOUR_BYTES Inbound Context String STRING 64 PRIMARY CAUSE An invalid service_ref in the SIP INVITE message coming from the AS has occurred PRIMARY ACTION Report the BTS 10200 and the AS with the received service_ref string to Cisco TAC Contact Cisco TAC DESCRIPTION Context in Call from Applic...

Page 173: ...ing 18 WARNING CALLP 19 Call Routed to Unprovisioned Subscriber Call Processing 19 WARNING CALLP 20 No Route or Trunk Group Available to Route Call Call Processing 20 WARNING CALLP 21 Call Released Due to Maximum Hop Counts Exceeded Call Processing 21 WARNING CALLP 22 Trunk Group Index Read Failure Call Processing 22 WARNING CALLP 23 Routing Error Termination is Not a Subscriber Call Processing 23...

Page 174: ...o Route Available for Carrier Dialed Call Processing 9 The No Route Available for Carrier Dialed event functions as a warning that is no route available for the dialed carrier The primary cause for the event is that no route is available for the IXC dialed The Orig Type one byte Orig Sub or TG ID eight bytes calling party number 20 called party number 20 and carrier code dialed 20 data words in th...

Page 175: ...Failure event functions as an informational alert that a NAS authentication failure has occurred The primary cause of the event is the AAA server denied the request Check the calling and called numbers Cable Modem Termination System Easily Recognizable Identification Not Found in Media Gateway Table Call Processing 15 The Cable Modem Termination System Easily Recognizable Identification Not Found ...

Page 176: ... trunk group in the route was not provisioned correctly To correct the primary cause of the event verify the route and trunk group provisioning Call Released Due to Maximum Hop Counts Exceeded Call Processing 21 The Call Released Due to Maximum Hop Counts Exceeded event functions as a warning that the call was released due to the maximum hop counts being exceeded The primary cause of the event is ...

Page 177: ...alid trunk group for the subscriber routing was selected The primary cause of the event is that the trunk group is not provisioned correctly for the specified subscriber To correct the primary cause of the event check and correct the BTS 10200 trunk group provisioning Unable to Route Blocked by Destination Subscriber Status Call Processing 27 The Unable to Route Blocked by Destination Subscriber S...

Page 178: ...d instructions on contacting Cisco TAC and opening a service request Country Code Dialing Plan Error Call Processing 32 The Country Code Dialing Plan Error event functions as a warning that a country code dialing plan error occurred The primary cause of the event is that the country code was not found in the dial plan To correct the primary cause of the event check and correct the BTS 10200 dial p...

Page 179: ...e of the event check the data words type of call four bytes calling number 20 called number 20 and failure indication 20 Once the data words are checked contact Cisco TAC to resolve the failure indicated Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Prior to contacting C...

Page 180: ...cal Number Portability Query with Location Routing Number of this BTS 10200 and the Directory Number event serves as an information alert that the provisioning of ported in subscriber has not been completed To correct the primary cause of the event verify whether the given DN has been provisioned correctly in the reporting BTS 10200 If there is an error in the LNP database SCP or other switch then...

Page 181: ...uest section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Context in call from Application Server not Found Call Processing 45 The Context in call from Application Server not Found event serves as a warning that an BTS 10200 has received an INVITE from an application server which contains a context ID in the route header which is invalid It may be tha...

Page 182: ...4 3 lists all of the call processing alarms in numerical order and provides cross reference to each subsection in this section Table 4 3 BTS 10200 Call Processing Alarms Alarm Type Alarm Name Alarm Severity CALLP 11 Feature Server One Link Down Call Processing 11 MINOR CALLP 12 Feature Server Both Links Down Call Processing 12 CRITICAL CALLP 38 Release Cause 25 Exchange Routing Error Received Call...

Page 183: ...of the methods below If the kstat command in Example 1 does not provide an output try the ndd commands in example 2 Example 1 mssol ca0 a kstat hme 0 hme0 link_up module hme instance 0 name hme0 class net link_up 1 mssol ca0 a kstat qfe 0 qfe0 link module qfe instance 0 name qfe0 class net link_duplex 2 link_up 1 mssol ca0 a kstat qfe 0 qfe0 ifspe module qfe instance 0 name qfe0 class net ifspeed ...

Page 184: ...s Oerrs should be close to 0 High input errors could indicate that the network is saturated host overload or physical network problem High output errors could indicate a saturated local network or a bad physical connection Step 8 Check statistics for ethernet interface s on the switch side Look for abnormal input output errors cyclic redundancy check CRC frame errors For a description of the outpu...

Page 185: ...g one of the methods below If the kstat command in Example 1 does not provide an output try the ndd commands in example 2 Example 1 mssol ca0 a kstat hme 0 hme0 link_up module hme instance 0 name hme0 class net link_up 1 mssol ca0 a kstat qfe 0 qfe0 link module qfe instance 0 name qfe0 class net link_duplex 2 link_up 1 mssol ca0 a kstat qfe 0 qfe0 ifspe module qfe instance 0 name qfe0 class net if...

Page 186: ...es to the Cisco output interpreter for further analysis of the interfaces https www cisco com cgi bin Support OutputInterpreter home pl Step 10 Check the physical cable the cable connectors and the cable connections Release Cause 25 Exchange Routing Error Received Call Processing 38 The Release Cause 25 Exchange Routing Error Received alarm minor indicates that a release with cause number 25 occur...

Page 187: ...n needed to monitor and correct the configuration events Troubleshooting Configuration Alarms Provides the information needed to troubleshoot and correct the configuration alarms Configuration Events and Alarms This section provides a brief overview of the configuration events and alarms for the Cisco BTS 10200 Softswitch in numerical order Table 5 1 lists all of the configuration events and alarm...

Page 188: ...Y Information INFO THRESHOLD 10000 THROTTLE 0 DESCRIPTION Signaling Media Gateway Adapter Wrongly Configured Domain Name Signaling MGA Wrongly Configured Domain Name SEVERITY INFO THRESHOLD 1 THROTTLE 1 DATAWORDS Configured Domain Na STRING 256 Cause STRING 128 PRIMARY CAUSE The domain name is invalid PRIMARY ACTION Check the domain name system DNS server and correct the domain name SECONDARY CAUS...

Page 189: ... KAM Mate Configuration Error SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Reason STRING 80 PRIMARY CAUSE Mate and local are configured to be of same side PRIMARY ACTION The configure side of the platform coming up to opposite of its mate SECONDARY CAUSE The mate is configured with wrong mate red DNS name SECONDARY ACTION Configure the mate DNS name properly for mate TERNARY CAUSE Mate red an...

Page 190: ...RESHOLD 100 THROTTLE 0 DATAWORDS Command Line DN STRING 128 Feature Server DN STRING 128 Platform Name STRING 32 PRIMARY CAUSE The feature server table is mis configured PRIMARY ACTION Reconfigure the feature server table to match command line host and port DESCRIPTION FIMXML Parse Error SEVERITY WARNING THRESHOLD 100 THROTTLE 0 PRIMARY CAUSE The BTS 10200 software is released with a file named FI...

Page 191: ...AUSE The external application server has returned a SIP response code that indicates a subscriber is not provisioned on the AS PRIMARY ACTION Provision the application server to handle all subscribers which are provisioned on the BTS 10200 to use the applications on that server DESCRIPTION BTS 10200 Provisioning for External Applications is not Complete SEVERITY WARNING THRESHOLD 100 THROTTLE 0 PR...

Page 192: ...rect domain name The secondary cause of the event is that the at least half of the local machine address does not match the MGCP domain name To correct secondary cause of the event check the DNS server for the domain name to ensure that the IP address is correct Mate Configuration Error Configuration 3 The Mate Configuration Error alarm major indicates that the mate configuration is incorrect To t...

Page 193: ...alization It defines how to handle certain features provided on an external application server The file may be missing or the file has been incorrectly modified To correct the primary cause of the FIMXML Parse Error event install the FIMXMLRules xml file in the appropriate directory To correct the secondary cause of the event install valid FIMXMLRules xml file Application Server Provisioning Error...

Page 194: ...the alarm properly configure the mate DNS name Configuration Error Configuration 4 The Configuration Error alarm critical indicates that a critical configuration error has occurred The primary cause of the alarm is that there is incorrect configuration information in the platform cfg file To correct the primary cause of the alarm correct the appropriate configuration parameters in the platform cfg...

Page 195: ...d to monitor and troubleshoot database events and alarms This chapter is divided into the following sections Database Events and Alarms Provides a brief overview of each database event and alarm Monitoring Database Events Provides the information needed to monitor and correct the database events Troubleshooting Database Alarms Provides the information needed to troubleshoot and correct the databas...

Page 196: ...play information about the event or alarm DATABASE 1 For additional information refer to the Test Report Database 1 section on page 6 22 Table 6 1 Database Events and Alarms by Severity Critical Major Minor Warning Info Not Used DATABASE 3 DATABASE 6 DATABASE 7 DATABASE 2 DATABASE 1 DATABASE 4 DATABASE 8 DATABASE 14 DATABASE 11 DATABASE 19 DATABASE 5 DATABASE 10 DATABASE 21 DATABASE 18 DATABASE 9 ...

Page 197: ...ve Database Out of Sync SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Error Code TWO_BYTES Error String STRING 20 Provisioning String STRING 80 PRIMARY CAUSE The master database under Oracle control in the Element Management System EMS was successfully updated but the subsequent update of the shared memory tables in the Call Agents CAs and or Feature Servers FSs were not properly updated PRI...

Page 198: ...rs nodestat dbadm r get_deferror dbadm r get_deferr dbadm r get_deftrandest dbadm r get_defcall_order Note Do not perform an EM01 switchover until the deferrors are removed DESCRIPTION There Are Errors in Element Management System Database DefError Queue Contact Database Administrator There are Errors in EMS Database DefError Queue Contact DBA SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS H...

Page 199: ... PUSH Job Broken EMS DB_Heart_Beat Replication PUSH Job Broken SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name STRING 30 Local Database STRING 10 Remote Database STRING 10 Job STRING 5 Time Stamp STRING 20 PRIMARY CAUSE The remote database is not accessible PRIMARY ACTION Re start up or restore the remote database SECONDARY CAUSE The remote database is down SECONDARY ACTION Restart ...

Page 200: ... Stamp STRING 20 PRIMARY CAUSE The DBHeartBeat process has been terminated by the system manager program SMG or stopped by the platform PRIMARY ACTION Restart the DBHeartBeat process by entering the dbinit H i start command as an Oracle user or by entering the platform start command as a root user DESCRIPTION Element Management System Database Replication DefTranDest Queue Overloaded EMS Database ...

Page 201: ... Database DefTran Queue is Overloaded EMS Database DefTran Queue is Overloaded SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name STRING 30 Database Name STRING 10 Threshold TWO_BYTES Time Stamp STRING 20 PRIMARY CAUSE The replication DefTranDest queue is overloaded PRIMARY ACTION Resume the replication activities SECONDARY CAUSE There are too many errors in the DefError queue SECONDARY A...

Page 202: ...ablespace is Out of Free Space SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name STRING 30 Database Name STRING 10 Tablespace Name STRING 30 Total Free Space TWO_BYTES Time Stamp STRING 20 PRIMARY CAUSE An increase data volume or transactions has occurred PRIMARY ACTION Add more space to the tablespace DESCRIPTION Urgent Element Management System Database Archive Log Directory is Getting...

Page 203: ...he Element Management System Database Backup Fails Database 10 section on page 6 31 DESCRIPTION Element Management System Database Backup Fails EMS Database Backup Fails SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name STRING 30 Database Name STRING 10 Message 1 STRING 200 Message 2 STRING 200 Time Stamp STRING 20 PRIMARY CAUSE The system or hardware are unstable PRIMARY ACTION Re start...

Page 204: ... 1 STRING 200 Message 2 STRING 200 Time Stamp STRING 20 PRIMARY CAUSE The probable cause of the ORA errors report in alert log file is documented in the ORACLE_HOME rdbms mesg oraus msg file Login to EMS system as oracle user or su oracle to view this file If more information is needed contact Cisco TAC for database support or query Oracle metalink library at http metalink oracle com PRIMARY ACTIO...

Page 205: ...instructions on contacting Cisco TAC and opening a service request DESCRIPTION Element Management System Database Process Died EMS Database Process Died SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name STRING 30 Database Name STRING 10 Error Source STRING 40 Message STRING 200 Time Stamp STRING 20 PRIMARY CAUSE Possible Error Source 1 Process Name if the local process is not running ...

Page 206: ...ervice Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Element Management System Database Performance Alert EMS Database Performance Alert SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name STRING 30 Database Name STRING 10 Stat Event Name STRING 80 Value 1 STRING 50 Value 2 FOUR_BYTES Message STRING 200 Time Stamp ST...

Page 207: ...db usage table_name string DATABASE 15 To troubleshoot and correct the cause of the alarm refer to the Table Size Exceeds Major Threshold Limit Database 15 section on page 6 33 DESCRIPTION Table Size Exceeds Minor Threshold Limit SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Table Name STRING 32 PRIMARY CAUSE The pre provisioned size for the stated table is nearing the licensed limit on the nu...

Page 208: ...e alarm refer to the Data Replication Failed Database 17 section on page 6 33 DESCRIPTION Table Size Exceeds Critical Threshold Limit SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Table Name STRING 32 PRIMARY CAUSE The critical threshold limit has been exceeded PRIMARY ACTION N A DESCRIPTION Data Replication Failed SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Replication Stage STRING 4...

Page 209: ... DESCRIPTION Unexpected Runtime Data Interaction SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Internal External In STRING 10 Table Name STRING 32 Table Entry STRING 10 Table Field Name STRING 32 Descriptive Data 1 STRING 64 Descriptive Data 2 STRING 64 Descriptive Data 3 STRING 64 Descriptive Data 4 STRING 64 PRIMARY CAUSE An unexpected data interaction has been detected at the runtime in t...

Page 210: ...lication module software problem has occurred PRIMARY ACTION The database restore procedure needs to be executed on the side of the system which goes active after the switchover Any alarms should be cleared manually after the recovery action is taken DESCRIPTION Database Statistics Collection Exception DB Statistics Collection Exception SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name S...

Page 211: ...uage and update the LANGUAGE table DESCRIPTION Element Management System Oracle Database Minor Error EMS Oracle Database ORA Minor Error SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS HostName STRING 30 DatabaseName STRING 10 Message1 STRING 200 Message2 STRING 200 TimeStamp STRING 20 PRIMARY CAUSE This ORA error is issued by an Oracle background process The probable cause of the ORA errors is ...

Page 212: ...NG 10 Message1 STRING 200 Message2 STRING 200 TimeStamp STRING 20 PRIMARY CAUSE This ORA error is issued by an Oracle background process The probable cause of the ORA errors is documented in the ORACLE_HOME rdbms mesg oraus msg file To view the file login to EMS system as root then su oracle If more information is needed contact Cisco TAC for database support or query Oracle Metalink library at ht...

Page 213: ...HRESHOLD 100 THROTTLE 0 DATAWORDS FileName STRING 128 Error STRING 50 PRIMARY CAUSE Unable to connect between active and standby call agents PRIMARY ACTION Verify communication between primary and CA On each CA ping the other node SECONDARY CAUSE Unable to login to remote host SECONDARY ACTION Verify that secure shell SSH keys have been pre configured for user root on both active and standby call ...

Page 214: ...o the Failure Setting the Index Table Soft Limit Database 27 section on page 37 DESCRIPTION Failure Setting the Index Table Soft Limit Failure Setting the IDX Table Soft Limit SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Reason STRING 200 PRIMARY CAUSE A corruption of the IDX framework for the table has occurred PRIMARY ACTION Running the tiat command will verify if there is corruption Fix th...

Page 215: ...ASE 9 Urgent Element Management System Database Archive Log Directory is Getting Full Database 9 CRITICAL DATABASE 10 Element Management System Database Backup Fails Database 10 MAJOR DATABASE 11 Element Management System Database Alert log Alerts Database 11 MAJOR DATABASE 12 Element Management System Database Process Died Database 12 CRITICAL DATABASE 13 Element Management System Database Perfor...

Page 216: ...anagement System Database DefError Queue Contact Database Administrator alarm critical indicates that there are errors in the EMS database DefError queue To troubleshoot and correct the cause of the There are Errors in Element Management System Database DefError Queue Contact Database Administrator alarm refer to the There Are Errors in Element Management System Database DefError Queue Contact Dat...

Page 217: ...abase Tablespace is Out of Free Space Database 8 The Element Management System Database Tablespace is Out of Free Space alarm major indicates that the EMS database table space is out of free space To troubleshoot and correct the cause of the Element Management System Database Tablespace is Out of Free Space alarm refer to the Element Management System Database Tablespace is Out of Free Space Datab...

Page 218: ... To troubleshoot and correct the cause of the Element Management System Database Performance Alert alarm refer to the Element Management System Database Performance Alert Database 13 section on page 6 32 Table Size Exceeds Minor Threshold Limit Database 14 The Table Size Exceeds Minor Threshold Limit alarm minor indicates that the table size has exceeded the minor threshold crossing limit To troub...

Page 219: ...ed Successfully event functions as an informational alert that the daily database backup has completed successfully The event is informational and no further action is required Replication Data Flush Timeout During Switchover Database 20 The Replication Data Flush Timeout During Switchover alarm major indicates that the replication data flush timed out during a switchover To troubleshoot and corre...

Page 220: ...m refer to the Element Management System Oracle Database Major Error Database 24 section on page 6 37 Secure File Transfer Protocol Transfer Failed Database 25 The Secure File Transfer Protocol Transfer Failed alarm alarm indicates that a SFTP file transfer had failed To troubleshoot and correct the cause of the Secure File Transfer Protocol Transfer Failed alarm refer to the Secure File Transfer ...

Page 221: ...R DATABASE 9 Urgent Element Management System Database Archive Log Directory is Getting Full Database 9 CRITICAL DATABASE 10 Element Management System Database Backup Fails Database 10 MAJOR DATABASE 11 Element Management System Database Alert log Alerts Database 11 MAJOR DATABASE 12 Element Management System Database Process Died Database 12 CRITICAL DATABASE 13 Element Management System Database...

Page 222: ...dditional causes of the alarm are that a request for update or delete on non existing data occurred or an unique constraint primary key was violated Correcting the cause of the alarm may require a manual update on database tables Contact Cisco TAC for assistance Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Ci...

Page 223: ...al troubleshooting information execute the following On one EMS server su oracle dbadm C rep On both EMS servers nodestat dbadm r get_deferror dbadm r get_deferr dbadm r get_deftrandest dbadm r get_defcall_order Note Do not perform an EM01 switchover until the deferrors are removed Element Management System Database HeartBeat Process Died Database 5 The Element Management System Database HeartBeat...

Page 224: ...n both EMS servers nodestat dbadm r get_deferror dbadm r get_deferr dbadm r get_deftrandest dbadm r get_defcall_order Note Do not perform an EM01 switchover until the deferrors are removed Element Management System Database DefTran Queue is Overloaded Database 7 The Element Management System Database DefTran Queue is Overloaded alarm minor indicates that the EMS database DefTran queue is overloade...

Page 225: ...deferr dbadm r get_deftrandest dbadm r get_defcall_order Note Do not perform an EM01 switchover until the deferrors are removed Urgent Element Management System Database Archive Log Directory is Getting Full Database 9 The Urgent Element Management System Database Archive Log Directory is Getting Full alarm critical indicates that the EMS database archive log directory is getting full The primary ...

Page 226: ...r should manually clear this alarm Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Element Management System Database Process Died Database 12 The Element Management System Database Process Died alarm critical indicates that the EMS database process has died The primary po...

Page 227: ...imit The primary cause of the Table Size Exceeds Major Threshold Limit alarm is the major threshold crossing limit has been exceeded No further action is required Table Size Exceeds Critical Threshold Limit Database 16 The Table Size Exceeds Critical Threshold Limit alarm critical indicates that the table size has exceeded the critical threshold crossing limit The primary cause of the alarm is tha...

Page 228: ... Database Statistics Collection Exception alarm minor indicates that the database statistics collection process had an exception To identify the primary cause of the alarm check the information listed in the Exception dataword field The correction action varies and is determined by the type of exception For more information about the ORA xxxxx errors execute an oerr ora xxxxx command as an Oracle ...

Page 229: ...EARING does not return check both the service and subscriber_service_profile tables btsadmin show service id mlstest ID mlstest FNAME1 MLS btsadmin show subscriber_service_profile sub id 2212437211 SUB_ID 2212437211 SERVICE_ID mlstest If you hear a reorder tone from a SIP phone ensure the status of the BTS announcement table is all of the following btsadmin status tt tgn id 889 cic all 889 1 ADMIN...

Page 230: ..._ eng_ and spa_ Recommended Action Change the filename length to less than 40 characters Element Management System Oracle Database Minor Error Database 23 The Element Management System Oracle Database Minor Error alarm minor indicates that a minor error has occurred in an Oracle background process The probable cause of the ORA errors report in alert log file is documented in the ORACLE_HOME rdbms ...

Page 231: ...fer has failed The primary cause of the alarm is that the SFTP was unable to connect between active and standby call agents To troubleshoot and correct the primary cause of the alarm verify communication between primary and secondary call agent CA On each CA ping the other node The secondary cause of the alarm is that the system was unable to login to the remote host To troubleshoot and correct th...

Page 232: ...6 38 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Chapter 6 Database Troubleshooting Troubleshooting Database Alarms ...

Page 233: ...tor and troubleshoot maintenance events and alarms This chapter is divided into the following sections Maintenance Events and Alarms Provides a brief overview of each maintenance event and alarm Monitoring Maintenance Events Provides the information needed to monitor and correct the maintenance events Troubleshooting Maintenance Alarms Provides the information needed to troubleshoot and correct th...

Page 234: ...CE 14 MAINTENANCE 57 MAINTENANCE 9 MAINTENANCE 90 MAINTENANCE 123 MAINTENANCE 15 MAINTENANCE 61 MAINTENANCE 10 MAINTENANCE 98 MAINTENANCE 16 MAINTENANCE 65 MAINTENANCE 19 MAINTENANCE 17 MAINTENANCE 69 MAINTENANCE 20 MAINTENANCE 22 MAINTENANCE 70 MAINTENANCE 21 MAINTENANCE 25 MAINTENANCE 73 MAINTENANCE 23 MAINTENANCE 27 MAINTENANCE 74 MAINTENANCE 26 MAINTENANCE 28 MAINTENANCE 82 MAINTENANCE 42 MAIN...

Page 235: ...AINTENANCE 122 Table 7 1 Maintenance Events and Alarms by Severity continued Critical Major Minor Warning Info Not Used DESCRIPTION Test Report SEVERITY Information INFO THRESHOLD 10000 THROTTLE 0 DESCRIPTION Report Threshold Exceeded SEVERITY INFO THRESHOLD 0 THROTTLE 0 DATAWORDS Report Type TWO_BYTES Report Number TWO_BYTES Threshold Level TWO_BYTES PRIMARY CAUSE Issued when the threshold for a ...

Page 236: ...DS Local State STRING 30 Mate State STRING 30 Reason STRING 80 Probable Cause STRING 80 PRIMARY CAUSE Can result from maintenance reports 5 6 9 10 19 or 20 PRIMARY ACTION Review the information from the command line interface CLI log report Usually a software problem restart the software using the installation and startup procedure SECONDARY CAUSE Manually shutting down the system using platform s...

Page 237: ... going to the faulty state PRIMARY ACTION Display the event summary on the faulty mate side using the report event summary command see the CLI Guide for command details SECONDARY ACTION Review the information in the event summary This is usually a software problem TERNARY ACTION After confirming the active side is processing traffic restart software on the mate side Log in to the mate platform as ...

Page 238: ... when changing from an active processor to a standby and the changeover fails PRIMARY ACTION Review information from CLI log report SECONDARY CAUSE This alarm is usually caused by a software problem on the specific platform identified in the alarm report SECONDARY ACTION Restart the platform identified in the alarm report TERNARY ACTION If platform restart is not successful reinstall the applicati...

Page 239: ...ired time period Soon after this event is issued one platform will go to the faulty state PRIMARY ACTION Review the information from CLI log report SECONDARY CAUSE This alarm is usually caused by a software problem on the specific platform identified in the alarm report SECONDARY ACTION Restart the platform identified in the alarm report TERNARY ACTION If platform restart is not successful reinsta...

Page 240: ...t CA or Feature Server FS SECONDARY CAUSE The mate detects itself faulty during changeover and then rejects changeover Note This attempted changeover could be caused by a forced operator switch or could be caused by secondary instance rejecting changeover as primary is being brought up SECONDARY ACTION If the mate is faulty not running then perform the corrective action steps listed for the MAINTE...

Page 241: ...ific mate platform identified in the alarm report TERNARY ACTION Restart the mate platform identified in the alarm report SUBSEQUENT ACTION If mate platform restart is not successful reinstall the application for this mate platform and then restart the mate platform again If necessary reboot the host machine this mate platform is located on Then reinstall and restart all applications on that machi...

Page 242: ... Local State STRING 30 Mate State STRING 30 PRIMARY CAUSE The local initialization has timed out PRIMARY ACTION Check that the binary files are present for the unit Call Agent Feature Server or Element Manager SECONDARY CAUSE When the event report is issued the system has failed and the re initialization process has failed SECONDARY ACTION If the files are not present then re install the files fro...

Page 243: ...THROTTLE 0 DATAWORDS Local State STRING 30 Mate State STRING 30 PRIMARY CAUSE The local initialization has been successfully completed PRIMARY ACTION This an informational event report and no further action is required DESCRIPTION Administrative State Change Admin State Change SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Facility Type STRING 40 Facility ID STRING 40 Initial Admin State STRING ...

Page 244: ... THRESHOLD 100 THROTTLE 0 DATAWORDS Call Agent ID STRING 40 Current Local State STRING 40 Current Mate State STRING 20 PRIMARY CAUSE Indicates that the call agent has changed operational state as a result of a manual switchover control command in CLI PRIMARY ACTION No action is required DESCRIPTION Feature Server Administrative State Change SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Feature ...

Page 245: ...g Process SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 40 Restart Type STRING 40 Restart Mode STRING 32 Process Group ONE_BYTE PRIMARY CAUSE A process is being started as the system is being brought up PRIMARY ACTION No action is required DESCRIPTION Invalid Event Report Received SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Report Type TWO_BYTES Report Number TWO_BYTES ...

Page 246: ...ocess Group FOUR_BYTES PRIMARY CAUSE This alarm is caused by a software problem PRIMARY ACTION If problem persists contact Cisco TAC technical support Contact Cisco TAC DESCRIPTION Process Manager Process Exceeded Restart Rate PMG Process Exceeded Restart Rate SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 40 Restart Rate FOUR_BYTES Process Group ONE_BYTE PRIMARY CAUSE This ...

Page 247: ...whether or not the network interface is down If it is down restore network interface and restart the software SECONDARY CAUSE The alarm can be caused by a router problem SECONDARY ACTION If the alarm is caused by a router problem repair the router and reinstall TERNARY CAUSE Soon after this event is issued one platform may go to the faulty state DESCRIPTION Keep Alive Module Network Interface Down...

Page 248: ...ection on page 7 98 DESCRIPTION Keep Alive Module Mate is Alive KAM Mate is Alive SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Local State STRING 30 Mate State STRING 30 DESCRIPTION Process Manager Process Failed to Complete Initialization PMG Process Failed to Complete Initialization SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 40 Process Group ONE_BYTE PRIMARY CAUSE ...

Page 249: ...AC and opening a service request MAINTENANCE 25 For additional information refer to the Process Manager Changing State Maintenance 25 section on page 7 73 DESCRIPTION Process Manager Restarting Process PMG Restarting Process SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 40 Restart Type STRING 40 Restart Mode STRING 32 Process Group ONE_BYTE PRIMARY CAUSE The software proces...

Page 250: ...on page 7 74 DESCRIPTION Process Manager Going Faulty PMG Going Faulty SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Reason STRING 40 PRIMARY CAUSE The system has been brought down because the system has detected a fault PRIMARY ACTION If it is not due to the operator intentionally bringing down the system then the platform has detected a fault and has shut down This is typically followed by a...

Page 251: ...trative state of a device has failed PRIMARY ACTION Monitor the system to see if any event reports indicate a database update failure SECONDARY ACTION If one is found analyze the cause of the failure Verify that the controlling element of the targeted device was in the ACTIVE state in order to service the request to change the ADMIN state of the device TERNARY ACTION If the controlling platform in...

Page 252: ...HROTTLE 0 DATAWORDS Process Name STRING 40 Process Group ONE_BYTE PRIMARY CAUSE Process is being notified to switch to the active state as the system is switching over from the standby state to the active state PRIMARY ACTION No action is necessary DESCRIPTION Process Manager Sending Go Standby to Process PMG Sending Go Standby to Process SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Process Na...

Page 253: ...ss SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 40 Process Group ONE_BYTE PRIMARY CAUSE The process is being notified to exit gracefully as the system is switching over to the standby state or it is shutting down The switchover or shutdown could be due to either of the following 1 The operator is taking the action to switch or shut down the system 2 The system has detected ...

Page 254: ...it Complete to Process SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 40 Process Group ONE_BYTE PRIMARY CAUSE The system is being brought up and all processes are being notified to start executing PRIMARY ACTION No action is necessary DESCRIPTION Process Manager Killing Process PMG Killing Process SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 40 Process...

Page 255: ...he Database PMG Clearing the Database SEVERITY INFO THRESHOLD 100 THROTTLE 0 PRIMARY CAUSE The system is preparing to copy data from the mate The system has been brought up and the mate side is running PRIMARY ACTION No action is necessary DESCRIPTION Process Manager Cleared the Database PMG Cleared the Database SEVERITY INFO THRESHOLD 100 THROTTLE 0 PRIMARY CAUSE The system is prepared to copy da...

Page 256: ...ful with Warning Admin State Change Successful with Warning SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Facility Type STRING 40 Facility Instance STRING 40 Initial State STRING 20 Target State STRING 20 Current State STRING 20 Warning Reason STRING 40 PRIMARY CAUSE The device was in a flux state PRIMARY ACTION Retry the command DESCRIPTION Keep Alive Module Number of Heartbeat Messages Rec...

Page 257: ...entation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Process Manager Process Failed to Come Up in Active Mode PMG Process Failed to Come Up in Active Mode SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 40 Process Group ONE_BYTE PRIMARY CAUSE The alarm is caused by a softwa...

Page 258: ... refer to the Network Interface Restored Maintenance 46 section on page 7 76 DESCRIPTION Application Instance State Change Failure SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Application Instance STRING 20 Failure Reason STRING 80 PRIMARY CAUSE The switchover of an application instance failed because of a platform fault PRIMARY ACTION Retry the switchover and if the condition continues call ...

Page 259: ...ng a service request DESCRIPTION Thread Watchdog Counter Expired for a Thread SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 5 Thread Type FOUR_BYTES Thread Instance FOUR_BYTES PRIMARY CAUSE The alarm is caused by a software error PRIMARY ACTION No action is required The system will automatically recover or shutdown DESCRIPTION Index Table Usage Exceeded Minor Usage Thres...

Page 260: ... section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Index Table Usage Exceeded Major Usage Threshold Level IDX Table Usage Exceeded Major Usage Threshold Level SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Table Name STRING 80 Table Size FOUR_BYTES Used FOUR_BYTES PRIMARY CAUSE Call traffic volume is above the design limits PRIMARY A...

Page 261: ...Usage Exceeded Critical Usage Threshold Level SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Table Name STRING 80 Table Size FOUR_BYTES Used FOUR_BYTES PRIMARY CAUSE Call traffic volume is above the design limits PRIMARY ACTION Verify that the traffic is within rated capacity SECONDARY CAUSE A software problem requiring manufacture analysis has occurred SECONDARY ACTION Contact customer supp...

Page 262: ...t section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Central Processing Unit Usage is Now Below the 50 Level CPU Usage is Now Below the 50 Level SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name STRING 40 PID STRING 40 Process Name STRING 40 CPU Usage STRING 40 PRIMARY CAUSE No probable cause is necessary PRIMARY ACTION No corre...

Page 263: ...l Processing Unit has Returned to Normal Levels of Operation The CPU has Returned to Normal Levels of Operation SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name STRING 40 CPU Usage STRING 40 PRIMARY CAUSE N A PRIMARY ACTION N A DESCRIPTION The Five Minute Load Average is Abnormally High SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name STRING 40 Load Average STRING 40 PRIMARY C...

Page 264: ...C and opening a service request DESCRIPTION The Load Average has Returned to Normal Levels SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name STRING 40 Load Average STRING 40 PRIMARY CAUSE N A PRIMARY ACTION N A DESCRIPTION Memory and Swap are Consumed at Critical Levels SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name STRING 40 Memory STRING 40 Swap STRING 40 PRIMARY CAUSE A...

Page 265: ...quest MAINTENANCE 59 MAINTENANCE 59 is not used MAINTENANCE 60 MAINTENANCE 60 is not used DESCRIPTION Memory and Swap are Consumed at Abnormal Levels SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name STRING 40 Memory STRING 40 Swap STRING 40 PRIMARY CAUSE A process or multiple processes have consumed an abnormal amount of memory on the system and the operating system is utilizing an abnor...

Page 266: ...he Interface SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Interface Name STRING 20 Interface IP Address STRING 50 PRIMARY CAUSE The local network interface is down PRIMARY ACTION Restore the local network interface SECONDARY CAUSE The mate network interface on the same sub net is faulty SECONDARY ACTION Restore the mate network interface TERNARY CAUSE Network congestion DESCRIPTION Link Mo...

Page 267: ...t Period Exceeded Limit Outgoing HB Period Exceeded Limit SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Max HB Period ms FOUR_BYTES HB Period ms FOUR_BYTES PRIMARY CAUSE This is caused by system performance degradation due to central processing unit CPU overload or excessive I O operations PRIMARY ACTION Identify the applications which are causing the system degradation by using the HMN CLI co...

Page 268: ...IPTION Disk Partition Critically Consumed SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Directory STRING 32 Device STRING 32 Percentage Used STRING 8 PRIMARY CAUSE A process or processes is are writing extraneous data to the named partition PRIMARY ACTION Perform disk a clean up and maintenance on the offending system DESCRIPTION Disk Partition Significantly Consumed SEVERITY MAJOR THRESHOL...

Page 269: ...vi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION The Free Inter Process Communication Pool Buffers Below Minor Threshold The Free IPC Pool Buffers Below Minor Threshold SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Free IPC Pool Buffer STRING 10 Threshold STRING 10 PRIMARY CAUSE The IPC pool buffers are not being properly freed by the application o...

Page 270: ... lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION The Free Inter Process Communication Pool Buffers Below Critical Threshold The Free IPC Pool Buffers Below Critical Threshold SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Free IPC Poll Buffer STRING 10 Threshold STRING 10 PRIMARY CAUSE The IPC pool buffers are not being properly freed by the ap...

Page 271: ...cal DNS Server Response Too Slow SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS DNS Server IP STRING 64 PRIMARY CAUSE The local domain name system DNS server too busy PRIMARY ACTION Check the local DNS server DESCRIPTION External Domain Name System Server Response Too Slow External DNS Server Response Too Slow SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS DNS Server IP STRING 64 PRIMARY CAU...

Page 272: ...nsive External DNS Server not Responsive SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS DNS Server IP STRING 64 PRIMARY CAUSE The DNS servers or the network may be down PRIMARY ACTION Check the DNS server s SECONDARY CAUSE There is an daemon called monitorDNS sh checking DNS server every minute or so It will issue an alarm if it cannot contact the DNS server or the response is slow But it wi...

Page 273: ... Server and Domain Name System Mismatch of IP Addr Local Server and DNS SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Host Name STRING 64 IP Addr Local Server STRING 64 IP Addr DNS Server STRING 64 PRIMARY CAUSE The DNS updates are not getting to the BTS 10200 from the external server or the discrepancy was detected before the local DNS lookup table was updated PRIMARY ACTION Ensure that the...

Page 274: ...Maintenance 79 section on page 7 82 DESCRIPTION Bulk Data Management System Admin State Change BDMS Admin State Change SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Application Instance STRING 40 Local State STRING 40 Mate State STRING 40 PRIMARY CAUSE The Bulk Data Management Server BDMS was switched over manually PRIMARY ACTION None DESCRIPTION Resource Reset SEVERITY INFO THRESHOLD 100 THROT...

Page 275: ...LD 100 THROTTLE 0 DATAWORDS Resource Type STRING 40 Resource Instance STRING 40 Warning Reason STRING 120 PRIMARY CAUSE Trunk Termination Subscriber Termination Media Gateway PRIMARY ACTION None DESCRIPTION Resource Reset Failure SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Resource Type STRING 40 Resource Instance STRING 40 Failure Reason STRING 120 PRIMARY CAUSE The informational event is th...

Page 276: ...CRIPTION Average Outgoing Heartbeat Period Exceeds Critical Limit Average Outgoing HB Period Exceeds Critical Limit SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Critical Threshold F FOUR_BYTES Current Average HB Peri FOUR_BYTES PRIMARY CAUSE The CPU is overloaded PRIMARY ACTION Shutdown the platform DESCRIPTION Swap Space Below Minor Threshold SEVERITY MINOR THRESHOLD 5 THROTTLE 0 DATAWORD...

Page 277: ...VERITY MAJOR THRESHOLD 5 THROTTLE 0 DATAWORDS Major Threshold MB FOUR_BYTES Current Value MB FOUR_BYTES PRIMARY CAUSE Too many processes are running PRIMARY ACTION Stop the proliferation of executables processes shell procedures SECONDARY CAUSE File spaces tmp or var run are over used SECONDARY ACTION Cleanup the file systems DESCRIPTION Swap Space Below Critical Threshold SEVERITY CRITICAL THRESH...

Page 278: ...equest MAINTENANCE 87 To troubleshoot and correct the cause of the alarm refer to the Status Update Process Request Failed Maintenance 87 section on page 7 107 DESCRIPTION System Health Report Collection Error SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS ErrString STRING 64 PRIMARY CAUSE An error occur while collecting the system health report PRIMARY ACTION Contact Cisco TAC for support DESC...

Page 279: ...pdate Error Maintenance 89 section on page 7 107 Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Status Update Process Database List Retrieval Error Status Update Process DB List Retrieval Error SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS ErrString STRING...

Page 280: ...on page 7 107 DESCRIPTION Disk Partition Moderately Consumed SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Directory STRING 32 Device STRING 32 Percentage Used STRING 8 PRIMARY CAUSE A process or processes is are writing extraneous data to the named partition PRIMARY ACTION Perform disk clean up and maintenance on the offending system DESCRIPTION Internet Protocol Manager Configuration File Er...

Page 281: ...face Failure Maintenance 93 section on page 7 108 DESCRIPTION Internet Protocol Manager Initialization Error IPM Initialization Error SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Reason STRING 128 PRIMARY CAUSE The IPM failed to initialize correctly PRIMARY ACTION Check the Reason dataword as to the cause of the error DESCRIPTION Internet Protocol Manager Interface Failure IPM Interface Failu...

Page 282: ...on on page 7 85 DESCRIPTION Internet Protocol Manager Interface State Change IPM Interface State Change SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Interface Name STRING 32 State STRING 16 PRIMARY CAUSE The IPM changed state on an interface up down PRIMARY ACTION None DESCRIPTION Internet Protocol Manager Interface Created IPM Interface Created SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS...

Page 283: ...AC and opening a service request DESCRIPTION Internet Protocol Manager Interface Removed IPM Interface Removed SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Hostname STRING 128 Logical IF Name STRING 32 IP Addr STRING 32 PRIMARY CAUSE The IPM removed a logical interface PRIMARY ACTION None DESCRIPTION Inter Process Communication Input Queue Entered Throttle State IPC Input Queue Entered Throttl...

Page 284: ...age lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Inter Process Communication Input Queue Depth at 25 of Its Hi Watermark IPC Input Queue Depth At 25 Of Its Hi Watermark SEVERITY MINOR THRESHOLD 500 THROTTLE 0 DATAWORDS Process Name STRING 10 Thread Type TWO_BYTES Thread Instance TWO_BYTES Hi Watermark FOUR_BYTES Lo Watermark FOUR_BYTES PRIMARY CAU...

Page 285: ...age lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Inter Process Communication Input Queue Depth at 50 of Its Hi Watermark IPC Input Queue Depth At 50 Of Its Hi Watermark SEVERITY MAJOR THRESHOLD 500 THROTTLE 0 DATAWORDS Process Name STRING 10 Thread Type TWO_BYTES Thread Instance TWO_BYTES Hi Watermark FOUR_BYTES Lo Watermark FOUR_BYTES PRIMARY CAU...

Page 286: ... IPC Input Queue Depth At 75 Of Its Hi Watermark SEVERITY CRITICAL THRESHOLD 500 THROTTLE 0 DATAWORDS Process Name STRING 10 Thread Type TWO_BYTES Thread Instance TWO_BYTES Hi Watermark FOUR_BYTES Lo Watermark FOUR_BYTES PRIMARY CAUSE The indicated thread is not able to process its IPC input messages fast enough hence the input queue has grown too large and is at 75 of the level at which it will e...

Page 287: ...tation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Thread Watchdog Counter Close to Expiry for a Thread SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 5 Thread Type FOUR_BYTES Thread Instance FOUR_BYTES PRIMARY CAUSE A software error has occurred PRIMARY ACTION None the sy...

Page 288: ...TION Aggregration Device Address Successfully Resolved SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS MGW IP Address STRING 17 MGW ID STRING 33 AGGR ID STRING 33 Network Address STRING 17 Subnet Mask ONE_BYTE PRIMARY CAUSE The event is informational PRIMARY ACTION No action needs to be taken DESCRIPTION Unprovisioned Aggregration Device Detected SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORD...

Page 289: ...LD 100 THROTTLE 0 DATAWORDS Reason Code STRING 17 MGW ID STRING 33 Reason STRING 128 PRIMARY CAUSE The auto AGGR identification ID resolution for the media gateway MGW IP failed due to a DNS lookup failure PRIMARY ACTION Check the provisioning of the DNS reverse lookup entry for the MGW IP address DESCRIPTION No Heartbeat Messages Received Through Interface From Router No HB Messages Received Thro...

Page 290: ...f the File With Full Path STRING 100 External Archive System STRING 50 PRIMARY CAUSE A problem in access to the external archive system has occurred PRIMARY ACTION Check the external archive system SECONDARY CAUSE The network to the external archive system is down SECONDARY ACTION Check the network TERNARY CAUSE The source log file is not present TERNARY ACTION Check the presence of log file DESCR...

Page 291: ...iled or File Corrupted SEVERITY MAJOR THRESHOLD 10 THROTTLE 0 DATAWORDS Full Path of LAF Configuration F STRING 50 PRIMARY CAUSE The LAF file is corrupted PRIMARY ACTION Check the log archive facility LAF configuration file SECONDARY CAUSE The file is missing SECONDARY ACTION Check the presence of LAF configuration file DESCRIPTION Cannot Login to External Archive Server SEVERITY CRITICAL THRESHOL...

Page 292: ...ngestion Status SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS System MCL Level ONE_BYTE PRIMARY CAUSE A change has occurred in the system overload level PRIMARY ACTION If the reported level remains continuously high adjust the system load or configuration DESCRIPTION Central Processing Unit Load of Critical Processes CPU Load of Critical Processes SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWOR...

Page 293: ... Processes SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Process Name STRING 5 Factor LEVEL ONE_BYTE Factor MCL ONE_BYTE PRIMARY CAUSE A change has occurred in the queue length of the critical processes PRIMARY ACTION If the reported level remains continuously high adjust the system load or configuration DESCRIPTION Inter Process Communication Buffer Usage Level IPC Buffer Usage Level SEVERITY ...

Page 294: ...l of the Feature Server CA Reports the Congestion Level of FS SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Domain Name of FS STRING 65 Feature Server ID STRING 20 PRIMARY CAUSE The Feature Server is congested PRIMARY ACTION No action is required DESCRIPTION Side Automatically Restarting Due to Fault SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Time of next restart attempt STRING 25 PRI...

Page 295: ... not Match Between the Primary Domain Name Server and the Internal Secondary Authoritative Domain Name Server DNS Zone Database does not Match Between the Primary DNS and the ISADS SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Zone Name STRING 64 Primary DNS Server IP STRING 64 Serial Number of that Zone in Sl EIGHT_BYTES Serial Number of that Zone in Ma EIGHT_BYTES PRIMARY CAUSE The zone t...

Page 296: ... to the Invalid SOAP Request Maintenance 121 section on page 89 DESCRIPTION Periodic Shared Memory Database Backup Success SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Details STRING 300 PRIMARY CAUSE Successful backup of the shared memory database PRIMARY ACTION N A DESCRIPTION Invalid SOAP Request SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Request STRING 256 Session ID STRING 20 PRIMAR...

Page 297: ...0 THROTTLE 0 DATAWORDS Prov Time at Seconds FOUR_BYTES Prov Time at Milli Seconds FOUR_BYTES Table Name STRING 40 Update String STRING 256 PRIMARY CAUSE The EMS hub maybe responding slowly PRIMARY ACTION Check to see if there are any hub alarms Take the appropriate action according to the alarms DESCRIPTION Northbound Provisioning Message Dropped Due To Full Index Table Northbound Provisioning Mes...

Page 298: ...intenance 12 INFO MAINTENANCE 13 Administrative State Change Maintenance 13 INFO MAINTENANCE 14 Call Agent Administrative State Change Maintenance 14 INFO MAINTENANCE 15 Feature Server Administrative State Change Maintenance 15 INFO MAINTENANCE 16 Process Manager Starting Process Maintenance 16 INFO MAINTENANCE 17 Invalid Event Report Received Maintenance 17 INFO MAINTENANCE 18 Process Manager Pro...

Page 299: ...d Maintenance 47 CRITICAL MAINTENANCE 48 Index Table Usage Exceeded Minor Usage Threshold Level Maintenance 48 MINOR MAINTENANCE 49 Index Table Usage Exceeded Major Usage Threshold Level Maintenance 49 MAJOR MAINTENANCE 50 Index Table Usage Exceeded Critical Usage Threshold Level Maintenance 50 CRITICAL MAINTENANCE 51 A Process Exceeds 70 of Central Processing Unit Usage Maintenance 51 MAJOR MAINT...

Page 300: ...81 Resource Reset Failure Maintenance 81 INFO MAINTENANCE 82 Average Outgoing Heartbeat Period Exceeds Critical Limit Maintenance 82 CRITICAL MAINTENANCE 83 Swap Space Below Minor Threshold Maintenance 83 MINOR MAINTENANCE 84 Swap Space Below Major Threshold Maintenance 84 MAJOR MAINTENANCE 85 Swap Space Below Critical Threshold Maintenance 85 CRITICAL MAINTENANCE 86 System Health Report Collectio...

Page 301: ...g Files Cannot be Transferred Maintenance 109 MAJOR MAINTENANCE 110 Access to Log Archive Facility Configuration File Failed or File Corrupted Maintenance 110 MAJOR MAINTENANCE 111 Cannot Login to External Archive Server Maintenance 111 CRITICAL MAINTENANCE 112 Congestion Status Maintenance 112 MAJOR MAINTENANCE 113 Central Processing Unit Load of Critical Processes Maintenance 113 INFO MAINTENANC...

Page 302: ...s become faulty To troubleshoot and correct the cause of the Mate Side has Become Faulty alarm refer to the Mate Side has Become Faulty Maintenance 4 section on page 7 92 Changeover Failure Maintenance 5 The Changeover Failure alarm major indicates that a changeover failed To troubleshoot and correct the cause of the Changeover Failure alarm refer to the Changeover Failure Maintenance 5 section on...

Page 303: ...Initialization Successful Maintenance 12 The Initialization Successful event functions as an informational alert that the initialization was successful The Initialization Successful event indicates that a local initialization has been successful The event is informational and no further action is required Administrative State Change Maintenance 13 The Administrative State Change event functions as...

Page 304: ... Process has Died Maintenance 18 The Process Manager Process has Died alarm minor indicates that a process has died To troubleshoot and correct the cause of the Process Manager Process has Died alarm refer to the Process Manager Process has Died Maintenance 18 section on page 7 97 Process Manager Process Exceeded Restart Rate Maintenance 19 The Process Manager Process Exceeded Restart Rate alarm m...

Page 305: ...efer to the Process Manager Restarting Process Maintenance 24 section on page 7 99 Process Manager Changing State Maintenance 25 The Process Manager Changing State event functions as an informational alert that a PMG process is changing state The primary cause of the event is that a side is transitioning from one state to another This is part of the normal side state change process Monitor the sys...

Page 306: ... is that the specified EMS has changed to the indicated state either naturally or via user request The event is informational and no action is necessary This is part of the normal state transitioning process for the EMS Monitor the system for related event reports if the transition was due to a faulty or out of service state Process Manager Sending Go Active to Process Maintenance 32 The Process M...

Page 307: ...ss did not come up when the system was brought up and had to be killed in order to restart it or a process did not come up when the system was brought up and had to be killed in order to restart it The event is informational and no further action is required Process Manager Clearing the Database Maintenance 38 The Process Manager Clearing the Database event functions as an informational alert that...

Page 308: ...indicates that the process has failed to come up in active mode To troubleshoot and correct the cause of the Process Manager Process Failed to Come Up in Active Mode alarm refer to the Process Manager Process Failed to Come Up in Active Mode Maintenance 43 section on page 7 99 Process Manager Process Failed to Come Up in Standby Mode Maintenance 44 The Process Manager Process Failed to Come Up in ...

Page 309: ...l To troubleshoot and correct the cause of the Index Table Usage Exceeded Major Usage Threshold Level alarm refer to the Index Table Usage Exceeded Major Usage Threshold Level Maintenance 49 section on page 7 101 Index Table Usage Exceeded Critical Usage Threshold Level Maintenance 50 The Index Table Usage Exceeded Critical Usage Threshold Level alarm critical indicates that the IDX table usage ha...

Page 310: ... 55 The Five Minute Load Average is Abnormally High alarm major indicates the five minute load average is abnormally high To troubleshoot and correct the cause of The Five Minute Load Average is Abnormally High alarm refer to the The Five Minute Load Average is Abnormally High Maintenance 55 section on page 7 102 The Load Average has Returned to Normal Levels Maintenance 56 The Load Average has Re...

Page 311: ...taining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request No Heartbeat Messages Received Through the Interface Maintenance 61 The No Heartbeat Messages Received Through the Interface alarm critical indicates that no HB messages are being received through the local network interface To troubleshoot and ...

Page 312: ...the disk partition consumption has reached the major threshold crossing level To troubleshoot and correct the cause of the Disk Partition Significantly Consumed alarm refer to the Disk Partition Significantly Consumed Maintenance 66 section on page 7 104 The Free Inter Process Communication Pool Buffers Below Minor Threshold Maintenance 67 The Free Inter Process Communication Pool Buffers Below Mi...

Page 313: ...efer to the The Free Inter Process Communication Pool Buffer Count Below Minimum Required Maintenance 70 section on page 7 105 Local Domain Name System Server Response Too Slow Maintenance 71 The Local Domain Name System Server Response Too Slow alarm major indicates that the response time of the local DNS server is too slow To troubleshoot and correct the cause of the Local Domain Name System Ser...

Page 314: ...cal DNS lookup table was updated Ensure the external DNS server is operational and sending updates to the BTS 10200 Mate Time Differs Beyond Tolerance Maintenance 77 The Mate Time Differs Beyond Tolerance alarm major indicates that the mate differs beyond the tolerance To troubleshoot and correct the cause of the Mate Time Differs Beyond Tolerance alarm refer to the Mate Time Differs Beyond Tolera...

Page 315: ...ates that the swap space has fallen below the minor threshold level To troubleshoot and correct the cause of the Swap Space Below Minor Threshold alarm refer to the Swap Space Below Minor Threshold Maintenance 83 section on page 7 106 Swap Space Below Major Threshold Maintenance 84 The Swap Space Below Major Threshold alarm major indicates that the swap space has fallen below the major threshold l...

Page 316: ...e 7 107 Disk Partition Moderately Consumed Maintenance 90 The Disk Partition Moderately Consumed alarm minor indicates that the disk partition is moderately consumed To troubleshoot and correct the cause of the Disk Partition Moderately Consumed alarm refer to the Disk Partition Moderately Consumed Maintenance 90 section on page 7 107 Internet Protocol Manager Configuration File Error Maintenance ...

Page 317: ...he thread is not able to process its IPC input messages fast enough hence the input queue has grown too large and is using up too much of the IPC memory pool resource To troubleshoot and correct the cause of the Inter Process Communication Input Queue Entered Throttle State alarm refer to the Inter Process Communication Input Queue Entered Throttle State Maintenance 97 section on page 7 108 Inter ...

Page 318: ... to Expiry for a Thread Maintenance 102 The Thread Watchdog Counter Close to Expiry for a Thread alarm critical indicates that the thread watchdog counter is close to expiry for a thread The primary cause of the alarm is that a software error has occurred No further action is required the BTS 10200 system will automatically recover or shutdown Central Processing Unit is Offline Maintenance 103 The...

Page 319: ...The primary cause of the event is that there is an access problem with the external archive system To correct the primary cause of the event check the external archive system The secondary cause of the event is that the network is having a problem To correct the secondary cause of the event check the network The ternary cause of the event is that the source log is not present To correct ternary ca...

Page 320: ...te the call load Queue Length of Critical Processes Maintenance 114 The Queue Length of Critical Processes event serves as an informational alert that a change has occurred in the queue length of critical processes If the reported level remains continuously high adjust the system load or configuration Inter Process Communication Buffer Usage Level Maintenance 115 The Inter Process Communication Bu...

Page 321: ...ance 119 section on page 7 111 Periodic Shared Memory Database Backup Success Maintenance 120 The Periodic Shared Memory Database Backup Success event serves as an informational alert that the periodic shared memory database backup was successfully completed The event is informational and no further action is required Invalid SOAP Request Maintenance 121 The Invalid SOAP Request event serves as an...

Page 322: ...n Maintenance 23 MAJOR MAINTENANCE 24 Process Manager Restarting Process Maintenance 24 MINOR MAINTENANCE 26 Process Manager Going Faulty Maintenance 26 MAJOR MAINTENANCE 40 Process Manager Binary Does not Exist for Process Maintenance 40 CRITICAL MAINTENANCE 42 Number of Heartbeat Messages Received is Less Than 50 of Expected Maintenance 42 MAJOR MAINTENANCE 43 Process Manager Process Failed to C...

Page 323: ...7 MAJOR MAINTENANCE 82 Average Outgoing Heartbeat Period Exceeds Critical Limit Maintenance 82 CRITICAL MAINTENANCE 83 Swap Space Below Minor Threshold Maintenance 83 MINOR MAINTENANCE 84 Swap Space Below Major Threshold Maintenance 84 MAJOR MAINTENANCE 85 Swap Space Below Critical Threshold Maintenance 85 CRITICAL MAINTENANCE 86 System Health Report Collection Error Maintenance 86 MINOR MAINTENAN...

Page 324: ...nd and then platform start command If a software restart does not resolve the problem or if the platform goes immediately to faulty again or does not start contact Cisco TAC It may be necessary to reinstall software If the alarm is reoccurring then the operating system or the hardware may have a problem Reboot host machine then reinstall and restart all applications The reboot will bring down the ...

Page 325: ...ing Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Changeover Timeout Maintenance 6 The Changeover Timeout alarm major indicates that a changeover timed out The cause of the alarm is that the system failed to changeover within time period Soon after this event is issued one platform will go to fault...

Page 326: ...e 4 section on page 7 92 Additionally if both systems local and mate are still running diagnose whether both instances are operating in stable state one in active and the other in standby If both are in a stable state wait 10 minutes and try the control command again If standby side is not in stable state bring down the standby side and restart software using the platform stop and platform start c...

Page 327: ...s of applicable EMS CA FS If only one platform is in a transitional state bring down platform that is in transitional state If both platforms are in a transitional state bring down platform that is in transitional _standby state Wait 3 minutes Determine current status of applicable EMS CA FS If not successful contact Cisco TAC for assistance Stable states for this step active_normal active_forced ...

Page 328: ...nd will affect service If approved by management Bring down secondary Is mate in stable state Was changeover successful Contact Cisco TAC for assistance No Attempt to bring up the primary instance of EMS CA or FS has failed because the secondary instance rejects changeover from active to standby MAINTENANCE event 7 was generated MATE rejected changeover Yes Restart the secondary instance Done Rest...

Page 329: ...he system has failed and the re initialization process has failed To correct the primary cause of the alarm check that the binary files are present for the unit Call Agent Feature Server Element Manager If the files are not present then re install the files from the initial or the backup media Then restart the failed device Local Initialization Timeout Maintenance 10 The Local Initialization Timeo...

Page 330: ... keepalive module connection to the mate has been lost The primary cause of the alarm is that a network interface hardware problem was occurred Soon after this event is issued one platform may go to faulty state To correct the primary cause of this alarm check whether the network interface is down If so restore network interface and restart the software The secondary cause of the alarm is a router...

Page 331: ...rm event Refer to the Local Side has Become Faulty Maintenance 3 section on page 7 92 Process Manager Binary Does not Exist for Process Maintenance 40 The Process Manager Binary Does not Exist for Process alarm critical indicates that the platform was not installed correctly The primary cause of the alarm is that the platform was not installed correctly To correct the primary cause of the alarm re...

Page 332: ...hover and if condition continues contact Cisco TAC Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Thread Watchdog Counter Expired for a Thread Maintenance 47 The Thread Watchdog Counter Expired for a Thread alarm critical indicates that a thread watchdog counter has expir...

Page 333: ...is that call traffic has exceeded design limits To correct the primary cause of the alarm verify that traffic is within the rated capacity The secondary cause of the alarm is that a software problem requiring additional analysis has occurred To correct the secondary cause of the alarm contact Cisco TAC Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi f...

Page 334: ...ritical Levels alarm critical indicates that memory and swap file usage have reached critical levels The primary cause of the alarm is that a process or multiple processes have consumed a critical amount of memory on the system and the operating system is utilizing a critical amount of the swap space for process execution This can be a result of high call rates or bulk provisioning activity To cor...

Page 335: ...ation Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request AverageOutgoingHeartbeatPeriodExceedsMajorAlarmLimit Maintenance 64 The Average Outgoing Heartbeat Period Exceeds Major Alarm Limit alarm major indicates that the average outgoing HB period has exceeded the major thresh...

Page 336: ...nd opening a service request The Free Inter Process Communication Pool Buffers Below Major Threshold Maintenance 68 The Free Inter Process Communication Pool Buffers Below Major Threshold alarm major indicates that the number of free IPC pool buffers has fallen below the major threshold crossing level The primary cause of the alarm is that IPC pool buffers are not being freed properly by the appli...

Page 337: ...The External Domain Name System Server Response Too Slow alarm major indicates that the response time of the external DNS server is too slow The primary cause of the alarm is that the network traffic level is busy or the nameserver is very busy To correct the primary cause of the alarm check the DNS server s The secondary cause of the alarm is that there is an daemon called monitorDNS sh checking ...

Page 338: ...use of the alarm cleanup the file systems Swap Space Below Major Threshold Maintenance 84 The Swap Space Below Major Threshold alarm major indicates that the swap space has fallen below the major threshold level The primary cause of the alarm is that too many processes are running To correct the primary cause of the alarm stop the proliferation of executables processes shell procedures The seconda...

Page 339: ...he alarm is that the MySQL DB on the EMS is not working properly To correct the primary cause of the alarm contact Cisco TAC Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Disk Partition Moderately Consumed Maintenance 90 The Disk Partition Moderately Consumed alarm minor...

Page 340: ...TAC and opening a service request Inter Process Communication Input Queue Depth at 25 of Its Hi Watermark Maintenance 98 The Inter Process Communication Input Queue Depth at 25 of Its Hi Watermark alarm minor indicates that the IPC input queue depth has reached 25 percent of its hi watermark The primary cause of the alarm is that the indicated thread is not able to process its IPC input messages f...

Page 341: ...switchover or automatic switchover No action needs to be taken the alarm is cleared when switchover is complete Service is temporarily suspended for a short period of time during this transition The primary cause of the alarm is that a system switchover occurred either due to manual switchover via CLI command failover or automatic switchover No action need to be taken the alarm is cleared itself w...

Page 342: ...f the network Access to Log Archive Facility Configuration File Failed or File Corrupted Maintenance 110 The Access to Log Archive Facility Configuration File Failed or File Corrupted alarm major indicates that access to the LAF configuration file failed or the file is corrupted The primary cause of the alarm is the LAF file is corrupted To correct the primary cause of the alarm check the LAF conf...

Page 343: ...er and the Internal Secondary Authoritative Domain Name Server Maintenance 118 The Domain Name Server Zone Database does not Match Between the Primary Domain Name Server and the Internal Secondary Authoritative Domain Name Server alarm critical indicates that the zone transfer between primary DNS and secondary DNS failed To troubleshoot and correct the cause of the Domain Name Server Zone Database...

Page 344: ...7 112 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Chapter 7 Maintenance Troubleshooting Troubleshooting Maintenance Alarms ...

Page 345: ...ons support system events Troubleshooting Operations Support System Alarms Provides the information needed to troubleshoot and correct the operations support system alarms Operations Support System Events and Alarms This section provides a brief overview of the operations support system events and alarms for the Cisco BTS 10200 Softswitch in numerical order Table 8 1 lists all of the operations su...

Page 346: ...ystem 2 section on page 8 13 Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Test Report SEVERITY Information INFO THRESHOLD 10000 THROTTLE 0 DESCRIPTION Undefined Variable in Known Set SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Module Name STRING 40 Fie...

Page 347: ...ge 8 14 Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Undefined Data Column Identification SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Noun STRING 40 Data Column ID STRING 40 PRIMARY CAUSE The database does not contain the required data column that was ...

Page 348: ... the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Structured Query Language Error While Getting Statistics SQL Error While Getting Statistics SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Statistics Category STRING 40 PRIMARY CAUSE An error occurred in accessing the Struc...

Page 349: ...Y WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Filename STRING 40 PRIMARY CAUSE The Management Information Base MIB file is missing or locked from access by the SNMP subsystem PRIMARY ACTION Contact Cisco TAC for support DESCRIPTION No Reply Received from Destination SEVERITY MINOR THRESHOLD 100 THROTTLE 5 DATAWORDS JMS Queue Name STRING 40 PRIMARY CAUSE This alarm is received when there is no respo...

Page 350: ...hentication Error SNMP Authentication Error SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS RemoteHost STRING 64 PRIMARY CAUSE The network management system NMS is issuing the wrong community string PRIMARY ACTION Ensure that the community string exists by using the CLI DESCRIPTION Queue Processing Module Database Management Index Failed with Error QAM DBM IDX Failed with Error SEVERITY MAJOR TH...

Page 351: ...ement Index Mismatch During Add or Delete QAM DBM IDX mismatch during Add or Delete SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Transaction ID STRING 32 Sequence Number EIGHT_BYTES Location Of Error STRING 16 DBM Result STRING 64 PRIMARY CAUSE The QAM is received a warning that either an entry already exists in DBM IDX during an add or an entry is nonexistent during a delete operation PRIM...

Page 352: ...ssion Count Exceeds Major Threshold Limit SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Session Type STRING 16 Session Maximum Limit STRING 3 Session Current Usage STRING 3 Session Usage Percentage STRING 3 PRIMARY CAUSE The user session usage has reached maximum allowed limit PRIMARY ACTION Use the report client_session command to view all login sessions Use the stop client_session command to...

Page 353: ...nt Severity OSS 1 Test Report Operations Support System 1 INFO OSS 2 Undefined Variable in Known Set Operations Support System 2 MAJOR OSS 3 Undefined Data Column Identification Operations Support System 3 MAJOR OSS 4 Request Handler Instantiation Error Operations Support System 4 MAJOR OSS 5 Structured Query Language Error While Getting Statistics Operations Support System 5 MINOR OSS 6 Structure...

Page 354: ...le Getting Statistics alarm minor indicates that an error occurred while accessing the SQL database for statistical information in the SNMP subsystem To troubleshoot and correct the cause of the Structured Query Language Error While Getting Statistics alarm refer to the Structured Query Language Error While Getting Statistics Operations Support System 5 section on page 8 14 Structured Query Langua...

Page 355: ...or alarm major indicates that the QAM is receiving an error because of a data mismatch between the information that is in the DBM IDX and the information that is in the EMS database Oracle To troubleshoot and correct the cause of the Queue Processing Module Database Management Index Failed with Error alarm refer to the Queue Processing Module Database Management Index Failed with Error Operations ...

Page 356: ...ount has exceeded the major threshold limit To troubleshoot and correct the cause of the User Session Count Exceeds Major Threshold Limit alarm refer to the User Session Count Exceeds Major Threshold Limit Operations Support System 14 section on page 8 15 Session Has Been Removed by Session Control Policy Operation Support System 16 The Session Has Been Removed by Session Control Policy event serv...

Page 357: ...at the database does not contain the required data column that was requested via the SNMP interface The primary cause of the alarm is that the database does not contain the required data column that was requested via the SNMP interface To correct the primary cause of the alarm contact Cisco TAC for support Note Refer to the Obtaining Documentation and Submitting a Service Request section on page l...

Page 358: ...r occurred while accessing the SQL database for statistical information in the SNMP subsystem The primary cause of the alarm is that an error occurred in accessing the SQL database for statistical information in the SNMP subsystem The alarm may result from schema error To correct the primary cause of the alarm contact Cisco TAC for support Note Refer to the Obtaining Documentation and Submitting a...

Page 359: ...imary cause of the alarm is that the NMS issuing the wrong community string To correct the primary cause of the alarm ensure the correct community string exists via CLI Queue Processing Module Database Management Index Failed with Error Operations Support System 10 The Queue Processing Module Database Management Index Failed with Error alarm major indicates that the QAM is receiving an error becau...

Page 360: ...8 16 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Chapter 8 Operations Support System Troubleshooting Troubleshooting Operations Support System Alarms ...

Page 361: ...rmation needed to monitor and correct the security events Troubleshooting Security Alarms Provides the information needed to troubleshoot and correct the security alarms Security Events and Alarms This section provides a brief overview of the security events and alarms for the Cisco BTS 10200 Softswitch in numerical order Table 9 1 lists all of the security events and alarms by severity Note Click...

Page 362: ...Security 2 section on page 9 5 DESCRIPTION Test Report SEVERITY Information INFO THRESHOLD 100 THROTTLE 0 DESCRIPTION Invalid Credentials Presented by a Session Initiation Protocol Phone Invalid Credentials Presented by a SIP Phone SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Authentication User Name STRING 33 From AOR STRING 65 SIP Request Type STRING 15 Sender IP STRING 20 PRIMARY CAUSE T...

Page 363: ...net Protocol Security Connection Down IPSEC Connection Down SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 PRIMARY CAUSE The kerberized management server KMS fails to establish the pf_key socket with the Internet Protocol security IPSEC engine This implies that the IPSEC engine is not running and that it may not be installed PRIMARY ACTION 1 Verify that the IPSEC is installed and running in the kernel 2 ...

Page 364: ...o establish the needed security association SA PRIMARY ACTION Remove or modify the security policy which caused the SA not found error This action assumes that security is provisioned DESCRIPTION Secure Session Initiation Protocol Endpoint Validation Failure Secure SIP Endpoint Validation Failure SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS AOR STRING 65 Secure Fqdn STRING 65 Source IP Addr...

Page 365: ...not running To troubleshoot and correct the cause of the Internet Protocol Security Connection Down alarm refer to the Internet Protocol Security Connection Down Security 3 section on page 9 7 Internet Protocol Security Media Terminal Adapter Key Establish Error Security 4 The Internet Protocol Security Media Terminal Adapter Key Establish Error event serves as a warning that the IPSEC MTA key est...

Page 366: ...ol Endpoint Validation Failure Security 6 The Secure Session Initiation Protocol Endpoint Validation Failure event serves as a warning that a secure SIP endpoint validation failed The primary cause of the event is that the BTS 10200 is incorrectly provisioned To correct the primary cause of the event check if correct value of secure fqdn is provisioned in the BTS 10200 system The secondary cause o...

Page 367: ...e IP security engine is not running The primary cause of the alarm is that the KMS has failed to establish the pf_key socket with the IPSEC engine The alarm implies that the IPSEC engine is not running and that it may not be installed To primary cause of the alarm verify that IPSEC is installed and running in the kernel and reboot the platform If problem persists or is recurrent contact Cisco TAC ...

Page 368: ...9 8 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Chapter 9 Security Troubleshooting Troubleshooting Security Alarms ...

Page 369: ...needed to monitor and correct the Signaling events Troubleshooting Signaling Alarms Provides the information needed to troubleshoot and correct the signaling alarms Caution The use of the UNIX ifconfig down command on any signaling interface to test or troubleshoot network or interface failures of the Cisco BTS 10200 Softswitch Signaling Interface may lead to undesirable consequences or conditions...

Page 370: ...ALING 47 SIGNALING 84 SIGNALING 40 SIGNALING 78 SIGNALING 32 SIGNALING 52 SIGNALING 48 SIGNALING 85 SIGNALING 59 SIGNALING 92 SIGNALING 33 SIGNALING 53 SIGNALING 56 SIGNALING 107 SIGNALING 63 SIGNALING 93 SIGNALING 34 SIGNALING 54 SIGNALING 67 SIGNALING 110 SIGNALING 66 SIGNALING 94 SIGNALING 60 SIGNALING 55 SIGNALING 123 SIGNALING 119 SIGNALING 68 SIGNALING 95 SIGNALING 70 SIGNALING 57 SIGNALING ...

Page 371: ... used SIGNALING 3 SIGNALING 3 is not used SIGNALING 122 SIGNALING 151 SIGNALING 166 SIGNALING 155 SIGNALING 125 SIGNALING 170 SIGNALING 167 SIGNALING 169 SIGNALING 126 SIGNALING 171 SIGNALING 168 SIGNALING 127 SIGNALING 177 SIGNALING 156 SIGNALING 157 SIGNALING 163 SIGNALING 164 SIGNALING 172 Table 10 1 Signaling Events and Alarms by Severity continued CRITICAL MAJOR MINOR WARNING INFO NOT USED DE...

Page 372: ...S Endpoint Name STRING 40 Message Type STRING 40 PRIMARY CAUSE This event is issued when a signaling adapter has received an invalid message from the specified endpoint PRIMARY ACTION Monitor the associated signaling link to see if there is an interruption of service on the link SECONDARY ACTION If there is a communication problem restart the link TERNARY ACTION Verify that the version of the prot...

Page 373: ...ING 64 Calling Function STRING 64 Index FOUR_BYTES PRIMARY CAUSE A signaling adapter has detected an error while accessing a database interface PRIMARY ACTION If the database that the adapter attempted to access is not available restart the associated process SECONDARY ACTION If an incompatible versions of the signaling adapter process and the database processes are present on the system correct t...

Page 374: ... occurred or an invalid network configuration exists PRIMARY ACTION Have your network administrator resolve the network errors Contact Cisco TAC if you need assistance Manually clear the alarm Restart this call agent instance using the platform start command DESCRIPTION Timeout on Internet Protocol Address Timeout on IP Address SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS MGW Term Name STRING...

Page 375: ...ssion Initiation Protocol Message Failed to Send Complete SIP Message SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Destination Address STRING 64 PRIMARY CAUSE Notifies the user that the Session Initiation Protocol SIP stack failed to send an SIP message due to it exceeding the maximum length of a UDP packet PRIMARY ACTION If encountered in normal network operations the message should be captu...

Page 376: ...er ID STRING 20 PRIMARY CAUSE The feature server platform is down or is not operating properly PRIMARY ACTION Restart the applicable feature server DESCRIPTION Signaling System 7 Signaling Link Down SS7 Signaling Link Down SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Link_Number ONE_BYTE Link_Name STRING 25 PRIMARY CAUSE The Signaling System 7 SS7 trunk group may be out of service OOS PRIMARY...

Page 377: ... DATAWORDS Link ONE_BYTE Link Name STRING 8 PRIMARY CAUSE Issued when the specified Signaling System 7 SS7 link is inhibited at the remote end PRIMARY ACTION Monitor the events at the network level for any related to the specified SS7 link Restorative actions need to be taken on the remote end DESCRIPTION Link is Locally Inhibited SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Link No ONE_BYTE ...

Page 378: ...mine if the traffic load on the specified SS7 link is too high on the local end or if the remote end is lagging in processing the traffic SECONDARY ACTION Verify that the SS7 link has not degraded in quality TERNARY ACTION Verify that the traffic load has not become unbalanced if multiple SS7 links are used SUBSEQUENT ACTION Verify that local SS7 signaling adapter process is running normally DESCR...

Page 379: ...RESHOLD 100 THROTTLE 0 DATAWORDS Link No ONE_BYTE Link Name STRING 8 PRIMARY CAUSE Issued when the specified SS7 link has experienced a processor outage PRIMARY ACTION Monitor the network level event reports for any events associated with the processing complex used by the specified SS7 link Verify that the SS7 link is returned to service DESCRIPTION Link Set Inaccessible SEVERITY MAJOR THRESHOLD ...

Page 380: ...t reports at the network level to determine if the traffic load on the specified SS7 link set is too high on the local end or if the remote end is lagging in processing the traffic SECONDARY ACTION Verify that the SS7 link set has not degraded in quality TERNARY ACTION Verify that the traffic load has not become unbalanced if multiple SS7 link sets are used SUBSEQUENT ACTION Verify that local SS7 ...

Page 381: ... at the network level to determine if the traffic load on the specified SS7 link set is too high on the local end or if the remote end is lagging in processing the traffic SECONDARY ACTION Verify that the SS7 link set has not degraded in quality TERNARY ACTION Verify that the traffic load has not become unbalanced if multiple SS7 link sets are used SUBSEQUENT ACTION Verify that local SS7 signaling...

Page 382: ...he event reports at the network level to determine if the traffic load to the specified DPC is too high on the local end or if the remote end is lagging in processing the traffic DESCRIPTION Unanswered Blocking Message Unanswered BLO SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when a blocking BLO messag...

Page 383: ... CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when a unblocking message UBL message was not acknowledged before the timer 15 T15 expired for the associated CIC PRIMARY ACTION Verify that the SS7 signaling adapter processes is running normally Verify that the call agent platform is active SECONDARY ACTION Verify that the SS7 interface hardware is in servic...

Page 384: ... DATAWORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when a circuit group blocking CGB message was not acknowledged before the timer 19 T19 expired for the associated CIC PRIMARY ACTION Verify that the SS7 signaling adapter processes is running normally Verify that the call agent platform is active SECONDARY ACTION Verify that the SS7 interface hardwar...

Page 385: ...0 DATAWORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when a circuit group unblocking CGU message was not acknowledged before the timer 21 T21 expired for the associated CIC PRIMARY ACTION Verify that the SS7 signaling adapter processes is running normally SECONDARY ACTION Verify that the call agent platform is active TERNARY ACTION Verify that the SS7...

Page 386: ...DS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when a circuit query message CQM message was not acknowledged before the timer 28 T28 expired for the associated CIC PRIMARY ACTION Verify that the SS7 signaling adapter processes is running normally SECONDARY ACTION Verify that the call agent platform is active Verify that the SS7 interface hardware is in s...

Page 387: ...TLE 0 DATAWORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when a circuit validation test CVT message was not acknowledged before the Tcvt expired for the associated CIC PRIMARY ACTION Verify that the SS7 signaling adapter processes is running normally SECONDARY ACTION Verify that the call agent platform is active TERNARY ACTION Verify that the SS7 inte...

Page 388: ...AWORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when a reset circuit RSC message was not acknowledged before the timer 17 T17 expired for the associated CIC PRIMARY ACTION Verify that the SS7 signaling adapter processes is running normally Verify that the call agent platform is active SECONDARY ACTION Verify that the SS7 interface hardware is in servi...

Page 389: ...WORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when a group reset GRS message was not acknowledged before the timer 23 T23 expired for the associated CIC PRIMARY ACTION Verify that the SS7 signaling adapter processes is running normally Verify that the call agent platform is active SECONDARY ACTION Verify that the SS7 interface hardware is in service ...

Page 390: ...ORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when a release REL message was not acknowledged before the timer 5 T5 expired for the associated CIC PRIMARY ACTION Verify that the SS7 signaling adapter processes is running normally SECONDARY ACTION Verify that the call agent platform is active Verify that the SS7 interface hardware is in service TERNARY...

Page 391: ...TLE 0 DATAWORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when an loop prevention acknowledgement LPA message was not acknowledged before the timer continuity check request TCCR expired for the associated CIC PRIMARY ACTION Verify that the SS7 signaling adapter processes is running normally SECONDARY ACTION Verify that the call agent platform is active...

Page 392: ...ection on page 10 137 SIGNALING 37 SIGNALING 37 is not used SIGNALING 38 SIGNALING 38 is not used SIGNALING 39 SIGNALING 39 is not used DESCRIPTION Trunk Locally Blocked SEVERITY MINOR THRESHOLD 500 THROTTLE 0 DATAWORDS CIC Number STRING 40 TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 MGW EP NAME STRING 64 MGW TSAP ADDR STRING 80 Reason STRING 80 PRIMARY CAUSE Issued when a BLO or CGB message was...

Page 393: ...AP ADDR STRING 80 PRIMARY CAUSE Issued when a BLO or CGB message was received on the specified CIC if it is SS7 trunk Issued when a SERVICE OOS message is received for Integrated Services Digital Network ISDN trunks Issued when Reverse Make Busy RBZ signal is received for channel associated signaling CAS operator trunk PRIMARY ACTION No action required Can manually recover from this condition loca...

Page 394: ...set Circuit Message on the Specified Circuit Identification Code RLC Received in Response to RSC Message on the Specified CIC SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when a release complete RLC message was received in response to a RSC message on the specified CIC PRIMARY ACTION No action required DESC...

Page 395: ...FO THRESHOLD 100 THROTTLE 0 DATAWORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when an unequipped circuit has been detected on the remote side PRIMARY ACTION Monitor the event reports at the network level to verify if an existing circuit was unequipped causing a status mismatch with the local end DESCRIPTION Specified Circuit Identification Code is In...

Page 396: ...er TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when a general SS7 processing error occurred due to all resources being busy or an invalid event occurring PRIMARY ACTION Verify the status of the signaling adapter process and the SS7 signaling interface to verify proper operation DESCRIPTION Unexpected Message for the Call State is Received Clear Call SEVERITY INFO T...

Page 397: ... THRESHOLD 100 THROTTLE 0 DATAWORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when the specified CIC is marked at remotely unequipped due to the CQM response indicating that it is unequipped at the far end PRIMARY ACTION Equip the trunk circuit at the far end DESCRIPTION Set Trunk State as NOT Remotely Blocked SEVERITY INFO THRESHOLD 100 THROTTLE 0 DAT...

Page 398: ...S CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when the specified CIC is marked as remotely blocked due to the CQM response indicating that it is remotely blocked at the far end PRIMARY ACTION Clear the blocking situation at the far end based on network level event reports DESCRIPTION Circuit Validation Test Aborted SEVERITY INFO THRESHOLD 100 THROTTLE 0 ...

Page 399: ...0 105 DESCRIPTION Circuit Validation Successful SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE Issued when the specified circuit was successfully validated PRIMARY ACTION No action required DESCRIPTION Continuity Recheck Failed SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES D...

Page 400: ...20 PRIMARY CAUSE Issued when a continuity recheck of the specified CIC was successful PRIMARY ACTION No action required DESCRIPTION Auto State Change for Integrated Services Digital Network Trunk Group by Media Gateway Auto State Change for ISDN Trunk Group by Media Gateway SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Trunk Group ID FOUR_BYTES Trunk Group Index FOUR_BYTES Media Gateway Name S...

Page 401: ...rmination Name STRING 40 Termination Index FOUR_BYTES Trunk Group ID FOUR_BYTES Trunk Group Index FOUR_BYTES Cause Value ONE_BYTE Call State ONE_BYTE PRIMARY CAUSE Issued when an ISDN status message was received containing an error indication for the specified termination PRIMARY ACTION If the specified termination is not operating normally place it in the service state DESCRIPTION Trunk Operation...

Page 402: ...TES Trunk Group ID FOUR_BYTES Trunk Group Index FOUR_BYTES Flag FOUR_BYTES PRIMARY CAUSE Issued when an ISDN restart message was received from the specified gateway PRIMARY ACTION Monitor the event reports at the network level to verify that the specified gateway and terminations are operating normally DESCRIPTION Media Gateway Termination Faulty SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS F...

Page 403: ...tting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Media Gateway Adapter Running out of Shared Memory Pools MGA Running out of Shared Memory Pools SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 PRIMARY CAUSE Issued when the Media Gateway Control Protocol MGCP signaling adapter was unable to allocate data storage for a...

Page 404: ...page lvi for detailed instructions on contacting Cisco TAC and opening a service request SIGNALING 67 SIGNALING 67 is not used DESCRIPTION Call Agent Internal Error Because of Which Media Gateway Adapter has to Start Automatically CA Internal Error Because of Which MGA has to Start Automatically SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Fully Qualified Name STRING 80 Reason STRING 80 Detai...

Page 405: ...Type of MGW STRING 32 Failure Cause STRING 80 Subscriber Info STRING 80 ICMP Ping Status STRING 80 PRIMARY CAUSE This could have been caused by 1 The trunking gateway has been administratively taken to the OOS state using the CLI command in the gateway GW 2 The trunk endpoint is alarmed or in the OOS state PRIMARY ACTION 1 Fix the trunking gateway state that has been administratively taken to the ...

Page 406: ...iled Communication SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Trunk Group ID FOUR_BYTES Trunk Group Index FOUR_BYTES PRIMARY CAUSE The ISDN signaling adapter is unable to restore a D channel due to incorrect backhaul provisioning at the media gateway or call agent PRIMARY ACTION Ensure that the provisioning of the backhaul port is correct at both the call agent and media gateway DESCRIPTI...

Page 407: ...S Trunk Group ID FOUR_BYTES Trunk Group Index FOUR_BYTES PRIMARY CAUSE The ISDN signaling adapter has lost calls due to a D channel being down as a result of a media gateway power loss or a loss of the connection between the private branch exchange PBX and the media gateway PRIMARY ACTION Resupply power to the media gateway and verify that the connection between the PBX and the media gateway is in...

Page 408: ...to Send the SERVICE Due to the SERVICE Timer Expired SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Termination Name STRING 40 Termination Index FOUR_BYTES Trunk Group ID FOUR_BYTES Trunk Group Index FOUR_BYTES Service Status FOUR_BYTES PRIMARY CAUSE The ISDN signaling adapter was unable to send a SERVICE message due to the expiration of the SERVICE timer PRIMARY ACTION Ensure that the SERVIC...

Page 409: ... 100 THROTTLE 0 DATAWORDS Port Number TWO_BYTES Hostname STRING 64 PRIMARY CAUSE The communication between the call agent and the remote instance is faulty PRIMARY ACTION No action needed DESCRIPTION Integrated Services Digital Network D channel Switchover for Non Facility Associated Signaling ISDN D channel Switchover for NFAS SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Trunk Group ID FOUR_B...

Page 410: ...nnel Down for Non Facility Associated Signaling ISDN Single D channel Down for NFAS SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Trunk Group ID FOUR_BYTES Trunk Group Idx FOUR_BYTES IS Primary D Channel FOUR_BYTES PRIMARY CAUSE One of the ISDN D channels in the primary rate interface PRI is down PRIMARY ACTION Check the gateway power and the gateway connection to the PBX DESCRIPTION Trunking ...

Page 411: ...co TAC and opening a service request DESCRIPTION Out of Bounds Memory Socket Error SEVERITY CRITICAL DATAWORDS Process Name STRING 40 Description STRING 40 Extra Info STRING 40 PRIMARY CAUSE Out of heap memory PRIMARY ACTION Notify customer support Increase the random access memory RAM Contact Cisco TAC SECONDARY CAUSE Out of IPC pool memory SECONDARY ACTION Resize the IPC pool size in the platfor...

Page 412: ...Signaling 82 section on page 10 140 Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Insufficient Heap Memory SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Gateway ID STRING 32 PRIMARY CAUSE Issued when the H 323 Protocol H323 signaling adapter is unable ...

Page 413: ...ause of the alarm refer to the Reached Maximum Socket Limit Signaling 84 section on page 10 140 DESCRIPTION Error While Binding to Socket SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Gateway ID STRING 32 Socket ID FOUR_BYTES Local TSAP Address STRING 32 Reason STRING 128 PRIMARY CAUSE An error has occurred while binding to a socket PRIMARY ACTION Contact Cisco TAC DESCRIPTION Reached Maxim...

Page 414: ...TION Initialization Failure SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Gateway ID STRING 32 Reason STRING 128 PRIMARY CAUSE A process initialization failure has occurred PRIMARY ACTION Check Dataword 2 Reason for the failure cause and take action accordingly DESCRIPTION Remote H323 Gateway is not Reachable SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Gateway ID STRING 32 Remote GW T...

Page 415: ...ubmitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION H323 Message Parsing Error SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Gateway ID STRING 32 Remote GW TSAP Addr STRING 32 PRIMARY CAUSE Unable to successfully parse an incoming H323 message PRIMARY ACTION This is a result of either a software bug or bad ...

Page 416: ... ID STRING 32 Gatekeeper ID STRING 32 GK TSAP Addr STRING 32 PRIMARY CAUSE The gatekeeper is not available or is unreachable PRIMARY ACTION Check network connectivity Check to ensure that the gatekeeper GK is reachable by trying to ping GK IP address If reachable then check to ensure that the GK is configured up DESCRIPTION Alternate Gatekeeper is not Responding SEVERITY MAJOR THRESHOLD 100 THROTT...

Page 417: ...ERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Gateway ID STRING 32 Gatekeeper ID STRING 32 GK TSAP Addr STRING 32 PRIMARY CAUSE An H323 security violation has occurred PRIMARY ACTION The password on the BTS 10200 and or the gatekeeper is wrong the H323 gateway H323GW table may not be provisioned properly or there is a time synchronization problem between the BTS 10200 and or gatekeeper and the Ne...

Page 418: ...ining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Invalid Call Reference Value SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Gateway ID STRING 32 Remote GW TSAP Addr STRING 32 Call ID EIGHT_BYTES Call Ref Value EIGHT_BYTES PRIMARY CAUSE The call ID was invalid or changed mid call ...

Page 419: ... Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Invalid Conference Identifier SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Gateway ID STRING 32 Reason STRING 32 Remote GW Port TWO_BYTES Call ID EIGHT_BYTES Conference ID EIGHT_BYTES PRIMARY CAUSE The call ID was invalid or changed mi...

Page 420: ...ng 96 section on page 10 143 Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Invalid Message from the Network SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Gateway ID STRING 32 Remote GW TSAP Addr STRING 32 Call ID EIGHT_BYTES Conf ID EIGHT_BYTES Call Ref V...

Page 421: ...and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Insufficient Information to Complete Call SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Gateway ID STRING 32 Call ID EIGHT_BYTES Conf ID EIGHT_BYTES Call Ref Value EIGHT_BYTES PRIMARY CAUSE Not enough initial call setup information was received to est...

Page 422: ...orrect the cause of the alarm refer to the Codec Negotiation Failed Signaling 100 section on page 10 144 DESCRIPTION Abnormal Call Clearing SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Gateway ID STRING 32 Call ID EIGHT_BYTES Reason STRING 128 PRIMARY CAUSE Unsupported or invalid message type received from network PRIMARY ACTION Call Cisco TAC Contact Cisco TAC DESCRIPTION Codec Negotiation F...

Page 423: ...VERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Gateway ID STRING 32 Call ID EIGHT_BYTES Gatekeeper ID STRING 32 PRIMARY CAUSE This is a future trap definition PRIMARY ACTION None DESCRIPTION H323 Network Congested SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Gateway ID STRING 32 Gatekeeper ID STRING 32 PRIMARY CAUSE The H323 application processes has depleted its resources No more calls can ...

Page 424: ...age 10 113 DESCRIPTION Aggregation Connection Down AGGR Connection Down SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS AGGR ID STRING 16 PRIMARY CAUSE The Transmission Control Protocol TCP connection is down PRIMARY ACTION Check the associated cabling and perform a PING to test the connectivity DESCRIPTION Aggregation Unable To Establish Connection AGGR Unable To Establish Connection SEVERITY I...

Page 425: ...ction Down Signaling 106 section on page 10 144 DESCRIPTION Aggregation Gate Set Failed AGGR Gate Set Failed SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS AGGR ID STRING 16 Error Code TWO_BYTES Sub Error Code TWO_BYTES PRIMARY CAUSE The gate set acknowledgement never came from the CMTS PRIMARY ACTION None DESCRIPTION Enhanced Subscriber Authentication BTS 10200 Delivery Function Connection Down...

Page 426: ...DATAWORDS Contact Domain Name STRING 128 Number of IP Addresses Resolved FOUR_BYTES Number of Virtual IP Addresses FOUR_BYTES PRIMARY CAUSE A contact name in the configuration file is not configured in the domain name system DNS PRIMARY ACTION Verify that the name in the DNS matches the name in the platform cfg and opticall cfg files SECONDARY CAUSE A contact could not be resolved to an IP address...

Page 427: ...SEVERITY MAJOR THRESHOLD 30 THROTTLE 0 DATAWORDS Host Domain Name STRING 128 PRIMARY CAUSE The hostname parameter is specified in the platform cfg file instead of contact parameter PRIMARY ACTION The BTS 10200 is configured as a SIMPLEX system DESCRIPTION Stream Control Transmission Protocol Association Failure SCTP Association Failure SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS SCTP Associa...

Page 428: ... out of service PRIMARY ACTION Make sure all Ethernet connections on the CA and the SGs are plugged in Also make sure all of the associated IP routers are operational SECONDARY CAUSE The MTP3 user adapter M3UA layer is down between the CA and the SGs SECONDARY ACTION Use the Cisco snooper application to determine why the M3UA layer is down DESCRIPTION Stream Control Transmission Protocol Associati...

Page 429: ...ntrol Transmission Protocol Association Configuration Error SCTP Association Configuration Error SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS SCTP Association ID STRING 17 PRIMARY CAUSE The destination IP address is invalid PRIMARY ACTION Input a new destination IP address see log for additional details SECONDARY CAUSE The local IP address is invalid SECONDARY ACTION Input new local IP addres...

Page 430: ...ared OFF status and are removed from CURRENT_ALARM table However not all cleared alarms OFF status are displayed on the subscriber terminal Only the first instance of the cleared alarms OFF status with a variation in TYPE NUMBER and COMPONENT ID are displayed Multiple instances of the cleared alarms OFF status where the TYPE NUMBER and COMPONENT ID are identical are not displayed DESCRIPTION Signa...

Page 431: ...PONENT ID are identical are not displayed SIGNALING 115 To monitor and correct the cause of the event refer to the Invalid Routing Context Received Signaling 115 section on page 10 115 DESCRIPTION Signaling Gateway Process is Out of Service SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Signaling Gateway STRING 17 PRIMARY CAUSE All of the SCTP associations between the SGP and the CA are out of ...

Page 432: ...ion Point Code User Part Unavailable DPC User Part Unavailable SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS DPC ID STRING 17 PRIMARY CAUSE A SGP sent a destination user part unavailable DUPU M3UA message to the CA indicating that a User Part is unavailable on a DPC PRIMARY ACTION Contact the SS7 Network Administrator to report the User Part Unavailable problem on the DPC so that communication...

Page 433: ...rcuit Verification Response Received with Failed Indication CVR Received with Failed Indication SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS CIC TWO_BYTES TGN ID EIGHT_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE A CIC mismatch occurred PRIMARY ACTION Perform an internal test such as checking that the CIC is assigned to a circuit between the sending and the receiving switch DESCRIPTION Sig...

Page 434: ...gnaling System 7 Module Signaling System 7 Adapter Faulty S7M S7A Faulty SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Reason STRING 36 PRIMARY CAUSE An OMNI failure has occurred PRIMARY ACTION Check the OMNI status a failover will occur in a duplex configuration DESCRIPTION Message Transfer Part 3 User Adapter Cannot Go Standby M3UA SUA Cannot Go Standby SEVERITY MAJOR THRESHOLD 100 THROTT...

Page 435: ...ansfer Part 3 User Adapter Cannot Go Active M3UA SUA Cannot Go Active SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Platform ID STRING 17 PRIMARY CAUSE No ACTIVE acknowledgement messages are being received from any SG or SCTP The associations are probably down PRIMARY ACTION Investigate any other alarms to see if the SGs are down or the SCTP associations are down Take corrective action accordi...

Page 436: ...rror SCCP Routing Error SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 PRIMARY CAUSE The signaling connection control part SCCP route is invalid or is not available PRIMARY ACTION Provision the right SCCP route DESCRIPTION Signaling Connection Control Part Binding Failure SCCP Binding Failure SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Local Point Code STRING 20 Local Subsystem Numb ONE_BYTE PRIMAR...

Page 437: ... monitor and correct the cause of the event refer to the Transaction Capabilities Application Part Reaches the Provisioned Resource Limit Signaling 132 section on page 10 117 DESCRIPTION Transaction Capabilities Application Part Reaches the Provisioned Resource Limit TCAP Reaches the Provisioned Resource Limit SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Dialogue Invoke ID FOUR_BYTES PRIMAR...

Page 438: ... THROTTLE 0 DATAWORDS Endpoint Name STRING 40 GTD Content Type STRING 40 PRIMARY CAUSE Issued when the generic transport descriptor GTD parser failed to decode a GTD message received from the specified endpoint PRIMARY ACTION Verify that the version of the GTD protocol used by the device at the remote endpoint is consistent with the version expected by the call agent SECONDARY ACTION Examine the a...

Page 439: ...tation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Signaling System 7 Message Decoding Failure SS7 Message Decoding Failure SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE An error in the ISUP stack or in a SAI messag...

Page 440: ... Cisco TAC and opening a service request DESCRIPTION Signaling System 7 Message Invalid Received SS7 Message Invalid Received SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS CIC Number TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 PRIMARY CAUSE An invalid message was received from the line in the ISUP stack PRIMARY ACTION Capture the SS7 trace of circuit for examination by support perso...

Page 441: ... PRIMARY CAUSE An ISUP message or a parameter received was not recognized or understood PRIMARY ACTION Check the log for more information including confusion CFN diagnostic output Capture a SS7 trace of the affected circuits If the diagnostic data indicates that messages or parameters that must be supported are being dropped refer the captured data to support along with a description of the call s...

Page 442: ...THRESHOLD 100 THROTTLE 0 DATAWORDS CIC TWO_BYTES TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 Near End State STRING 64 Far End State STRING 64 Resolution Action STRING 64 PRIMARY CAUSE A discrepancy between the local and the remote trunk states has occurred PRIMARY ACTION Automatic corrective action is enforced when using American National Standards Institute ANSI ISUP DESCRIPTION Unanswered Info...

Page 443: ...rovisioning DESCRIPTION Session Initiation Protocol Trunk Operationally Out of Service SIP Trunk Operationally out of Service SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Trunk Group Description STRING 21 Trunk SIP Element ID STRING 65 Trunk Server Group ID STRING 65 PRIMARY CAUSE Issued when the BTS 10200 is unable to communicate with a remote SIP party Call Agent or Proxy over a SIP or a...

Page 444: ...44 section on page 10 155 DESCRIPTION Internet Protocol Interface Link to the Signaling System 7 Signaling Gateway is Down IP Interface Link to the SS7 Signaling Gateway is Down SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Interface Name STRING 65 Interface IP Address STRING 65 PRIMARY CAUSE A hardware problem has occurred PRIMARY ACTION Check the link interfaces DESCRIPTION All Internet Prot...

Page 445: ...ddress STRING 65 PRIMARY CAUSE A hardware problem has occurred PRIMARY ACTION Check the link interfaces DESCRIPTION All Retransmission Attempts of Session Initiation Protocol Request or Response Failed All Retransmission Attempts of SIP Request or Response Failed SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS SIP Request Type STRING 15 Sender IP STRING 20 PRIMARY CAUSE SIP request All retrans...

Page 446: ...ervice Addresses Exhausted DNS SRV Addresses Exhausted SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS SRV Hostname STRING 256 PRIMARY CAUSE The DNS service SRV hostname resolution to the IP address is exhausted PRIMARY ACTION Add an entry to the SRV in the DNS server Fix the BTS 10200 provisioning DESCRIPTION Stream Control Transmission Protocol Association Congested SCTP Association Congeste...

Page 447: ...DS End Point Termination STRING 54 Media Gateway Type STRING 54 Error Details STRING 54 PRIMARY CAUSE The residential gateway returned an error code in response to a command from the MGW PRIMARY ACTION Try controlling subscriber termination to OOS and back to INS using the BTS 10200 CLI command If the problem persist after more calls check the configuration in the BTS 10200 and the RGW If the erro...

Page 448: ...ks Become Locally Blocked SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS CIC Number STRING 40 TGN ID FOUR_BYTES DPC STRING 20 OPC STRING 20 MGW EP Name STRING 64 MGW TSAP_ADDR STRING 80 Reason STRING 80 PRIMARY CAUSE Issued when an emergency trunk CAS SS7 or ISDN gets locally blocked PRIMARY ACTION No action is required DESCRIPTION Emergency Trunks Become Remotely Blocked SEVERITY CRITICAL T...

Page 449: ...0 DATAWORDS AGGR ID STRING 16 Subscriber IP Address STRING 32 Gate Direction STRING 16 PRIMARY CAUSE An error condition has been encountered by the CMTS PRIMARY ACTION Check the alarms and warnings from the CMTS DESCRIPTION Integrated Services Digital Network Signaling Gateway Down ISDN Signaling Gateway Down SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Media Gateway ID STRING 16 Media Gatewa...

Page 450: ...ignaling Gateway Inactive ISDN Signaling Gateway Inactive SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Media Gateway ID STRING 16 PRIMARY CAUSE A shutdown command was executed in the application server on the ISDN gateway side PRIMARY ACTION No action is needed The ISDN gateway will be automatically recovered DESCRIPTION Invalid Integrated Services Digital Network Interface Identification Inv...

Page 451: ...D 100 THROTTLE 0 DATAWORDS N A PRIMARY CAUSE No ACTIVE acknowledgement messages are being received from any signaling gateway The ISDN signaling gateway or the SCTP associations are probably down PRIMARY ACTION Investigate other alarms to see if the signaling gateways are down or the SCTP associations are down Take corrective action according to those alarms DESCRIPTION Integrated Services Digital...

Page 452: ...pt and 911 ringback calls PRIMARY ACTION Upgrade or re provision the remote switch so it can process incoming SIP UPDATE messages DESCRIPTION Session Initiation Protocol Server Group Element Operationally Out of Service SIP Server Group Element Operationally out of Service SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS Server Group Description STRING 64 TSAP Address of the SIP Element STRING...

Page 453: ...TRING 17 PRIMARY CAUSE INACTIVE ACK messages were received from a Signaling Gateway The SG or the SCTP associations are probably down PRIMARY ACTION Investigate other alarms to see if the SGs are down or the SCTP associations are down Take corrective action according to those alarms Also check the AS status for the routing context on ITP DESCRIPTION Signaling Gateway Traffic Mode Mismatch SEVERITY...

Page 454: ...oned to add the P DCS Billing Info header to outgoing SIP requests and responses PRIMARY ACTION Provision the originating switch to add the P DCS Billing Info header to outgoing messages SECONDARY CAUSE The header could have been stripped off by an intermediate proxy SECONDARY ACTION Determine if it has been stripped off by an intermediate proxy and configure for corrective action if so TERNARY CA...

Page 455: ...ation problem between the ITP and the BTS 10200 has occurred PRIMARY ACTION Check the communication path between the BTS 10200 and the ITP DESCRIPTION A Session Initiation Protocol Server Group has no Child Elements Provisioned A SIP Server Group has no Child Elements Provisioned SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Server Group ID STRING 64 PRIMARY CAUSE Issued when a SIP Server Gr...

Page 456: ...Disabled SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS SIP Element ID STRING 64 PRIMARY CAUSE A SIP Element was provisioned with SRV enabled and is associated to at least one or more Server Groups PRIMARY ACTION The SRV flag will be assumed disabled However to resolve this informational message provision the SRV flag disabled on the SIP element DESCRIPTION Residential Gateway Endpoints are out ...

Page 457: ...tity Name STRING 40 General Context STRING 40 Specific Context STRING 40 Failure Context STRING 40 PRIMARY CAUSE A MGCP signaling interop error has occurred with the residential media gateway PRIMARY ACTION Check the IP connectivity status between BTS 10200 call agent and the trunking gateway If the residential gateway is not physically connected but controlled INS at the BTS 10200 DESCRIPTION Mul...

Page 458: ...Signaling 174 section on page 10 159 DESCRIPTION ENUM Server Domain Cannot be Resolved into Any IP Address SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS ENUM Server Domain STRING 128 ENUM Profile ID STRING 64 PRIMARY CAUSE Misconfiguration in the DNS PRIMARY ACTION Fix the configuration in the DNS according to the documentation DESCRIPTION ENUM Server Unavailable SEVERITY CRITICAL THRESHOLD...

Page 459: ...ilable to Launch ENUM Query Signaling 176 section on page 10 159 DESCRIPTION ENUM Server Farm Unavailable SEVERITY CRITICAL THRESHOLD 100 THROTTLE 0 DATAWORDS ENUM Server Farm Name STRING 128 ENUM Profile ID STRING 64 PRIMARY CAUSE A network or server problem has occurred PRIMARY ACTION Fix the network or server problem DESCRIPTION No Resources Available to Launch ENUM Query SEVERITY CRITICAL THRE...

Page 460: ... Active State Signaling 177 section on page 10 126 DESCRIPTION ISDN Unable to Restore D Channel into In Service Active State SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS D Chan ID STRING 20 D Chan Index FOUR_BYTES D Chan Type STRING 10 PRIMARY CAUSE The BTS 10200 does not receive the Service Ack from the remote end in response to Service message to make the D Channel active PRIMARY ACTION V...

Page 461: ...nk Down Signaling 13 MAJOR SIGNALING 14 Link is Remotely Inhibited Signaling 14 MINOR SIGNALING 15 Link is Locally Inhibited Signaling 15 MINOR SIGNALING 16 Link is Congested Signaling 16 MINOR SIGNALING 17 Link Local Processor Outage Signaling 17 MINOR SIGNALING 18 Link Remote Processor Outage Signaling 18 MINOR SIGNALING 19 Link Set Inaccessible Signaling 19 MAJOR SIGNALING 20 Link Set Congestio...

Page 462: ...k Successful Signaling 58 INFO SIGNALING 59 Auto State Change for Integrated Services Digital Network Trunk Group by Media Gateway Signaling 59 MAJOR SIGNALING 60 Integrated Services Digital Network STATUS Message Containing Error Indication Received Signaling 60 WARNING SIGNALING 61 Trunk Operational State Changed by Service Message Signaling 61 INFO SIGNALING 62 Received Integrated Services Digi...

Page 463: ...G 87 H323 Message Parsing Error Signaling 87 MAJOR SIGNALING 88 H323 Message Encoding Error Signaling 88 MAJOR SIGNALING 89 Gatekeeper not Available Reachable Signaling 89 MAJOR SIGNALING 90 Alternate Gatekeeper is not Responding Signaling 90 MAJOR SIGNALING 91 Endpoint Security Violation Signaling 91 MAJOR SIGNALING 92 Invalid Call Identifier Signaling 92 MINOR SIGNALING 93 Invalid Call Reference...

Page 464: ...g 120 CRITICAL SIGNALING 121 Message Transfer Part 3 User Adapter Cannot Go Standby Signaling 121 MAJOR SIGNALING 122 Message Transfer Part 3 User Adapter Cannot Go Active Signaling 122 MAJOR SIGNALING 124 Remote Subsystem is out of Service Signaling 124 MINOR SIGNALING 125 Signaling Connection Control Part Routing Error Signaling 125 MAJOR SIGNALING 126 Signaling Connection Control Binding Failur...

Page 465: ...ital Network Signaling Gateway Inactive Signaling 157 MAJOR SIGNALING 158 Invalid Integrated Services Digital Network Interface Identification Signaling 158 WARNING SIGNALING 159 Integrated Services Digital Network User Adaptation Layer can not Go Active Signaling 159 WARNING SIGNALING 160 Integrated Services Digital Network User Adaptation Layer can not Go Standby Signaling 160 WARNING SIGNALING ...

Page 466: ...ction call has failed The primary cause of the event is that a signaling adapter has detected an error while accessing a database interface To correct the primary cause of the event restart the associated process if the database that the adapter attempted to access is not available If incompatible versions of the signaling adapter process and the database processes are present on the system correc...

Page 467: ...ssion Initiation Protocol Control Block Signaling 11 The Failed to Allocate Session Initiation Protocol Control Block alarm major indicates that a SIP control block allocation failed To troubleshoot and correct the cause of the Failed to Allocate Session Initiation Protocol Control Block alarm refer to the Failed to Allocate Session Initiation Protocol Control Block Signaling 11 section on page 10...

Page 468: ... Signaling 17 section on page 10 134 Link Remote Processor Outage Signaling 18 The Link Remote Processor Outage alarm minor indicates that the SS7 link has experienced a remote processor outage To troubleshoot and correct the cause of the Link Remote Processor Outage alarm refer to the Link Remote Processor Outage Signaling 18 section on page 10 134 Link Set Inaccessible Signaling 19 The Link Set ...

Page 469: ...apter processes is running normally Verify that the call agent platform is active Verify that the SS7 interface hardware is in service Verify that the associated SS7 signaling link is available Verify that the T13 timer is set to an appropriate level Verify that the SS7 link is not congested Unanswered Unblocking Message Signaling 26 The Unanswered Unblocking Message event serves as a warning that...

Page 470: ...message was not answered The primary cause of the event is that a CVT message was not acknowledged before the Tcvt expired for the associated CIC To correct the primary cause of the event verify that the SS7 signaling adapter processes is running normally Verify that the call agent platform is active Verify that the SS7 interface hardware is in service Verify that the associated SS7 signaling link...

Page 471: ...ardware is in service Verify that the associated SS7 signaling link is available Verify that the T13 timer is set to an appropriate level Verify that the SS7 link is not congested Trunk Locally Blocked Signaling 36 The Trunk Locally Blocked alarm minor indicates that the trunk is locally blocked To troubleshoot and correct the cause of the Trunk Locally Blocked alarm refer to the Trunk Locally Blo...

Page 472: ...of the event verify that the SS7 provisioning tables are properly configured at the circuit level A General Processing Error Encountered Signaling 49 The A General Processing Error Encountered event functions as a informational alert that a general processing error has occurred The primary cause of the event is that a general SS7 processing error occurred due to all resources being busy or an inva...

Page 473: ...ports Circuit Validation Test Aborted Signaling 54 The Circuit Validation Test Aborted event functions as an informational alert that the circuit validation test has been aborted The primary cause of the event is that the circuit specified failed a validation test due to an internal failure To correct the primary cause of the event verify the SS7 signaling adapter process and SS7 interface is oper...

Page 474: ...ormally Trunk Operational State Changed by Service Message Signaling 61 The Trunk Operational State Changed by Service Message event functions as an informational alert that the trunk operational state was changed by a service message The primary cause of the event is that the specified trunk group operational status was changed via a service message from the specified gateway To correct the prima...

Page 475: ...ing 66 The Call Agent Internal Error Because of Which Media Gateway Adapter has to Start Automatically alarm major indicates that a call agent internal error has occurred causing the restart of the MGCP signaling adapter To troubleshoot and correct the cause of the Call Agent Internal Error Because of Which Media Gateway Adapter has to Start Automatically alarm refer to the Call Agent Internal Err...

Page 476: ... improper provisioning of the network or user side To correct the primary cause of the event verify the correct provisioning at the media gateway Integrated Services Digital Network Calls Lost Due to D channel Down for Period of Time Signaling 72 The Integrated Services Digital Network Calls Lost Due to D channel Down for Period of Time event serves as a warning that calls were lost due to the D c...

Page 477: ... primary cause of the event is that communication between call agent and remote instance is faulty The event is informational and no further action is required Integrated Services Digital Network D channel Switchover for Non Facility Associated Signaling Signaling 77 The Integrated Services Digital Network D channel Switchover for Non Facility Associated Signaling event functions as an information...

Page 478: ...ing 81 section on page 10 140 Insufficient Shared Memory Pools Signaling 82 The Insufficient Shared Memory Pools alarm critical indicates that there is insufficient shared memory pools To troubleshoot and correct the cause of the Insufficient Shared Memory Pools alarm refer to the Insufficient Shared Memory Pools Signaling 82 section on page 10 140 Error While Binding to Socket Signaling 83 The Er...

Page 479: ...Gatekeeper not Available Reachable Signaling 89 The Gatekeeper not Available Reachable alarm major indicates that the gatekeeper is not available or the gatekeeper is not reachable To troubleshoot and correct the cause of the Gatekeeper not Available Reachable alarm refer to the Gatekeeper not Available Reachable Signaling 89 section on page 10 141 Alternate Gatekeeper is not Responding Signaling ...

Page 480: ... Call Processing Error Signaling 96 The Internal Call Processing Error alarm minor indicates that an internal call processing error has occurred To troubleshoot and correct the cause of the Internal Call Processing Error alarm refer to the Internal Call Processing Error Signaling 96 section on page 10 143 Insufficient Information to Complete Call Signaling 97 The Insufficient Information to Comple...

Page 481: ...To troubleshoot and correct the cause of the H323 Network Congested alarm refer to the H323 Network Congested Signaling 102 section on page 10 144 Aggregation Connection Down Signaling 103 The Aggregation Connection Down alarm major indicates that the aggregation AGGR TCP connection is down To troubleshoot and correct the cause of the Aggregation Connection Down alarm refer to the Aggregation Conn...

Page 482: ...orrect the cause of the Simplex Only Operational Mode alarm refer to the Simplex Only Operational Mode Signaling 108 section on page 10 145 Stream Control Transmission Protocol Association Failure Signaling 109 The Stream Control Transmission Protocol Association Failure alarm major indicates that the SCTP association failed To troubleshoot and correct the cause of the Stream Control Transmission ...

Page 483: ... the Signaling Gateway Process Is Out of Service Signaling 114 section on page 10 152 Invalid Routing Context Received Signaling 115 The Invalid Routing Context Received event serves as a warning that an invalid routing context was received The primary cause of the event is that the routing context was configured improperly on the CA or the SG To correct the primary cause of the event reconfigure ...

Page 484: ...y To troubleshoot and correct the cause of the Signaling System 7 Module Signaling System 7 Adapter Faulty alarm refer to the Signaling System 7 Module Signaling System 7 Adapter Faulty Signaling 120 section on page 10 153 Message Transfer Part 3 User Adapter Cannot Go Standby Signaling 121 The Message Transfer Part 3 User Adapter Cannot Go Standby alarm major indicates that the M3UA process canno...

Page 485: ...e Transaction Capabilities Application Part Binding Failure alarm refer to the Transaction Capabilities Application Part Binding Failure Signaling 127 section on page 10 155 Transaction Capabilities Application Part Reaches the Provisioned Resource Limit Signaling 132 The Transaction Capabilities Application Part Reaches the Provisioned Resource Limit event serves as a warning that the TCAP proces...

Page 486: ...ervice request Signaling System 7 Message Invalid Received Signaling 136 The Signaling System 7 Message Invalid Received event functions as an informational alert that an invalid SS7 message was received The primary cause of the event is that an invalid message was received from the line in the ISUP stack To correct the primary cause of the event verify the SSP sending the message to the CA is cor...

Page 487: ...k was Found to be in Erroneous State event functions as an informational alert that a SS7 trunk was found to be in an erroneous state The primary cause of the event is that a discrepancy exists between the local and the remote trunk states The corrective action is automatically enforced when using ANSI ISUP Unanswered Information Message Signaling 140 The Unanswered Information Message event funct...

Page 488: ... alarm refer to the Internet Protocol Interface Link to the Signaling System 7 Signaling Gateway is Down Signaling 143 section on page 10 155 All Internet Interface Links to Signaling System 7 Signaling Gateway are Down Signaling 144 The All Internet Interface Links to Signaling System 7 Signaling Gateway are Down alarm critical indicates that all IP interface links to the SS7 signaling gateway ar...

Page 489: ...DNS SRV hostname resolution to IP address is exhausted To correct the primary cause of the event add an entry to the SRV in the DNS server and fix the BTS 10200 provisioning Stream Control Transmission Protocol Association Congested Signaling 150 The Stream Control Transmission Protocol Association Congested alarm minor indicates that the SCTP association is congested To troubleshoot and correct t...

Page 490: ...tegrated Services Digital Network Signaling Gateway Down Signaling 156 The Integrated Services Digital Network Signaling Gateway Down alarm major indicates that the BTS 10200 system cannot communicate with the ISDN gateway To troubleshoot and correct the cause of the Integrated Services Digital Network Signaling Gateway Down alarm refer to Integrated Services Digital Network Signaling Gateway Down...

Page 491: ...otocol Update not Allowed for Operator Service Position System Calls Signaling 161 The Session Initiation Protocol Update not Allowed for Operator Service Position System Calls event serves as a warning that the remote switch is not allowing the BTS 10200 to send SIP UPDATE messages The UPDATE messages are mandatory in the CMSS and are used exclusively by the BTS 10200 for operator service calls o...

Page 492: ...de error at the sending switch To correct the ternary cause of the event determine if a SIP message encode error occurred at the adjacent switch and if so call the technical assistance center to determine a fix for the problem No Routing Keys are Active Signaling 166 The No Routing Keys are Active event serves as a warning that no routing keys are active The primary cause of the event is that the ...

Page 493: ...ignaling 171 The Residential Gateway Unreachable alarm minor indicates that a MGCP signaling interop error has occurred with the residential media gateway To troubleshoot and correct the primary cause of the Residential Gateway Unreachable alarm refer to the Residential Gateway Unreachable Signaling 171 section on page 10 158 Multimedia Terminal Adapter Effective Aggr Id Becomes Unavailable Due to...

Page 494: ...le Signaling 175 section on page 10 159 No Resources Available to Launch ENUM Query Signaling 176 The No Resources Available to Launch ENUM Query alarm critical indicates that no resources are available to launch the ENUM query The primary cause of the alarm is that there is internal or network congestion or that the server response is slow To troubleshoot and correct the primary cause of the alar...

Page 495: ...ssor Outage Signaling 17 MINOR SIGNALING 18 Link Remote Processor Outage Signaling 18 MINOR SIGNALING 19 Link Set Inaccessible Signaling 19 MAJOR SIGNALING 20 Link Set Congestion Signaling 20 MAJOR SIGNALING 21 Route Set Failure Signaling 21 MAJOR SIGNALING 22 Route Set Congested Signaling 22 MINOR SIGNALING 23 Destination Point Code Unavailable Signaling 23 MAJOR SIGNALING 24 Destination Point Co...

Page 496: ...he Network Signaling 95 MINOR SIGNALING 96 Internal Call Processing Error Signaling 96 MINOR SIGNALING 97 Insufficient Information to Complete Call Signaling 97 MINOR SIGNALING 98 H323 Protocol Inconsistencies Signaling 98 MINOR SIGNALING 99 Abnormal Call Clearing Signaling 99 MINOR SIGNALING 100 Codec Negotiation Failed Signaling 100 MINOR SIGNALING 101 Per Call Security Violation Signaling 101 M...

Page 497: ...erface Link to the Signaling System 7 Signaling Gateway is Down Signaling 143 MINOR SIGNALING 144 All Internet Protocol Interface Links to Signaling System 7 Signaling Gateway are Down Signaling 144 CRITICAL SIGNALING 145 One Internet Protocol Interface to Signaling System 7 Signaling Gateway is Down Signaling 145 MINOR SIGNALING 150 Stream Control Transmission Protocol Association Congested Signa...

Page 498: ...e Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Media Gateway Control Protocol The Socket Failure alarm is issued when there is a failure in creating the UDP port used by the MGCP stacks Some other application may already be active on the same UDP port and IP address to which the Call Agent MGCP stack is assigned Reconfigure the MGCP st...

Page 499: ...ce Restart this Call Agent instance using the platform start command Timeout on Internet Protocol Address Signaling 9 The Timeout on Internet Protocol Address alarm major indicates that an IP address has timed out The alarm is issued when the OptiCall is unable to communicate with a gateway To correct the primary cause of the alarm verify that the gateway is both configured for service and that it...

Page 500: ...ge lvi for detailed instructions on contacting Cisco TAC and opening a service request Failed to Allocate Session Initiation Protocol Control Block Signaling 11 The Failed to Allocate Session Initiation Protocol Control Block alarm major indicates that a SIP control block allocation failed The primary cause of the alarm is that there is not enough memory to allocate a SIP Call Control Block To cor...

Page 501: ...n ISUP message during the link failure or if the 120 second timer has expired Should the call agent fail over for any reason the state of the 120 second timer or any indication of a request for an outgoing message that could not be sent will not be preserved If the signaling links are in the failure state on the stand by side the 120 second timer will be re started however if the links should rest...

Page 502: ...g 18 The Link Remote Processor Outage alarm minor indicates that the SS7 link has experienced a remote processor outage The primary cause of the alarm is that the specified SS7 link has experienced a processor outage To correct the primary cause of the alarm monitor the network level event reports for any events associated with the processing complex used by the specified SS7 link Verify that the ...

Page 503: ...l to determine the failing component and verify its restoral to service Route Set Congested Signaling 22 The Route Set Congested alarm minor indicates that the specified route set is congested The primary of the alarm is that the specified route set is experiencing congestion To correct the primary cause of the alarm monitor event reports at the network level to determine if the traffic load on th...

Page 504: ...ge Transfer Part 3 User Adapter Cannot Go Active Signaling 122 section on page 10 154 Otherwise proceed to Step 3 Step 3 If you arrive at this step there is probably communication between the BTS 10200 and ITP at the M3UA and SCCP user adapter SUA layers and a communication problem exists between the ITP and the unavailable DPC To confirm this log on to each ITP get into enable mode and enter show...

Page 505: ...BLO or CGB message was received on the specified CIC if it is SS7 trunk Issued when SERVICE OOS message is received for ISDN trunks Issued when Reverse Make Busy rbz signal is received for CAS operator trunk No action is required The system can be manually recovered from this condition locally by controlling the effected trunks to UEQP state and back INS Auto State Change for Integrated Services D...

Page 506: ...sco TAC for assistance Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Call Agent Internal Error Because of Which Media Gateway Adapter has to Start Automatically Signaling 66 The Call Agent Internal Error Because of Which Media Gateway Adapter has to Start Automatically a...

Page 507: ...or Non Facility Associated Signaling alarm minor indicates that one of the ISDN D channels in the PRI is down The primary cause of the alarm is that one of the ISDN D channels in PRI is down To correct the primary cause of the alarm check the gateway power and the gateway connection to the PBX Trunking Gateway Unreachable Signaling 79 The Trunking Gateway Unreachable alarm major indicates that the...

Page 508: ...rvice Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Error While Binding to Socket Signaling 83 The Error While Binding to Socket alarm critical indicates that an error occurred while binding to the socket To correct the primary cause of the alarm contact Cisco TAC for corrective action Note Refer to the Obtaining Documentation and Submi...

Page 509: ...uest H323 Message Encoding Error Signaling 88 The H323 Message Encoding Error alarm major indicates that a H323 message encoding error has occurred The primary cause of the alarm is that the system was unable to encode an H323 message for sending The alarm is indicative a software bug To correct the primary cause of the alarm contact Cisco TAC Note Refer to the Obtaining Documentation and Submitti...

Page 510: ...orrect the primary cause of the alarm contact Cisco TAC Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Invalid Call Reference Value Signaling 93 The Invalid Call Reference Value alarm minor indicates that the Call ID was invalid or changed mid call The primary cause of th...

Page 511: ... detailed instructions on contacting Cisco TAC and opening a service request Insufficient Information to Complete Call Signaling 97 The Insufficient Information to Complete Call alarm minor indicates that there was insufficient information to complete a call The primary cause of the alarm is that there was not enough initial call setup information received to establish the call To correct the prim...

Page 512: ...t will not be issued in the 3 x release series H323 Network Congested Signaling 102 The H323 Network Congested alarm indicates minor that the H323 application process has depleted its resources and no more calls can be completed The primary cause of this alarm is that the H323 application processes has depleted it s resources and no more calls can be completed The high water mark has been reached ...

Page 513: ...DNS resolves to the IP addresses reserved for process on the BTS 10200 The ternary cause of the alarm is that the IP address manager is not running To correct the ternary cause of the alarm verify that the IPM process is running and check for alarms from IPM The subsequent cause of the alarm is mis configuration during installation or manual changes made after installation To correct the subsequen...

Page 514: ...g command at the BTS 10200 CLI prompt status sctp assoc id sctp assoc name If the response displays ADMIN STATE ADMIN_OOS the SCTP association has been taken administratively out of service and needs to be put back in service b Enter the following command to put the SCTP association in service control sctp assoc id sctp assoc name mode forced target state INS c If the administrative state is ADMIN...

Page 515: ...erver hrnbtsjs 1 cisco com Address 10 82 70 199 Name mgcp HRN11CA hrndevtest cisco com Addresses 10 0 5 136 10 128 1 147 The BTS 10200 configured local IP addresses are given in the Addresses line b Determine the ITP configured values of the ITP BTS 10200 IP addresses and port Log on to the ITP and get into enable mode Enter the following command show run Hit enter until the ASP configurations are...

Page 516: ...what has been provisioned in the BTS 10200 for the destination IP interfaces of the SCTP association by typing the following command show sctp association id sctp association id Information similar to the following will appear and display the destination IP addresses REMOTE_TSAP_ADDR1 10 0 1 54 REMOTE_TSAP_ADDR2 10 128 1 239 b Ping each of the destination IP addresses If one of the addresses does ...

Page 517: ...ion from the ITP side by performing the following steps a Log on to the ITP and get into enable mode b Get into configure mode by typing configure terminal c Type t he following commands to bounce the SCTP association back in service va 2651 82 config cs7 asp hrn11asp va 2651 82 config cs7 asp shut va 2651 82 config cs7 asp no shut va 2651 82 config cs7 asp end d Determine if the SCTP association ...

Page 518: ... Association Degraded One of Two Internet Protocol Connections Down Signaling 111 The Stream Control Transmission Protocol Association Degraded One of Two Internet Protocol Connections Down alarm major indicates that the SCTP association is degraded The primary cause of the alarm is that a single Ethernet connection on CA or SGP is unplugged or severed To correct the primary cause of the alarm plu...

Page 519: ...o correct the ternary cause of the alarm have the system administrator configure IP routing table Message Transfer Part 3 User Adapter Troubleshooting Procedure This alarm indicates that there is a provisioning error keeping the BTS 10200 from properly configuring the SCTP association Perform the following steps to resolve the problem Step 1 To get more information about this alarm look at the pla...

Page 520: ...down Signaling Gateway Process Is Out of Service Signaling 114 Note When a port on a ITP is removed from service using the shut command multiple Signaling 113 and 114 alarms are raised ON status When the port is recovered by cycling the ITP power all alarms raised are cleared OFF status and are removed from CURRENT_ALARM table However not all cleared alarms OFF status are displayed on the subscrib...

Page 521: ...provision the CIC Circuit Verification Response Received with Failed Indication Signaling 118 The Circuit Verification Response Received with Failed Indication alarm minor indicates that a CVR message was received with a failure indication The primary cause of the alarm is that a CIC mismatch has occurred To correct the primary cause of the alarm perform an internal test such as checking that the ...

Page 522: ... are down or if the SCTP associations are down Take corrective action according to those alarms This alarm is raised at initial startup or during failover by the BTS 10200 node that is trying to go into platform Active mode It occurs when this BTS 10200 node is unable to communicate properly with any SGs to tell them that all active call traffic should be routing towards the BTS 10200 See the Stre...

Page 523: ...onally Out of Service alarm critical indicates that the SIP trunk is operationally out of service The primary cause of the alarm is that the BTS 10200 is unable to communicate with a remote SIP party Call Agent or Proxy over a SIP or SIP T trunk To correct the primary cause of the alarm verify that the DNS resolution exists if TSAP address of the remote entity is a domain name Verify the remote en...

Page 524: ...ppressed by adding a new entry in the MGCP RETCODE ACTION table and change EP ACTION to RESET NONE Note The following additional troubleshooting information is applicable to Release 5 0 MR2 and above If the VXSM is OOS at the GW side a 501 error message for CRCX AUEP may be transmitted This generally occurs if there is resource state mismatch between the BTS 10200 ACTV IDLE and the VXSM DOWN For a...

Page 525: ... Integrated Services Digital Network Signaling Gateway Inactive Signaling 157 The Integrated Services Digital Network Signaling Gateway Inactive alarm major indicates that a shutdown command has been executed in the application server on the ISDN gateway side No action needed The application server will be automatically recovered Session Initiation Protocol Server Group Element Operationally Out o...

Page 526: ...g 170 The Residential Gateway Endpoints are out of Service at the Gateway alarm minor indicates that the residential gateway has been administratively taken OOS using the command at the gateway To troubleshoot and correct the primary cause of the Residential Gateway Endpoints are out of Service at the Gateway alarm bring the residential gateway administratively into INS using the command at the ga...

Page 527: ... network or server problem has occurred To troubleshoot and correct the cause of the alarm fix the network or server problem ENUM Server Farm Unavailable Signaling 175 The ENUM Server Farm Unavailable alarm critical indicates that a network or server problem has occurred To troubleshoot and correct the cause of the alarm fix the network or server problem No Resources Available to Launch ENUM Query...

Page 528: ...10 160 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Chapter 10 Signaling Troubleshooting Troubleshooting Signaling Alarms ...

Page 529: ...stics Alarms Provides the information needed to troubleshoot and correct the statistics alarms Statistics Events and Alarms This section provides a brief overview of all of the statistics events and alarms for the Cisco BTS 10200 Softswitch in numerical order Table 11 1 lists all of the statistics events and alarms by severity Note Click the statistics message number in Table 11 1 to display infor...

Page 530: ...nished Statistics 3 section on page 11 13 DESCRIPTION Test Report SEVERITY Information INFO THRESHOLD 10000 THROTTLE 0 DESCRIPTION Call Agent Measurement Collection Started SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Start Time STRING 40 PRIMARY CAUSE Indicates that the 15 minute traffic measurement collection process has started on the Call Agent PRIMARY ACTION No action is necessary DESCRIP...

Page 531: ...Data Link Monitor Feature Server Measurement Collection Started POTS CTX TDM Feature Server Measurement Collection Started SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Start Time STRING 40 PRIMARY CAUSE Indicates that the 15 minute traffic measurement collection process has started on the plain old telephone service POTS Centrex Tandem Feature Server PRIMARY ACTION No action is necessary DESCR...

Page 532: ...ature Server Measurement Collection Started AIN Feature Server Measurement Collection Started SEVERITY INFO THRESHOLD 100 THROTTLE 0 DATAWORDS Start Time STRING 40 PRIMARY CAUSE Indicates that the 15 minute traffic measurement collection process has started on the Advanced Intelligent Network AIN feature server PRIMARY ACTION No action is necessary DESCRIPTION Advanced Intelligent Network Feature ...

Page 533: ...NING THRESHOLD 100 THROTTLE 0 DATAWORDS Originating Process STRING 40 Traffic Mgr Msg Type STRING 40 Traffic Agent Msg Type STRING 30 PRIMARY CAUSE The originating process for Call Agent CA Feature Server FS or Element Management System EMS is not in active state or is shut down PRIMARY ACTION Check for other alarms and events generated from this component CA FS or EMS SECONDARY CAUSE The hub is d...

Page 534: ...ment Table Structured Query Language Read Error Measurement Table SQL Read Error SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Measurement Table Na STRING 40 PRIMARY CAUSE There is no database connection or the connection is faulty PRIMARY ACTION Check to see if there are any other events generated that indicate there is a database problem SECONDARY CAUSE The traffic measurement table s are ...

Page 535: ...ent Table Structured Query Language Write Error Measurement Table SQL Write Error SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Measurement Table Na STRING 40 PRIMARY CAUSE There is no database connection or the connection is faulty PRIMARY ACTION Check to see if there are any other events are generated that indicate there is a database problem SECONDARY CAUSE The traffic measurement table s...

Page 536: ...ssued when the traffic measurement subsystem on the call agent or the feature server encounters difficulties when it tries to collect measurement values from one of the processes PRIMARY ACTION Execute a status command for the affected call agent or feature server SECONDARY CAUSE The originating process for CA FS or EMS is not in active state or is shut down SECONDARY ACTION If the status report i...

Page 537: ...e Error Schema Inconsistency Statistics 12 section on page 11 17 DESCRIPTION Measurement Handshake Error Schema Inconsistency SEVERITY MAJOR THRESHOLD 5 THROTTLE 0 DATAWORDS Schemas Out of Synchronization STRING 64 PRIMARY CAUSE Counters were added or deleted from the schema in Oracle but not in the call agent PRIMARY ACTION Add or delete the counters on the CA SECONDARY CAUSE Load is installed in...

Page 538: ...R THRESHOLD 1 THROTTLE 10 DATAWORDS TMM Error FOUR_BYTES PRIMARY CAUSE Unable to initialize shared memory PRIMARY ACTION Reconfigure and restart system SECONDARY CAUSE Unable to attach to shared memory SECONDARY ACTION Restart offending process TERNARY CAUSE Shared memory table overflow TERNARY ACTION Reconfigure restart or fix problematic application SUBSEQUENT CAUSE Shared memory exhaustion SUBS...

Page 539: ...o the Trunk Group has Reached the MDII Alarm Threshold Statistics 16 section on page 16 DESCRIPTION Threshold Crossing Alert SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Description STRING 256 PRIMARY CAUSE A threshold crossing has occurred PRIMARY ACTION Reduce the provisioning workload DESCRIPTION Trunk Group has Reached the MDII Alarm Threshold SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DAT...

Page 540: ... Statistics 3 INFO STATISTICS 4 Plain Old Telephone Service Centrex Telecommunications Data Link Monitor Feature Server Measurement Collection Started Statistics 4 INFO STATISTICS 5 Plain Old Telephone Service Centrex Telecommunications Data Link Monitor Feature Server Measurement Collection Finished Statistics 5 INFO STATISTICS 6 Advanced Intelligent Network Feature Server Measurement Collection ...

Page 541: ...d Telephone Service Centrex Telecommunications Data Link Monitor Feature Server Measurement Collection Finished Statistics 5 The Plain Old Telephone Service Centrex Telecommunications Data Link Monitor Feature Server Measurement Collection Finished event functions as an informational alert that the POTS CTX TDM feature server measurement collection has finished The event is informational and no fu...

Page 542: ...n the Call Agent and the EMS and there will be additional events being issued These other events will indicate the nature of the communication difficulties The repair procedures for the other event reports should be followed to correct the Call Agent EMS communication issue Check for other alarms and events generated from this component CA FS or EMS If the originating process is down try to bring ...

Page 543: ...has failed The primary cause of the event is that the report is issued when the traffic measurement subsystem on the call agent or feature server encounters difficulties when it tries to collect measurement values from one of the processes To correct the primary cause of the event execute a status command for the affected call agent or feature server The secondary cause of the event is that the or...

Page 544: ...age 11 17 MDII Trunk Statistics 14 The MDII Trunk event serves as a warning that the calls on the MDII trunk termination are not being successfully completed The BTS 10200 system may take the MDII trunk out of service if it does not take the full trunk group out of service Threshold Crossing Alert Statistics 15 The Threshold Crossing Alert alarm minor indicates that a threshold crossing has occurr...

Page 545: ...Module Application Programming Interface Failure alarm major indicates that the TMM API failed The primary cause of the alarm is that the system is unable to initialize the shared memory To correct the primary cause of the alarm reconfigure and restart the system The secondary cause of the alarm is that a process is unable to attach to the shared memory To correct the secondary cause of the alarm ...

Page 546: ...11 18 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Chapter 11 Statistics Troubleshooting Troubleshooting Statistics Alarms ...

Page 547: ...the system events Troubleshooting System Alarms Provides the information needed to troubleshoot and correct the system alarms System Events and Alarms This section provides a brief overview of all of the system events and alarms for the Cisco BTS 10200 Softswitch in numerical order Table 12 1 lists all of the system events and alarms by severity Note Click the system message number in Table 12 1 t...

Page 548: ...rvice Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Test Report SEVERITY Information INFO THRESHOLD 100 THROTTLE 0 PRIMARY CAUSE This is a test report for the SYSTEM category PRIMARY ACTION No action is required DESCRIPTION Inter Process Communication Queue Read Failure IPC Queue Read Failure SEVERITY MINOR THRESHOLD 100 THR...

Page 549: ...cting Cisco TAC collect the message pool and message queue statistics To collect the statistics use the pdm CAxxx script in the opt OptiCall CAxxx bin directory Example pdm CA146 1 IPC Controls 2 Message Pool Stats 6 MEssage Queue Stats Also use the top script to collect the current CPU usage DESCRIPTION Inter Process Communication Message Allocate Failure IPC Message Allocate Failure SEVERITY MIN...

Page 550: ...tacting Cisco TAC and opening a service request DESCRIPTION Inter Process Communication Message Send Failure IPC Message Send Failure SEVERITY MINOR THRESHOLD 50 THROTTLE 0 DATAWORDS Error Code FOUR_BYTES Destination Process FOUR_BYTES Message Number FOUR_BYTES Location Tag STRING 30 PRIMARY CAUSE The process for which the message is intended is not running PRIMARY ACTION Check to ensure that all ...

Page 551: ...tation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Unexpected Inter Process Communication Message Received Unexpected IPC Message Received SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Source Process Type ONE_BYTE Source Thread Type ONE_BYTE Message Number TWO_BYTES Location Tag STRING 30 PRI...

Page 552: ...n and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Index List Remove Error IDX List Remove Error SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS List Name STRING 20 Index of Entry Being FOUR_BYTES Location Tag STRING 30 PRIMARY CAUSE An internal error has occurred PRIMARY ACTION If the problem persist...

Page 553: ...ge 12 16 Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Timer Start Failure SEVERITY WARNING THRESHOLD 100 THROTTLE 0 DATAWORDS Timer Type STRING 20 Location Tag STRING 30 PRIMARY CAUSE Process was unable to start a platform timer PRIMARY ACTION If the problem...

Page 554: ...Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request DESCRIPTION Index Table Add Entry Error IDX Table Add Entry Error SEVERITY MINOR THRESHOLD 100 THROTTLE 0 DATAWORDS Table Name STRING 20 Index of Entry Being FOUR_BYTES Error Code FOUR_BYTES Location Tag STRING 30 PRIMARY CAUSE An internal error has occurred PRIMARY ACTION If the pro...

Page 555: ... for Queue Depth FOUR_BYTES Low Mark for Queue Depth FOUR_BYTES PRIMARY CAUSE Messages are flooding from a malfunctioning network element PRIMARY ACTION Check the messages to process SECONDARY CAUSE Resource congestion or slow processing of messages from queue has occurred SECONDARY ACTION Check the process and the system resources You may need to failover DESCRIPTION Multiple Readers and Multiple...

Page 556: ...th Reached SEVERITY MAJOR THRESHOLD 100 THROTTLE 0 DATAWORDS Throttle Mark for Queue Depth FOUR_BYTES Throttle Clear Mark for Queue De FOUR_BYTES PRIMARY CAUSE Inbound network messages are arriving at a rate much higher than the processing capacity is able to handle PRIMARY ACTION Determine the cause of increase in inbound network traffic and try to control the traffic externally SECONDARY CAUSE R...

Page 557: ... has failed To troubleshoot and correct the cause of the Inter Process Communication Message Allocate Failure alarm refer to the Inter Process Communication Message Allocate Failure System 3 section on page 12 15 Table 12 2 BTS 10200 System Events Event Type Event Name Event Severity SYSTEM 1 Test Report System 1 INFO SYSTEM 2 Inter Process Communication Queue Read Failure System 2 MINOR SYSTEM 3 ...

Page 558: ...ndex List Insert Error alarm minor indicates that an error has been inserted in the index list To troubleshoot and correct the cause of the Index List Insert Error alarm refer to the Index List Insert Error System 6 section on page 12 15 Index List Remove Error System 7 The Index List Remove Error alarm minor indicates that an index list remove error has occurred To troubleshoot and correct the ca...

Page 559: ...tiple Writers Maximum Q Depth Reached alarm critical indicates that the multiple readers and multiple writers MRMW maximum queue depth has been reached To troubleshoot and correct the cause of the Multiple Readers and Multiple Writers Maximum Q Depth Reached alarm refer to the Multiple Readers and Multiple Writers Maximum Q Depth Reached System 13 section on page 12 17 Multiple Readers and Multipl...

Page 560: ...ction on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Table 12 3 BTS 10200 System Alarms Alarm Type Alarm Name Alarm Severity SYSTEM 2 Inter Process Communication Queue Read Failure System 2 MINOR SYSTEM 3 Inter Process Communication Message Allocate Failure System 3 MINOR SYSTEM 4 Inter Process Communication Message Send Failure System 4 MINOR SYSTEM 6 ...

Page 561: ... Message Pool Stats 6 MEssage Queue Stats Also use the top script to collect the current CPU usage Inter Process Communication Message Send Failure System 4 The Inter Process Communication Message Send Failure alarm minor indicates that the IPC message send has failed The primary cause of the alarm is that the process for which the message is intended is not running To correct the primary cause of...

Page 562: ...isco TAC Note Refer to the Obtaining Documentation and Submitting a Service Request section on page lvi for detailed instructions on contacting Cisco TAC and opening a service request Index Update Registration Error System 10 The Index Update Registration Error alarm minor indicates that an index update registration error has occurred The primary cause of the alarm is that an application unsuccess...

Page 563: ...cess and system resources The system may need to be failed over Multiple Readers and Multiple Writers Queue Reached Low Queue Depth System 14 The Multiple Readers and Multiple Writers Queue Reached Low Queue Depth alarm minor indicates that the MRMW queue has reached the low queue depth threshold The primary cause of the alarm is a high rate of messages from the network To correct the primary caus...

Page 564: ...12 18 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Chapter 12 System Troubleshooting Troubleshooting System Alarms ...

Page 565: ...s section describes the procedures to locate the cause of the problem These procedures describe an iterative process that must be performed in order When a problem is found and resolved perform the procedure again from the beginning This section describes how to perform the following procedures Verify the Stream Control Transmission Protocol Association Status page 13 2 Verify the Configuration pa...

Page 566: ...he SCTP association name provisioned on the BTS 10200 Information similar to the following is displayed c2651 48 show cs7 asp name TB2 PRI AIN Effect Primary ASP Name AS Name State Type Rmt Port Remote IP Addr SCTP TB2 PRI AIN TB02 LNP NC active SUA 12520 10 89 225 209 323 TB2 PRI AIN TB02 SUALNP shutdown SUA 12520 10 89 225 209 323 TB2 PRI AIN TB02 800A NC active SUA 12520 10 89 225 209 323 TB2 P...

Page 567: ...sctp_prof REMOTE_PORT 14001 REMOTE_TSAP_ADDR1 10 89 232 48 PLATFORM_ID FSAIN520 DSCP NONE IP_TOS_PRECEDENCE FLASH LOCAL_RCVWIN 64000 MAX_INIT_RETRANS 5 MAX_INIT_RTO 1000 STATUS INS ULP XUA Reply Success Entry 1 of 1 returned c2651 48 show cs7 sua Sigtran SUA draft version 14 SUA Local port 14001 State active SCTP instance handle 2 Local IP address 10 89 232 48 Number of active SUA peers 8 Max numb...

Page 568: ...MaxRestartExceededInDuplex yes EndPlatformIfMaxRestartExceededWhenMateFaulty yes Restart rate n m where n Max restarts m interval in hours RestartRate 0 1 b On the ITP enter show run begin asp name Information similar to the following is displayed c2651 48 show run begin TB2 PRI AIN cs7 asp TB2 PRI AIN 12520 14001 sua remote IP 10 89 225 209 remote IP 10 89 226 209 c If the sctpport on the BTS 102...

Page 569: ...P the SCTP association will be taken down by the ITP Make sure the AS using the ASP is provisioned before bringing up the SCTP association corresponding to the same ASP If the ASP is used by any AS continue to the next section Otherwise correct it and continue Verify the Internet Protocol Transfer Point T1 Card Provisioning Enter show controller t1 slot bay port on the ITP Verify if trunk level on...

Page 570: ... Queueing strategy fifo Output queue 0 40 0 drops input queue 0 75 23 drops 30 second input rate 0 bits sec 0 packets sec 30 second output rate 0 bits sec 0 packets sec 1912000 packets input 9866017 bytes 0 no buffer Received 0 broadcasts 0 runts 17 giants 0 throttles 3356 input errors 128 CRC 2641 frame 0 overrun 0 ignored 587 abort 1163961 packets output 13234709 bytes 0 underruns 0 output error...

Page 571: ... available on the ITP by entering the following command show cs7 linkset ls name Information similar to the following is displayed c2651 48 show cs7 linkset lsn ls_to_mgts_lic_10 apc 1 101 0 state avail avail links 1 1 SLC Interface Service PeerState Inhib 00 Serial0 0 0 avail Step 2 If the status is not available and at least one of the serial interfaces is available the problem could be the poin...

Page 572: ...utes 6 Destination Prio Linkset Name Route 1 8 1 24 INACC 1 ls_to_mgts_lic_10 UNAVAIL 1 12 1 24 acces 5 ls_to_mgts_lic_10 avail 1 101 0 24 acces 1 ls_to_mgts_lic_10 avail 7 44 120 24 acces 1 ls_to_inet12_pod_1 avail 7 44 121 24 acces 1 ls_to_inet12_pod_1 avail 7 212 112 24 acces 1 ls_to_inet12_pod_1 avail Routing table XUA Destination Type 7 2 1 24 acces AS 7 2 3 24 acces AS 7 44 1 24 acces AS 7 4...

Page 573: ...ates that database jobs 3 4 5 and 6 are broken the database administrator needs to restart the jobs using the following procedure Step 1 Login to oracle su oracle Step 2 Restart database job 3 java dba rep RepAdmin enable job 3 Step 3 Restart database job 4 java dba rep RepAdmin enable job 4 Step 4 Restart database job 5 java dba rep RepAdmin enable job 5 Step 5 Restart database job 6 java dba rep...

Page 574: ...13 10 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Chapter 13 Network Troubleshooting Troubleshooting a Network Failure ...

Page 575: ...on responses Protocol Troubleshooting Provides the troubleshooting information for resolving BTS 10200 protocol problems File Configuration bts properties Provides instructions for editing and configuring the bts properties file Privacy Screening Troubleshooting Provides instructions for troubleshooting privacy screening Call Agent Controlled Mode for RFC 2833 DTMF Relay Troubleshooting Describes ...

Page 576: ...IIF a part of the Alliance for Telecommunications Industry Solutions ATIS organization has published a document ATIS NIIF 0017 that includes detailed steps that service providers should follow when LNP problems are encountered The document is titled Guidelines for Reporting Local Number Portability Troubles in a Multiple Service Provider Environment and it is available at https www atis org docsto...

Page 577: ...old service provider the NPAC SMS logs the failure to respond and allows the new service provider to proceed with activation when the new service provider due date is reached If the NPAC SMS does not receive a concurring notification from the new service provider the NPAC SMS logs the failure to respond cancels the request and notifies both service providers of the cancellation If the service prov...

Page 578: ...se audits are executed through queries to the appropriate service provider s network and corrected by means of downloads to those same networks Local service providers are also responsible for comparing database extracts of Subscription data written to an File Transfer Protocol FTP site by the NPAC SMS with their own versions of the same Subscription data The NPAC SMS selects a random sample of ac...

Page 579: ...NOTIFY The feature identifier can be any unique string of up to 16 ASCII characters chosen by the service provider If you are not sure of the name used in your system for this feature use the SHOW FEATURE command and view the system response to find the name Example SHOW FEATURE SERVER SHOW FEATURE FNAME ALERT_NOTIFY SHOW CA CONFIG TYPE DEFAULT OFFICE SERVICE ID SHOW SERVICE ID the value of the de...

Page 580: ...e product documentation The BTS 10200 does not send any provisioning or status control commands to the 3PTYFS Step 8 Verify that the 3PTYFS and peripheral devices are operating properly according to the applicable product documentation Command Responses This section describes success and failure responses to commands as well as values for the term reason and trunk reason responses Note In this sec...

Page 581: ...hile in trunk table ADM fail while looking to find trunk index ADM fail while getting MGW administration state ADM fail while getting trunk group administration state ADM fail while looking for MGW index ADM administration state invalid ADM failed to allocate inter process communication IPC message s ADM failed to dispatch IPC message s ADM operational state invalid ADM MGW s state change and pend...

Page 582: ... 5 0 x OL 8723 19 Chapter 14 General Troubleshooting Command Responses ADM found entity unequipped in initial state ADM operation not allowed because D Channel s is down The H 323 Gateway was not found in database management DBM ADM found unknown failure reason s ...

Page 583: ...r error Facility failure Failure of a grouping of trunks Incompatible protocol version Insufficient bandwidth at this time Insufficient bandwidth Internal consistency in local connection options Internal hardware failure Invalid call ID Invalid conn identifier Invalid or unsupported command parameter Invalid or unsupported LocalConnectionOptions Loss of lower connectivity Loss of lower layer conne...

Page 584: ...on could not be executed because the endpoint does not have sufficient resources at this time The transaction could not be executed because the endpoint does not have enough resources available permanent condition The transaction could not be executed because the endpoint is restarting The transaction could not be executed due to a transient error The transaction could not be executed due to some ...

Page 585: ... mode FORCED MAINT BLOCK trunk termination is manually controlled OOS controlled mode GRACE MAINT BUSY trunk termination is in maintenance state controlled to MAINT MAINT OOS trunk termination is manually controlled OOS There is no difference between this and a BLOCK NON FAULTY Not blocked available for service OUTGOING_RESTRICTED the outgoing call is not allowed SIGNALLING FAULT Cannot exchange m...

Page 586: ...or the ITP informs that the user part is available The first incoming message will also clear this response ACT_LOC_VALIDATE This is set when the a diagnostic command is issued from the CLI to perform a circuit validation and remains set until circuit validation test CVT and CVR messages are exchanged with the remote switch ACT_LOC_COTTEST This is set when the a diagnostic command is issued from t...

Page 587: ...nk is available for calls REMOTE_GRACE Trunk is blocked remotely because of a CLI command on the remote switch REMOTE_FORCE Trunk is blocked remotely because of a hardware failure on the remote switch RESERVE_SPARE1 Reserved for future use RESERVE_SPARE2 Reserved for future use TERM_GRACE Trunk is gracefully blocked because of an RSIP from the MGW ...

Page 588: ...ected The transaction could not be executed because the command contained an unrecognized extension The transaction could not be executed because the gateway is not equipped to detect one of the requested events The transaction could not be executed because the gateway is not equipped to generate one of the requested signals The transaction could not be executed because the gateway cannot send the...

Page 589: ...g url http www cisco com univercd cc td doc product wireless moblwrls itp This section contains the following sections Cisco Internet Protocol Transfer Point Troubleshooting Procedures Viewing BTS 10200 Logs Troubleshooting Transaction Capabilities Application Part with the Query Verification Tool Cisco Internet Protocol Transfer Point Troubleshooting Procedures The following procedures are useful...

Page 590: ...s7 as Retrieves the application server AS status show cs7 asp Retrieves the application service provider ASP status show cs7 linkset Retrieves the Signaling System 7 SS7 linkset status show cs7 route Retrieves the SS7 route status show cs7 group state Retrieves the signaling gateway SG group status Controlling Internet Protocol Transfer Point Resources Change the administrative state of an ITP res...

Page 591: ...to display and clear alarms To display all the currently active alarms enter the following command at a CLI prompt show alarm To display all alarms of a specific type enter show alarm type alarm type To clear an alarm enter the following command clear alarm id alarm id For a detailed description of the CLI commands that are used please see the BTS 10200 Command Line Interface Reference Guide Relea...

Page 592: ... when searching the BTS 10200 logs M3UA Interface Module MIM CFG SCTP Display how the SCTP has been configured at startup MIM PDU Trace incoming and outgoing ISUP messages at the MIM layer MIM STATUS DPC Display how the DPC status has changed in the system MIM STATUS SCTP Display how the SCTP status has changed in the system MIM PLATFORM Determine if a platform state change has been issued to the ...

Page 593: ...tify the following problems Missing Call Agent CA CONFIG table Missing SLHR PROFILE table Missing Service Logic Host Route SLHR table Missing DPC table Missing OPC table Missing SUBSYSTEM PROFILE table Missing SUBSYSTEM table Missing signaling connection control part SCCP network NW table Missing SCCP ROUTE table Missing ROUTING KEY table Missing SG group GRP table Missing SG table Missing SGP tab...

Page 594: ...n Subsystem Failure Unequipped User Network Failure Network Congestion Unqualified Error In Message Transport Destination Cannot Perform Reassembly Signaling Connection Control Part Failure Segmentation Not Supported Segmentation Failure Query Verification Tool Timeout Command Line Interface Timeout No Translation for an Address of Such Nature No Translation for this Specific Address Layer SCCP Ve...

Page 595: ... ANSI88 ANSI92 Location SCP Cause The SCP is down or the subsystem of the SCP is down Solution Verify that the SCP point code is correct Layer SCCP Subsystem TCAP Version ITU88 ITU92 ITU96 ANSI88 ANSI92 Location SCP Cause The SCCP user is not equipped Solution Verify that the SCP point code is correct Layer MTP3 MTP2 MTP1 or SCTP Version ITU88 ITU92 ITU96 ANSI88 ANSI92 for NTP3 2 1 Internet Engine...

Page 596: ...y indicates insufficient central processing unit CPU power Hardware needs to be upgraded or more BTS 10200 need to be added to offload traffic Layer SCCP Version ITU88 ITU92 ITU96 ANSI88 ANSI92 Location STP or SCP Cause Unknown Solution Contact the support team or developer for assistance Layer SCCP Version ITU92 Location STP Cause There was a failure in message transportation Solution Contact the...

Page 597: ...tart the platform providing the service Feature Server for AIN services FSAIN or Feature Server for POTS Tandem and Centrex services FSPTC Layer SCCP Version ITU96 ANSI92 Location STP Cause The maximum hop count is exceeded during the message routing Solution Make sure the hop count value provisioned in the SCCP NW table is not too small Verify that the SS7 network provider does not have any route...

Page 598: ...eeper Has Placed Routing Into a Loop Outgoing H 323 Calls Routed to Incorrect Endpoint When Using Registration Admissions and Status Outgoing H 323 Calls Routed to Incorrect Endpoint When Using Direct Signaling Registration Admissions and Status Still Used When Outgoing H 323 Call is Provisioned to Use Direct Signaling Location Local Cause The SCP filed to respond or the TSA is out of service Solu...

Page 599: ...s Solution Use CLI to place the trunk_grp in service as follows CLI control trunk_grp id 8991 target state ins mode forced Outgoing H 323 Gateway is Out of Service WARNING 10 47 44 793 H3A3 01 1 Main Call failure H3A BCM sent for csaid 41 callid 0 reason H323 gateway gk seem to be OOS h3a_sig_sai c 1525 This is an indication that the outgoing H 323 gateway is out of service Verify the status of ou...

Page 600: ...nvolved If an request rejected RRJ message is received by gateway with reject reason of invalidAlias this is symptomatic of an alias list being provided to the gatekeeper but the gatekeeper not configured to be responsible for the prefix associated with each of the aliases H3A3 value RasMessage gatekeeperReject H3A3 H3A3 requestSeqNum 11 H3A3 protocolIdentifier 0 0 8 2250 0 3 H3A3 rejectReason inv...

Page 601: ...10085 bts os src bts_debug c 200 4 10 52 44 824 H3A3 01 1 H_EVT state 0 cch323_traverse_enum_contact_list in gw src cch323_gw_api c 4652 bts os src bts_debug c 200 I3 10 52 44 824 H3A3 01 1 Main Rel message from STACK H3A for callid 5 cause code 16 h3a_sig_sai c 1549 This is an indication that the outgoing H 323 gateway needs to use Registration Admissions and Status RAS to complete the call but b...

Page 602: ...ether configuration is correctly registered with the GK Step 1 Log on to gatekeeper Step 2 show gatekeeper endpoints Step 3 Examine the display of the GK which should look similar to the example below Step 4 Verify that both the CA and H 323 Protocol H323 GW are registered with the GK with Flags E as shown in the above example No Matching Dial Plan Found on Incoming H 323 Trunk Group I2 14 45 24 2...

Page 603: ... in a admission confirmation ACF A registered E 164 address in this case highly unlikely A static route configured has been observed Default routing highest frequency of problem Outgoing H 323 Calls Routed to Incorrect Endpoint When Using Registration Admissions and Status I3 14 25 17 215 H3A3 01 1 Main Rel message from STACK H3A for callid 8 cause code 3 h3a_sig_sai c 1549 After the SETUP message...

Page 604: ...this case the remote endpoint does not service the destination address cause code 3 Verify the value assigned to the SOFTSW_TSAP_ADDR for the outgoing H 323 trunk group CLI show trunk_grp id 8991 Reply Success Entry 1 of 1 returned ID 8991 CALL_AGENT_ID CA146 TG_TYPE H323 SOFTSW_TSAP_ADDR 10 89 227 119 TG_PROFILE_ID CHINA STATUS INS DIRECTION BOTH SEL_POLICY ASC GLARE SLAVE ALT_ROUTE_ON_CONG N SIG...

Page 605: ...nism is to try RAS to determine the remote endpoint Verify the configuration for the outgoing trunk group with CLI CLI show trunk_grp id 8991 Reply Success Entry 1 of 1 returned ID 8991 CALL_AGENT_ID CA146 TG_TYPE H323 TG_PROFILE_ID CHINA STATUS INS DIRECTION BOTH SEL_POLICY ASC GLARE SLAVE ALT_ROUTE_ON_CONG N SIGNAL_PORTED_NUMBER N DIAL_PLAN_ID cdp1 DEL_DIGITS 0 OPER_STATUS NF QOS_ID china_qos1 T...

Page 606: ...ate of Trunk group in service for the trunk group non facility associated signaling NFAS or D channel backup should also show ISDN_DCHAN_STATE_INS for the D channel operational status then If the response is Trunk group out of service go to Step 1 If the response is Trunk group restore session fail normal switchover or maint go to Step 2 If the response is Trunk group restore establish fail normal...

Page 607: ...the Call Agents actually have the IP addresses or DNS names specified d Can the mgw tsap addr be pinged from both Call Agents e Does the trunk grp isdn profile exist show isdn tg profile ID Keep in mind that the profile ID is case sensitive f Snoop can be used to see if the gateway and Call Agent are talking in order to establish the session Snoop can be used to eliminate whether the problem is on...

Page 608: ...DN_DCHAN_STATE_WAIT ISDN_DCHAN_STATE_MB This means the states at the far end are sending a SERVICE_ACK to the BTS 10200 service message for primary rate interface PRI Things to Check Make sure the far end is sending a SERVICE_ACK There are some other trunk group states that happen after the ISDN backhaul is successfully established Step 5 Trunk group states for FAS Trunk group down session set fai...

Page 609: ... reason available TRUNK STATIC STATE ACTV TRUNK DYNAMIC STATE TRNS TRUNK REASON NON_FAULTY The above status shows that the trunk is active in idle mode and ready for call use If the trunk static state is set to locally blocked go to the Trunk Static State Is Set to Locally Blocked section If the trunk static state is set to transient TRNS go to the Trunk Static State Is Set to Transient section If...

Page 610: ...trunk group OPER STATE is IN SERVICE see the How to Troubleshoot When the Integrated Services Digital Network Trunk Group Fails to Restore section in case of any issues Clear any SIGNALLING FAULT states Trunks must be in active or idle mode and ready for use Step 3 If TRUNK REASON is set to TERM_FAULT then the terminations are in FAULTY states Verify the IP connectivity from the CA to the gateway ...

Page 611: ...nation to OOS then control trunk termination to INS in case there is an error on the PBX side Also set the isdn service supp token in the ISDN trunk group profile per the procedures supported by the PBX Termination Status Is Faulty The TERM_STATUS_FAULTY message indicates an MGCP problem Things to Check Is the media gateway MGW profile set up for the trunk Is the MGW in the in service operational ...

Page 612: ...ock timezone CDT 6 network clock base rate 56k network clock select 3 T1 0 network clock select 1 T1 1 network clock select 2 system SCB IP subnet zero IP domain name ipclab cisco com IP name server 10 89 224 1 IP audit notify log IP audit po max events 100 backhaul session manager set isdn1 client ft group group1 set isdn1 group group2 set isdn1 session group group1 10 89 225 223 9000 10 89 227 2...

Page 613: ... 23 bandwidth 64000 no IP address no logging event link status isdn switch type primary ni isdn protocol emulate network isdn incoming voice voice isdn bind l3 backhaul grp1 no cdp enable IP classless IP route 0 0 0 0 0 0 0 0 10 89 227 254 no IP http server IP pim bidir enable access list 1 permit 10 0 24 0 0 0 0 255 call rsvp sync voice port 1 23 mgcp mgcp call agent mgcp SYS01CA ipclab cisco com...

Page 614: ...ting Guide Release 5 0 x OL 8723 19 Chapter 14 General Troubleshooting Protocol Troubleshooting dial peer voice 1 pots application mgcpapp port 1 23 line con 0 line aux 0 line 2 3 line vty 0 4 password xxxxxxxx login ntp server 10 89 227 254 end ...

Page 615: ...p id TG ID number call agent id CA ID If the trunk group status has still not changed to ADMIN_INS even though the commands in Steps 1 and 2 were successful then it means that communication to the MGW has been lost Notify your system administrator and have the problem corrected before continuing Step 4 Check trunk termination and trunk status The single command shown here queries both the trunk te...

Page 616: ...h the Call Agent into one of two states ACTIVE STANDBY STANDBY ACTIVE When the control command is entered on a Call Agent connected to an ISDN trunk group D channel the Call Agent switchover time is approximately 20 seconds During all but the first 6 seconds of this switchover time the ISDN D channel is temporarily down However the D channel comes back up and the ISDN trunk group automatically ret...

Page 617: ...pted without Cisco TAC support or supervision Table 14 1 Default Content of the bts properties File Parameter Variable reportDir opt ems report reportDirSize 50000000 reportSuffix html logDefLevel INFO logMaxFileSize 50000000 logMinFileSize 1000 logDefFileSize 4000000 logDir opt ems log logName BtsEms logSuffix log logBakupSuffix bak usersDir opt ems users etcDir opt ems etc ftpErrorsAllowed 3000 ...

Page 618: ... properties file to enable debugging To enable the debugging mode and to collect the debugging information for a CLI session use the following instruction Note Modification of the bts properties file should not be attempted without Cisco TAC support or supervision Step 1 Open the opt ems etc bts properties file Step 2 Set logDefLevel DEBUG valid logging options are INFO DEBUG ERROR WARNING FATAL S...

Page 619: ... the softsw tsap addr is set to an IP address and not a domain name For the trunk group against the DN that is mapped to the PS_MANAGE App verify that the softsw tsap addr is set to a domain name and not an IP address Symptom Three Privacy Screening is not able to collect digits or record with a Cisco 2421 Solution Three In the Cisco 2421 media gateway execute the following command mgcp rtp payloa...

Page 620: ...FC 2833 DTMF relay See the MGW vendor documentation for information on MGW features Resolution If the MGW does not support CA controlled RFC 2833 DTMF relay set dtmf telephone event enabled N in the QoS table associated with the endpoint No DTMF Relay Involving H 323 Endpoints Problem RFC 2833 DTMF relay does not work on the H 323 gateway or endpoint Symptom DTMF tones do not go through Diagnosis ...

Page 621: ...d timeout has increased by a factor of 2 Symptoms Increased memory usage on slow networks and during major outages of MGCP devices Diagnosis Memory usage returns to normal when network connectivity to MGCP devices is restored Resolution Great care needs to be taken when provisioning MGCP T HIST and MGCP T MAX parameters in ca config table If necessary reduce the values of MGCP T HIST and or MGCP T...

Page 622: ...solution If the endpoint reports e on in its capabilities but provisioned echo cancellation settings are ignored verify that EC SUPP is enabled for the MGCP device in the mgw profile table see the Provisioning MGCP Command Timeout and QoS Parameters section Upgrade the MGCP device firmware Contact the device manufacturer if the MGCP device advertises support for silence suppression echo cancellati...

Page 623: ...ou hear a reorder tone from a SIP phone ensure the status of the BTS announcement table is all of the following btsadmin status tt tgn id 889 cic all 889 1 ADMIN_INS TERM_ACTIVE_IDLE ACTV IDLE NON_FAULTY If you hear a click from an MGCP phone ensure the status of the BTS announcement table is all of the following btsadmin status tt tgn id 889 cic all 889 1 ADMIN_INS TERM_ACTIVE_IDLE ACTV IDLE NON_...

Page 624: ...m an individual MGCP based MGW or termination When such a flood occurs the BTS 10200 automatically throttles messages coming from that specific resource If the flood condition stops the system releases the throttle The system also deletes wild carded messages for example RSIP tid mgw net received from any MGW This detection and throttling mechanism is not customer configurable The system displays ...

Page 625: ... and limitations for this feature Special conditions or scenarios for which these features might not work or might behave in an unexpected manner Prerequisites The Cisco BTS 10200 Softswitch must be upgraded to Release 5 0 for the Cisco BTS 10200 Softswitch Core File Monitor feature to be active Restrictions and Limitations The Cisco BTS 10200 Softswitch must be upgraded to Release 5 0 for the Cis...

Page 626: ..._DELETE This is the maximum time in hours that a core file can exist before it is automatically deleted Default is 72 hours CORE_FILE_AGE_DELETE_ENABLE If set to true core files are deleted automatically when their maximum age is reached Default is true CORE_FILE_SPACE_DELETE_ENABLE If set to true the oldest core files are deleted when free file space is low Default is true Table 2 Core File Monit...

Page 627: ...nk Termination Tests page 15 10 Channel Associated Signaling Trunk Termination Tests page 15 11 Announcement Trunk Termination Tests page 15 13 Troubleshooting Using Snoop page 15 15 Query Verification Tool and Translation Verification Tool page 15 19 Network Loopback Test for Network based Call Signaling Media Gateway Control Protocol Endpoints page 15 41 Session Initiation Protocol Subscriber Re...

Page 628: ... in the MAINT state Step 1 Force the media gateway into MAINT state control mgw id c2421 65 mode forced target state maint Reply Example Reply Success CLI change successful MGW ID c2421 65 INITIAL STATE ADMIN_INS REQUEST STATE ADMIN_MAINT RESULT STATE ADMIN_MAINT FAIL REASON ADM found no failure REASON ADM executed successful RESULT ADM configure result in success Step 2 Display the Test Menu diag...

Page 629: ...ESULT TEST SUCCESS REASON PASSED Reply Diagnostic command executed diag mgw id ubr 03 test 3 Reply Example MEDIA GATEWAY LINE DIAGNOSTIC TEST EXECUTED diag mgw ID ubr 03 TEST TYPE ADM MGW NETW CONNECTIVITY TEST TEST DURATION 11 RESULT TEST SUCCESS REASON PASSED MEDIA GATEWAY LINE DIAGNOSTIC TEST EXECUTED diag mgw ID ubr 03 TEST TYPE ADM MGW MGCP CONNECTIVITY TEST TEST DURATION 0 RESULT TEST SUCCES...

Page 630: ...ubscriber termination id sub2 ctx2 mode forced target state maint Step 2 Display the Test Menu diag subscriber termination Reply Example Reply Diagnostic Subscriber Menu 1 Subscriber MGCP Connectivity Test 2 Subscriber Termination Connection Test 3 Subscriber Termination Ring Test 4 ALL Note Test 1 tests if MGCP has access to the termination Test 2 tests if there is a path to the device ping Test ...

Page 631: ...SUBSCRIBER LINE DIAGNOSTIC TEST EXECUTED diag subscriber termination ID sub ubr3 1 Cisco com TEST TYPE ADM TERM CONNECTION TEST TEST DURATION 55 RESULT TEST SUCCESS REASON PASS successfully Reply Diagnostic command executed diag subscriber termination id sub ubr3 1 cisco com test 3 ring duration 10 Reply Example SUBSCRIBER LINE DIAGNOSTIC TEST EXECUTED diag subscriber termination ID sub ubr3 1 Cis...

Page 632: ...Display the Test Menu diag ss7 trunk termination Reply Example Reply Diagnostic SS7 Trunk Group Menu Test 1 SS7 MGCP Connectivity Test Test 2 SS7 Termination Connection Test Test 3 SS7 COT Test Test 4 SS7 CQM Test Test 5 SS7 CVT Test Test 6 SS7 CIC Audit Test 0 ALL Tests Note Test 1 tests if MGCP has access to the SS7 trunk termination Test 2 tests if there is a path to the device ping Test 3 test...

Page 633: ... tgn id 103 cic 13 test 2 Reply Example TRUNK DIAGNOSTIC TEST EXECUTED diag trunk TG NUM 103 CIC 13 TEST TYPE ADM TERM CONNECTION TEST TEST DURATION 33 RESULT TEST SUCCESS REASON PASS successfully Reply Diagnostic command executed diag ss7 trunk termination tgn id 103 cic 14 test 3 Reply Example TRUNK DIAGNOSTIC TEST EXECUTED diag trunk TG NUM 103 CIC 14 TEST TYPE ADM SS7 COT TEST TEST DURATION 0 ...

Page 634: ...tion into INS state control ss7 trunk termination tgn id 103 mode forced target state ins Remote State Local State CIC 1 CS_IDLE ACTV IDLE CIC 2 CS_IDLE ACTV IDLE CIC 3 CS_IDLE ACTV IDLE CIC 4 CS_IDLE ACTV IDLE CIC 5 CS_IDLE ACTV IDLE CIC 6 CS_IDLE ACTV IDLE CIC 7 CS_IDLE ACTV IDLE CIC 8 CS_IDLE ACTV IDLE CIC 9 CS_IDLE ACTV IDLE CIC 10 CS_IDLE ACTV IDLE CIC 11 CS_IDLE ACTV IDLE CIC 12 CS_IDLE ACTV...

Page 635: ... CS_IDLE_REMBLOC Idle and remotely maintenance blocked CS_IDLE_BOTH_BLOC Idle and Remotely and Locally Maintenance Blocked CS_HW_LOCBLOC Locally Hardware Blocked CS_HW_LOCBLOC_LOCBLOC Locally Hardware and Locally Maintenance Blocked CS_HW_LOCBLOC_REMBLOC Locally Hardware and Remotely Maintenance Blocked CS_HW_LOCBLOC_BOTHBLOC Locally Hardware and Remotely and Locally Maintenance Blocked CS_HW_REMB...

Page 636: ...e Test 1 tests if MGCP has access to the ISDN termination Test 2 tests if there is a path to the device ping Test 3 performs tests 1 and 2 Step 3 To perform a specific test use the following examples as a guide diag isdn trunk termination test 1 tgn id 17 cic 1 Reply Example TRUNK DIAGNOSTIC TEST EXECUTED diag trunk TG NUM 17 CIC 1 TEST TYPE ADM MGW MGCP CONNECTIVITY TEST TEST DURATION 0 RESULT TE...

Page 637: ... has access to the CAS termination Test 2 tests if there is a path to the device ping Test 3 performs tests 1 and 2 Step 3 To perform a specific test use the following examples as a guide diag cas trunk termination tgn id 64 cic 1 test 1 Reply Example TRUNK DIAGNOSTIC TEST EXECUTED diag trunk TG NUM 64 CIC 1 TEST TYPE ADM MGW MGCP CONNECTIVITY TEST TEST DURATION 0 RESULT TEST SUCCESS REASON PASSED...

Page 638: ... trunk TG NUM 64 CIC 1 TEST TYPE ADM TERM CONNECTION TEST TEST DURATION 32 RESULT TEST SUCCESS REASON PASS successfully Reply Diagnostic command executed Step 4 Force the CAS trunk termination into INS state control cas trunk termination tgn id 64 mode forced target state ins Note On some Cisco voice gateways ringback tone may be played when the subscriber line is idle If the router statement voic...

Page 639: ...he MAINT state for testing Step 1 Force the Announcement trunk termination into MAINT state control annc trunk termination tgn id 13 mode forced target state maint Step 2 Display the Test Menu diag annc trunk termination Reply Example Reply Diagnostic ANC Trunk Group Menu 1 ANC MGCP Connectivity Test 2 ANC Termination Connection Test 3 ALL Note Test 1 tests if MGCP has access to the announcements ...

Page 640: ...le TRUNK DIAGNOSTIC TEST EXECUTED diag trunk TG NUM 13 CIC 1 TEST TYPE ADM TERM CONNECTION TEST TEST DURATION 33 RESULT TEST SUCCESS REASON PASS successfully Reply Diagnostic command executed diag annc trunk termination test 3 tgn id 13 cic 1 Reply Example TRUNK DIAGNOSTIC TEST EXECUTED diag trunk TG NUM 13 CIC 1 TEST TYPE ADM MGW MGCP CONNECTIVITY TEST TEST DURATION 11 RESULT TEST SUCCESS REASON ...

Page 641: ...u are snooping from a separate Unix machine you will just snoop the span destination interface in that case In this example destination Internet Protocol IP 10 0 0 1 is in question The fully qualified domain name FQDN can be used if it is resolvable by domain name system DNS Issue the command several times as there may be redundant routes mssol ca0 a route get 10 0 0 1 route to 10 0 0 1 destinatio...

Page 642: ...udp Stream Control Transmission Protocol SCTP MTP3 user adaptation M3UA ISDN user part ISUP example Since these protocols are not TEXT based as the ones mentioned above use the o option with snoop to capture packets in an Ethereal readable format Ethereal can decode SCTP M3UA ISUP or SCTP SCCP user adapter SUA Transaction Capabilities Application Part TCAP 10 0 0 1 is a Signaling Gateway acting as...

Page 643: ...O 500 STATUS INS ULP XUA snoop d qfe0 o suaain cap host 10 0 0 1 and port 2907 b Use Control C to stop the packet capture Open the file in Ethereal and inspect SCTP SUA TCAP example 2 10 0 0 1 is a Signaling Gateway acting as an SUA peer with the BTS 10200 The goal is to capture all offnet automatic callback and automatic rollback ACAR queries a Follow the same syntax as for the M3UA case except f...

Page 644: ...GE 30 MAX_VOIP_CALLS 65535 HIGH_WATER_MARK 0 LOW_WATER_MARK 0 IRR_BANDWIDTH_SUPP N IPTOS_SIG_LOWDELAY Y IPTOS_SIG_THROUGHPUT N IPTOS_SIG_RELIABILITY N IPTOS_SIG_PRECEDENCE FLASH BRQ_SUPP Y ANNEXE_RETRANSMIT_TIMER 500 ANNEXE_RETRANSMIT_MULTIPLIER 2 ANNEXE_RETRANSMIT_ATTEMPTS 8 CALL_START_MODE FAST_START ANNEXE_SUPP N ANNEXR_SUPP N STATUS_ENQ_TIMER 4 CODEC_NEG_TIMER 200 CODEC_NEG_ATTEMPTS 4 SOURCE_B...

Page 645: ...rification Tool This section describes the Query Verification Tool QVT and the Translation Verification Tool TVT and is organized into the following sub sections Tool Requirements page 15 19 Query Verification Tool page 15 20 Translation Verification Tool page 15 27 Using Query Verification Tool and Translation Verification Tool Together page 15 30 Tool Requirements The following requirements are ...

Page 646: ...P queries to external databases through the command line interface CLI interface The types of queries supported are Line information database LIDB Generated by the POTS Feature Server Toll Free Generated by the AIN Feature Server LNP Generated by the AIN Feature Server Command Format The QVT command uses the following format query lidb toll free lnp parameter value Examples query lidb calling dn 8...

Page 647: ...nating Line Information OLI Optional parameter used if the message type is IN 1 0 default POTS Bearer Capability Valid values are Speech f31KhzAudio b56kbps or b64kbps Trigger Criteria Valid values are 3 6 7 8 9 or 10 Table Info Specifies whether or not you want to see the tables accessed when processing the query Y N default N OPC Id Origination Point Code ID Calling DN The caller s directory num...

Page 648: ...rmation about error messages and problem resolution refer to the Query Errors section on page 15 22 Query Line Information Database Response Parameters Additional parameters returned in response to a query lidb command include Calling DN Caller ID Name String Caller ID Name Privacy Query Toll Free Parameters The following additional parameters are returned in response to a query toll free command ...

Page 649: ...ctive execute the following command to confirm that the selective call acceptance SCA process is running ps aef grep sca If the process is not running start it through process debug manager PDM or by stopping and restarting the platform If the platform is Active execute the following command to confirm that the TCAP signaling adapter TSA process is running ps aef grep tsa If the process is not run...

Page 650: ...bsystem is IN 1 The error response is similar to the following example CLI query toll free called dn 8002550002 user type calling dn user id 2182640018 lata 100 bearer capability speech trigger criteria 9 opc id opc Reply Failure Missing CALLING_DN for the IN 1 query CLI Reissue the command in the IN 1 format To see what message type is specified for the Toll Free subsystem enter the following com...

Page 651: ...l free called dn 8002550002 calling dn 2182640018 lata 101 trigger criteria 9 opc id opc Reply Failure LATA 101 does not exist CLI To resolve this error enter a provisioned local access and transport area LATA Local Network Failure When communication is lost between the BTS 10200 and the IP transfer point ITP gateway a local network failure may occur The most likely reason for this error is that t...

Page 652: ...rated by the AIN Feature Server for the QVT feature Table 15 3 identifies the measurements generated by the POTS Feature Server for the QVT feature Table 15 2 QVT AIN Tools Counters Counter Label Counter Description TOOLS_LNP_QUERY_ATTMP The total number of times the reporting feature server received a request to perform an LNP query from the QVT tool TOOLS_LNP_QUERY_SUCC The total number of times...

Page 653: ...TVT is a diagnostic tool that simulates a call from the originator to a specific destination based on dialed digits It enables a user to check system translations and determine if routing will occur as expected without making a call Command Format The TVT command uses the following format translate line trunk parameter value Examples translate line calling dn 2189722345 called dn 8002550005 Syntax...

Page 654: ...LAN_ID dp1 LOCAL_PFX1_OPT NR TOLL_PFX1_OPT RQ POP_ID 1 OLI 0 EA_USE_PI C1 Y TABLE DIAL_PLAN_PROFILE ID dp1 DESCRIPTION dialingplanprofile NANP_DIAL_PLAN Y DNIS_DIGMAN_ID dp1 TABLE DIAL_PLAN ID dp1 DIGIT_STRING 408555 DEST_ID ssp1dest SPLIT_NPA NONE DEL_DIGITS 0 MIN_DIGITS 10 MAX_ DIGITS 10 NOA NATIONAL TABLE DESTINATION DEST_ID ssp1dest CALL_TYPE LOCAL ROUTE_TYPE ROUTE ROUTE_GUIDE_ID ssp1rg ZERO_P...

Page 655: ...es the reporting feature server received a request to perform an LNP query from the QVT tool TOOLS_LNP_QUERY_SUCC The total number of times the reporting feature server received a request to perform an LNP query from the QVT tool and completed it successfully TOOLS_TOLLFREE_QUERY_ATTMP The total number of times the reporting feature server received a request to perform a Toll Free query from the Q...

Page 656: ...ollowing example illustrates verifying routing of a call from 972 233 1286 to 800 255 3002 Step 1 Execute a TVT translate command CLI translate line calling dn 9722331286 called dn 8002553002 TRANSLATE LINE ON CALL AGENT CA146 IS TABLEINFO TOLL FREE CALL NEEDS AN 800 QUERY Reply Success CLI Step 2 The translate command indicates that a Toll Free query is required Perform the QVT query to do the nu...

Page 657: ...OLI 0 EA_USE_PIC1 N TABLE DIAL_PLAN_PROFILE ID dp1 DESCRIPTION dialing plan profile ID 1 NANP_DIAL_PLAN Y DNIS_DIGMAN_ID dp_svc TABLE DIAL_PLAN ID dp1 DIGIT_STRING 703432 DEST_ID ssp1 dest SPLIT_NPA NONE DEL_DIGITS 0 MIN_DIGITS 7 MAX_ DIGITS 10 NOA NATIONAL TABLE DESTINATION DEST_ID ssp1 dest CALL_TYPE LOCAL ROUTE_TYPE ROUTE ROUTE_GUIDE_ID ssp1 rg ZERO_PLUS N INTR A_STATE Y GAP_ROUTING N CLDPTY_CT...

Page 658: ... IN SEL_POLICY DSC GLARE SLAVE ALT_ROUTE_ON_CONG N SIGNAL_PORTED_NUMBER Y POP_ID hun1 DIAL_PLAN_ID dp_t rk_itu DESCRIPTION TG IN from Inet 116 DEL_DIGITS 0 TRAFFIC_TYPE LOCAL ANI_BASED_ROUTING N CALL_CTRL_ROUTE_ ID cc_rte_i116_tg5 MGCP_PKG_TYPE T ANI_SCREENING N SEND_RDN_AS_CPN N STA TUS_MONITORING N SEND_EARLY_BKWD_MSG N EARLY_BKWD_MSG_TMR 5 SCRIPT_SUPP N VOICE_LAYER1_USERINFO AUTO VOICE_INFO_TRA...

Page 659: ...NSLATE ON CALL AGENT CA146 IS TABLEINFO TABLE TRUNK_GRP ID inet116_rg1 POLICY_TYPE ROUTE POLICY_ID inet116_rte TABLE ROUTE ID inet116_rte TGN1_ID 6 DEL_DIGITS1 0 DEL_DIGITS2 0 DEL_DIGITS3 0 DEL_D IGITS4 0 DEL_DIGITS5 0 DEL_DIGITS6 0 DEL_DIGITS7 0 DEL_DIGITS8 0 DEL_DIG ITS9 0 DEL_DIGITS10 0 TG_SELECTION SEQ NEXT_ACTION NONE TABLE TRUNK_GRP ID 6 CALL_AGENT_ID CA146 TG_TYPE SS7 NUM_OF_TRUNKS 24 DPC 5...

Page 660: ...ACTIVE BILLING_DN 11501511 DN1 11501511 PRIVACY NONE RING_TYPE_DN1 1 PIC1 NONE PIC2 NONE PIC3 NONE GRP N USAGE_SENS Y SUB_PROFILE_ID hungary_prof TERM_TYPE SIP IMMEDIATE _RELEASE N TERMINATING_IMMEDIATE_REL N AOR_ID 11501511 192 168 54 124 SEND_BDN_AS_CPN N SE ND_BDN_FOR_EMG N PORTED_IN N BILLING_TYPE NONE VMWI Y SDT_MWI Y TABLE DN2SUBSCRIBER OFFICE_CODE_INDEX 15 DN 1160 STATUS PORTED_OUT RING_TYP...

Page 661: ...NE PIC3 NONE GRP N USAGE_SENS Y SUB_PROFILE_ID hungary_prof TERM_TYPE SIP IMMEDIATE _RELEASE N TERMINATING_IMMEDIATE_REL N AOR_ID 11501511 192 168 54 124 SEND_BDN_AS_CPN N SE ND_BDN_FOR_EMG N PORTED_IN N BILLING_TYPE NONE VMWI Y SDT_MWI Y TABLE TRUNK_GRP ID 6 CALL_AGENT_ID CA146 TG_TYPE SS7 NUM_OF_TRUNKS 24 DPC 5 2 4 TG_PROFILE_ID tgprof_inet1 16 STATUS INS DIRECTION OUT SEL_POLICY DSC GLARE SLAVE...

Page 662: ... N AOR_ID 11501511 192 168 54 124 SEND_BDN_AS_CPN N SE ND_BDN_FOR_EMG N PORTED_IN N BILLING_TYPE NONE VMWI Y SDT_MWI Y TABLE SUBSCRIBER_PROFILE ID hungary_prof DIAL_PLAN_ID dp_sub_itu LOCAL_PFX1_OPT NR TOLL_PFX1_OPT RQ POP_ID hun1 OLI 0 EA_USE_PIC1 Y INTERLATA_PFX1_OPT RQ TABLE SUBSCRIBER ID sipata2 CATEGORY INDIVIDUAL NAME h15 sipata2 Larry STATUS ACTIVE BILLING_DN 11501512 DN1 11501512 PRIVACY N...

Page 663: ...apter 15 Diagnostic Tests Query Verification Tool and Translation Verification Tool btsadmin btsadmin QUERY LNP calling dn 11501511 called dn 11501512 QUERY ON FEATURE SERVER FSAIN205 IS RESULT Called Number 11501512 Routing Number RN Not Found QUERYSTATUS Query Success Reply Success ...

Page 664: ...IC2 NONE PIC3 NONE GRP N USAGE_SENS Y SUB_PROFILE_ID hungary_prof TERM_TYPE SIP IMMEDIATE _RELEASE N TERMINATING_IMMEDIATE_REL N AOR_ID 11501511 192 168 54 124 SEND_BDN_AS_CPN N SE ND_BDN_FOR_EMG N PORTED_IN N BILLING_TYPE NONE VMWI Y SDT_MWI Y TABLE SUBSCRIBER_PROFILE ID hungary_prof DIAL_PLAN_ID dp_sub_itu LOCAL_PFX1_OPT NR TOLL_PFX1_OPT RQ POP_ID hun1 OLI 0 EA_USE_PIC1 Y INTERLATA_PFX1_OPT RQ T...

Page 665: ..._TYPE SIP IMMEDIATE _RELEASE N TERMINATING_IMMEDIATE_REL N AOR_ID 11501511 192 168 54 124 SEND_BDN_AS_CPN N SE ND_BDN_FOR_EMG N PORTED_IN N BILLING_TYPE NONE VMWI Y SDT_MWI Y TABLE SUBSCRIBER_PROFILE ID hungary_prof DIAL_PLAN_ID dp_sub_itu LOCAL_PFX1_OPT NR TOLL_PFX1_OPT RQ POP_ID hun1 OLI 0 EA_USE_PIC1 Y INTERLATA_PFX1_OPT RQ TABLE TRUNK_GRP ID 6 CALL_AGENT_ID CA146 TG_TYPE SS7 NUM_OF_TRUNKS 24 D...

Page 666: ...MBER Y POP_ID hun1 DIAL_PLAN_ID dp_trk_itu DESCRIPTION TG IN from Inet 116 DEL_DIGITS 0 TRAFFIC_TYPE LOCAL ANI_BASED_ROUTING N CALL_CTRL_ROUTE_ID cc_rte_i116_tg5 MGCP_PKG_TYPE T ANI_SCREENING N SEND_RDN_AS_CPN N STATUS_MONITORING N SEND_EARLY_BKWD_MSG N EARLY_BKWD_MSG_TMR 5 SCRIPT_SUPP N VOICE_LAYER1_USERINFO AUTO VOICE_INFO_TRANSFER_CAP A UTO ACCESS_TYPE COMBINED POI INTER_ENDOFFICE PERFORM_LNP_Q...

Page 667: ...Y INDIVIDUAL NAME h15 sipata2 Larry STATUS ACTIVE BILLING_DN 11501512 DN1 11501512 PRIVACY NONE RING_TYPE_DN1 1 PIC1 NON E PIC2 NONE PIC3 NONE GRP N USAGE_SENS Y SUB_PROFILE_ID hungary_prof TERM_TYPE SIP IMMEDIA TE_RELEASE N TERMINATING_IMMEDIATE_REL N AOR_ID 11501512 192 168 54 124 SEND_BDN_AS_CPN N SEND_BDN_FOR_EMG N PORTED_IN N BILLING_TYPE NONE VMWI Y SDT_MWI Y Reply Success btsadmin Network L...

Page 668: ... the test MTA TGW has capability to perform multiple connections PARALLEL TEST CONN SUPP Y The BTS 10200 allows the system level configuration to specify whether the network loopback and network continuity test calls will be service affecting or not service affecting Restrictions Although you can test this feature by using the regular MTA as the testing device by configuring the endpoints as subsc...

Page 669: ...e Most of these tables have additional tokens that are not included in the examples For a complete list of all CLI tables and tokens see the BTS 10200 Command Line Interface Reference Guide Global Configuration Example Use the following procedure as a global configuration example Step 1 Add ca config NLB TEST SERVICE AFFECTING add ca config type NLB TEST SERVICE AFFECTING value N Step 2 Add ca con...

Page 670: ...NLB testing device configuration example Step 1 Add MGW profile add mgw profile id BRIX vendor Tollgrade mgcp version mgcp_1_0 MGCP VARIANT NCS 1 0 Step 2 Add cas tg profile add cas tg profile id BRIX_TG sig type LINE TEST LINE Y TEST LINE TYPE NLB LINE TEST Step 3 Add MGW add mgw id brix1 tsap addr mgw DNS IP address mgw profile id BRIX type TGW call agent id CA146 Step 4 Add trunk grp add trunk ...

Page 671: ... agent id CA146 tg type CAS cas tg profile BRIX_TG mgcp pkg type LINE Step 5 Add termination add termination prefix aaln port start 1 port end 2 type TRUNK mgw id c925 172 Step 6 Add trunk add trunk termination prefix aaln termination port start 1 termination port end 2 cic start 1 cic end 2 tgn id 100 Step 7 Add dial plan profile add dial plan profile id dp1 description NA_Default Step 8 Add dial...

Page 672: ... DNS IP address mgw profile id UBR925 call agent id CA103 Step 3 Add termination add termination prefix aaln port start 0 port end 1 type line mgw id c925 172 mgcp pkg type line ncs Step 4 Add destination add destination dest id local call route type sub call type local Step 5 Add dial plan profile add dial plan profile id dp1 description NA_Default Step 6 Add dial plan add dial plan id dp1 digit ...

Page 673: ...e TEST CALL call subtype NCT LINE TEST route type SUB add destination dest id DEST_NLB_TRUNK call type TEST CALL call subtype NLB TRUNK TEST route type ROUTE route guide id abc add destination dest id DEST_NCT_TRUNK call type TEST CALL call subtype NCT TRUNK TEST route type ROUTE route guide id abc Step 2 Add dial plan profile add dial plan profile id test nanp dial plan N Step 3 Add dial plan add...

Page 674: ... 4 Equip trunk termination equip trunk termination tgn id 100 cic all Step 5 Control trunk termination control trunk termination tgn id 100 cic all target state INS mode FORCED Step 6 Status trunk termination status trunk termination id 100 cic all Step 7 Reset trunk termination reset trunk termination id 100 cic all Using Operating the Network Loopback Test for Network based Call Signaling Media ...

Page 675: ...TG TYPE CAS Step 2 Associate the trunk group to CAS TG PROFILE specific to network loopback test origination type CAS TG PROFILE TEST LINE Y CAS TG PROFILE TEST LINE TYPE NLB LINE NCT LINE NLB TRUNK NCT TRUNK Step 3 Add all test lines in the testing device as trunk termination Shared Test Trunk Group In addition to dedicated test trunk groups the BTS 10200 allows a shared test trunk group where th...

Page 676: ...or the testing device as CAS trunk group TRUNK GRP TG TYPE CAS Step 2 Associate the trunk grp to CAS TG PROFILE specific to network loopback test origination CAS TG PROFILE TEST LINE Y CAS TG PROFILE TEST LINE TYPE NTE Step 3 Configure all test lines in Testing device as trunk termination Step 4 Configure the test dial plan destination with the exact type of test call Step 5 Configure the main sub...

Page 677: ...ing examples show the usage of the sip reg contact CLI command The first example provides an example of a expired contact and the second example provides an example of a registered contact or current contact Example 1 CLI to check the registration status of an address of record AOR CLI status sip reg contact CLI AOR_ID 4692551119 sia SYS44CA146 ipclab cisco com AOR ID 4692551119 sia SYS44CA146 ipc...

Page 678: ...nd already scheduled to execute at midnight every 24 hours To schedule multiple SHR command s at different times the command scheduler add command can be issued multiple times CLI add scheduled command verb report noun system health recurrence DAILY start time keys period values Use the following command to remove any scheduled SHR command s CLI delete scheduled command id NNN Use the following co...

Page 679: ...ts that use other unix scripts and utilities to perform full database and table audits of the databases on the various network elements of the system and synchronize the mismatches found The bts_audit tool determines the tables when performing full database audit by analyzing the catalog of the CA FSPTC and FSAIN databases The scripts will create copies of the data from the tables in a standardize...

Page 680: ...EMS machine ca is the hostname of the active CA machine platforms is a list of the platforms to be audited without spaces and separated by commas tables is a list of tables to be audited without spaces and separated by commas Example bts_audit ems priems01 ca prica01 platforms CA146 FSAIN205 tables SUBSCRIBER MGW_PROFILE The bts_sync command takes a list of filenames to be used for correcting erro...

Page 681: ...ls are identified by provisioning the call type and call subtype tokens in the Destination table The BTS 10200 cannot perform network loopback test calls that originate from another switch and does not route calls from a testing device on an H 323 or SIP interface Note The network loopback test cannot be performed if the status of the subscriber to be tested is unequipped UEQP or operational out o...

Page 682: ...group Step 1 Add a trunk group for the testing device as an ISDN PRI trunk group if it does not already exist trunk grp tg type isdn Step 2 Configure the test dial plan destination with the exact type of test call Step 3 Configure a main subscriber ID for the testing trunk group if necessary Call Agent Configuration Table The system defaults for the Call Agent Configuration ca config table may req...

Page 683: ...ples of network loopback NLB line loopback tests over ISDN trunks NLB Line Loopback Test over an ISDN Trunk This section provides example steps to use the feature for NTE NLB trunk test over an ISDN trunk Then 1 From the test equipment dial the NTE NLB trunk test call 551 xxx xxx xxxx The BCM does not notify the Feature Server of this call and the call is looped back 2 Hang up the test call and ve...

Page 684: ...he test call is still active 4 Hang up the test call and verify the Billing call type Perform the Following Command or Action Purpose Step 1 add destination dest id nlb line test call type test call call subtype nlb line test Provision the Destination table Step 2 add dial plan id xxx digit string 551 dest id nlb line test split npa none del digits 0 min digits 13 max digits 13 noa national Note W...

Page 685: ...riber Call is set up and the test call is released 4 Hang up the test call and verify the Billing call type Perform the Following Command or Action Purpose Step 1 add destination dest id nlb line test call type test call call subtype nlb line test Provision the Destination table Step 2 add dial plan id xxx digit string 551 dest id nlb line test split npa none del digits 0 min digits 13 max digits ...

Page 686: ...set up and the test call is still active 4 Hang up the test call and verify the Billing call type Perform the Following Command or Action Purpose Step 1 add destination dest id nlb line test call type test call call subtype nlb line test Provision the Destination table Step 2 add dial plan id xxx digit string 551 dest id nlb line test split npa none del digits 0 min digits 13 max digits 13 noa nat...

Page 687: ... 4 Verify the Billing call type NCT Tests This section provides examples of line loopback network continuity tests NCT over ISDN Perform the Following Command or Action Purpose Step 1 add destination dest id nlb line test call type test call call subtype nlb line test Provision the Destination table Step 2 add dial plan id xxx digit string 551 dest id nlb line test split npa none del digits 0 min ...

Page 688: ... Purpose Step 1 add destination dest id nlb line test call type test call call subtype nct line test Provision the Destination table Step 2 add dial plan id xxx digit string 552 dest id nlb trunk test split npa none del digits 0 min digits 13 max digits 13 noa national Note Where xxx is an existing dial plan The dial plan id must match to the LB test prefix for example 552 in the digit string Prov...

Page 689: ...riber Call is set up and the test call is released 4 Hang up the test call and verify the Billing call type Perform the Following Command or Action Purpose Step 1 add destination dest id nlb line test call type test call call subtype nct line test Provision the Destination table Step 2 add dial plan id xxx digit string 552 dest id nlb line test split npa none del digits 0 min digits 13 max digits ...

Page 690: ... and the test call is still active 4 Hang up the test call and verify the Billing call type Perform the Following Command or Action Purpose Step 1 add destination dest id nlb line test call type test call call subtype nct line test Provision the Destination table Step 2 add dial plan id xxx digit string 552 dest id nlb line test split npa none del digits 0 min digits 13 max digits 13 noa national ...

Page 691: ...as precedence For example 1 if the test call is on circuit xxx and a normal call comes in on the same circuit then the normal call is set up and the test call is released 2 if a normal call is on circuit xxx and a test call comes in on same circuit then the normal call stays up and the test call is released Perform the Following Command or Action Purpose Step 1 add destination dest id nlb line tes...

Page 692: ...test split npa none del digits 0 min digits 11 max digits 11 noa national Note Where xxx is an existing dial plan The dial plan id must match to the LB test prefix for example 553 in the digit string Provision the Dial Plan table The digit string plus the min digits and max digits total depends on the settings configured if any in the Call Agent Configuration Table section Perform the Following Co...

Page 693: ...ormation includes both statistical and performance details The mechanism used to manage the data generated and transported from the Cisco BTS 10200 system follows legacy type procedures and are documented in the following sections Measurement Data Transport and Access The measurement data collected on the Cisco BTS 10200 can be accessed through several different mechanisms The Command Line Interfa...

Page 694: ...TDM Measurement Collection Finished Issued whenever the traffic process running on the POTS Feature Server platform completes a collection cycle for the current interval STATISTICS 6 Informational AIN Measurement Collection Started Issued whenever the traffic process running on the AIN Feature Server platform begins a new collection cycle for the current interval STATISTICS 7 Informational AIN Mea...

Page 695: ... the BTS 10200 Operations and Maintenance Guide STATISTICS 12 Major Schemas out of Synchronization Issued whenever system detects a mismatch between the counter schema in Oracle on the BDMS and the internal schema of the call agents and or feature servers STATISTICS 13 Major TMM API Failure Issued whenever the TMM collection process is unable to initialize or attach to shared memory STATISTICS 14 ...

Page 696: ... counters POTS SLE Screening List Editing counters POTS ACAR Auto Callback Recall counters POTS COS Class Of Service counters POTS COT Customer Originated Trace counters AINSVC AIN Feature Service related information ISUP ISDN User Part SS7 signaling protocol related information in a Signaling Gateway configuration AUDIT Auditing related information SIA SIP Interface Adapter related information BI...

Page 697: ...rting interval is to encompass for the given report type The reporting interval is always synchronized to zero minutes after the hour for consistency This is an optional token that is pre provisioned with a value of 15 at the factory Changing this value does not take effect until the completion of the current collection interval based on the previous time interval setting Either this token and or ...

Page 698: ...elease 5 0 x OL 8723 19 Chapter 15 Diagnostic Tests Enhanced Traffic Measurement show measurement prov type tg usg show measurement prov type h323 show measurement prov type call tools show measurement prov type ain tools show measurement prov type pct tools ...

Page 699: ...ut sim report output type csv report measurement pots local summary start time 2002 03 27 10 00 00 end time 2002 03 27 12 00 00 feature server id PCT01 output pots local report output type csv report measurement pots misc summary start time 2002 03 27 10 00 00 end time 2002 03 27 12 00 00 feature server id PCT01 output pots misc report output type csv report measurement pots sle summary start time...

Page 700: ...put type csv report measurement snmp summary start time 2002 03 27 10 00 00 end time 2002 03 27 12 00 00 output snmp report output type csv report measurement tg usage summary start time 2002 03 27 10 00 00 end time 2002 03 27 12 00 00 tgn id dallas01 call agent id CA146 output tg report output type csv report measurement tg usage summary start time 2002 03 27 10 00 00 end time 2002 03 27 12 00 00...

Page 701: ...l of the collected intervals persisted on disk ALL If this token is used on the command line it overrides start time and end time tokens that are specified If entered the corresponding call agent id or feature server id must be specified There is no default value for this token If this token and the start time token are not entered by the user the last collected interval is reported sum This token...

Page 702: ...ture server that collected the measurement data This is an optional token that defaults to all feature servers and is only applicable to the following measurement types pct tools ain tools ainsvc sccp tcap pots local pots misc pots sle pots acar pots cos pots cot tgn id The trunk group numbers used to report measurement data This is an optional token that is only applicable to the following measur...

Page 703: ...he following measurement types m3ua sua sctp assoc id The sctp association id for reporting measurement data This is an optional token that is only applicable to the following measurement type sctp aggr id The aggregation id for reporting measurement data This is an optional token that is only applicable to the following measurement type dqos Table 15 6 Command Line Tokens Associated with Measurem...

Page 704: ...mary call agent id CA146 output sim partial report interval current output type csv report measurement pots local summary feature server id PCT01 output pots local partial report interval current output type csv report measurement pots misc summary feature server id PCT01 output pots misc partial report interval current output type csv report measurement pots sle summary feature server id PCT01 ou...

Page 705: ... call agent id CA146 output anm partial report interval current output type csv report measurement h323 summary call agent id CA146 output h323 partial report interval current output type csv report measurement call tools summary call agent id CA146 output call tools partial report interval current output type csv report measurement ain tools summary feature server id AIN01 output ain tools partia...

Page 706: ...S database This is a mandatory token output The name of the file to be created and location to place the resulting measurement data The file name is pre pended with the string Tm_ and placed in the opt ems report directory on the active EMS output type The format of the output file it can be in comma separated value csv or XML format call agent id The identity of the call agent that collected the ...

Page 707: ... optional and is used to specify that a report be generated that contains counter information for the interval currently under collection CURRENT or all of the collected intervals persisted on disk ALL If this token is used on the command line it will override start time and end time tokens if they are specified If entered the corresponding call agent id or feature server id must be specified Ther...

Page 708: ...local summary feature server id PCT01 clear measurement pots misc summary feature server id PCT01 clear measurement pots sle summary feature server id PCT01 clear measurement pots acar summary feature server id PCT01 clear measurement pots cos summary feature server id PCT01 clear measurement pots cot summary feature server id PCT01 clear measurement ainsvc summary feature server id AIN01 clear me...

Page 709: ...ent id The identity of the call agent that collected the measurement data This is an optional token that defaults to all call agents and is only applicable to the following measurement types call tools billing callp mgcp isdn audit sia sim anm H 323 m3ua em sctp feature server id The identity of the feature server that collected the measurement data This is an optional token that defaults to all f...

Page 710: ... on which counters are maintained within each measurement area A description of the meaning of each counter is also provided The name of each counter is an exact ASCII match to the label that is printed within the reports issued by the Cisco BTS 10200 These labels can then be used for automation purposes in testing and retrieving data from the BTS 10200 via the command line or FTP interfaces ...

Page 711: ...sages received by the reporting call agent ISDN_CONNECT_ACK_TX The number of ISDN CONNECT ACK messages sent from the reporting call agent ISDN_CONNECT_ACK_RX The number of ISDN CONNECT ACK messages received by the reporting call agent ISDN_DISCONNECT_TX The number of ISDN DISCONNECT messages sent from the reporting call agent ISDN_DISCONNECT_RX The number of ISDN DISCONNECT messages received by th...

Page 712: ...all agent Note This counter is only applicable to ETSI PRI ISDN_SUSPEND_ACK_RX The number of ISDN Suspend Acknowledge messages received by the reporting call agent Note This counter is only applicable to ETSI PRI ISDN_SUSPEND_REJ_TX The number of ISDN Suspend Reject messages sent from the reporting call agent Note This counter is only applicable to ETSI PRI ISDN_SUSPEND_REJ_RX The number of ISDN S...

Page 713: ...ONG_CNTL_TX The number of ISDN Congestion Control messages sent from the reporting call agent Note This counter is only applicable to ETSI PRI ISDN_CONG_CNTL_RX The number of ISDN Congestion Control messages received by the reporting call agent Note This counter is only applicable to ETSI PRI ISDN_SEGMENT_TX The number of ISDN Segment messages sent from the reporting call agent Note This counter i...

Page 714: ...empts that failed on the reporting call agent CALLP_ISDN_TERM_FAIL The number of ISDN terminating call attempts that failed on the reporting call agent CALLP_ISDN_CALL_SUCC The number of successful ISDN originating and terminating call attempts on the reporting call agent CALLP_ISDN_CALL_ABAND The number of ISDN originating call attempts that were abandoned on the reporting call agent CALLP_SS7_OR...

Page 715: ...ALLP_CAS_TERM_FAIL The number of CAS terminating call attempts that failed on the reporting call agent CALLP_CAS_CALL_SUCC The number of successful CAS originating and terminating call attempts on the reporting call agent CALLP_CAS_CALL_ABAND The number of CAS originating call attempts that were abandoned on the reporting call agent CALLP_ISDN_SS7_CALL The number of successfully completed calls fr...

Page 716: ...ed calls from an MGCP originator to an SIP terminator on the reporting call agent CALLP_MGCP_MGCP_CALL The number of successfully completed calls from an MGCP originator to an MGCP terminator on the reporting call agent CALLP_MGCP_CAS_CALL The number of successfully completed calls from an MGCP originator to an CAS terminator on the reporting call agent CALLP_CAS_SS7_CALL The number of successfull...

Page 717: ...abandoned on the reporting call agent CALLP_LOCAL_ATTMP The number of Local call attempts on the reporting call agent CALLP_LOCAL_FAIL The number of Local call attempts that failed on the reporting call agent CALLP_LOCAL_SUCC The number of Local call attempts that completed successfully on the reporting call agent CALLP_LOCAL_ABAND The number of Local call origination attempts that were abandoned ...

Page 718: ...mber of successfully completed calls from an H323 originator to an MGCP terminator on the reporting call agent CALLP_H323_CAS_CALL The total number of successfully completed calls from an H323 originator to a CAS terminator on the reporting call agent CALLP_H323_H323_CALL The total number of successfully completed calls from an H323 originator to an H323 terminator on the reporting call agent CALL...

Page 719: ...ort Unneeded Reason Code 812 that are received in the DLCX messages on the reporting call agent CALLP_NAS_PORT_PREEMPTED The total number of Port Pre empted Reason Code 813 that are received in the DLCX messages on the reporting call agent CALLP_NAS_PORT_SUSPENDED The total number of Port Suspended Reason Code 814 that are received in the DLCX messages on the reporting call agent CALLP_NAS_SERVICE...

Page 720: ...gent This counter includes both call setup failures and resource failures These are test calls abnormally released by the call agent due to reasons such as resource priorities CALLP_LB_TEST_SUCC The total number of successful TDM Loop Back 108 Tests completed by the reporting call agent CALLP_TEST_ROUTE_SUCC The total number of successful TDM Loop Back 108 Tests with DN dialed out in outgoing mess...

Page 721: ... call agent MGCP_MDCX_TX The number of MGCP MDCX messages sent by the reporting call agent MGCP_DLCX_RX The number of MGCP DLCX messages received from gateways by the reporting call agent MGCP_DLCX_TX The number of MGCP DLCX messages s ent by the reporting call agent MGCP_DLCX_ACK_RX The number of MGCP DLCX acknowledgement messages received by the reporting call agent MGCP_DLCX_NACK_RX The number ...

Page 722: ...AUCX_ACK_RX The number of AUCX ACK Audit Connection Acknowledgement messages that were received by the reporting call agent FUTURE R5 0 MGCP_AUCX_NACK_RX The number of AUCX NACK Audit Connection NotAcknowlegement messages that were received by the reporting call agent FUTURE R5 0 Table 15 11 MGCP Adapter Counters Counter Label Counter Context ...

Page 723: ...porting call agent or feature server sent SIS_T1_TIMER_EXPIRED The number of SIP T1 Timer expirations that occurred on the reporting call agent or feature server received over the collection interval SIS_T2_TIMER_REACHED The number of SIP T2 Timer expirations that occurred on the reporting call agent or feature server received over the collection interval SIS_INVITE_RX The number of SIP INVITE mes...

Page 724: ...ssages the reporting call agent or feature server received SIS_3xx_TX The number of 3xx class REDIRECTION messages the reporting call agent or feature server sent SIS_4xx_RX The number of 4xx class REQUEST FAILURES messages the reporting call agent or feature server received SIS_4xx_TX The number of 4xx class REQUEST FAILURES messages the reporting call agent or feature server sent SIS_5xx_RX The ...

Page 725: ...ACES_RX The number of SIP REFER with REPLACES messages the reporting call agent or feature server received SIS_INVITE_REPLACES_TX The number of SIP INVITE REPLACES messages the reporting call agent or feature server sent SIS_INVITE_REPLACES_RX The number of SIP INVITE REPLACES messages the reporting call agent or feature server received SIS_REL100_RX The number of REL100 class TRYING messages the ...

Page 726: ...ment and uses the default TCP port numbers of the Platform Application Services PAS server Table 15 13 on each side of the network element to establish an SSL connection to it and to obtain information Figure 15 1 Sample BTSSTAT Output prica16 btsstat ID SIDE HOST CA146 A prica16 CA146 B secca16 VERSION 900 05 00 00 I06 900 05 00 00 I06 RED REPL STATE STANDBY Replicating ACTIVE Replicating ID SIDE...

Page 727: ...y nondefault port numbers to status for any of the BTS 10200 network elements The command line options in Table 15 15 are supported to specify the port numbers Table 15 14 BTSSTAT Configuration File Format Element Setting CA_SIDE_A_HN CA_SIDE_B_HN prica11 secca11 FSAIN_SIDE_A_HN FSAIN_SIDE_B_HN prica11 secca11 FSPTC_SIDE_A_HN FSPTC_SIDE_B_HN prica11 secca11 EMS_SIDE_A_HN EMS_SIDE_B_HN priems11 sec...

Page 728: ...parate host Figure 15 2 BTSSTAT System Context Prerequisites The BTSSTAT software utility needs Apache xerces c library Version 2 6 0 or higher to parse and serialize the XML message This shared library must be present in the host in order for you to run BTSSTAT Installing Use the following procedures to install the BTSSTAT software utility on a BTS 10200 host and on a non BTS 10200 host HOSTprica...

Page 729: ...or fallback the two files can be simply replaced by the previous version Call Tracer CTRAC The BTS 10200 Call Tracer CTRAC feature provides a mechanism that uniquely marks each BTS 10200 system call to provide a system call trace troubleshooting capability In the BTS 10200 releases prior to Release 5 0 TRACE logs captured the detailed operations of the call processing system had a practical proble...

Page 730: ... Isolating Calls Based on Billing Record To isolate calls based on the billing record take the following steps Step 1 For a given call of interest note through CLI the value of the CTRAC ID billing record parameter This value is the CTRAC ID for the call For this example assume that it is M0000001 The CTRAC billing cdr parameter is CTRACID It can be obtained by using the CLI report billing record ...

Page 731: ...correlates with the CTRAC ID for all calls that originated from the specified endpoint Step 3 Use the CTRAC ID to filter out the trace log lines from the BTS 10200 logs Isolating Calls Based on a Given Terminating End Point To isolate calls based on a given terminating end point take the following steps Step 1 Go to the desired target platform log directory cd opt OptiCall platform instance name b...

Page 732: ...arameter is named CTRACID Troubleshooting The BTS 10200 CTRAC feature is intended as a troubleshooting enabler No specific troubleshooting steps are required other than the use of grep to filter out the TRACE lines corresponding to a CTRAC ID Tabular Display of Events and Alarms The BTS 10200 Tabular Display of Events and Alarms feature enables the BTS 10200 to display current alarms alarm history...

Page 733: ...ormation The Cisco BTS 10200 Softswitch Pre Manual Switchover Switch Integrity Diagnostic Utility feature provides the system health information of the switchover target so that Cisco BTS 10200 Softswitch customers can decide if they want to perform manual switchover The customer service provider operation organizations often periodically perform switchovers during maintenance windows after midnig...

Page 734: ...match between the EMS and the CA 2 Query for the following DB replication related alarms 3 Run the shared memory integrity tool to validate the shared memory integrity of the CA processes System Time Check The Cisco BTS 10200 Softswitch Pre Manual Switchover Switch Integrity Diagnostic Utility feature queries the system for the following alarms to see there is any system time drift Type and Number...

Page 735: ...ded Limit MAJOR MAINTENANCE 64 Average Outgoing Heartbeat Period Exceeds Major Alarm Limit Average Outgoing HB Period Exceeds Maj Alarm Limit MAJOR MAINTENANCE 65 Disk Partition Critically Consumed CRITICAL MAINTENANCE 66 Disk Partition Significantly Consumed MAJOR MAINTENANCE 68 The Free Inter Process Communication Pool Buffers Below Major Threshold The Free IPC Pool Buffers Below Major Threshold...

Page 736: ...re server configuration errors Operating System Issues in var adm messages Check The Cisco BTS 10200 Softswitch Pre Manual Switchover Switch Integrity Diagnostic Utility feature checks var adm messages for any operating system errors The feature searches for the following Solaris event types kern err kern crit kern em Software Configuration Check The Cisco BTS 10200 Softswitch Pre Manual Switchove...

Page 737: ...appId password field Root password of the Cisco BTS 10200 Softswitch nodes target field Is optional parameter and is used to specify the switchover target Allowed values are CA FSPTC FSAIN If target field is specified the mate of the ACTIVE side of the specified application is checked If target field is not specified the mate of the ACTIVE side of all three applications is checked CLI Reference Gu...

Page 738: ...o stores detailed information in a log file Log File The script output log file Contains a timestamp so that its name is unique Is stored in the opt ems log directory on the node that the script utility is running from Is managed by the Cisco BTS 10200 Softswitch log archiving utility Result Summary The script utility uses the following format for the result summary The summary is displayed on the...

Page 739: ...group and a specific trunk within the trunk group to test It then instructs the near end test equipment which is connected to the OTL and switch to select the specified trunk and the destination number for the far end test set The near end switch then selects the Trunk Under Test TUT and if the TUT is idle dials the destination number through CAS or nonassociated signaling methods common to normal...

Page 740: ... test calls conduct tests record measurements and report marginal or inferior trunk performance Test Equipment Test equipment capable of seizing the test line outpulsing digits preferably MF format recognizing supervision and supporting 1XX tests While Cisco does not recommend any vendor SAGE Instruments 930 or 935 series test equipment with the proper options are examples for use Test Line Many g...

Page 741: ... Test Lines 1 2 3 4 5 7 9 8 Comment MW QT Communications Test MW Signal Supervisory 2 Way Tests CAROT ROTL Responder Echo Digital Loopback Data Transmission K P S T 5 6 7 8 9 10 11 12 17 18 1 2 3 4 13 14 15 16 Test Line Test Type Trunk Group Number Member Trunk 1 0 0 Dialed Digits 100 101 102 103 104 105 N A 107 108 109 Transmission Tests To Standard Test Lines 1 2 3 4 5 7 9 8 Comment MW QT Commun...

Page 742: ...ing an answer signal recognizing supervision and supporting 1XX tests Although Cisco does not recommend any vendor SAGE Instruments 930 or 935 series test equipment with the proper options are good choices Test Line Many gateway products can satisfy the OTL requirements Preferred capabilities include Must be supported by the Cisco BTS 10200 T1 access line to connect to the test equipment to minimi...

Page 743: ...nd the test to be performed TG for example 0003 Trunk number for example 0018 The complete trunk address in this example is 00030018 Test type 10x for example 104 The technician dials KP 00030018 104 ST 5 The BTS 10200 automatically inserts either 9581 or 9591 in front of the test type digits to create a dialing string The complete test string in this example is PREFIX 00030018 9581104 END Note Al...

Page 744: ...sts record measurements and report marginal or inferior trunk performance Test Equipment Test equipment capable of seizing the test line outpulsing digits recognizing supervision and supporting 1XX tests Although Cisco does not recommend any vendor SAGE Instruments 930 or 935 series test equipment with the proper options are good choices Test Line OTL requirements are specific to the Far End switc...

Page 745: ...ment with the proper options are good choices Test Line OTL requirements are specific to the far end switch product as well as far end service provider enterprise test methods and procedures This is outside the scope of this document TTL Dial Plan Test lines are typically assigned 958 11XX numbers as depicted in Figure 15 8 5 6 7 8 9 10 11 12 17 18 1 2 3 4 13 14 15 16 Test Line Test Type 9 5 8 1 1...

Page 746: ...10200 is supporting the TTL capability test call originated at another switch the BTS 10200 receives the 958 or 959 call recognizes the 958 or 959 type and routes the test to the appropriate test line The BTS 10200 enables a TDM based testing device to perform continuity testing over an MF CAS TDM trunk interface An MGCP based trunking gateway must be present in the test path The TDM test type is ...

Page 747: ...seconds This helps failed T1 lines to regain frame synchronization and may also be used for other purposes This test may be used to determine the far to near loss and or C Notched noise of the trunk under test It may also be used for other far to near test purposes 103 Test Signaling and Supervisory The 103 test provides a connection to a supervisory and signaling test circuit for overall testing ...

Page 748: ...provides a digital loopback within the gateway The T108 test line feature supports manual and automated testing The T108 test line sequence is as follows 1 The near end switch originates the test sequence by placing a test call identifying the trunk to be selected and the test line number A digital test pattern generator is used in the test setup shown in Figure 15 3 2 The near end switch uses the...

Page 749: ... platform the system automatically restarts the process that shut down Restarting the process is a preferred alternative to switching over to the mate because the restart preserves stable calls and also attempts to preserve transient calls When a process is restarted the process audits information such as resource states and attempts to repair inconsistencies If a process experiences a high failur...

Page 750: ...Sun Solaris 10 Operating System CD 1 Note Sun Solaris 10 can be download at http www sun com website Console access Restored Host name Internet Protocol IP address and netmask of restored system Location of an archive Enabling negotiation on the 2900 switch for the primary interface of the system Example c2924 118 A config t c2924 118 A config t int fastEthernet 0 1 c2924 118 A config if no speed ...

Page 751: ...sc 2 to continue Step 14 Press Esc 2 to continue use default setting Mark X on Yes for System part of a subnet Step 15 Enter Netmask then press Esc 2 to continue Step 16 Press Esc 2 to continue use default setting Mark X on No for Enable IPv6 Step 17 Confirm the network information and press Esc 2 to continue Step 18 Press Esc 2 to continue use default setting Mark X on No for Configure Kerberos S...

Page 752: ...ut 15 30 minutes Step 38 Press exclanation sign to exit if prompted Step 39 Verify a etc vfstab and a etc system files contain no disk mirroring information Example of a etc vfstab device device mount FS fsck mount mount to mount to fsck point type pass at boot options dev dsk c1d0s2 dev rdsk c1d0s2 usr ufs 1 yes fd dev fd fd no proc proc proc no dev dsk c1t0d0s3 swap no dev dsk c1t0d0s0 dev rdsk ...

Page 753: ...t init bye Step 3 Copy the nsswitch conf file to etc directory cp p nsswitch conf etc Step 4 Copy the nsswitch conf file to etc directory cp p nsswitch conf etc Step 5 Copy the resolv conf file to etc directory cp p resolv conf etc Step 6 Copy init file to etc default directory cp p init etc default Step 7 Set up interfaces cd opt setup setlogic_EMS sh Run this script to set up interfaces on EMS b...

Page 754: ...ostname cp mnt db hostname gz opt hostname gzip cd opt oradata hostname gz tar xvf Step 4 Reboot the system hostname sync sync hostname shutdown y g0 i6 Step 5 Bring up BTS application hostname platform start Step 6 Restore platform startup script hostname mv etc rc3 d saved S99platform etc rc3 d S99platform Step 7 Refer to Jumpstart documentation to set up disk mirroring Note Cisco recommends run...

Page 755: ... analyze the logs for details that may provide the cause of the failure Power Failure Scenarios The following power failure scenarios are discussed in the following sections Power Failure on Single Host Computer page 16 7 Only one host of the two mated host computers is affected by the power outage Power Failure on Both Call Agent Computers page 16 9 Both mated host Call Agent computes are affecte...

Page 756: ...he mate to be restarted Step 3 When power is restored verify that all platforms are running by entering the nodestat command on the recovering host computer Step 4 Restart the failing platform on the host computer that is not affected by the power outage The platform should come up as Standby Platform Failure Not Due To Bad State of the Shared Memory on the Mate Host Computer or Any Failure on the...

Page 757: ...from the EMS to the CA e Enter the command audit db ems to make sure everything is in sync Step 3 Discrepancies will have to be fixed via command line interface CLI commands Caution This may take hours to complete and during this time call processing is lost that is why it is critical that there is no common single point of failure in the power feeds Power Failure on Both Element Management System...

Page 758: ...everything is in sync Step 6 Discrepancies will have to be fixed via CLI commands Caution This may take hours to complete and during this time call processing is lost that is why it is critical that there is no common single point of failure in the power feeds Element Management System Database Recovery from Hot Backup This section provides procedures to restore your Oracle EMS database data files...

Page 759: ...00201101800 from 18 00 archive log backup must be restored Timestamp is formatted as YYYYMMDDhhmm If your database crashes before the archive log backup you only need to restore the 2 00 am hot backup file sets If your system does not perform extra archive log backup daily by ora_arch_backup ksh use backup file sets from hot backup only In this sample scenario the primary EMS database will be reco...

Page 760: ...dm disable svc system cron Note Execute platform stop all and stop_cron sh on the secondary EMS also if the secondary EMS platform is active Step 4 Log in as oracle user or su oracle Step 5 Enter the following command to verify that there is enough free disk space df k opt oraback The EMS system must have enough disk space in the opt oraback directory to restore all database data files and archive...

Page 761: ...nly the hot backup files Backup files from 2 00 hot backup optical1_arc_full_1_167 200201100200 Z optical1_arc_full_1_168 200201100200 Z optical1_ctl_binary 200201100200 Z optical1_ctltrc 200201100200 tar Z optical1_hot_full_1_166 200201100200 Z optical1_ora_hot_full_backup_200201100200 log Back up files from the 18 00 archivelog backup optical1_arc_incr_1_169 200201101800 Z optical1_ctl_binary 20...

Page 762: ...200201101800 db1 control01 ctl cp opt oraback optical1_ctl_binary 200201101800 db2 control02 ctl Step 9 Recover the database using the recover_db_until_time ksh script The recover_db_until_time ksh script uses the restored binary control file to mount the database restores all data files from the restored database data sets applies all applicable archivelog files through the restored archivelog fi...

Page 763: ...datafiles RMAN 2 3 4 5 6 7 8 Thu Jan 10 16 40 15 CST 2002 All datafiles are restored Thu Jan 10 16 40 15 CST 2002 Begin to recover database Recover database until time 20020111 14 00 13 until time is always the restored timestamp 1day Last logseq 6782 thread 1 RMAN msglog file opt oracle tmp recover_db_until_time_200201101636 log RMAN 2 3 4 5 6 7 8 You can Ignore RMAN error messages regarding to I...

Page 764: ...ckup Note This process can take more than 10 minutes to complete depending on the volume of data in the database Text similar to the following is displayed This process performs the following tasks 1 Shutdown optical1 database on priems09 2 Backup opt oracle admin directory except arch dump and log 3 Cold backup database 4 Backup oracle crontab file 5 Startup database The following backup files ar...

Page 765: ...w ready to shut down the Oracle database and start the platform and cron process Step 6 Log in as root or su root Step 7 Enter the following commands su root platform stop i oracle platform start svcadm enable svc system cron nodestat The recovery of the primary EMS database is now complete To recover the secondary EMS database copy data from the primary EMS database Refer to the Recovering the El...

Page 766: ...similar to the following is displayed 2 Y 0 declare rc binary_integer begin rc sys dbms_defer_s ys push destination OPTICAL1 stop_on_error FALSE delay_seconds 0 parallelism 1 end Step 2 Shut down all processes on the corrupted EMS site a On the corrupted EMS site login as root b Stop the cron process and shut down the platform svcadm disable svc system cron platform stop all nodestat Verify whethe...

Page 767: ... Enter the following command mkdir p opt ems utils c FTP the file opt ems utils Version from the active EMS to the corrupted EMS then rename the file to Version save for reference Enter the following commands cd opt ems utils cat Version save 900 xx yy zz VVV d Enter the following commands to create the version file from Version save but change the version number to D00 D zero zero This D00 versio...

Page 768: ...s for the primary EMS and optical2_DB_upd tar gz is for the secondary EMS If the tar file is not in opt oraback directory and the same file still exists in the opt oracle tmp directory copy this file from opt oracle tmp to opt oraback directory If the file does not exist on either directory restore this file from remote FTP server to opt oraback directory then execute the steps in this section to ...

Page 769: ...pted database Caution During this step the dbadm A copy_all process will truncate local tables first then copy data from the tables on the other site Make sure that you execute this step on the corrupted EMS side only a On the corrupted EMS side log in as oracle or su oracle Note Make sure that you are on the corrupted database site b Enter the following command dbadm A copy_all Text similar to th...

Page 770: ...straints for BILLING Mon Jan 24 11 40 28 CST 2005 INFO DMMgr Enabling triggers for BILLING Mon Jan 24 11 40 29 CST 2005 INFO DMMgr Disabling Foreign Key constraints fo r OAMP Mon Jan 24 11 40 29 CST 2005 INFO DMMgr Disabling triggers for OAMP Mon Jan 24 11 40 29 CST 2005 INFO copy table OAMP CALL_TRACE Mon Jan 24 11 40 30 CST 2005 INFO copy table OAMP CALL_TRACE OK 0 row Mon Jan 24 11 41 41 CST 20...

Page 771: ...Beat process automatically enables the broken push job From active EMS side a Truncate replication queues Enter the following commands su oracle dbadm A truncate_def dbadm r get_unpushed_trans Text similar to the following is displayed no rows selected b Start the DBHeartBeat process Enter the following command dbinit H i start dbadm r get_broken_jobs Text similar to the following is displayed 2 N...

Page 772: ...Enter the following command dbadm C db Response similar the following example is displayed Checking table OPTICALL AGGR OK Checking table OPTICALL ANI OK Number of tables to be checked xxx Number of tables checked OK xxx Number of tables out of sync 0 OPTICAL1 Deftrandest is empty YES OPTICAL1 dba_repcatlog is empty YES OPTICAL1 Deferror is empty YES OPTICAL1 Deftran is empty YES OPTICAL1 Has no b...

Page 773: ...bles checked OK 127 Number of tables out of sync 3 List of out of sync tables OAMP TABLE_NAME 22 0 In this example one table owned by the OAMP is out of sync Follow the steps below to synchronize the contents of the tables Note Execute these commands on the corrupted EMS database to synchronize the table Truncate the content of the table on the local database then copy the data from the remote dat...

Page 774: ...ery from another database is now complete Fresh Download The Fresh Download command refreshes data in Call Agent shared memory It recovers the data in the Call Agent shared memory in the event that shared memory data cannot be recovered by any other means The fresh download wipes out Call Agent shared memory data and causes a total outage Caution Do not use this command on any live traffic product...

Page 775: ...form stop all command on both call agents Step 3 Delete the following data directories on the primary and secondary call agents cd opt OptiCall rm rf bin data Step 4 Start the primary call agents or feature servers with the platform start command Step 5 FTP the files created in Step 1 to the primary call agent download CA ftp to opt OptiCall CAxxx bin download FSPTC ftp to opt OptiCall FSPTxxx bin...

Page 776: ...nk grps oos control h323gws oos unequip trunk terminations control h323gws ins control trunk grps ins equip trunk terminations control trunks ins b Bring subscribers into service utilizing the cs control tool A script is generated with all the subscriber terminations An example of the script is as follows control subscriber termination id sub1 mode forced target state INS control subscriber termin...

Page 777: ...fic production systems Contact Cisco TAC regarding the use of this command for disaster recovery Caution Do not download the database through the console port because the TTY can cause long delays Downloads of the CA FSAIN and FSPTC applications can be done in parallel You can perform the download database command by ID The download by ID command allows you to copy database information from the EM...

Page 778: ...n queue you should login as ciscouser Examples delete transaction queue target CAxxx transaction id id delete transaction queue target FSPTCxxx transaction id id delete transaction queue target FSAINxxx transaction id id If there are thousands of entries stuck in the transaction queue it is recommended to flush all the entries from Oracle after logging in as oamp user Perform the following steps t...

Page 779: ...uccessfully Step 7 Enter the shared memory command to recover the CA and FS databases Enter following CLI download database commands download database target CA file tmp download CA download database target FSAIN file tmp download FSAIN download database target FSPTC file tmp download FSPTC Step 8 Stop both CA platforms Enter the following command platform stop all Step 9 Delete the following data...

Page 780: ...lowing from opt OptiCall FSPTCxxx bin command line dbm_sql FSPTCxxx data catalog download FSPTC c Enter the following from opt OptiCall FSAINxxx bin command line dbm_sql FSAINxxx data catalog download FSAIN Step 13 Start the secondary CA and FS platforms after Step 12 completes platform start Step 14 Continue to the following procedure Restoring Subscriber and Trunk Terminations to Service Restori...

Page 781: ... forced target state INS Note The completed script will be placed in the opt OptiCall CAxxx bin directory on the primary CA The cs control script is invoked from opt OptiCall CAxxx bin directory to place subscribers in service as follows cs control data name of the script used in the step above Table 16 1 Controlling Trunks and Trunk Groups Examples CLI Command Example control trunks oos control t...

Page 782: ... the Call Agent or Feature Server platform to be restored to an operational state in a much shorter time Additionally note that the data in the Automatic Shared Memory Backup ASMB can be up to 24 hours old Provisioning done in the time since the backup will be restored only after an audit and sync with the EMS is completed at the end of the procedure Before You Begin This procedure should only be ...

Page 783: ...neither NE will return to service after a platform start command If either side is in service or can be returned to service exit this procedure Step 3 Choose which side of the NE to restore first We recommend Side A as a convention and therefore will refer to Side A in this procedure Exit from the console of the other system Step 4 On Side A run the script restoreSharedMemory This script will remo...

Page 784: ...ommand Restore Shared Memory Script This section describes actions performed by the restoreSharedMemory script The restoreSharedMemory script completes these actions automatically These steps do not need to be performed by the user as part of the recovery procedure Step 1 Verifies that the platform is OOS FAULTY This procedure should not be run otherwise Step 2 Verifies that the installed software...

Page 785: ...ess The processing associated with the BTS 10200 Automatic Restart feature has two main phases In the first phase the transition is made to OOS FAULTY During this phase the forensic information for offline analysis must be collected and stored In the second phase the attempts are made to bring the platform from OOS FAULTY to STANDBY Transition to OOS FAULTY When the platform is transitioning to OO...

Page 786: ... initiated and KAM initiated the shutdowns are grouped by the state of the platform Table 16 2 OOS FAULTY Behavior Description Result PMG initiated Shutdowns Bringup One or more processes did not come up to correct state 1 SDAMR Unable to open log file var adm events_to_ plat SDAMR During bring up to ACTIVE one or more processes in a bad state SDAMR During bring up to STANDBY one or more processes...

Page 787: ...R Init thread failed SDAMR HB timer failure SDAMR Time out of sync with mate SDAMR Mate wrong side SDAMR Mate wrong ip address SDAMR Mate standby state SDAMR Mate not active SDAMR Active Active heartbeat failure and mate is OK SDAAR Standby Heartbeat with mate not working properly SDAAR Timeout waiting for Call data updates completion SDAAR DB copy timeout SDAAR DB copy failed SDAAR Switchover fai...

Page 788: ...A signaling links See Note 2 MGA When all the needed interfaces are either faulty or not stabilized See Note 2 SGA When both signaling gateways are down See Note 2 EPA When the EMS is not reachable See Note 2 NIM When all of the HUB interfaces are faulty See Note 2 Standby BMG2 Billing Manager Problems parsing command line options See Note 2 H3A When there are no good H3A signaling links See Note ...

Page 789: ...default is 10 minutes The newly active side is allowed time to begin processing fully before needing to support the auto restart of the faulty side by DB copies etc Platform Configuration The following parameter is added to the platform cfg file SystemAutoRestartNumSavedTraceLogs This variable indicates the number of most recent consecutive trace logs to save The following parameters are changed i...

Page 790: ... the saved debug directory to a system for offline debugging An example filename is saved debug 2007_02_16__16_07_09 tar Z Step 2 Remove the tar file from the disk after it is downloaded to conserve disk space Switchover in Progress Maintenance 101 The Switchover in Progress alarm critical indicates that a system switchover is in progress This alarm is issued when a system switchover is in progres...

Page 791: ...s in order to replace the defective disk Before starting the disk replacement procedures the following procedures must be completed Identify the disk that needs to be replaced by viewing the var adm messages Perform a full system backup The part number of the replacement disk must be the same as the part number of the defective disk or the size of the replacement disk must be equal to or larger th...

Page 792: ...200 system The procedure is performed before and after any software upgrade or for routine maintenance purposes Note It is recommended that this procedure be performed during a maintenance window or when traffic volume is low Before you start the backup procedure obtain the following information NFS server hostname or IP address Shared directory from NFS server Root user access Note Provisioning i...

Page 793: ... hostname FSAIN gz hostname tar cf opt OptiCall FSPTCxxx bin data gzip fast mnt data hostname FSPTC gz Note Where xxx is the instance number Step 7 Start all platforms hostname platform start nocopy Step 8 Verify all platforms are up on STANDBY hostname nodestat Step 9 Create excluded directories file for flash archive hostname vi tmp excluded_dir opt OptiCall CAxxx bin data hostname vi tmp exclud...

Page 794: ... all platforms Perform the following commands from the Active EMS hostname ssh optiuser hostname cli control feature server id FSAINxxx target state standby active cli control feature server id FSPTCxxx target state standby active cli control call agent id CAxxx target state standby active Note Where xxx is the instance number of each platform Step 13 Repeat the same steps for the PRIMARY CA FS ...

Page 795: ... file for flash archive hostname vi tmp excluded_dir data1 oradata hostname vi tmp excluded_dir opt data oradata hostname vi tmp excluded_dir opt Build Step 6 Start all platforms hostname platform start nocopy Step 7 Verify all platforms are up on STANDBY hostname nodestat Step 8 Backup the system hostname mv bin date bin date archive hostname mv bin date bin date hostname flarcreate n hostname I ...

Page 796: ... this node are ACTIVE then you need to switchover to STANDBY side using the CLI command Step 3 Stop all BTS 10200 platforms hostname platform stop all Step 4 Perform the following commands to detach all submirror metadevice hostname metadetach f d2 d0 hostname metadetach f d14 d12 hostname metadetach f d11 d9 hostname metadetach f d5 d3 hostname metadetach f d8 d6 Step 5 Perform the following comm...

Page 797: ...d to bring the new Disk 0 on line hostname cfgadm c configure c1 dsk cxt0d0 Example Secca38 cfgadm c configure c1 dsk c1t0d0 Ap_Id Type Receptacle Occupant Condition c0 scsi bus connected configured unknown c0 dsk c0t0d0 CD ROM connected configured unknown c1 scsi bus connected configured unknown c1 dsk c1t0d0 disk connected configured unknown c1 dsk c1t1d0 disk connected configured unknown c2 scs...

Page 798: ...on the new disk hostname metadb a f c 3 dev dsk cxt0d0s4 Step 16 Perform the following commands to create submirror metadevice hostname metainit f d0 1 1 dev dsk cxt0d0s0 hostname metainit f d12 1 1 dev dsk cxt0d0s6 hostname metainit f d9 1 1 dev dsk cxt0d0s5 hostname metainit f d3 1 1 dev dsk cxt0d0s1 hostname metainit f d6 1 1 dev dsk cxt0d0s3 Step 17 Install the boot block on the new disk hostn...

Page 799: ...ync Disk 0 new disk with Disk 1 hostname metattach d2 d0 hostname metattach d14 d12 hostname metattach d11 d9 hostname metattach d5 d3 hostname metattach d8 d6 Note The disk sync process with take awhile depending on the disk size To find out the percentage completion of this process you can run the metastat grep command Step 19 Start all BTS 10200 platforms hostname platform start nocopy ...

Page 800: ...this node are ACTIVE then you need to switchover to STANDBY side using the CLI command Step 3 Stop all BTS 10200 platforms hostname platform stop all Step 4 Perform the following commands to detach all submirror metadevice hostname metadetach f d2 d1 hostname metadetach f d14 d13 hostname metadetach f d11 d10 hostname metadetach f d5 d4 hostname metadetach f d8 d7 Step 5 Perform the following comm...

Page 801: ...nd to bring the new Disk 1 on line hostname cfgadm c configure c1 dsk cxt1d0 Example Secca38 cfgadm c configure c1 dsk c1t1d0 Ap_Id Type Receptacle Occupant Condition c0 scsi bus connected configured unknown c0 dsk c0t0d0 CD ROM connected configured unknown c1 scsi bus connected configured unknown c1 dsk c1t0d0 disk connected configured unknown c1 dsk c1t1d0 disk connected configured unknown c2 sc...

Page 802: ... on the new disk hostname metadb a f c 3 dev dsk cxt1d0s4 Step 16 Perform the following commands to create submirror metadevice hostname metainit f d0 1 1 dev dsk cxt1d0s0 hostname metainit f d12 1 1 dev dsk cxt1d0s6 hostname metainit f d9 1 1 dev dsk cxt1d0s5 hostname metainit f d3 1 1 dev dsk cxt1d0s1 hostname metainit f d6 1 1 dev dsk cxt1d0s3 Step 17 Install the boot block on the new disk host...

Page 803: ...ync Disk 1 new disk with Disk 0 hostname metattach d2 d1 hostname metattach d14 d13 hostname metattach d11 d10 hostname metattach d5 d4 hostname metattach d8 d7 Note The disk sync process with take awhile depending on the disk size To find out the percentage completion of this process you can run the metastat grep command Step 19 Start all BTS 10200 platforms hostname platform start nocopy ...

Page 804: ...17 14 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Chapter 17 Disk Replacement Call Agent Feature Server or Element Management System Disk 1 Replacement ...

Page 805: ...nection was deleted F N N N R 400 Transient error If this code is received the BTS 10200 tries to recover by retransmitting the MGCP message with a different transaction identifier for a predefined number of times before declaring the endpoint faulty The number of times is configurable at platform startup time its default is 2 R R R R R 401 Phone is already off hook If this code is received the BT...

Page 806: ...mplicated F F N N R 509 Error in RemoteConnectionDescriptor F F F N F 510 The transaction could not be executed because some unspecified protocol error was detected Automatic recovery from such an error will be very difficult and hence this code SHOULD only be used as a last resort R F N N R 511 The transaction could not be executed because the command contained an unrecognized extension This code...

Page 807: ...N R 524 Internal inconsistency in LocalConnectionOptions F F N N R 525 Unknown extension in LocalConnectionOptions This code SHOULD be used for unsupported mandatory vendor extensions x F F F N R 526 Insufficient bandwidth In cases where this is a transient error error code 404 SHOULD be used instead F F N N R 527 Missing RemoteConnectionDescriptor F F F N R 528 Incompatible protocol version F F F...

Page 808: ...ction CRCX create connection MDCX modify connection RQNT notification request Note Normal N other than 200 error code the BTS 10200 clears the call recoverable R the BTS 10200 clears the call and starts finite recovery non recoverable NR or faulty F the BTS 10200 clears the call Note For 400 error codes the BTS 10200 re transacts RT the corresponding MGCP message The BTS 10200 also retransmits RX ...

Page 809: ...mmands from the Call Agent How to Use this Document The keepalive functions were modified in BTS 10200 Release 5 0 Maintenance Release 1 1 MR1 1 Therefore this document is divided into two parts Keepalive for Release 5 0 Prior to Maintenance Release 1 1 page B 1 Keepalive for Release 5 0 Maintenance Release 1 1 and Later page B 19 The primary change in MR1 1 is that the parameter mgcp t hist which...

Page 810: ... list are set to their default values This has the following effect Enabling the KA process Using the default values of mgw monitoring enabled Y and keepalive method AUEP enables the sending of AUEP messages from the BTS 10200 to the MGW Note We recommend that you keep these tokens set to their default values which enables the KA process unless you have some other method of determining MGW connect...

Page 811: ...lares this MGW to be in down status and continues to perform the KA procedure Note To learn more about the MGW operational states reported by the BTS 10200 see the MGW table in the Cisco BTS 10200 Softswitch CLI Database The system uses the parameters mgcp max1 retries and mgcp max2 retries to limit the number of retransmissions of the same AUEP You can adjust the parameters mgcp max1 retries and ...

Page 812: ... messages including AUEP Initial Transmission Waiting Period mgcp t tran The initial transmission waiting period the period that the system waits after sending an initial AUEP transmission before repeating it is equal to the greater of the following The average response time between the sending of an MGCP message and receiving a response A specified lower limit provisioned as mgcp t tran default 4...

Page 813: ... does not receive an ACK message but an ACK is received in response to retransmissions Figure B 1 One IP Address in DNS ACK Received after Retransmissions Note See the additional information about mgcp t tran in the Initial Transmission Waiting Period mgcp t tran section on page B 4 Time AUEP initial transmission ACK received trying 1st IP address mgcp t tran 2 mgcp t tran 4 mgcp t tran 8 mgcp t t...

Page 814: ...t IP addresses for any single MGW listed in the DNS server This maximum number of IP addresses is not provisionable Time AUEP initial transmission ACK received trying 2nd IP address mgcp t tran 2 mgcp t tran 4 mgcp t tran 8 mgcp t tran mgcp max2 retries for 2nd IP address default 3 mgcp max1 retries completed If no ACK is received proceed to next IP address and continue 250021 Time AUEP initial tr...

Page 815: ... max Tip Terminology Note that the first transmission is called the initial transmission The second transmission of the same AUEP message is called the first retransmission or the first retransmission Figure B 3 Example of Retransmission Timing Upper Limit mgcp rto max Note See the additional information about mgcp t tran in the Initial Transmission Waiting Period mgcp t tran section on page B 4 T...

Page 816: ...onal status of each of the terminations is marked as TERM_UNREACH If keepalive functionality is disabled on the system the system marks the operational status of each of the terminations as UNKNOWN Figure B 4 shows how the system handles AUEP retransmissions and takes action if no ACK is received In this example mgcp max2 keepalive attempts are completed before the expiration of mgcp t max In gene...

Page 817: ...Progress Scenario 3 MGW Unreachable Note The following sections refer to two types of MGWs residential gateways RGWs and trunking gateways TGWs A MGW is identified as either a RGW or TGW according to the provisioning of the type token in the mgw table type rgw or type tgw Scenario 1 MGW Reachable This scenario applies to the following types of MGWs when the MGWs are reachable Trunking gateways TGW...

Page 818: ...ee Figure B 1 and Figure B 2 The system takes the following actions in the scenario shown in Figure B 5 If KA attempts are successful and stable calls exist The system continues this pattern of KA attempts Scenario 1 MGW Reachable For RGWs with 16 or fewer endpoints If KA attempts are successful and no stable calls exist The system implements the KA pattern shown in Scenario 2 See Scenario 2 Speci...

Page 819: ... process is expedited even though the next KA attempt is scheduled for a later time This condition is shown in Figure B 6 Figure B 6 MGW Reachable with Stable Calls and MGCP Message Unsuccessful 250025 KA 1 KA 2 KA 3 KA 5 KA 6 KA 7 Time Repeating time intervals with no MGCP command activity mgcp keepalive interval default 60 seconds Time KA 4 An MGCP message is sent and is unsuccessful no ACK rece...

Page 820: ... scenario is applicable to RGWs with 16 or fewer endpoints but not to TGWs or RGWs with more than 16 endpoints The BTS 10200 always uses Scenario 1 MGW Reachable procedures for reachable TGWs and reachable RGWs with more than 16 endpoints regardless of whether there are stable calls or not Three variations of this scenario are described in this section Basic Scenario MGCP Message Sent and Is Unsuc...

Page 821: ...1 and Figure B 2 The system takes the following actions in the scenario shown in Figure B 21 If KA attempts are successful and stable calls exist The system implements the KA pattern shown in Scenario 1 See Scenario 1 MGW Reachable If KA attempts are successful and no stable calls exist The system continues this pattern of KA attempts Scenario 2 Special Case Reachable RGW with 16 or Fewer Endpoint...

Page 822: ... the message is unsuccessful no ACK received the KA process is expedited even though the next KA attempt is scheduled for a later time This condition is shown in Figure B 9 Figure B 9 MGW Reachable with No Stable Calls and MGCP Message Unsuccessful 250028 Time Time An MGCP message is sent and is unsuccessful no ACK received The next scheduled KA attempt is expedited KA 1 KA 2 KA 3 KA 4 KA 5 KA 6 K...

Page 823: ...GW Reachable with No Stable Calls and MGCP Message Successful Scenario 3 MGW Unreachable This scenario applies to residential gateways RGWs and trunking gateways TGWs The following conditions are described KA Attempts Unsuccessful MGW Unreachable and Stable Call Is Present KA Attempts Unsuccessful MGW Unreachable and No Stable Calls Present MGW Unreachable New Call Termination Attempt Occurs 25002...

Page 824: ...epalive attempt KA 1 1 KA 1 2 KA 2 1 KA 3 1 and further has a unique MGCP transaction ID Note For detailed examples of successful KA attempts see Figure B 1 and Figure B 2 The system takes the following actions in the scenario shown in Figure B 24 If a KA attempt is successful and stable calls exist The system implements the KA pattern shown in Scenario 1 See Scenario 1 MGW Reachable For RGWs with...

Page 825: ...ive functionality is disabled If all KA attempts are unsuccessful through mgcp keepalive retries the system declares the MGW unreachable KA Attempts Unsuccessful MGW Unreachable and No Stable Calls Present This condition is similar to the condition described in KA Attempts Unsuccessful MGW Unreachable and Stable Call Is Present section on page B 16 However the intervals that occur after the KA att...

Page 826: ... is scheduled for a later time If the MGW is still in down status the KA attempts continue This condition is shown in Figure B 13 Note In this case the BTS 10200 fails the call immediately after receiving the call termination attempt without sending any MGCP message to the MGW Figure B 13 MGW Unreachable New Call Termination Attempt Occurs 250032 KA 1 1 KA 1 2 KA 1 3 Time KA attempts all unsuccess...

Page 827: ...in this document assumes that all of the tokens in the above list are set to their default values This has the following effect Enabling the KA process Using the default values of mgw monitoring enabled Y and keepalive method AUEP enables the sending of AUEP messages from the BTS 10200 to the MGW Note We recommend that you keep these tokens set to their default values which enables the KA process ...

Page 828: ...y the BTS 10200 see the MGW table in the Cisco BTS 10200 Softswitch CLI Database The system uses the parameters mgcp max1 retries and mgcp max2 retries to limit the number of retransmissions of the same AUEP You can adjust the parameters mgcp max1 retries and mgcp max2 retries if necessary to improve response if there are network bandwidth or reliability issues or if a MGW is slow in responding to...

Page 829: ...s show the initial waiting period as mgcp t tran Note The drawings in this section are not to scale Scenarios with AUEP Message Retransmissions and ACK Received Figure B 14 is applicable when one IP address is provisioned for the MGW in the DNS server It illustrates the scenario in which the initial AUEP transmission does not receive an ACK message but an ACK is received in response to retransmiss...

Page 830: ...ent IP addresses for any single MGW listed in the DNS server This maximum number of IP addresses is not provisionable Time AUEP initial transmission ACK received trying 2nd IP address mgcp t tran 2 mgcp t tran 4 mgcp t tran 8 mgcp t tran mgcp max2 retries for 2nd IP address default 3 mgcp max1 retries completed If no ACK is received proceed to next IP address and continue 250021 Time AUEP initial ...

Page 831: ...o max Tip Terminology Note that the first transmission is called the initial transmission The second transmission of the same AUEP message is called the first retransmission or the first retransmission Figure B 16 Example of Retransmission Timing Upper Limit mgcp rto max Note See the additional information about mgcp t tran in the Initial Transmission Waiting Period mgcp t tran section on page B 2...

Page 832: ... operational status of each of the terminations is marked as TERM_UNREACH If keepalive functionality is disabled on the system the system marks the operational status of each of the terminations as UNKNOWN Figure B 17 shows how the system handles AUEP retransmissions and takes action if no ACK is received In this example mgcp max2 keepalive attempts are completed before the expiration of mgcp t ma...

Page 833: ... Progress Scenario 3 MGW Unreachable Note The following sections refer to two types of MGWs residential gateways RGWs and trunking gateways TGWs A MGW is identified as either a RGW or TGW according to the provisioning of the type token in the mgw table type rgw or type tgw Scenario 1 MGW Reachable This scenario applies to the following types of MGWs when the MGWs are reachable Trunking gateways TG...

Page 834: ...akes the following actions in the scenario shown in Figure B 18 If KA attempts are successful and stable calls exist The system continues this pattern of KA attempts Scenario 1 MGW Reachable For RGWs with 16 or fewer endpoints If KA attempts are successful and no stable calls exist The system implements the KA pattern shown in Scenario 2 See Scenario 2 Special Case Reachable RGW with 16 or Fewer E...

Page 835: ... process is expedited even though the next KA attempt is scheduled for a later time This condition is shown in Figure B 19 Figure B 19 MGW Reachable with Stable Calls and MGCP Message Unsuccessful 250025 KA 1 KA 2 KA 3 KA 5 KA 6 KA 7 Time Repeating time intervals with no MGCP command activity mgcp keepalive interval default 60 seconds Time KA 4 An MGCP message is sent and is unsuccessful no ACK re...

Page 836: ...s scenario is applicable to RGWs with 16 or fewer endpoints but not to TGWs or RGWs with more than 16 endpoints The BTS 10200 always uses Scenario 1 MGW Reachable procedures for reachable TGWs and reachable RGWs with more than 16 endpoints regardless of whether there are stable calls or not Three variations of this scenario are described in this section Basic Scenario MGCP Message Sent and Is Unsu...

Page 837: ...14 and Figure B 15 The system takes the following actions in the scenario shown in Figure B 21 If KA attempts are successful and stable calls exist The system implements the KA pattern shown in Scenario 1 See Scenario 1 MGW Reachable If KA attempts are successful and no stable calls exist The system continues this pattern of KA attempts Scenario 2 Special Case Reachable RGW with 16 or Fewer Endpoi...

Page 838: ...the message is unsuccessful no ACK received the KA process is expedited even though the next KA attempt is scheduled for a later time This condition is shown in Figure B 22 Figure B 22 MGW Reachable with No Stable Calls and MGCP Message Unsuccessful 250028 Time Time An MGCP message is sent and is unsuccessful no ACK received The next scheduled KA attempt is expedited KA 1 KA 2 KA 3 KA 4 KA 5 KA 6 ...

Page 839: ...GW Reachable with No Stable Calls and MGCP Message Successful Scenario 3 MGW Unreachable This scenario applies to residential gateways RGWs and trunking gateways TGWs The following conditions are described KA Attempts Unsuccessful MGW Unreachable and Stable Call Is Present KA Attempts Unsuccessful MGW Unreachable and No Stable Calls Present MGW Unreachable New Call Termination Attempt Occurs 25002...

Page 840: ...new attempts spaced according to the provisioned value of mgcp keepalive interval Each keepalive attempt KA 1 1 KA 1 2 KA 2 1 KA 3 1 and further has a unique MGCP transaction ID Note For detailed examples of successful KA attempts see Figure B 14 and Figure B 15 The system takes the following actions in the scenario shown in Figure B 24 If a KA attempt is successful and stable calls exist The syst...

Page 841: ...tempt is unsuccessful The system raises one or more alarms and continues this pattern of KA attempts Scenario 3 MGW Unreachable For any stable calls on this MGW the system takes the action specified in the spare1 supp token in the mgw profile table Typical alarms for this condition include SIGNALING 36 Trunk locally blocked applicable to CAS ISDN and SS7 trunks SIGNALING 79 Trunking Gateway unreac...

Page 842: ...Attempts Unsuccessful MGW Unreachable and No Stable Calls Present KA 1 1 KA 1 2 KA 1 3 KA 1 4 KA 2 1 KA 3 1 Interval between KA attempts Starts at 10 seconds and increases as shown but is not allowed to exceed 60 seconds not provisionable 10 seconds 20 seconds 40 seconds KA attempt failure Gateway unreachable Occurs if no ACK is received at any time during this AUEP transaction mgcp keepalive retr...

Page 843: ... is scheduled for a later time If the MGW is still in down status the KA attempts continue This condition is shown in Figure B 26 Note In this case the BTS 10200 fails the call immediately after receiving the call termination attempt without sending any MGCP message to the MGW Figure B 26 MGW Unreachable New Call Termination Attempt Occurs 250032 KA 1 1 KA 1 2 KA 1 3 Time KA attempts all unsuccess...

Page 844: ...B 36 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Appendix B System Usage of MGW Keepalive Parameters Release 5 0 Keepalive for Release 5 0 Maintenance Release 1 1 and Later ...

Page 845: ...l types of networks SIP SS7 ISDN MGCP H 323 Table C 1 Overload Control Phases Overload Control Phase Actions 1 Detection Measures and compares factors to threshold values Determines system congestion and machine congestion level MCL Detects BTS machine congestion conditions in 5 levels none mild moderate severe emergency 2 Control Decreases overload this is configurable and varies with MCL but usu...

Page 846: ...critical IPC queue IPC buffer pool usage BTS monitors the proportion of available buffers in the IPC buffers pool this reflects MCL the higher the usage the greater the congestion BTS detects its own MCL in five levels MCL0 No congestion and no need for any abatement MCL1 Mild congestion Call rejection starts as configured in olm cfg MCL2 Moderate congestion Call rejection increases as configured ...

Page 847: ...owing occurs FS notifies CA once of its congested status CA sends only emergency triggers to FS as it manages FS s congestion abatement Slowing Overload Reduction Sudden abatement reduction may cause MCL to rapidly increase again To counteract MCL bouncing MCL reduces one MCL level at a time regardless of how low computed MCL becomes This permits the system MCL to reduce gracefully over a number o...

Page 848: ...evel platform_id CA146 mcl 2 MACHINE CONGESTION LEVEL ON CALL AGENT CA146 IS ADMIN MCL NO_CONGESTION 0 COMPUTED MCL NO_CONGESTION 0 EFFECTIVE MCL NO_CONGESTION 0 FEATURE SERVER CONGESTION FSAIN205 IS NOT CONGESTED FSPTC235 IS NOT CONGESTED REASON ADM executed successfully RESULT ADM configure result in success Reply Success at 2006 02 28 09 54 27 by btsadmin Configuring the SIP Response Code When ...

Page 849: ...isplay the EMERGENCY NUMBER LIST enter show emergency number list DIGIT_STRING 911 Reply Success at 2006 02 28 09 48 45 by btsadmin Entry 1 of 1 returned To delete a number from the EMERGENCY NUMBER LIST enter a command similar to the following delete emergency number list digit_string 911 Reply Success at 2006 02 28 09 52 20 by btsadmin MNT delete successful Transaction 934823480106794504 was pro...

Page 850: ...Unsolicited NOTIFY Repression If overloaded BTS does not send unsolicited NOTIFY messages MWI requests to endpoints However even if overloaded BTS does receive and process unsolicited NOTIFY requests UDP Messages BTS drops messages like STUN if they are less than the configured size This applies to UDP messages Message Rejection Logic When the BTS rejects an incoming SIP call it responds with 500 ...

Page 851: ... the output includes congestion status of FSs as seen by the CA Without this parameter the command displays the MCL of all platforms on the system Editing the OLM CFG File The Overload Manager OLM section appears in the platform cfg file Overload Control uses the following configuration files Table C 3 Configuration Files Used by Overload Control Name Location Description olm cfg opt OptiCall ca b...

Page 852: ...vel 4 is triggered for this factor Alarm Step Size info_alarm_step_size 0 The number of percentage point steps that will cause an info alarm for this factor Slots slot_array_size 10 The number of slots in the array of factor levels over which the mean is taken SUPEROBJECT Global data olm_sampling_interval 2 In milliseconds how often OLM wakes up and performs its computations olm_printing_cycle 0 I...

Page 853: ...y of factor levels over which the mean is taken Thresholds in level_1_threshold THRESH1 The value at which MCL level 1 is triggered for this factor level_2_threshold THRESH2 The value at which MCL level 2 is triggered for this factor level_3_threshold THRESH3 The value at which MCL level 3 is triggered for this factor level_4_threshold THRESH4 The value at which MCL level 4 is triggered for this f...

Page 854: ...CT The total number of calls rejected by OLM CALLP_OLM_ACCEPT_MCL0 Calls accepted by OLM at MCL0 CALLP_OLM_ACCEPT_MCL1 Calls accepted by OLM at MCL1 CALLP_OLM_ACCEPT_MCL2 Calls accepted by OLM at MCL2 CALLP_OLM_ACCEPT_MCL3 Calls accepted by OLM at MCL3 CALLP_OLM_REJECT_MCL1 Calls rejected by OLM at MCL1 CALLP_OLM_REJECT_MCL2 Calls rejected by OLM at MCL2 CALLP_OLM_REJECT_MCL3 Calls rejected by OLM...

Page 855: ...by SIM which tracks number of emergency triggers forced for all the FS SIM will update this counter every time when it forces an emergency trigger TRIGGER_911 to FS SIM_OC_TRIG_FORCED The number of triggers forced when the FS is overloaded a single counter is used by SIM which tracks the number of forced triggers for all the FSs SIM will update this counter every time when it forces a trigger Tabl...

Page 856: ... major shows MCL changes System MCL Level This is the effective MCL or the greater of the computed MCL and the administrative MCL When a new MAINTENANCE 112 alarm appears old MAINTENANCE 112 alarms clear When the system MCL falls to 0 the new alarm clears Dampen this alarm using alarm_damping_time in olm cfg The value of alarm_damping_time is the minimum amount of time that passes before the alarm...

Page 857: ...r direction for this factor but it is required to pass the next higher or lower level before appearing again For additional information refer to the MAINTENANCE 114 section on page 7 61 IPC Buffer Usage Level Maintenance 115 The IPC Buffer Usage Level alarm info shows MCL for IPC buffer usage factor crossed a multiple of the info_alarm_step_size in olm cfg Factor Level and Factor MCL This alarm is...

Page 858: ...C 14 Cisco BTS 10200 Softswitch Troubleshooting Guide Release 5 0 x OL 8723 19 Appendix C Overload Control Troubleshooting ...

Page 859: ... acknowledge ACL access control list automatic congestion level ACTV active ADDR address ADM administrative AGGR aggregation AIN Advanced Intelligent Network ALT alternate ANC announcements module ANI automatic number identification ANSI American National Standards Institute AOR address of record API application programming interface ARQ admission request AS application server ASC associate ASP ap...

Page 860: ...bleshooting Guide Release 5 0 x OL 8723 19 AUCX audit connection AUEP audit endpoint B B8ZS binary 8 zero substitution BCM Basic Call module BDMS Bulk Data Management System BLG billing BLO blocking BTS Broadband Telephony Softswitch BW band width ...

Page 861: ...CFN confusion CGB circuit group blocking CGU circuit group unblocking CIC circuit identification code carrier identification code CLI command line interface CLLI common language location identifier CMJ circuit reservation rejected CMTS cable modem termination system CNAM calling name delivery CODEC coder decoder compression decompression COPS Common Open Policy Service Protocol CORBA Common Object...

Page 862: ...TX Centrex CVM circuit verification message CVR circuit verification response CVT circuit validation test D DB database DBA database administrator DBM database management DF delivery function DHCP Dynamic Host Configuration Protocol DLCX delete connection DLY delay DN directory number DNIS dialed number identification service DNS domain name system DOC document DPC destination point code DSCP diff...

Page 863: ...stem EP endpoint EPCF endpoint connection ER easily recognizable ESA enhanced subscriber authentication ESF extended super frame F FAS facility associated signaling FPGA field programmable gate array FQDN fully qualified domain name FS Feature Server FSAIN Feature Server for AIN services FSPTC Feature Server for POTS Tandem and Centrex services FTP File Transfer Protocol FXS Foreign Exchange Stati...

Page 864: ...gabits GCF gatekeeper confirmation GK gatekeeper GMT Greenwich Mean Time GRA group reset acknowledge GRJ gatekeeper reject GRP group GRQ gatekeeper request GRS group reset GTD generic transport descriptor GTT global title translation GW gateway H H323 H 323 Protocol H323GW H323 gateway H3A H 323 signaling adapter HB heartbeat ...

Page 865: ...ask Force IN intelligent network INF information INFO information INIT initialize INR information request INS in service IOS Integrated Operating System IP Internet Protocol IPC inter process communication IPM Internet Protocol Manager IPSEC Internet Protocol security ISDN Integrated Services Digital Network ISP Internet service provider ISUP ISDN user part ITP IP transfer point IVR interactive vo...

Page 866: ...AM keepalive module KB kilobit KMS Kerberized management server L LAF log archive facility LATA local access and transport area LBLK locally blocked LIDB line information database LNP local number portability LPA loop prevention acknowledgement LRN local routing number LSA local serving area ...

Page 867: ...edia gateway adapter MGCP Media Gateway Control Protocol MGW media gateway MIB Management Information Base MIM M3UA Interface Module MRMW multiple readers and multiple writers ms millisecond MTA media terminal adapter Metropolitan Trading Area MTP message transfer part MTP1 message transfer part 1 MTP2 message transfer part 2 MTP3 message transfer part 3 MTU maximum transmission unit ...

Page 868: ...m NMS network management system NNN numeric number nomenclature NOA nature of address NPA Numbering Plan Area NPAC Number Portability Administration Center NR non recoverable NTP Network Time Protocol NU network unit NUM number NW network O OAMP operations administration maintenance and provisioning OLI originating line information OMNI Ulticom SS7 stack OMS OptiCall Messaging System OPC originati...

Page 869: ...s debug manager PDU protocol decode unit PIC presubscribed interexchange carrier point in call PID process ID PMG process manager POP point of presence POTS plain old telephone service PRI primary rate interface PSTN public switched telephone network Q QAM queue processing module QoS quality of service QVT Query Verification Tool ...

Page 870: ...W residential gateway RKEY routing key RKS record keeping system RLC release complete RMAN recovery manager ROM read only memory RPF registration and profiling tool RQNT request for notification RQNT notification request RR resource record RRJ request rejected RRQ registration request RSC reset circuit RSIP restart in progress RT re transact RTM routing module RTO retransmission time out RTP Real ...

Page 871: ...ignaling connection control SCCP signaling connection control part SCP service control point signal control point SCTP Stream Control Transmission Protocol SDN software defined network SFTP Secure File Transfer Protocol SG signaling gateway SGP signaling gateway process SHR System Health Report system health SIA SIP adapter SID system identification number SIP Session Initiation Protocol SLC signa...

Page 872: ...e 5 0 x OL 8723 19 SNMP Simple Network Management Protocol SP service provider SQL Structured Query Language SRV service SS7 Signaling System 7 SSH secure shell SSN subsystem number SSP signal switching point STP signal transfer point SUA SCCP user adapter SUB subscriber ...

Page 873: ... request TAC Technical Assistance Center TCAP Transaction Capabilities Application Part TCP Transmission Control Protocol TDM telecommunications data link monitor TEI terminal endpoint identifier TFC transfer controlled TG trunk group trunking gateway TGID trunk group ID TGN ID trunk group ID TMM Traffic and Measurements module TOS type of service TPM terminating point master TRNS transient TSA TC...

Page 874: ...ation Tool U UBL unblocking message UDP User Datagram Protocol UEQP unequipped ULP Upper Layer Protocol URI uniform resource identifier URL universal resource locator US United States USA United States of America V VOIP voice over IP VTOC volume table of contents W X XUA extended unit association XUDTS extended unit data service Y Z ...

Reviews:

No comments

Brands by name

Popular brands

Load more brands