Cisco NCS 1002 Troubleshooting Manual Download Page 24

Page: 24 / 86

NAME: "Rack 0", DESCR: "Network Convergence System 1002 20 QSFP28/QSFP+ slots"
PID: NCS1002-K9

, VID: V03, SN: CAT2116B170

NAME: "0/FT0", DESCR: "Network Convergence System 1000 Fan"
PID: NCS1K-FTA

, VID: V01, SN: N/A

NAME: "0/FT1", DESCR: "Network Convergence System 1000 Fan"
PID: NCS1K-FTA

, VID: V01, SN: N/A

NAME: "0/FT2", DESCR: "Network Convergence System 1000 Fan"
PID: NCS1K-FTA

, VID: V01, SN: N/A

NAME: "0/PM0", DESCR: "Network Convergence System 1000 2KW AC PSU"
PID: NCS1K-2KW-AC

, VID: V01, SN: POG2041J0BW

NAME: "0/PM1", DESCR: "Network Convergence System 1000 2KW AC PSU"
PID: NCS1K-2KW-AC

, VID: V01, SN: POG2041J01C

What to do next

Environment parameter anomalies are logged in the syslog. Hence, if an environment parameter displayed in
the

show environment

command output is not as expected, check the syslog using the

show logging

command.

The syslog provides details on any logged problems.

Troubleshooting Firmware Upgrade Failure

Step 1

show hw-module fpd

Verify the firmware version. Displays the firmware information of various hardware components of the NCS 1002.

The following example is for Release 6.0.1:

Example:

RP/0/RP0/CPU0:ios#

show hw-module fpd

Tue Apr 12 09:04:14.935 UTC
FPD Versions
=================
Location Card type HWver FPD device ATR Status Running Programd
------------------------------------------------------------------------------
0/0

NCS1002 2.4 CDSP_PORT_05

CURRENT 3.56 3.56

0/0

NCS1002 2.4 CDSP_PORT_06

CURRENT 3.56 3.56

0/0

NCS1002 2.4 CDSP_PORT_12

CURRENT 3.56 3.56

0/0

NCS1002 2.4 CDSP_PORT_13

CURRENT 3.56 3.56

0/0

NCS1002

CDSP_PORT_19

UPGD FAIL

0/0

NCS1002 2.4 CDSP_PORT_20

CURRENT 3.56 3.56

0/0

NCS1002 2.4 CDSP_PORT_26

CURRENT 3.56 3.56

0/0

NCS1002 2.4 CDSP_PORT_27

CURRENT 3.56 3.56

0/0

NCS1002

CFP2_PORT_05

NOT READY

0/0

NCS1002 2.0 CFP2_PORT_06

CURRENT 4.38 4.38

0/0

NCS1002

CFP2_PORT_12

NOT READY

0/0

NCS1002

CFP2_PORT_13

NOT READY

0/0

NCS1002

CFP2_PORT_19

NOT READY

0/0

NCS1002 2.1 CFP2_PORT_20

CURRENT 5.19 5.19

0/0

NCS1002

CFP2_PORT_26

NOT READY

0/0

NCS1002

CFP2_PORT_27

NOT READY

0/0

NCS1002 0.1 CTRL_BKP_LOW

CURRENT 1.22

Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x

General Troubleshooting

Troubleshooting Firmware Upgrade Failure

Summary of Contents for NCS 1002

Page 1: ...oting Guide for Cisco NCS 1002 IOS XR Release 6 3 x Americas Headquarters Cisco Systems Inc 170 West Tasman Drive San Jose CA 95134 1706 USA http www cisco com Tel 408 526 4000 800 553 NETS 6387 Fax 408 527 0883 ...

Page 2: ... IMPLIED INCLUDING WITHOUT LIMITATION THOSE OF MERCHANTABILITY FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING USAGE OR TRADE PRACTICE IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT SPECIAL CONSEQUENTIAL OR INCIDENTAL DAMAGES INCLUDING WITHOUT LIMITATION LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE T...

Page 3: ...bmitting a service request and gathering additional information see What s New in Cisco Product Documentation To receive new and revised Cisco technical content directly to your desktop you can subscribe to the RSS feeds are a free service 2018 Cisco Systems Inc All rights reserved ...

Page 4: ......

Page 5: ...ware Version 16 Troubleshooting Optical Connectivity 18 Using Loopbacks 18 Using Link Layer Discovery Protocol Snooping 20 Using Trail Trace Identifier 21 Troubleshooting the Trunk Port 23 Troubleshooting Breakout Ports 25 Troubleshooting Breakout Patch Panel 26 Troubleshooting a Failed Commit Configuration 26 Removing and Re inserting DIMMs on the Controller Card 27 Verifying Wavelength and Chann...

Page 6: ...E CPU PROCESSOR HOT Alarm 47 Clear the DAISYDUKE CPU PROCESSOR HOT Alarm 47 DGD Alarm 48 Clear the DGD Alarm 48 EQPT FAIL SLICE Alarm 48 Clear the EQPT FAIL SLICE Alarm 48 FAM FAULT TAG UNIT UNSUPPORTED ALARM 49 Clear the FAM FAULT TAG UNIT UNSUPPORTED Alarm 49 FAM FAULT TAG PMO UNIT UNSUPPORTED ALARM 49 Clear the FAM FAULT TAG PMO UNIT UNSUPPORTED Alarm 49 FAN FAIL Alarm 49 Clear the FAN FAIL Ala...

Page 7: ...e LOM Alarm 56 LOS P Alarm 56 Clear the LOS P Alarm 56 LOWER CTRL FPGA PCIE ERROR 57 Clear the LOWER CTRL FPGA PCIE ERROR Alarm 57 MEA Alarm 57 Clear the MEA Alarm 57 MIXED AC DC PTS Alarm 58 Clear the MIXED AC DC PTS Alarm 58 MKA LOCALLY SECURED Alarm 58 Clear the MKA LOCALLY SECURED Alarm 58 MKA START FAIL MISSING PSK Alarm 58 Clear the MKA START FAIL MISSING PSK Alarm 59 MKA UNSECURE MISSING PS...

Page 8: ...63 OTUK SF Alarm 63 Clear the OTUK SF Alarm 63 OTUK TIM Alarm 63 Clear the OTUK TIM Alarm 63 PATCH PANEL POWER REDUNDANCY LOST ALARM 63 Clear the PATCH PANEL POWER REDUNDANCY LOST Alarm 64 PEM MODULE INSERT Alarm 64 Clear the PEM MODULE INSERT Alarm 64 PEM MODULE REMOVAL Alarm 64 Clear the PEM MODULE REMOVAL Alarm 64 POWER MODULE OUTPUT DISABLED Alarm 64 Clear the POWER MODULE OUTPUT DISABLED Alar...

Page 9: ...rm 68 Clear the SIGLOSS Alarm 68 SYNCLOSS Alarm 68 Clear the SYNCLOSS Alarm 69 SQUELCHED Alarm 69 Clear the SQUELCHED Alarm 70 TEMPERATURE Alarm 70 Clear the TEMPERATURE Alarm 71 TX POWER PROV MISMATCH Alarm 71 Clear the TX POWER PROV MISMATCH Alarm 71 UNC WORD Alarm 72 Clear the UNC WORD Alarm 72 UPPER CTRL FPGA PCIE ERROR Alarm 72 Clear the UPPER CTRL FPGA PCIE ERROR Alarm 72 VOLTAGE Alarms 73 C...

Page 10: ...Troubleshooting Guide for Cisco NCS 1002 IOS XR Release 6 3 x x Contents ...

Page 11: ...shooting Optical Connectivity on page 18 Troubleshooting the Trunk Port on page 23 Troubleshooting Breakout Ports on page 25 Troubleshooting Breakout Patch Panel on page 26 Troubleshooting a Failed Commit Configuration on page 26 Removing and Re inserting DIMMs on the Controller Card on page 27 Verifying Wavelength and Channel Mapping for Optics Controllers on page 28 Verifying the Performance Mon...

Page 12: ...le RP 0 RP0 CPU0 ios show install active Wed Nov 11 06 06 40 221 UTC Node 0 RP0 CPU0 RP Boot Partition xr_lv0 Active Packages 1 ncs1k xr 6 0 0 22I version 6 0 0 22I Boot image The following sample output displays active software packages in the system admin EXEC mode sysadmin vm 0_RP0 show install active Wed Nov 11 06 06 47 804 UTC Node 0 RP0 RP Active Packages 1 ncs1k sysadmin 6 0 0 22I version 6...

Page 13: ...etail Wed Dec 9 01 19 18 680 UTC Dec 09 01 19 07 Install operation 49 started by root install add source tftp 10 105 236 167 ncs1k k9sec rpm Dec 09 01 19 08 Action 1 install add action started Dec 09 01 19 08 ERROR Either file is not proper or error in getting rpm metadata from rpm file Dec 09 01 19 08 ERROR failed to complete install add precheck Dec 09 01 19 09 Install operation 49 aborted Dec 0...

Page 14: ...es You can verify the state of the hardware modules in the following scenarios Node is not reachable Node recovers from a problem Node had a power cycle Node reboot Node upgrade Node settles down after the Cisco IOS XR has continuously reloaded Step 1 show platform When you execute this command from the Cisco IOS XR EXEC mode the status of the Cisco IOS XR is displayed Verify that the node state i...

Page 15: ...0 CPU0 ios admin Step 3 show platform Displays information and status for each node in the system Example sysadmin vm 0_RP0 show platform Tue Feb 27 10 26 58 763 UTC Location Card Type HW State SW State Config State 0 0 NCS1002 OPERATIONAL N A NSHUT 0 RP0 NCS1002 RP OPERATIONAL OPERATIONAL NSHUT 0 FT0 NCS1K FTA OPERATIONAL N A NSHUT 0 FT1 NCS1K FTA OPERATIONAL N A NSHUT Verify that all the modules...

Page 16: ... OPERATIONAL SW Oper State OPERATIONAL Configuration NSHUT RST HW Version 0 1 Last Event UNKNOWN Last Event Reason UNKNOWN Step 5 show inventory Displays the details of the physical entities of the NCS 1002 along with the details of QSFPs and CFPs when you execute this command in the Cisco IOS XR EXEC mode You can verify if any QSFP or CFP has been removed from the NCS 1002 Example RP 0 RP0 CPU0 i...

Page 17: ... 0 0 Optics0 0 0 22 LANE1 DESCR Cisco 10G SFP LR Pluggable Optics Module PID SFP 10G LR VID V01 SN SPC190707YS NAME 0 0 Optics0 0 0 23 DESCR Cisco 40GE QSFP SR4 Pluggable Optics Module PID QSFP 40G SR4 VID V03 SN JFQ2033201H NAME 0 0 Optics0 0 0 24 DESCR Non Cisco QSFP28 100G LR4 Pluggable Optics Module PID FTLC1151RDPL VID A0 SN UWD2QMM NAME 0 0 Optics0 0 0 25 LANE1 DESCR Cisco 10G SFP ER Pluggab...

Page 18: ...n 6 0 0 26I Copyright c 2013 2015 by Cisco Systems Inc Build Information Built By xxxx Built On Tue Dec 1 17 02 18 PST 2015 Build Host build lnx 100 Workspace auto build lnx 106 san1 r60x ws6 nightly_r60x 151201B_ncs1k workspace Version 6 0 0 26I Location opt cisco XR packages cisco NCS1K processor System uptime is 4 days 12 hours 20 minutes What to do next Verify the result to ascertain whether a...

Page 19: ... buffer failures 0 output buffers swapped out 0 carrier transitions a In the above result the management interface is administratively down Use the no shut command to enable the management interface The following example shows sample output from the show running config interface mgmtEth command when the management interface is in the no shut state RP 0 RP0 CPU0 ios show running config interface mg...

Page 20: ...ig Failure Client Bitrate 100 Trunk Bitrate 100 In the above example the slice provisioning has failed because of an ETNA configuration failure Some of the failure reasons that can appear in the command output are CFG INIT Config Failure Indicates DP FPGA download issue PPM Config Failure Indicates PPM Driver failure CDR Config Failure Indicates CDR failure ETNA Config Failure Indicates ETNA devic...

Page 21: ... show environment fan Wed Nov 11 02 04 58 161 UTC Fan speed rpm Location FRU Type FAN_0 0 FT0 NCS1K FTA 4800 0 FT1 NCS1K FTA 4800 0 FT2 NCS1K FTA 4680 0 PM1 NCS1K 2KW AC 8064 The table below lists the temperature threshold values for the different fan speeds Falling Min Temperature C Falling Max Temperature C Rising Max Temperature C Rising Min Temperature C Fan speed rpm 127 27 28 127 4800 28 29 ...

Page 22: ...69 4A Location Card Type Power Power Status Allocated Used Watts Watts 0 0 S L NCS1K P0 70 RESERVED 0 RP0 NCS1002 RP 35 ON 0 FT0 NCS1K FTA 40 ON 0 FT1 NCS1K FTA 40 ON 0 FT2 NCS1K FTA 40 ON The following example shows sample output from the show environment command with the voltages keyword sysadmin vm 0_RP0 show environment voltages location 0 RP0 Thu Aug 6 09 35 09 211 UTC Location VOLTAGE Value ...

Page 23: ...o QSFP28 100G LR4 Pluggable Optics Module PID SPQCELRCDFB VID 01 SN GAV2008935 NAME 0 0 Optics0 0 0 11 LANE1 DESCR Cisco 10G SFP LR Pluggable Optics Module PID SFP 10G LR VID V01 SN SPC190707YP NAME 0 0 Optics0 0 0 17 LANE1 DESCR Cisco 10G SFP SR Pluggable Optics Module PID SFP 10G SR VID V03 SN JUR1904073P NAME 0 0 Optics0 0 0 18 DESCR Non Cisco QSFP28 100G LR4 Pluggable Optics Module PID FTLC115...

Page 24: ...sion Displays the firmware information of various hardware components of the NCS 1002 The following example is for Release 6 0 1 Example RP 0 RP0 CPU0 ios show hw module fpd Tue Apr 12 09 04 14 935 UTC FPD Versions Location Card type HWver FPD device ATR Status Running Programd 0 0 NCS1002 2 4 CDSP_PORT_05 CURRENT 3 56 3 56 0 0 NCS1002 2 4 CDSP_PORT_06 CURRENT 3 56 3 56 0 0 NCS1002 2 4 CDSP_PORT_1...

Page 25: ... In the above output DP FPGA Version indicates the image of the datapath FPGA Here F 203 is the image version of the 40 G image The CURRENT value of the HW Status parameter indicates that the firmware version is the latest T indicates 10 G and H indicates 100 G image versions If Need UPG appears in the output you must upgrade the slice to get the updated DP FPGA using the upgrade hw module slice s...

Page 26: ... 0 NCS1002 CFP2_PORT_05 NOT READY 0 0 NCS1002 2 0 CFP2_PORT_06 CURRENT 4 38 4 38 0 0 NCS1002 CFP2_PORT_12 NOT READY 0 0 NCS1002 CFP2_PORT_13 NOT READY 0 0 NCS1002 CFP2_PORT_19 NOT READY 0 0 NCS1002 2 1 CFP2_PORT_20 CURRENT 5 19 5 19 0 0 NCS1002 CFP2_PORT_26 NOT READY 0 0 NCS1002 CFP2_PORT_27 NOT READY 0 0 NCS1002 0 1 CTRL_BKP_LOW B CURRENT 1 22 0 0 NCS1002 0 1 CTRL_BKP_UP B CURRENT 1 22 0 0 NCS100...

Page 27: ... ios show hw module slice 2 Tue Apr 12 09 09 24 527 UTC Slice ID 2 Status Provisioned Client Bitrate 40 Trunk Bitrate 200 DP FPGA Version F 203 HW Status CURRENT Client Port Trunk Port CoherentDSP0 0 0 20 Traffic Split Percentage FortyGigECtrlr0 0 0 14 100 FortyGigECtrlr0 0 0 15 100 FortyGigECtrlr0 0 0 16 100 FortyGigECtrlr0 0 0 17 100 FortyGigECtrlr0 0 0 18 100 In the above output DP FPGA Version...

Page 28: ...fiber and pluggables using this loopback Internal loopback An internal loopback tests the data path as it passes through various components of the device and loops back After applying an internal loopback on a port use a test set to run traffic over the loopback You can verify issues related to the programming of the device using this loopback You can use loopback to troubleshoot some of the follo...

Page 29: ...ig eth ctrlr loopback internal b loopback line Configures line loopback Example RP 0 RP0 CPU0 ios config eth ctrlr loopback line Step 6 commit Saves the configuration changes and remains within the configuration session Example RP 0 RP0 CPU0 ios config eth ctrlr commit Step 7 You can verify the internal or line loopback configuration using the following show commands a show controllers controllert...

Page 30: ...ntenance snip Using Link Layer Discovery Protocol Snooping LLDP snooping is enabled on the Ethernet controllers when you provision the controllers You can use LLDP snooping to troubleshoot problems in the client ports For example to verify the far end device connected to the client interface You can troubleshoot connectivity issues using LLDP snooping using the following procedure show controllers...

Page 31: ...ection on the trunk side Use the following procedure to configure and verify the TTI Step 1 controller coherentDSP R S I P tti sent expected ascii string Configures the transmit and expected TTI strings The ASCII text string can be a maximum of 64 characters The TTI string has to be configured on both the trunk ports that are inter connected to each other If a pattern mismatch occurs a TIM alarm i...

Page 32: ...0000000000000000000000 OTU TTI Received OPERATOR SPECIFIC ASCII abc OPERATOR SPECIFIC HEX 616263000000000000000000000000 00000000000000000000000000000000 OTU TTI Expected OPERATOR SPECIFIC ASCII abc OPERATOR SPECIFIC HEX 616263000000000000000000000000 00000000000000000000000000000000 FEC mode Soft Decision 7 Network SRLG values Not Configured Step 3 show alarms brief card location R S I P active D...

Page 33: ...larm Information LOS 2 LOF 1 LOM 2 OOF 0 OOM 0 AIS 0 IAE 0 BIAE 0 SF_BER 0 SD_BER 0 BDI 0 TIM 0 FECMISMATCH 0 FEC UNC 0 Detected Alarms LOS OTU TTI Sent OPERATOR SPECIFIC ASCII OPERATOR SPECIFIC HEX 00000000000000000000000000000000 00000000000000000000000000000000 OTU TTI Expected OPERATOR SPECIFIC ASCII OPERATOR SPECIFIC HEX 00000000000000000000000000000000 00000000000000000000000000000000 FEC mo...

Page 34: ...B Polarization Change Rate 3 00 rad s Differential Group Delay 7 30 ps In the above output you can verify the state of the controller LED state wavelength TX power RX power OSNR and the alarms Step 3 If there is an LOS alarm on the trunk port a Verify the fiber continuity to the port of the NCS 1002 and fix the fiber connection b Verify the wavelength and the channel mapping of the optics controll...

Page 35: ...7 1 07 1 07 23 98 NO 7 5 NO In the following example Lane 2 is monitored within the Optics 0 1 0 0 corresponding to the 10G Ethernet controller 0 1 0 0 2 RP 0 RP0 ios show controllers optics 0 1 0 0 pm current 15 min optics 2 Tue Feb 10 14 59 10 936 UTC Optics in the current interval 14 45 00 14 59 11 Tue Feb 15 2015 Optics current bucket type Valid MIN AVG MAX Threshold Min TCA enable Threshold M...

Page 36: ... 09 dBm 10 50 dBm Step 3 show tech support ncs1k Collects the output logs What to do next Verify the PM parameters of the Ethernet Controller For more information on these parameters see Verifying the Performance Monitoring Parameters of Controllers on page 29 Troubleshooting Breakout Patch Panel Step 1 show tech support ncs1k Collects the output logs to troubleshoot breakout patch panel Step 2 Co...

Page 37: ...displaced BIOS will boot however Cisco IOS XR does not boot due to insufficient memory If both DIMMs are displaced BIOS will not boot In both the scenarios it is recommended to remove and re insert DIMMs on the controller card Before you begin Follow the standard electrostatic discharge ESD rules according to local site practice before replacing DIMMs Step 1 Remove DIMMs Push the connector latches...

Page 38: ...hannel mapping for optics controllers RP 0 RP0 CPU0 ios show controllers optics 0 0 0 11 dwdm carrrier map Thu Aug 27 15 59 00 385 UTC DWDM Carrier Band C Band MSA ITU channel range supported 1 97 DWDM Carrier Map table ITU Ch G 694 1 Frequency Wavelength Num Ch Num THz nm 1 60 196 10 1528 773 2 59 196 05 1529 163 3 58 196 00 1529 553 4 57 195 95 1529 944 5 56 195 90 1530 334 6 55 195 85 1530 725 ...

Page 39: ... displays the current performance monitoring parameters of the Optics controller in 15 minute intervals Example RP 0 RP0 ios show controllers optics 0 1 0 0 pm current 15 min optics 1 Tue Feb 10 14 59 06 945 UTC Optics in the current interval 14 45 00 14 59 05 Tue Feb 15 2015 Optics current bucket type Valid MIN AVG MAX Threshold Min TCA enable Threshold Max TCA enable LBC mA 735 735 735 0 NO 0 NO...

Page 40: ...alid ES NE 0 Threshold 500 TCA enable YES ESR NE 0 00000 Threshold 0 00000 TCA enable NO SES NE 0 Threshold 500 TCA enable YES SESR NE 0 00000 Threshold 0 00000 TCA enable NO UAS NE 0 Threshold 500 TCA enable YES BBE NE 0 Threshold 10000 TCA enable YES BBER NE 0 00000 Threshold 0 00000 TCA enable NO FC NE 0 Threshold 10 TCA enable YES ES FE 0 Threshold 500 TCA enable YES ESR FE 0 00000 Threshold 0...

Page 41: ...d operation and traffic is not impacted If hitless restart is not enabled the slice is initialized in the stateless mode and traffic is impacted Example RP 0 RP0 CPU0 ios show hw module slice 1 internal Thu Nov 19 03 46 35 968 UTC Slice ID 1 Status Provisioned Client Bitrate 10 Trunk Bitrate 100 Headless Internal Information State data 0xA1B2C3D4 In the above example the State data is 0xA1B2C3D4 I...

Page 42: ...visioning of the slice to prevent any traffic impact Example RP 0 RP0 CPU0 ios show hw module slice 0 internal Tue Feb 9 05 24 48 075 UTC Slice ID 0 Status Reprovisioning Aborted DP Access Failure hitless reload Client Bitrate 10 Trunk Bitrate 100 Headless Internal Information State data 0xA1B2C3D4 Step 2 show alarm brief card location location active Displays active alarms You can verify if the e...

Page 43: ...or changed on that slice Example The following example displays the statistics collected for the Ethernet controller during the last headless operation RP 0 RP0 CPU0 ios show controllers fortyGigECtrlr 0 0 0 7 headless stats Thu Aug 30 06 32 58 936 UTC Started in Stateful mode Yes Headless Start Time Thu Aug 30 06 31 09 2018 Headless End Time Thu Aug 30 06 32 34 2018 Ethernet Headless Statistics R...

Page 44: ...f operation Headless End Time Time at which the NCS 1002 came out of the headless mode SmBip Section Monitoring Backward Error Indicator SmBei Section Monitoring Bit Interleaved Parity Fec EC Forward error correction Errors Corrected Fec UC Forward Error Correction Uncorrected Words Slices that start in the stateful mode are not reset during the last CPU OIR or reload operation Hence the traffic i...

Page 45: ... the number of times an operation was done on an OID show snmp mib access time Displays the timestamp of the last operation on an OID show snmp trace requests Displays a log of the high level PDU processing trace points debug snmp packet Displays information about every SNMP packet sent or received by the NCS 1002 debug snmp requests Displays information about every SNMP request made by the SNMP m...

Page 46: ...ng clients Tue Dec 8 07 49 14 846 UTC Netconf clients client session ID NC version client connect time last OP time last OP type lock 1188487019 1 1 0d 16h 56m 50s 01 17 13 get No 3445210079 1 1 0d 16h 56m 38s 01 16 54 get No 3027026318 1 1 0d 16h 56m 1s 01 16 50 get No 2653293062 1 1 0d 16h 56m 33s 01 16 53 get No 96573454 1 1 0d 16h 56m 15s 01 17 13 get No 2771481091 1 1 0d 16h 56m 45s 01 17 14 ...

Page 47: ...ce Debugs and verifies Netconf Example RP 0 RP0 CPU0 ios show netconf yang trace Tue Dec 8 07 50 54 590 UTC 12 08 15 07 30 37 851 UTC 1046d3 4942 TRC nc_sm_session_find_session_id 1386 Found session 3027026318 0x1852f68 12 08 15 07 30 37 851 UTC 1046d4 4942 DBG nc_sm_yfw_response_cb 2816 Received OK response for session id 3027026318 for message id 856615 which has NO ERROR and DATA 12 08 15 07 30...

Page 48: ...01 13 34 34 HundredGigECtrlr0 0 0 14 Carrier Loss On The LAN In the maintenance mode all alarms are suppressed and the show alarms command will not show the alarms details Use the show controllers controllertype R S I P command to view the client and trunk alarms Note What to do next For more information about alarms and steps to clear them see the Alarm Troubleshooting on page 45 Using Onboard Fa...

Page 49: ...e Count 4 UPTIME INFORMATION Start Time Stamp End Time Stamp Card Uptime info mm dd yyyy hh mm ss mm dd yyyy hh mm ss Weeks Days Hrs Min Sec 11 27 2015 20 44 32 11 27 2015 22 11 18 0 0 1 26 46 11 27 2015 22 11 18 11 28 2015 17 21 59 0 0 19 10 41 11 28 2015 17 21 59 11 28 2015 17 51 59 0 0 0 30 0 Capturing Logs Step 1 show logging Displays the contents of the logging buffers You can also view detai...

Page 50: ...t alarm mgr Sat Jan 30 21 41 53 894 UTC Show tech start time 2016 Jan 30 214154 UTC Sat Jan 30 21 41 56 UTC 2016 Waiting for gathering to complete Sat Jan 30 21 44 02 UTC 2016 Compressing show tech output Show tech output available at 0 RP0 CPU0 harddisk showtech showtech alarm_mgr 2016 Jan 30 214154 UTC tgz Show tech end time 2016 Jan 30 214402 UTC Step 4 show tech support ptah Collects the Cisco...

Page 51: ...bytes total 935528 kbytes free What to do next You should gather the above information before calling the Cisco Technical Assistance Center TAC Verifying Process Details and Crash Dump Step 1 show processes Displays information about active processes Example The following example shows the output of the show processes command in the EXEC mode RP 0 RP0 CPU0 ios show processes File saved at 17 22 13...

Page 52: ...l core MAINMEM Max core 0 Level 200 Placement None startup_path opt cisco XR packages ncs1k os support 2 0 0 0 r61101I rp startup mxp_driver startup Ready 134 140s Process cpu time 0 000 user 15061 040 kernel 15061 040 total JID TID Stack pri state NAME rt_pri 148 3795 0K 20 Sleeping Infra Agent 0 148 4176 0K 20 Sleeping lwm_debug_threa 0 148 4177 0K 20 Sleeping mxp_driver 0 148 4181 0K 20 Sleepin...

Page 53: ...11 20150423 083922 sysadmin vm 0_RP0 009d5 core gz Core dump time 2015 04 23 08 39 23 058000000 0000 Process Core was generated by opt cisco calvados bin aaad Build information XR Information User aaaa Host agl ads 2232 Workspace nobackup aaaa xspeed new Lineup proj xspeed XR version 6 0 0 01I Signal information Program terminated with signal 11 Segmentation fault Faulting thread 1463 Registers fo...

Page 54: ...Troubleshooting Guide for Cisco NCS 1002 IOS XR Release 6 3 x 44 General Troubleshooting Verifying Process Details and Crash Dump ...

Page 55: ...logical objects These logical objects represent physical objects such as cards ports and logical objects such as trunk The table below lists all logical alarm objects used in this chapter Table 1 Alarm Logical Object Type Definitions Definition Logical Object Fan tray assembly FT Pluggable port module PPM also called SFP referring to MXP and TXP cards PPM Power Supply Unit PSU Chassis equipment Ch...

Page 56: ...FT The ALL FAN TRAY REMOVAL alarm is raised on the NCS 1002 when all the fan trays are removed from the chassis Clear the ALL FAN TRAY REMOVAL Alarm Insert all three fans into the chassis If the alarm does not get cleared you need to report a Service Affecting SA problem Log into the Technical Support Website at http www cisco com c en us support index html for more information or log into http ww...

Page 57: ...rity Critical CR Service Affecting SA Logical Objects EQUIPMENT The DAISYDUKE CPU FPGA PCIE ERROR alarm occurs when the Daisy Duke CPU FPGA is unable to communicate with the CPU controller due to a Peripheral Component Interconnect Express PCIe error Clear the DAISYDUKE CPU FPGA PCIE ERROR Alarm If the alarm does not get cleared you need to report a Service Affecting SA problem Log into the Techni...

Page 58: ...f 0 01 ps Step 3 If the value is within the range of the differential group delay threshold contact Cisco TAC If the condition does not clear log into the Technical Support Website at http www cisco com c en us support index html for more information or call Cisco TAC 1 800 553 2447 EQPT FAIL SLICE Alarm Default Severity Critical CR Service Affecting SA Logical Objects Slice The Equipment Fail EQP...

Page 59: ...SUPPORTED alarm is raised when unsupported power module is inserted in PEM slot The unsupported power module is NCS1K 2KW AC2 Clear the FAM FAULT TAG PMO UNIT UNSUPPORTED Alarm Replace the power module with the power module supported in NCS 1002 If the alarm does not get cleared you need to report a Service Affecting SA problem Log into the Technical Support Website at http www cisco com c en us s...

Page 60: ...ING Alarm Step 1 Verify that a fan is missing or is not correctly inserted Step 2 Insert a fan The fan should run immediately when correctly inserted If the alarm does not get cleared you need to report a Service Affecting SA problem Log into the Technical Support Website at http www cisco com c en us support index html for more information or log into http www cisco com c en us support web tsd ci...

Page 61: ...controller optics R S I P lbc high threshold command in the config mode Laser bias typically starts at about 30 percent of the manufacturer maximum laser bias specification and increases as the laser ages If the HI LASERBIAS alarm threshold is set at 100 percent of the maximum the laser usability has ended If the threshold is set at 90 percent of the maximum the card is still usable for several we...

Page 62: ...shold Step 4 Change the threshold value or use an attenuator to reduce the input power to the desired level If the condition does not clear log into the Technical Support Website at http www cisco com c en us support index html for more information or call Cisco TAC 1 800 553 2447 HI TXPOWER Alarm Default Severity Minor MN Non Service Affecting NSA Logical Objects PPM The Equipment High Transmit P...

Page 63: ...ue to this limitation the end user cannot be able to potentially see HIBER through CLI since Ncs1k XR Alarm reporting soak time is only 2 seconds for raising Clear the HIBER Alarm The alarm clears under the following conditions When the card port does not receive a high bit error rate When the optical connectors are cleaned If the alarm does not get cleared you need to report a Service Affecting S...

Page 64: ... threshold range configure the low receive power threshold using the controller optics R S I P rx low threshold command in the config mode The range is from 400 to 300 dBm in units of 0 1 dBm Step 3 If the value is within the range of the low receive power threshold physically verify by using a standard power meter that the optical input power overcomes the expected power threshold Ensure that a p...

Page 65: ... If the value is within the range of the low transmit power threshold physically verify by using a standard power meter that the optical output power is overcoming the expected power threshold If so the pluggable module should be replaced at first opportunity If the condition does not clear log into the Technical Support Website at http www cisco com c en us support index html for more information...

Page 66: ...blem does not clear the transmitter at the other end of the optical line could be failing and require replacement If the alarm does not get cleared you need to report a Service Affecting SA problem Log into the Technical Support Website at http www cisco com c en us support index html for more information or log into http www cisco com c en us support web tsd cisco worldwide contacts html to obtai...

Page 67: ...ree Technical Support numbers for your country MEA Alarm Default Severity Critical CR Service Affecting SA Logical Objects PPM The Missing Equipment Attributes MEA alarm for the PPM or QSFP is raised on a pluggable port of the NCS 1002 when there is a mismatch in the configured client data rate and the supported QSFP physical data rate For example if the configured client data rate is 10G and the ...

Page 68: ...KA session becomes locally secured due to the loss of an active programmed peer A node loses its peer under the following conditions When the node shuts down its interface When the node issues no macsec command for more than 6 times Clear the MKA LOCALLY SECURED Alarm The alarm clears after confirming the macsec configuraiton and the interface state on the nodes If the alarm does not get cleared y...

Page 69: ... to the expiry of existing key If the alarm does not get cleared you need to report a Service Affecting SA problem Log into the Technical Support Website at http www cisco com c en us support index html for more information or log into http www cisco com c en us support web tsd cisco worldwide contacts html to obtain a directory of toll free Technical Support numbers for your country MORE THAN ONE...

Page 70: ...ion or log into http www cisco com c en us support web tsd cisco worldwide contacts html to obtain a directory of toll free Technical Support numbers for your country NOT ABLE TO COMMUNICATE WITH PATCH PANEL Alarm Default Severity Minor MN Logical Objects Software This alarm is raised on NCS 1002 when communication with the breakout patch panel is lost The alarm is also raised when the connected i...

Page 71: ...erity Not Reported NR Non Service Affecting NSA Logical Objects TRUNK An Alarm Indication Signal AIS signal communicates to the receiving node when the transmit node does not send a valid signal AIS is not an error The OTUK AIS alarm is raised by the receiving node on each input when it detects the AIS instead of a real signal OTUK AIS is a generic AIS signal with a repeating AIS PN 11 sequence Th...

Page 72: ...tical Transport Unit Loss of Frame OTUK LOF alarm is raised when a frame loss is detected by an invalid frame alignment in the received frames This alarm indicates that the card has lost frame delineation on the input data Loss of frame occurs when the optical transport unit overhead frame alignment FAS area is invalid for more than five frames and that the error persists more than three milliseco...

Page 73: ...hnical Support Website at http www cisco com c en us support index html for more information or call Cisco TAC 1 800 553 2447 OTUK TIM Alarm Default Severity Major MJ Service Affecting SA Logical Objects PORT The OTUK TIM alarm or the Trail Trace Identifier TTI Mismatch alarm is raised on the NCS 1002 when the expected TTI string does not match the received section trace string Clear the OTUK TIM ...

Page 74: ...ally If the condition does not clear log into the Technical Support Website at http www cisco com c en us support index html for more information or call Cisco TAC 1 800 553 2447 PEM MODULE REMOVAL Alarm Default Severity Major MJ Service Affecting SA Logical Objects PEM The PEM MODULE REMOVAL alarm is raised on the NCS 1002 when one of the redundant PEM module is removed from the system Clear the ...

Page 75: ...problem Log into the Technical Support Website at http www cisco com c en us support index html for more information or log into http www cisco com c en us support web tsd cisco worldwide contacts html to obtain a directory of toll free Technical Support numbers for your country PLL FAIL SLICE Alarm Default Severity Major MJ Service Affecting SA Logical Objects SLICE The PLL FAIL SLICE alarm is ra...

Page 76: ...e is removed when encrypted slice is available The macsec_mka_aipc_driver library is contained in the k9sec package Clear the PKG K9SEC REQ Alarm This alarm clears after you install the k9sec package containing the macsec_mka_aipc_driver library If the alarm does not get cleared you need to report a Service Affecting SA problem Log into the Technical Support Website at http www cisco com c en us s...

Page 77: ...eared you need to report a Service Affecting SA problem Log into the Technical Support Website at http www cisco com c en us support index html for more information or log into http www cisco com c en us support web tsd cisco worldwide contacts html to obtain a directory of toll free Technical Support numbers for your country REMOTE FAULT Alarm Default Severity Major MJ Service Affecting SA Logica...

Page 78: ...r more information or call Cisco TAC 1 800 553 2447 SIGLOSS Alarm Default Severity Major MJ Service Affecting SA Logical Objects CLIENT The Signal Loss on Data Interface alarm is raised on the client side QSFP of the NCS 1002 when there is a loss of Ethernet signal Clear the SIGLOSS Alarm Step 1 Ensure that the port connection at the near end of the client peer router is operational Step 2 Verify ...

Page 79: ...configured under the client controller Laser squelch is supported only on QSFP QSFP28 LR4 and QSFP28 CWDM4 pluggables Note Laser squelching occurs on a QSFP pluggable when all the four lanes operating in the 10 GE client mode are turned off after the upstream receive facility has experienced a loss of signal such as optical LOS LOF A MXP or TXP client facility detects that an upstream receive faci...

Page 80: ...l CR Service Affecting SA Logical Objects EQUIPMENT The temperature alarms are raised on the NCS 1002 when the temperature is not within the operating range When this condition occurs critical devices like DSP and ETNA automatically shut down The alarm can appear in one of the following formats sensor name high temperature alarm sensor name low temperature alarm The sensor name high temperature al...

Page 81: ...isco com c en us support web tsd cisco worldwide contacts html to obtain a directory of toll free Technical Support numbers for your country TX POWER PROV MISMATCH Alarm Default Severity Minor MN Non Service Affecting NSA Logical Objects PPM The Provisioned Optics Transmit Power Not Supported TX POWER PROV MISMATCH alarm is raised when the laser transmit power configured is not within the range of...

Page 82: ... equipment consult the manufacturer Step 4 If the optical power level is good verify that the optical receive levels are within the acceptable range Step 5 If receive levels are good clean the fibers at both ends Step 6 If the condition does not clear verify that a single mode fiber is used Step 7 Verify if the fiber is of single mode type Step 8 Clean the fiber connectors at both ends for a signa...

Page 83: ...agement All sensors managed at the rack level have RM as the prefix RM denotes Rack management Some alarms that come in this category have the alarm tags as FAM FAULT NM L FAM FAULT NM H FAM FAULT RM L FAM FAULT RM H Clear the VOLTAGE Alarms This alarm clears when the voltage falls within the operating range Verify the voltage of the power source The voltage rating value for AC power ranges betwee...

Page 84: ...problem Log into the Technical Support Website at http www cisco com c en us support index html for more information or log into http www cisco com c en us support web tsd cisco worldwide contacts html to obtain a directory of toll free Technical Support numbers for your country Troubleshooting Guide for Cisco NCS 1002 IOS XR Release 6 3 x 74 Alarm Troubleshooting Clear the WVL OUT OF LOCK Alarm ...

Page 85: ...51 HI RXPOWER 52 HI TXPOWER 52 HIBER 53 I IMPROPRMVL 53 L LLDP snooping 20 LO RXPOWER 54 LO TXPOWER 55 LOCAL FAULT 55 logs 38 LOM 56 loopback 18 LOS P 56 LOWER CTRL FPGA PCIE ERROR 57 M Management Interface 8 MEA 57 MIXED AC DC PTS 58 MKA LOCALLY SECURED 58 MKA START FAIL MISSING PSK 58 MKA UNSECURE MISSING PSK 59 MORE THAN ONE FAN TRAY REMOVED 59 N Netconf 35 NM I2C ACCESS ERROR 59 NM SHUTDOWN CA...

Page 86: ...s 39 REMOTE FAULT 67 RM DEVICE OUT OF TOL 68 S SIGLOSS 68 Slice Provisioning 10 troubleshooting 10 Software Package 1 Troubleshooting 1 SQUELCHED 69 SYNCLOSS 68 T TEMPERATURE 70 troubleshoot 31 headless state settings 31 hitless restart 31 trunk port 23 TT1 21 TX POWER PROV MISMATCH 71 U UNC WORD 72 unreachable node 3 UPPER CTRL FPGA PCIE ERROR 72 V VOLTAGE 73 W Wavelength and Channel Mapping 28 W...

Search results

Summary of Contents for NCS 1002

Reviews:

Related manuals for NCS 1002

Brands by name

Popular brands

Cisco NCS 1002, Troubleshooting Manual

Search results

Summary of Contents for NCS 1002

Reviews:

Related manuals for NCS 1002

Brands by name

Popular brands