PTP230 - v13.2 - 100% cpu

Lemaitre · November 19, 2014, 2:18pm

Hello,

I am seeing this message in the vent log since I've upgraded a ptp230 link to v13.2. Is this normal ?

CPU Utilization (Cur/Max): (100%/100%)
Total Time : 1975809 us

TASK TASK % RT Tot TASK Tot S T A C K Task PC
NAME PRI RT MAX Cyc Preempt CtxSw (Sz/Cur%/Max%)OV Status Addr
-------------------------------------------------------------------------------------
SYNC 4 ( 0%) 834 6725 0 9 (12284/ 2%/30%) PendEvFlgGrp 0x67ed4
WDOG 5 ( 0%) 51 243 0 6 (12284/ 2%/ 8%) Ready 0xa29b1c
LEDT 6 ( 0%) 72 347 0 6 (12284/ 2%/ 9%) Ready 0x67ed4
DIAG 10 ( 0%) 0 0 0 0 (12284/ 2%/10%) PendEvFlgGrp 0x67ed4
APMT 11 ( 0%) 0 0 0 0 (12284/ 2%/ 9%) PendEvFlgGrp 0x67ed4
trap 14 ( 0%) 0 0 0 0 (12284/ 2%/32%) PendEvFlgGrp 0x67ed4
SESS 15 ( 0%) 0 0 0 0 (12284/ 2%/47%) PendEvFlgGrp 0x67ed4
SOCK 16 ( 0%) 235 757 4 8 (12284/ 6%/26%) Suspend 0x67ed4
COMM 17 ( 0%) 67 67 0 1 (12284/ 2%/29%) PendEvFlgGrp 0x67ed4
VLAN 20 ( 0%) 0 0 0 0 (12284/ 2%/10%) PendEvFlgGrp 0x11
APPT 22 ( 0%) 0 0 0 0 (12284/ 2%/ 9%) PendEvFlgGrp 0x67ed4
ctic 23 ( 0%) 534 3666 8 28 (12284/ 2%/ 9%) Ready 0x67ed4
Inet 24 ( 0%) 0 0 0 0 (12284/ 2%/24%) Suspend 0x67ed4
BDMT 27 ( 0%) 51 96 0 2 (12284/ 2%/ 9%) PendEvFlgGrp 0x67ed4
BDQT 28 ( 0%) 474 2146 0 20 (12284/ 2%/12%) PendEvFlgGrp 0x67ed4
AUTH 31 ( 0%) 0 0 0 0 (12284/ 3%/11%) PendEvFlgGrp 0x67ed4
SNMP 32 ( 0%) 0 0 0 0 (12284/ 3%/40%) Suspend 0x67ed4
teln 34 ( 0%) 66 213 0 4 (12284/ 6%/25%) Suspend 0x67ed4
TEL1 35 ( 0%) 0 0 0 0 (12284/ 2%/ 8%) Suspend 0x67ed4
TEL2 36 ( 0%) 0 0 0 0 (12284/ 2%/ 8%) Suspend 0x67ed4
TEL3 37 ( 0%) 0 0 0 0 (12284/ 2%/ 8%) Suspend 0x67ed4
TEL4 38 ( 0%) 0 0 0 0 (12284/ 2%/ 8%) Suspend 0x67ed4
FTPs 39 ( 0%) 53 199 0 4 (12284/ 7%/27%) Suspend 0x67ed4
ROOT 46 ( 0%) 0 0 0 0 (12284/ 2%/40%) Suspend 0x67ed4
UPDM 47 ( 0%) 0 0 0 0 (12284/ 2%/11%) PendEvFlgGrp 0x67ed4
UPDT 48 ( 0%) 0 0 0 0 (12284/ 2%/ 9%) PendEvFlgGrp 0x67ed4
HTTP 50 ( 0%) 53 206 0 4 (12284/ 7%/27%) Suspend 0x67ed4
PROX 51 ( 0%) 278 633 0 4 (12284/ 7%/27%) Suspend 0x67ed4
TFT0 53 ( 0%) 0 0 0 0 (12284/ 2%/ 8%) PendQ 0x67ed4
nvrm 55 ( 0%) 0 0 0 0 (12284/ 2%/ 9%) PendEvFlgGrp 0x67ed4
PING 56 ( 0%) 0 0 0 0 (12284/ 2%/ 8%) Suspend 0x67ed4
LLDT 57 ( 0%) 185 350 0 2 (12284/ 2%/ 9%) PendEvFlgGrp 0x67ed4
STAT 60 (16%) 19251 341146 6 24 ( 8192/ 3%/14%) Ready 0x67ed4
IDLE 61 (80%) 49944 1619015 68 68 ( 8192/523272%/14%) Ready 0x67cb4
PRI PC ID

Lemaitre · November 19, 2014, 3:10pm

worse:

******System Startup******
System Reset Exception -- Watchdog Reset
Software Version : CANOPY 13.2 BHUL-DES
Board Type : P11
Device Setting : 5.4GHz SISO OFDM - Backhaul - Timing Slave - 0a-00-3e-b0-48-35
FPGA Version : 082914
FPGA Features : DES, Sched;
01/01/2011 : 00:00:02 UTC : : Bridge/OS Core : FatalError() NULL exception reset
01/01/2011 : 00:00:02 UTC :
Stack Dump information:
Current context Task: IDLE
Current Stack: 3%
Max Stack: 14%

then

******System Startup******
System Reset Exception -- Watchdog Reset
Software Version : CANOPY 13.2 BHUL-DES
Board Type : P11
Device Setting : 5.4GHz SISO OFDM - Backhaul - Timing Slave - 0a-00-3e-b0-48-35
FPGA Version : 082914
FPGA Features : DES, Sched;
11/19/2014 : 15:06:07 UTC : : Bridge/OS Core : Time Set

going back to v13.1.3

J_Mandziara · November 19, 2014, 4:13pm

We have received several reports of issues with PTP230's running 13.2. We are working on determining a root cause for this problem now. We apologize for the inconvenience that this may have caused you.

Tom_Dobbertin · November 19, 2014, 4:27pm

Lemaitre - DId you have the chance to grab a CNUT capture prior to falling back to 13.1.3?

Lemaitre · November 19, 2014, 5:03pm

yes, I sent them to Aaron.

Ryan_2 · November 19, 2014, 5:12pm

this is my event log after upgrading to 13.2 on a ptp250 5.7Ghz

downgraded back to 11.2

******System Startup******
System Reset Exception -- Watchdog Reset
Software Version : CANOPY 13.2 BHUL-DES
Board Type : P11
Device Setting : 5.7GHz SISO OFDM - Backhaul - Timing Master - 0a-00-3e-38-6b-e6 - 5775.0 MHz - 20.0 MHz - 1/4 - CC 108
FPGA Version : 082914
FPGA Features : DES, Sched, US/ETSI;
12/31/2010 : 19:00:03 EST : Bridge/OS Core : Acquired sync pulse from Power Port.
12/31/2010 : 19:00:04 EST : : Bridge/OS Core : FatalError() NULL exception reset
12/31/2010 : 19:00:04 EST :
Stack Dump information:
Current context Task: IDLE
Current Stack: 3%
Max Stack: 14%
r0: 00000000 r1: 00c0ffee r2: 00000000 r3: 00000000
r4: 00a20cd8 r5: 0000003d r6: 000000f4 r7: 00000003
r8: 00a2169c r9: 00000000 r10: 001fb854 r11: 0000002e
r12: 00000000 r13: 0000000a r14: f9f8a321 r15: bb694be5
r16: 00000000 r17: 00000000 r18: 00000000 r19: 00000000
r20: 00000000 r21: 00000000 r22: 00000000 r23: 00000000
r24: 00a1e5f4 r25: deadbeef r26: 00578be4 r27: 14031cf0
r28: 00000000 r29: 00307300 r30: deadbeef r31: 00000000
Task Stack Dump:
0x14031c28: 0000001d 00307300 0000001e deadbeef
0x14031c38: 0000001f 00000000 00000000 14031cc8
0x14031c48: 00000000 00a1e5f4 00000800 0003ffff
0x14031c58: 00000000 00a20d3c 00000000 00000000
0x14031c68: 00000000 00000000 00000000 3d050720
0x14031c78: 80000000 0000007b 00000000 00000000
0x14031c88: 00000000 00000000 00a205f4 00000112
0x14031c98: 00000478 49444c45 00000000 00000000
0x14031ca8: 14031c44 000003ca 00002000 00000024
0x14031cb8: 94020070 00000001 14031c9c deffcf04
0x14031cc8: 00000004 0000001b 0000001b 003b5594
0x14031cd8: 00ad2df5 00480f04 00000000 14031cec
0x14031ce8: 001641b0 00000000 00000000 00000000
0x14031cf8: 00000000 00000000 00000000 00000000
0x14031d08: 00000000 00000000 00000000 00000000
0x14031d18: 00000000 00000000 00000000 00000000
0x14031d28: 00000000 00000000 00000000 00000000
0x14031d38: 00000000 00000000 00000000 00000000
0x14031d48: 00000000 00000000 00000000 00000000
0x14031d58: 00000000 00000000 00000000 00000000
0x14031d68: 00000000 00000000 00000000 00000000
0x14031d78: 00000000 00000000 00000000 00000000
0x14031d88: 00000000 00000000 00000000 00000000
0x14031d98: 00000000 00000000 00000000 00000000
0x14031da8: 00000000 00000000 00000000 00000000
0x14031db8: 00000000 00000000 00000000 00000000
0x14031dc8: 00000000 00000000 00000000 00000000
0x14031dd8: 00000000 00000000 00000000 00000000
0x14031de8: 00000000 00000000 00000000 00000000
0x14031df8: 00000000 00000000 00000000 00000000
0x14031e08: 00000000 00000000 00000000 00000000
0x14031e18: 00000000 00000000 00000000 00000000
0x14031e28: 00000000 00000000 00000000 00000000
0x14031e38: 00000000 00000000 00000000 00000000
0x14031e48: 00000000 00000000 00000000 00000000
0x14031e58: 00000000 00000000 00000000 00000000
0x14031e68: 00000000 00000000 00000000 00000000
0x14031e78: 00000000 00000000
12/31/2010 : 19:00:04 EST :
CPU Utilization (Cur/Max): (9%/100%)
Total Time : 0 us

TASK TASK % RT Tot TASK Tot S T A C K Task PC
NAME PRI RT MAX Cyc Preempt CtxSw (Sz/Cur%/Max%)OV Status Addr
-------------------------------------------------------------------------------------
SYNC 4 ( 0%) 0 0 0 34 (12284/ 3%/30%) Ready 0x67ed4
WDOG 5 ( 0%) 0 0 0 13 (12284/ 2%/ 8%) Ready 0x67ed4
LEDT 6 ( 0%) 0 0 0 13 (12284/ 2%/ 9%) Ready 0x67ed4
DIAG 10 ( 0%) 0 0 0 1 (12284/ 2%/10%) PendEvFlgGrp 0x67ed4
trap 14 ( 0%) 0 0 0 2 (12284/ 2%/32%) PendEvFlgGrp 0x67ed4
SESS 15 ( 0%) 0 0 0 9 (12284/ 2%/25%) PendEvFlgGrp 0x67ed4
SOCK 16 ( 0%) 0 0 0 20 (12284/ 6%/29%) Suspend 0x67ed4
COMM 17 ( 0%) 0 0 0 2 (12284/ 2%/11%) PendEvFlgGrp 0x67ed4
EAPR 18 ( 0%) 0 0 0 37 (12284/ 3%/13%) PendEvFlgGrp 0x67ed4
VLAN 20 ( 0%) 0 0 0 3 (12284/ 2%/10%) PendEvFlgGrp 0x67ed4
APPT 22 ( 0%) 0 0 0 1 (12284/ 2%/ 9%) PendEvFlgGrp 0x67ed4
ctic 23 ( 0%) 0 0 0 53 (12284/ 2%/ 9%) Ready 0x67ed4
Inet 24 ( 0%) 0 0 0 3 (12284/ 2%/24%) Suspend 0x67ed4
BDMT 27 ( 0%) 0 0 0 4 (12284/ 2%/ 9%) PendEvFlgGrp 0x67ed4
BDQT 28 ( 0%) 0 0 0 37 (12284/ 2%/ 9%) PendEvFlgGrp 0x67ed4
AUTH 31 ( 0%) 0 0 0 2 (12284/ 3%/11%) PendEvFlgGrp 0x67ed4
SNMP 32 ( 0%) 0 0 0 2 (12284/ 3%/40%) Suspend 0x67ed4
teln 34 ( 0%) 0 0 0 9 (12284/ 6%/25%) Suspend 0x67ed4
TEL1 35 ( 0%) 0 0 0 1 (12284/ 2%/ 8%) Suspend 0x67ed4
TEL2 36 ( 0%) 0 0 0 1 (12284/ 2%/ 8%) Suspend 0x67ed4
TEL3 37 ( 0%) 0 0 0 1 (12284/ 2%/ 8%) Suspend 0x67ed4
TEL4 38 ( 0%) 0 0 0 1 (12284/ 2%/ 8%) Suspend 0x67ed4
FTPs 39 ( 0%) 0 0 0 9 (12284/ 7%/27%) Suspend 0x67ed4
NTP 44 ( 0%) 0 0 0 3 (12284/ 2%/26%) Ready 0x67ed4
NTPS 45 ( 0%) 0 0 0 1 (12284/ 2%/ 8%) Ready 0x67ed4
ROOT 46 ( 0%) 0 0 0 29 (12284/ 2%/40%) Ready 0x67ed4
UPDT 48 ( 0%) 0 0 0 1 (12284/ 2%/11%) PendEvFlgGrp 0x67ed4
HTTP 50 ( 0%) 0 0 0 8 (12284/ 7%/27%) Suspend 0x67ed4
PROX 51 ( 0%) 0 0 0 8 (12284/ 7%/27%) Suspend 0x67ed4
TFT0 53 ( 0%) 0 0 0 1 (12284/ 2%/ 8%) PendQ 0x67ed4
nvrm 55 ( 0%) 0 0 0 1 (12284/ 2%/ 9%) PendEvFlgGrp 0x67ed4
PING 56 ( 0%) 0 0 0 1 (12284/ 2%/ 8%) Suspend 0x67ed4
LLDT 57 ( 0%) 0 0 0 4 (12284/ 2%/ 9%) PendEvFlgGrp 0x67ed4
STAT 60 ( 0%) 0 0 0 44 ( 8192/ 3%/14%) Ready 0x67ed4
IDLE 61 ( 0%) 0 0 0 123 ( 8192/ 3%/14%) Ready 0x4
PRI PC ID
----------------
12/31/2010 : 19:00:00 EST : : Bridge/OS Core :
01/01/2011 : 00:00:00 UTC : File src/common/psos_wrap.c : Line 960 : Bridge/OS Core : Time Set
12/31/2010 : 19:00:00 EST : Bridge/OS Core :

Lemaitre · November 21, 2014, 10:52am

don't know if it's related, but even after the downgrade, we had another issue yesterday on this link.

Bh master is in autosync + freerun.

11/19/2014 : 15:41:10 UTC : Bridge Core : Acquired sync pulse from Timing Port/UGPS.
11/20/2014 : 15:28:25 UTC : : Bridge Core : Loss of sync pulse. Switching to Free Run Mode!
11/20/2014 : 15:28:27 UTC : Bridge Core : Acquired sync pulse from Power Port.

and the link went down

the problem is that there is no cmm on this site, and the master takes its synchro from a ptp100 slave on the timing port. is it possible to have, as in the APs, a "remote BH" option, to be sure the unit won't try the power port ?