Need help debugging syslog messages for a bouncing SM

Products ePMP
1

Have a client with a Force 180 on firmware 3.0.0.  It is conencted to a 1000 connectorized (non-GPS) on an onmi also running 3.0.0.  Client is ~200 meters away from AP with clear LOS.

Client reports his conection "flaps" from time to time. Too quick of a "flap" for our SNMP monitoring to catch it but when he reported his last two "flaps", I did see this in the syslog of his SM radio:

Aug 23 12:37:13 4332 Ulysses Way DEVICE-AGENT[8544]: Not received PONG for the last ping (1)
Aug 23 12:37:38 4332 Ulysses Way DEVICE-AGENT[8544]: Not received PONG for the last ping (2)
Aug 23 12:37:54 4332 Ulysses Way DEVICE-AGENT[8544]: Not received PONG for the last ping (3)
Aug 23 12:38:10 4332 Ulysses Way DEVICE-AGENT[8544]: Not received PONG for the last ping (4)
Aug 24 03:49:18 4332 Ulysses Way DEVICE-AGENT[8544]: callback_websocket: LWS_CALLBACK_CLOSED
Aug 24 03:49:25 4332 Ulysses Way DEVICE-AGENT[8544]:  SMs pmac [00:04:56:C8:B8:70]
Aug 24 03:49:25 4332 Ulysses Way DEVICE-AGENT[8544]: Unable to discover cnMaestro URL (re-discover in 307 seconds)
Aug 24 03:49:25 4332 Ulysses Way DEVICE-AGENT[8544]: Attempting (re)connection in 5 minutes

 Those times (12:37 and 03:49) do line up with times he thought he lost conenctions.  I'm not really sure what those mean...

2

Are the on 5180 - 5240 channel? If so I think its another bug, I figured out on 2 of my sectors customers randomly flapped on any of thoes channels. All had -60 -55 db and SnR of 30, well within spec, I got the same messages when they flapped. It was ony 3 of 6 on the one sector but it was never ending, and all on another. Moved them to a 57 channel.

3

Hi Chadwick,

Messages you have attached just says cnMaestro agent on ePMP device can not establish connection to cnMaestro server.

I can assume we should debug messages which were written to Syslog a bit earlier.

It should be something like:

SM disassociated from AP....

Thank you.

4

No, this one is at a channel in 5.7

I am having a number of issues crop up right around the 3.0 upgrade. May be related, may not.

In addition to this SM flapping (with no messages in the log to indicate anything), I have a Force 180 that is acting as an AP for 2 SMs that goes down totally for no reason. It is getting powered from the AUX POE port of a 1000 connectorized. Prior to 3.0, it was rock solid.  Since about the time it upgraded, it goes down every day. It looks like the issue is in the Force 180, not the 1000 connectorized that is feeding it.

And then there is the DFS issue that others are having.  We have had to move ALL of our DFS channel APs off DFS since upgrading to 3.0.  They get constant radar hits on all DFS channels that never used to happen.

5

I need to revisit this.  A few times a night, our SYSLOG server gets hits with a bunch of messages from nearly all of our APs at the same time:

Sep 15 06:16:59 Lookout North 2000 AP DEVICE-AGENT[2094]: callback_websocket: LWS_CALLBACK_CLOSED

Nothing seems to go off line. This same info is repeated in the logs of most other APs.

What causes this?

6

Hi Chadwick,

Regarding your issue with Force 180

Did you check whether Crashlog message is saved on the board?

You can check it by command in CLI:

debug crashlog

Regarding message from syslog:

Sep 15 06:16:59 Lookout North 2000 AP DEVICE-AGENT[2094]: callback_websocket: LWS_CALLBACK_CLOSED

That is message from cnMaestro device-agent.

It can be caused by by many external reasons such as packets loss, reconfiguration, remote end closing, etc.

Thank you.