Counts, Reg-count, re-reg count?

We have the same problem on certain towers with CCM4.

They lost the sync and all SM disconnect (sometimes only 20-30 at time but I'm not sure it's the same problem).

A single AP with a similar problem has beed dowgraded from 14.1.1 (beta 14) to 14.1.1 official and we don't have disconnections from 2 days.

In Generating Sync mode we don't have problems

Okay, 48 hours now and still running without disconnect.  This inspired me to also test a PTP450 that I was having related issues with.

 

PTP450 is the hardware equivalent to the PMP450 SM.  PTP450 cannot be timed via power, so I use a uGPS to provide sync via the sync port.  I am seeing reg counts on separately located PTP450 as well.  I moved a PTP450 link to generate sync instead of sync port timing and for the last 24 hours there have been no new reg counts.  I will continue to test this for the next week as well.

 

I can verify that I am not losing the pulse.  There are no events logged indicating a loss of sync.  My PMP100 APs have not lost sync.  I have personally witnessed a client stop working from both the AP side and the client side.  The AP reports receiving sync, the client reports itself connected.  Communication stops.  The AP seems to end the stalled connection after a timeout.  The SM loses signal.  The SM initiates a new registration.  Everything is back up and running.  A process that consumes about 25 seconds, ten times per day in some cases.

 

This is occurring on every 3Ghz PMP450 we have that receives sync.  The issue appears to affect near-line-of-sight SMs more than line-of-sight subscribers, which threw off my troubleshooting, making this a complicated issue.

 

In summary,  PMP450 with received timing causes stalled connections, which forces SMs to initiate a new registration, logged as a registration (not a re-reg).  The issue occurs on all SMs, but is exacerbated by non-line-of-sight conditions.


@Will wrote:

Okay, 48 hours now and still running without disconnect.  This inspired me to also test a PTP450 that I was having related issues with.

 

PTP450 is the hardware equivalent to the PMP450 SM.  PTP450 cannot be timed via power, so I use a uGPS to provide sync via the sync port.  I am seeing reg counts on separately located PTP450 as well.  I moved a PTP450 link to generate sync instead of sync port timing and for the last 24 hours there have been no new reg counts.  I will continue to test this for the next week as well.

 

I can verify that I am not losing the pulse.  There are no events logged indicating a loss of sync.  My PMP100 APs have not lost sync.  I have personally witnessed a client stop working from both the AP side and the client side.  The AP reports receiving sync, the client reports itself connected.  Communication stops.  The AP seems to end the stalled connection after a timeout.  The SM loses signal.  The SM initiates a new registration.  Everything is back up and running.  A process that consumes about 25 seconds, ten times per day in some cases.

 

This is occurring on every 3Ghz PMP450 we have that receives sync.  The issue appears to affect near-line-of-sight SMs more than line-of-sight subscribers, which threw off my troubleshooting, making this a complicated issue.

 

In summary,  PMP450 with received timing causes stalled connections, which forces SMs to initiate a new registration, logged as a registration (not a re-reg).  The issue occurs on all SMs, but is exacerbated by non-line-of-sight conditions.


Will,

Thanks so much for testing 14.1.2b14. This is our highest priority to address. We are currently unable to reproduce this in our lab, so your feedback is very valuable.

I agree that the problem is around GPS Sync.

We are coming out with a new load this week 14.1.2 (Build 15) that will absolutely provide us with more information in the logs regarding this issue. It also contains a fix that keeps AP's frequecy from jumping, that could address the issue. Would you be willing to run it and give some captures for it?

-Charlie

Senior Staff Software Engineer
Cambium Networks

1 Like

I'll give it a try.

Here i s my last update on the generate sync test:

It's been five days on the PMP450, still sitting at 1 reg, so using generate sync is a verified work-around.  However, I cannot keep this configuration and will absolutely require sync going forward.

It's been four days on the PTP450.  I'm sitting at 3 reg on that one.  Those two additional registrations occurred about two days apart.

Charlie,  I have a CNUT capture for you of an AP and a Client on 14.2(15).  Where should I send it?

FYI, I have six clients on this test site.  One client inparticular is having the most issues since it is installed with near-line-of-sight.  It is equipment with a reflector aiming through trees.  The trees are just sparse enough to make out the tower structure that hold the AP, but barely.

Last night we had rain and the signal went from a -65 to -75.  This crippled the connection and the SM was incrementing the reg count constantly.  Again, not re-reg, just reg counts... same symptoms I've been describing all along.  I performed a sepctrum scan at the AP and there was a flat noise floor of -98.

I tried moving back to generate sync, but that did not fix it for this particular client.  I'm wondering if I should trying gonig back to the older firmware, because it's never been this bad.

Note about Timing:  I see if I change the Device Type in the general tab, I can supposedly turn off power port timing, which I assume will force it to internal GPS.  I haven't been able to do that, because since the rain the internal GPS lost its signal... really getting tired of these terrible bugs on such an expensive product...

Just to give you a frustrating example, I can run a UBNT 5ghz connection through these same trees and not get disconnects.  Sad that a cheapo UBNT  5Ghz radio is more reliable than a 3Ghz Cambium with near-line-of-sight.