PMP450b HG SM's & R22 are unable to connect to cnMaestro after 50+ days of uptime

I think we’ve found one, possibly two issues…

  1. After what we think is 50-53 days of uptime, PMP450b HG 3GHz and 5GHz SM’s, running firmware R22, AND running in NAT mode, will no longer be able to connect to cnMaestro. The cnMaestro agent will continue to try over and over but will not be able to connect. The only way to resolve the issue is to reboot the SM. This issue is apparent in more and more SM’s and we now have dozens of examples. The numbers grow everyday as we see the disconnected SM counts in cnMaestro increase every day.

  2. Some SM’s appear to not be connected to cnMaestro, but then, after logging into the SM, the SM will immediately connect to cnMaestro. cnMaestro will then show the SM as being online again. This issue is much more difficult to track, but we believe it’s related to the first issue.

I’m going to log a ticket with Cambium as well, but I’m curious if anyone else is seeing this issue. Thanks!

1 Like

:frowning: That’s not good. Any chance that it happens after 49.7 days? That would be a sign of a 32-bit millisecond counter wrapping.

2 Likes

Thanks for finding it and posting @Eric_Ozrelic.

You might be exactly right, @Simon_King. If you’re right, then 450i/450b/MicroPoP will all be seeing this issue after 49.7 days of uptime running 22.0.

We are investigating and will for sure have a fix in the next release.

In the meantime, if you’re past or approaching that uptime, you can reboot your device and it will reconnect.

1 Like

we noticed a similar issue as well on our network, but only on the SM’s that have the field blank or the field filled in with cloud.cambiumnetworks.com , if you put https://cloud.cambiumnetworks.com in and then run a ping from the radio to cloud.cambiumnetworks.com it generally shows back up online with no reboot. just went through and fixed about 40 today that had this issue. We also noticed on our cnpilot R195 200 and 201s that are deployed. and it was anything onboarded over 45 days that seemed to be affected not 49.7

1 Like

I can confirm we’re seeing a similar issue where after 53 days of uptime, the SMs show offline in cnMaestro but we can login to them via IP just fine. They’re all running NAT and were all fine before this time.

I would prefer not to open a ticket but can share tech files if needed to help resolve the problem. These are 450b 3ghz. The connection status on the SM just keeps showing reconnecting with the 5 minute countdown. They all have internet access.

2 Likes

Thanks for posting. We have the issue understood and fixed and it will be included in 22.1 BETA which we are hoping to release soon.

1 Like

Per the release notes, this was supposedly fixed in 22.0.1. We installed this version about 50 days ago and started to see this behavior for the first time over the last few days.

this seems to be resolved in 22.0.2.
22.0 and 22.0.1 both have the problem though. it affects the Access points and the Subscriber Modules