ePMP SMs running 4.6.0.1 rebooting every ~5 minutes

Since sometime this morning, all but a few SMs running 4.6.0.1 are rebooting exactly every 5 minutes.

I disabled cnMaestro on a few of them to see if some interaction between the two was causing the reboots, but that didn’t help. I’m updating a few of them to 4.6.1 right now to see if that resolves things.

Anyone else seeing similar? There is a thread on the Facebook Cambium group where others seem to be seeing the same or similar.

1 Like

Is there any watchdog configured?
What do logs say?

No watchdog is configured. Nothing in the logs after a reboot. These same SMs report that they are unable to check for firmware updates since this started. I hope there isn’t some hidden watchdog “feature” that broke because something changed on Cambium’s side.

Updating the SMs to 4.6.1 solves the problem. I’m guessing there is some date-related bug in 4.6.0.1.

epmp 1000 SMs running 4.6.0.1 don’t exhibit this behavior. It is only Force 300 units it looks like.

The other “fix” for this behavior is to put a junk string in the cnMaestro URL field for the affected radios. Once they stop talking to cnMaestro the reboots stop.

I’m using the cloud cnMaestro.

1 Like

Whatever mechanism in cnMaestro that is causing these watchdog-like reboots is of incredibly poor design.

I’ve seen a report that certificate errors are logged on affected radios right before reboot, so I would guess maybe an ssl certificate expired somewhere and that has triggered this.

3 Likes

Just woken up to hundreds of alerts with the message “Device Firmware does not meet minimum version required to maintain connectivity to cnMaestro”

What has Cambium done to suddenly break things?

1 Like

Yeah, I also have a bunch of 4.6.0.1 radios doing this, random reboots every 5 mins. As well as other devices prior to 4.6.1 doing this. Cambium should urgently roll back whatever they broke in cnMaestro.

2 Likes

We have reproduced this issue in our lab and working on it with CnMaestro team.

I’m seeing a lot of our ePMP radios throughout our network rebooting 5-7 minutes after reconnecting. Some with older firmware. Started about 16:00hrs on the 18/05/2022 at around 16:00hrs AEST

1 Like

Ok, I removed the cloud settings on an ePMP300-16 V4.4.3 and it is now stable. Been up for 30+mins now. Was rebooting every 5-7mins before the change

Can someone PM me tech support file from affected SM? I need to see configuration. We have some reports but it does not affect all customers with outdated FW.

Hi Andrii - I included two in support ticket 306124 if you have access to that. If not, I can email you directly.

Thank you Jacob, I’ve already found yours!

But we need more of them! So, please, don’t hesitate to send me some or open a support ticket and attach there.

We have a small update. Versions 4.6.1 and 4.6.2 are not affected by this issue for sure.
Work in progress.

I still can’t understand why Cambium didn’t immediately roll back changes to cnMaestro, assuming cnMaestro was changed?

1 Like

Hi @jessearcher!
Fe have found out it has just coincided with CnMaestro update. The issue is not related to this update. Please check the notice below.

1 Like