Hi Ben, Justin,
Actually described issue is not in the list of known issues.
I would be really appreciate it if you could send me e-mail at fedir.trutsko@cambiumnetworks.com.
Then we will schedule troubleshooting activities.
Thank you.
I've got the same issue now on a Force 180 SM. A few days after taking the entire tower to 3.2.1, I lost all access to an SM. It is online, passing traffic and doing its job but cnMaestro thinks it is down and I can't get at the web interface.
Sadly, I have SSH turned off for most of our SMs for security reasons. Dumb mistake.
Does this mean I now need to visit this subscrober and physically plug into the radio to fix this?
Most likely this issue is related to snmpd. It is known issue in 3.2.1 release and we are doing our best to fix it asap.
Sometimes snmpd hangs and the only known workaround is to reboot device via power feeding.
However sometimes snmpd may restar and issue will be fixed after that.
So I’d like to recommend you to wait for one day and then if issue will not be solved reboot device.
Sorry for inconveniences.
Thank you.
I'm guessing this also explains why a 1000 GPS AP running 3.2.1 stopped reporting any SNMP data until a reboot?
Hmm..i might agree with SNMPD but why would putting a device on the same subnet fix the problem. I have emailed you if you want to do any further troubleshooting.
-Ben
Hi Ben
Please, check this if it could help
1 Like
Yes that works, that also explains why the laptop is able to connect direct behind it, since I was typing the IP directly. They must of changed something with the headers they are using in version 3.2
Thank you for performed investigation.
We will check and fix this issue asap.
Thank you.
1 Like
May or may not be related to 3.2.1 but a SM that we took to 3.2.1 five days ago seemed to stop passing traffic to the AUX port suddenly. Could not reboot from web interface but SSH reboot worked. Rebooting it brought back the AUX port. We downgraded it to 3.1 as soon as it came back.
@Au Wireless wrote:May or may not be related to 3.2.1 but a SM that we took to 3.2.1 five days ago seemed to stop passing traffic to the AUX port suddenly. Could not reboot from web interface but SSH reboot worked. Rebooting it brought back the AUX port. We downgraded it to 3.1 as soon as it came back.
We've noticed this issue since all the way back to 2.5 when we first started using the aux port. Every once in awhile it freezes up and we have to reboot.
Did this release take care of the random reboot? I'm still running 2.6.1
Thanks,
Yes, it corrected random reboots for us with our 180s
Thanks Chris_Bay,
We are having it with ePMP 1000 GPS Lite's
see anything with those?
Thanks,
I had a few issues with them on the initial release firmware, and the cold weather issue with 2.5.1 but I don’t have the problem on 2.6.2.1 on them. 3.2.1 has solved our 180s that would reboot sometimes
1 Like
@Nick R wrote:I would also like to know if the issue with unable to reboot devices through GUI/SSH has been fixed specifically before upgrading, but can't see a mention of these specifically.
Has this been fixed in 3.2.2 I dont see it specified either way in the notes for the release. We are holding off on any upgrades until its resolved.
I've had 3.2.2 running on a couple sites for a week now and it has been fine for me. I did have some stuff on 3.2.1 and *did* experience the inability to reboot some devices. If you have them onboarded in cnMaestro, then the device agent is still able to reboot them if snmpd is crashed. I've had no issues with this on 3.2.2.
Other than a couple DFS hits after initial updates, which could have certainly been real radar events, and I haven't seen any more, it looks like 3.2.2 is good to go. The faster GUI is soooo much better.
1 Like
@George Skorup wrote:I've had 3.2.2 running on a couple sites for a week now and it has been fine for me. I did have some stuff on 3.2.1 and *did* experience the inability to reboot some devices. If you have them onboarded in cnMaestro, then the device agent is still able to reboot them if snmpd is crashed. I've had no issues with this on 3.2.2.
Other than a couple DFS hits after initial updates, which could have certainly been real radar events, and I haven't seen any more, it looks like 3.2.2 is good to go. The faster GUI is soooo much better.
George,
That faster GUI is one of the reasons I was looking to upgrade. We have stuck with 2.6.x until now. Several of our sites requires snow machines and snow shoes to access so remote upgrades on something buggy is not on my list of fun stuff to do.
No cnMaestro configured yet just have not got around to it. Its on the list.
Ill give it a week or two and see if anyone reports issues and then upgrade.
Thanks for responding.
That faster GUI is one of the reasons I was looking to upgrade. We have stuck with 2.6.x until now. Several of our sites requires snow machines and snow shoes to access so remote upgrades on something buggy is not on my list of fun stuff to do.
There are a few things that we do to ensure successful firmware upgrades.
1. Don't let your radios get too far behind the current/stable firmware version... the larger the gap between the radio's firmware and the current/stable, the more likely you are to have issues.
2. If the radio has a very high uptime... like many months or even years, reboot it first before attempting an upgrade.
3. Make sure that the radio is connected to a power source that you can remotely power cycle, like an APC power control unit, PacketFlux, or LastMile CTM. If it's a backhaul, make sure that whatever device powers it has a watchdog in place to auto-reboot it in case it stops responding.
4. In general, we make sure that all of our radios are configured with a minimal amount of services enabled... don't need syslog? disable it. Don't need SNMP? disable it. Don't need discovery/CDP/LLDP... disable it. Don't need SSH or telnet? Disable it.
In the entire time we've been adhearing to the above rules, we've never bricked an ePMP radio (or an Airmax radio for that matter), and we've never had to drive to a site to reboot a device... although, we have had to occasionally use our remote reboot devices, but it's pretty rare.
1 Like
I have had the issue where you are unable to reboot an AP via the GUI or SSH on 3.2.2. A reboot from cnMaestro works. This has only happened to one AP so far...
One more thing to add... I've had several clients, typically ones with Apple router connected to the ePMP client. The Apple router will put itself in a bridge mode because it detects NAT on the ePMP and it avoids double nat'ing, and ePMP and the Apple will not get along. You have to disable the ePMP's LAN port, then reboot the ePMP, and then you'll be able to upgrade the firmware, then enable ePMP LAN port. As of the GUI refresh, just hitting the reboot button on the ePMP no longer works... you'll have to force a reboot through cnMaestro... assuming the radio is on-boarded. I've reported this issue to Cambium some time ago, fall of last year. This is a 100% reproducible issue... but... still no fix. Investigation of this issue may lead to a fix for many of the issues caused by unresponsive UI, unable to update, and the notorious 'Board still in reboot state' issues.