ePMP Software Release 4.7 is now available

The first three e3K APs I tried upgrading from 4.6.2 to 4.7 had the same lockup behavior (could only ping, and connected SMs continued to work just fine) The next few APs I power-cycled before upgrading and they upgraded fine. I’ll try to reproduce the issue on the next couple APs I upgrade and will grab support files beforehand.

We have been upgrading our lighter loaded APs to start. We’ve tried about 20 APs and 35-40 SMs. So far, we’ve had 5 APs that did not upgrade from cnMaestro. Error message is: SW update file download failed. If we reboot the AP, then try again from cnMaestro, the upgrade still fails. So far, if we upgrade from the GUI, the upgrade is successful. So far it has been all 2000 APs that fail. Fedor, I will send you a tech support file from one of the APs.

We have also had 2 SMs that upgraded, but did not re-connect to the AP until they were power cycled.

Hi all,

If you are upgrading your devices from older fw version to 4.7 through cnMaestro, please make sure that ePMP devices are getting proper time from NTP server.
Otherwise the incorrect time doesn’t allow to pass the certificate validation and the upgrade fails.

Thank you.

2 Likes

Thanks for the update. :+1: Perhaps that’s an alert that cnMaestro could monitor and report? We all can audit our whole network manually, but it’s there a way for cnM to let us know?

Just upgraded a couple of SM running 4.6.4 to 4.7. These devices are in NAT mode running pppoe-auto on the WAN interface.
Lan address is 192.168.1.1
I now get a red error “Wireless (WAN) and Ethernet (LAN) Networks must not overlap”
They don’t!

I have updated a couple 3000 epmp sectors, all ac clients, and the only issue I have seen so far are random Clients dropping off of Maestro. Log into the said client and re-input cloud address and all is right again.

Just finished moving the whole network to 4.7, no operationg issues except it disables my snmp despite the communities not being default!
Why should I have to re-enable something that is already properly setup?
How about implementing web auth lockouts? 99% of any attempts are not snmp based, they are either ssh, ftp, telnet or web login.
Snmp can easily be secured by adding a hosts/subnets list (linux has this already) or using a properly configured router to block outside snmp requests like your supposed to do, so stop turning off my snmp!

is the community’s name at least 8 characters long?

10 characters and both are different from each other.

By default, after upgrade the radio presents you with a user dialog box and promptly disables snmp regardless of the configuration. Most annoying.

Diabling unused users is normal, but why diable a properly configured option? cant the firmware check the config status for this and pop up a dialog box?
Better yet, add proper security measurs such as limiting whoch IP addresses can talk to the radio and adding a login failure blocker like fail2ban?

1 Like

That’s weird, it happend to me ONLY when the community was less than 8 chars or default one.
I just upgraded ~200 SM/AP to 4.7

our experience seems to be different here, but this still does not change the fact that proper security measures that are available for the version of linux these things run need to be implemented instead of forcing some third party policy on us operators. A simple nag-box and refusal to save config until these important settings are changed is sufficient to make sure Cambium is doing the basics of enforcment. Adding proper security abilities to the devices is more important than deciding what my config should be and changing it without my authorization, again a simple pop-up box with a warning and an option to keep the setting enabled and check if the communities are default and over the minimum character limit is all that was/is needed.

I know I am more than bitter about this and will be for a long time. Security isnt just disabling a service, its a whole system!

hi all, in CNmastro it still says 4.6.1 is recommended. Will this be changed soon? ive upgraded a few links to test 4.7…

1 Like

I’m surprised I’m not seeing anyone else with the issues we have seen. We have ~300 radios that were upgraded to 4.7.0STA - which I understand is the same firmware as 4.7.0 but with the additional 5.9 frequency.

Since that upgrade, we have had issues all over the place. None of these issues existed before 4.7.0:

  • Device Name will not remain through a power cycle. We can set the device name, it forces a reboot, device name stays but as soon as that radio is hard power cycled, the device name reverts back to what it was pre 4.7.0. I can duplicate this on 100% of our radios (Force 180, 190, 200, 300-16, 300-25). Have not tested on an AP yet.

  • GPS coordinates do not remain through a power cycle. Same behavior as Device Name. They will remain through reboot commands but as soon as you hard power cycle the radio, the GPS reverts to the data that was there pre 4.7.0 upgrade. Can reproduce on 100% of my client radios.

  • Radios crash with various commands pushed from cnMaestro. Sometimes a reboot command will crash them, sometimes a data vlan ID change will crash them (the two things we have tested). When they crash, they are up but will not associate with a tower. We can log in locally, they act weird but are accessible and will not associate until they are hard rebooted with a power cycle. The reboot button in the GUI never initiates a reboot when in this state. We have seen this on Force 180, 190, 200 and various 300-x client radios.

  • So far, 4 client radios have reverted to a factory default state when they were sent a vlan ID change command from cnMaestro. We logged in locally (after a truck roll) and it was totally reset to factory. We have the power sequence factory reset option turned off on all radios.

  • Radios lose the ability to be logged into. We can ping the IP and the IP shows up in the AP/Monitor list but trying to access it from the GUI fails. It’s like there is nothing at that IP. This is only fixed with a hard power cycle. Then it works again. See this on all SM models. No pattern we can tell when they decide to hang.

  • Every single ePMP 1000 AP we have will stop responding to SNMP requests once the airtime utilization creeps up. I can’t tell you the threshold yet but in the evening, our SNMP status board is red on most of our 1000 APs showing SNMP is no longer responding. Ping times to the AP remain unchanged. We do not see this on 2000 or 3000 series APs but we do see it on Force 180 radios in AP mode.

As an ePMP customer for 7 years, this is the worst upgrade we’ve been through. I am ready to revert everything back to a previous version but I am afraid of how many radios we will lose control of in that process and how many more truck rolls we will have to make.

Here is our vlan ID change code that is crashing things. You can see it is not doing anything special:

{
    "device_props": {
        "dataVLANEnable":	"1",
	"dataVLANVID":	"301",
        "dataVLANVP":	 "6"

    }
}
3 Likes

We had to stop deployment as well for reasons stated above. Sticking /w 4.6.1 for now.

1 Like

I am going to recommend we downgrade all sites to 4.6.1 as well. This current version is costing us customer churn and way too much support time. So far, 5 truck rolls and two cancelled customers as a result of problems directly related to the upgrade to 4.7.0.

1 Like

Just lost another radio with a simple data VLAN change. Never came back online after saving that one change. Yet another truck roll. I am terrified how many radios (and customers) we are going to lose in the downgrade process. I wish we had any idea what was causing this.

1 Like

We are going to have to downgrade as well. We didn’t see any problems with 4.7 for quite a while, but more & more stations stop passing traffic, we can’t log into gui - but thankfully can be rebooted via cnMaestro.

Also unfortunately means we can’t get any support file downloaded from the radios.

Ticket has been opened, no reply yet.

We’ve been working with Cambium support on this and we do have a clue, it may have something to do with the SNMPd having issues. We also only seen this issue with 802.11n clients. We were able to grab some logs from a wedged SM that was still connected to cnMaestro. Our current strategy is to use cnMaestro to reboot all of our ePMP SM’s late at night on a semi-weekly basis. This issue doesn’t appear to happen until past 7 days of uptime, and it only happens to a small % of our SM’s.

Here’s a dump of those logs:

Jan 3 20:06:24 Client CN7744 kernel: [2019276.450000] SM disassociated from AP[00:04:56:22:29:bf] F=5240 11naht20. Reason: 32 (NO ALLOCATION ON AP)
Jan 3 20:07:05 Client CN7744 kernel: [2019317.240000] SM associated with AP[00:04:56:22:29:bf]
Jan 3 20:07:07 Client CN7744 miniupnpd[3126]: ioctl(s, SIOCGIFADDR, …): Cannot assign requested address
Jan 3 20:07:07 Client CN7744 miniupnpd[3126]: Failed to get IP for interface ath0
Jan 12 14:11:15 Client CN7744 kernel: [2775567.930000] SM disassociated from AP[00:04:56:22:29:bf] F=5240 11naht20. Reason: 33 (GPFs MISS)
Jan 12 16:10:09 Client CN7744 kernel: [2782701.510000] SM associated with AP[00:04:56:22:29:bf]
Jan 12 16:10:11 Client CN7744 miniupnpd[3126]: ioctl(s, SIOCGIFADDR, …): Cannot assign requested address
Jan 12 16:10:11 Client CN7744 miniupnpd[3126]: Failed to get IP for interface ath0
Feb 7 12:32:56 Client CN7744 snmpd[7593]: DFS status: N/A
Feb 8 09:24:01 Client CN7744 snmpd: Watchdog:Abnormal SNMPd stop occured, restarting…
Feb 8 10:56:02 Client CN7744 snmpd: Watchdog:Abnormal SNMPd stop occured, restarting…
Feb 8 10:56:03 Client CN7744 snmpd[7661]: DFS status: N/A
Feb 8 11:08:01 Client CN7744 snmpd: Watchdog:Abnormal SNMPd stop occured, restarting…
Feb 8 11:08:02 Client CN7744 snmpd[9997]: DFS status: N/A
Feb 8 11:32:01 Client CN7744 snmpd: Watchdog:Abnormal SNMPd stop occured, restarting…
Feb 8 11:34:01 Client CN7744 snmpd: Watchdog:Abnormal SNMPd stop occured, restarting…
Feb 8 11:34:02 Client CN7744 snmpd[12503]: DFS status: N/A
Feb 8 12:08:01 Client CN7744 snmpd: Watchdog:Abnormal SNMPd stop occured, restarting…
Feb 8 12:08:03 Client CN7744 snmpd[16077]: DFS status: N/A
Feb 8 19:32:02 Client CN7744 snmpd: Watchdog:Abnormal SNMPd stop occured, restarting…
Feb 9 05:26:01 Client CN7744 snmpd: Watchdog:Abnormal SNMPd stop occured, restarting…

1 Like

I may try this tonight, rather than downgrading everything.

Okay, cambium replied to my ticket, said it is a ‘strange issue, and they have not come across any other account’ such as this. :person_shrugging: So, they didn’t make it sound like the have any clue.

They also say they may need access to a device when the problem is present, I’m not sure how to accommodate that, since we can’t SSH or HTTP into them at that time. Plus of course, the customer is without internet, So The priority is usually rebooting the SM and getting them back online. However, we may do a truck roll and see if we can access it from their Ethernet side of things.

For us, I’m pretty sure it has been a mix of ePMP 1000 / Force 200 as well as Force 300. I will triple check with the staff. Virtually all of our 1000/200 clients are 2.4 GHz, almost all of our 5ghz is 3000/300, but there are a few straggling 1000/180/200s in 5ghz out there - not very many. But, I will triple check and make sure. :+1:

1 Like