It is strongly recommended to upgrade ePMP AP and SM devices that are managed by cnMaestro to the latest release, ePMP version 2.5.2-RC6.
The key feature in this software load is a significant improvements in the management protocol between the ePMP device and cnMaestro.
Other than the critical improvement mentioned above, this software version does not introduce any new ePMP features and has a few other minor bug fixes.
This software version is only available via cnMaestro’s software upgrade page.
I keep getting "Device got timed out during update. Last know status was: Sent the software update command to the device"
Then if I reboot the device (aps or sms epmp1000) they then will show offline in cnMaestro. The devices appear online with cns server. Reboot and disable reable remote managment will not do the fix.
Attempted to upgrade SM from 2.5.1 to the recommended 2.5.2-RC6 from Maestro. SM showed as online in Maestro, was accessible and pinging well with less than 40ms response time.
1st attempt - Failed with error message "Device got timed out during update. Last known status was: Sent the software update command to device."
Verified DNS settings, SM shows as " Connected" on the Home screen under management status. Disabled management and saved. Enabled management, retyped the credentials and saved. Device just shows "Connecting..." under management status. Rebooted the SM. After a few minutes, device now shows as "Connected" under management status.
2nd attempt - Failed with error message "Version applied was 2.5.2-RC2 and device came with version 2.5.1"
EDIT: After the second attempt, SM now shows "Connecting..." under management status. Rebooted. Showed "Connecting" again for almost 5 minutes, then showed as "Connected".
Attempted with the same SM, again it failed with the message "Device got timed out during update. Last known status was: Sent the software update command to device." Device is still accessible, pinging well and shows green in Maestro. Rebooted, device stuck at "Connecting" under management status. Several minutes later it showed as Connected, tried again with the same failure message "Device timed out..."
Attempted to re-push 2.5.1 to the same SM to check connectivity. Same failure message.
Attempted another SM, also accessible, showing green in Maestro and returning good pings. Failed with the same above message: "Device got timed out during update. Last known status was: Sent the software update command to device."
Attempted a 3rd SM on a different AP and tower, same failure message as reported above.
According to the User Manual, "Devices supported by cnMaestro need to be able to access https://cloud.cambiumnetworks.com over port 443. Because the devices initiate the connection, they can be located within a private subnet behind a NAT firewall."
This has been checked and is correct on our end. Is there any other setting that may be preventing upgrade communication but allows monitoring?
Is your SM connected to cnMaestro when you tried to upgrade it?
They were all running 2.5.1.
If the upgrade failed, I logged into the radio and rebooted it. Usually the upgrade would then work. Sometimes it needed a second reboot. All upgraded now.
Edit: I also got a few timeouts. They needed several minutes to re-connect to cnMaestro. Some required another flash, some required a reboot and a flash. The process was not seamless, but I'm looking forward to the new stability.
Those of mine that have failed so far (all that I've tried have failed) have been running 2.5.1. I just added another batch and am awaiting the outcome. I'll reboot the radios manually if needed to see if that forces the upgrade.
I've chosen one SM to focus on. It has now failed the update 3 times in as many manual reboots of the radio. I have waited each time to initiate the update until it connects to cnMaestro and shows as online.
In the syslog of the radio I see this if it is helpful. This is post-reboot and update attempt:
Sep 1 00:00:27 hc-***-c110 DEVICE-AGENT[1931]: get_stats_block: connect() failed errno=2
Sep 1 00:00:28 hc-***-c110 DEVICE-AGENT[1931]: get_stats_block: connect() failed errno=2
Sep 1 00:00:28 hc-***-c110 DEVICE-AGENT[1931]: get_stats_block: connect() failed errno=2
Sep 1 00:00:29 hc-***-c110 DEVICE-AGENT[1931]: get_stats_block: connect() failed errno=2
Sep 1 00:00:29 hc-***-c110 DEVICE-AGENT[1931]: get_stats_block: connect() failed errno=2
Sep 1 00:00:29 hc-***-c110 DEVICE-AGENT[1931]: get_stats_block: connect() failed errno=2
Sep 1 00:00:30 hc-***-c110 DEVICE-AGENT[1931]: get_stats_block: connect() failed errno=2
Sep 1 00:00:30 hc-***-c110 DEVICE-AGENT[1931]: get_stats_block: connect() failed errno=2
Sep 1 00:00:30 hc-***-c110 DEVICE-AGENT[1931]: get_stats_block: connect() failed errno=2
Sep 1 00:00:50 hc-***-c110 DEVICE-AGENT[1931]: EINPROGRESS in connect()
Sep 1 00:00:50 hc-***-c110 DEVICE-AGENT[1931]: SSL_ERROR_WANT_READ try again
Sep 1 00:00:50 hc-***-c110 DEVICE-AGENT[1931]: SSL_ERROR_WANT_READ try again
Sep 1 00:00:50 hc-***-c110 DEVICE-AGENT[1931]: Server certificate is verified and it is valid
Sep 1 00:00:50 hc-***-c110 DEVICE-AGENT[1931]: Received Headers : "HTTP/1.1 302 Moved Temporarily
Sep 1 00:00:55 hc-***-c110 DEVICE-AGENT[1931]: SMs pmac [00:04:56:C9:C7:D3]
Sep 1 00:00:57 hc-***-c110 DEVICE-AGENT[1931]: callback_websocket: LWS_CALLBACK_CLIENT_ESTABLISHED
Sep 1 00:00:57 hc-***-c110 DEVICE-AGENT[1931]: handle_cns_msg: MSG_REGISTER_SUCCESS received
Nov 13 23:19:01 hc-***-c110 DEVICE-AGENT[1931]: Not received PONG for the last ping
Nov 13 23:22:15 hc-***-c110 DEVICE-AGENT[1931]: platform_sw_update: ENTRY
Nov 13 23:22:41 hc-***-c110 DEVICE-AGENT[4675]: do_sw_update2: Exited while loop sw_upd state=0
Then it fails again. I've spent all the time on this I can for now.
Re-attempted using the original SM. New error this time, "Skipped the device to update as it went offline."
Rebooted the SM. Prior to reboot, shows 2d21h System Uptime, shows Connected to Maestro in the SM, and shows green and conected in Maestro. After reboot, Green in Maestro, Connected in the SM, smooth double-digit pings to the SM. Started upgrade again. Pings ran for several minutes, then dropped just as the upgrade failed with the error "Device got timed out during update. Last known status was: Sent the software update command to the device."
== Attempted to update an AP with a single SM. AP failed with "Device got timed out during update, last known status was: Sent the software update command to the device." AP dropped no pings during the update attempt. SM then showed the same error.
In order to make sure everything works, we tried to update couple of our device from 2.5.1 to 2.5.2-RC6 now and everything went smooth and I dont see any timeout or error.
If we can do a screenshare or hangout session it would really help us debug your issue.
Please share us your details at rkh001@cambiumnetworks.com so that I can get in touch with you.
I'm still seeing "Device got timed out during update" when attempting to update my radio at home (the only one I can do during business hours on this sector). Mac is 00:04:56:CE:75:8D.
In order to make sure everything works, we tried to update couple of our device from 2.5.1 to 2.5.2-RC6 now and everything went smooth and I dont see any timeout or error.
If we can do a screenshare or hangout session it would really help us debug your issue.
Please share us your details at rkh001@cambiumnetworks.com so that I can get in touch with you.
An update to the situation. We do not use the default SNMP Community String in our network. So, as a troubleshooting method, we changed the SNMP Read-Only and Read-Write strings to the defaults of public/private as seen in a ePMP 1000 on defaults. Tried to upgrade the same SM as previous and it was successful!
Then moved to another SM on the same tower site but connected to a different AP, also successful. Attempted another tower site with 2 SMs connected, AP and both SMs were able to upgrade after changing the SNMP strings back to default.
I have had to reboot one SM, but otherwise the other devices we have tried to upgrade while using the default strings appear to be upgrading succcessfully. Have successfully updated 4 SMs and 5 APs in this manner.
Change the Read-Only string to public, change the Read-Write string to private. Perform the update. Change the strings back to normal settings. We have been in discussion with Cambium about the issue and hope to reach a resolution that will allow Maestro to work with our existing Comunity String settings.
