About 10% of our radios are not allowing an upgrade from 4.6 to 4.6.1. The message is the SM must be rebooted before it will accept new firmware. When we hit the reboot button in the software interface, the radio reboots and does a factory reset apparently as well and never comes back online until we roll a truck and plug into the radio to re-program it.
This happens 100% of the time when we get the “you must reboot first” message and that message is popping up on about 10% of our upgrades. So far, only Force 180, 190 and 200 radios are experiencing this. We have stopped upgrading further until we can figure out what is causing the factory reset and how to stop it.
I have a handful more radios that failed on the upgrade via our cnMaestro. When I log into any of them to download the tech support file to help you truboushoot, I am given this error when I try to download it:
**11:50:49** Error: UI cannot get the resources.
Then the radio goes offline and does not come back up again - just by requesting the tech support file from the radio.
I am dropping radios all over the city after trying to push firmware via cnMaestro. All offending radios were attempted to be upgraded from 4.6 to 4.6.1 and none were rebooted before the firmware was pushed from cnMaestro.
if there is not enough free memory space to accept the new firmware then a reboot is usually required to perform the download and extraction. This does not normally reset the radio to factory defaults.
We updated most of our radios to 4.6.1-rc26 and rc27 prior to updating to 4.6.1-final but we had quite a few that jumped from 4.6.0.1 to 4.6.1 without any real issues. Had to reboot about 6 radios that had uptimes measuring 120+days though.
if you get the UI cannot get resources error, login to the radio via ssh and simply use the reboot command. This fixes the issue that causes this and since it is not a common error issue, we have deemed it to be akin to a file system segment fault which requires a reboot to fix.
what version of cnMaestro are you using? Cloud? on-prem?
a power cycle should clear it too, but that should also be unnecessary as the radio just needs a memory re-initialization.
have you tried the in between firmware 4.6.0.1? can you reproduce this on the test bench?
I would be interested in what your techs says they do when they reprogram the radio. Do they power cycle it to a factory reset or do they just plug in and start programming?
I have accessed two of the radio so far one is a force 180 and one is a force 190. Both of them appear to have been bricked. They are both in a constant reboot cycle and nothing I do will snap them out of it. I am now having to replace these with new hardware.
What brick these was trying to run a firmware upgrade late at night through cnmaestro. When that failed and we logged into the radio to see why we rebooted the radio from the web interface. After that the radio was dead.
This won’t be particularly helpful… but I upgraded over 200 radios the other night, just selected that segment and clicked 4.6.1 and ‘schedule for 3 AM’. That included ePMP1000/Force200 @ 2.4Ghz, and Force300 (13s 16s 25s CSM)
We were mostly upgrading from 4.6.1-RC27, but some of the 2.4Ghz gear was 3.5.6, and some of the F300’s were 4.6.0.1 (because that’s still the ‘recommended’ version for some reason, so we had accidently downgraded by mistake… again grrr! )
Anyway, we had no such issues. So I’m wondering if your SM’s are configured to run extra services or something which is running them out of resources? I don’t really have any concrete suggestions, but maybe there are services or features which can be shut off, and maybe you could reboot them all before you attempt an upgrade, and maybe that would make yours all succeed like ours all did?
These radios are not really doing anything special. They are in bridge mode, no firewall running and no special services.
We ran an upgrade from our cnMaestro on-prem to 39 radios. Almost all were running 4.6 - some were 4.6.0.1. All were N series radios (1000 AP, Force 180, 190, 200 SMs).
Of those 39 upgrades, 7 failed. All 7 have been up for over 170 days without a reboot. All 7 are telling us we need to reboot the radio before we can upgrade it. On 4 of those 7, we requested a reboot from the web interface and all 4 of those are now dead. The radio will power up but the Ethernet link to anything just goes up and down. We can not ping 169.254.1.1.
The radios did not loose power during the upgrade process. They are still reporting 172 days uptime to cnMaestro. We logged into one of them and tried to download the tech support file. That process crashed the radio and it is now also dead. We never requested a reboot on that one.
Now that we have a few of these on the bench, I will attempt a tftp upload. We turn off the feature allowing us to factory reset via a power sequence. That burned us in the past with some customers and bad power.
I have attempted a tftp recovery of these radios. No dice. Despite a couple attempts, the radio remains in a reboot state and we are not able to access it via the 169.254 or the 192.161 addresses. It appears they are fried.
If any are in warranty, I suggest you go that route for them.
Stuck in reboot state: does the power light do anything specific other than just come on?
The problem you are describing is a corrupt firmware image, unfortunately none of the f180/190/200s have dual image support. If the tftp recovery doesnt work, try forcing a factory reset by the button method. This sometimes works to force an SM to come up if there is a problem with the bootloader.
Use tftpd to make an unsecured tftp server with windows firewall and all anti-virus programs disabled on a private network. I have had much better success with using a cheap netgear 5 port switch between my laptop and the radio as it keeps the network config alive on the laptop (removes the windows discovery problem). I have had only one radio that I could not recover and it was an F190. Then again it just started to act up and then just as fast died.
This problem exists in the 4.6.1 firmware branch as well. We just attempted to upgrade a Force 200 from 4.6.1 to 4.6.2 via cnMaestro. The radio had been up without a reboot for 88 days. The upgrade got stuck at the validating image portion of the process and never moved past that. I eventually logged into the radio and rebooted it from the web interface. It never came back up and is now bricked. Just like the Force 180s we lost when trying to upgrade from 4.6.0.
There is a memory leak of some kind in this firmware that seems to impact N series radios. We tried to TFTP restore all of our Force 180s that died in the past and about half of them would not allow a restore (and were RMAd). Now we have a Force 200 in that same boat.
So, our new process is to always reboot a ePMP radio prior to attempting a firmware upgrade. Probably good habit but we never needed to do this prior to 4.6.x
We have had to reboot radios when upgrading from 4.6.0 to 4.6.1 but have not had any loose their configuration on upgrade and we have lots of radios out there.
I have such issues at hand now with F180 and I need a solution to it, the radio has been upgraded to 4.6.1 firmware but got stuck when trying to make changes of parameter, it doesn’t save and come up with a message “can’t save password file” I tried factory reset both soft and hard but no changes still retain the old parameters that was previously configured, I upgrade to 4.6.2 and reboot still no change was established, I downgrade it still the same, I tried all my best still the same. What is Probably the cause and the solution to it?
First, always make sure your browser cache is cleared before doing anything. This will ensure your working from the current config save number and not a cached one. Remember that edge and chrome are the preferred browsers for the N based radios (e1000-Integrated, F180,F190,F200)
You should pull the config from the radio prior to upgrading just in case you get one that does this. Normally there is not an issue but it can happen. If your upgrading an F180 from before 4.1 to current you really should factory reset using the power cycle reset option to ensure a clear config file set then import the config back into the radio.
If your doing remote firmware upgrades, if any power problems happen then very weird things can happen to the radio and your guess is as good as any as to what is wrong.