cnMaestro Template help - pushing unintended changes

ninedd · December 13, 2022, 6:54pm

So, I need some help or advice on cnMaestro templates please.

A couple days ago, I made the template to disable the INSTALLER, HOME, and READONLY users. I was getting ready to push FW 4.7 to all our ePMP, and I wanted to disable those extra accounts.

My template contained just the three sections for those three accounts, and nothing else. I basically cut and pasted from the examples here.

However, after I applied the template everywhere, and then at 3:00 a.m. I applied the 4.7 firmware to everything – and then the next day, we noticed some of our ePMP had different ‘device names’ than they should have.

We noticed it on backhauls and APs (It’s pretty obvious if an F300 was named ‘Kingston-Slave’ And then the next morning it’s named 'Fred Smith).

So, I suspect the ones that were changed, we’re probably radios that we configured months ago for some purpose or did testing, and then changed our minds or reprogram these for where they are now. So it seems like cnMaestro went and now ‘corrected’ what it thought should have been the device names of these devices? It didn’t change SSIDs or IPs or things like that, so nothing really broke… but we ended up with a bunch of radios which The device name are different (and who knows what else?).

So, is this something we did wrong? I’m confident there was nothing else in the template. Is there some other checkbox needed to be checked on, or checked off, to get it to only affect the sections we intended? My biggest concern is what it may have done to hundreds of SM’s. It was pretty obvious to see (and repair) these changes at a glance on the infrastructure devices, but looking through hundreds of SMs for mislabeled ones will be a daunting task.

Fedor · December 14, 2022, 12:29am

Is the new “old” name displayed on cnMestro only?
Or it’s displayed both on device’s UI and cnMaestro?

Thank you.

ninedd · December 14, 2022, 2:58am

Yes, both. We noticed it in cnMaestro first… looking down our BACKHAULS network in cnMaestro, it was obvious there were some re-named Masters/Slaves. Logging into the radios themselves, the ‘Device Names’ were changed there, although the SSID and IP’s and stuff all seemed correct still.

We then looked in our ‘default’ network, and noticed that some of the AP’s where renamed as well, and we logged into the radios and changed them back. Again, the SSID’s were correct to the IP’s, but the ‘device names’ were incorrect.

So, that was about a half-dozen changes, so not a big deal. My fear is that maybe there might be dozens (or more?) of client SM’s with changed device names now, and I don’t know how we would even find them, or know what to change them back to.

So – I need to apply the SNMP password changes, but I’m afraid to do that now, without understanding what went wrong.

ninedd · December 14, 2022, 3:04am

This was basically the whole template…

{
	"device_props": {
		"cambiumSysAccountsTable": [{
				"cambiumSysAccountsName": "installer",
				"cambiumSysAccountsHash": "",
				"cambiumSysAccountsUID": "2000",
				"cambiumSysAccountsGID": "100",
				"cambiumSysAccountsDir": "\/tmp",
				"cambiumSysAccountsShell": "\/bin\/false"
			},
			{
				"cambiumSysAccountsName": "home",
				"cambiumSysAccountsHash": "",
				"cambiumSysAccountsUID": "3000",
				"cambiumSysAccountsGID": "100",
				"cambiumSysAccountsDir": "\/tmp",
				"cambiumSysAccountsShell": "\/bin\/false"
			},
			{
				"cambiumSysAccountsName": "readonly",
				"cambiumSysAccountsHash": "",
				"cambiumSysAccountsUID": "4000",
				"cambiumSysAccountsGID": "100",
				"cambiumSysAccountsDir": "\/tmp",
				"cambiumSysAccountsShell": "\/bin\/false"
			},
			{
				"cambiumSysAccountsName": "dashboard",
				"cambiumSysAccountsHash": "",
				"cambiumSysAccountsUID": "5000",
				"cambiumSysAccountsGID": "100",
				"cambiumSysAccountsDir": "\/tmp",
				"cambiumSysAccountsShell": "\/bin\/false"
			}
		]
	}
}

Fedor · December 14, 2022, 2:45pm

Thank you for the clarification.
I completely understand your fear.
We are looking into the issue now and I may revert to you for extra details and remote access.

Thank you.

ninedd · December 15, 2022, 5:44pm

Well, yes, I think these did revert to Device Names that had been used in testing previously. They were not ‘random’ strings, they all changed to devices names which were ‘familiar’ labels used while testing in the shop years ago or whatnot.

Au_Wireless · February 12, 2023, 8:05pm

Following. We are not able to keep Device Names through a power cycle (not a reboot). Since 4.7.0, both Device Name and GPS coordinates will revert back to their pre-4.7.0 state upon a power cycle to the radio. Can reproduce on 100% of our 4.7.0 client radios. Spent considerable man hours fixing radios only to find them revert as soon as the radio was power cycled. Something has changed in the json file and I don’t think it’s writing to the correct field anymore. Lasts through reboots but not power cycles.

dkeltgen · March 3, 2023, 2:14pm

We’ve run into this, but we’re running an older version of on-site cn, so maybe this is fixed. The problem, for us, was when we renamed a device that’s already been onboarded in cn via the radio’s web interface. cn will show the new (correct) device name on the main page, however, if you click to the “configuration” tab it will sill show the previous hostname. Then, once you push a config change via cn, the old hostname will get applied to the radio.

ninedd · July 28, 2023, 8:17am

Was there ever any resolution to this? Any suggestions from Cambium, or anyone - any help from anyone would be appreciated

We need to apply another template again, and I’m scared of how much stuff it’s going to wreck. Last time it changed a whole bunch of host names and stuff.

I stopped using templates since, but now I need to go everywhere and turn off that ‘reset on power sequence’ - I thought we had it off everywhere, but the last large power outage, we had dozens of EPMP defaulted. Nothing else on our network defaults during a power outage.

Anyway, I need to apply a template to all AP/SM and last time I did a template, we had a bunch of radios renamed and random stuff changed and reverted.

Is there any update on that? Any suggestions or ideas why that happened, or how I can avoid it?

Fedor · July 30, 2023, 3:36am

Unfortunately, no major updates there.
We were not able to replicate the issue in the lab despite of many hours spent on that.
We would really appreciate it if someone can help us to reproduce the issue and collect the debug information on it.

Thank you.

ninedd · July 30, 2023, 7:31am

That would be nice, but it was when we used cnMaestro to do template and firmware upgrades, so the ePMP units were rebooted as part of that. We noticed them later, a bunch of them named incorrectly. It’s like some of the info, on some of the radios out in the wild, was restored from a previously used configuration from years ago.

But they had all been rebooted since the templates, so no real logs or anything to collect I don’t think.

Fedor · July 31, 2023, 5:17pm

If you can drop the Tech Support file from one of those radios that would help.
There are configuration files stored for every major firmware version on the device, so we can start digging from there.

Thank you.

Jacob_Turner · November 8, 2023, 5:52am

I’m bumping this thread as we are seeing this continue to happen on ePMP radios randomly either when updating software or pushing minor config templates.

Has Cambium come to any conclusions on this behavior?

Edit: We are using cnMaestro cloud, in case that is relevant.

aka · November 8, 2023, 12:10pm

Hello @Jacob_Turner,

please check you PM box. I would like to work on this with you!

Jacob_Turner · November 8, 2023, 2:28pm

Hey @aka , unfortunately it happens randomly. I’ll contact you directly and we’ll see if we can get to the bottom of it.