ePMP2000 - Stop passing data AP to SMs

I've been noticing this somewhat randomly on various AP's now since July of 2015 and have raised this issue with numerous Cambium technical folks with no resolution. Sometimes rebooting the AP makes it go away, or changing the framing mode (from 75/25 to flexible). It happens so sporadically and with only a few AP's that I haven't kept on Cambium because I feel like I'm beating a dead horse and it's incredible difficult to pin point the issue. The micro outages are small enough and happen to so few clients that no one has complained yet. The below graph shows a rather extreme version of the issue that was happening before we switched out an AP or changed the frame timing.. I can't remember which... but it's from back in 2015. Fast forward to today, and we only have one AP that I've been noticing this issue on, and it only has like 10 clients on it, so again, I've just let it go...

Hi Mirko,

Could you please check System Log on SM after issue occurred?

Also I would be really grateful if you could try to downgrade AP to 3.4.1 and monitor whether issue is still reproducible?

You can contact me at fedor.trutsko(at)cambiumnetworks(dot)com for faster communication.

Thank you.

Hi Fedor, thanks for your kindness in person.

We've a ticket opened with cambium support #144858.

Today we've found a PSU voltage that sometimes drops from 50V to 42V.

Could this affect the device with the problem I saw?

We still change it this morning, and also downgrade to 3.2.2 (yesterday)

Now we're waiting for 3 days if the problem occur

Hi Mirko, 

The ePMP 2000 needs at least 44V to operate. So please ensure the PSU supplies that. 

Thanks,

Sriram

Mirko,

I will keep my eyes on the support ticket.

Thank you.

I think I am having the same issue with an AP1000 w/GPS.

http://community.cambiumnetworks.com/t5/ePMP-2000-and-1000/Random-temporary-loss-of-connectivity/m-p/84551#M13124

We have checked the SM's and there is nothing in their log files.  Also the AP has nothing in the log file.

Rolled back to 3.4.1 and the random connectivity issue seems to have stopped. It has now been 1 day 19 hours with no drops. We were having several per hour. So maybe something in the 3.5.1 firmware.

2 Likes

After changing the PSU and downgrade to 3.2.2, problem disappear on all APs.
(I can not say which of the two changes solved the cause of the problem)

All 4 APs except for ONE, but with a different behaviour:

It disconnect alla SMs one time in 10 days and in the log I see these strange
TSF Alignment :0
TSF Alignment :1
What is?

1 Like

Hi guys, same issue here on some AP (ePMP1000 GPS Synched)  upgraded to 3.5.1.

We are planning to Downgrade, you can confirm us that Downgrade to 3.4.1 solve the issue?

Thank you.

Regards

Alessandro Bertacco

Over the weekend I upgraded 1 POP (3x GPS Sync 5Ghz 1000 APs) and customer radios (about 100 total across those 3 APs)  to 3.5 (was running 3.4.1) and within a few hours a customer called saying their Internet was not working.  I could not reach their radio but the AP showed the radio as in session.  Had the customer power cycle the radio and all was good.  A few hours later another customer called, same problem. Had them reboot and problem solved.  A few hours later yet another called, appeared to be the same problem but the connection started working on its own while they were on the phone with us.  Then , several hours later the 2nd person that had called earlier called again, same problem again.

We had seen this problem in the past, pre-3.4.1, looks like it was brought back in 3.5 

Was thinking I would move up to 3.5.1 but looking at posts here it looks like whatever bug 3.5 re-introduced 3.5.1 didn't un-introduce it. We have been sending out new radios with 3.5 on them for several months now (all APs were still 3.4.1)  and haven't noticed this problem before so I'm guessing that upgrading the AP's to 3.5 is the problem ?  Going to roll the AP's back to 3.4.1 and leave the customer radios at 3.5  and see if it resolves the problem.

Hey brubble, did rolling back the AP to 3.4.1 resolve your issues?

Yes,it looks like it did.  I thought maybe the people were just power cycling their radios now instead of calling us but when I look at the session time on the AP's they show almost ever single radio has been in session for 7 days 2+ hours now (so 95% of them have been in session since the AP's came back up after I rolled them back and the customers are not just power cycling the radios themselves).

So , AP's on 3.4.1  and all CPE's on 3.5 is working so far.

I wanted to let anyone know that's following this thread that Cambium engineers have had me try a number of things to the handful of afflicted AP's, along with logging into them directly and getting more information. At the moment they recommend rolling back to 3.4.1 if you're experiencing this issue. They're working on a fix. I'll update this thread if there's any news.

5 Likes

Just coming from :[THIS LINK]

Chiming in that we're seeing this too.

Cambium is working on a fix and we're going to do more testing today.

1 Like

Also having this issue

So, at long last they've identified the issue and it's going to be included in the 3.5.2 release comming soon.

I hope it's coming soon, getting super frustrating this issue. Getting calls all the time and having to de-register SM's is annoying. 

I would upload a copy of the firmware-3.5.2-RC18.tar.gz beta image that fixes this problem, but I'm having a lot of issues with the forum's uploading security checks. Perhaps Fedor can upload a copy to the regular beta location for people to try?

I've been loading the R15 and R17 when it happens on the SM and the issue still happens, do you know if it has to go on the AP too to fix this problem or is it fixed in R18?