All SMs disconnecting from ePMP3000 on 4.5.4

I am seeing this in our logs on a 3000 AP running 4.5.4:

Sep 16 05:19:05 Bear Tooth 3000 kernel: [1688212.474159] SM[00:04:56:2b:2f:36] aid=14 disassociated. Reason: 48 (COMMUNICATION LOST)

When it happens, all SMs are disconnected from the AP and then reconnect. The frequency is completely clean. All SMs are also running 4.5.4.

The advice will be to upgrade to the 4.5.5 which has been released for a couple weeks, and see if that resolves it. There are lots of fixes between 4.5.4, and 4.5.5 which have been made.

Chad,
More than likely you are witnessing a phy reset which is a work around we put in instead of a full AP reboot. This has been an intermittent problem plaguing some customers for a while. It has been a tough problem to address but finally we have some light at the end of the tunnel. The team is going to work with some customers to validate the solution and we will then push out to additional customers. Could you elaborate how often you see this? Thx Sakid

Im seeing the same issue on 2 different ap. Both are on 4.5.5 . Hopefully this gets fixed soon.

I replied direct but did not repost here… On one AP, we have seen this 28 times in 28 days in our AP logs. On another AP, it happens less frequently. I have upgraded both to 4.5.5 but it sounds like that may not fix it.

I had disconnects happening all the time. However it is stable now after one critical setting change. The primary reason for disconnects that I’ve seen is SNR issues with the up links from the SM to the AP. This is in a area where the AP shows great down links. I believe the problem is an error in the automatic power control logic where the AP tells the SM to manage power to hit a target value. The AP appears to lower the SM power too much. So try changing the Client target level to -50 from the -60 default. After doing this my connections have stabilized.
Note - this problem has dogged my network since the initial 3000 release.
I have 3 sectors on 4.4.3 and one where I’m testing 4.5.5.
All of these are now relatively stable.

I mentioned this in other threads - with a 2x2 radio, a -60 might be comprised of chains which are -63 plus -63 if perfectly balanced. Or more typically something like -62 plus a -66. But the point is, a -60 will usually be comprised of both chains being pretty good.

In 4x4, that same -60 can be comprised of (if perfectly balanced) -66 plus -66 plus -66 plus -66, if all perfectly balanced. It’s also possible that your 4x4 -60 client, the chains might be something like -65 -71 -64 -74.

So, the per-chain signal levels of an aggregate -60 is not the same in 2x2 vs 4x4 - they will be on average -3 db worse, and yet add up to the same -60 aggregate. A target of -60 as an aggregate signal might fine on a 2x2, but in 4x4, -60 is not be enough.

AND, I’ve asked repeatedly if we can somehow see and graph the receive signals on the AP - from the AP point of view. I’m told that’s not a feature, that it’s not needed, and it’s not necessary.

Respectfully Cambium, I totally disagree.

In my opinion, it’s absolutely imperative to be able to see the per-chain signals on both the SM side, and on the AP side. I’ll bet you dollars-to-donuts, if you could have seen your clients per-chain-signals, I’ll bet that your most frequently disconnecting clients, probably had some of the chains in the 70’s — and had you been able to see that, you would have had a solution years ago.

1 Like

We noticed the same problem with two different APs on two different sites. We open a ticket with Cambium Support and we are waiting for some answer.

Last time we noticed this problem, the proposed solution was to upgrade from the 4.5.0 to the 4.5.5, but the problem appears again after 15 days with all apparatus (APs and SMs) with the same firmawe version.

Hope to receive some news.

Chad,
Could you possibly share the tech support file from the APs that are dropping the SMs. We need to verify something and hopefully shoot you over something we are working on.

Thx
Sakid

I did send them to support (from one of the APs). Do you need it again? What address would you like it sent to?

sakid.ahmed@Cambiumnetworks.com

If you send it today, hopefully I can turn around a possible resolution back today. Thx

Please send me a tech support dump at sakid.ahmed@cambiumnetworks.com
We want to verify if its the same issue. While you are at it, can you tell me the ticket number that you submitted to support.

Dear Sakid,

thank you for your availability. I just send to you the tech support files.

Please let me know if you need any other information to help to solve our issue.

Kind regards,

Giorgio

Hello everyone, everything good?
I have the same problem in Epmp 3000 in version 4.5.6 any news? I also noticed that you don’t have the options: Compatibility and Mimo mode to enable or disable. I’m from Brazil

I am seeing these logs on EPMP 3000 running firmare version 4.6.1:

Oct 20 13:51:18 ePMP19 - nuevo kernel: [497565.213499] SM[00:04:56:c4:b8:5e] aid=36 disassociated. Reason: 48 (COMMUNICATION LOST)
Oct 20 13:51:15 ePMP19 - nuevo kernel: [497561.578468] SM[00:04:56:c5:f1:30] aid=90 disassociated. Reason: 48 (COMMUNICATION LOST)
Oct 20 13:51:03 ePMP19 - nuevo kernel: [497549.427474] SM[00:04:56:e5:02:a7] aid=9 associated with AP
Oct 20 13:51:01 ePMP19 - nuevo kernel: [497547.718763] SM[00:04:56:f3:fc:e5] aid=74 disassociated. Reason: 48 (COMMUNICATION LOST)
Oct 20 13:51:01 ePMP19 - nuevo kernel: [497547.353791] SM[58:c1:7a:4b:66:8d] aid=63 disassociated. Reason: 48 (COMMUNICATION LOST)

the equipment is an epmp 3000 and the drawback is that all SMs unlink for about 3 or 5 seconds and then re-link to the AP.

Attached capture of the result of the spectrum analysis of the AP.

The firmware version of both the AP and the SMs is 4.6.1.