SM disassociated from AP (ePMP F190)

Hi everyone,
I was wondering if someone could help us with one of our links.
It is an ePTP link between our sites. We had various reports of 30 minutes long disconnections - this was caused by DFS. Once we moved the channel, we had another issue:
Oct 1 14:22:17 alexandreinglis kernel: [2643741.190000] SM disassociated from AP[00:04:56:64:54:b7] F=5800 11naht20. Reason: 51 (AP INITIATED DISCONNECT)
Oct 1 14:22:34 alexandreinglis snmpd[2546]: DFS status: N/A
Oct 1 14:23:38 alexandreinglis snmpd[2546]: DFS status: Channel Availability Check
Oct 1 14:24:33 alexandreinglis kernel: [2643878.160000] SM associated with AP[00:04:56:64:54:b7]
Oct 1 14:24:33 alexandreinglis snmpd[2546]: DFS status: In-Service Monitoring
Oct 3 19:11:10 alexandreinglis kernel: [2833874.730000] SM disassociated from AP[00:04:56:64:54:b7] F=5800 11naht20. Reason: 51 (AP INITIATED DISCONNECT)

We tried 3.5.6 and 4.4.3 softwares and issue still persists. However the link had really long wireless session time (21 days) after updating to 4.4.3 , but the problem appeared again.
No known interference on the AP after performing Spectrum Analysis.
RSSI : -64dBm
SNR: 33dB

Could somebody help us please?
Thank you
David
Lothian Broadband

Hello,

Could you please attach logs from AP and SM please?
It’s important to see disconnect reasons on both sides of the link.

Thank you.

Hi, thank you for your reply:
AP backlog:
Dec 4 04:40:27 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 4 04:40:27 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 4 04:42:24 alexandreinglis kernel: [2522167.060000] SM[00:04:56:60:db:61] aid=1 associated with AP
Dec 4 18:22:04 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 4 18:22:16 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 4 18:22:16 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 5 03:29:37 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 5 03:29:39 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 5 03:29:39 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 5 09:24:03 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 5 09:24:04 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 5 09:24:04 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 5 09:46:10 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 5 09:53:31 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 5 09:54:34 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 5 09:54:34 alexandreinglis DEVICE-AGENT[2597]: [src/epmp.c:183 json_config_show] request failed, ret = 2 (Param not found)
Dec 5 14:52:47 alexandreinglis kernel: [2645190.030000] SM[00:04:56:60:db:61] aid=1 disassociated. Reason: COMMUNICATION LOST
Dec 5 14:55:04 alexandreinglis kernel: [2645327.010000] SM[00:04:56:60:db:61] aid=1 associated with AP
Dec 7 19:41:40 alexandreinglis kernel: [2835322.550000] SM[00:04:56:60:db:61] aid=1 disassociated. Reason: COMMUNICATION LOST
Dec 7 19:43:57 alexandreinglis kernel: [2835459.540000] SM[00:04:56:60:db:61] aid=1 associated with AP

SM backlog :
Sep 23 21:08:20 alexandreinglis kernel: [1976905.300000] SM associated with AP[00:04:56:64:54:b7]
Sep 23 21:08:34 alexandreinglis snmpd[2546]: DFS status: In-Service Monitoring
Sep 30 04:09:38 alexandreinglis snmpd[2546]: DFS status: N/A
Sep 30 04:10:49 alexandreinglis snmpd[2546]: DFS status: Channel Availability Check
Sep 30 04:11:52 alexandreinglis kernel: [2520717.580000] SM associated with AP[00:04:56:64:54:b7]
Sep 30 04:11:53 alexandreinglis snmpd[2546]: DFS status: In-Service Monitoring
Oct 1 14:22:17 alexandreinglis kernel: [2643741.190000] SM disassociated from AP[00:04:56:64:54:b7] F=5800 11naht20. Reason: 51 (AP INITIATED DISCONNECT)
Oct 1 14:22:34 alexandreinglis snmpd[2546]: DFS status: N/A
Oct 1 14:23:38 alexandreinglis snmpd[2546]: DFS status: Channel Availability Check
Oct 1 14:24:33 alexandreinglis kernel: [2643878.160000] SM associated with AP[00:04:56:64:54:b7]
Oct 1 14:24:33 alexandreinglis snmpd[2546]: DFS status: In-Service Monitoring
Oct 3 19:11:10 alexandreinglis kernel: [2833874.730000] SM disassociated from AP[00:04:56:64:54:b7] F=5800 11naht20. Reason: 51 (AP INITIATED DISCONNECT)
Oct 3 19:11:23 alexandreinglis snmpd[2546]: DFS status: N/A
Oct 3 19:12:33 alexandreinglis snmpd[2546]: DFS status: Channel Availability Check

System time in SM is wrong. Both are on 4.4.3 software.
Thank you

Hi,

I would propose you to try 4.6-RC35 firmware that is available in Beta now.
We have done multiple fixes related to disconnects in the latest firmware versions.
From the logs, I can see AP stops hearing SMs. It can happen because of multiple reasons, including strong interference in Uplink.

Thank you.

Hi Fedor,
OK I will try Beta version.
I have done Spectrum Analysis on the SM too and did not find any interference. eDetect tool also did not identify anything.
I moved the channel twice to avoid interference and DFS events in the past.
Thank you

Hello everyone,

I do not want to create duplicate of this threat so I will continue here as we are having absolutely same issue with our other F200s PtP radios too -
again the ‘‘Communication Error 52/Connection lost’’ errors in the log - identical issue to previous one.

It drops once a day - MCS15/15 and RSSI -65/65dBm (No interference detected even in Spectrum Analysis).
FW is 4.6.1 (Recommended).

The previous issue was never resolved and we had to put the link down and find another route.

Can someone help please?
Thank you very much
David Riedl

Just because you cant see the interference doesnt mean its not there, especially with e1k/e2k/f200 gear. The spectrum scanner is not fast enough to pickup fast interference. The only way to prove this is to hook up a real spectrum analyzer to a force 110 dish and just let it gather data. You would be amazed after looking at a long duration spectrum scan to see what is out there that the epmp gear just cant see!

Thank you Douglas.
The thing is that this would need to be interference affecting every channel across 5GHz spectrum (as long this PtP drops on any quiet channel in Band B/Band C) and this is the only Cambium radio affected. Other radios at both sites stays connected at time of drop outs.

However thank you for your hint to install Spectrum Analyzer. We will try to do that or we will install different radios of different brand to save time as this affects many customers.

Regards
DavidR

If your spectrum is quiet, then one of two things are possible before we determine a hardware issue. 1) GPS sync loss or 2) DFS channel

If on a DFS channel, it only takes a rf power spike a couple milli-seconds long and the radio will stop transmitting. Here in Canada, we have 5190 to 5240 and 5745 to 5825 (40mhz channels) for non-DFS. So make sure you are not using DFS channels for your region.

GPS sync is both a blessing and can be a curse. The blessing is when you have to reuse channels and you place your masters on the same tower. The curse is that you need to ensure that all links are in master-slave/slave-master/master-slave and that you maintain that at all times especially with DFS channels.
Sync loss causes a timer to run and if sync is not reestablished by the time the timer runs out then they stop transmitting. Increasing the hold-off timer will give more time to re-sync but does not fix the underlying problem. We have found that relying on the internal GPS is not a good idea, yes Cambium has been working on its reliability and have made huge improvements but the costs of downtime may be worth a sync-injector and a syncbox jr from packetflux. The only sync radio that you can not provide sync over power is the e3kL.

Hi Douglas,

Thanks for the reply.

Radio is in non-DFS channel band (Band C in the UK is non-DFS). It is in ePTP Master-Slave mode. 3.6km link lenght. Radio link is under the 24hours surveillance - RSSI: -64dBm, MCS15/15, Signal to Noise Ratio: 33dB at all times. These numbers do not bounce around, they stay the same.

All ePMP radios in range have at least 10MHz gap between their channels as long this is the quiet edge of our network.

We have hold-off timer set for maximum on every ePMP radio in the case of GPS-sync failure.

As a last try, before we swap HW, I will try to raise Tx power to maximum EIRP limit of 36dBm to get better RSSI than -64dBm and SNR of 40dB and maybe ensure that wireless session won’t drop in the case of short-term interferences (farm cameras, security cameras, radars etc.)
Expected RSSI will be -58dBm which should be solid enough to avoid total drop out due to interference.

I will update you once maximum Tx power is set.

Thank you for your advice.
David R.

Swapping the equipment has resolved the issue. Thank you