Problem SM massive disconnection from AP

Hi all,

in a high interference site, we have an ePMP1000 Gps Sync AP using a 120° sector antenna that lately is giving us some weird problems.

Especially in the evening, we're experiencing a massive disconnection of all SM connected (about 7-8 SM clients).

We've already tried to find some better frequency to use but no way... the problem persist.

We've this problem especially in the evening and at night (every 30-40 minutes or so), during the day we've this problem about every 2-3 hours...

Someone else experienced a problem like this?

Sorry for my bad english! :)

Best regards,

Paolo

Hi Paolo,

-Is it a new installation or existing one?

-Was that worked fine before?

-May I know the software version running on the AP and SMs?

-Please run the Spectrum analyser at AP and SMs in question and share the result (Screenshots) with us.

-May I know the weather condition at the time of issue?

-May I know the farthest SM distance from AP?

-Is there any other AP on the same tower running with same frequency band?

Please send an email with all the above information to our support email address support@cambiumnetworks.com . So that, we can have a further discussion & remote session to help you to resolve this issue.

Regards,

Vijay Gnanamurtthi

I've reported the same - a bunch of SM's disconnect and reconnect at the same time on one of our AP's.  This is an AP that is close to a competitors tower, so we suspect lots of noise, although the signals and SNR's are pretty good and a spectrum from the AP does''t look any worse than the other Sectors.  And these clients work well and test well while they are connected.  When they disconnect, they re-registere right away and continue to work well again.  In our case, there are other Sectors on the tower, and only this one (North) has this issue, the East and South AP's don't disconnect like this.

SO, it might be simply intermitent overwhelming noise, but it's every several hours or couple days, so it's difficult to pin down if that's what it is or not.  All Sectors/SM's are running 3.1 here.

 
And in our case, occasionally when they dis-connect and re-connect, some client(s) may be shown without an IP and without a Device Name displayed on the AP.  Yes, as slown below in the Monitor -> Performance screen, that client still works fine and transfers data.  If I manually 'deregister' this client right now, it'll reconnect and it'll have an IP and Name displayed.  Again, this is still 3.1 - I might upgrade everyone to 3.2 after the weekend to see if that changes the sitaution.

RDH-N_Performance_Dec11_16.png

ninedd I think we've exactly the same problem!

We have tried upgrade all SMs to the same firmware of the AP, but it doesn't really change something.

Now we are all working with the latest 3.2 firmware.

Yes of course near the tower where's our AP, there are a couple of other competitors APs, using different brand devices (Ubiquiti, Mikrotik, .....). On our tower we've 2 more sectors APs at about 2 meters each from the AP that's suffering of this problem but they are not ePMP, they're old Ubnt Rocket TI and they works really bad since there are a lot of radio interferences in that place. Obviously we're using different frequency! :)

We've tried to find the best frequency using Spectrum Analyzer mode over the AP.

I can't do Spectrum Analyzer from a SM in a short time, since they're pretty far from us and we've to reach them and we cannot do remotely the Spectrum Analyzer.

I can do it from the AP and I'll send you results.

Anyway client SMs works pretty fine (when they are connected to the AP!!!!), wireless link test shows about 60/25 Mbps for example. It looks like MCS in upload is pretty bad... MCS 1 or 2, not more. Instead in download is pretty good (MCS 10 o MCS 11).

For the record - I don't doubt that it is interference as the core cause.  I'm just confused why the SM's seem to work OK (so the interference isn't so overwhelming that they are non-functional at all) and I've been watching when multiple people are transferring data and everything seems OK - and then a chunk of them will all dis- re-connect at once. And when they re-connect seconds later, those clients will resume transferring data at their QOS right away again, no apparent problems.

So - it probably is interference (given the environment) but it also feels like the way the ePMP is handling the interference.  Generally speaking, it's awesome how it handles and performs in interference - I'm frankly blown away how well it performs in places where other gear falls on it's face.  But, this place seems strange for ePMP.  It feels like it should be able to react better, since it seems to be going along fine, and then to suddently drop and reconnect a bunch of people all at once... and then to continue happily along for another couple hours or a couple days again.

I agree with you ninedd.

The problem is that we're having massive SMs disconnection every 30-60 minutes, we can't have a stable situation for more than 2-3 hours!!!

During the evening it get worst (almost 30-40 minutes!), during the day maybe SMs can go fine without disconnections for about 4-6 hours too.

I hope Cambium team will get a solution for this problem. It's really frustating.

Best regards,

Paolo

Hi,

Could you please also attach System Log from AP and SM after disconnect has happened.

Thank you.


@pftech wrote:

I agree with you ninedd.

The problem is that we're having massive SMs disconnection every 30-60 minutes, we can't have a stable situation for more than 2-3 hours!!!

During the evening it get worst (almost 30-40 minutes!), during the day maybe SMs can go fine without disconnections for about 4-6 hours too.

I hope Cambium team will get a solution for this problem. It's really frustating.

Best regards,

Paolo


have you tried using the EPMP2000 with beam steering antenna?  if its truly noise, that may have a significant impact on your situation.


@Chris_Bay wrote:
have you tried using the EPMP2000 with beam steering antenna?  if its truly noise, that may have a significant impact on your situation.

I'd LOVE to...   This is a 2.4Ghz site - you got a Beamsteering Antenna for the 2.4Ghz ePMP????   ;)  


@ninedd wrote:

@Chris_Bay wrote:
have you tried using the EPMP2000 with beam steering antenna?  if its truly noise, that may have a significant impact on your situation.

I'd LOVE to...   This is a 2.4Ghz site - you got a Beamsteering Antenna for the 2.4Ghz ePMP????   ;)  


I can only wish, and of course I'd share them with ya!  ;)

speaking of strange 2.4 disconects, I've  found some really screw ball interference that was bothering one of my subscribers just a few days ago, very randomly too.    few mintues a day at random they can't keep a connection to save thier tail. the rest of the day, MCS 12/13 all day.   no wifi channels show up in 5, 10 or 20 mhz channel widths when i checked in wifi mode and in analizer mode, i found a signal pattern that i think would look like a 20 mhz wide DSSS type signal.   never found the source because it is saturates a intrated radio for nearly 1000' (it maxes out the Spectrum anasyer tool... peaking around -10 to -15 ish all around this subs home.  the house is off in the woods and i couldn't find anything around that looked resposible for that kind of insaine power output only 4 wifi routers had shown up in the wifi mode check, none louder than 80 db and all of them were our customers on channel 1.  Wonder if some kind of new toy out there is really being bothersome and causing you the headache i found a few days ago?  but what ever i was seeing peaking, i doubt it was within legal power limits.   

oh and please forgive me taking a picture of my screen...


@Chris_Bay wrote:

have you tried using the EPMP2000 with beam steering antenna?  if its truly noise, that may have a significant impact on your situation.

That would be probably an interesting idea.... we haven't got an ePMP 2000 yet, but I think in about one month we'll can try it.

BTW we're in 5 Ghz ETSI band (Italy), but as I can see the problem persists even on 2,4 Ghz!!

Chris - that kinda looks like what we see at one customer's house with their leaky microwave oven.  They cook lunch and their wifi goes off the air.  I'm telling them that a 1000 watt -20 signal leaking out of their microwave... that might not be all that healthy for the person standing right there watching the timer count down.

I've upgraded this AP and all the SM's to 3.2 tonight, they previously were all 3.1


So, we will see if that has any effect.  As mentioend before - I'm very, very impressed how well the Cambium ePMP work in the face of noise, and this is a 2.4 Ghz Access Point which is in a fairly noisy environment near our other non-Cambium (non-Synced) gear as well as other competative gear.  The fact that it even works is amazing.

However, it does seem that it periodically does this thing where it de-registers and re-registered 80% of the SM's all at once, and then they are OK for the next 2 hour or 2 days.  So - we wil see if 3.2 has any effect on that behavior. 

ninedd,  Did the upgrade to 3.2 have any effect? 

For Paolo, ninedd or anyone else reading, here are my experiences.  I've found the ePMP gear deals well with interference from wifi routers and other typical sources.  I did expereince this same issue, however, and it was due to self-interferene in a fequency re-use arrangement.  This may not be applicable to Paolo or ninedd's situation, but it may help others reading this.

I had a 2.4 GHz 4 sector arrangement using frequency re-use and GPS sync. As soon as we hooked up a few clients close to the tower (under 1/4 mile), we started seeing session drops. The drops would increase as the clients became more active in the evening, up to serveral drops per hour during busy times.  Moving from a 20 Mhz channel re-use arrangement to a 10 Mhz non-overlapping channel arrangment eliminated the issue completely.  Now links are typically up for days without a drop. 

The issue was that the clients close to the tower were causing interference with the sector on the opposite side of the tower.  What I learned was that frequency re-use requires careful planning.  You have to use the right sector antennas, they have to be downtilted the right amount and the client antenna's have to be chosen and aimed carefully.  Particularly, if you have clients near and far from the tower, a standard sector antenna, with typical vertical beamwidth of 7 to 11 degrees, may not work in a frequency re-use environment.  The "effective" front to back ratio of opposing sector antennae, when considering clients close to the tower, may not be good enough for frequency re-use.

Again, I found the ePMP generally handles noisy environements well.  This issue was particular to a frequency re-use deployment.

1 Like

Sector tilt is critical I. Dealing with reuse interference. The default target power of -60 is also to hot for this with sharp up angles subs. (Generally just the close ones) if the floor allowed, reduce to 65. Maintaining that 28+ db between cpes gets critical. As well as the cpe hearing the APs. Without that gap, I’ve observed cpes getting confused and showing rssi power bouncing between the beconed power of both APs. Hitting that 28 db threshold stops it entirely, or not using local reuse. Ive never seen the bulk drop in subs related to that re-use issue you described, but the shoe fits. A few entirely drowned out frames in a roe by noise would do the job. Wonder if reduced target RSSI would make a sudden improvement, or just reduce one and see if the problem doesn’t track the opposite AP.

1 Like

Hi.  No, I don't think 3.2 made any change vs 3.1 on that 'North' sector.  We don't have anyone too terribly close to these AP's.

North = 6.6 KM to 16.2 KM
South = 1.8 KM to 20.8 KM

East = 2.7 KM to 16 KM

However, I'll try balancing the power a bit - we do have the South Sector's target set to -59 so perhaps one of the 1.8 or 2.1 KM clients are too loud and bouncing the guys on North.  I think it's competitive interference, since there is a tower north of our tower, so our North Sector is pointed towards it.  I dont' really now what's on it that tower - UBNT something, but those guys often put one extra sectors pointed at us, I suspect just to mess things up for us.