Mikrotik showing 1000's of RX Pause frames on ports connected to ePMP2000 APs

ePMP 2000 APs running v3.2.1

Mikrotik CCR Switch with 10 other devices (ePMP , ubiquiti, Pi)

ePMP 2000 APs show 1000/FD

Mikrotik shows 1000/FD

This only effects  the 2 ePMP 2000 APs on the switch. The other 10 devices do not show a single RX pause. The two ePMP 1000's we replaced with the 2000's did not have a single RX pause.

Setting Mikrotik to tx/rx flow control auto/off/on has no noticeable effect.

The APs rarely pass more than 40Mbps and even when only passing a few 100kbps the RX pause counter continues to increase at 1000s per second.

Discovered this trying to hunt down the reason for erratic latency and packet loss on these two AP's.. Latency on both 2000s is all over the place from 12ms to250ms  and everything in between (this is between the customer and the AP ) and while we have not seen PL between the customer and the AP we see sporadic PL beyond the AP (it will run with just wildly variable latency for 10 minutes or half and hour or longer then suddenly 100% PL for 10 or 15 seconds).  Not sure if this is related to the RX pause .

I tried setting one of the ePMP 2000 APs to 100/FD just to see what effect it would have but when I did this I could no longer reach the AP via the ethernet side and it did not appear to be passing customer data however I was able to ping it...  Connected to it via RF interface and set ethernet back to Auto (it goes to 1000/FD when on auto) and everything was good then. Didn't want to cause another outage for customers so I stopped messing with. At this point I don't know if that was just a firmware glitch or something relevant to the problem.  

I don't know why there would ever be pause frames between two devices running at the same speed/duplex and especially when only moving a few 100kpbs to maybe 40Mbps on a 1000/FD ethernet link.  

Any ideas ?



Also, I could have sworn I posted this earlier this morning... I even when back and made an edit to my post when I realized I had misspelled something  but now not only does it not show up on the board but it does not show up in my post history. So if others are seeing my first post also, I apologize, I can not find it so I'm posting again.

I believe that is normal and proper use of the RX pause... you are going from 1Gbit to  what 70mbps on the client?

I have an ePMP 2000 connected to a netonix switch. it is pushing 40-60mbps and is getting 4k or so RX pauses per second. the buffer on the switch is handling it correctly so that it does not affect the Routers port.

The way you can test to see if the RX frames are affecting the ping times is just ping the AP. the RX pause affects the entire interface not just things behind it. even with my 2000's 4k/sec RX pause's out of 1000 pings from my core I get 0min Avg 1 and max of 5ms

"I believe that is normal and proper use of the RX pause..."

Then none of my other devices on this switch are running correctly ? Because nothing else is sending pause frames.

"you are going from 1Gbit to  what 70mbps on the client?"

 

On my old 900Mhz Canopy I'm going from 1Gbit to 256kbps on the client but that device is not sending pause frames.

Of the 12 devices on that switch only two are sending any pause frames at all and that's the two ePMP 2000 APs. 

"I have an ePMP 2000 connected to a netonix switch. it is pushing 40-60mbps and is getting 4k or so RX pauses per second. the buffer on the switch is handling it correctly so that it does not affect the Routers port."

I have APs on my system that are pushing 100+Mbps and even on switches that have been up for 8+months the most pause frames I see on any device (other than these two epmp 2000's)  is 1183. So in 8 months the most pause frames any other device has sent is 1183  while these two epmp 2000's send twice that many in one second and only handle a fraction of the traffic of other devices.

Right now one of the two epmp 2000's is only moving , on average, about 20kbps , no one is using it  it's just chatter and yet pause frames still increment at 100+ per second.

 

Just curious.  Has technical support or engineering been involved with this at all?

Seems like something they would want to jump on ASAP.

Paul

Funny I was just coming here to bring this up again and there was your post. 

Attached is a cap of one of our switches at a tower next to our NOC. I highlighted every port that has an ePMP plugged into it and as you can see that is also every device with Rx Pause. While the ePMP radios are currently running 3.4.1 (because every firmware version after that breaks something important for us) it has been this way since we started using ePMP 3+ years ago.  I have other towers with ePMP 2000 GPS Sync AP's on them and they have tons of Rx Pause also.

On every switch at every tower every ePMP device has rx pause while no other devices on any other switches have them.  I don't know that it is causing a problem but it is odd that only the ePMP's are doing this.

Below is what is plugged into each port, things with Rx Pause are bold (which happens to be every single ePMP even ones that are not in use !)

Port 1 -  another tik 

Port 2 - banana pi

Port 3 - Airfiber 5 

Port 8 - ePMP 1000 GPS PtP

Port 10 - Ubiquiti 5Ghz Rocket

Port 13 - ePMP 1000 GPS PtP

Port 15 - ePMP 1000 GPS AP

Port 16 -  Canopy 900Mhz AP

Port 19 - ePMP 1000 GPS PtP (not even linked to anything)

Port 20 - ePMP 1000 GPS AP (No links no customers on this AP)

Port 21 - ePMP 1000 GPS PtP

Port 22 - ePMP 1000 GPS AP

Port 23 - AirFiber 3x

Port 24 - ePMP 1000 GPS AP

Hello.

I´ve been checking this out since I started with sudden Ethernet stopping RX data... Same scenario... 5 of 8 Cambium products generating a lot of Rx pauses frames... I thinks this is slowing me down my Mikrotik...

Hi all,

There is another thread on that topic.
We have tested fix for that issue and it will be included into 3.5.5 FW soon.

Thank you.

1 Like

Thanks for the info Fedor