How are Pause Frames Handled?

Products ePMP
1

How does the ePMP line handle pause frames?  In a wireless broadband network there is a lot of half-duplex operation, and when you connect that to a switch which is fed from a full-duplex high-rate device (e.g. a 700meg fiber connection) switch buffer overflow can occurr causing massive network slow down.

Pause frames fix this by allowing the radios to tell the sending device to momentarily stop sending traffic so that the switch buffer doesn't overflow.

In some other vendor's outdoor wireless equipment they do not support pause frames on the CPE/AP interfaces.

Does the ePMP line support pause frames on the wireless/Ethernet interfaces?

2

Anybody?

3

they send and receive  pause frames, my netonix switch with a very busy panel has a few hundred marked TX and RX.  I would imagine if your using a switch with a small buffer, you'll see a good increase from the switch sending to the radio

the epmp radios have a pretty good chunk of memory and has much better frame control, so getting flow control to fire from these takes a significant higher load than a lot of other 802 based gear. 

if your seeing a fair amount of pause frames, upgrades are due.  you shouldn't see pause frames in a network that isn't bottle necking

2 Likes
4

if your seeing a fair amount of pause frames, upgrades are due.  you shouldn't see pause frames in a network that isn't bottle necking

------

I agree, but how do you propose doing that when you're feeding a tower with a 300meg full-duplex link that then goes down to subscribers that are 10 and 20 meg subs on half duplex wireless links?

5

half duplex and time divided duplex links are very diffrent animals.     if your maxing out your radio framing you'll see pause frames happen on the ethernet side.  if you are not maxing out your radio framing on the Radios you will not see pause frames.     pause frames will only be sent to a switch from a radio when the radio has  almost ran out of buffer to accept more incoming data. same from a switch. the pause frames will only fire with the buffer cannot accept anymore frames. 

lets assume you've got 6 PTMP radios attached to a switch and your 300 meg backhaul.    when your get a pause frame from  a PTMP radio on one of the ports its attached to, your flooding your PTMP radio link.     add another AP to relive load and problem will reduce.  

if your backhaul is getting pause frames from the switch, but the switch isn't getting them from your PTMP radios, your switch can't handle the work, get a switch that can.    

toughswitches are notorious for having that problem.  they have very small buffers and do a very poor job mixing gig inbound dataflows to fast ethernet outbound flows.    another way to see if its your buffer issue on your switch, during peak, check the data rate coming into your backhaul and use a single session TCP session for a file transport. you should be able to nearly fill your backhauls transport. if you cannot but you can with multiple streams, you've got a reduction in performance in your switching system some place.       

netonix is probably the cheepest switch out there that will give you a reasonable line rate without reducing that flow.     we've got 60 to 80 subs on some of our 5ghz panels attached to both juniper and netonix swithes, neither count many pause frames.       our old netgear and the few toughswitches we had, would flood out easy and we just changed them with junipers for our busy sites and recently netonix, problem solved. 

3 Likes
6

I agree, the Netonix switches are amazing for the price ( http://www.netonix.com ).

Let me throw a scenario at you though:

You have a 500meg Internet pipe, which goes to a switch.  The switch then feeds off to 5 or 6 access points.  Those access points have 100meg Ethernet ports and then funnel the traffic down even further to clients which have 5, 10, 20megabit connections  on half duplx wireless connections.

Even if those clients have great MCS rates, the packets are still going to come in full line rate (500meg) from the Internet, hit the 100meg Ethernet port, then hit the throttle when it hits the air interface (~ 50megabits running a 20meg channel) then get throttled by the traffic shaping on the CPE.

When this happens TCP doesn't back off gracefully and so the switch buffer the AP is plugged into begins to backup, causing the switch to issue a pause frame to the backhaul feeding the switch.

Are you saying you see limited to no pause frames when using Netonix with your ePMP network?  Do the ePMP APs have Gigabit Ethernet ports?

7

Again, half duplex and time decided duplex are VERY DIFFRENT to a data stream. Latency will graceful back down a TCP session and is introduced via a tdd. A good tdd system will cause balance in TCP. If you have a billion megabit connection going to a 1 meg connection, and you’re not overloading buffers, pause frames will not happen. Now when you over demand your buffers, they will begin to show. With Netflix, downs ect. Ect. Ect. Until you’ve over run the capacity of your radios buffer, you will not see a pause frame. Starting more Netflix sessions than your radio specifically can handle, might show a pause frame, however it likely will get kicked in L3 session handlers and not show. The pause frame will ONLY show a an Ethernet buffer has been maxed. And again pause frames show a need to upgrade. The bandwidth throttles you talk about happen in the wireless Mac and cause intentionally slow the stream (latency) and causes TCP to respond. Poor tdd radios literally spot the streams until thier turn go tx rather than control the rate inwitch the cache fills in the baseststion. The epmp radios will not throw a pause frame until they can no longer buffer incoming packets. They can handle much greater loads than any other 802.11 chip I’ve watch operate as a ptmp tdd radio.

A good tdd radio will trip TCP sessions controllers long before pause frames.

8

Yes epmp sync aps have gig ports, they can dependably deliver 180 ish meg at mcs15 in 40 mhz channels

9

Thanks Chris,

So what you're saying is if lots of pause frames are being seen it's an issue with the traffic overrunning the buffer in the radio because the radio is not properly handeling TCP flows to the end CPEs, yes?

10

It's not that the radio isn't handling it properly, its the radio is overwhelmed for one reason or another. The pause frames happen when a peering device is bottle necked to tell the neighboring device to hold on a sec while it catches up

it can also be the switch. if the switch is receiving pause frames, the radio is clogged. if the switch is sending pause frames, the switch is clogged.   

11

I'm not contradicting you, I'm trying to learn here.

Wouldn't stuffing a 500megabit Internet feed down to a 20megabit wireless subscriber constitute "overwhelmed"?

12

it wouldn't overwhelm it,     TCP will quickly realize and get the flow set to 20 mbps so its utilized, but not overwhelmed.      when a 2nd data stream is introduced to the flow, TCP will get pushbacks to its layer session controls and cause the streams to fall to 10 mbps each, this will continue to split down until the packet sizes begin to shrink to fit in the stream,    your available bandwidth per stream will fall as expected to that individual customer but the link doesn't become overwhelmed until it is unable to collect frames in the radios buffer and has to start sending pauses or dumping packets.      each radio will handle a different mix of steams and packet types before becoming overwhelmed and with good QoS controls and a solid MAC can keep on overwhelmed link kinda usable.  

the pause frames get sent when the ETHERNET side of the radio becomes overwhelmed and can't cache anymore in the radio. 

for example, a EPMP sync ap with MCS 15 to 50 different subs. assume everything is perfect, no noise, no NACKs ect. all customers are set to 20 meg service.  

the  radio can deliver 170 ish megs downlink to the subscribers.  lets also assume that this is the only active AP on a tower being fed with a 300 meg backhaul.  

all 50 subs begin to start a single session file download all at the same time.    you've got a situation that your radio cannot supply 50 people with 20 megs of service each.   the radio would be at "capacity" but not overwhelmed.    the customers will 3.4 mbps on average each. which is significantly less than the assigned rates.     in this situation you will likely not see any pause frames fire, or very few.   the reason being the radios TDD mechanism will cause enough delay in the layer 3 session controls that the sending side won't try and send 300 meg based on the delay in the handshack process.    it will start some place in the middle, the next packet group will also have a reasonable delay on the path and will further reduce the data rate being sent along the stream. once the TCP session controls have a good idea of an acceptable information rate, it will attempted to continue to provide that rate.    since all of these data streams experience the same latency being implemented on the first few packets, they all will see the need to slow down and the regulate the data flows.  most of the speed adjustments and delays are done within the AP and the first few very high data rate packets that came in, are slowed down in the APs buffers, which cause the delay going back to the original source of that data download.    the reason the link isn't actually overwhelmed is because of the packet scheduler.   yes this situation is clearly over subscribed, but your radio will only have 100 sessions active, not exceeding the packet rate of the radio, likely not over flowing the buffers thanks the internal delay in the data rate.      this is part of the magic of using a TDD radio with a solid MAC layer that includes a good packet scheduler.  

the same situation on a wifi MAC or polling based MAC wouldn't work out so well.  since the MAC isn't as controlled and defined delays that are dictated by the AP the ACKs that go all the way back to the source would be less aligned and consist which results in unstable latency, which in turn causes information rates to fluctuate and in turn causes to fast of streams to hit the AP, resulting in Pause frames being sent. reducing the overall actual usable throughput being sent down the link.     now add in a few of the other eliminates that can happen in a polling environment, like a NACK out of turn, and you've got a possible snow ball effect that will cause pause frames to be kicked.       when the polling radio has free air time, this likely will be fine, but with full air time, comes chaos.   (that why some systems turn down hill all at once rather than slowly)   

smooth constant latency that's introduced with a pre-mapped out schedule like wimax, LTE, epmp, cambium mac will cause more effective use the TCP session controls than a poling system which produces more wild shifts in latency, which intern causes a lot of fluctuations in the data flows that hit the AP before hitting the air interface... 

the radios become overwhelmed with they're buffers spill over and discard and or pause packets.  

overwhelmed in a wifi situation, happens significantly sooner than overwhelmed in a good scheduled TDD MAC 

2 Likes
13

You could have the opposite problem: licensed radios with packet buffers that are too small to even utilize the full capacity of the RF link. *cough* Trango *cough*

You also have to be really careful with flow control / pause frames. This can severely degrade VoIP performance. I enabled flow control on said Trango radios and voice quality beacme so horrendous because of the constant pauses and it had to be disabled. Priority traffic like voice is fine without it because the radios get that traffic in and out of the ethernet buffer first, but at the expense of best-effort traffic obviously because the buffer is being overrun.

The only solution to a problem such as this is to upgrade the hardware. :(

14

Ouch, that stinks… We had ran into a similar problem with some old netgear switches we used, flow control murdered voice service, without the flow control it was decent but over all best effort service turned jumpy, but still decent. Does that backhaul support multiple data queues ? Or a priority vlan ? It may be enough of a priority to correct the pause from haukting the frame to much, and a high jitter buffer in a ata might fix it to, as long as the voice frames stay in order

1 Like
15

They say they're going to have a new software load that will improve the flow control between the internal switch and modem. We'll see.