QoS inside PPP with bridge mode

Hello,

we have this setup:

PPPoE Server <--cable--> ePMP AP(bridge) <-- wireless--> ePMP STA(bridge) <--cable--> Router (PPPoE Client)
               |----------------------------------------------------PPP Tunnel----------------------------------------------------|

Does the ePMP/PMP recognize the DSCP or the CoS tags that pass encapsulated in the PPP tunnel?

I mean, if I change the DSCP header inside a PPP packet, would the ePMP/PMP with a proper QoS configuration apply the correct priority?

This is a crucial information that I couldn't find in any documentation.

Thanks.

Enrico

we use  ppp too the best way to deal with that is use vlan priority. Because the nature of ppp the traffic is encapsulated. 

1 Like

Hello Enrico,

The scenario you describe should work if you use TDD mode. The implementation of PPP takes into account the additional PPP header so QoS rules can be applied correctly.

Regards

Hello Luis,

please not the both of the radios are in bridge network configuration, the PPPoE session is established by a router behind the STA lan.

Does this case apply here?

Thanks.

Hello Enrico,

My mistake. If the PPPoE client is NOT the ePMP SM, then QoS rules cannot be applied to the PPP encapsulated traffic.

Regards

Luis et al.,

Is there any possibility that we might someday see this feature (applying QoS rules to a PPPoE tunnel bridged through SM) show up on ePMP platform?  My understanding is that this is possible with PMP450 (though maybe that's not accurate either? I intend to follow up with a post to that thread as well), and it would be great to see this feature show up on ePMP as well...it would be another really great incentive to Elevate existing networks to ePMP for sure!

Thanks,

-- Nathan

What you want is impossible with PPP due to the encapsulation methods used to make it work. If you want VoIP to have a higher priority, then it must be set at the PPPoE client prior to encapsulation. The ePMP-SM does not do deep packet inspection and this tends to break the PPPoE encapsulation (tried to do this and found our the hardway that it breaks).

Using radius to set UL/DLMIR and some QoS settings by VSA in both the ePMP-SM and the PPPoE server works surprisingly well to do the same/similar bandwidth control. VoIP has been a challange as PPPoE makes the network look like a single hop right to the edge, not a bad thing but you loose control of your trunk (backhaul) bandwith and placing traffic controls wont work as expected as most of the data is PPP and not individually definable to http, voip, snmp, gre, etc. Using the ePMP-SM as the PPPoE client, the MIR/QoS settings work so well that we do not even set the PPPoE server VSAs in ePMP radius profiles anymore. Using any other PPPoE client, It does not matter what the PPPoE client is, if you can not set QoS in the device, then you have no control.

I am kind of curious as to why you are giving clients bridged connections? That is a L2 connection directly into your network. My position is NAT or Routed, leave bridged for PTP links. With Routed you can set priority Vlans for Voip and traffic engineering on your backhaul network provides the ability to control the network and provide high priority data the correct priority. And yes we use PPPoE on our data connections for the AAA accounting for data caps. (yes we set data caps, but are reasonable about it. Basic packages are for basic needs not for cheaply hosting torrent servers, why should the few heavy users be subsidized by the many average and light users?) Voip on a second Vlan with its own PPPoE allows us to give that Vlan a higher priority than the data vlan. Still not what you want but it does work.

* This was edited due to my not being fully on topic nor answering the question the first time around.

1 Like

Well, it is indeed possible if using DPI correctly, PMP450 series is working just right in this configuration.

http://community.cambiumnetworks.com/t5/PMP-450/VoIP-QoS-issue-using-PPPoE-from-client-router/m-p/59986#M3417

Anyway, we are using bridged connections simply because we like to mainstream all of our CPEs and having the PPP interface directly on the router permits us not to double-nat our customers.

This problem wouldn't exists if we were using ePMP stations as customer's routers.

1 Like

Use the R190 and R200 routers in bridge mode and you wont double NAT if the rooftop radio is in router mode. 

Netgear and many other consumer routers support AP mode and do the same thing. 

A third option is to DMZ your traffic to the WAN of the router inside the house and although still a double NAT, it still allows customers to do their own port forwards. 

We just enable uPNP rather than port forwards. 

1 Like

I somehow managed to miss Douglas' response to this thread last November.

Speaking only for the network I help to maintain, there are many reasons why we choose to bridge instead of route at the SM.  I am a firm believer in using the right tool for each job vs. the "kitchen sink" approach.  The job of the SM is to attach a subscriber to the network and, by coordinating with the AP, schedule the moving of bits.  That's all it should be expected to do.  An SM is never going to be a first-class router and we shouldn't expect last-mile infrastructure manufacturers to shape it into one.  We used to do the router-mode-at-the-SM model many years ago with a different set of vendors, and each time it was a disaster because that simply isn't where their core competencies are.  (The only exception to this policy on our network right now is on the LTE side, since the LTE radio interface is not "etherlike" and the various hacky ways that exist for layering Ethernet frames atop it are all littered with compromises, even when they are built into the vendor's platform.)

There are many advantages to keeping the SM in bridge mode.  First it establishes a CLEAR demarc between our network and the customer's.  (If you want to do managed router services, though, this doesn't stop you from selling the customer one of "your" routers.)  It also gives the customer the freedom (if they so choose & are a power user or a business with their own IT staff) to choose to "BYOR" and set it up the way that they need or prefer.  Such customers can then log into their own router and define firewall rules, port forwards, etc. without having to open tickets with the ISP Every. Single. Time.  (DMZ / 1-to-1 NAT at the SM is a possible way to work around this if it's in router mode, but needlessly messy, and even then some customers are not going to be happy with it...they'll insist that *their* router be *directly* assigned the IP in question, even if they can't justify their "need".  Ask me how I know.)

Second it ensures endpoint diversity when it comes to those devices on your network that are actually reachable from the world.  We don't give SMs routable IPs, but the PPPoE session does get a routable IP.  This means SMs themselves are not exposed to a hostile environment.  And because we allow for "BYOR", when a router zero-day hits, it isn't "all or nothing" for our network when it comes to exposure because not every customer is likely to be using a vulnerable device.  (See: Ubiquiti AirOS "MF Worm" from a couple of years back.  That was a fun one to deal with...for our competitors who actually had their CPEs configured in router mode with reachable IPs, that is! :-D)

It simply isn't true that true DPI is required to implement what we are talking about in this thread: having the AP and SM both honor the DSCP bits in the IP packets encapsulated by the PPPoE tunnel that the SM is bridging.  The implementation would be straightforward: only support this for PPPoE tunnels that are unencrypted, uncompressed, and non-multilink.  As long as that is the case, the offset of the start of the IP packet in each frame is going to be at a *constant* 8 bytes in.  So the logic, it seems to me, is simple: on either the AP or the SM side, when queueing a frame for transmit, if the frame ethertype is 0x8864 (PPPoE payload), just scan ahead 8 bytes and then treat the frame from a QoS perspective exactly the same way that you would if it were a raw IPoE frame.  I'm sure that this is exactly what the PMP450 platform is already doing.

-- Nathan

1 Like

First I need to address a supposition that is being made about pppoe. Pppoe does not mean double nat nor does routed mode in the epmp-sm. Pppoe is an access control method that allows control over how a subscriber can connect and at what times/rates. IPoE is an updated version of this. IPoE is not pppoe but they are very similar.
Routed mode does not mean nat, though they do tend to go together. Routed mode means the SM can accept a block of IP addresses that it will be the gateway for. Eg you buy a /16 and decide to send a /20 to a tower. Now most clients will just need one IP and nat inside their own networks. So your tower router routes that /20 to your clients on an as needed basis. Then you get a client that for some reason needs a /29 for their use and it must be static. So you add a route statement that places a route from your tower to the client for that /29. The client gets their 4 addresses and points their default route to your router which uses an address that is part of that /29. Because you use a router system using an IGP between towers and your inter-exchange points you can aggregate your route broadcast to include your client and it does not matter which IXP their traffic comes in on, your network knows how to deliver it.
In the ePMP-SM their is also NAT mode which makes the SM act like a common wifi router and this may not be what is best as now to get a port forward working you have to allow clients to access that page or have them call in to support to get the change placed. uPNP does mitigate this a bit but not for incoming port forwards as uPNP forwards expire if they are not kept up by the device that places them.

Now on to the next point.
Their are perfectly valid reasons to bridge an SM. My favorite is the managed router scenario. Since we only place Cisco routers in this position and never a Soho router this makes sense. We are contractually responsible for that router and its config and thus expect to be called at all hours of the clock to fix/change things. We are charging a monthly fee that covers this eventuality.
Another good reason for bridging is no access lane spanning. The client does not want internet access but does want to span their land over our network. We place a bridge connection at each location and then assign a p-clan for the client. The clients clans are used as c-vans and thus they never see the rest of the network and the network can not see them. Though we are moving to using an mpls configured router on each end point and tunneling through our network that way. This allows us to have better control over the bandwidth and the path for the traffic.

For all other clients we are using routes mode with pppoe for the addressing and accounting. Yes the SM can be polled and the accounting system updated, but this system is already in place and works great. Once the SM can use IPoE vsa configs we will make the change.

Wow inside ppp can be made to work but the OP was asking at the SM in bridge mode. The SM does not do this and is not intended to do this in that mode. It simply passes bits from ath0 to eth0 (these are linux based and these are the correct interface names).
If you wish to bridge your network and have qos inside pppoe then you must place the qos before the pppoe point. By its nature a pppoe stream is point to point and it does not matter the network it is on as long as it can ping both endpoints it will work. But if anything tries to interfere with the pppoe stream then it breaks. DPI can see what is in the packet but then all you can do is queue the stream to let higher priority packets through. If you try to lower available bandwidth then the stream will need to drop and reconnect.


@Douglas Generous wrote:
First I need to address a supposition that is being made about pppoe. Pppoe does not mean double nat nor does routed mode in the epmp-sm.

Nobody said either one of those things necessitated double-NAT.  This (pre)supposition was not made by anybody.  Maybe I missed something, but I think you might be reading things into other people's posts that isn't there.

Pppoe is an access control method that allows control over how a subscriber can connect and at what times/rates. IPoE is an updated version of this. IPoE is not pppoe but they are very similar.

...uh, no, IPoE is not an "updated" version of PPPoE.  The way people use the term "PPPoE", they probably should instead be saying IPoPPPoE (IP over PPP over Ethernet), because that's almost surely the only thing they use PPPoE for: encapsulating IP datagrams with PPP, but over an Ethernet network instead of a serial interface.  IPoE *literally* just means IP datagrams being encapsulated in an Ethernet frame...as in, exactly what you would see on your typical Ethernet-based LAN.  That's it.  PPPoE is kind of a "layer 2.5" thing that sits in between the IP packet and the Ethernet frame.  And yes, it's typically used as a means of access control, but that's kinda beside the point.

Routed mode does not mean nat, [...]

[snip long explanation of IP routing]

Again, nobody said it did.  And sure, of course you can disable NAT mode in the SM, and then route a /30 or other prefix to it, either while using PPP or not.  We do this all the time for customers of ours that need multiple routables (/29 or shorter prefix).

So if my goals were to 1) use PPPoE, 2) have ePMP QoS honor the DSCP bits of IP packets flowing through the SM, and 3) not use any NAT at all, then sure, using the tools available to us today, I could put the SM in router mode, have it act as the PPPoE client, and route a /30 to it.  But that would be a waste of good IPv4 space, especially if I did it for every customer (4 IPs extra per), now wouldn't it?

Now on to the next point.
Their are perfectly valid reasons to bridge an SM.

Yes, there are, but you only listed reasons why you might want *some* of your SMs in bridge mode.  My point is that we want *all* of ours in bridge mode because in our experience, the routing engines in most vendors' CPE models suck.  Plus, we also prefer bridge mode so that we can reduce the homogenous surface area on our network available to worms attacking zero-day security flaws that are discovered in said CPEs.  I don't want any of our SMs that we buy and use from any of our vendors to be reachable from the internet, period.  (Again, unfortunately not realistic on the LTE side of things right now, so we're doing what we can with what we've got.)

Wow inside ppp can be made to work but the OP was asking at the SM in bridge mode.

As was I. (And I assume "wow" was a typo?)

The SM does not do this and is not intended to do this in that mode. [...] But if anything tries to interfere with the pppoe stream then it breaks. DPI can see what is in the packet but then all you can do is queue the stream to let higher priority packets through. If you try to lower available bandwidth then the stream will need to drop and reconnect.

I don't think you're picking up what some of us in this thread are throwing down.

Correct, the SM does not do this currently.  But you talk about this as if it is not even technically feasible.  I am directly challenging that claim.

We know that it is technically feasible because PMP450 already does this (QoS bridged PPPoE frames based on the encapsulated IP packet's DSCP)!

Which means that your claim that nothing mid-stream (like a bridge!) is allowed to "interfere" with the timing of the delivery of PPPoE payloads can't be correct, because there is already another platform out there that has demonstrated it can be done!  Also, if you simply look at the structure of a PPPoE session frame, it's clear that the only kind of "state" it keeps track of is the PPPoE session ID itself, not the state of packets that are being encapsulated by it...the PPPoE header does not have "packet sequence" numbering included in it at all whatsoever.  So it does not matter if data payloads encapsulated by PPPoE arrive at their destination "out of order" (or, rather, it doesn't make any MORE of a difference with PPPoE-encapsulated IP than it does with plain IP traffic).

We also know that ePMP SM can already QoS bridged IP(oE) datagrams.

So, again, the logic seems simple:

/* ...in the middle of existing bridge & QoS code, add... */

if ( frame.ethertype == 0x8864 ) {

    frame.payload_pointer += 8;

}

/* continue on with QoS DSCP check */

This might require rummaging around in the Linux kernel itself to accomplish, but it is hardly complicated nor CPU-intensive.  But if you want to call looking ahead by 8 octets when a PPPoE Session frame is seen in the bridge "deep packet inspection", then knock yourself out...

-- Nathan

2 Likes

Has there been any update to this?

'nathana' is correct in that this has nothing to do with deep packet inspection or anything complicated. Also it doesn't matter if QoS is set by the CPE router if the radio itself doesn't look at the DSCP/CoS tags and act upon them

QoS may be setup and performing perfectly in the backbone and even on the CPE router, but it doesn't matter at all if the radio link to the customer is congested and it's treating all packets on a 'first in first out' basis. That radio needs to bump all PPPoE packets with appropriate DSCP/QoS tags to the front of its queue and transmit them first, otherwise its doing nothing and VoIP will perform 'exactly' the same with or without QoS anywhere in the network (assuming the backhaul isn't congested)

I wouldn't expect the radio itself to look at packets and determine "oh this is a VoIP packet so 'I' will tag it", I expect that role to be done on the CPE router. But I definitely do want the radio to look at the packets going across and say "oh hang on there's a CoS mark already applied? i'm bumping this packet and giving it priority"

It doesn't have to modify anything in the packet or add any tags, it just has to act on those tags

1 Like

This is an important function to most wisps. I am wondering if the new generation ePMP3000 has this ability or if perhaps it is on the roadmap?  As Mazamars312 stated we are only looking for the AP/CPE to respect existing DSCP/QoS tags already applied within the PPPoE tunnel.

Hello,

The functionality (QoS for PPPoE-encapsulated packets) is implemented in the Legacy ePMP products and it will also be available in the current generation of ePMP products once QoS is available.

Regards

1 Like