Help installing secondary BH between tower sites

We are trying to install a second BH20 link between tower A and tower B to increase the amount of pipe being fed to tower B. Whenever we put a secondary link in and turn it on our entire network goes down, like it doesn’t know where to route the traffic. Has anyone had success with implementing this type of “poor man’s” capacity upgrade between tower locations? We have two tower sites that are seeing congestion during peak times of usage and if we could successfully get a second BH20 to feed those locations to increase available bandwidth to those sites, it would really help our customers speeds.

Thanks for any input!

Custcomp

basically when you plug that second bh in you are creating a loop. what you need is to have a router at each end of the bh’s in order to “bond” them together to make it look like one singular “pipe”. you could do this with a pair of mikrotik routerboards very easily. they even have examples of how to do it on their wiki.

I just looked at their website. What routerboard model would you recommend? Also, how will that affect our ability to to ping monitoring from the NOC to APs and SMs on the far side of that bonded link? Will we still be able to see our radio subnet on the other side of these units? We currently us a 172.16.x.x class B subnet to monitor all of the radios on the network. Of course those addresses do not show up in a tracert b/c of how canopy works, but we will still need to be able to do our monitoring with our servers alive ping monitoring software. Have you ever implemented this type of bonding solution? Thanks.

Custcomp

the RB450G would do it with room to spare. shouldnt affect anything on the other side of the link. basically it is seen as one pipe not two so everything else should work just as it did before. no i have never implemented it, but definitely would if we ran in to an issue that need it.

make sure both bh’s are running the same speed and downlink %. set bonding on both routers for balance-alb. then you should be good to go. here is the link forn the bonding wiki on mikrotik.http://wiki.mikrotik.com/wiki/Bonding

then once you have bonding setup, bridge the bonding interface to one of the other physical interfaces to plug into your switch.

What does this mean…

"Mode is basically the same as balance-tlb but incoming traffic is also balanced. Only additional downside of this mode is that it requires device driver capability to change mac address. Most of the cheap cards do not support this mode."

In regards to where it says something about changing the mac address in that mode and that some cards do not support that mode. Will this be a problem for the application we are going to use this setup for?

custcomp wrote:
Will this be a problem for the application we are going to use this setup for?


No, the MAC problem could happen if you use two pc as routers (with Routerboard OS installed).
If you use Mikrotik hardware boxes, the problem doesn't exist.

as stated above, if you use mikrotik routerboards you do not have to worry about that issue.

Thanks. I am planning on ordering two of these routerboards today. I will let you know how it goes or post any questions I may have once we get them in and work on getting them configured. How are you handling timing on the BHs, are you running on timing port since you cannot time them through a CMM since they will be plugging directly into the routerboard.

you could use a packetflux parasitic or another ap receving sync with a packetflux syncsplitter(cheapest solution) or use the ctm which has one data port for every antenna. Sync Splitter
SyncPipe Parisitic

we have used a syncpipe parsitic at several locations with just one ap. never had any issues whatsoever with them going out or not working. seems to be more reliable than a cmm as far as gps sync goes.

I’d use switches rather than those routers, less lat and cheeper, 2 150$ netgear managed switches will bond up to 8 ports to 1 trunk so you could actually add even more over time simple and no extra routing to configure.

2nd the syncpipe parsitic we dropped CMM micro’s, tomany failing and blowing power supplies. we’ve switched to all packet flux devices and all of our towers last october and not an once of trouble at all 18 sites

What is the model on that netgear switch you are using?

Regarding the netgear switches, do you need to have the two links bonded on both ends of the link with two of these switches, in order to have the bonding work correctly? I do not really have experience with bonding but it seems like this is the answer to our congestion issue, to bond two BH20’s to those two tower sites.

Yes, both the MikroTik routers and most switches use some of the same methods of bonding. You will need the same equipment at both ends otherwise the load balancing will only work in one direction (like the image for balance-tlb) or the link might not work at all.

One thing to be aware of is the management of your backhauls. Most methods are designed to be used over plain cabling and some may not permit you access to the backhauls due to how packets are expected to be returned or are sent out in the first place (IE balance-rr). Another gotcha is if a backhaul goes down: All of these Layer 2 methods depend on a loss of ethernet link to trigger a failover. If one of the wireless links drop manual intervention may be required to restore service. (For example, in balance-rr you would lose every other packet, in -xor, -tlb, and -alb you will lose whichever “flows” are running across the secondary link so it would appear that you lost a random 50% of the network)

(Disclaimer: I have no MikroTik experience)

Is there a way to setup bonding so it has any automatic failover/load balancing so if one of the BHs goes down it will automatically roll all traffic through the link that is still up, but if the down link comes back up it will automatically revert back to original config of the twin BHs bonded?

We have setup Mikrotik Bondings between a few of our towers in the past as a measure to alleviate congestion, some notes for you :
1. Bonding Description Names MUST be the same on both ends
2. Ideal setup would be to “route” each tower to each other if possible
3. Monitoring of the 4 Backhauls will be almost impossible
4. Use Balanced Round Robin as Bonding Mode - works best if you have failure on a link


Just a few hopefully helpful hints …
If you want an IP Diagram I can write one up for you to give you an idea on how to implement that…

Thanks
THE BRAD

THEBRAD,

Could you please post an IP diagram? That would be great! We just got the two Routerboards delivered on Friday so we are ready to get them programmed this week.

Thanks,

Custcomp

Tower A:
Interface 1 (AP Side) - 172.16.1.0/24 - Private Network
Interface 1 (AP Sure) - Public IP Range 1 - DHCP Pool For Customers

Interface 2 - Bonded (Bonding1)
Interface 3 - Bonded (Bonding1)

Bonding1 - 172.17.0.1/30
mode=balance-rr
slaves=Interface2,Interface3

Tower B:

Interface 1 (AP Side) - 172.16.2.0/24 - Private Network
Interface 1 (AP Side) - Public IP Range 2 - DHCP Pool for Customers

Interface 2 - Bonded (Bonding1)
Interface 3 - Bonded (Bonding1)

Bonding1 - 172.17.0.2/30
mode=balance-rr
slaves=Interface2, Interface3

Now you can either - statically route the networks or you can use a routing protocol like OSPF (Which the mikrotiks support and work well )
I would suggest OSPF as this will be easier on you as you grow and add additional towers in the future and will also provided added redundancy at that time.

As for public IP’s I would setup each router with its own public IP Pool in the same manor as the private IP Addressing scheme as long as you have that ability.
What do you have available to you for public IP’s either from ARIN or from Your Provider?
If you dont have the ability to setup 2 separate DHCP servers then I would setup DHCP Relay from whatever core server you are using now.

We are not using any DHCP on the network. Every radio has a static 172.16.x.x IP address that routes back to our head end router. We run NAT on the SMs which then hand out a 192.168.10.x IP to whatever is connected on the LAN side of the SM. If we have a customer that needs a public IP, then we bridge their SM and then give them a static public IP to program on their premise router equipment behind the SM.

So, that being said, we will not need any DHCP configuration on this setup. Will that change anything you have posted above in your recommended configuration? Currently on our AP/BHs we use the 172.16.0.x/24 subnet. As an example, the BH link we have right now looks as follows:

Tower A BH to Tower B: 172.16.0.49
Tower B BH to Tower A: 172.16.0.50

What should we setup the second BH IPs to be? Thanks for your input.

Custcomp

be sure to use arp monitoring to facilitate with failover.

Yeah that is true nucoles ! Thanks

unless you have someone on staff that understands routing I would probably just leave it a bridged network by :

Create Bond between 2 interfaces (Interface 2 -3)
and then create a Bridge between (Bonding1 and Interface 1 on Tower B Router)

Meanwhile Doing the Same thing on Tower A as well.

I have never setup a bridge across a bond before but it should act properly…

I take it as you have every one of your customers Double nat’d ? Unless they request it otherwise?