Ours is a flat network. We have a single Cisco 3600 series providing routing for the following:
2 Class C Public Ranges
1 Class C NAT’d (at the 3600) range (only about 30 IPs on this)
1 Class B Private 10.0.x.x Management IP range (about 150 devices)
I suspect that we have pushed this about as far as we can (or should). I think it may be time to start implementing VLAN’s to segement AP’s, and towers.
How are you using VLAN’s to segment your network? How is it working for you? Not being a real Networking guru like some of you, I would rather go with something tried and true than experiment on my own.
We don’t have issues with broadcast storms, we have that filtered at the SM’s. We do occasionally run into a situation where the bridge tables will get buggered up and I have to reboot everything from the router to the AP’s.
Thanks in advance.
By the way, anyone seen VJ?
I have about 900 people on a flat network. since i filtered everything on the SM and a few managed switches I havnt had any problems.
however im waiting for this to bite me in the arse. I would be interested in learning how to correctly do this as well.
thanks 
same deal here…we’re not at 100 yet but i’d love to see us there. i’ve implemented a 2nd router using NAT and another public IP at our newest tower site…i had hoped to move most all the sites to that implementation eventually. We’re DSL at the core…but if i could do this with VLANs it’d be better.
yep, this would be interesting to know…
I am building a network with subnets and vLANs from the start.
There are a bazillion ways to do this, much like a zillion ways to work the algebra problem. You just have to follow the rules and it will work, but, not every method is “the best way” or the shortest way or the least expensive way. You have to understand the dynamics of a large ethernet, not exactly guru-packet-level type of understanding, but a good qualitative idea of how ethernet works at the Layer 2 and Layer 3 level.
If you have only one router between your T-1s (or whatever you have) and your clients, then that router minimally has to be vLAN capable. You also need, at a minimum, vLAN-aware capable devices on “the other end” of your network. Fortunately, Moto has provided that in the APs and SMs, and because of this, your vLAN design can be quite simple. Such as, creating vLAN interfaces on your router and assigning the APs/SMs to those vLANs. This simple design means that from your one router to the AP is still a bridge, but it will be segmented according to your vLAN design. I don’t like that design because that forces up to twice the traffic across a BH radio link. I prefer to route (and vLAN) at the AP site. Yes, this means each tower/AP site gets it’s own router. And none of my BH traffic is vLAN’d/vLAN-aware at all.
Basically, what I’ve designed is a bridge for all my BHs, and, each AP is on it’s own bridge, in a vLAN (and eventually, ALSO in it’s own subnet/vLAN), on the other side of the router at the tower.
I use Mikrotik software on mini-ITX PCs for my routers. I will use Mikrotik Routerboards on smaller sites (the projected load at the tower being the determinate).
I did not want to introduce much latency in my network via traversing lotsa routers, so I bridged my BHs and put each AP in it’s own vLAN or subnet, as applicable. This way, between the customer and my T-1s, there are only two routers, and one of them is absolutely unavoidable: the T-1 router. The other one is there for network design/management/segmentation/feature addition/firewalling/etc purposes.
Is this the “right” way? I’m not sure. But I know there is nothing wrong with what I’ve done, and, if I later discover a better way to do things, I know that I will not have to destroy what I’ve built – just change it a bit. The Mikrotik router software is very feature-full so I know that I can do anything “advanced” later on if I need to. And if for some reason I want to bridge all my APs and BHs together, the router software will let me do that, too.
From what I’ve been able to measure, under light load, the routers introduce 0-1ms of latency to the whole thing.
I purchased (fanless) 800MHz mini-ITX PCs to keep it that way. (The Routerboards presently max out at 330MHz - or something close to that).
Read and experiment. If you’ve made it this far, with some pain, you’ll be able to figure out what to do. That’s what I did. I have no formal network training/certs – I’m just smart enough, conscientous and committed, that’s all.
vince wrote: I have about 900 people on a flat network. since i filtered everything on the SM and a few managed switches I havnt had any problems.
900 !??! Do these 900 have their own NAT routers?
if they dont have a router they have nat enabled on thier sm
Jerry,
I wouldn’t say that I’m a network guru, but I will outline what we have done and some advantages/disadvantages. I would also like to see some other designs from other ISPs. We used to be on a flat network as well. The longer you wait to make the change to a vlan/routed network, the harder it will be. m_b_h says he likes to route at the AP site. We used to do that as well, but we changed our design.
We currently have ~30 tower sites. Like many ISPs, our network spans hundreds of miles. All of our BH are wireless. We VLAN with cisco layer-2 managed switches at the tower sites. Each AP gets its own subnet/port on the switch. We chose not to use the VLAN capability on the Canopy APs for a couple of reasons. First, we can use the outstanding Cisco support when needed (Canopy support needs a lot of work). Second, we are familiar with cisco, and a new IOS doesn’t break your network (eg, 8.1 and NAT). Some subnets are /24, some /25, some/26, depeding on the density on the AP. Traffic is trunked back to our NOC to Cisco layer-3 switches.
Disadvantages:
1. Clients are on the same VLAN. Clients on the same AP will “see” some traffic from other clients. A per-customer-VLAN is optimal, but this seems like a management headache of epic proportions. If anyone is doing per-customer-VLANs successfully, please respond with your design.
2. Clients with multiple locations are on different VLANs. Customers with multiple locations not on the same VLAN, but on the same tower, must be routed across the BHs to the layer-3 switch and routed back. This is not as bad as it seems, as our BH latency is ~5ms. Also, most of our customers who have multiple locations are not on the same tower. We could use layer-3 devices at the tower, but the benefit to expense is too low.
3. Broadcast/multicast traffic is sent across the BH links.
Advantages:
1. Traffic is segmented. IMHO, you need to segment at least on class Cs to get the best performance.
2. Problem CPEs are easier to find. Consider a CPE with a spoofed IP or MAC address. ARP poisoning is another huge issue with a bridged network.
3. Problem CPEs do not affect the entire network. A spewing CPE can eat a lot of bandwidth before it is discovered.
4. Broadcast/multicast traffic is sent across the BH links. I know this is also a disadvantage. Sometimes, nothing beats a raw “on the wire” packet capture. You can plug in Ethereal or similiar, and see exactly what is traversing the network, including broadcast/multicast packets. This can be done from the comfort of the NOC. With a router, broadcast/multicast is blocked (typically) at the tower site.
Hope this helps.
dats why did you decide to change the design from how m_b_h explained?
You could also make the customer’s with miltiple sites members of a second VLAN for traffic from sm to sm so it does not have to go accross the BH to the layer-3 switch.
The CMMmicro ver2.2 software supports port based VLAN switching that will allow you to isolate the ports from each other this will prevent AP to AP traffic on the same cluster.
ver 8.1 it supports sm isolation accross a cluster so sm’s on the same AP are isolated from each other, but sounds like you’re not a 8.1 fan due to NAT. I do not use NAT at the sm and I have not switched to 8.1 accross the network yet. still testing at a couple of sites.
attitude0330,
You could also make the customer’s with miltiple sites members of a second VLAN for traffic from sm to sm so it does not have to go accross the BH to the layer-3 switch.
We use cisco switches at the tower sites using port-based VLANs. A port can only be a member of one port-based vlan.
The CMMmicro ver2.2 software supports port based VLAN switching that will allow you to isolate the ports from each other this will prevent AP to AP traffic on the same cluster.
This is currently done with the cisco switches.
ver 8.1 it supports sm isolation accross a cluster so sm’s on the same AP are isolated from each other, but sounds like you’re not a 8.1 fan due to NAT. I do not use NAT at the sm and I have not switched to 8.1 accross the network yet. still testing at a couple of sites.
We use NAT at the SM as much as possible. I went to a canopy user group event on Monday. I mentioned the NAT issues to one of the presenters (he was the main software developer for canopy), and he insisted that there is no problem with NAT and 8.1. Has anybody else had good luck with NAT and 8.1?
Thanks
Wasn’t paying attention to that small detail. Sounds like you could benefit from 8.1 if it works with NAT.