We are having extreme trouble in the administration of our Canopy network due to the fact that without apparent reason the Web page of certain AP´s and BH units just stops reacting. The only way to recover is to reset the devices via Telnet …but this causes havoc in the network due to the bridge timeout issues etc etc etc so that we need an urgent solution for this problem.
We have noticed that this issue usually occurs in AP´s and BH´s that have a high constant traffic load, but for some or other reason this has also happened in AP´s that do not have much traffic at all but it takes a lot longer after a reset for the low traffic AP´s to stop reacting to the WEB administration. The BH with high traffic loose the capability of being managed within a very short time (less than a day, sometimes less than an hour).
We have upgraded all units to 4.23 and Boot 3.0 and fpga 062403. All units that we use are DES and synchronized via CMM and CMM-Micro´s. All the IP´s are private (172.20.x.x)
We need an urgent solution to this problem because everytime we want to add an SM, we need to reset the AP which obviously affects all the rest of the customers in a sector.
I have this problem like many others. The worst cluster of 4 AP requires a daily reboot, and all 4 stop working at different times over the course of the next 24 hours. This problem cluster has gotten WORSE since we updated the network 6.1 (software scheduler).
I have tried all the mentioned configurable workarounds. Our bridge tables are set at 360 minutes (6 hours), while the ARP ageing at our router is 4 hours (Cisco default). In addition, all our AP have always been assigned private 172 addresses, and I have added filtering at our routers so that only our internal management stations have access. Nothing has helped so far.
Its not possible for this symptom to be caused by a network configuration or reachability problem - ping and telnet still functions. A misconfigured firewall could block, but that effect would remain past rebooting your AP.
Port 80 (http) packets go through, but the connection is FINished before any data is returned. It is very obvious to me, that the httpd software in the Canopy unit is failing. If you’d like some tcpdump output, ask me. I can generate a few packet logs for analysis.
Perhaps Motorola should waste some flash space, and use Apache or Tux - known working httpd services. Of course the http interface wouldn’t even be required if the CLI offered more unhidden functionality.
I see a couple posts suggesting the problem may be related to the Display-Only and Full-Access passwords. All of the units I’ve installed have always had both passwords set the same, and I’ve never experienced this problem.
Could just be a coincidence.
Even with 6.1 and password in both “view” and “change” it´s still the same thing. I am now changing technology because Motorola has not been able to solve this and the VoIP issues.
The only problem with this forum is that Motorola hosts it but never looks at what is going on and is not interested in their customers. You can let off a bit of steam but that has´nt helped to solve this and other problems in over 14 months!!! I am really unhappy with this Motorola attitude. I´d really appreciate someone taking an interest in our problems from Motorola…but I think that´s not going to happen, so all that we´re left with is to make our views public on a message board such as this one and invest time and money in finding other technologies that work!
If you’re willing to spend a bit more time at this, I’ll see what I can do to help.
Web Login: Are the “view” and "change passwords the same password? Do all units have the problem, or just some; just APs and BHs, or are SMs also affected; is it always the same units that have the problem? Can you duplicate the problem; can you describe a scenario so I can try and duplicate the problem? Is the web page blocked from local or remote access, or both?
VoIP: Fill me in. You apparently haven’t posted anything on this.
As a counter example to the problems everyone seems to be having, here is our setup that does not experience the problem at all.
The whole system is 5.7. All non-advantage. Running 6.1 firmware.
Both passwords are the same
Bridge Timeout is set to 1440 minutes
All radios are on 192.168.0.0/24 subnet
The only problem we ever had was when we gave only the APs public routable IPs. The web interface did not lockup, but the units rebooted themselves quite often. When they were switched back to private IPs, all went back to normal, and has been flawless since.
Hope that helps.
here we go again - I do´nt believe it´s going to solve the problem but…:
- 172.20.x.x addressing
- view and change passwords the same
- it is usually the high usage AP´s and BH that present this problem
- the time till we cannot log in is now arround 2 days max for high usage AP´s and BH units and 1 to 2 weeks for low usage units ( not dependant on number of SM it is really related to high or low throughput)
- we have SNMP enabled (but have also disabled it with no change)
- there is no remote access to the AP´s and BH from remote sites because of the private addressing so it is the local web access that get´s blocked.
- 2 of the BH units have public adresses and it is a matter of hours untill they get blocked (constant 6MB up and 3MB down on a 20MB unit)
The VoIP issue is something that Motorola has built in not being able to handle more than 8 to 10 simultaneous calls per AP. For this reason we are also looking into alternatives.
Thanks for trying. But after more than a year with this issue I really do´nt believe that you or Motorola are able to solve this issue unless someone at Motorola sit´s down and actually does some debugging on thier equipment.
Thanks anyway for the trouble.
Are all you guys monitoring port 80 with a program such as WhatsUP Gold? I have had a few devices from various manufacturers fail due to WhatsUP pinging them, or making quick connections to services and dropping them, etc. Usually the device runs out of memory or doesn’t close the maximum number of HTTPD servers running. Perhaps there is a bug in the radios where HTTPD is respawning a new process and not closing the previous connection? I have some switches that had this problem - both telnet and httpd died after a few weeks of WhatsUP connecting/disconnecting.
Web Login: Motorola’s response to the problem of the web server not responding is that a virus or worm is attacking the unit. Their suggestion is to not assign public IPs to the units, not create 1-to-1 NAT mappings, and strictly limit the number of PCs on the management network. They say the web server experiences a stack overflow when accessed repeatedly.
I don’t know if this is intentional (as a security measure), or if it’s a bug, or if it was a compromise made in the design in order to conserve resources. Pete’s comment about the same problem being caused by What’s Up is a reminder that “attacks” don’t have to be malicious.
VoIP: The limitation on VoIP sessions should be strictly related to bandwidth. If the AP and SMs only had to handle VoIP, you could handle 200 sessions at 12K each; 16K would be marginal. The problem I imagine you’re having is that there’s plenty of other traffic being passed, leaving little for VoIP. Using the High-Priority setting will help VoIP, but it’ll kill the other traffic.
If you’re running out of bandwidth, your only option with Canopy is to reduce the SM-to-AP ratio.
speschko wrote: The only problem with this forum is that Motorola hosts it but never looks at what is going on and is not interested in their customers.
We look at as many posts as we possibly can and we are very interested in what customers have to say. We have said it before but I will say it again; this forum is NOT formal support. Many times we have contacted customers offline to help them with their problems posted on this forum and other forums not hosted by us. How many other companies follow items posted online and pursue the customer? We do this, we want to help.
We are fully aware of the web lockup issue. We have issued a work around for the problem. If you have a problem with the work around then please call into support. This way we can formally log your concern.
Hmmm… I have read all the posts here. We have been running Canopy since March, 2002 and must be doing something “right” in this area, as we do not experience this problem. All our radios, as with any infrastructure gear in the entire company run private IP’s on the same “wires” or subnets as the devices that HAVE to have public IP’s. Not to say Motorola is right, but if you ever watched the amount of Internet “junk” flying around through analizers, you would certainly understand at least the high potentional to have “non-anticipated” types of packets hitting your radios.
We simply have never expereienced that kind of problem with the way our system is setup. The only remotely related problem we have accessing the radios, is accessing SM’s through an AP. Using the default for what the radio assigns to talk to the SM’s (192.168.101.x), we have found some problems accessing the radios, so we use an obscure subnet (10.231.34.x) or something (same one on every AP), and that completely elliminated that problem.
I don’t really understand why you would need to have your radios or almost any infrastructure device on public IPs. We run “interlock” devices or other control PC’s that can be telneted, ssh’d, remote desktop’d, etc. to THEN acces the other devices. Been doing this in our ISP for 8 years and it works wonderfully.
well I am really gettin irratated with this same problem that MOTOROLA cant seem to fix. I have tried almost every concievable idea of how to fix this???
my equip dosent reboot but the http interface keeps dying on all AP’s & BH’s
I don’t think there is a fix. A “workaround” is an admission that they can’t fix it. We are living with the problem but don’t like it either. You would think that somebody at MOTOROLA (not exactly a bunch of guys working out of a garage) could come up with a way to have a working interface on their equipment, especially since most every other equipment maker can do it.
EXACTLY!!!
how they phrase it a "workaround"
Lets issue a FIX for this problem.
I have many clients that are running Citrix sessions, and my rebooting to access the http interface is making them mad, and threatening to leave.
Have you guys performed the workaround that Motorola has recommended? I doubt it. Until you do, perhaps you should stop the flames, and perform the workaround. Or would you rather pay double for a radio that has the ability to fight off all of the bogus traffic?
With all of the killer performance we get from Motorola, I can live with having to implement ONE workaround.
I’m not trying to start an argument, but I have been watching this post for a long time, and the bottom line is, the problem cannot be fixed with the current design. It’s safe to assume that a new design will require costly R&D which we as ISP’s get to pay for. Put that into your ROI and see where it puts you. I think you will find that the cost of implementing the workaround will be less.
Jerry Richardson
airCloud Communications
I see mention of a workaround for this problem, but I’m not sure I know the details of the workaround. Is it the stuff about assigning a private address, max out the bridge timeout and all that? I’ve done that and still have lockup issues.
Maybe its related to how often our techs click the ‘refresh’ button while monitoring an AP or SM. I know that I can reproduce the problem myself by hammering the refresh button about 5 - 6 times, and then I have to telnet and reset the AP…
Is there any more to this workaround than I’ve mentioned?
Yes I have tried thier “workaround” it helps, but dosent entirely fix the problem, and you talk about “awesome” performance, I am failing to see that, maybe If I upgraded to all advantage gear it would help, but with 15 AP’s and 8 BH’s just in my location, We cannot afford to just throw those AP’s away and buy new ones. Moto should have a trade up to advantage promo, for those of us who have spent asmuch as we have to be able to do it. Try it yourself, 60 active sessions per AP, awesome does not explain it.
Iam not flaming them, but letting them know the real issue still isnt resolved.
I have 11 AP’s running with a mix and match of 900 integrated, 2400, and 5700 - some advantage, some not. All of the radios are running 6.1 or later (That upgrade was fun right on the heels of upgrading everything to 4.2).
Maybe I am not trying to sell the same thing you are. Our fastest service is 1024/512 - and we deliver this consistently regardless of the radio they are on. I have done downloads from our server within our network and see ~3MB transfer. My connection at my house is not limited so I see about 1200/450 through our single T1. Maybe when we add the second T1 and try to pull 2.5-3.0Mbps off the 'Net I will see limitations, but with transfers off the server at 3.0MB I doubt it will be a problem.
I did recently have a problem with a radio limiting bandwidth so I reset it to all factory defaults and reprogrammed it and it started behaving properly. Maybe you just need to burn your network down and reprogram all the AP’s and BH’s from scratch. Also, are you running 6.1 or later? I noticed a very significant improvement from 4.2 to 6.1, and even better performance with 7.0.
As far as the radios dropping the interface, it’s a pain in the ass, but the network runs so well, I rarely need to look at the interface anyway. If a radio is mis-behaving, I telnet to it and check the logfile. If it has been resetting or otherwise shows problems, I reset it and then go into the Web interface. That’s just me, and may not work for you.
The biggest problem I have is with routers getting buggered up. If I have the customer reboot the router, the problems go away. Most of the time this is being caused by hijacked machines spewing network sessions and overloading the router. When I find this I am relieved because it puts the responsibility back on the customer to clean up thier machines. I always want to assume it’s my network, but more often than not it’s something in the customer’s network.
Jerry