CMM Micro (2.2), AP (7.3.6) Problem

Well, I know there’s been a lot of discussion regarding CMMs & APs, so I’ll add another one.

I’m having a problem with one AP of my 6 AP cluster.

It seems as though once every few weeks (read 2), the AP locks up and is not reachable.

If I reset it via the CMM, it boots back up, passes two pings, then won’t respond.

If I power it off via the CMM, and wait, then boot it back up, it works as normal.

It is set to auto-negotiate on the CMM, as all the other APs are also. I’ve also changed ports on the CMM, which didn’t seem to help out.

The APs are set to private addresses (10.3.x.x) network. I don’t notice flooding traffic to it or it’s webserver as after the last issue, I implemented some counters to watch for traffic on it. Nothing abnormal.

Sync is showing OK.

These are 900MHz APs I’m referring to.

Any ideas on what to check for? It’s kinda frustrating and I can’t replicate the issue with any test APs.

Thanks,

Rich

bump

The AP has locked up now about 10 times over the course of the holiday weekend.

Not very fun to deal with. Still can get by with shutting it off via CMM and turning it back on.

edit: it appears this is happening around the same time each time it goes down. around noon. that’s weird to me.

Can you post the event log? When it locks up can you Telnet to it?

Look at the Ethernet Stats and see if you are getting any errors

rjk,

Sounds like it could just be a bad end. Have you replaced the ends and tested the cable?

I have replaced the CMM end, not the top of the tower end.

ethernet stats errors:

inerrors count: 3
CRC error: 3

Things seem consistent in there (event log), although, I did notice a Hardware Watchdog reset a few different times. Not sure if that’s the key I’m looking for. Trying to research it a little bit.

I’ll post more as I know. This AP we had some troubles with prior to putting it up, but now the not reachable issues are becoming more and more frequent, and I noticed after analyzing it for a pattern/reason, I had seen around noon is when this is happening.

I’ll check the telnet idea when/if it goes down tomorrow around noon.

I apologize for the large post!!

[ EVENT LOG START]



System Event Log
00:00:00 UT : 01/01/01 : File root.c : Line 945 Software Version : CANOPY 7.2.9 Jul 23 2005 01:49:03 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 949 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 955 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 959 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1449 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 940 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 945 Software Version : CANOPY 7.2.9 Jul 23 2005 01:49:03 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 949 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 955 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 959 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1449 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 940 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 945 Software Version : CANOPY 7.2.9 Jul 23 2005 01:49:03 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 949 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 955 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 959 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1449 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 940 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 945 Software Version : CANOPY 7.2.9 Jul 23 2005 01:49:03 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 949 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 955 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 959 FPGA Features : DES Sched
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 940 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 945 Software Version : CANOPY 7.2.9 Jul 23 2005 01:49:03 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 949 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 955 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 959 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1449 Acquired GPS Sync Pulse.
00:01:46 UT : 01/01/01 : File httptask.c : Line 733 Reboot from Webpage.
00:01:46 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1522 Loss of GPS Sync Pulse.
00:01:28 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:01:28 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset WatchDog
00:01:28 UT : 01/01/01 : File root.c : Line 940 System Startup
00:01:28 UT : 01/01/01 : File root.c : Line 945 Software Version : CANOPY 7.2.9 Jul 23 2005 01:49:03 AP-DES
00:01:28 UT : 01/01/01 : File root.c : Line 949 Software Boot Version : CANOPYBOOT 3.0
00:01:28 UT : 01/01/01 : File root.c : Line 955 FPGA Version : 070605
00:01:28 UT : 01/01/01 : File root.c : Line 959 FPGA Features : DES Sched
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 940 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 945 Software Version : CANOPY 7.2.9 Jul 23 2005 01:49:03 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 949 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 955 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 959 FPGA Features : DES Sched
00:00:43 UT : 01/01/01 : File httptask.c : Line 733 Reboot from Webpage.
00:00:32 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:32 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset WatchDog
00:00:32 UT : 01/01/01 : File root.c : Line 940 System Startup
00:00:32 UT : 01/01/01 : File root.c : Line 945 Software Version : CANOPY 7.2.9 Jul 23 2005 01:49:03 AP-DES
00:00:32 UT : 01/01/01 : File root.c : Line 949 Software Boot Version : CANOPYBOOT 3.0
00:00:32 UT : 01/01/01 : File root.c : Line 955 FPGA Version : 070605
00:00:32 UT : 01/01/01 : File root.c : Line 959 FPGA Features : DES Sched
00:00:43 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1449 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 940 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 945 Software Version : CANOPY 7.2.9 Jul 23 2005 01:49:03 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 949 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 955 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 959 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1449 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 940 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 945 Software Version : CANOPY 7.2.9 Jul 23 2005 01:49:03 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 949 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 955 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 959 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1449 Acquired GPS Sync Pulse.
00:03:32 UT : 01/01/01 : File root.c : Line 3182 Reset from Telnet command line.
00:03:32 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1522 Loss of GPS Sync Pulse.
00:03:32 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:03:32 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset WatchDog
00:03:32 UT : 01/01/01 : File root.c : Line 941 System Startup
00:03:32 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:03:32 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:03:32 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:03:32 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:03:43 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:09:21 UT : 01/01/01 : File httptask.c : Line 733 Reboot from Webpage.
00:09:21 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1525 Loss of GPS Sync Pulse.
00:08:36 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:08:36 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset WatchDog
00:08:36 UT : 01/01/01 : File root.c : Line 941 System Startup
00:08:36 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:08:36 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:08:36 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:08:36 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:08:46 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
13:54:23 UT : 01/01/01 : File httptask.c : Line 733 Reboot from Webpage.
13:54:23 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1525 Loss of GPS Sync Pulse.
13:54:21 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
13:54:21 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset WatchDog
13:54:21 UT : 01/01/01 : File root.c : Line 941 System Startup
13:54:21 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
13:54:21 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
13:54:21 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
13:54:21 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
13:54:31 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
23:29:17 UT : 01/02/01 : File httptask.c : Line 733 Reboot from Webpage.
23:29:17 UT : 01/02/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1525 Loss of GPS Sync Pulse.
23:29:04 UT : 01/02/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
23:29:04 UT : 01/02/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset WatchDog
23:29:04 UT : 01/02/01 : File root.c : Line 941 System Startup
23:29:04 UT : 01/02/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
23:29:04 UT : 01/02/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
23:29:04 UT : 01/02/01 : File root.c : Line 956 FPGA Version : 070605
23:29:04 UT : 01/02/01 : File root.c : Line 960 FPGA Features : DES Sched
23:29:15 UT : 01/02/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:10 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 1062 Time set
00:00:00 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/syslog.c : Line 918 System Reset Exception – External Hard Reset
00:00:00 UT : 01/01/01 : File root.c : Line 941 System Startup
00:00:00 UT : 01/01/01 : File root.c : Line 946 Software Version : CANOPY 7.3.6 Oct 24 2005 12:06:56 AP-DES
00:00:00 UT : 01/01/01 : File root.c : Line 950 Software Boot Version : CANOPYBOOT 3.0
00:00:00 UT : 01/01/01 : File root.c : Line 956 FPGA Version : 070605
00:00:00 UT : 01/01/01 : File root.c : Line 960 FPGA Features : DES Sched
00:00:11 UT : 01/01/01 : File C:/ISIPPC/pssppc.250/bsps/devices/whisp/rfhwsched.c : Line 1452 Acquired GPS Sync Pulse.

[END EVENT LOG]




Thanks for the replies.[/code]

Nothing in there that gives any clues…

CRC errors should be 0. If you are getting any at all you probably have a cabling issue.

I don’t worry too much about ethernet CRC errors as long as they are less than .01% of total errors. Obviously, if you see a 1% error rate
or anything above that, you have a service-impacting issue. If you have a 10% error rate, or more then you have a very serious issue. But if you have 7340 CRC errors out of 73,400,000 packets, who really cares?

If the equipment is working properly otherwise and you can’t measure any noticeble level of errors with other techniques then there’s no reason to worry about a fraction of a fraction of a percent of error… It’s not like modulation/demodulation of an electrical or optical signal is going to work flawlessly all the time!

Oh, and on the actual problem, have you tried running at 10Mbps FDX instead of 100Mbps? Obviously this problem doesn’t fall in the category I was attempting to describe. And the very low CRC error count doesn’t describe the actual problem…

I have Canopy APs near a strong FM transmitter with very high quality shielded, grounded ethernet cables that won’t talk to a CMM at 100Mbps reliably. They’ve got a 10-15% packet loss rate at 100Mbps, no matter what I do. (No, I never worked with it long enough to verify if the packet loss was in the CMM->AP direction, AM-> CMM, or both)

So I run them at 10Mbps and they are rock-solid… The cable terminations were even done multiple times by people with traditionally VERY low error rates at ethernet termination, new cables were run multiple times, and no matter what, we could not avoid the problems at 100Mbps. 10Mbps FDX should be just fine for most Canopy APs, anyways…

Other outdoor ethernet (Atheros SoC ethernet at least) seem to work at 100Mbps just fine in this hostile environment, but they are talking to various Netgear gigabit switch ports instead of the Canopy CMM. And then we also have used gear that was not reliable at 100Mbps or even 10Mbps at this tower site due to the noise. (Trango Atlas PtP, for instance. The ethernet port would lock up all the time, requiring reboot. Trango was unable to reproduce the problem after they released some software upgrade, but the upgrade didn’t fix my equipment, and after that point their support people just ignored me. And Trango’s engineers had no clue about the ethernet chips or the drivers that actually ran their products, they basically just write user interface code and depend on third party vendors for all the drivers. I can understand using someone else’s drivers, but at least have the source code and a fucking clue on how to make sense of it…)

After looking more, I do see there are CRC errors on all of the APs. Very low, with 2 being relatively higher than the rest of the cluster.

Negotiation is set to Auto…

I should manually set it to see if it helps resolve this.

This is actually getting a little elusive.

Setting to 10M might temporarily help, but I suspect that there may a larger issue looming such as a bad connector at the AP, or since this AP had issues previously the AP may be going bad.

If, after re-terminating the connector at the AP you still have issues, try swagging a completely new cable from the CMM to the AP.

Another possible solution might be to set it to factory defaults and then reconfigure. Continued issues may require you to reload the software.

rjk

After one of your outages, do you happen to notice the uptime of the AP? Has it been reset? IF it has then it may be losing power. Then you probably have a cable issue. Or the port in the cmm is failing.

Well, I can’t determine the uptime of the AP due to not being able to reach it.

There is just one thing that keeps me going crazy. If it were truely a bad cable, would it not be having problems ALL the time? This is almost between certain hours of the day.

I thought about something last night before sleep. It has been unusually warm the past few days here in Ohio.

This may have something to do with it. I thought ‘thermal’. So I took a look at prizm’s graphs. I noticed something.

Just before the first time of the outtage on this past Thursday, I noticed the Temp go from 93f to 80f within 10 min (poll). Immediately after that it jumped to 107f, then the next poll 111f.

It has consistently risen (temp. wise). With each time the AP has locked up there being a dramatic temperature increase (20-25 degrees).

I know the temperature readings aren’t exact according to the stat page via Prizm.

Is it possible that this COULD be an thermal issue?

Maybe the sun is hitting the AP directly between 12 and 2 every day right now and it’s just not liking it.

As crazy as it sounds, I like the thermal idea moreso than cabling issues. Just simply because of the idea of if it were cabling, it would be acting up consitently, not a specific time of day. Maybe a coincidence but I’m not sure.

Rich

It’s possible you have a marginal connector that is changing with temperature. But again, this AP had issues before you installed it, so it’s possible that those issues are getting worse with temperature.