Have a 3.5 year old 2.4GHz ePMP 1000 GPS AP that losses sync in the evening times. 4.4.3 for AP and all SM’s. We have never had any major GPS issues with this AP (or any AP on this tower).
For the last week+ AP has been losing sync around prime internet usage time (early to mid evenings). Will track 8-10 satellites with SNR’s from the teens up into the high 40’s nearly all day. Come early evening time, we start to see 7-8 tracked. A little later, 3-4 tracked. Eventually none are tracked and the AP stops transmitting. SNR’s drop drastically as well.
There are few weird things going on here. First, tracked satellites begin to reduce as traffic ramps up. Second, AP GUI lags more than other 2.4GHz 1000 GPS AP’s with the same amount of traffic or more. We also have other 2.4 1000 GPS AP’s with more CPU load (or at least that is what we see) that do not display this GUI lag as bad, and do not reduce tracked satellites. Third, no matter what amount of time we set “Syncronization Hold off Time” (1000, 1500, 2000 seconds, etc) to, GPS is restored in less than 2 minutes after it falls out (according to AP system log).
It almost seems like something is going screwy with this AP as traffic ramps up and causes GPS sync to fail. The fact that GPS sync comes back within 2 minutes of transmitter stopping (according to AP system log), no matter hold off time settings is really baffling. It is like once SM’s disconnect, the AP has the resources available to track satellites again…until traffic ramps up again.
With the drop in seen and tracked satellites, and drop in SNR of tracked satellites, I would suspect the GPS puck. But… this is only happening in the evenings when traffic ramps up.
Any one else had this before?..or any suggestions?
Yes sir, AXN_1.51_2838, with message, GPS firmware is up-to-date. Sorry, meant to add that to the first post.
Last night was the first night in several nights GPS did not fall out. BUT…AP never hit max frame time. Had peaks around 60Mbps averaging 32Mbps at 8:22PM ET. Usually see peaks of 70Mbps averaging around 50 Mbps around this time. By 10PM we were averaging 24Mbps, so a slow night for this AP.
Also, last night, number of GPS satellites seen and tracked reduced slightly during this time, but SNR’s were back up into the 40’s as they are when traffic is light to medium. I may have 10 seconds or so where no satellites were tracked, but still 8-10 satellites were seen. After tracking no satellites for 10-20 seconds, we started to track 4, 6, 8, etc. satellites again…repeat. After 10-10:30PM GPS sync was back to normal. Crazy stuff.
AP started losing sync at different times of day now. Still gets sync again within less than a minute of stopping transmit. Ordered packet flux and another 2.4GHz AP just in case packet flux does not fix it. Will run all 4 AP’s off packet flux regardless if AP is bad or not.
Which puck do you use? If its the round one, get a square one they are better and have less issues. Better yet use packetflux.
Are you using the “recommended” position in the sector antenna? If so, dont! Use the tower or mast extention and point the flat face south (if in the northern latitudes). If your near the equator then you will need to point it up and get a tuna tin with holes in the bottom to act as an rf shield.
If its the same time each night, I suspect that you are getting some RF bounced interference that is killing your gps signal.
Upgrade to 4.5.4 firmware. There is an SPI interface issue in the older firmware. The SPI interface is what the GPS card uses to talk to the rest of the radio.
Actually, I do. For some reason it seems a good thing to do.
You are having issues that point to the GPS system not communicating properly. A known issue.
You asked for help and suggestions. I provided that. I did see that you have ordered packetflux gear a whole 2hrs before I posted. My comment still stands as it is a suggestion like you requested.
You dont have to be condensending though. Keep in mind you could be seeing a post that was opened on an background tab and a response sent without seeing your additional post as it is not on the tab at that time. Since I usually open several and then respond as I have a moment, this does happen.
Remember, we are here helping because we can and want to give back to the group that has helped us. You dont have to follow what anyone says and can just as easily ignore what you want.
Hey Douglas, I was actually being silly seeing I had pretty much closed the topic with the entrance of packetflux. Troubleshooting phase is/was over. AP has been working fine going on 4 years. At this point I did not feel like climbing tower and playing musical GPS pucks or AP’s (too busy for one thing). Packet flux was ordered last week and will arrive tomorrow.
We have given up on the onboard gps except for 3000L units, which has given an education on what to not do with the pucks and how to mitigate issues.
I hear you about climbing to play musical pucks and changing a radio is a new juggling game that at this time of year isnt a keen option.
Did you order the correct sync injector? One does canopy sync (cmm3) and the other does cambium sync (cmm4). The timing profiles are very different but you can use either in a system together as long as you follow the epmp user guide for cmm3 vs cmm4 sync planning.
Canopy sync means you need to set the AP sync to cmm3, there is a difference but if setup right they can work together. We only get the gigbit sync injector, saves headaches when trying to figure out why its not linking at 1gbps.
Site monitor is a good thing!
You could have just used a sync box jr, but next time.
mmmm…I did order the 1Gbps version…as stated in my post ( POWERINJECTOR PLUS CANOPY-STYLE SYNC - GIGABIT VERSION) . It is compatible with cmm4, which Cambium states in the frequency reuse manual as compatible with “onboard GPS”.
It was the sync box jr. that we purchased (packetflux reference it as sync box basic in some instances). Packetflux was delivered today. Have it running with a ePMP 1000 GPS Sync on test bench now…in cmm4 mode.
The only other gigabit version of packetflux, that I see is compatible with cmm5, which the ePMP 1000’s will not do.
It sounds like you got the correct hardware. We had a lot of trouble with the two versions of gigabit sync injector as there is a difference between the two sync types.
There was several sync sources: sync pipe, sync box, sync box jr. And the difference is really just price and available ports. Since we dont need 6p6c sync ports, the sync box isnt needed and is more costly than the jr. which just provides sync to a sync injector.
I just double checked my epmp1000’s and they are either cmm3 (canopy sync) or cmm4 (cambium sync) for selection. They as you said do not do the cmm5 sync. The difference between the two is how you orientate the secors for frequency reuse. As long as all towers are the same sync type, then you dont have to worry. We have both types in our network and when ordering new we opt for the cambium sync injector to keep from expanding our towers sync pattern in a weird pattern.
This is just friendly advice from a graduate of the School of Hard Lessons. Its simple hardware, plug it in, wait 2 mins and presto its providing sync. But like all technology, the devil is in the details and sync type and sync settings can make very frustrating things happen!
Both will work together… e.g. e1k using CMM4 or Canopy Sync, and PMP450 2.4GHz using CMM5 or Cambium Sync will sync just fine, we’ve done this in the field with towers that have both ePMP and PMP450.
Cambium developed Cambium sync in order to fix timing issues over gigabit ethernet on the 450 line. It is not recommended that you run older 450 radios at gigabit ethernet speeds and use Canopy sync, you’ll have issues.