APs lost connection to cnMaestro since 3.5

Au_Wireless · October 26, 2017, 9:51pm

One of my 1000 GPS APs has been problematic since its 3.5 upgrade. It shows down in cnMaestro while all the devices behind it show up. I have re-entered my cnMaestro credentials into the AP and rebooted it. It can ping the Internet just fine but my logs are full of this:

Oct 26 14:06:34 Belvedere AP DEVICE-AGENT[14928]: Cleaning stale websock_fd 22
Oct 26 14:06:34 Belvedere AP DEVICE-AGENT[14928]: Attempting (re)connection in 5 minutes
Oct 26 14:12:37 Belvedere AP DEVICE-AGENT[14928]: lws_connect_timeout_cb: Unable to connect websocket to cnMaestro, trying again
Oct 26 14:14:47 Belvedere AP DEVICE-AGENT[14928]: lws_connect_timeout_cb: Unable to connect websocket to cnMaestro, trying again
Oct 26 14:16:55 Belvedere AP DEVICE-AGENT[14928]: lws_connect_timeout_cb: Exhausted max websocket connect attempts(10), restarting DA
Oct 26 14:17:02 Belvedere AP device-agent: Watchdog:Abnormal Device-Agent stop occured, restarting...
Oct 26 14:17:04 Belvedere AP DEVICE-AGENT[11982]: Attempting (re)connection in 5 seconds
Oct 26 14:18:12 Belvedere AP DEVICE-AGENT[11982]: lws_connect_timeout_cb: Unable to connect websocket to cnMaestro, trying again
Oct 26 14:20:23 Belvedere AP DEVICE-AGENT[11982]: lws_connect_timeout_cb: Unable to connect websocket to cnMaestro, trying again
Oct 26 14:22:26 Belvedere AP DEVICE-AGENT[11982]: lws_connect_timeout_cb: Unable to connect websocket to cnMaestro, trying again
Oct 26 14:24:42 Belvedere AP DEVICE-AGENT[11982]: lws_connect_timeout_cb: Unable to connect websocket to cnMaestro, trying again
Oct 26 14:24:42 Belvedere AP DEVICE-AGENT[11982]: Cleaning stale websock_fd 19
Oct 26 14:24:42 Belvedere AP DEVICE-AGENT[11982]: Attempting (re)connection in 5 minutes
Oct 26 14:30:45 Belvedere AP DEVICE-AGENT[11982]: lws_connect_timeout_cb: Unable to connect websocket to cnMaestro, trying again
Oct 26 14:33:02 Belvedere AP DEVICE-AGENT[11982]: lws_connect_timeout_cb: Unable to connect websocket to cnMaestro, trying again
Oct 26 14:35:10 Belvedere AP DEVICE-AGENT[11982]: lws_connect_timeout_cb: Unable to connect websocket to cnMaestro, trying again
Oct 26 14:37:18 Belvedere AP DEVICE-AGENT[11982]: lws_connect_timeout_cb: Unable to connect websocket to cnMaestro, trying again
Oct 26 14:37:18 Belvedere AP DEVICE-AGENT[11982]: Cleaning stale websock_fd 22
Oct 26 14:37:18 Belvedere AP DEVICE-AGENT[11982]: Attempting (re)connection in 5 minutes
Oct 26 14:43:22 Belvedere AP DEVICE-AGENT[11982]: lws_connect_timeout_cb: Unable to connect websocket to cnMaestro, trying again

Au_Wireless · October 28, 2017, 8:32pm

Logs on this AP are also full of this since upgrading to 3.5:

Oct 27 18:01:55 Belvedere AP kernel: [5542636.940000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 27 20:58:33 Belvedere AP kernel: [5553235.270000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 27 21:24:13 Belvedere AP kernel: [5554775.450000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 05:33:30 Belvedere AP kernel: [5584132.450000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 05:33:33 Belvedere AP kernel: [5584135.150000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 06:50:25 Belvedere AP kernel: [5588746.890000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 06:55:00 Belvedere AP kernel: [5589022.510000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 07:30:50 Belvedere AP kernel: [5591172.360000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 07:36:15 Belvedere AP kernel: [5591497.210000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 07:37:07 Belvedere AP kernel: [5591548.980000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 07:37:27 Belvedere AP kernel: [5591569.010000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 07:48:59 Belvedere AP kernel: [5592260.600000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 08:05:49 Belvedere AP kernel: [5593271.210000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 08:15:15 Belvedere AP kernel: [5593837.500000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 09:12:34 Belvedere AP kernel: [5597276.370000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 11:29:55 Belvedere AP kernel: [5605516.970000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 11:33:01 Belvedere AP kernel: [5605703.340000] GPF: tdd_gpf_send tdd_maptx_add_gpf error

There was an old post about this message a while back but no solution that I can find.

Fedor · October 29, 2017, 8:13pm

@Au Wireless wrote:

Logs on this AP are also full of this since upgrading to 3.5:

Oct 27 18:01:55 Belvedere AP kernel: [5542636.940000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 27 20:58:33 Belvedere AP kernel: [5553235.270000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 27 21:24:13 Belvedere AP kernel: [5554775.450000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 05:33:30 Belvedere AP kernel: [5584132.450000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 05:33:33 Belvedere AP kernel: [5584135.150000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 06:50:25 Belvedere AP kernel: [5588746.890000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 06:55:00 Belvedere AP kernel: [5589022.510000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 07:30:50 Belvedere AP kernel: [5591172.360000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 07:36:15 Belvedere AP kernel: [5591497.210000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 07:37:07 Belvedere AP kernel: [5591548.980000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 07:37:27 Belvedere AP kernel: [5591569.010000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 07:48:59 Belvedere AP kernel: [5592260.600000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 08:05:49 Belvedere AP kernel: [5593271.210000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 08:15:15 Belvedere AP kernel: [5593837.500000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 09:12:34 Belvedere AP kernel: [5597276.370000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 11:29:55 Belvedere AP kernel: [5605516.970000] GPF: tdd_gpf_send tdd_maptx_add_gpf error
Oct 28 11:33:01 Belvedere AP kernel: [5605703.340000] GPF: tdd_gpf_send tdd_maptx_add_gpf error

There was an old post about this message a while back but no solution that I can find.

_______________________________________________________________________________________

Hi Chadwick,

Could you please contact me at e-mail and provide as much as possible information on this issue?

How many APs are affected, FW versions of AP and SMs, configuration files, etc.

Very similar issue was fixed long time ago in 2.3 FW Release.

Thank you.

Au_Wireless · September 13, 2019, 4:15am

This is back again in 3.5.6 I have three devices out of hundreds with:

device-agent: Watchdog:Abnormal Device-Agent stop occured, restarting...

in the log and they are no longer showing as up in cloud hosted cnmaestro. I can ping IPs and host names from the three offending devices so they are online and able to talk out to the Internet. Two of the devices are SMs and one is an AP. That AP has one SM and that SM is one of the two that is down. The other is in a totally different spot on our network. All three are running 3.5.6 and this problem did not start with the upgrade to 3.5.6 but popped up 2 days ago and 21 hours for all three devices. We have not made any network changes in the last 3 days.

Fedor · September 13, 2019, 10:13am

@Au Wireless wrote:

This is back again in 3.5.6 I have three devices out of hundreds with:
device-agent: Watchdog:Abnormal Device-Agent stop occured, restarting...
in the log and they are no longer showing as up in cloud hosted cnmaestro. I can ping IPs and host names from the three offending devices so they are online and able to talk out to the Internet. Two of the devices are SMs and one is an AP. That AP has one SM and that SM is one of the two that is down. The other is in a totally different spot on our network. All three are running 3.5.6 and this problem did not start with the upgrade to 3.5.6 but popped up 2 days ago and 21 hours for all three devices. We have not made any network changes in the last 3 days.

Hi Chad,

DA restart has been fixed in the 4.x FW versions.

Thank you.