No data read from unix domain socket - syslog message

Nekomata · January 26, 2018, 9:43pm

I've come across this message in a few of our ePMP 1000 access points, any idea what this means? Attached some logs with this message.

system_log (2).txt (8.11 KB)
system_log (3).txt (3.07 KB)

Nekomata · January 26, 2018, 9:45pm

Syslog file is kinda messy, so here's the copy/paste

Jan 18 01:34:04 AP-2119 snmpd[8830]: DFS status: N/A
Jan 18 21:07:37 AP-2119 DEVICE-AGENT[2869]: Accept failed
Jan 18 21:07:37 AP-2119 DEVICE-AGENT[2869]: No data read from unix domain socket, nbytes = -1
Jan 20 15:47:21 AP-2119 DEVICE-AGENT[2869]: Accept failed
Jan 20 15:47:21 AP-2119 DEVICE-AGENT[2869]: No data read from unix domain socket, nbytes = -1
Jan 20 15:47:22 AP-2119 DEVICE-AGENT[2869]: Accept failed
Jan 20 15:47:22 AP-2119 DEVICE-AGENT[2869]: No data read from unix domain socket, nbytes = -1
Jan 21 23:19:09 AP-2119 DEVICE-AGENT[2869]: Accept failed
Jan 21 23:19:09 AP-2119 DEVICE-AGENT[2869]: No data read from unix domain socket, nbytes = -1
Jan 21 23:19:09 AP-2119 DEVICE-AGENT[2869]: Accept failed
Jan 21 23:19:09 AP-2119 DEVICE-AGENT[2869]: No data read from unix domain socket, nbytes = -1
Jan 22 10:11:35 AP-2119 DEVICE-AGENT[2869]: Accept failed
Jan 22 10:11:35 AP-2119 DEVICE-AGENT[2869]: No data read from unix domain socket, nbytes = -1
Jan 22 10:11:35 AP-2119 DEVICE-AGENT[2869]: Accept failed
Jan 22 10:11:35 AP-2119 DEVICE-AGENT[2869]: No data read from unix domain socket, nbytes = -1
Jan 23 13:27:38 AP-2119 DEVICE-AGENT[2869]: Accept failed
Jan 23 13:27:38 AP-2119 DEVICE-AGENT[2869]: No data read from unix domain socket, nbytes = -1
Jan 24 08:24:47 AP-2119 DEVICE-AGENT[2869]: Not received PONG for the last ping (1)
Jan 25 21:49:43 AP-2119 DEVICE-AGENT[2869]: Accept failed
Jan 25 21:49:43 AP-2119 DEVICE-AGENT[2869]: No data read from unix domain socket, nbytes = -1

CWB · January 30, 2018, 2:33am

I am seeing this in one of our AP's as well (2.4ghz synced ePMP 1000)...have the same question. Just noticed it about a week ago.

Dmitry_Moiseev · January 30, 2018, 3:41pm

Hi,

It looks like device is having trouble connecting to cnMaestro. I wonder if your device is onboarded. If it is not you might benifit adding it to cnMaestro or as alst measure disable cnMaestro in settings to avoid this message in syslog.

Thanks,

Dmitry

CWB · January 30, 2018, 3:43pm

Been added to cnMaestro for almost a year now. Shows up fine in cnMaestro.

Cambium_Emilio · January 30, 2018, 5:38pm

Nekomata and CWB,

Approximately how many devices are showing this? For devices that are still generating these messages, what is the current device state in cnMaestro, Online or Offline?

Thansk for provinding the attached device syslogs, our engineering team will review them.

Nekomata · January 30, 2018, 6:54pm

I have found 3 or 4 access points with this message, but all are connected to cnMaestro and have been for at least a week

ashutoshdatta · January 31, 2018, 12:53pm

Hi,

Can you please provide the firmware version running on these devices (probably a screenshot of the device GUI dashboard would help).

can you try to download the tech support from cnMaestro for these devices? It will help us analyse and will also confirm the connectivity of the device to cnMaestro.

From the logs attached, it does not look as if the devices are online in cnMaestro.

Sep 1 00:00:25 AP-2119 DEVICE-AGENT[2869]: Attempting (re)connection in 5 seconds
Sep 1 00:00:51 AP-2119 DEVICE-AGENT[2869]: Non-cambium/cloud cnMaestro URL [cnm.sbbnet.com], disabling Server Cert check
Sep 1 00:00:51 AP-2119 DEVICE-AGENT[2869]: Non-cambium/cloud cnMaestro URL [https://cnm.sbbnet.com/device], disabling Server Cert check
Jan 14 11:15:37 AP-2119 kernel: [ 139.750000] GPS Sync Restored. (19:15:37:313)
Jan 14 12:54:56 AP-2119 DEVICE-AGENT[2869]: Accept failed
Jan 14 12:54:56 AP-2119 DEVICE-AGENT[2869]: No data read from unix domain socket, nbytes = -1
Jan 15 13:55:26 AP-2119 DEVICE-AGENT[2869]: Accept failed
Jan 15 13:55:26 AP-2119 DEVICE-AGENT[2869]: No data read from unix domain socket, nbytes = -1
Jan 16 03:23:22 AP-2119 DEVICE-AGENT[2869]: Accept failed
Jan 16 03:23:22 AP-2119 DEVICE-AGENT[2869]: No data read from unix domain socket, nbytes = -1

Nekomata · January 31, 2018, 9:28pm

All are firmware 3.4.1 as you can see here, we are in the middle of upgrading all of our units to 3.5.1. It's possible that the APs lost connection during the time these errors showed up in the syslog, but it is unsure. Tried to attach tech support files but the forum doesn't seem to like the extension. What email address can I send these to?

ashutoshdatta · February 1, 2018, 6:29am

Hi Nekomata,

Are you seeing these logs coming when the AP is onboarded /online in cnMaestro? This can happen when the connection is down. Please confirm.

What is the impact that you are observing ? Are you getting events for these APs in cnMaestro properly (when online) ?

You can send the tech support dump this to email ashutosh.datta(at)cambiumnetworks.com

thanks

ashutosh

Nekomata · February 1, 2018, 4:49pm

All APs are onboarded to cnMaestro, but so far I have not witnessed this log event at the time of its occurrence, only a few hours after so I can not confirm whether or not it has occurred while any APs were online, I came across it while checking access points that had an irregular number of station drop events, so I was curious as to what the message meant. From what you are saying the message can just indicate loss of communication with cnMaestro / the AP going offline so perhaps there is another issue I should be investigating instead of the message itself. I've sent you the tech support files

ashutoshdatta · February 2, 2018, 5:37am

Hi

Even when looking back in time, you should be able to get this information by checking the timestamp of the log and the "Device is Offline" event in cnMaestro event history. (Assuming that the device and cnMaestro are time synced)

I shall look at the tech support.

Can you please elaborate on "checking access points that had an irregular number of station drop events"

thanks

ashutosh

Nekomata · February 2, 2018, 2:41pm

This is the only access point affected that went offline around the same time as this message started showing up, but you can see that it came back rather quickly (looks like it was due to dfs). I checked the event logs for all other affected APs and none of them have gone offline since it first showed up.

As for the station drops, we've noticed a few of our APs have subs that are dropping and coming back periodically. Some of these look to be due to GPS Sync issues that we are trying to resolve, but some of them have no indication of losing sync (recently at least) and instead have this "No data read from unix domain socket" message. Just trying to figure out what this message is really indicating, and whether or not it may be related to the station disconnects.

ashutoshdatta · February 6, 2018, 7:39am

Hi Nekomata,

thanks for sharing the tech support files. I did go through all of them, and it appears that in all the cases the error message is seen when the device connectivity to cnMaestro is down.

I do not see any direct correlation between this log and the client connectivity issue that you are observing.

thanks

ashutosh

Nekomata · February 6, 2018, 1:31pm

Thank you ashutosh, this well help me better diagnose our AP logs.