New MicroPOP AP dropping all SMs and DL frame utilization flat lining at 72%

New MicroPOP Omni AP 5 GHz Lite running 16.3.2.1-b1 dropping all SMs after running a couple of days flawlessly. SMs have 6x and 8x connections with no re-regs and excellent link tests. Spectrum is clean. On-board GPS locked seeing 20 birds. No other APs in this 5.1 band in range. Frame utilization goes from steady 1-5% to 72% in one step and stays pegged at 72% with SMs no longer connected. CPU utilization doesn’t change at ~5%. No hint in the event logs. Reboot AP and SMs return immediately and run perfectly for days… until the problem repeats. SMs are 450b mid-gain running 16.3.2 and 16.3.2.1-b1. Have ticket 242419 open with Cambium. Sent engineering log files. They suggested changing Rx target to -60 from factory default -52. Didn’t help. Anyone else seeing this? Ideas on how to fix?

I have not heard of this issue yet. Frame Utilization is a clue sounds like.

Would you be willing to try 20.0.1 beta on there?
You’ll have to upgrade both SMs and AP though, otherwise if just AP is upgraded will run at 2X till the SM is upgraded too.

We are planning to release 20.0.1 soon, and very stable as far as I know.

What’s the ticket number? I can try and take a look soon.

Ticket number in OP.
R20 feels like such a big step I had planned to hold off but this is a small site so I guess we could try it. Rather deploy non-beta code so maybe we hold for that? I would also feel better if the problem could be found and fixed on the 16.2 code line as I feel WISPs are going to keep that in production for a while. Not relevant for this issue but please don’t forget the 16.1 code line needs to stay viable for a while longer too! Let me know what I can do to help with any of this.

Same thing happened again. Forth time. We expected this because we have not changed anything besides reboot. Providing this post just to update and hope that someone has seen similar and has ideas for solution. This time DL frame utilization jumps instantly to 66% and flat lines at 66% until reboot. Last time it was 72%. I will add that the AP log does actually show something of interest. Log has 10 identical entries “Dropping Control Packet (44)”. These are the only entries besides normal startup/shutdown. Interestingly the first of these messages start when the SMs all drop and the frame utilization goes to 66%. Then 9 more identical entries over a 90 minute period then no entries for 10 hours until we manually reboot. CPU% 4-5% the entire time. Engineering support files sent to Cambium support under open ticket.

Hey Paul, sorry for the slow response. I’m looking at the logs now and will get back to you soon.