NOTICE ePMP SMs reboots started on May 18, 2022

[aka (Andrii)]

The problem statement:	F300s Subscriber Modules reboots started on May 18, 2022.
Affected equipment:	Force 300s SMs
Affected software version:	All Firmware versions older than 4.6.1(e.g. 4.6.0.1, .4.6.0)
Configuration conditions:	cnMaestro Cloud URL is entered on SM’s UI or URL field is blank, DHCP is enabled on SM.
Problem description:	Cambium investigation revealed that a combination of updated number of IP addresses co-related to DNS entries for CnMaestro and DHCP renewals caused crashes on SMs with software release older than 4.6.1. The issue is not seen on releases 4.6.1 and 4.6.2.However, URLs for CnMaestro is hardcoded in ePMP code and when the number of IPs that were resolved as part of the DNS query were increased and a DHCP renewal event occurred, the SM code on the affected firmware versions was unable to process as it provided updates to the AP while executing the SM click through feature. (i.e. Ability to get to SM UI via IP address listed in AP’s monitor wireless screen)
Corrective action:	The most obvious corrective action is to upgrade the network to the latest release of 4.6.2. However, we realize on large networks CnMaestro is a critical part of the upgrade process. In order to use CnMaestro to do the upgrade, we recommend significantly increasing the DHCP lease time so that the SMs are stable enough to take the mass upgrade. In parallel, Cambium Networks is working on a proposal to reduce the number of IP addresses so that the instability does not occur and customers are able to upgrade the network.
Future preventive measures:	We acknowledge the challenge this problem has posed for various networks. While we make an absolute best effort to test and build protection for all scenarios, this was an example of a miss where more than just ePMP or CnMaestro is involved. Our future plans are to ensure more robust testing that includes all aspects of changes to cloud settings while incorporating radio features.

If you have any further questions, please contact Cambium support!

[aka (Andrii)]

UPDATE

We have decreased IP addresses amount in AWS DNS response. Now it is limited to 6.
It should prevent SMs reboots on FW before 4.6.1.
We warmly suggest to update all Cambium equipment to current versions. To utilize it’s stability and new features.

~$ nslookup cloud.cambiumnetworks.com
Server:		172.17.48.13
Address:	172.17.48.13#53

Non-authoritative answer:
cloud.cambiumnetworks.com	canonical name = us-east-1-rtr-1-578252723.us-east-1.elb.amazonaws.com.
Name:	us-east-1-rtr-1-578252723.us-east-1.elb.amazonaws.com
Address: 35.170.225.177
Name:	us-east-1-rtr-1-578252723.us-east-1.elb.amazonaws.com
Address: 3.209.229.179
Name:	us-east-1-rtr-1-578252723.us-east-1.elb.amazonaws.com
Address: 3.91.129.167
Name:	us-east-1-rtr-1-578252723.us-east-1.elb.amazonaws.com
Address: 23.23.97.104
Name:	us-east-1-rtr-1-578252723.us-east-1.elb.amazonaws.com
Address: 54.211.98.197
Name:	us-east-1-rtr-1-578252723.us-east-1.elb.amazonaws.com
Address: 35.174.228.255