Software release 3.11.2-r2 DNS problem

I updated my e410 cn pilot radio to release 3.11.2-r2 last night.  After the reboot, the radio never came back to cnmaestro.  I have been able to ssh into the radio and I can see that it is on the new 3.11.2-r2 release and that it is accepting clients.  While I am in the radio, via ssh, I am able to ping an external ip address but can not ping anything via name.  The name server is pointing to my local router, as it always has been, and should be fine.  

Obviously the reason it can not talk to cnmaestro is that it can't resolve the name.

Anyone else experiencing this?  Any idea how I can resolve this issue without having to go to a static IP configuration?

Thanks

mg

thanks for doing a round of troubleshooting already to narrow this down. Could you please run 'show ip name-server' from the CLI and send us the output?

Also, is your router reachable through vlan 1 on the AP? If so, on the CLI under 'interface vlan 1' if you add 'ip dhcp request-option-all' (if that line does not exist) does that change the behaviour?

I had this problem after an upgrade and the radio lost it's default gateway. Check that. 


@Ryan Ray wrote:

I had this problem after an upgrade and the radio lost it's default gateway. Check that. 


How was the issue resolved?

Hi Mike,
Please email me tech-support of AP after upgrade. This will help us to know if DNS servers are learnt by device. My email id is shashank.tadakamadla@cambiumnetworks.com

Appreciate your support.


@firefly wrote:

@Ryan Ray wrote:

I had this problem after an upgrade and the radio lost it's default gateway. Check that. 


How was the issue resolved?


Re-adding the default gateway into the Vlan. It had a static entry 


@Ryan Ray wrote:

@firefly wrote:

@Ryan Ray wrote:

I had this problem after an upgrade and the radio lost it's default gateway. Check that. 


How was the issue resolved?


Re-adding the default gateway into the Vlan. It had a static entry 


thanks. so the gateway was static, but that was lost on upgrade? Could you please share your config (either the whole thing with private info such as ssids/passphrases deleted, or just the section relating to interface-vlan) and what was the old software version before upgrade to 3.11.2?

Backwards compatibility and continued connectivity is very important for us to maintain, so it is concerning to see an AP lose connectivity on upgrade and we want to root-cause and fix this asap. Thanks a lot for your help!

thanks....

"show ip name-server" results in just simply a return..... nothing returned.

LowerLevel(config-vlan-1)# ip dhcp request-option-all
LowerLevel(config-vlan-1)# ping google.com
ping: bad address 'google.com'

LowerLevel(config)# ping 4.2.2.1
PING 4.2.2.1 (4.2.2.1): 56 data bytes
64 bytes from 4.2.2.1: seq=0 ttl=56 time=23.304 ms
64 bytes from 4.2.2.1: seq=1 ttl=56 time=23.291 ms

So to answer your question, no the behaviour did not change

More info:


LowerLevel(config)# show ip dhcp-client-info
VLAN-1:
-------
ip=10.10.10.111
siaddr=10.10.10.1
subnet=255.255.255.0
mask=24
router=10.10.10.1
dns=10.10.10.1
hostname=LowerLevel
domain=centralpark.local
broadcast=10.10.10.255
lease=3600
serverid=10.10.10.1
opt53=05
opt58=00000668
LowerLevel(config)#

---------------------------

LowerLevel(config)# show config
!
no management telnet
management ssh
management user admin password (removed by Mike)
no management radius-auth
management http port 80
management http
management cambium-remote
management cambium-remote validate-server-cert
management https
management https port 443
led
lldp
no poe-out
country-code US
wpa2-handshake-retry 4 4
wpa2-handshake-timeout 100 500 1000
wpa2-handshake-log-level 4
placement indoor
!
wireless radio 1
no shutdown
channel auto
channel-width 20
channel-list all-channels
data-rate unicast 1b 2b 5.5b 11b 12 18 24 36 48 54
data-rate non-unicast highest-basic
power auto
no airtime-fairness
antenna-gain 5
beacon-interval 100
off-channel-scan dwell-time 50
auto-rf channel-selection-mode interference
auto-rf chan-hold-time 120
auto-rf interference-diff-threshold 10
auto-rf packet-error-rate-threshold 30
auto-rf channel-utilization-threshold 25
multicast-to-unicast max-stream 40
multicast-to-unicast max-stream-per-client 25
mesh-xtnded-dev-list
wmm-parameters downstream txoplimit vi 3008
wmm-parameters downstream txoplimit vo 1504
wmm-parameters upstream txoplimit vi 3008
wmm-parameters upstream txoplimit vo 1504
!
wireless radio 2
no shutdown
channel auto
channel-width 80
channel-list all-channels
data-rate unicast 6b 9 12b 18 24b 36 48 54
data-rate non-unicast highest-basic
power auto
no airtime-fairness
antenna-gain 5
beacon-interval 100
off-channel-scan dwell-time 50
auto-rf channel-selection-mode interference
auto-rf chan-hold-time 120
auto-rf interference-diff-threshold 10
auto-rf packet-error-rate-threshold 30
auto-rf channel-utilization-threshold 25
multicast-to-unicast max-stream 40
multicast-to-unicast max-stream-per-client 25
mesh-xtnded-dev-list
wmm-parameters downstream txoplimit vi 3008
wmm-parameters downstream txoplimit vo 1504
wmm-parameters upstream txoplimit vi 3008
wmm-parameters upstream txoplimit vo 1504
!
wireless wlan 1
ssid Guest
no shutdown
vlan 10
security wpa2-psk
no protected-mgmt-frames
passphrase (removed by mike)
band both
dtim-interval 1
max-associated-client 127
band-steer normal
mac-authentication policy deny
passpoint interworking internet
passpoint interworking asra
passpoint interworking access-network-type private
no guest-access
no guest-access redirect http-only
!
wireless wlan 2
ssid Private
no shutdown
vlan 1
security wpa2-psk
no protected-mgmt-frames
passphrase (removed by mike)
band both
dtim-interval 1
max-associated-client 127
band-steer normal
mac-authentication policy deny
passpoint interworking internet
passpoint interworking asra
passpoint interworking access-network-type private
no guest-access
no guest-access redirect http-only
!
interface eth 1
switchport mode trunk
switchport trunk native vlan 1
switchport trunk allowed vlan 1,10
!
interface vlan 1
ip dhcp request-option-all
ip address zeroconf
management-access all
ip address dhcp
!

ip route default 10.10.10.1
!
timezone America/Detroit
hostname LowerLevel
snmp-server
ip gw-source-precedence static 1
ip gw-source-precedence dhcpc 2
ip gw-source-precedence pppoe 3
logging syslog 7
!
LowerLevel(config)#

--------------------------------------------------

What else can I send you?

Mike


@Wireless_Mike wrote:

thanks....

"show ip name-server" results in just simply a return..... nothing returned.

LowerLevel(config-vlan-1)# ip dhcp request-option-all
LowerLevel(config-vlan-1)# ping google.com
ping: bad address 'google.com'

LowerLevel(config)# ping 4.2.2.1
PING 4.2.2.1 (4.2.2.1): 56 data bytes
64 bytes from 4.2.2.1: seq=0 ttl=56 time=23.304 ms
64 bytes from 4.2.2.1: seq=1 ttl=56 time=23.291 ms

So to answer your question, no the behaviour did not change

More info:


LowerLevel(config)# show ip dhcp-client-info
VLAN-1:
-------
ip=10.10.10.111
siaddr=10.10.10.1
subnet=255.255.255.0
mask=24
router=10.10.10.1
dns=10.10.10.1
hostname=LowerLevel
domain=centralpark.local
broadcast=10.10.10.255
lease=3600
serverid=10.10.10.1
opt53=05
opt58=00000668
LowerLevel(config)#

---------------------------

LowerLevel(config)# show config
!
no management telnet
management ssh
management user admin password (removed by Mike)
no management radius-auth
management http port 80
management http
management cambium-remote
management cambium-remote validate-server-cert
management https
management https port 443
led
lldp
no poe-out
country-code US
wpa2-handshake-retry 4 4
wpa2-handshake-timeout 100 500 1000
wpa2-handshake-log-level 4
placement indoor
!
wireless radio 1
no shutdown
channel auto
channel-width 20
channel-list all-channels
data-rate unicast 1b 2b 5.5b 11b 12 18 24 36 48 54
data-rate non-unicast highest-basic
power auto
no airtime-fairness
antenna-gain 5
beacon-interval 100
off-channel-scan dwell-time 50
auto-rf channel-selection-mode interference
auto-rf chan-hold-time 120
auto-rf interference-diff-threshold 10
auto-rf packet-error-rate-threshold 30
auto-rf channel-utilization-threshold 25
multicast-to-unicast max-stream 40
multicast-to-unicast max-stream-per-client 25
mesh-xtnded-dev-list
wmm-parameters downstream txoplimit vi 3008
wmm-parameters downstream txoplimit vo 1504
wmm-parameters upstream txoplimit vi 3008
wmm-parameters upstream txoplimit vo 1504
!
wireless radio 2
no shutdown
channel auto
channel-width 80
channel-list all-channels
data-rate unicast 6b 9 12b 18 24b 36 48 54
data-rate non-unicast highest-basic
power auto
no airtime-fairness
antenna-gain 5
beacon-interval 100
off-channel-scan dwell-time 50
auto-rf channel-selection-mode interference
auto-rf chan-hold-time 120
auto-rf interference-diff-threshold 10
auto-rf packet-error-rate-threshold 30
auto-rf channel-utilization-threshold 25
multicast-to-unicast max-stream 40
multicast-to-unicast max-stream-per-client 25
mesh-xtnded-dev-list
wmm-parameters downstream txoplimit vi 3008
wmm-parameters downstream txoplimit vo 1504
wmm-parameters upstream txoplimit vi 3008
wmm-parameters upstream txoplimit vo 1504
!
wireless wlan 1
ssid Guest
no shutdown
vlan 10
security wpa2-psk
no protected-mgmt-frames
passphrase (removed by mike)
band both
dtim-interval 1
max-associated-client 127
band-steer normal
mac-authentication policy deny
passpoint interworking internet
passpoint interworking asra
passpoint interworking access-network-type private
no guest-access
no guest-access redirect http-only
!
wireless wlan 2
ssid Private
no shutdown
vlan 1
security wpa2-psk
no protected-mgmt-frames
passphrase (removed by mike)
band both
dtim-interval 1
max-associated-client 127
band-steer normal
mac-authentication policy deny
passpoint interworking internet
passpoint interworking asra
passpoint interworking access-network-type private
no guest-access
no guest-access redirect http-only
!
interface eth 1
switchport mode trunk
switchport trunk native vlan 1
switchport trunk allowed vlan 1,10
!
interface vlan 1
ip dhcp request-option-all
ip address zeroconf
management-access all
ip address dhcp
!

ip route default 10.10.10.1
!
timezone America/Detroit
hostname LowerLevel
snmp-server
ip gw-source-precedence static 1
ip gw-source-precedence dhcpc 2
ip gw-source-precedence pppoe 3
logging syslog 7
!
LowerLevel(config)#

--------------------------------------------------

What else can I send you?

Mike


Scanning through this the config looks ok, we'll load this on an AP and try it out, to make sure.

After that config change of dhcp-request-option-all, did you also do an 'apply' or 'commit' from the CLI to activate the config? ie:

LowerLevel(config-vlan-1)# ip dhcp request-option-all

<< needs an 'apply' CLI command done here, CLI commands only go live after an apply/commit>>

LowerLevel(config-vlan-1)# ping google.com

Also, what version did you upgrade from? you can check the older version from the CLI with 'show boot', the backup software version will be what you upgraded to 3.11.2 from.

Thanks.

Thanks for the help..... since I never use the CLI I didn't know I had to do the "apply"...... hopefully what I am doing now is helpful.

LowerLevel(config-vlan-1)# ip dhcp request-option-all
LowerLevel(config-vlan-1)# apply
[Apply OK]
LowerLevel(config-vlan-1)# show boot
Active Software Version : 3.11.2-r2
Backup Software Version : 3.5.2-r4

LowerLevel(config-vlan-1)#
LowerLevel(config-vlan-1)# ping www.google.com
ping: bad address 'www.google.com'
LowerLevel(config)# ping 4.2.2.1
PING 4.2.2.1 (4.2.2.1): 56 data bytes
64 bytes from 4.2.2.1: seq=0 ttl=56 time=22.693 ms
64 bytes from 4.2.2.1: seq=1 ttl=56 time=22.902 ms
^C
--- 4.2.2.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 22.693/22.797/22.902 ms
LowerLevel(config)#

Mike,

Could you please send me a private message to shashank.tadakamadla@cambiumnetworks.com.


@firefly wrote:

@Wireless_Mike wrote:

thanks....

"show ip name-server" results in just simply a return..... nothing returned.

LowerLevel(config-vlan-1)# ip dhcp request-option-all
LowerLevel(config-vlan-1)# ping google.com
ping: bad address 'google.com'

LowerLevel(config)# ping 4.2.2.1
PING 4.2.2.1 (4.2.2.1): 56 data bytes
64 bytes from 4.2.2.1: seq=0 ttl=56 time=23.304 ms
64 bytes from 4.2.2.1: seq=1 ttl=56 time=23.291 ms

So to answer your question, no the behaviour did not change

More info:


LowerLevel(config)# show ip dhcp-client-info
VLAN-1:
-------
ip=10.10.10.111
siaddr=10.10.10.1
subnet=255.255.255.0
mask=24
router=10.10.10.1
dns=10.10.10.1
hostname=LowerLevel
domain=centralpark.local
broadcast=10.10.10.255
lease=3600
serverid=10.10.10.1
opt53=05
opt58=00000668
LowerLevel(config)#

---------------------------

LowerLevel(config)# show config
!
no management telnet
management ssh
management user admin password (removed by Mike)
no management radius-auth
management http port 80
management http
management cambium-remote
management cambium-remote validate-server-cert
management https
management https port 443
led
lldp
no poe-out
country-code US
wpa2-handshake-retry 4 4
wpa2-handshake-timeout 100 500 1000
wpa2-handshake-log-level 4
placement indoor
!
wireless radio 1
no shutdown
channel auto
channel-width 20
channel-list all-channels
data-rate unicast 1b 2b 5.5b 11b 12 18 24 36 48 54
data-rate non-unicast highest-basic
power auto
no airtime-fairness
antenna-gain 5
beacon-interval 100
off-channel-scan dwell-time 50
auto-rf channel-selection-mode interference
auto-rf chan-hold-time 120
auto-rf interference-diff-threshold 10
auto-rf packet-error-rate-threshold 30
auto-rf channel-utilization-threshold 25
multicast-to-unicast max-stream 40
multicast-to-unicast max-stream-per-client 25
mesh-xtnded-dev-list
wmm-parameters downstream txoplimit vi 3008
wmm-parameters downstream txoplimit vo 1504
wmm-parameters upstream txoplimit vi 3008
wmm-parameters upstream txoplimit vo 1504
!
wireless radio 2
no shutdown
channel auto
channel-width 80
channel-list all-channels
data-rate unicast 6b 9 12b 18 24b 36 48 54
data-rate non-unicast highest-basic
power auto
no airtime-fairness
antenna-gain 5
beacon-interval 100
off-channel-scan dwell-time 50
auto-rf channel-selection-mode interference
auto-rf chan-hold-time 120
auto-rf interference-diff-threshold 10
auto-rf packet-error-rate-threshold 30
auto-rf channel-utilization-threshold 25
multicast-to-unicast max-stream 40
multicast-to-unicast max-stream-per-client 25
mesh-xtnded-dev-list
wmm-parameters downstream txoplimit vi 3008
wmm-parameters downstream txoplimit vo 1504
wmm-parameters upstream txoplimit vi 3008
wmm-parameters upstream txoplimit vo 1504
!
wireless wlan 1
ssid Guest
no shutdown
vlan 10
security wpa2-psk
no protected-mgmt-frames
passphrase (removed by mike)
band both
dtim-interval 1
max-associated-client 127
band-steer normal
mac-authentication policy deny
passpoint interworking internet
passpoint interworking asra
passpoint interworking access-network-type private
no guest-access
no guest-access redirect http-only
!
wireless wlan 2
ssid Private
no shutdown
vlan 1
security wpa2-psk
no protected-mgmt-frames
passphrase (removed by mike)
band both
dtim-interval 1
max-associated-client 127
band-steer normal
mac-authentication policy deny
passpoint interworking internet
passpoint interworking asra
passpoint interworking access-network-type private
no guest-access
no guest-access redirect http-only
!
interface eth 1
switchport mode trunk
switchport trunk native vlan 1
switchport trunk allowed vlan 1,10
!
interface vlan 1
ip dhcp request-option-all
ip address zeroconf
management-access all
ip address dhcp
!

ip route default 10.10.10.1
!
timezone America/Detroit
hostname LowerLevel
snmp-server
ip gw-source-precedence static 1
ip gw-source-precedence dhcpc 2
ip gw-source-precedence pppoe 3
logging syslog 7
!
LowerLevel(config)#

--------------------------------------------------

What else can I send you?

Mike


Scanning through this the config looks ok, we'll load this on an AP and try it out, to make sure.

After that config change of dhcp-request-option-all, did you also do an 'apply' or 'commit' from the CLI to activate the config? ie:

LowerLevel(config-vlan-1)# ip dhcp request-option-all

<< needs an 'apply' CLI command done here, CLI commands only go live after an apply/commit>>

LowerLevel(config-vlan-1)# ping google.com

Also, what version did you upgrade from? you can check the older version from the CLI with 'show boot', the backup software version will be what you upgraded to 3.11.2 from.

Thanks.


can you please delete the default gateway from config, and have that also learnt from DHCP:

ip route default 10.10.10.1

Looks like we have an issue where the older firmware used to allow static gateway but DNS from DHCP and newer ones dont.

That works!!!!!

Thanks

Mike

------------------------------------------

LowerLevel(config)# ip route default 10.10.10.1
LowerLevel(config)# apply
[Apply OK]
LowerLevel(config)# ping google.com
PING google.com (172.217.9.174): 56 data bytes
64 bytes from 172.217.9.174: seq=0 ttl=52 time=48.208 ms
64 bytes from 172.217.9.174: seq=1 ttl=52 time=47.995 ms
64 bytes from 172.217.9.174: seq=2 ttl=52 time=48.359 ms

--- google.com ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 47.995/48.187/48.359 ms
LowerLevel(config)#
LowerLevel(config)#

Per Cambium's request, I have executed the following and everything is working great.

E410-93F184(config)# no ip route default 

E410-93F184(config)# apply

And after verifying that I was able to ping google.com I performed a "Save"

Perfect.  Thanks team Cambium!

Mike

More info.....

It appears that the gateway static precedence of 1 means that it looks for the gateway as static before looking at dhcp.  When I manually put in a static gateway, even though I'm using DHCP, it worked.  When I deleted the gateway, it also worked.

So I tested this theory by creating a bad gateway and with the gw-source-precedence static set to 1 I could not ping google.com.  As soon as I change the static precedence to 5 (just to make it a higher number than what the dhcp was set to) I was able to ping google.com again.

So it appears to look for the gateway based on this precedence number and by default must be treating static as a higher level than DHCP.... since I was using DHCP the static gateway must have been wrong.

To solve when using dhcp, for now do ONE of the following:

1.  "no ip route default" 

2.  "ip gw-source-precedence static 5"

and you should be fine.....

Mike