ePMP Release 3.3 is now available.

Products ePMP
41

We have 3.3 on all EPMP force 200 and 180 units. We dont usually monitor client CPE but when we started to get a lot of calls we investigated and saw that logging into the carious CPE they uptime was from 1 min to a few hours then they would reboot.

You have to know that when a CPE or AP reboots the LOG files get wiped this is a known issue I have brought up MANY MANY MANY times when you guys ask us to check log files. Unless you are using a log file server to remotely log whats going on just before it reboots your log file on the CPE is useless! They reboot in Flex or  TDD mode we have both. We did not upgrade all our AP's until we figured out what was going on.

Please offer up a better solution to diagnose other than to check a log file that is not wiped away on a reboot.


Thanks

42

This is interesting. I’ve been seeing SMs reboot as well under 3.3. Figured it was customers moving stuff around. Then I had 2 different APs reboot randomly. Thought it was odd but couldn’t find a cause. Until I read this…

43

Crash log is not the System Log.
Crash log is always saved after cras happened.
You can check it in CLI using the command - “debug crashlog”.
This information is really important for reboot reason identification.

Thank you.

44

This guys radio rebooted several hours ago..

Matt Parsons>debug crashlog
Crashlog not found

45

I finally found a radio that the crash log worked..The last radio matt parsons rebooted but no crash log.. This customers JUST rebooted 10 min ago so I am wondering after a certain time the crash log gets erased? I have tried aprox 15 different radios now that have rebooted for no reason and show an uptime from min to hours and there is NO crashlog.


  ********************************************
  *             ePMP command line interface             *
  ********************************************

Kevin Armentano>debug crashlog
Time: 1441074614.804698
Modules:        tdd_umac@c0cf1000+12381c        tdd_link_test@c0b45000+100b     tdd_dev@c0af3000+3d929      tdd_spectral@c0a87000+9d13      tdd_dfs@c093b000+131174 tdd_hal@c086d000+a35d8  tdd_acm@c07dc000+1e06       tdd_unblockSA@c07cd000+2b5      tdd_pps@c07bf000+2b7f   gpslic@c07ae000+46f     tdd_cdf@c07a1000+111c       tdd_adf@c0792000+26c5   tdd_asf@c0782000+1b93   athrs_gmac@c0739000+ca58   tdd_netlink_socket@c0456000+20b  macvlan@c0449000+1140   nf_conntrack_netlink@c0439000+2820      ebt_ulog@c0428000+1080      ebt_ip@c041b000+520     ebt_arp@c040f000+650    ebt_vlan@c0403000+650   ebt_pkttype@c03f6000+260    ebt_limit@c03ea000+400  ebt_802_3@c03de000+2f0  ebtable_nat@c03cc000+3a0   ebtable_filter@c03ba000+3b0      ebtable_broute@c03a8000+300     ebtables@c0399000+3ac5  nfnetlink@c0388000+7df      nf_conntrack_tftp@c036f000+9b0  xt_HL@c0355000+560      xt_hl@c0349000+3e0      ipt_ECN@c033d000+580        xt_CLASSIFY@c0331000+240        xt_tcpmss@c0325000+430  xt_statistic@c0319000+380   xt_DSCP@c030d000+5d0    xt_dscp@c0301000+440    xt_quota@c02f5000+340   xt_pkttype@c02e9000+2a0     xt_physdev@c02dd000+590 xt_owner@c02d1000+300   ipt_REDIRECT@c02c5000+2f0       ipt_NETMAP@c02b9000+2f0     ipt_MASQUERADE@c02ad000+440     iptable_nat@c02a1000+9b8        nf_nat@c0292000+2dbxt_CONNMARK@c0282000+360 xt_recent@c0275000+1600 xt_helper@c0266000+390  xt_conntrack@c025a000+830  xt_connmark@c024d000+2e0 xt_connbytes@c0241000+530       xt_NOTRACK@c0235000+270 iptable_raw@c0229000+2e0    xt_state@c021d000+350   nf_conntrack_ipv4@c0210000+1fb2 nf_defrag_ipv4@c0200000+306     nf_conntrack@c01e9000+a219  pppoe@c01cc000+20c0     pppox@c01bd000+59a      ipt_REJECT@c01b0000+710 xt_TCPMSS@c01a3000+a50      ipt_LOG@c0195000+1110   xt_multiport@c0187000+750       xt_mac@c017b000+2a0xt_limit@c016f000+440    iptable_mangle@c0163000+430     iptable_filter@c0156000+350     ip_tables@c0147000+2255     xt_tcpudp@c0138000+730  x_tables@c012a000+278e  ppp_async@c0119000+1960 ppp_generic@c0106000+4b55   slhc@c00f1000+112b      ts_fsm@c00e4000+a80     ts_bm@c00d7000+600      ts_kmp@c00cb000+590 crc_ccitt@c00bb000+42b  cambium_iprst@c00a8000+d00      leds_gpio@c0097000+5c0  button_hotplug@c008a000+a70 gpio_buttons@c007d000+880       input_polldev@c0070000+643      input_core@c005e000+469e
0000 @ 00000000 (usable)
<6>Initrd not found or empty - disabling initrd
<4>Zone PFN ranges:
<4>  Normal   0x00000000 -> 0x00004000
<4>Movable zone start PFN for each node
<4>early_node_map[1] active PFN ranges
<4>    0: 0x00000000 -> 0x00004000
<7>On node 0 totalpages: 16384
<7>free_area_init_node: node 0, pgdat 802d2dc0, node_mem_map 81b6a000
<7>  Normal zone: 128 pages used for memmap
<7>  Normal zone: 0 pages reserved
<7>  Normal zone: 16256 pages, LIFO batch:3
<4>Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 16256
<5>Kernel command line: console=ttyS0,115200 root=31:02 noinitrd mtdparts=ath-nor0:256k(u-boot),64k(u-boot-env),384K(config),7360K(uImage),64K(uImageI),64k(ART) mem=64M
<6>PID hash table entries: 256 (order: -2, 1024 bytes)
<6>Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
<6>Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
<4>Primary instruction cache 64kB, VIPT, 4-way, linesize 32 bytes.
<4>Primary data cache 32kB, 4-way, VIPT, cache aliases, linesize 32 bytes
<6>Writing ErrCtl register=00000000
<6>Readback ErrCtl register=00000000
<6>Memory: 36812k/65536k available (2008k kernel code, 28652k reserved, 501k data, 25060k init, 0k highmem)
<6>SLUB: Genslabs=13, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
<6>Hierarchical RCU implementation.
<6>NR_IRQS:128
<4>plat_time_init: plat time init done
<6>Calibrating delay loop... 279.34 BogoMIPS (lpj=1396736)
<4>Mount-cache hash table entries: 512
<6>NET: Registered protocol family 16
<4>bio: create slab <bio-0> at 0
<6>Switching to clocksource MIPS
<6>NET: Registered protocol family 2
<6>IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
<6>TCP established hash table entries: 2048 (order: 2, 16384 bytes)
<6>TCP bind hash table entries: 2048 (order: 1, 8192 bytes)
<6>TCP: Hash tables configured (established 2048 bind 2048)
<6>TCP reno registered
<6>NET: Registered protocol family 1
<4>ATH GPIOC major 0
<6>squashfs: version 4.0 (2009/01/31) Phillip Lougher
<4>Registering mini_fo version $Id$
<6>JFFS2 version 2.2 (NAND) (SUMMARY)  (ZLIB) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.
<4>yaffs Mar 31 2017 17:35:10 Installing.
<6>msgmni has been set to 72
<6>io scheduler noop registered
<6>io scheduler deadline registered (default)
<6>Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled
<6>serial8250.0: ttyS0 at MMIO 0xb8020000 (irq = 19) is a 16550A
<6>console [ttyS0] enabled
<4>Flash Manuf Id 0xef, DeviceId0 0x40, DeviceId1 0x17
<4>8MB Flash Detected
<5>6 cmdlinepart partitions found on MTD device ath-nor0
<5>Creating 6 MTD partitions on "ath-nor0":
<5>0x000000000000-0x000000040000 : "u-boot"
<5>0x000000040000-0x000000050000 : "u-boot-env"
<5>0x000000050000-0x0000000b0000 : "config"
<5>0x0000000b0000-0x0000007e0000 : "uImage"
<5>0x0000007e0000-0x0000007f0000 : "uImageI"
<5>0x0000007f0000-0x000000800000 : "ART"
<4>*** ar8216_init ***
<6>TCP westwood registered
<6>NET: Registered protocol family 17
<5>Bridge firewalling registered
<6>802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
<6>All bugs added by David S. Miller <davem@redhat.com>
<4>athwdt_init: Registering WDT success
<4>ath_otp_init: Registering OTP success
<4>ath_clksw_init: Registering Clock Switch Interface success
<4>apPx_spi_init: initialize Cambium New SPI Driver....
<4>Board SKU .... 0xc
<4>Boot loader SKU<FF> Linux Board SKU<0C> Using <0xC> as board SKU
<4>Entering ath_hw_config_init()
<4>GPIO 14 and 15 set for XLNA0 and XLNA1
<4>Initialize GPIO PINs
<4>apPx_rst_button_init: platform_device_regsiter: avenger_keys_device  status = 0
<6>Freeing unused kernel memory: 25060k freed
<6>Button Hotplug driver version 0.3.1
<4>
<4>gpio_keys_probe: Initialize RST_BUTTON Driver....GPIO(16)
<4>gpio_keys_probe:gpio_keys_isr registered to IRQ = 48
<6>input: gpio-keys as /devices/platform/gpio-keys/input/input0
<4>**** drop_caches_sysctl_handler: all done timer added ...****
<6>PPP generic driver version 2.4.2
<6>ip_tables: (C) 2000-2006 Netfilter Core Team
<6>NET: Registered protocol family 24
<4>nf_conntrack version 0.5.0 (967 buckets, 3868 max)
<4>CONFIG_NF_CT_ACCT is deprecated and will be removed soon. Please use
<4>nf_conntrack.acct=1 kernel parameter, acct=1 nf_conntrack module option or
<4>sysctl net.netfilter.nf_conntrack_acct=1 to enable it.
<4>nf_nat_ftp: Unknown symbol nf_nat_ftp_hook
<4>Netfilter messages via NETLINK v0.30.
<6>Ebtables v2.0 registered
<4>ctnetlink v0.93: registering with nfnetlink.
<4>tdd_netlink_socket: module license 'Proprietary' taints kernel.
<4>Disabling lock debugging due to kernel taint
<4>ATHR_GMAC: Length per segment 1722
<4>ATHR_GMAC: fifo cfg 3 01f00140
<4>ATHR_GMAC: mac number:0
<4>ATHR_GMAC: RX TASKLET - Pkts per Intr:16
<4>ATHR_GMAC: Mac address for unit 0:bfff0000
<4>ATHR_GMAC: 00:04:56:fc:e3:50
<4>ATHR_GMAC: Max segments per packet :   1
<4>ATHR_GMAC: Max tx descriptor count :   128
<4>ATHR_GMAC: Max rx descriptor count :   128
<4>ATHR_GMAC: Mac capability flags    :   102381
<4>athr_gmac_ring_alloc Allocated 2048 at 0x837c4000
<4>athr_gmac_ring_alloc Allocated 2048 at 0x837c4800
<4>WASP  ----> AR803x (AR8035) PHY
<4>Setting Drop CRC Errors, Pause Frames and Length Error frames
<6>camb_mii_bus: probed
<4>mac:0 Registering AR803x...
<4>Setting PHY...0
<4>AR803x_PHY phy_reg_read ID1:(0x004d) 0
<4>AR803x_PHY AR8035 Detected
<4>Wait for Autoneg to complete
<4>AR803x_PHY: Port 0, Neg Success
<4>AR803x_PHY: unit 0 phy addr 0
<4>checking for the az feature of 803x...
<4>Disabling 802.3az feature...
<4>Restart auto-negotiation
<4>TXFCTL enabled in Mac:0
<6>device eth0 entered promiscuous mode
<6>tdd_hal: 0.9.17.1 (AR9380, DEBUG, WRITE_EEPROM, 11D)
<6>tdd_dfs: Version 2.0.0
<6>Copyright (c) 2005-2006 Atheros Communications, Inc. All Rights Reserved
<6>tdd_spectral: Version 2.0.0
<6>Copyright (c) 2005-2009 Atheros Communications, Inc. All Rights Reserved
<6>SPECTRAL module built on Mar 31 2017 17:38:19
<6>tdd_dev: Copyright (c) 2001-2007 Atheros Communications, Inc, All Rights Reserved
<6>ath_ahb: 9.2.0_U10.5.13 (Atheros/multi-bss)
<4>AH_CAL_IN_FLASH_AHB defined
<4>__ath_attach: Set global_scn[0]
<4>TxBuf flow control is disabled
<4>hal_conf_parm.calInFlash 1
<6>Bootstrap clock 40MHz
<6>Wasp Enterprise mode: 0x00000000
<6>ar9300RadioAttach: Need analog access recipe!!
<6>Restoring Cal data from Flash
<6>Restoring Cal data from Second Radio in Flash
<4>Don't allow 5.9 channels: cal peer[7]=5825
<7>dfs_attach: use DFS enhancements
<4>Use default DFS regisers
<7>DFS min filter rssiThresh = 11
<7>DFS max pulse dur = 201 ticks
<4>ath_get_caps[6034] rx chainmask mismatch actual 3 sc_chainmak 0
<4>ath_get_caps[6009] tx chainmask mismatch actual 3 sc_chainmak 0
<4>tdd_classifier_init: ic=835e02c0 classifier=835f754c
<4>ath_descdma_setup: tx DMA: 1024 buffers 1 desc/buf 128 desc_len
<4>ath_descdma_setup: tx DMA map: 81000000 (135168) -> 1000000 (135168)
<4>ath_descdma_setup: success, name = tx, nbuf = 1024
<4>ath_descdma_setup: beacon DMA: 8 buffers 1 desc/buf 128 desc_len
<4>ath_descdma_setup: beacon DMA map: a110f000 (4096) -> 110f000 (4096)
<4>ath_descdma_setup: success, name = beacon, nbuf = 8
<4>ath_rx_edma_init: cachelsz 32 rxbufsize 1816
<6>wifi0: Atheros 9340: mem=0xb8100000, irq=2
<4>ATH_MAC_TIMER: enet unit:0 is up...
<4>Mii 100Mbps full duplex
<4>ATH_MAC_TIMER: done cfg2 0x7115 ifctl 0x10000 miictrl
<6>br-lan: port 1(eth0) entering forwarding state
<4>TXFCTL enabled in Mac:0
<4>wlan_vap_create : enter. devhandle=0x835e02c0, opmode=IEEE80211_M_STA, flags=0x1
<4>latency control is ON, latency value is 150000000
<4>wlan_vap_create : exit. devhandle=0x835e02c0, opmode=IEEE80211_M_STA, flags=0x1.
<4>SWB: CAC time = 67000 MILLIseconds
<4>SWB: NOP time = 1800000 MILLIseconds
<4>50, 30, 20
<6>device ath0 entered promiscuous mode
<6>br-lan: port 2(ath0) entering forwarding state
<4>ieee80211_ioctl_getparam : parameter drv:tdd 0x284 not supported
<4>DEBUG: config value 0
<4>Wait for Autoneg to complete
<4>AR803x_PHY: Port 0, Neg Success
<4>AR803x_PHY: unit 0 phy addr 0
<4>ATH_MAC_TIMER: unit 0: phy 0 not up carrier 1
<6>br-lan: port 1(eth0) entering disabled state
<4>ATH_MAC_TIMER: enet unit:0 is up...
<4>Mii 100Mbps full duplex
<4>ATH_MAC_TIMER: done cfg2 0x7115 ifctl 0x10000 miictrl
<6>br-lan: port 1(eth0) entering forwarding state
<4>TPC set initial Tx-Power to 29dbm
<4>[   41.150000] Flags 0, Scheduler Mode 0
<4>[   41.170000] connection is UP
<4>[   41.180000] tdd_ratectrl_stat_add: suspicious ARQ statistic, ARQ fn = 33557, RA last FN = 0
<4>[   41.210000] SM associated with AP[00:04:56:d2:0e:42]
<4>[   42.350000] Adding WDS entry for 00:04:56:fc:e3:51, through ni=00:04:56:d2:0e:42
<1>[ 9019.240000] CPU 0 Unable to handle kernel paging request at virtual address c38ee0b4, epc == c38ee0b4, ra == c0d6e2cc
<4>[ 9019.250000] Oops[#1]:
<4>[ 9019.250000] Cpu 0
<4>[ 9019.250000] $ 0   : 00000000 00000001 0001c908 00e48400
<4>[ 9019.250000] $ 4   : 005c006c 8113f870 00000000 00000007
<4>[ 9019.250000] $ 8   : 812709c6 8017ba18 00000718 00000000
<4>[ 9019.250000] $12   : 00000000 00000000 00000000 83781b40
<4>[ 9019.250000] $16   : 81270000 8113f87a 00000004 810fd1a0
<4>[ 9019.250000] $20   : 83580000 812709c0 810fd180 810fd180
<4>[ 9019.250000] $24   : 00000000 8017b370
<4>[ 9019.250000] $28   : 83780000 83781978 835e02c0 c0d6e2cc
<4>[ 9019.250000] Hi    : 00000000
<4>[ 9019.250000] Lo    : 041bd902
<4>[ 9019.250000] epc   : c38ee0b4 0xc38ee0b4
<4>[ 9019.250000]     Tainted: P
<4>[ 9019.250000] ra    : c0d6e2cc ath_net80211_rx+0x11c/0xbb0 [tdd_umac]
<4>[ 9019.250000] Status: 1100ff03    KERNEL EXL IE
<4>[ 9019.250000] Cause : 00800008
<4>[ 9019.250000] BadVA : c38ee0b4
<4>[ 9019.250000] PrId  : 0001974c (MIPS 74Kc)
<4>[ 9019.250000] Modules linked in: tdd_umac tdd_link_test(P) tdd_dev(P) tdd_spectral(P) tdd_dfs(P) tdd_hal(P) tdd_acm(P) tdd_unblockSA(P) tdd_pps(P) gpslic tdd_cdf tdd_adf tdd_asf(P) athrs_gmac tdd_netlink_socket(P) macvlan nf_conntrack_netlink ebt_ulog ebt_ip ebt_arp ebt_vlan ebt_pkttype ebt_limit ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables nfnetlink nf_conntrack_tftp xt_HL xt_hl ipt_ECN xt_CLASSIFY xt_tcpmss xt_statistic xt_DSCP xt_dscp xt_quota xt_pkttype xt_physdev xt_owner ipt_REDIRECT ipt_NETMAP ipt_MASQUERADE iptable_nat nf_nat xt_CONNMARK xt_recent xt_helper xt_conntrack xt_connmark xt_connbytes xt_NOTRACK iptable_raw xt_state nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack pppoe pppox ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables ppp_async ppp_generic slhc ts_fsm ts_bm ts_kmp crc_ccitt cambium_iprst leds_gpio button_hotplug gpio_buttons input_polldev input_core [last unloaded: nf_nat_tftp]
<4>[ 9019.250000] Process sh (pid: 22719, threadinfo=83780000, task=8082e440, tls=00000000)
<4>[ 9019.250000] Stack : 0000b33d c0d140b8 00000001 00000081 83781b40 83781990 00000001 00000020
<4>[ 9019.250000]         00000800 00000000 8365b920 8365b900 000000e0 800dab24 0024e781 30339f7e
<4>[ 9019.250000]         f7f00000 005e0000 802c0a6c 00000131 81100000 801d0fd4 00000780 802c06c0
<4>[ 9019.250000]         00000020 801cfe74 835f3714 835f3838 83580000 810c6070 835f3700 835e0000
<4>[ 9019.250000]         00000737 00000020 00000000 00000718 83781b40 801d0fd4 83580000 c0b21c98
<4>[ 9019.250000]         ...
<4>[ 9019.250000] Call Trace:
<4>[ 9019.250000] [<c0d140b8>] tdd_recv_gpf+0x4b4/0x1a00 [tdd_umac]
<4>[ 9019.250000] [<800dab24>] __kmalloc_track_caller+0xd8/0x138
<4>[ 9019.250000] [<801d0fd4>] dev_alloc_skb+0x1c/0x4c
<4>[ 9019.250000] [<801cfe74>] __alloc_skb+0x7c/0x150
<4>[ 9019.250000] [<801d0fd4>] dev_alloc_skb+0x1c/0x4c
<4>[ 9019.250000] [<c0b21c98>] ath_rxbuf_alloc+0x48/0x170 [tdd_dev]
<4>[ 9019.250000] [<c07924d4>] __adf_nbuf_alloc+0x38/0xd4 [tdd_adf]
<4>[ 9019.250000] [<c086d420>] ath_hal_reg_read+0x30/0x44 [tdd_hal]
<4>[ 9019.250000] [<c0b21c98>] ath_rxbuf_alloc+0x48/0x170 [tdd_dev]
<4>[ 9019.250000] [<c0b20760>] ath_rx_indicate+0xf4/0x128 [tdd_dev]
<4>[ 9019.250000] [<c0875778>] ar9300ChangeRangeTo_95dbm+0x0/0x50 [tdd_hal]
<4>[ 9019.250000] [<c0875778>] ar9300ChangeRangeTo_95dbm+0x0/0x50 [tdd_hal]
<4>[ 9019.250000] [<c0b07834>] ath_rx_process+0x270/0x948 [tdd_dev]
<4>[ 9019.250000] [<c086d420>] ath_hal_reg_read+0x30/0x44 [tdd_hal]
<4>[ 9019.250000] [<c0b16da0>] ath_rx_edma_requeue+0x474/0xaf4 [tdd_dev]
<4>[ 9019.250000] [<c0daba70>] ieee80211_notify_queue_status+0xc4/0x108 [tdd_umac]
<4>[ 9019.250000] [<c086d46c>] ath_hal_reg_write+0x38/0x5c [tdd_hal]
<4>[ 9019.250000] [<c0b17460>] ath_rx_edma_tasklet+0x40/0x2b8 [tdd_dev]
<4>[ 9019.250000] [<c0afce44>] ath_tdd_scheduling_start_timer_handler+0x1bc/0x179c [tdd_dev]
<4>[ 9019.250000] [<c0b20000>] ath_green_ap_state_mc+0x310/0x44c [tdd_dev]
<4>[ 9019.250000] [<c0afdc58>] ath_tdd_scheduling_start_timer_handler+0xfd0/0x179c [tdd_dev]
<4>[ 9019.250000] [<c0afcd4c>] ath_tdd_scheduling_start_timer_handler+0xc4/0x179c [tdd_dev]
<4>[ 9019.250000] [<c0b20000>] ath_green_ap_state_mc+0x310/0x44c [tdd_dev]
<4>[ 9019.250000] [<c0db01a4>] ath_isr_generic+0x110/0x14c [tdd_umac]
<4>[ 9019.250000] [<800857e0>] tasklet_action+0x88/0xdc
<4>[ 9019.250000] [<800ad72c>] handle_IRQ_event+0x84/0x16c
<4>[ 9019.250000] [<800b8834>] get_page_from_freelist+0x154/0x46c
<4>[ 9019.250000] [<80085fc4>] __do_softirq+0xb0/0x148
<4>[ 9019.250000] [<800860a4>] do_softirq+0x48/0x6c
<4>[ 9019.250000] [<8006baac>] ret_from_irq+0x0/0x4
<4>[ 9019.250000] [<8007629c>] copy_user_highpage+0x84/0x130
<4>[ 9019.250000] [<800c94fc>] __pte_alloc+0x3c/0x17c
<4>[ 9019.250000] [<800c81c8>] do_wp_page+0x568/0x87c
<4>[ 9019.250000] [<800c9d04>] handle_mm_fault+0x6c8/0x7a4
<4>[ 9019.250000] [<8007ece0>] dup_mm+0x2ec/0x3c8
<4>[ 9019.250000] [<80075bb0>] do_page_fault+0x100/0x2f0
<4>[ 9019.250000] [<8007a598>] activate_task+0x70/0x98
<4>[ 9019.250000] [<8007fd58>] do_fork+0x280/0x304
<4>[ 9019.250000] [<800e18f4>] sys_stat64+0x34/0x48
<4>[ 9019.250000] [<800db460>] filp_close+0x68/0x8c
<4>[ 9019.250000] [<8006baa0>] ret_from_exception+0x0/0xc
<4>[ 9019.250000] [<800db484>] sys_close+0x0/0xe4
<4>[ 9019.250000]
<4>[ 9019.250000]
<4>[ 9019.250000] Code: (Bad address in epc)
<4>[ 9019.250000]
<0>[ 9019.720000] Kernel panic - not syncing: Fatal exception in interrupt
Kevin Armentano>

46

Thank you for provided crash log.

We will analyze it asap.

Crash log saved after crash will be erased after next device reboot.

I will contact you at your e-mail address on this issue.

Thank you.

47

+1 on Force 180 SMs randomly rebooting on v3.3… (hadn’t updated APs yet) Looked like a pattern yesterday, decided today that I’d rollback to 3.2.2 late tonight. Now I’m leery of that. I assume reports of bricked radios are due to a random reboot occuring during flash, not sure how to reduce that chance. (maybe manual reboot right before?) Throughput monitoring may show the reboots (external via snmp) since the counters all reset - ours shows that the reset interval has no discernible pattern.

j

48

Thanks  you so much

----------------------------------------------------------------

lam bang dai hoc - lam bang dai hoc gia re - lam bang dai hoc uy tin - lam bang cap 3

49

+1 We are seeing the same on Force 200's radios constantly rebooting. We have about 10 or so. 

Have not seen this happen on an AP yet 

50
@Mario606 wrote:

+1 We are seeing the same on Force 200's radios constantly rebooting. We have about 10 or so. 

Have not seen this happen on an AP yet 

Hi Mario,

Could you please contact me at fedor.trutsko(at)cambiumnetworks(dot)com on this issue.

Thank you.

51

Correction - it does look like the problem is limited to Force200 radios, not Force180 as I previously stated.  We had a few brought back to the shop after techs replaced them on service calls.  Two nights ago I downgraded 520 SMs from 3.3 back to 3.2.2 and we were able to close about 20 open service tickets that were rooted in the reboots.

j

52

We see it on both tpes of radios. We have not confirmed it yet on the older force 110 radios? but for sure on both 180 and 200 units.

has anyone been able to "unbrick" a force 200 radio that possibly failed to load firmware and rebooted during the downgrade. Would a TFTP client possibly work? I have one here at my desk I am going to play around with and see what it is doing. Wish it had a monitor output heheheh..

53
@accelwireless wrote:

We see it on both tpes of radios. We have not confirmed it yet on the older force 110 radios? but for sure on both 180 and 200 units.

has anyone been able to "unbrick" a force 200 radio that possibly failed to load firmware and rebooted during the downgrade. Would a TFTP client possibly work? I have one here at my desk I am going to play around with and see what it is doing. Wish it had a monitor output heheheh..

HI,

TFTP recovery works properly starting 2.6.2 Release.

Please find link to guide how to perform it.

http://community.cambiumnetworks.com/t5/ePMP-Troubleshooting/Recovery-of-flash-corrupted-ePMP-devices-caused-due-to-power/m-p/54712

Thank you.

54

Thats great but it talks abotu extracting the BIN file. When I use my "extraction software" to find the .bin file in the tar.gz file all it finds is another .tar file? where do we locate the .bin file???? or are you simply renaming the .tar file to .bin?

55
@accelwireless wrote:

Thats great but it talks abotu extracting the BIN file. When I use my "extraction software" to find the .bin file in the tar.gz file all it finds is another .tar file? where do we locate the .bin file???? or are you simply renaming the .tar file to .bin?

Hi, 

You have to untar the .tar file which will then list all the files. One of the files will be the firmware.bin file. 

Thanks,

Sriram

1 Like
56

Okay extract tar.gz then .tar.

I found it and it worked I was able to recover this unit so we can put it back into production.

Thank you this is good info to know when we come across other ones if it happens again.

1 Like
57

Anyone else seeing issues on 3.3? We have been running a few force 200 and force 180 on it and so far no reboots but I dont want to keep putting it out in the wild if its going to cause customers issues.

Right now the installers are still getting SM's with it on them.

Cambium should we role back to 3.2?

58
@Brandon Yuchasz wrote:

Anyone else seeing issues on 3.3? We have been running a few force 200 and force 180 on it and so far no reboots but I dont want to keep putting it out in the wild if its going to cause customers issues.

Right now the installers are still getting SM's with it on them.

Cambium should we role back to 3.2?

Hi Brandon,

For now we didn't observe reboot issue with 3.3 Release.

Several cases were solved with hardware replacement and hardware issue has been confirmed.

Thank you.

59

We saw probably 15-20 clients impacted by this. (out of about 500 using ePMP) The first several we rolled trucks and replaced the radios, but for the remainder their troubles were solved by downgrading to 3.2.2.

So you're saying those are all hardware problems that were exposed by 3.3?

j

60
@newkirk wrote:

We saw probably 15-20 clients impacted by this. (out of about 500 using ePMP) The first several we rolled trucks and replaced the radios, but for the remainder their troubles were solved by downgrading to 3.2.2.

So you're saying those are all hardware problems that were exposed by 3.3?

j

Hi Joel,

We need to investigate your perticular case to make any conclusions.

I will contact you via e-mail.

Thank you.