Import data from 1.2.1: Error Failed to import data!

Just did an export on my 1.2.1 server and tried to import it into a fresh 1.2.2 install.  Getting this in the operations.log:

2016-11-02 21:28:30,393 - cns.operation - INFO - Starting operation migration
2016-11-02 21:28:30,561 - cns.migration - INFO - ['-o', 'migration', '-a', 'restore', '--account', 'cnmaestro', '--force']
2016-11-02 21:28:30,567 - cns.migration - INFO - Started Restore cnmaestro, False!
2016-11-02 21:28:30,568 - cns.migration - INFO - Check Jobs runnig in soruce db!
2016-11-02 21:28:30,591 - cns.migration - INFO - Check jobs running is source db done in 0 (sec)
2016-11-02 21:28:30,593 - cns.migration - INFO - Started Dump! cnmaestro, True
2016-11-02 21:28:30,593 - cns.migration - INFO - Check Jobs runnig in soruce db!
2016-11-02 21:28:30,595 - cns.migration - INFO - Check jobs running is source db done in 0 (sec)
2016-11-02 21:28:30,596 - cns.migration - INFO - Checking dir mongodump exists
2016-11-02 21:28:30,597 - cns.migration - INFO - Checking dir pgdump exists
2016-11-02 21:28:30,597 - cns.migration - INFO - Checking dir nocFiles exists
2016-11-02 21:28:30,598 - cns.migration - INFO - Dump existing noc files!
2016-11-02 21:28:30,600 - cns.migration - INFO - Executing command 'dpkg -s cnmaestro-router | grep Version'
2016-11-02 21:28:30,648 - cns.migration - INFO - Noc files dump done!
2016-11-02 21:28:30,650 - cns.migration - INFO - Started Posgtres dump!
2016-11-02 21:28:30,650 - cns.migration - INFO - Started Posgtres dump router!
2016-11-02 21:28:30,650 - cns.migration - INFO - Executing command 'PGPASSWORD=postgres pg_dump -d cnmaestro_router -h 127.0.0.1 -p 5432 -U postgres -f /tmp/migration/dumps/pgdump/cnmaestro_router.sql'
2016-11-02 21:28:31,056 - cns.migration - INFO - Started Posgtres dump server!
2016-11-02 21:28:31,058 - cns.migration - INFO - Executing command 'PGPASSWORD=postgres pg_dump -d cnmaestro_server -h 127.0.0.1 -p 5432 -U postgres -f /tmp/migration/dumps/pgdump/cnmaestro_server.sql'
2016-11-02 21:28:31,270 - cns.migration - INFO - Started Posgtres dump controller!
2016-11-02 21:28:31,273 - cns.migration - INFO - Executing command 'PGPASSWORD=postgres pg_dump -d controller -h 127.0.0.1 -p 5432 -U postgres -f /tmp/migration/dumps/pgdump/controller.sql'
2016-11-02 21:28:31,545 - cns.migration - INFO - Skipped dumping mongo collection  cnmaestro.alarm db notifications, as collection not exists
2016-11-02 21:28:31,546 - cns.migration - INFO - Executing command 'mongodump --host 127.0.0.1:27017 --db notifications --collection cnmaestro.event --out /tmp/migration/dumps/mongodump/cnmaestro.event'
2016-11-02 21:28:31,646 - cns.migration - INFO - Successfully dumped mongo collection cnmaestro.event db notifications
2016-11-02 21:28:31,670 - cns.migration - INFO - Executing command 'mongodump --host 127.0.0.1:27017 --db stats --collection cnmaestro.inventory --out /tmp/migration/dumps/mongodump/cnmaestro.inventory'
2016-11-02 21:28:31,785 - cns.migration - INFO - Successfully dumped mongo collection cnmaestro.inventory db stats
2016-11-02 21:28:31,786 - cns.migration - INFO - Skipped dumping mongo collection  cnmaestro.clients db stats, as collection not exists
2016-11-02 21:28:31,788 - cns.migration - INFO - Skipped dumping mongo collection  cnmaestro.clients.failures db stats, as collection not exists
2016-11-02 21:28:31,788 - cns.migration - INFO - Skipped dumping mongo collection  cnmaestro.hrchy.days db stats, as collection not exists
2016-11-02 21:28:31,788 - cns.migration - INFO - Skipped dumping mongo collection  cnmaestro.stats.hours db stats, as collection not exists
2016-11-02 21:28:31,788 - cns.migration - INFO - Skipped dumping mongo collection  cnmaestro.stats.days db stats, as collection not exists
2016-11-02 21:28:31,788 - cns.migration - INFO - Skipped dumping mongo collection  cnmaestro.template db stats, as collection not exists
2016-11-02 21:28:31,789 - cns.migration - INFO - Executing command 'mongodump --host 127.0.0.1:27017 --db stats --collection cnmaestro.upgrade.job --out /tmp/migration/dumps/mongodump/cnmaestro.upgrade.job'
2016-11-02 21:28:31,853 - cns.migration - INFO - Successfully dumped mongo collection cnmaestro.upgrade.job db stats
2016-11-02 21:28:31,855 - cns.migration - INFO - Executing command 'mongodump --host 127.0.0.1:27017 --db stats --collection cnmaestro.config.job --out /tmp/migration/dumps/mongodump/cnmaestro.config.job'
2016-11-02 21:28:31,924 - cns.migration - INFO - Successfully dumped mongo collection cnmaestro.config.job db stats
2016-11-02 21:28:31,925 - cns.migration - INFO - Skipped dumping mongo collection  cnmaestro.edetect.days db stats, as collection not exists
2016-11-02 21:28:32,075 - cns.migration - INFO - Deleted Mongo dump files exists in path /tmp/migration/dumps
2016-11-02 21:28:32,076 - cns.migration - INFO - Publishing json data {'action': 'restore', 'status': 'PROGRESS', 'account': 'cnmaestro', 'value': 25}
2016-11-02 21:28:32,092 - cns.migration - ERROR - Error in extract tar file /tmp/migration/uploads/cnmaestro-import.tar.gz to destination dir /tmp/migration
2016-11-02 21:28:32,094 - cns.migration - INFO - Cleaning restore files and directories!
2016-11-02 21:28:32,094 - cns.migration - ERROR - Failed restore!
2016-11-02 21:28:32,094 - cns.migration - INFO - Publishing json data {'action': 'restore', 'status': 'ERROR', 'account': 'cnmaestro', 'code': 'RESTORE_FAILED', 'cause': 'Failed to import data!'}
2016-11-02 21:28:32,095 - cns.migration - ERROR - Error json data {'action': 'restore', 'status': 'ERROR', 'account': 'cnmaestro', 'code': 'RESTORE_FAILED', 'cause': 'Failed to import data!'} for account cnmaestro
2016-11-02 21:28:32,095 - cns.migration - ERROR - invalid header
2016-11-02 21:28:32,096 - cns.migration - INFO - Closing mongo connection! mongodb://127.0.0.1:27017/notifications
2016-11-02 21:28:32,097 - cns.migration - INFO - Closing mongo connection! mongodb://127.0.0.1:27017/stats
2016-11-02 21:28:32,098 - cns.migration - INFO - Disconnecting redis connection pool 127.0.0.1:6379:5
2016-11-02 21:30:01,806 - cns.operation - INFO - Starting operation sys-monitor
2016-11-02 21:30:01,926 - cns.monitor - INFO - Start System Monitoring
2016-11-02 21:30:02,943 - cns.monitor - INFO - Created the monit data!
2016-11-02 21:30:02,945 - cns.monitor - INFO - Created the monit hours data! <pymongo.results.InsertOneResult object at 0x7fdb600e1640>
2016-11-02 21:30:03,060 - cns.monitor - INFO - Deleted the hour data successfully! match {'ts': {'$lt': 1478032202944.077}} result {u'ok': 1, u'n': 0}
2016-11-02 21:30:03,062 - cns.monitor - INFO - Closing mongo connection! mongodb://127.0.0.1:27017/monitor
2016-11-02 21:30:03,063 - cns.monitor - INFO - System Monitoring done 1 (sec)!

We will debug this issue and get back to you.
Please let us know the steps you did to understand the scenario.

Was able to get our 1.2.1 inventory to 1.2.2 by doing a series of steps.

  1. Export any sort of backup from 1.2.0
  2. Export our current backup from 1.2.1
  3. Extract both files, Overwrite the pgdump folder from 1.2.1 to 1.2.0
  4. Compress the the 1.2.0 folder backup as .tar.gz ( I renamed it similar to the naming convention Ex: cnmaestro-export_cnmaestr....)
  5. Upload and import.

    History stats are missing, but inventory was successfully transferred over.

Hope this helps.

yes, i have found a similar solutions changing same folders inside (i have found >70 mbps of alarms) but i need to upgrade without loose the history...

thanks

Hi,

We have been trying to replicate this issue locally but haven't succeeded so far. It would be really great if you could share your exported file from 1.2.0/1.2.1 version to investigate the issue further and find the exact root cause.

Please email me at ajay.singh@cambiumnetworks.com with the shared location details.

Thanks,

Ajay

file sent.

thanks.

Thanks for sharing the exported data. It helped us in identifying the real issue. We have already started working on the fix and will try to make the patch available early next week.

Thanks,

Ajay

Hi, any update about the patch?

thanks,

Michele Pietravalle

Hi, I have a slightly different problem with IMPORT DATA. It seems to have imported OK but, All devices are off line and unreachable. I have tried deleting an AP then upgrading the AP to 3.1 firmware and restarting the AP which used to automatically show up in the OnBoarding queue but, it doesn't now! It is like either the ePMP device was a flag set that says it has already "onboarded" so it will not try fresh. I am stumped. Thanks for any help and happy Thanksgiving!

Hi David,

Once the import is done successfully , the devices will take 5 minutes to show online in the server again. Since after the import the server gets restarted all the device web connections will be closed and the device retries to connect to cnMaestro after 5 minutes. If it is not connecting after 5 minutes also please send the server tech support dump as well as the device agent logs of the device while it tries to connect to the server to the mail id

krishna.reddum@cambiumnetworks.com

Also please go through the following link for debugging the issue

http://community.cambiumnetworks.com/t5/cnMaestro/Troubleshooting-cnMaestro-on-premises-device-on-boarding-issue/td-p/58439

All,

Please apply the below patch before importing the data into On-Premises VM running with version - 1.2.2-r3 from older versions – 1.2.0/1.2.1:

https://s3.amazonaws.com/cnmaestro-on-premises/packages/cnmaestro-_package_v1.2.3-r1_XhdhEW9nkQ.tar.gz

Here are the steps to apply the patch:

  1. Go to Manage -> Server from top level menu.
  2. Navigate to Operations tab.
  3. On operations tab, go to “Apply cnMaestro Package” section.
  4. Select the patch file and hit the “Apply Update” button.

Please try out the patch update and let us know if it worked successfully. We will be making another On-Premises release early next week with other significant improvements including this fix.

Thanks,

Ajay

For me this patch works

Thank you

Great thank you for the confirmation!

What is the eta on the next update?  We want to patch but if there is an update I'd rather not go through the process 2x. 

We are targeting to release it by coming Friday(02-Dec).

-Ajay

1 Like