Operation External Restore failed.

[Eclipse]

This is what I got for notifications when trying to import, exported email accounts, here is the error log

This is an automated notification from ZxBackup about External Restore.

Operation External Restore failed.

- exception -
java.lang.NullPointerException
at com.zimbra.cs.account.DataSource.encryptData(DataS ource.java:455)
at com.zextras.backup.ZEBAccount.restoreDataSources(U nknown Source)
at com.zextras.backup.ZEExternalRestoreOperation.doOp eration(Unknown Source)
at com.zextras.op.ZEOperation.exec(Unknown Source)
at com.zextras.backup.ZEBackupThread.execOperation(Un known Source)
at com.zextras.backup.ZEBackupThread.run(Unknown Source)

- debug info -
-- parameters --
humanDate = last available
OpStartTime = 1314523085063
notificationeMails =
accountNameList = private email addresses - ********@******-******.com
backupPath = /test/export
date = null
domainNameList = ******-******.com
-- context --
zContext = com.zimbra.cs.mailbox.OperationContext@3e7403c9
numItemsRestored = 105
extMap = com.zextras.backup.ZEBackupExtMap@32a4a127
lastAccountId = a9dd3000-1f0d-49f7-bbfd-a3b81642b9dd

I commented out the email addresses and domain names with asterisks....

Anyone know why a simple export and import will not work with my Zimbra 7.1.1?

When your asking for $8.60 per mailbox I would expect better support and a fewer problems before considering to even make a purchase....

Anyone else ever had this problem while importing?

[ekpaxp]

Hi, I have also encountered the same error when importing a backup whilst testing a migration scenario.

This is an automated notification from ZxBackup about External Restore.
Operation External Restore failed.
- exception -
java.lang.NullPointerException
at com.zimbra.cs.account.DataSource.encryptData(DataS ource.java:455)
at com.zextras.backup.ZEBAccount.restoreDataSources(U nknown Source)
at com.zextras.backup.ZEExternalRestoreOperation.doOp eration(Unknown Source)
at com.zextras.op.ZEOperation.exec(Unknown Source)
at com.zextras.backup.ZEBackupThread.execOperation(Un known Source)
at com.zextras.backup.ZEBackupThread.run(Unknown Source)

I tested a migration from an old server with 40GB of data to a fresh install and it worked perfectly. Then a couple of days later I created a new backup set on the source server and tried importing again onto the new server to see if it worked OK and will only import the missing emails. However, this error occured after 17mins of running.

Does anyone have any pointers to see where I can look to see what may be causing this? Thanks

[d0s0n]

Hello guys and thanks for reporting.

Our dev team are trying to reproduce the issue in all possible cases.
It is undoubtedly connected to the restoration of a DataSource (rss folder or imap/pop3 external account) belonging to an account which was already present on the server. If the restoration had taken place on a new server, or on a server where that specific account didn't already exist this problem would not have arisen.
This is, in fact, an atypical scenario for the use of the import/export functions, probably just as test.

In order to properly investigate the scenario a few more detail are need, however:

- version of ZeXtras involved (I surmise the latest available, the 0.99.7)
- the account which was being restored when the problem came (check mailbox.log for the last account mentioned as being restored)
- the version of Zimbra active when the DataSources in question were created (5.x,6.x,7.x ecc.)
- informations regarding the datasources of the account in question, which can be extracted via either zmprov

zmprov gds <user>

or ldapsearch

ldapsearch -x -D `zmlocalconfig -m nokey -s zimbra_ldap_userdn` -w `zmlocalconfig -m nokey -s zimbra_ldap_password` -h `hostname -f` "(objectClass=zimbraDataSource)"

[Obviously mask passwords and the likes].

The latter is needed in order to check for missing ldap attributes which have been added and become mandatory in recent versions of Zimbra. Zimbra 6/7 requires those attributes and ZeXtras follows suit in that regard.

As for support, we're here to assist at the best of our capabilities every ZeXtras Suite Trial and Migration Tool user through our public forums, in addition to our customers, partners and betatesters through private forums and direct contact if necessary.

Have a nice day,
D0s0n

[Eclipse]

Hey sorry for the very late response, I became busy with other things and ended up not having to do a backup or restore and so I put it off.


Anyways, I did a full backup of all the domains today and it worked fine.

I deleted 12,000 emails in an account after that where all the emails were information about the server that were old and unneeded.

I then imported that domain of the 12,000 emails with Zextras, I got the email that it started the restore of every email account in that domain, then I got the second email of it finishing successfully.

However, once I go to the account where the 12,000 old emails were, they do not appear, making me think the restore, restored nothing...

In the debug info, it lists 135 items restored....shouldn't that be a bit more then 135 if I had 12,000 emails originally after backing up the domain?

Any ideas? Would you like me to run those commands?

[Cine]

Hello Eclipse.

First of all, I'd like you to give us some additional informations about your setup:
- Zimbra version on Source machine
- Zimbra version on Destination machine
- ZeXtras Suite version

Another question: What do you mean by Full Backup? You mean a Full Scan or an external backup with the "Export Backup" button? Can you please explains step by step what you've done to start the backup and restore?

Have a nice day,
Cine

[Eclipse]

Hello Eclipse.

First of all, I'd like you to give us some additional informations about your setup:
- Zimbra version on Source machine
- Zimbra version on Destination machine
- ZeXtras Suite version

Another question: What do you mean by Full Backup? You mean a Full Scan or an external backup with the "Export Backup" button? Can you please explains step by step what you've done to start the backup and restore?

Have a nice day,
Cine

- Zimbra version on Source machine - 7.1.2, its running on CentOS
- Zimbra version on Destination machine - 7.1.2, note, that I am trying to import to the same machine as a test. I have no other machine running Zimbra.
- ZeXtras Suite version - 0.99.7

What I am trying to do right now is to export a domain and then re-import it to the same server. I start by going to the Zxbackup tab in the Zextras zimlet, I am then scrolling down to the import and backup buttons and exporting one domain to a folder with read and write access. I am then deleting emails on an email account in that domain, I then import that same domain I just exported to try and get those emails back. Note that the backing up and importing is done on the exact same machine.

Can I not backup and import on the same machine for a practice run?

[Cine]

Hello Eclipse.

We are currently investigating the problem and trying to reproduce it.

Importing data to an existent account, however, is not a proper use of the software. In order to restore deleted data on an account, the correct way to proceed is to use the Restore functions of ZeXtras Suite.

A more proper import test for your case would be:
- Delete the 'map_[serverId]' file from the /opt/zimbra/backup/zextras/ folder. (or move it to another folder)
- Export a Domain
- Delete All the accounts within the domain
- Delete the Domain
- Import back the Domain and all of its accounts on the server


Have a nice day,
Cine

[Eclipse]

Hello Eclipse.

We are currently investigating the problem and trying to reproduce it.

Importing data to an existent account, however, is not a proper use of the software. In order to restore deleted data on an account, the correct way to proceed is to use the Restore functions of ZeXtras Suite.

A more proper import test for your case would be:
- Delete the 'map_[serverId]' file from the /opt/zimbra/backup/zextras/ folder. (or move it to another folder)
- Export a Domain
- Delete All the accounts within the domain
- Delete the Domain
- Import back the Domain and all of its accounts on the server


Have a nice day,
Cine

I just did a system rebuild and I started with a domain completely seperate from the domains I exported.

I copied the exported data into a folder in the root directory, I then imported all the accounts and domains at the same time, and it looked like everything went fine, however I got the confirmation email and it skipped A LOT of emails for multiple accounts, and it gives no reason why....

Here is the email log....

This is an automated notification from ZxBackup about External Restore.

Operation External Restore Completed.

- skipped items -

account *****@**********  search query:    item: 261
account *****@**********  search query:    item: 262,263,265,266
account *****@**********  search query:    item: 265,266,267,268,269,274,275,276,277,282,283,288,290,293,295,296,304,308,309,313,314,315,318,319,320,322,326,327,329,332,337,338,340,342,343,344,346,348,350,351,353,355,359,362,363,365,367,368,369,373,377,378,380,382,383,385,386,387,388,393,394,395,396,397,398,404,405,407,410,411,412,414,415,416,418,419,421,423,424,427,428,429,432,433,434,435,436,439,440,445,446,447,452,454,455,458,459,460,461,462,465,468,472,475,476,477,479,480,483,485,487,489,490,493,494,497,498,500,501,502,503,507,509,517,518,519,520,521,523,524,525,526,533,534,536,538,539,542,545,546,547,549,550,552,554,556,557,562,563,565,568,569,570,571,572,573,574,575,576,578,579,580,581,582,586,589,591,594,595,596,597,599,603,607,608,609,610,612,613,616,617,618,624,627,628,630,634,637,640,643,646,649,657,658,661,663,666,667,672,673,675,678,679,681,687,688,693,694,696,699,700,706,708,709,712,715,717,720,726,730,732,735,736,741,742,744,748,751,753,757,759,760,763,765,766,768,772,774,775,777,778,781,784,787,792,795,796,801,802,805,849,900,920
account *****@**********  search query:    item: 260
account *****@**********  search query:    item: 265,268,272,274,275,276,280,282,283,285,290,292,294,296,297,299,301,302,305,309,311,313,315,319,322,324,327,334,336,340,341,344,345,346,347,380,400,420,440,462,463,465
account *****@**********  search query:    item: 480,487,488,500,502,504,506,508,511,512,513,521,522,526,529,530,531,533,535,536,537
account *****@**********  search query:    item: 12964,12970,12973,12992,12995,12996,13000,13003,13007,13011,13017,13022,13023,13026,13027,13028,13029,13030,13034,13042,13045,13048,13054,13064,13068,13076,13077,13081,13083,13090,13096,13101,13105,13108,13116,13119,13123,13131,13132,13135,25547,25548,25549,25550,25552,25558,25559,25565,25566,25580,25581,25582,25583,25585,25586,25600
account *****@**********  search query:    item: 257,263,266,269,270,271,276,277,282,285,286,287,288,289,292,293,295,296,301,302,303,305,307,380,381,500,520
account *****@**********  search query:    item: 270,282,290,293,299,302,307,308,309,312,313,316,317,320,321,323,325,326,327,328,330,331,333,336,337,339,343,345,347,351,352,354,357,358,360,364,371,377,385,388,389,390,394,395,396,399,400,401,402,440,441,443,460,462,463,480,481,483,500,520,580,581,600
account *****@**********  search query:    item: 260,262,264,266,268,269,271,272,282,284,286,290,292,294,295,296,297,300,304,305,306,309,310,312,313,314,315,316,318,320,325,326,327,332,340
account *****@**********    item: 301,302
account *****@**********  search query:    item: 260
account *****@**********  search query:    item: 302,306,307,309,312
account *****@**********  search query:    item: 260,261
account *****@**********  search query:    item: 257,873,972,1170,1205,1330,1332,1361,1390,1461,1488,1618,1667,1747,1885,1980,1996,1998,2020,2082,2092,2093,2096,2098,2106,2107,2109,2161,2164,2168,2171,2173,2174,2175,2177,2178,2179,2182,2190,2191,2209,2259,2261,2262,2264,2267,2271,2290,2312,2314,2316,2318,2319,2320,2321,2322,2326,2327,2329,2333,2336,2337,2339,2340,2341,2357,2361,2364,2366,2367,2368,2369,2370,2371,2373,2374,2377,2378,2380,2389,2391,2397,2415,2416,2418,2419,2420,2423,2424,2425,2437,2439,2441,2442,2450,2460,2461,2462,2463,2464,2465,2468,2472,2473,2494,2498,2505,2508,2537,2560,2561,2624,2629,2633,2636,2637,2640,2641,2643,2645,2646,2647,2648,2649,2651,2662,2667,2671,2672,2693,2694,2696,2699,2700,2701,2703,2716,2720,2724,2731,2732,2733,2737,2739,2740,2742,2743,2745,2749,2752,2754,2756,2759,2760,2800,2801,2804,2805,2806,2807,2809,2820,2821,2825,2826,2827,2842,2843,2845,2846,2855,2860,2866,2868,2869,2870,2873,2876,2882,2885,2886,2891,2893,2895,2897,2908,2912,2915,2917,2921,2935,2936,2937,2945,2949,2951,2964,2965,2970,2971,2972,2988,2989,3245,3247,3249,3251,3252,3254,3256,3259,3260,3261,3275,3283,3284,3285,3286,3288,3289,3290,3292,3299,3300,3339,3341,3344,3347,3360,3365,3366,3368,3369,3376,3377,3381,3386,3400
account *****@**********  search query:    item: 280
account *****@**********  search query:    item: 257
account *****@**********  search query:    item: 261
account *****@**********  search query:    item: 282,286,300,302,303,305,322,323,360,400,402,421,423,440,441,442,443,444,446,447,460
account *****@**********  search query:    item: 262,264
account *****@**********  search query:    item: 262,263,266,267,268,270,272,274,276,278,280,281,282,283,291,292,293,301,302,304,305,306,307,308,310,312,321,324,342,343,344,345,346,350,353,387,389,390,394,396,398,400,403,421,423,427,428,429,431,432,433,434,435,436,437,441,449,460,463,467,470,471,480,483,500,521,541,544,548,549,550,561,562,564,566,581,582,583,586,588,589,590,593,596,598,601,603,605,606,608
account *****@**********  search query:    item: 263,265,268,270,271,275,280,281,283,284,285,286,288,290,293,301,302,305,308,323,324,326,327,328,329,330,331,332,333,336,337,360,367,380,382,420,421,443,452,453,460,461,463,480,481,485,489,490,492,494,495,501
account *****@**********  search query:    item: 261,262
account *****@**********  search query:    item: 257,261,263,264,265,266,267,269,270,275,279,281,285,320,321,322,323,325,360,361,381,400,420,421,440,460,472,474,476,478
account *****@********** search query:    item: 258,260,263,264,265,284,286,289,291,292,293,294,295,299

(The email addresses are not all the same size, I just censored them...)

Any idea how I can get these items to import sucessfully or find out why they are not importing?

[d0s0n]

Hi Eclipse,

the skipped items reported in mail notification are unrestorable items. To view them on the source accounts you can use the search queries reported in the e-mail on the webclient, so you can see if these items are correctly stored and they can be correctly viewed.
The reasons for these errors can be found in mailbox.log files during import process. If you find some strange errors on that log, feel free to post it on this thread.

Have a nice Sunday

D0s0n

[Eclipse]

Hi Eclipse,

the skipped items reported in mail notification are unrestorable items. To view them on the source accounts you can use the search queries reported in the e-mail on the webclient, so you can see if these items are correctly stored and they can be correctly viewed.
The reasons for these errors can be found in mailbox.log files during import process. If you find some strange errors on that log, feel free to post it on this thread.

Have a nice Sunday

D0s0n

Thank you for the quick reply, can you go into detail a bit more about viewing the search queries?

I'm trying to think what could make them unrestorable, could it be each one of those emails had an attachment?

I dont suppose it's possible to just copy and paste the email's into the Zimbra server...?

I check the mailbox log....

Update:

One of the many ZeXtras import mailbox log errors

2011-09-25 02:56:06,839 WARN [Thread-50] [] extensions - [68e0257b-c182-41f8-b212-ea228c03327a] ExternalRestore: Unable to restore item 257, account *****@******** (8f465ef1-a22a-4ce0-be72-6d11b4aa354e):
Unknown Exception in restore
at com.zextras.backup.ZEBItem.restore(Unknown Source)
at com.zextras.backup.ZEExternalRestoreOperation.rest oreItemsFromExternal(Unknown Source)
at com.zextras.backup.ZEExternalRestoreOperation.doOp eration(Unknown Source)
at com.zextras.op.ZEOperation.exec(Unknown Source)
at com.zextras.backup.ZEBackupThread.execOperation(Un known Source)
at com.zextras.backup.ZEBackupThread.run(Unknown Source)
Caused by: Unable to restore MailItem
... 6 more
Caused by: java.io.FileNotFoundException: /Zimbra_Domain_Backup-9-24-11/items/yN/yN,txgvociS1ehnFzn80NzqYWrk= (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.jav a:120)
at com.zextras.backup.ZEBItem.createZimbraItem(Unknow n Source)
... 6 more

It appears that its trying to restore a file that doesn't exist in the exported folder....

I exported every single thing and let the copy finished, I don't know why it doesn't exist....