Import process running but not completing
Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: Import process running but not completing

  1. #1
    Member
    Join Date
    Jan 2012
    Posts
    20

    Import process running but not completing

    Hi all
    I was testing the ZeXtras backup suite with data exported (from a test copy) of our mail server through the Migration Tool to have a better idea on a real scenario in case of migration/recovery/etc

    The original server is running with ubuntu 10.4 64 bit and Zimbra 7.1.4 OSE with 160 mailbox for 40 gb of data and the latest version downloaded two days ago of the migration tool plugin. The export went ok with no error message as showed on the report

    Code:
    This is an automated notification from ZxBackup about Export.
    
    Operation Export Completed.
    
    Operation Id: 4a54baf9-fdb5-4941-ac43-0fff2dd4f1e3
    Operation Host: **********.com
    
    - stats -
                       new accounts: 161
                   accounts updated: 0
                       item updated: 0
                       new metadata: 487604
                          new files: 377187
                      checked items: 487681
                        backup path: /backup/migration
                      skipped items: 0
                          items/sec: 71.149155
      additional notification mails:  
           Exceptions: None

    The new destination server is clean machine that has the same ubuntu/zimbra software version with V1.2.2 of the ZeXtras Suite

    Once started the import the procedure the users was immediately created with their own preferences and the mail recovered from the backup starting from the admin acccounts

    Code:
    This is an automated notification from ZxBackup about External Restore.
    
    Operation External Restore is still running and is now starting items restoring.
    
    Operation Id: 76bc0d72-6b4f-4f2a-91f0-b254e630a67d
    Operation Host: **********.com
    Operation Log path: /opt/zimbra/log/zextras_ZxBackup_ExternalRestore_76bc0d72-6b4f-4f2a-91f0-b254e630a67d.log
    
    Warning: the following admin accounts have been restored: *****.*****@**********.com, *****.*****@**********.com, *****.*****@**********.com
    
    Warnings: 0
    Errors: 0
    The problem is that now, after 24 hours procedure has not ended yet, i see a high cpu activity from a java task but the import is like in a freezed status at yesterday evening (7 hours after start)

    Looking at the cli monitor this is what i get

    Code:
    Operation log path: /opt/zimbra/log/zextras_ZxBackup_ExternalRestore_76bc0d72-6b4f-4f2a-91f0-b254e630a67d.log
     Warnings/Errors: 0/0
     Restored Accounts(only attributes):           158/158
     Restored Accounts(common items):              93/158
     Restored Accounts(only shared & datasources): 0/158
     Current Account: *****.*****@*******.com
     Restored/Skipped Items: 284467/275
     Unrestored Items: 1376
    and the message is repeated endless, while in the log file the last entry is

    Code:
    2012-01-30 21:43:00,573 INFO Adding item 20415[76bc0d72-6b4f-4f2a-91f0-b254e630a67d] ExternalRestore Adding item 20415
    Any idea on what i can check?


    Other than in the log i see other errors like this one

    Code:
    2012-01-30 21:05:42,673 INFO Adding item 2056[76bc0d72-6b4f-4f2a-91f0-b254e630a67d] ExternalRestore Adding item 2056
    2012-01-30 21:05:42,733 WARN Unable to restore item 2056, account *****.*****@**********.com(40159096-6d6b-48a8-8f3d-33f7b5a19270):
    Unknown Exception in restore
    	at com.zextras.backup.ZEBItem.restore(ZEBItem.java:547)
    	at com.zextras.backup.ZEExternalRestoreOperation.restoreItemsFromExternal(ZEExternalRestoreOperation.java:548)
    	at com.zextras.backup.ZEExternalRestoreOperation.doOperation(ZEExternalRestoreOperation.java:438)
    	at com.zextras.op.ZEOperation.exec(ZEOperation.java:584)
    	at com.zextras.backup.ZEBackupThread.execOperation(ZEBackupThread.java:199)
    	at com.zextras.backup.ZEBackupThread.run(ZEBackupThread.java:109)
    Caused by: com.zimbra.common.service.ServiceException: invalid request: Message content is invalid.
    ExceptionId:ZxBackup Thread:1327953942733:53dcbc66f1220a3e
    Code:service.INVALID_REQUEST
    	at com.zimbra.common.service.ServiceException.INVALID_REQUEST(ServiceException.java:260)
    	at com.zimbra.cs.mailbox.Mailbox.addMessage(Mailbox.java:4670)
    	at com.zextras.backup.ZEBItem.createZimbraItem(ZEBItem.java:1127)
    	at com.zextras.backup.ZEBItem.restore(ZEBItem.java:522)
    	... 5 more
    [76bc0d72-6b4f-4f2a-91f0-b254e630a67d] ExternalRestore Unable to restore item 2056, account *****.*****@**********.com(40159096-6d6b-48a8-8f3d-33f7b5a19270):
    Unknown Exception in restore
    	at com.zextras.backup.ZEBItem.restore(ZEBItem.java:547)
    	at com.zextras.backup.ZEExternalRestoreOperation.restoreItemsFromExternal(ZEExternalRestoreOperation.java:548)
    	at com.zextras.backup.ZEExternalRestoreOperation.doOperation(ZEExternalRestoreOperation.java:438)
    	at com.zextras.op.ZEOperation.exec(ZEOperation.java:584)
    	at com.zextras.backup.ZEBackupThread.execOperation(ZEBackupThread.java:199)
    	at com.zextras.backup.ZEBackupThread.run(ZEBackupThread.java:109)
    Caused by: com.zimbra.common.service.ServiceException: invalid request: Message content is invalid.
    ExceptionId:ZxBackup Thread:1327953942733:53dcbc66f1220a3e
    Code:service.INVALID_REQUEST
    	at com.zimbra.common.service.ServiceException.INVALID_REQUEST(ServiceException.java:260)
    	at com.zimbra.cs.mailbox.Mailbox.addMessage(Mailbox.java:4670)
    	at com.zextras.backup.ZEBItem.createZimbraItem(ZEBItem.java:1127)
    	at com.zextras.backup.ZEBItem.restore(ZEBItem.java:522)
    	... 5 more
    about unrestored items, how i can identify what is missing (and maybe why)


    Also one more question on restoring a domain from scratch.... if i have to restore the domain CAT.com the right way is to create a CAT.com domain on a new machine and then recover all the data (excluding the special account like ham. spam and viruses) or is better create a temp domain like DOG.com restore CAT.com and then delete DOG ones?

    Sorry if question sound dumb but i've not found on the wiki and i'm a new zimbra user in learning mode

    Thanks

  2. #2
    CTO ZeXtras Employee d0s0n's Avatar
    Join Date
    Apr 2011
    Posts
    565
    Hi Davide,

    I notified the Dev team of the error, but I need some more information about your migration enviroment:

    • How did you transferred data from source to destination servers? Maybe there was a transfer problem?
      I suspect a corruption because I see many "Unrestored Items" (1376).
    • Can you check the particular item (2056)?
      You can find it on /exported_data_path/accounts/40159096-6d6b-48a8-8f3d-33f7b5a19270/items/56/2056
      If you read the file you can see its "digest", a string like this: "3BOxfMVzme,tCPaxiseaKX,MjkU=", so the real content of item is stored on this path(gzipped):
      /exported_data_path/items/3B/3BOxfMVzme,tCPaxiseaKX,MjkU=
      You can copy this file in a tmp folder and gunzip it with this command:
      Code:
      cp /exported_data_path/items/3B/3BOxfMVzme,tCPaxiseaKX,MjkU= /tmp/test.gz
      gunzip /tmp/test.gz
      ..and view content with a simpler:
      Code:
      cat /tmp/test


    With these info, Dev Team can check the problem more deeply, because your problem is very unusual, expecially for a migration from the same zimbra version.

    Thanx for your input,
    D0s0n
    ZeXtras Website # ZeXtras Wiki # ZeXtras Store

    Head of ZeXtras System Administrators

  3. #3
    CTO ZeXtras Employee d0s0n's Avatar
    Join Date
    Apr 2011
    Posts
    565
    Quote Originally Posted by davidesjhs View Post
    ...about unrestored items, how i can identify what is missing (and maybe why)
    At the end of import you will be notified about all unrestorable items, sorted by account, As you can find HERE you can use the search box on the source account to find the unrestored items with the search string reported by the notification, like this:
    Code:
    item: 262,263,265,266
    Quote Originally Posted by davidesjhs View Post
    Also one more question on restoring a domain from scratch.... if i have to restore the domain CAT.com the right way is to create a CAT.com domain on a new machine and then recover all the data (excluding the special account like ham. spam and viruses) or is better create a temp domain like DOG.com restore CAT.com and then delete DOG ones?
    The way first is best.

    Quote Originally Posted by davidesjhs View Post
    Sorry if question sound dumb but i've not found on the wiki and i'm a new zimbra user in learning mode
    All we are in learning mode and we need feedback to improve our product and our documentation.

    D0s0n
    ZeXtras Website # ZeXtras Wiki # ZeXtras Store

    Head of ZeXtras System Administrators

  4. #4
    Member
    Join Date
    Jan 2012
    Posts
    20
    Hi, sorry for the delay but i've been a bit busy lately

    I transferred the data via virtual disk, disconnected from the original vm and then connected to the one importing data

    I have checked the item 2056 as you asked and once decompresses i'm able to read the message from the x-header trough the contents. It's plain text where i can read the answer to a mail that look like a html formatted table.
    I can forward somewhere if it can be of any help

    Davide

  5. #5
    Member
    Join Date
    Jan 2012
    Posts
    20
    Quote Originally Posted by d0s0n View Post

    The way first is best.
    Ok

    that mean once all the data has been imported i can delete the "default" COS created during the install, optionally rename the restored ones (default_restored_******.com. calendar_restored_******.com) and choose the new default.
    There are other step to be done? because i just noticed that under Server settings / MTA - MTA Trusted Networks is missing an ip that i've added on the source machine, not sure if this can be related to my restore problems or if the problem it's just me

    Davide

  6. #6
    CTO ZeXtras Employee d0s0n's Avatar
    Join Date
    Apr 2011
    Posts
    565
    Quote Originally Posted by davidesjhs View Post
    I have checked the item 2056 as you asked and once decompresses i'm able to read the message from the x-header trough the contents. It's plain text where i can read the answer to a mail that look like a html formatted table.
    I can forward somewhere if it can be of any help
    You could attach it to this thread or, if you prefer, you can send it (as attacment) via mail to our community manager. Is also possible to open a bug report.

    Thanks for your contribution.
    ZeXtras Website # ZeXtras Wiki # ZeXtras Store

    Head of ZeXtras System Administrators

  7. #7
    CTO ZeXtras Employee d0s0n's Avatar
    Join Date
    Apr 2011
    Posts
    565
    Quote Originally Posted by davidesjhs View Post
    that mean once all the data has been imported i can delete the "default" COS created during the install, optionally rename the restored ones (default_restored_******.com. calendar_restored_******.com) and choose the new default.
    You can refer to Disaster Recovery Guide. But I think that you cannot rename or delete the default cos, because it is required by Zimbra installation and its zimbraId is a constant string. However you can use new restored COSes without problems and you can rename them as you want.

    Quote Originally Posted by davidesjhs View Post
    There are other step to be done? because i just noticed that under Server settings / MTA - MTA Trusted Networks is missing an ip that i've added on the source machine, not sure if this can be related to my restore problems or if the problem it's just me
    For now all server settings are not restored by the import procedure, including any MTA settigs or specific customizations.

    D0s0n
    ZeXtras Website # ZeXtras Wiki # ZeXtras Store

    Head of ZeXtras System Administrators

  8. #8
    Member
    Join Date
    Jan 2012
    Posts
    20
    Hello again and thanks for the support

    I have sent the file by email to the community manager, meanwhile i'm still doing some test and experiment
    Davide

  9. #9
    ZeXtras Community Manager ZeXtras Employee Cine's Avatar
    Join Date
    Apr 2011
    Posts
    2,342
    Hi Davide,

    I got your email, we'll check it ASAP


    Have a nice day,
    Cine
    IT Support Team Contact Form
    Sales Team Contact Form

    ZeXtras Website
    # ZeXtras Wiki # ZeXtras Store

    Have ZeXtras Suite or ZeXtras Migration Tool been helpful to you?
    Share your experience in the Zimbra Gallery!

    ZeXtras Suite on the Zimbra Gallery
    ZeXtras Migration Tool on the Zimbra Gallery

  10. #10
    Member
    Join Date
    Jan 2012
    Posts
    20
    Hi All

    I have done another couple of testing in exporting again the data and importing on a new machine.
    I verified that what is causing the import process to stuck is a specific user, if i exclude that one from in the account selection the operation end correctly (though if with a loooot of skipped items) but no way to complete the import if this person is selected (even as single account to restore or disabling zimbra av)

    I've seen that an update of ZxMig is available going to try again with this one

    Davide

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •