Extreme number of unrestored items
Results 1 to 6 of 6

Thread: Extreme number of unrestored items

  1. #1
    Active Member
    Join Date
    Dec 2012
    Posts
    4

    Extreme number of unrestored items

    Hello there.

    I am performing a test migration, hopefully the basis of a permanent migration. I am using the latest version of the migration tool and the Backup Suite. Source server is 7.1.4 running on Ubuntu 10.4 LTS and destination server is 8.0.2 running on Ubuntu 12.0.4 LTS. I used sshfs to cp the data across to the new server. Rsync was just too slow. There is about 180 GB of mail. It still took 6 hours to move all of the data (and unfortunately it is not possible to move the data with a shared disk of any sort and trying to write the migration data directly to the destination server using a sshfs mount point failed - this would be a great method if I can get it to work).

    I restored one account as a test and it looks like everything restored except any mail. Shared items, such as a shared contact lists, restored nicely. Unfortunately I did not see that there was no mail until after I kicked off the full restore of the server accounts. Now I've noticed that the shared items are being restored but none of the accounts have any mail in them. Folders are being created, but again, no items in the folders.

    While this is a test migration, I had hoped that I would be able to use this as the first step in the incremental migration b/c of the extreme amount of email that some of these people have (1 account alone is 40 gb ).

    I looked at the Wiki site that the notification email states. However, there is nothing in the notification email concerning the query that I can enter in to the account of the user, as the wiki page indicates I should do.

    Could there be a permissions issue with some of the folders on the destination server?
    Could I create a mount on the destination server pointing to the source server's migration folder in order to save time with the transfer of the data?
    What is the typical throughput of the backup/restore process?

    Please advise on how I can move forward.

  2. #2
    Active Member
    Join Date
    Dec 2012
    Posts
    4
    Looks like it was a permissions issue on the Items directory. I modified the permissions and started another restore on a rebuilt server and fresh Zimbra install.

    However, the restore looks like it will take awhile. Am I able to perform a migration with the Zextras Migration tool in an incremental fashion such as what is detailed with the Backup tool? Or does the incremental ONLY work with the Backup tool? I've got the full recovery going now but I suspect it will take days to restore. If I can run another Zextras Migration scan on the production server at the end of the week only capturing the new mail then I can complete the migration over the weekend. Or do I have to scrap this Migration backup and do the incremental backup using the Backup Tool?

    Thanks for your thoughts.

  3. #3
    CTO ZeXtras Employee d0s0n's Avatar
    Join Date
    Apr 2011
    Posts
    570
    Hi mornic and welcome to the forums.

    Quote Originally Posted by morinic View Post
    However, the restore looks like it will take awhile. Am I able to perform a migration with the Zextras Migration tool in an incremental fashion such as what is detailed with the Backup tool? Or does the incremental ONLY work with the Backup tool?
    This is precisely the main difference between ZxMig and ZxBackup, if you are evaluting ZeXtras Backup for the first time you have 30 days to perform your migration following the incremental migration guide.

    Quote Originally Posted by morinic View Post
    I've got the full recovery going now but I suspect it will take days to restore. If I can run another Zextras Migration scan on the production server at the end of the week only capturing the new mail then I can complete the migration over the weekend. Or do I have to scrap this Migration backup and do the incremental backup using the Backup Tool?

    Thanks for your thoughts.
    In order to not scrap the initial export on the source server, you can install the ZxBackup and use the export folder as "backup path" of Zxbackup module, but you should do so after completing the first import on the target server.

    Theoretically, the update of the export folder content, during the import operation should not cause problems, but it depends on what changes are made ​​on the source server, so you should install the Zextras Suite on source server only when the first import is completed.

    Keep us updated!

    D0s0n
    ZeXtras Website # ZeXtras Wiki # ZeXtras Store

    Head of ZeXtras System Administrators

  4. #4
    Active Member
    Join Date
    Dec 2012
    Posts
    4
    I had to scrap the initial backup for several reasons, one being that I wanted to start from scratch. I have since installed the Backup tool on both the source and the destination servers. Created a sshfs mount from source to destination which saved copying the data across. However, the restore has died twice due to some network issue, I believe. Not really sure why it would have died but the log file wasn't changing and no new data was being restored. Zextras Notifications did not show that the job had stopped. I ended up using the Zextras Backup console to stop all running backup processes. Waited a few minutes and then started a new restore process. However, I have since noticed that the restore is now restoring duplicate emails into the new accounts. Is this a result of having to restart the restore? Are there some how duplicate processes running that would cause it to restore emails twice? Do I need to scratch Zimbra install and start a new restore?

    Thanks for your help.

  5. #5
    CTO ZeXtras Employee d0s0n's Avatar
    Join Date
    Apr 2011
    Posts
    570
    Quote Originally Posted by morinic View Post
    I had to scrap the initial backup for several reasons, one being that I wanted to start from scratch.
    Hi mornic, I can assure that this was not necessary.

    Quote Originally Posted by morinic View Post
    I have since installed the Backup tool on both the source and the destination servers. Created a sshfs mount from source to destination which saved copying the data across. However, the restore has died twice due to some network issue, I believe.
    Network connections are often unsafe, perhaps it could be more performant to use NFS instead of SSHFS.

    Quote Originally Posted by morinic View Post
    Not really sure why it would have died but the log file wasn't changing and no new data was being restored. Zextras Notifications did not show that the job had stopped. I ended up using the Zextras Backup console to stop all running backup processes. Waited a few minutes and then started a new restore process. However, I have since noticed that the restore is now restoring duplicate emails into the new accounts. Is this a result of having to restart the restore? Are there some how duplicate processes running that would cause it to restore emails twice? Do I need to scratch Zimbra install and start a new restore?

    Thanks for your help.
    The duplication you noticed is caused by the missing of the map_files on backup path in the destination server, because in there files all imported item ids are saved. Probably before you restarted the import process you have scrapped that path also.

    PS: We have just builded a new version of the zextras suite that fix a little problem with the tags in the import from a zimbra 7 to the zimbra 8. It will not be published because the next week will be released the 1.8.0 version. If you have many tags in your mailboxes you should try it. If you want I send you a link in pm.

    D0s0n
    ZeXtras Website # ZeXtras Wiki # ZeXtras Store

    Head of ZeXtras System Administrators

  6. #6
    Active Member
    Join Date
    Dec 2012
    Posts
    4
    To be clear, I had/have the SSHFS to facilitate the placing of the files on the destination server during the backup process eliminating the need to rsync the data across. It took a little bit longer but saved time in the long run.

    During the middle of the night I got a notification that the stats and sasl services were restarted a couple of times on the destination server. Would this have caused the restore to stop? Is it possible that the restore didn't stop but the logging of the restore did? No emails were being restored while I watched the 'tail' of the log file. The timing of the stoppage kind of correlates with the Zimbra Backup timing. Can I or should I stop that while doing this lengthy restore?

    The account that has all of the duplicates does not use any tags and I don't suspect that many people do in this company.

    And can I delete the mailbox with the duplicates and have the system restore it during the refresh?

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •