ZxBackup Import Running Almost 2 Weeks
Results 1 to 8 of 8

Thread: ZxBackup Import Running Almost 2 Weeks

  1. #1
    Active Member
    Join Date
    Oct 2014
    Posts
    3

    ZxBackup Import Running Almost 2 Weeks

    Hello ZeXtras Community,

    I'm evaluating the ZeXtras suite of products and testing a migration from Zimbra v.7.0.1 --> 8.5.0 (following ZxBackup:_Incremental_migration_with_ZeXtras_Backu p). I realize that this topic has been discussed a few times here and I've read and followed the advice on each thread...but still cannot find a solution. For my test, I am migrating a single domain of users ~8mailboxes totalling ~30GB of data. The export from the source server was seamless and completed in ~24hrs (which is expected as it is an old/slow 32-bit server). However, the import on the target is taking much-much longer...it has been running for almost 2weeks and has restored only 3/4 of the data.

    Here's my hardware configuration...
    -AMD FX(tm)-8150 Eight-Core Processor (3.6GHz)
    -8GB RAM
    -Internal hard drive

    Here are some troubleshooting steps that I've completed...
    -Followed the jvm and mysql optimization guide
    -Ensure that the import is still running using the suggested `zxsuite backup monitor <ID>` command
    -Run `iostat -xm 1` which shows iowait between 0-->12% maxmium
    -Ensure that the restore was started without "Apply HSM" option
    -I have submitted a zmthrdump to community[at]zextras.com a few days ago

    Will disabling the HSM Policy althogether make any difference in this case? This is the next step that I was going to try...but thought I would turn to the community for help before that.

    Thank you for your time,



    Jason

  2. #2
    ZeXtras Community Manager ZeXtras Employee Cine's Avatar
    Join Date
    Apr 2011
    Posts
    2,361
    Hello jasons,
    welcome to the forums!

    In the vast majority of migrations like yours the main bottleneck is the storage, and unfortunately a server with a single drive is not the most performing solution.
    Thus said, two weeks for a 30gb import is surely not common at all, may I ask what kind of disk you are using and to send me the output of a 1-2 minutes run of iostat -xm 1?

    Disabling the HSM Scheduling might help but I'm afraid it won't be a game breaking change... I'd try to disable the Realtime Scanner, if enabled, but my first suggestion is to wait for ZeXtras Suite 1.10.0, the first stable version to support Zimbra 8.5.0, which will be released in a couple of hours.

    Have a nice day,
    Cine
    the ZeXtras Team


    p.s.: Please don't send log files or help requests to community@zextras.com unless there's a related forum post, "orphaned" emails are not taken in charge by our staff...

  3. #3
    Active Member
    Join Date
    Oct 2014
    Posts
    3
    Hi Cine,

    Thanks for the prompt reply! I am just using a mirrored pair of SATA-3 disks (Seagate ST3000DM001). I've also verified that the Realtime Scanner is disabled...so no gain there. I've sent you 2min of iostat to the community email address.

    I've noticed that ZeXtras Suite v.1.10.0 is now available...I will try to upgrade and re-run the import over the weekend. I'll report the results on Monday.

    I appreciate the assistance!



    Jason

  4. #4
    Active Member
    Join Date
    Oct 2014
    Posts
    3
    I have installed and tested ZeXtras Suite v.1.10.0...same results as the previous version. However, I think I may have narrowed-down the issue. The domain that I was testing the initial migration with included a logging account with tens-of-millions of very small log messages. It seems that indexing this large volume of KB sized messages was causing the slowness??? I repeated the migration with a different domain of similiar total size ~40GB and the import completed much quicker (~24hrs).

    @Cine, as I've spent quite some time troubleshooting this issue...my trial license has almost run out. Could I requrest an extension in order to complete my migration and really evaluate the product?

    Thanks for your help!


    Jason

  5. #5
    ZeXtras Community Manager ZeXtras Employee Cine's Avatar
    Join Date
    Apr 2011
    Posts
    2,361
    Hello Jason,

    glad that the cause of the slow import was found!

    I'd suggest to export the contents of said account via Zimbra's own Import/Export feature, run a SmartScan on the source server, empty the account on the destination server and proceed with the External Restore skipping the affected account whose content can be imported while the Restore is running.

    In order to obtain an Extended Trial License, please get in touch with the ZeXtras Sales Team via THIS form including the link to this thread in the "Enquiry" field (to speed things up, register to the ZeXtras Store and include your username aswell).

    Have a nice day,
    Cine
    the ZeXtras Team

  6. #6
    Junior Member ZeXtras Reseller
    Join Date
    Nov 2013
    Posts
    1
    Hiya guys,

    I had a similar situation, where a new server was about to be installed.
    I have a 1.5TB install of zimbra, and at 50GB/day, it took me some considerable amount of days without end-users having access to their emails.

    It raised a few feature requests from my side:

    1. Can we have addressbook and calendar populated after the accounts and preferences have been set up, please, as that way all users see something is happening - rather fast.

    2. Whilst the first import took a few weeks, the old server was still running once thefirst import was completed, I wanted to do another import, incremental of course to'top up' thepreviou import. I was told this is notpossible. Please make it possible. I ended up copying the /opt/zimbra/ with rsync (and got MUCH faster transfer rates) and the server up andrunning rather quick.

    3. I would like to be able to, if possible, select what account to import, and also, what folder. I have a case now where s'user have deleted a folder, and wants it back. I am thus restoring the who account, into a new account, and subsequently moving ut over to where it ought to be.

    Best wishes,

    Miguel

  7. #7
    ZeXtras Community Manager ZeXtras Employee Cine's Avatar
    Join Date
    Apr 2011
    Posts
    2,361
    Hello Miguel!

    Please feel free to open an RFE requests for points 1 and 3 of your last post in the ZeXtras Bugzilla in order for both the community and the ZeXtras R&D team to discuss your ideas - by the way, you can choose which mailboxes you wish to import both via the External Restore wizard and the related CLI command, while selecting a single folder to be restored might be a useful addition to both Restore on New Account and Undelete Restore modes...

    About point 2, however, I need to clarify that Incremental Migrations are possible when using ZeXtras Backup on both the source and destination server, and a guide can be found here: I don't remember your specific case right now, but I'll check our support history to see why the incremental migration was not a viable option.

    Have a nice day,
    Cine
    the ZeXtras Team

  8. #8
    ZeXtras Community Manager ZeXtras Employee Cine's Avatar
    Join Date
    Apr 2011
    Posts
    2,361
    Hello!


    I'm glad to inform you that the next release of ZeXtras Suite will include an optimization to the "External Restore" operation made to improve import times by reducing the operation's overhead: in optimal cpu/storage/memory conditions the improvement will be remarkable, but it should be noticeable even in sub-par situations (e.g. low I/O trhoughput or not enough iops).


    Keep an eye on the "News and announcements" section for more info about the release of ZeXtras Suite 2.0!


    Have a nice day,
    Cine
    the ZeXtras Team

    (This is a bulk update to several similar threads, but don't hesitate to reply!)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •