Epic Data Corruption In Need Of Help
Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: Epic Data Corruption In Need Of Help

  1. #1
    Member
    Join Date
    May 2012
    Posts
    46

    Epic Data Corruption In Need Of Help

    Hello,

    I'm still in trial of a zextras backup installation and my raid 1 got degraded and I lost a lot of data. I managed to get a full scan yesterday on a nas device. Also I managed to recover my store and index folder of the live one.

    I am reinstalling zimbra right now on a new computer, installing zextras backup.

    • Can I restore the data from my zextras backup of my old installation, if I change the folder to the nas share, will he break my backup? Will I be able to restore the data?
    • If I have a live copy of store and index and I overwrite the data after the zextras restore, run zmfixperms, should I have the live data up and running?


    Thank you for your support.
    cypher.

  2. #2
    ZeXtras Developer ZeroZeroOne ZeXtras Reseller
    Participant

    Join Date
    Jun 2011
    Posts
    23
    Quote Originally Posted by iamcypher View Post
    Hello,
    Hi, I'm one of the developer of ZeXtras.

    Quote Originally Posted by iamcypher View Post
    • Can I restore the data from my zextras backup of my old installation, if I change the folder to the nas share, will he break my backup? Will I be able to restore the data?
    • If I have a live copy of store and index and I overwrite the data after the zextras restore, run zmfixperms, should I have the live data up and running?
    Yes, absolutely!

    I suggest you to mount fullscan data from the NAS (or copy them for better performances) to the new server and run an external import to restore your data.

    The store and index directories are not of any help because LDAP data for all users and their information are different on the new server (ex. account and folder IDs are different).

    Best regards,
    --
    kral
    Last edited by Cine; 05-11-2012 at 05:33 PM.

  3. #3
    Member
    Join Date
    May 2012
    Posts
    46
    Thank you for the answer. I need a way of recovering using the store and index folder otherwise I lose a lot of emails. There really is no way at all?

  4. #4
    ZeXtras Community Manager ZeXtras Employee Cine's Avatar
    Join Date
    Apr 2011
    Posts
    2,356
    Hello iamcypher,
    this is Cine.

    If the Real Time Scanner was enabled on your old server, just perform an external import using the old Backup Path as the import path. This will restore all your mailboxes exactly as they were one second before the RAID crash.

    Else, the last available backup is the one contained in the NAS... The store and index folders are of no use unless you also have all other Zimbra folders available - these folders contain the actual data but the structure of such data is contained in the LDAP and in the mysql database.

    (I edited Kral's post to remove some inconsistencies... Devs are not supposed to explain how their software works )
    Last edited by Cine; 05-11-2012 at 05:33 PM.

  5. #5
    Member
    Join Date
    May 2012
    Posts
    46
    Big problem. I get an exception when I try to restore the backup:

    2012-05-12 00:34:45,531 INFO [ZxBackup Thread] [] extensions - begin restoreFromExternalBackup
    2012-05-12 00:34:45,616 WARN [ZxBackup Thread] [] extensions - ZxBackup Thread: java.lang.NullPointerException
    at com.zextras.backup.ZEBackupServer.getBackup(ZEBack upServer.java:250)
    at com.zextras.backup.ZEExternalRestoreOperation.doOp eration(ZEExternalRestoreOperation.java:320)
    at com.zextras.op.ZEOperation.exec(ZEOperation.java:5 81)
    at com.zextras.backup.ZEBackupThread.execOperation(ZE BackupThread.java:200)
    at com.zextras.backup.ZEBackupThread.run(ZEBackupThre ad.java:110)

    ZxBackup version 1.2.7 (latest) .

    I tried chmodding and chowning to user zimbra and to user root and I keep getting this error. Please advise.

    [zimbra@email bin]$ ./zxsuite backup doExternalRestore /restore/nas


    server email.censored.com
    operationId a2d67a38-9909-4b0f-9fd2-23b0a28a21fa
    I tried running as root and as zimbra.

    Here are the contents of /restore/nas :

    [root@email bin]# ls /restore/nas
    accounts backupstat dirtyList items lock_8c6a7eb4-41bb-4947-8ffd-e9f987ecc5b1 server
    Here is debug data:

    2012-05-12 00:43:11,915 INFO [ZxBackup Thread] [] extensions - ZxBackup Thread: New Operation: External Restore
    2012-05-12 00:43:11,915 INFO [ZxBackup Thread] [] extensions - ZeXtras Notifier: Sending notification: This is an automated notification from ZxBackup about External Restore.

    Operation External Restore Started.

    Operation Id: de6df55b-2fc5-471e-8b36-720bf7cb3133
    Operation Host: email.censored.com
    Operation Log path: /opt/zimbra/log/zextras_ZxBackup_ExternalRestore_de6df55b-2fc5-471e-8b36-720bf7cb3133.log

    - debug info -
    -- parameters --
    humanDate = last available
    notificationeMails =
    accountNameList = null
    backupPath = /restore/nas
    date = null
    domainNameList = null
    -- context --

    2012-05-12 00:43:12,014 INFO [ZxBackup Thread] [] extensions - begin restoreFromExternalBackup
    2012-05-12 00:43:12,015 INFO [ZxBackup Thread] [] extensions - ZeXtras Notifier: Sending notification: This is an automated notification from ZxBackup about External Restore.

    Operation External Restore failed.

    Operation Id: de6df55b-2fc5-471e-8b36-720bf7cb3133
    Operation Host: email.censored.com

    - exception -
    java.lang.NullPointerException
    at com.zextras.backup.ZEBackupServer.getBackup(ZEBack upServer.java:250)
    at com.zextras.backup.ZEExternalRestoreOperation.doOp eration(ZEExternalRestoreOperation.java:320)
    at com.zextras.op.ZEOperation.exec(ZEOperation.java:5 81)
    at com.zextras.backup.ZEBackupThread.execOperation(ZE BackupThread.java:200)
    at com.zextras.backup.ZEBackupThread.run(ZEBackupThre ad.java:110)

    - debug info -
    -- parameters --
    humanDate = last available
    OpStartTime = 1336776192013
    notificationeMails =
    accountNameList = null
    backupPath = /restore/nas
    date = null
    domainNameList = null
    -- context --
    completed = 1

    2012-05-12 00:43:12,110 WARN [ZxBackup Thread] [] extensions - ZxBackup Thread: java.lang.NullPointerException
    at com.zextras.backup.ZEBackupServer.getBackup(ZEBack upServer.java:250)
    at com.zextras.backup.ZEExternalRestoreOperation.doOp eration(ZEExternalRestoreOperation.java:320)
    at com.zextras.op.ZEOperation.exec(ZEOperation.java:5 81)
    at com.zextras.backup.ZEBackupThread.execOperation(ZE BackupThread.java:200)
    at com.zextras.backup.ZEBackupThread.run(ZEBackupThre ad.java:110)
    Last edited by iamcypher; 05-11-2012 at 11:44 PM.

  6. #6
    ZeXtras Community Manager ZeXtras Employee Cine's Avatar
    Join Date
    Apr 2011
    Posts
    2,356
    Hello iamcypher,

    the error you experienced is caused by an I/O error on the "server" subfolder of your backup.

    I see 2 possible scenarios:

    * The "zimbra" user lacks read permissions on the "server" subfolder (or on the entire /restore/nas/ folder)
    * The "server" subfolder is corrupted

    Please check if the Zimbra user has read permissions on the /restore/nas/ folder and in all its subfolders...
    If permissions are ok, please check the "server" folder itself to see if there is anything abnormal (try running "ls -la /restore/nas/", "ls -la /restore/nas/server/", "touch /restore/nas/server/testfile", "rm /restore/nas/server/testfile" and post the outputs).

    Have a nice day,
    Cine

  7. #7
    Member
    Join Date
    May 2012
    Posts
    46
    Hello,

    Thank you for your answer. It started restoring now however it's skipping a LOT of emails because he says that they don't exist. My log is being spammed with a lot of restoring items and every 3 or 4 successful restores I get a missing file:

    2012-05-12 09:37:53,010 WARN [ZxBackup Thread] [] extensions - Unable to restore item 272, account admin@censored.com(708b0c35-129d-4953-8708-fdfc5deb0b33):
    Unknown Exception in restore
    at com.zextras.backup.ZEBItem.restore(ZEBItem.java:56 9)
    at com.zextras.backup.ZEExternalRestoreOperation.rest oreItemsFromExternal(ZEExternalRestoreOperation.ja va:584)
    at com.zextras.backup.ZEExternalRestoreOperation.doOp eration(ZEExternalRestoreOperation.java:474)
    at com.zextras.op.ZEOperation.exec(ZEOperation.java:5 81)
    at com.zextras.backup.ZEBackupThread.execOperation(ZE BackupThread.java:200)
    at com.zextras.backup.ZEBackupThread.run(ZEBackupThre ad.java:110)
    Caused by: Unable to restore MailItem
    at com.zextras.backup.ZEBItem.restore(ZEBItem.java:55 8)
    ... 5 more
    Caused by: java.io.FileNotFoundException: /restore/nas/items/dm/dmWlDXju+qFJQh8bPNXQ00kkrOI= (No such file or directory)
    at java.io.FileInputStream.open(Native Method)
    at java.io.FileInputStream.<init>(FileInputStream.jav a:120)
    at com.zextras.backup.ZEBItem.createZimbraItem(ZEBIte m.java:1224)
    at com.zextras.backup.ZEBItem.restore(ZEBItem.java:52 9)
    ... 5 more
    A LOT OF THEM.

    Is this possible? I mean how come he is missing so many files? I am sure I copied a verbatim copy of whatever was saved on the nas device.

    Does this mean that only a part of my emails will get restored or they are false positive alerts?

    Thank You.

    Edit: I just checked and the file in the exception /restore/nas/items/dm/dmWlDXju+qFJQh8bPNXQ00kkrOI= exists, belongs to user zimbra (uid 500) and to group zimbra (gid 500) and is has chmod 0777.

    Here is the output you requested:

    [zimbra@email bin]$ ls -la /restore/nas
    total 56
    drwxrwxrwx. 5 zimbra zimbra 4096 May 11 21:15 .
    drwxr-xr-x. 3 zimbra zimbra 4096 May 12 00:19 ..
    drwxrwxrwx. 62 zimbra zimbra 4096 May 11 20:45 accounts
    -rwxrwxrwx. 1 zimbra zimbra 353 May 11 20:45 backupstat
    -rwxrwxrwx. 1 zimbra zimbra 296 May 11 20:45 dirtyList
    drwxrwxrwx. 1446 zimbra zimbra 32768 May 11 21:15 items
    -rwxrwxrwx. 1 zimbra zimbra 0 May 11 21:15 lock_8c6a7eb4-41bb-4947-8ffd-e9f987ecc5b1
    drwxrwxrwx. 2 zimbra zimbra 4096 May 11 21:15 server
    [zimbra@email bin]$ ls -la /restore/nas/server
    total 464
    drwxrwxrwx. 2 zimbra zimbra 4096 May 11 21:15 .
    drwxrwxrwx. 5 zimbra zimbra 4096 May 11 21:15 ..
    -rwxrwxrwx. 1 zimbra zimbra 75957 May 11 21:15 backup_10_05_12#11_01_27
    -rwxrwxrwx. 1 zimbra zimbra 75957 May 11 21:15 backup_10_05_12#11_01_36
    -rwxrwxrwx. 1 zimbra zimbra 75994 May 11 21:15 backup_10_05_12#23_28_27
    -rwxrwxrwx. 1 zimbra zimbra 75993 May 11 21:15 backup_11_05_12#00_06_00
    -rwxrwxrwx. 1 zimbra zimbra 75993 May 11 21:15 backup_11_05_12#00_06_10
    -rwxrwxrwx. 1 zimbra zimbra 75993 May 11 21:15 backup_11_05_12#04_01_12
    [zimbra@email bin]$ touch /restore/nas/server/testfile
    [zimbra@email bin]$ rm -f /restore/nas/server/testfile
    Edit2: I correct myself the file does not exist, there is a small error:

    /restore/nas/items/dm/dmWlDXju+qFJQh8bPNXQ00kkrOI=

    actually is

    /restore/nas/items/dM/dmWlDXju+qFJQh8bPNXQ00kkrOI=

    What could possibly cause this error in the zextras indexes?

    Edit3: I noticed someone else was having a simillar problem, is there any chances as this is a known problem and there is a fix for it?

    Edit4: Result:

    The result is unacceptable. Please advise.
    Last edited by iamcypher; 05-12-2012 at 09:50 AM.

  8. #8
    ZeXtras Community Manager ZeXtras Employee Cine's Avatar
    Join Date
    Apr 2011
    Posts
    2,356
    Hello iamcypher,

    we are checking the issue as we speak and we'll provide a solution as soon as possible.

    Have a nice day,
    Cine

  9. #9
    ZeXtras Community Manager ZeXtras Employee Cine's Avatar
    Join Date
    Apr 2011
    Posts
    2,356
    Hello iamcypher.

    A quick question: was the NAS mounted as a CIFS share when you performed the Full Scan operation?

  10. #10
    Member
    Join Date
    May 2012
    Posts
    46
    Hello,

    I was using the your advice to store all the backup data on a cifs share and yes the full scan happened on a cifs share not locally. Actually all the backup data was stored on a cifs share. When I did the restore I copied all the data from the cifs share back to the computer.

Page 1 of 2 12 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •