Please help, problem migrating.

[fishbros]

Ok So I have probably caused my own issue here but.
I am migrating my users from ZCS 6.0.9 NE to 8.6.0 NE and thought all was going well.
I had created a migration point on the old server using Migration tool on 21/4 and imported that on to the new server - all seemed fine.
I then after switching mailflow on 01/05 started a backup on the old server using the zextras backup utility which although it took its time completed ok.
I then started to restore the backup on to the new server yesterday but have noticed on the accounts I have so far restored it is creating duplicate items - contacts, emails in folders and seems generally to be duplicating everything prior to 01/05.
Have I done something wrong ? I have only restored 28 accounts out of 175 so far in this way.
I still have access to the old server but the new one is in a production environment now.

Begging for assistance please.

[fishbros]

I picked up a post elsewhere on the forum from Cine (hope he is reading) and wondered if I added some info to my scenario would this help.

Does the data I am importing have to be in the same location on the new server that it was on the old one ?

I (due to time & I/O constraints having previously tried to copy the data sideways) tar'd the backup folder to a usb drive (156gb file) and then extracted that on to the usb drive and have then tried to restore it from there.

[Cine]

Hello Fishbros,
welcome to the forums!

The actual import data location does not affect the External Restore process, as it uses a mapfile system to keep track of accounts and items that have already been imported. All of those mapfiles are created in the destination server's Backup Path during the import: did you change or clear out the backup path between imports?

Have a nice day,
Cine
the ZeXtras Team

[fishbros]

A big thanks for the reply.
I cant remember what location I used for the initial migration export (21/04) but I think I created a folder /mig/zcs, the one I used for the backup(smartscan) (01/05) was /opt/zimbra/backup/zextras and I am trying to restore from /mnt/zextras/zextras is that likely to be my issue ?

Just checked folder locations in the ZxNotifications tab on the old server and yes thats the locations I used.

[Cine]

Hello Fishbros!

Import locations are ok, the issue might be on the destination server's "backup path" (default /opt/zimbra/backup/zextras/): did you change it after the first migration or did you delete the files it contained?

[fishbros]

Thanks again Cine.
The path showing in the destination server is /tmp. I have a feeling I may changed it after doing the initial restore from the migration tool but I cna seem able to check - as a lazy admin used GUI to do operations, fingers burnt all done from CLI from now on if possible.

[fishbros]

Just checked, I think I deleted the folder of the original migration due to limited space

[fishbros]

Is there a log file for Zextras that I could check to see if I did change it please ?
Or...
If I was to create a new Zextras backup on the new server and then copy the old backup to the same location would that work ?
Or...
If I moved the old backup to the location /tmp and then ran a smartscan to /tmp would that help ?
Please help I am running out of time to do this.

[Cine]

Hello Fishbros!

Restore logs can be found in /opt/zimbra/log/, and there you can find the item IDs of all items restored in the second pass and thus duplicated with

cat /opt/zimbra/log/zextras_ZxBackup_ExternalRestore_OPERATIONID | grep -i "Adding Item" | awk {'print $8'}

(this is just a basic search, make sure to manually check the items returned or crosscheck those with the logs of the first restore)

You can then remove those using zmmailbox if needed, but if the original maps have been deleted there is no way to reprise an incremental migration...

[fishbros]

Thankyou again.
Dang and blast looks like I am going to have to accept defeat on this thanks all the same. I have found a python script (https://github.com/quentinsf/IMAPdedup) that uses IMAP access to deduplicate within the mail folders (exceptions being Contacts and Recent & Suggested Contacts) and with a bit of bash scripting from another (http://news.numlock.ch/tag/imapdedup) and then fiddling by my very novice hands modifying someone else's basic bash script come up with an almost viable solution, well slightly less painful than going through everyones mail folders and manually deduping anyway.
I would be quite happy to post the code bash script I butchered to help automate the process if wanted but I will post the link as its not all my work and I dont intend to take rights for someone elses intellectual efforts.