zmblobchk: org.openzal.zal.extension.InternalOverrideStoreMan ager is not support

[orsomannaro]

Hi all.

I seem that I'm the first one having this error.

Running zmblobchk start -v on Zimbra 8.0.6 (Release 8.0.6.GA.5922.UBUNTU12.64 UBUNTU12_64 FOSS edition) I have:

[] INFO: Setting mysql connector property: maxActive=100
[] INFO: Setting mysql connector property: maxActive=100
Checking mailbox 1.
com.zimbra.common.soap.SoapFaultException: invalid request: org.openzal.zal.extension.InternalOverrideStoreManager is not supported
ExceptionId:qtp1303093998-21457:https://127.0.0.1:7071/service/admin/soap/CheckBlobConsistencyRequest:1473844978023:6eff4b624e9fdb26
Code:service.INVALID_REQUEST
        at com.zimbra.common.soap.Soap12Protocol.soapFault(Soap12Protocol.java:88)
        at com.zimbra.common.soap.SoapTransport.extractBodyElement(SoapTransport.java:333)
        at com.zimbra.common.soap.SoapTransport.parseSoapResponse(SoapTransport.java:308)
        at com.zimbra.common.soap.SoapHttpTransport.invoke(SoapHttpTransport.java:255)
        at com.zimbra.common.soap.SoapHttpTransport.invoke(SoapHttpTransport.java:167)
        at com.zimbra.common.soap.SoapTransport.invoke(SoapTransport.java:386)
        at com.zimbra.common.soap.SoapTransport.invokeWithoutSession(SoapTransport.java:372)
        at com.zimbra.cs.account.soap.SoapProvisioning.invokeRequest(SoapProvisioning.java:442)
        at com.zimbra.cs.account.soap.SoapProvisioning.invoke(SoapProvisioning.java:450)
        at com.zimbra.cs.store.file.BlobConsistencyUtil.checkMailbox(BlobConsistencyUtil.java:244)
        at com.zimbra.cs.store.file.BlobConsistencyUtil.run(BlobConsistencyUtil.java:201)
        at com.zimbra.cs.store.file.BlobConsistencyUtil.main(BlobConsistencyUtil.java:328)

Can someone help me to solve the problem?

TIA

[Cine]

Hello Orsomannaro,
welcome to the forums!

Could you please let me know the Zextras Migration Tool version you have installed?

Have a nice day,
Cine
the Zextras Team

[orsomannaro]

Hi Cine.

Thank very much for your help.

Could you please let me know the Zextras Migration Tool version you have installed?

I want to perform an incremental migration Backup.

So I have uninstalled Zextras Migration Tool to install the last version of ZeXtras Suite 2.4.0 (ups... maybe I'm writing on the wrong thread... :/)

[Cine]

Hello Orsomannaro!

Thanks for the additional information, I'll run some tests and report back to you as soon as possible!
In the meantime, could you please check whether the same error happens with the `zxsuite powerstore doCheckBlobs` command?

Have a nice day,
Cine
the Zextras Team

P.s. Don't worry, I'll move the post to the appropriate section...

[orsomannaro]

It seem all is ok:

Volumes to check: all
Mailboxes to check: all
Search all volumes: true
Traced: false


2016-09-14 14:04:23,318 INFO Checking volume: 1 - message1
2016-09-14 14:04:23,623 INFO Checking mailbox 2 for account account1@mydomain.com
2016-09-14 14:04:24,812 INFO Checking mailbox 3 for account account@mydomain.com
...
2016-09-14 15:33:12,821 INFO Checking Store Finished

Many thanks!

[orsomannaro]

I tryed to run the migration. All seem ok, but some mailbox have different folders named "UnknownFolder_xxxx" (xxxx is a number)...

[orsomannaro]

some mailbox have different folders named "UnknownFolder_xxxx" (xxxx is a number)...

I found that rsync created a file lock_3cb2d1bb-6921-42f3-81f1-8fb2766bd6da instead of map_3cb2d1bb-6921-42f3-81f1-8fb2766bd6da

Running the zxsuite backup doFixShares command with the correct map_ file, I have this error:

Operation Shares Fix failed.

...

- exception -
com.zextras.lib.Error.EmptyMergedMapError: Merged map is empty, check logs
        at com.zextras.op.backup.ZEFixShares.doOperation (ZEFixShares.java:266)
        at com.zextras.op.ZEOperation.exec (ZEOperation.java:757)
        at com.zextras.op.LegacyOperationProxy.exec (LegacyOperationProxy.java:83)
        at com.zextras.op.StatefulOperationImpl.exec (StatefulOperationImpl.java:115)
        at com.zextras.op.OperationStarterActivity.run (OperationStarterActivity.java:45)
        at com.zextras.lib.activities.ActivityThread.run (ActivityThread.java:122

[orsomannaro]

After the end of the first SmartScan I have this directory tree:

ls -l /opt/zimbra/backup/zextras/
drwxr-x---   22 zimbra zimbra  4096 set 19 18:40 accounts
-rw-r-----    1 zimbra zimbra   601 set 19 19:34 backupstat
drwxr-x--- 4098 zimbra zimbra 69632 set 19 17:27 items
-rw-r-----    1 zimbra zimbra     0 set 19 16:49 lock_3cb2d1bb-6921-42f3-81f1-8fb2766bd6da
drwxr-x---    2 zimbra zimbra  4096 set 19 19:34 server

My question is: why the smartscan script didn't delete the lock file lock_3cb2d1bb-6921-42f3-81f1-8fb2766bd6da?


TIA

[Cine]

Hello Orsomannaro!

The "lock_[serverID]" file is created by Zextras Suite when a backup path is initialized to avoid multiple initializations of the same from different servers, which would break the backup's consistency, while the "map_[serverID]" should have been created on the destination server's backup path (or in the default /opt/zimbra/backup/zextras/ if the backup path is not initialized), so the doFixShares command should be executing using the latter as a reference.

The "UnknownFolder" issue happens when the folder's data in the backup is corrupted or not accessible: I'd suggest to run a "deep" SmartScan - `zxsuite backup doSmartScan deep TRUE` on the source server to make sure the backup data is fully healthy, but unfortunately once the items have been restored running the restore again won't fix the folder names...

Have a nice day,
Cine
the Zextras Team

[orsomannaro]

the "map_[serverID]" should have been created on the destination server's backup path (or in the default /opt/zimbra/backup/zextras/ if the backup path is not initialized), so the doFixShares command should be executing using the latter as a reference

I Cine.

I rebuilded the destination Zimbra server and ran an export/import process.

Now all is ok, even the Shares.

But...

zxsuite backup doCheckShares

return some problems and when I run:

zxsuite backup doFixShares /opt/zimbra/backup/zextras/map_3cb2d1bb-6921-42f3-81f1-8fb2766bd6da

I still have the EmptyMergedMapError exception:

- exception -
com.zextras.lib.Error.EmptyMergedMapError: Merged map is empty, check logs
        at com.zextras.op.backup.ZEFixShares.doOperation (ZEFixShares.java:266)
        at com.zextras.op.ZEOperation.exec (ZEOperation.java:757)
        at com.zextras.op.LegacyOperationProxy.exec (LegacyOperationProxy.java:83)
        at com.zextras.op.StatefulOperationImpl.exec (StatefulOperationImpl.java:115)
        at com.zextras.op.OperationStarterActivity.run (OperationStarterActivity.java:45)
        at com.zextras.lib.activities.ActivityThread.run (ActivityThread.java:122)