Unexpected Blobs in zmblobchk
Results 1 to 8 of 8

Thread: Unexpected Blobs in zmblobchk

  1. #1
    Member
    Join Date
    May 2012
    Posts
    46

    Unexpected Blobs in zmblobchk

    Hello,

    I'm planning a cleanup on my email server and i'm seeing 17k+ of unexpected blobs in the blob check process. I need some information on this, what causes unexpected blobs and is it safe to delete them?

    Thank You,
    Paul.

    Sample output:

    Mailbox 6, volume 4, /home/store/0/6/msg/0/1658-1993.msg: unexpected blob. File size is 16212.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/2073-2356.msg: unexpected blob. File size is 48530.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/2682-2892.msg: unexpected blob. File size is 2312.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/2186-2455.msg: unexpected blob. File size is 186145.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/1075-1471.msg: unexpected blob. File size is 7778.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/1768-2088.msg: unexpected blob. File size is 85919.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/2505-2742.msg: unexpected blob. File size is 261357.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/1839-2151.msg: unexpected blob. File size is 2808214.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/2310-2564.msg: unexpected blob. File size is 1002.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/3124-3302.msg: unexpected blob. File size is 222642.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/2320-2572.msg: unexpected blob. File size is 39664.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/1983-2277.msg: unexpected blob. File size is 892702.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/1478-1838.msg: unexpected blob. File size is 44148.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/1892-2200.msg: unexpected blob. File size is 71605.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/2703-2910.msg: unexpected blob. File size is 6927.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/2061-2344.msg: unexpected blob. File size is 54776.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/2461-2701.msg: unexpected blob. File size is 90434.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/2376-2624.msg: unexpected blob. File size is 2347.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/1350-1722.msg: unexpected blob. File size is 19414.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/2721-2925.msg: unexpected blob. File size is 45392.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/2189-2458.msg: unexpected blob. File size is 201283.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/3355-3555.msg: unexpected blob. File size is 11467.
    Mailbox 6, volume 4, /home/store/0/6/msg/0/1465-1825.msg: unexpected blob. File size is 3624.
    I did lookup on these wiki: Ajcody-Notes-No-Such-Blob - Zimbra :: Wiki
    and these posts: Total Innodb/MySQL failure No such blob -> delete references

    I'm still left in a complete void. I have no idea how to fix these problems, and I would rather not loose about 17k emails from my server.

  2. #2
    ZeXtras Community Manager ZeXtras Employee Cine's Avatar
    Join Date
    Apr 2011
    Posts
    2,356
    Hello iamcypher!

    Basically, an "unexpexted blob" is a file in the Zimbra Store which is correctly not referenced in the Zimbra database (the opposite situation is the "no such blob").
    The causes for such an error are many, for example:
    - Incomplete item deletion caused
    - Orphaned Briefcase file revisions
    - Corrupted database entry
    - Zimbra update gone wrong
    - Zimbra "community script" migration gone wrong

    Is it safe to delete them? I'd say "maybe", and that's an optimistic answer.

    First of all, I suggest you to run a permission fix (run `/opt/zimbra/libexec/zmfixperms --verbose --extended` as root, will take quite some time) and a database integrity check (run `/opt/zimbra/libexec/zmdbintegrityreport -m -v` as the zimbra user). Then re-run the blob check utility and see if there is any improvement.

    If there is no improvement, I suggest you to run some sample checks picking random "unexpected blob" files, looking at the content with `less` or `zless` and trying to find out where the item should be (e.g.: the missing blobs in the output extract you posted all belong to the user with mailboxId=6, as the parent directory of the "msg/" directory is named after the owner's mailboxId. To see the mailbox id of a user run `zmprov gmi user@yourdomain` as the zimbra user). Check if you can find the items in the proper mailbox.. If you don't, it's quite safe to delete the file while if you do find the item but experience some display/content issue then the deletion becomes a lot riskier.
    In the best case scenario, you'll see a common pattern in the files and you will be able to identify the issue causing the unexpected blobs.

    Unfortunately everything revolving around BLOB issues is very tied to the history and use of the server, so I'm not able to provide any targeted suggestion or straightforward solution, but if you find any additional information about the issue feel free to post it here so that I'll try to narrow down the options.

    Have a nice day,
    Cine

    p.s.: this is a quick one-liner that prints all accounts and the related mailbox id, run as the zimbra user and redirect the output to a file in order to make it permanent.
    Code:
    for i in `zmprov -l gaa`; do echo "$i `zmprov gmi $i`" | grep mailboxId;  done

  3. #3
    Member
    Join Date
    May 2012
    Posts
    46

    Unexpected Blobs in zmblobchk

    Hello Cine,

    Thank you so much for your reply, I will try it out.

    I think this might be caused by the failed import I've had in the 'epic data loss' thread. I eventually quit importing using zextras and pulled up the data from pst files on each computer. Is it possible that the failed import to copy only blob messages and skip registering them?

    Paul.

    Edit: I've done a mysql query before to get the ids and email because I didn't know other way:

    use zimbra
    select id,comment from mailbox;

    And in comment there was the email address
    Last edited by iamcypher; 10-15-2012 at 12:42 PM.

  4. #4
    ZeXtras Community Manager ZeXtras Employee Cine's Avatar
    Join Date
    Apr 2011
    Posts
    2,356
    Hello Paul,

    it is very unlikely that your previous restore issues caused this, as ZeXtras Backup works at a higher level by interacting with Zimbra itself to restore the items instead of "forcing" them into the DB/Store.
    I'll run some tests tho, just to be sure

    Have a nice day,
    Cine

  5. #5
    Member
    Join Date
    May 2012
    Posts
    46
    Hello Cine,

    I've went ahead and moved all the unexpected blobs to some folder and I've done a couple of mistakes and lost about 3.4k of blobs, but the good part is that they were not needed. Now blobchk is running with no problems at all.

    However when I tried to export using zextras I got:

    additional notification mails:
    Exceptions: item: 23369 on account claudio@mydomain.com
    reason:Unable to access zimbra file: ruNsf2DChw8WHPyHgnOhKiue32c=
    at com.zextras.backup.ZEBItem.synchronize(ZEBItem.jav a:941)
    at com.zextras.backup.ZEBAccount.synchronize(ZEBAccou nt.java:1225)
    at com.zextras.backup.ZEScanOperation.doItemsScan(ZES canOperation.java:366)
    at com.zextras.backup.ZEScanOperation.doAllItemsScan( ZEScanOperation.java:323)
    at com.zextras.backup.ZEScanOperation.doAccountScan(Z EScanOperation.java:279)
    at com.zextras.backup.ZEScanOperation.doDomainScan(ZE ScanOperation.java:60)
    at com.zextras.backup.ZEScanOperation.doDomainScan(ZE ScanOperation.java:51)
    at com.zextras.backup.ZEExportOperation.doOperation(Z EExportOperation.java:329)
    at com.zextras.op.ZEOperation.exec(ZEOperation.java:5 81)
    at com.zextras.backup.ZEBackupThread.execOperation(ZE BackupThread.java:200)
    at com.zextras.backup.ZEBackupThread.run(ZEBackupThre ad.java:110)
    Caused by: com.zimbra.cs.mailbox.MailServiceException: No such blob: mailbox=6, item=23369, change=50334
    ExceptionId:ZxBackup Thread:1350298670426:ebd15a725c866b1b
    Code:mail.NO_SUCH_BLOB
    at com.zimbra.cs.mailbox.MailServiceException.NO_SUCH _BLOB(MailServiceException.java:269)
    at com.zimbra.cs.mailbox.MailItem.getBlob(MailItem.ja va:971)
    at com.zimbra.cs.mailbox.MailItem.getContentStream(Ma ilItem.java:990)
    at com.zextras.backup.ZEBItem.synchronize(ZEBItem.jav a:934)
    ... 10 more
    I did a search for item: 23369 on that mailbox and no results were found. Any ideas?

    Paul.

  6. #6
    ZeXtras Community Manager ZeXtras Employee Cine's Avatar
    Join Date
    Apr 2011
    Posts
    2,356
    Hello Paul,

    a similar issue has been discussed HERE, it is probably a legitimate deletion happened during the Export operation...
    In this case you can ignore the error, but I suggest you to check the mailbox.log file to see if there has been any deletion or purge on the claudio@mydomain.com mailbox during the export.

    Have a nice day,
    Cine

  7. #7
    Member
    Join Date
    Nov 2012
    Posts
    26
    I know this is a VERY old thread but it is related enough to my question that I figured it would be nice to keep it together. I ran doCheckBlobs on my 8.6 zimbra in preparation for upgrading to 8.8. I have a couple dozen Unexpected Blobs but many of the .msg files referenced in the unexpected blob message don't exist. What should I do in that case? I have run the permissions fix and checked the integrity of the DB.

    I have just a couple of Missing Blobs and 3 Unexpected mailbox directories. What is to be done about those or will they not necessarily cause problems upgrading?

    Stephen

  8. #8
    Member
    Join Date
    Nov 2012
    Posts
    26
    @Cine ^ Not sure if vBulletin supports @mentions

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •