ZxPowerstore: Checking the message BLOBs health

From ZeXtras Suite Wiki

Jump to: navigation, search
ZxPowerStore logo box.png
Available since version: 1.3.0
Latest Version: 2.12.2
Released on: January 2nd, 2019
Compatibility List
Admin Guide
FAQ
Troubleshooting
ZspPowerstore logo box.png
Language: English  • español • português

Volumes and BLOBs

In Zimbra, all message data is stored into the so-called "Message Volumes" in the form of .msg files archived in a strict directory tree.

Every item in Zimbra is composed by 2 parts: the "Data", which is the actual content of the item, and the "Metadata" which is a set of additional informations regarding the item. A BLOB file is a file containing the "Data" part of an item, stored in a Message Volume. The exact path of the file depends on different factors.

BLOB path example

The /opt/zimbra/store/0/128/msg/2/12124-23531.msg file's location and name are calculated as follows

/opt/zimbra/store/ is the volume's path.

0/ is a number calculated by applying a right bitshift on the mailbox_id of the owner of the item of a number of bits set in the "mailbox_bits" property of the volume.

128/ is the mailbox_id of the owner of this item.

msg/ is a default subfolder for message BLOBs.

2/ is a number calculated by applying a right bitshift on the itemID of the item of a number of bits set in the "file_bits" property of the volume.

12124-23531.msg is the name of the file in the [itemID]-[revision].msg form.

What can go wrong

Different issues can affect the health of a volume. The most common are:

  • Missing BLOB: Metadata for an item are stored in Zimbra's database but the corresponding BLOB file cannot be found.
  • Unexpected BLOB: A BLOB file exists but the corresponding metadata cannot be found in Zimbra's database.
  • Size Mismatch: Item size informations stored in the metadata are different than the corresponding BLOB file's size.
  • Digest Mismatch: Item digest informations stored in the metadata are different than the corresponding BLOB file's digest.

"Unexpected BLOB" errors are commonly caused by a problem in the deletion process of an item.

"Missing BLOB" errors are particularily pesky, as since the item's metadata exists in Zimbra's database this very item is usually visible in the Zimbra Web Client. An error will be shown to the user when trying to access an item which BLOB file is missing.

Checking the BLOB status with ZeXtras Powerstore

ZeXtras Powerstore includes a CLI command that analyzes all BLOBs in a volume or mailbox to find out any possible issue: "doCheckBlobs"

Syntax:
   zxsuite powerstore doCheckBlobs {start} [attr1 value1 [attr2 value2...]]


PARAMETER LIST

NAME                                   TYPE            EXPECTED VALUES       DEFAULT
action(M)                              String          start                 
volumes(O)                             String[,..]     message1,hsm[,...]    
mailbox_ids(O)                         Integer[,..]    2,9,27                
missing_blobs_crosscheck(O)            Boolean         true|false            true
traced(O)                              Boolean         true|false            false
fix_incorrect_compressed_digests(O)    Boolean         true|false            false

(M) == mandatory parameter, (O) == optional parameter

Usage example:

Usage examples:
zxsuite powerstore doCheckBlobs start
Perform a BLOB coherency check on all message volumes

zxsuite powerstore doCheckBlobs start volumes message1,hsm
Perform a BLOB coherency check on volumes message1 and hsm

zxsuite powerstore doCheckBlobs start mailbox_ids 2,9,27
Perform a BLOB coherency check on mailboxes 2,9 and 27

zxsuite powerstore doCheckBlobs start missing_blobs_crosscheck false
Perform a BLOB coherency check without checking on other volumes

zxsuite powerstore doCheckBlobs start traced true
Perform a BLOB coherency check, logging even the correct checked items

Volume names are case sensitive. Run 'zxsuite powerstore getAllVolumes' for a complete list of the volumes.

Fixing BLOB errors

Zimbra offers a CLI command to fix some BLOB issues: `zmblobchk` (Before running this utility, remember to stop ZxPowerstore service with `zxsuite powerstore doStopService module`).

Also, one of the most common causes of BLOB/volume issues is bound to filesystem permissions, which can be fixed using the `zmfixperms` Zimbra CLI command.

A great article about "Missing BLOB" errors and all related informations can be found on the Zimbra Wiki at http://wiki.zimbra.com/wiki/Ajcody-Notes-No-Such-Blob.

Extreme caution is advised when dealing with this kind of issues, as dealing with volumes and BLOB files level means dealing with Zimbra's "raw data". Before taking any action make sure that you find out the cause of the issue and that you have a backup of your system in order to easily revert any possible data loss.

Personal tools