First make yourself a backup of the entire current /opt/zimbra (and any linked directories).
Is everything else fine, just the store is bad? Restore as much as you can into /opt/zimbra/store from backups.
Start by making sure permissions are fixed:
su - root
chown -R zimbra:zimbra /opt/zimbra/ (might just do this on the /opt/zimbra/store if everything else is fine)
cd /opt/zimbra/libexec/
./zmfixperms (not needed if you're saying it's just the store that's bad)
Ok, so there's the MySQL metadata DB, the Lucene index, and the actual blob files on disk in /opt/zimbra/store. If you can click on a folder in the web UI and get results, it means MySQL is up and happy. If you open a message and get the error, it means that the message files on disk aren't where they're supposed to be.
If you're managed to restore a bunch try re-indexing first, but you're probably gonna have to remove some stuff from the DB (example">
http://www.zimbra.com/forums/administra ... ml>example) if your absolutely missing those blobs (the previously stated 2 days of email) then re-index again.
In 5.0.6 there's Bug">
http://bugzilla.zimbra.com/show_bug.cgi?id=19927>Bug 19927 - tool to do consistency checks and repair for missing blob for ID x while the mailbox.log "com.zimbra.common.service.ServiceException: system failure: missing blob for id: 19512, change: 67500" can tell you just as much, zmblobchk is needed because you don't get all those errors in the log in an order, you get them when accessed, and you might want to spit out a list.
In 5.0.8 (target) this will take all the tough work out of it: Bug">
http://bugzilla.zimbra.com/show_bug.cgi?id=27958>Bug 27958 - Add support for 'repair-mode' to the zmblobchk tool
I would read Account">
http://wiki.zimbra.com/index.php?title= ... re>Account mailbox database structure - Zimbra :: Wiki so you understand the below.
Below is a Perl script that will reconcile the user's database to account for the missing blob(s). Run it like this: ./mailboxfixdb455.pl
user@domain.com
The script will remove references to the missing blob. After running the script, it'll probably be a good idea to run reindex again just to make sure things are set.
mailboxfixdb.pl:
#!/usr/bin/perl
# This script compatible with Zimbra version 4.5.x only. Do not use with any other version.
# OK, there's 2 MAILBOX_*_BITS values in the VOLUME table.
# Take the mailbox ID, right-shift it by MAILBOX_BITS, and take the lowest MAILBOX_GROUP_BITS of the result.
# That's your mailbox hash.
# Take the message ID, right-shift it by FILE_BITS, and take the lowest FILE_GROUP_BITS of the result. That's your msgid hash.
# I think.
# //msg//-.msg
my ($fbits, $fgbits, $mbits, $mgbits, $basepath) = split (' ',`echo "select file_bits,file_group_bits,mailbox_bits,mailbox_group_bits,path from volume where type='1'" | mysql -N zimbra`);
my $ARGV = shift @ARGV;
chomp $basepath;
my $mbmask = sprintf "1" x $mgbits;
my $fmask = sprintf "1" x $fgbits;
foreach (`echo "select id, group_id, account_id, comment from mailbox where comment = '$ARGV'" | mysql -N zimbra`) {
chomp;
my $path = "$basepath/";
my ($id, $grid, $aid, $nm) = (split);
my $mbhash = $id >> $mbits;
$mbhash &= $mbmask;
$path .= $mbhash."/".$id."/msg/";
foreach my $msgstuff (`echo "select id, mod_content, type from mail_item where blob_digest is not null and mailbox_id=${id};" | mysql -N mboxgroup${grid}`) {
chomp $msgstuff;
if ($msgstuff eq "") {next;}
my ($msgid, $modContent, $type) = split (' ',$msgstuff);
my $msghash = $msgid >> $fbits;
# $msghash &= $fmask;
my $nm = $msgid;
if ($modContent) {$nm .= "-$modContent";}
my $npath = $path.$msghash."/".$nm.".msg";
if (-e $npath) {print $npath." OK
";}
else {
print $npath." NOT OK
";
# not ok, remove the entry from the database so it is not a nuisance
print "Delete from mail_item where MSGID is ${msgid} and MAILBOXID is ${id}
";
print "Uncomment line below me in script to have me delete.
";
# `echo "delete from mail_item where id=${msgid} and mailbox_id=${id}" | mysql -N mboxgroup${grid}`;
if ($type eq "11") {
print "Delete from appointment where MSGID is ${msgid} and MAILBOX_ID is ${id}
";
print "Uncomment line below me in script to have me delete.
";
# `echo "delete from appointment where item_id=${msgid} and mailbox_id=${id}" | mysql -N mboxgroup${grid}`;
}
}
# print $npath."
";
}
}
There are several ways to reindex:
The admin console GUI. (Which might be time consuming if you have a large # of accounts.)
http://i30.tinypic.com/izqgsm.jpg />
SOAP ReIndexRequest on each account (starts a java thread to re-index).
BTW when you do upgrade (I haven't read up on your other threads to find out why your mgt is objecting but it seems you sorted out
http://www.zimbra.com/forums/installati ... -04-a.html) re-indexing is a native zmprov feature in 5.0: zmprov rim
user@domain.com start. (status & cancel are also useful) Bug">
http://bugzilla.zimbra.com/show_bug.cgi?id=15348>Bug 15348 - zmprov should be able to control reindexing from the command line
Quick script for that reindex_mailboxes:
for i in `zmprov gaa -s `
do
echo "reindexing $i"
zmprov rim $i start
done
It can easily slam your disk and CPU i/o. So you might put a "sleep 120" between each one to space it out a bit if you don't care how fast it got done - just need it to do it for all accounts without all at the same time.
Another method:
1. Set the problematic account into maintance mode
2. Get the mailbox id of the account as described.
3. Shutdown Zimbra (zmcontrol stop)
4. Delete the folder called /opt/zimbra/index/0/ (probably better to NOT delete and move this to a safe place)
5. Startup Zimbra - You will see that zimbra rebuilds the index of the mailbox. (zmcontrol start)
6. Wait until the reindexing is finished
7. Make the account active again.
UserID is system wide
MailboxID is per server store
To get those:
zmprov ga
user@domain.com| grep -i zimbraId
If your on the appropriate mailserver:
zmprov getMailboxInfo
user@domain.com
or globally:
/opt/zimbra/bin/mysql -e "use zimbra; select id from mailbox where account_id = 'UserID HERE including the leading 0'"
SCRIPTS:
This is from FRANKLIN/ZimbraServer/src/perl/soap/reindex.pl (The 5.0 script just seems to be a slight 2 line correction from the FRANK 4.5 script on password inputs - pw instead of p).
reindex.pl:
#!/usr/bin/perl -w
#
# ***** BEGIN LICENSE BLOCK *****
#
# Zimbra Collaboration Suite Server
# Copyright (C) 2005-2008 Zimbra, Inc.
#
# Software distributed under the License is distributed on an "AS IS"
# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied.
#
# ***** END LICENSE BLOCK *****
#
use Time::HiRes qw ( time );
use strict;
use lib '.';
use LWP::UserAgent;
use Getopt::Long;
use XmlElement;
use XmlDoc;
use Soap;
use ZimbraSoapTest;
my $ACCTNS = "urn:zimbraAdmin";
my $MAILNS = "urn:zimbraAdmin";
# If you're using ActivePerl, you'll need to go and install the Crypt::SSLeay
# module for htps: to work...
#
# ppm install
http://theoryx5.uwinnipeg.ca/ppms/Crypt-SSLeay.ppd
#
# app-specific options
my ($mbox, $action, $types, $ids);
#standard options
my ($user, $pw, $host, $help); #standard
GetOptions("u|user=s" => $user,
"pw=s" => $pw,
"h|host=s" => $host,
"m|mbox=s" => $mbox,
"a|action=s" => $action,
"t|types=s" => $types,
"ids=s" => $ids,
"help|?" => $help);
if (!defined($user)) {
die "USAGE: $0 -u USER -m MAILBOXID -a ACTION [-pw PASSWD] [-h HOST] [-t TYPES] [-ids IDS]";
}
my $z = ZimbraSoapTest->new($user, $host, $pw);
$z->doAdminAuth();
my %args = ( 'action' => $action );
my $d = new XmlDoc;
$d = new XmlDoc;
$d->start('ReIndexRequest', $MAILNS, %args); {
my %mbxArgs = ( 'id' => $mbox );
if (defined $ids) {
$mbxArgs{'ids'} = $ids;
}
if (defined $types) {
$mbxArgs{'types'} = $types;
}
$d->add('mbox', $MAILNS, %mbxArgs);
} $d->end();
print "
OUTGOING XML:
-------------
";
my $out = $d->to_string("pretty");
$out =~ s/ns0://g;
print $out."
";
my $start = time;
my $firstStart = time;
my $response = $z->invokeAdmin($d->root());
print "
RESPONSE:
--------------
";
$out = $response->to_string("pretty");
$out =~ s/ns0://g;
print $out."
";
Run usage is ./reindex.pl -u USER -m MAILBOXID -a ACTION [-pw PASSWD] [-h HOST] [-t TYPES] [-ids IDS]
Here's another script zmreindexbyname.pl:
#!/usr/bin/perl
#
# ***** BEGIN LICENSE BLOCK *****
#
# Zimbra Collaboration Suite Server
# Copyright (C) 2005-2008 Zimbra, Inc.
#
# Software distributed under the License is distributed on an "AS IS"
# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied.
#
# ***** END LICENSE BLOCK *****
#
use strict;
use lib "/opt/zimbra/zimbramon/lib/zimbrapm";
use lib "/opt/zimbra/zimbramon/lib/zimbrapm/SOAP";
use lib "/opt/zimbra/zimbramon/lib";
use Getopt::Std;
use POSIX ":sys_wait_h";
use LWP::UserAgent;
use Zimbra::SOAP::XmlElement;
use Zimbra::SOAP::XmlDoc;
use Zimbra::SOAP::Soap;
use Time::Local;
# Exit if software-only node.
exit(0) unless (-f '/opt/zimbra/conf/localconfig.xml');
my $ACCTNS = "urn:zimbraAccount";
my $MAILNS = "urn:zimbraMail";
my $ADMINNS = "urn:zimbraAdmin";
my $SOAP = $Zimbra::SOAP::Soap::Soap12;
my $authToken;
my $START;
my $END;
my $PERIOD;
our %GlobalOpts;
sub usage { exit 1; }
sub get_auth_token {
my $host = shift;
my $user = shift;
my $pass = shift;
# print "Authenticating to $host as $user/$pass
";
my $url = https://$host:7071/service/admin/soap/";
">https://$host:7071/service/admin/soap/";
> my $d = new Zimbra::SOAP::XmlDoc;
$d->start('AuthRequest', $ADMINNS);
$d->add('name', undef, { by => "name"}, "$user");
$d->add('password', undef, undef, "$pass");
$d->end();
my $authResponse;
eval {
$authResponse = $SOAP->invoke($url, $d->root());
};
if (!$authResponse) {return undef;}
my $authToken = $authResponse->find_child('authToken')->content;
# print "authToken($authToken)
";
return $authToken;
}
#
#
#
#
#
# []
#
# -- Progress data is currently ONLY returned by the "status" and "cancelled" calls
#
# Access: domain admin sufficient
#
sub reIndexMailboxes {
my $name = shift;
my $action = shift;
my $host = "localhost";
my $user = "zimbra";
chomp $user;
my $pass = `zmlocalconfig -s -m nokey zimbra_ldap_password`;
chomp $pass;
$authToken = get_auth_token ($host, $user, $pass );
if (!defined ($authToken)) {exit 1;}
my $context = $SOAP->zimbraContext($authToken, "");
my $url = https://$host:7071/service/admin/soap/";
">https://$host:7071/service/admin/soap/";
> print "Indexing mailbox for $name...
";
my $d = new Zimbra::SOAP::XmlDoc;
$d->start('GetAccountRequest', $ADMINNS);
$d->add('account', undef, { by => "name"}, "$name");
$d->end();
my $response = $SOAP->invoke($url, $d->root(), $context);
if (!defined($response) || $response->{name} ne "GetAccountResponse") {
print "FAILED - can't get ID
";
return;
}
my $id = $response->find_child('account')->attr('id');
my $date = `date +%Y%m%d:%H%M%S`;
print "ID: $id...";
print "ACTION: $action
";
print "$date
";
$d = new Zimbra::SOAP::XmlDoc;
$d->start('ReIndexRequest', $ADMINNS, {action => $action});
$d->add('mbox', undef, { id => "$id"}, undef);
$d->end();
my $response = $SOAP->invoke($url, $d->root(), $context,360000);
if (defined($response) && $response->{name} eq "ReIndexResponse") {
print "".$response->to_string("pretty")."
";
} else {
print "Reindex for $name failed
";
return;
}
$d = new Zimbra::SOAP::XmlDoc;
$d->start('ReIndexRequest', $ADMINNS, {action => "status"});
$d->add('mbox', undef, { id => "$id"}, undef);
$d->end();
while (1) {
print "Status for $name...
";
my $response = $SOAP->invoke($url, $d->root(), $context,360000);
#if ($response->{name} eq "ReIndexResponse") {
if (defined ($response)) {
print "".$response->to_string("pretty")."
";
} else {
#print "".$response->to_string("pretty")."
";
print "$name complete
";
return;
}
sleep 120;
}
}
$|=1;
my @userList = <>;
foreach (@userList) {
chomp;
reIndexMailboxes($_, "start");
}
Should you be able to scrounge up some individual message blobs from backups some later on, see some of the info here:
Appointment">
http://www.zimbra.com/forums/installati ... #post64962
Appointment blobs can be tricky...
Good luck!