Zimbra Administration Server Status all turned red

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
Post Reply
chengcsw
Posts: 1
Joined: Sat Sep 13, 2014 3:21 am

Zimbra Administration Server Status all turned red

Post by chengcsw »

I have read all posts in the forum but none of them work
Here's what I have checked
The hostname should be correct...
[zimbra@zimbra root]$ zmprov gacf | grep zimbraLogHostname

zimbraLogHostname: zimbra.mydomain.com
To make sure the log is written, I execute it manually
[root@zimbra log]# su zimbra -c /opt/zimbra/libexec/zmstatuslog


The status is written to the log and the format should be correct with rsyslog
[root@zimbra log]# tail zimbra-stats.log | grep STATUS

Sep 29 20:53:19 zimbra zimbramon[8971]: 8971:info: 2013-09-29 20:53:03, STATUS: zimbra.mydomain.com: logger: Running

Sep 29 20:53:19 zimbra zimbramon[8971]: 8971:info: 2013-09-29 20:53:03, STATUS: zimbra.mydomain.com: mailbox: Running

Sep 29 20:53:19 zimbra zimbramon[8971]: 8971:info: 2013-09-29 20:53:03, STATUS: zimbra.mydomain.com: memcached: Running

Sep 29 20:53:19 zimbra zimbramon[8971]: 8971:info: 2013-09-29 20:53:03, STATUS: zimbra.mydomain.com: mta: Running

Sep 29 20:53:19 zimbra zimbramon[8971]: 8971:info: 2013-09-29 20:53:03, STATUS: zimbra.mydomain.com: opendkim: Running

Sep 29 20:53:19 zimbra zimbramon[8971]: 8971:info: 2013-09-29 20:53:03, STATUS: zimbra.mydomain.com: proxy: Running

Sep 29 20:53:19 zimbra zimbramon[8971]: 8971:info: 2013-09-29 20:53:03, STATUS: zimbra.mydomain.com: snmp: Running

Sep 29 20:53:19 zimbra zimbramon[8971]: 8971:info: 2013-09-29 20:53:03, STATUS: zimbra.mydomain.com: spell: Running

Sep 29 20:53:19 zimbra zimbramon[8971]: 8971:info: 2013-09-29 20:53:03, STATUS: zimbra.mydomain.com: stats: Running

Sep 29 20:53:19 zimbra zimbramon[8971]: 8971:info: 2013-09-29 20:53:03, STATUS: zimbra.mydomain.com: zmconfigd: Running


Make sure logswatch is running
[zimbra@zimbra root]$ /opt/zimbra/libexec/zmloggerinit

cp: cannot stat `/opt/zimbra/logger/db/work/*.gif': No such file or directory

Stopping logswatch...done.

Starting logswatch...done.


Yet, zmsoap returns status as 0...
[root@zimbra ~]# /opt/zimbra/bin/zmsoap -z GetServiceStatusRequest





0

0

0

0

0

0

0

0

0

0

0

0

User avatar
L. Mark Stone
Ambassador
Ambassador
Posts: 2796
Joined: Wed Oct 09, 2013 11:35 am
Location: Portland, Maine, US
ZCS/ZD Version: 10.0.6 Network Edition
Contact:

Zimbra Administration Server Status all turned red

Post by L. Mark Stone »

Have the red X marks been there from day one or is this new? If new, have you done anything like Zimbra or operating system updates recently? Any other changes?
Basically to get the green check marks you need the service statuses to make it into /var/log/zimbra.log and then to have the Zimbra logger service be able to parse them. Sounds like the first part is working but not the second part... SOmetimes the logger service just dies and Zimbra is unable to restart it by itself.
What happens if you do as the zimbra user? :
zmloggerctl stop; zmloggerctl start; zmloggerctl status
If the service starts up with no errors then within ten minutes the red X's should change to the green checkmarks.
Let us know!

Mark
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
AWS Certified Solutions Architect-Associate
huuphan
Posts: 4
Joined: Sat Nov 12, 2016 1:33 am
ZCS/ZD Version: zimbra 8.6
Contact:

Re: Zimbra Administration Server Status all turned red

Post by huuphan »

azam wrote:How can i Solved this RED Mark error ?
Hi azam! solve problem "RED" . you can run command line all zimbra server as bellow:
1. update /etc/rsyslog.conf /opt/zimbra/libexec/zmsyslogsetup ( as root account)
2. zmupdateauthkeys command to update and fetching key between zimbra server. ( as zimbra account)

Link below help to you:
http://www.huuphan.com/2016/11/zimbra-s ... nning.html
linuradu
Posts: 4
Joined: Mon Apr 24, 2017 6:18 pm

Re: Zimbra Administration Server Status all turned red

Post by linuradu »

I followed this post viewtopic.php?t=60723 and did the following:

I had a similar issue and I've tried what you described here but didn't worked for me.

I analysed the file "zmstatuslog" from (/opt/zimbra/libexec/zmstatuslog) and found there that there is a TIMEOUT set for 60 seconds;
The time executaion of "/opt/zimbra/bin/zmcontrol status" took around 90 seconds; Because of that, the code from this file was not executed properly
and this is the reason of "not sync status".

My solution was:
1. open and edit the file: #nano /opt/zimbra/libexec/zmstatuslog
2. change the line: "my $TIMEOUT=60;" to "my $TIMEOUT=360;"
3. Save the file and wait around 5 minutes.

This worked for me; Hopefully will work also for other people :)

If you type: "tail -f /var/log/zimbra-stats.log" you should see the status like, in the log:
Apr 24 11:41:07 mail zimbramon[28337]: 28337:info: 2017-04-24 11:40:02, STATUS: mail.test.com: antispam: Running
Apr 24 11:41:07 mail zimbramon[28337]: 28337:info: 2017-04-24 11:40:02, STATUS: mail.test.com: antivirus: Running
Apr 24 11:41:07 mail zimbramon[28337]: 28337:info: 2017-04-24 11:40:02, STATUS: mail.test.com: dnscache: Running
Apr 24 11:41:07 mail zimbramon[28337]: 28337:info: 2017-04-24 11:40:02, STATUS: mail.test.com: ldap: Running
....
mastermindkhan
Posts: 1
Joined: Tue Jul 24, 2018 4:36 am

Re: Zimbra Administration Server Status all turned red

Post by mastermindkhan »

Dear friends,

This is quite an old post, but i had to reply to it, because this issue just made me NUTS :shock: and I looked for a solution for the whole night but could not find it till finally i came across this and I want to share it to make someone's life easier. Just execute the following command and everything will be OK.

zmsoap -v -a <admin@domain.com> -p <password> GetServiceStatusRequest

Please keep in mind, that this solution is for the actual post which began this thread, not for any other related issue. This works when nothing else works. First you need to try the other methods and if all fail just type the above command and check your admin console. All will be well from then on. :D

Regards,

Bilal
User avatar
gabrieles
Outstanding Member
Outstanding Member
Posts: 233
Joined: Tue Feb 14, 2017 9:40 am

Re: Zimbra Administration Server Status all turned red

Post by gabrieles »

mastermindkhan wrote:Just execute the following command and everything will be OK.
zmsoap -v -a <admin@domain.com> -p <password> GetServiceStatusRequest
Does not work for me. Keep in mind that GetServiceStatusRequest is a soap get api. It does not modify anything, maybe you fixed it in another way, and now GetServiceStatusRequest gives you correct results.
esegarra
Posts: 1
Joined: Sat Apr 11, 2020 7:19 pm

Re: Zimbra Administration Server Status all turned red

Post by esegarra »

I have the same problem, I have followed his good recommendations to the letter but nothing, he is currently running on ubuntu 18, I don't know if it is because I recently installed csf, I can't find another explanation
rpocamilo
Posts: 2
Joined: Wed Jul 01, 2020 5:18 pm

Re: Zimbra Administration Server Status all turned red

Post by rpocamilo »

I did all mentioned above with no luck.

Then i found that the script /opt/zimbra/libexec/zmstatuslog was running well if i executed it manually (it writes STATUS lines in the log). But when running in crontab it wasn't writing STATUS lines to the log (/var/log/zimbra-stats.log).

I inspected the code inside the script and compared with another zimbra server that was working fine and found some diferences. So i copied the script from the OK server to the server with the Red X in status web page and everithing worked well again ( Green status on webpage and STATUS lines got logged on /var/log/zimbra-stats.log ).

This is the "good script" content in case someone needs to use it.

Code: Select all

#!/usr/bin/perl
#
# ***** BEGIN LICENSE BLOCK *****
# Zimbra Collaboration Suite Server
# Copyright (C) 2005, 2006, 2007, 2008, 2009, 2010, 2013, 2014, 2015, 2016 Synacor, Inc.
#
# This program is free software: you can redistribute it and/or modify it under
# the terms of the GNU General Public License as published by the Free Software Foundation,
# version 2 of the License.
#
# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
# without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
# See the GNU General Public License for more details.
# You should have received a copy of the GNU General Public License along with this program.
# If not, see <https://www.gnu.org/licenses/>.
# ***** END LICENSE BLOCK *****
#

use strict;

use lib "/opt/zimbra/common/lib/perl5";
use  Zimbra::Util::Common;
use Zimbra::Mon::Logger;

# Exit if software-only node.
exit(0) unless (-f "/opt/zimbra/conf/localconfig.xml");

$SIG{ALRM} = \&catchAlarm;

my $platform=qx(/opt/zimbra/libexec/get_plat_tag.sh);
chomp $platform;

my $pidFile="/opt/zimbra/log/zmstatuslog.pid";

my $TIMEOUT=60;
my $MNTCMD;
my $DFCMD;
if ($platform eq "MACOSX") {
        $MNTCMD = "mount -t hfs";
        $DFCMD = "df -ml ";
} else {
        $MNTCMD = "mount -t ext3";
        $DFCMD = "df -ml ";
}

my $dt = qx(date "+%Y-%m-%d %H:%M:%S");
chomp $dt;

my $hostname;

checkPid();
logStatus();
clearPid();

exit 0;

sub logStatus {
        my @status = ();
  alarm($TIMEOUT);
        open STATUS, "/opt/zimbra/bin/zmcontrol status |" or die "Can't get status: $!";
        @status = <STATUS>;
        close STATUS;
        foreach my $s (@status) {
                if ($s =~ /is not/) {
                        next;
                }
                chomp $s;
                if ($s =~ /^Host (.*)/) {
                        $hostname = $1;
                        next;
                }
                $s =~ s/ webapp//;
                my ($service, $stat) = split ' ', $s, 2;
                Zimbra::Mon::Logger::LogStats( "info", "$dt, STATUS: ${hostname}: $service: $stat" );
        }
  alarm(0);
}

sub checkPid {
  if (-f "$pidFile") {
    my $P = qx(cat $pidFile);
    chomp $P;
    if ($P ne "") {
      system("kill -0 $P 2> /dev/null");
      if ($? == 0) {
        print "$0 already running with pid $P\n";
        exit 0;
      }
    }
  }
  qx(echo $$ > "$pidFile");
}

sub clearPid {
  unlink($pidFile);
}

sub catchAlarm {
                Zimbra::Mon::Logger::LogStats( "info", "zmstatuslog timeout after $TIMEOUT seconds");
    exit 1;
}
lcxat
Posts: 1
Joined: Tue Jan 25, 2022 7:43 am

Re: Zimbra Administration Server Status all turned red

Post by lcxat »

If someone stumbles upon this thread this is what helped me:

The issue was CAUSED by commenting following line in my rsyslog.conf, as I prefer ISO standardised timestamps.

Code: Select all

grep DefaultTemplate /etc/rsyslog.conf
# $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
so in order to make Zimbra status work properly again, you need to do something like

Code: Select all

grep stats /etc/rsyslog.d/60-zimbra.conf
local1.*                -/var/log/zimbra-stats.log;RSYSLOG_TraditionalFileFormat
and restart rsyslog
Post Reply