Bugfix: zmstat.out dramatically increasing in size with 'uninitialized' warnings for list variable

[OliverR]

After fresh install of Zimbra 8.8 on Ubuntu 18.04.2 LTS I encountered issues with zmstat.out file increasing dramatically in short time (several hundred gigs in minutes) along with high CPU load.

zmstat.out mentioned issues with uninitialized variable $list in the /opt/zimbra/libexec/zmstat-io perl script on line 76.

After checking the line and comparing the output of /usr/bin/iostat, I figured the latter had no : after the Device entry.

Code: Select all

zimbra@sol:~$ iostat
Linux 4.15.0-50-generic (sol)   05/29/2019      _x86_64_        (2 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          20.41    8.96    9.80    7.28    0.00   53.55

Device             tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
sdc              27.82      1117.57      4601.60  119308677  491252548
sda              74.38      4111.18      2056.85  438896128  219582696
sdb              46.77      2488.03      4601.61  265614472  491253852
md1               0.00         0.04         0.00       4528         60
md3              65.43      3585.30      4580.76  382754949  489026940
md2              15.71        21.68        41.18    2314220    4396760

The zmstat-io perl script should thus have replaced all 5 occasions of the search string Device: by Device

Example:

Wrong code:

Code: Select all

while ($line !~ /^avg-cpu/ && $line !~ /^Device:/) {

Corrected code:

Code: Select all

while ($line !~ /^avg-cpu/ && $line !~ /^Device/) {

Hth!

[DualBoot]

Hi,

nice job. I got into that problem with Ubuntu 16.04 ZCS OSE 8.8.12 and I could solved it.

Regards,

[davidkillingsworth]

Oh my goodness, you are such a life saver. Thank you so much for posting this in the forum!

I just upgraded a server 8.8.15 Patch 19 from Ubuntu 14.04 to 16.04 and then 18.04.

All looked fine and I thought the upgrade was a success, then a couple of hours later, I got notifications that the disk space in /opt/zimbra/ had filled up to 100%.

I tried deleting all logs and I found this command which deleted old zmstats from the last 5 years that I didn't know that I had.

Code: Select all

 su - zimbra
% /opt/zimbra/libexec/zmstat-cleanup --help

Usage: /opt/zimbra/libexec/zmstat-cleanup [-k <keep count>] [-h] [-l] [-p]

    -h|--help    Help/usage information
    -k|--keep    Number of archived copies of zmstat data to retain.
    -l|--list    Generate a listing of log dates archived.
    -p|--purge   Purge ALL archived zmstats data.

I found that the /opt/zimbra/zmstat/zmstat.out had grown 250GB withe the error.

Code: Select all

Use of uninitialized value $line in pattern match (m//) at /opt/zimbra/libexec/zmstat-io line 76

I found your post, deleted /opt/zimbra/zmstat/zmstat.out and edited the perl script and rebooted the server.

It seems to be running ok for now, but I'll definitely be keeping an eye on it for any further disk growth.