[SOLVED] Logrotate not processing nginx.access.log and nginx.log

[liverpoolfcfan]

Zimbra version: Release 8.8.15_GA_3829.RHEL7_64_20190718141144 RHEL7_64 FOSS edition, Patch 8.8.15_P7. (Upgraded all the way from 6.0)

I recently discovered that /opt/zimbra/log/nginx.access.log and nginx.log are not being logrotated (with the first being multiple GB) - Can anyone spot what might be the issue? Thanks.

/etc/logrotate.d/zimbra

Code: Select all

/opt/zimbra/log/nginx.log /opt/zimbra/log/nginx.access.log {
    daily
    sharedscripts
    missingok
    notifempty
    create 0644 zimbra zimbra
    postrotate
      kill -USR1 `cat /opt/zimbra/log/nginx.pid 2> /dev/null` 2> /dev/null || true
    endscript
    rotate 7
    compress
    delaycompress
    su zimbra zimbra
}

And, here are the results of manually force running the log rotation in order to troubleshoot

Code: Select all

rotating pattern: /opt/zimbra/log/nginx.log /opt/zimbra/log/nginx.access.log  forced from command line (7 rotations)
empty log files are not rotated, old logs are removed
switching euid to 997 and egid to 995
considering log /opt/zimbra/log/nginx.log
  log needs rotating
considering log /opt/zimbra/log/nginx.access.log
  log needs rotating
rotating log /opt/zimbra/log/nginx.log, log->rotateCount is 7
dateext suffix '-20200430'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
previous log /opt/zimbra/log/nginx.log.1 does not exist
renaming /opt/zimbra/log/nginx.log.7.gz to /opt/zimbra/log/nginx.log.8.gz (rotatecount 7, logstart 1, i 7),
renaming /opt/zimbra/log/nginx.log.6.gz to /opt/zimbra/log/nginx.log.7.gz (rotatecount 7, logstart 1, i 6),
renaming /opt/zimbra/log/nginx.log.5.gz to /opt/zimbra/log/nginx.log.6.gz (rotatecount 7, logstart 1, i 5),
renaming /opt/zimbra/log/nginx.log.4.gz to /opt/zimbra/log/nginx.log.5.gz (rotatecount 7, logstart 1, i 4),
renaming /opt/zimbra/log/nginx.log.3.gz to /opt/zimbra/log/nginx.log.4.gz (rotatecount 7, logstart 1, i 3),
renaming /opt/zimbra/log/nginx.log.2.gz to /opt/zimbra/log/nginx.log.3.gz (rotatecount 7, logstart 1, i 2),
renaming /opt/zimbra/log/nginx.log.1.gz to /opt/zimbra/log/nginx.log.2.gz (rotatecount 7, logstart 1, i 1),
renaming /opt/zimbra/log/nginx.log.0.gz to /opt/zimbra/log/nginx.log.1.gz (rotatecount 7, logstart 1, i 0),
rotating log /opt/zimbra/log/nginx.access.log, log->rotateCount is 7
dateext suffix '-20200430'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
previous log /opt/zimbra/log/nginx.access.log.1 does not exist
renaming /opt/zimbra/log/nginx.access.log.7.gz to /opt/zimbra/log/nginx.access.log.8.gz (rotatecount 7, logstart 1, i 7),
renaming /opt/zimbra/log/nginx.access.log.6.gz to /opt/zimbra/log/nginx.access.log.7.gz (rotatecount 7, logstart 1, i 6),
renaming /opt/zimbra/log/nginx.access.log.5.gz to /opt/zimbra/log/nginx.access.log.6.gz (rotatecount 7, logstart 1, i 5),
renaming /opt/zimbra/log/nginx.access.log.4.gz to /opt/zimbra/log/nginx.access.log.5.gz (rotatecount 7, logstart 1, i 4),
renaming /opt/zimbra/log/nginx.access.log.3.gz to /opt/zimbra/log/nginx.access.log.4.gz (rotatecount 7, logstart 1, i 3),
renaming /opt/zimbra/log/nginx.access.log.2.gz to /opt/zimbra/log/nginx.access.log.3.gz (rotatecount 7, logstart 1, i 2),
renaming /opt/zimbra/log/nginx.access.log.1.gz to /opt/zimbra/log/nginx.access.log.2.gz (rotatecount 7, logstart 1, i 1),
renaming /opt/zimbra/log/nginx.access.log.0.gz to /opt/zimbra/log/nginx.access.log.1.gz (rotatecount 7, logstart 1, i 0),
fscreate context set to unconfined_u:object_r:usr_t:s0
renaming /opt/zimbra/log/nginx.log to /opt/zimbra/log/nginx.log.1
creating new /opt/zimbra/log/nginx.log mode = 0644 uid = 997 gid = 995
fscreate context set to unconfined_u:object_r:usr_t:s0
renaming /opt/zimbra/log/nginx.access.log to /opt/zimbra/log/nginx.access.log.1
creating new /opt/zimbra/log/nginx.access.log mode = 0644 uid = 997 gid = 995
running postrotate script
running script with arg /opt/zimbra/log/nginx.log /opt/zimbra/log/nginx.access.log : "
      kill -USR1 `cat /opt/zimbra/log/nginx.pid 2> /dev/null` 2> /dev/null || true
"
removing old log /opt/zimbra/log/nginx.log.8.gz
error: error opening /opt/zimbra/log/nginx.log.8.gz: No such file or directory
switching euid to 0 and egid to 0

[danielb]

I just debuged the same issue this morning. The problem is that Zimbra's log have wrong SELinux type set, so SELinux prevents logrotate to touch them. To fix it, use

Code: Select all

semanage fcontext -a -t var_log_t "/opt/zimbra/log(/.*)?"
restorecon -Rv /opt/zimbra/log

[liverpoolfcfan]

I had suspected it might be Selinux and had turned off Enforcing to manually try to logrotate but it hadn't worked. I had given up on that being the issue. Perhaps nginx also needs to be restarted after the Selinux change for logrotate to be able to do it's thing on the files?

Anyway, after setting the contexts on the files using your instructions, and the server being bounced for a backup overnight, the logrotate worked in the early hours of this morning.

Thanks a million @danielb for your help.