[SOLVED] Antivirus not running, sending all mail to deferred queue

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
Ron Haines
Posts: 15
Joined: Fri Sep 12, 2014 10:37 pm

[SOLVED] Antivirus not running, sending all mail to deferred queue

Post by Ron Haines »

I had my Zimbra server crash hard today, requiring a power cycle to get up again. After the reboot zmcontrol reported antivirus was not running. Soon after I noticed mail getting diverted into the deferred queue and this in the logs:
Jul 2 10:01:35 Zimbra-Computer amavis[684]: (00684-01) (!)ClamAV-clamd: Can't connect to INET socket 127.0.0.1:3310: Connection refused, retrying (2)

Jul 2 10:01:41 Zimbra-Computer amavis[684]: (00684-01) (!!)ClamAV-clamd av-scanner FAILED: Too many retries to talk to 127.0.0.1:3310 (Can't connect to INET socket 127.0.0

.1:3310: Connection refused) at (eval 54) line 269.

Jul 2 10:01:41 Zimbra-Computer amavis[684]: (00684-01) (!!)WARN: all primary virus scanners failed, considering backups

Jul 2 10:01:41 Zimbra-Computer amavis[684]: (00684-01) (!!)TROUBLE in check_mail: virus_scan FAILED: virus_scan: ALL VIRUS SCANNERS FAILED: ClamAV-clamd av-scanner FAILED:

Too many retries to talk to 127.0.0.1:3310 (Can't connect to INET socket 127.0.0.1:3310: Connection refused) at (eval 54) line 269.

Jul 2 10:01:41 Zimbra-Computer amavis[684]: (00684-01) (!)PRESERVING EVIDENCE in /opt/zimbra/amavisd/tmp/amavis-20070702T100133-00684
After a clean shutdown and reboot the situation persists, and now I also see snmp is not running ('swatch not running'). I have tried turning off antivirus in the server settings (web interface). Still incoming mail is getting stuck in the deferred queue for the same reason as above.
Any and all help would be welcomed. Looking though the forum archives I see a few posts about this, but none that give me a solution.
The server is: Release 4.5.5_GA_838.MACOSX, Zimbra, Inc. MACOSX NETWORK edition
More details: after turning off antivirus and antispam (using web admin), then turning them back on I was able to get mail going into mailboxes with repeated use of postqueue -f. Now incoming mail does appear to be getting into mailboxes. However 'zmcontrol status' still reports:

antivirus Stopped

zmclamdctl is not running

ldap Running

logger Running

mailbox Running

mta Running

snmp Stopped

swatch is not running
Curiously ps aux shows:

USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND

zimbra 21633 96.0 1.8 40500 15424 ?? Rs 4:01PM 0:33.89 /opt/zimbra/clamav/sbin/clamd --config-file /opt/zimbra/conf/clamd.conf
Clamd.log shows no errors, just lots of instances of:

Mon Jul 2 15:46:25 2007 -> +++ Started at Mon Jul 2 15:46:25 2007

Mon Jul 2 15:46:25 2007 -> clamd daemon 0.90.2 (OS: darwin8.7.0, ARCH: ppc, CPU: powerpc)

Mon Jul 2 15:46:25 2007 -> Log file size limited to 20971520 bytes.

Mon Jul 2 15:46:25 2007 -> Reading databases from /opt/zimbra/clamav/db
Looks like clamd is trying to start and failing over and over.
thanks,

Ron.
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

[SOLVED] Antivirus not running, sending all mail to deferred queue

Post by phoenix »

You haven't got another version of amavisd running have you?
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
Ron Haines
Posts: 15
Joined: Fri Sep 12, 2014 10:37 pm

[SOLVED] Antivirus not running, sending all mail to deferred queue

Post by Ron Haines »

[quote user="10330phoenix"]You haven't got another version of amavisd running have you?[/QUOTE]
No, well not at least intentionally. This is a plain install for OS X with no customisation. 'ps aux' shows the following amavisd processes.
zimbra 5226 0.0 2.1 79240 18104 ?? Ss 2:15PM 0:03.70 amavisd (master)

zimbra 5262 0.0 3.5 80228 29716 ?? S 2:15PM 0:04.21 amavisd (ch2-avail)

zimbra 5263 0.0 3.4 79892 29020 ?? S 2:15PM 0:03.60 amavisd (ch3-avail)

zimbra 5264 0.0 3.4 80192 29092 ?? S 2:15PM 0:03.61 amavisd (ch2-avail)

zimbra 5265 0.0 3.4 79892 28792 ?? S 2:15PM 0:02.56 amavisd (ch2-avail)

zimbra 5266 0.0 3.4 80540 29172 ?? S 2:15PM 0:09.89 amavisd (ch5-avail)

zimbra 5267 0.0 3.6 80212 30428 ?? S 2:15PM 0:10.67 amavisd (ch4-avail)

zimbra 5268 0.0 2.0 80192 16932 ?? S 2:15PM 0:01.67 amavisd (ch1-avail)

zimbra 5269 0.0 3.6 81360 30480 ?? S 2:15PM 0:08.83 amavisd (ch6-avail)

zimbra 5270 0.0 3.5 80220 29592 ?? S 2:15PM 0:09.22 amavisd (ch4-avail)

zimbra 5271 0.0 3.7 80384 31200 ?? S 2:15PM 0:05.91 amavisd (ch2-avail)
Does this look as expected? I don't have anything to compare this with.

Thanks for your time.

Ron.
User avatar
jholder
Ambassador
Ambassador
Posts: 4824
Joined: Fri Sep 12, 2014 10:00 pm

[SOLVED] Antivirus not running, sending all mail to deferred queue

Post by jholder »

can you

cat /opt/zimbra/conf/clamd.conf
Ron Haines
Posts: 15
Joined: Fri Sep 12, 2014 10:37 pm

[SOLVED] Antivirus not running, sending all mail to deferred queue

Post by Ron Haines »

[quote user="jholder"]can you

cat /opt/zimbra/conf/clamd.conf[/QUOTE]
Certainly, here it is. I have not made any changes to this file, and it doesn't appear to be corrupted. But somewhere there is something corrupted or locked after the kernel panic that stopped the machine.
dosy:~/log zimbra$ cat /opt/zimbra/conf/clamd.conf

##

## Example config file for the Clam AV daemon

## Please read the clamd.conf(5) manual before editing this file.

##


# Comment or remove the line below.

# Example
# Uncomment this option to enable logging.

# LogFile must be writable for the user running daemon.

# A full path is required.

# Default: disabled

LogFile /opt/zimbra/log/clamd.log
# By default the log file is locked for writing - the lock protects against

# running clamd multiple times (if want to run another clamd, please

# copy the configuration file, change the LogFile variable, and run

# the daemon with --config-file option).

# This option disables log file locking.

# Default: disabled

#LogFileUnlock
# Maximal size of the log file.

# Value of 0 disables the limit.

# You may use 'M' or 'm' for megabytes (1M = 1m = 1048576 bytes)

# and 'K' or 'k' for kilobytes (1K = 1k = 1024 bytes). To specify the size

# in bytes just don't use modifiers.

# Default: 1M

LogFileMaxSize 20M
# Log time with each message.

# Default: disabled

LogTime yes
# Also log clean files. Useful in debugging but drastically increases the

# log size.

# Default: disabled

#LogClean no
# Use system logger (can work together with LogFile).

# Default: disabled

LogSyslog yes
# Specify the type of syslog messages - please refer to 'man syslog'

# for facility names.

# Default: LOG_LOCAL6

LogFacility LOG_LOCAL0
# Enable verbose logging.

# Default: disabled

#LogVerbose no
# This option allows you to save a process identifier of the listening

# daemon (main thread).

# Default: disabled

PidFile /opt/zimbra/log/clamd.pid
# Optional path to the global temporary directory.

# Default: system specific (usually /tmp or /var/tmp).

#TemporaryDirectory /var/tmp
# Path to the database directory.

# Default: hardcoded (depends on installation options)

DatabaseDirectory /opt/zimbra/clamav/db
# The daemon works in a local OR a network mode. Due to security reasons we

# recommend the local mode.
# Path to a local socket file the daemon will listen on.

# Default: disabled

# LocalSocket /opt/zimbra/clamav/clamav.sock
# Remove stale socket after unclean shutdown.

# Default: disabled

# FixStaleSocket
# TCP port address.

# Default: disabled

TCPSocket 3310
# TCP address.

# By default we bind to INADDR_ANY, probably not wise.

# Enable the following to provide some degree of protection

# from the outside world.

# Default: disabled

#TCPAddr 127.0.0.1
# Maximum length the queue of pending connections may grow to.

# Default: 15

#MaxConnectionQueueLength 30
# Clamd uses FTP-like protocol to receive data from remote clients.

# If you are using clamav-milter to balance load between remote clamd daemons

# on firewall servers you may need to tune the options below.
# Close the connection when the data size limit is exceeded.

# The value should match your MTA's limit for a maximal attachment size.

# Default: 10M

StreamMaxLength 10240000
# Limit port range.

# Default: 1024

#StreamMinPort 30000

# Default: 2048

#StreamMaxPort 32000
# Maximal number of threads running at the same time.

# Default: 10

#MaxThreads 20
# Waiting for data from a client socket will timeout after this time (seconds).

# Value of 0 disables the timeout.

# Default: 120

#ReadTimeout 300
# Waiting for a new job will timeout after this time (seconds).

# Default: 30

#IdleTimeout 60
# Maximal depth directories are scanned at.

# Default: 15

#MaxDirectoryRecursion 20
# Follow directory symlinks.

# Default: disabled

#FollowDirectorySymlinks
# Follow regular file symlinks.

# Default: disabled

#FollowFileSymlinks
# Perform internal sanity check (database integrity and freshness).

# Default: 1800 (30 min)

#SelfCheck 600
# Execute a command when virus is found. In the command string %v will

# be replaced by a virus name.

# Default: disabled

#VirusEvent /usr/local/bin/send_sms 123456789 "VIRUS ALERT: %v"
# Run as a selected user (clamd must be started by root).

# Default: disabled

User zimbra
# Initialize supplementary group access (clamd must be started by root).

# Default: disabled

#AllowSupplementaryGroups
# Stop daemon when libclamav reports out of memory condition.

#ExitOnOOM
# Don't fork into background.

# Default: disabled

#Foreground
# Enable debug messages in libclamav.

# Default: disabled

#Debug
# Do not remove temporary files (for debug purposes).

# Default: disabled

#LeaveTemporaryFiles
# By default clamd uses scan options recommended by libclamav. This option

# disables recommended options and allows you to enable selected ones below.

# DO NOT TOUCH IT unless you know what you are doing.

# Default: disabled

#DisableDefaultScanOptions
##

## Executable files

##
# PE stands for Portable Executable - it's an executable file format used

# in all 32-bit versions of Windows operating systems. This option allows

# ClamAV to perform a deeper analysis of executable files and it's also

# required for decompression of popular executable packers such as UPX, FSG,

# and Petite.

# Default: enabled

#ScanPE
# With this option clamav will try to detect broken executables and mark

# them as Broken.Executable

# Default: disabled

#DetectBrokenExecutables
##

## Documents

##
# This option enables scanning of Microsoft Office document macros.

# Default: enabled

#ScanOLE2
##

## Mail files

##
# Enable internal e-mail scanner.

# Default: enabled

#ScanMail
# If an email contains URLs ClamAV can download and scan them.

# WARNING: This option may open your system to a DoS attack.

# Never use it on loaded servers.

# Default: disabled

#MailFollowURLs


##

## HTML

##
# Perform HTML normalisation and decryption of MS Script Encoder code.

# Default: enabled

#ScanHTML


##

## Archives

##
# ClamAV can scan within archives and compressed files.

# Default: enabled

#ScanArchive
# Due to license issues libclamav does not support RAR 3.0 archives (only the

# old 2.0 format is supported). Because some users report stability problems

# with unrarlib it's disabled by default and you must uncomment the directive

# below to enable RAR 2.0 support.

# Default: disabled

#ScanRAR
# The options below protect your system against Denial of Service attacks

# using archive bombs.
# Files in archives larger than this limit won't be scanned.

# Value of 0 disables the limit.

# Default: 10M

ArchiveMaxFileSize 100M
# Nested archives are scanned recursively, e.g. if a Zip archive contains a RAR

# file, all files within it will also be scanned. This options specifies how

# deep the process should be continued.

# Value of 0 disables the limit.

# Default: 8

#ArchiveMaxRecursion 9
# Number of files to be scanned within an archive.

# Value of 0 disables the limit.

# Default: 1000

#ArchiveMaxFiles 1500
# If a file in an archive is compressed more than ArchiveMaxCompressionRatio

# times it will be marked as a virus (Oversized.ArchiveType, e.g. Oversized.Zip)

# Value of 0 disables the limit.

# Default: 250

#ArchiveMaxCompressionRatio 300
# Use slower but memory efficient decompression algorithm.

# only affects the bzip2 decompressor.

# Default: disabled

#ArchiveLimitMemoryUsage
# Mark encrypted archives as viruses (Encrypted.Zip, Encrypted.RAR).

# Default: disabled

ArchiveBlockEncrypted
# Mark archives as viruses (e.g. RAR.ExceededFileSize, Zip.ExceededFilesLimit)

# if ArchiveMaxFiles, ArchiveMaxFileSize, or ArchiveMaxRecursion limit is

# reached.

# Default: disabled

#ArchiveBlockMax


##

## Clamuko settings

## WARNING: This is experimental software. It is very likely it will hang

## up your system!!!

##
# Enable Clamuko. Dazuko (/dev/dazuko) must be configured and running.

# Default: disabled

#ClamukoScanOnAccess
# Set access mask for Clamuko.

# Default: disabled

#ClamukoScanOnOpen

#ClamukoScanOnClose

#ClamukoScanOnExec
# Set the include paths (all files in them will be scanned). You can have

# multiple ClamukoIncludePath directives but each directory must be added

# in a seperate line.

# Default: disabled

#ClamukoIncludePath /home

#ClamukoIncludePath /students
# Set the exclude paths. All subdirectories are also excluded.

# Default: disabled

#ClamukoExcludePath /home/guru
# Don't scan files larger than ClamukoMaxFileSize

# Value of 0 disables the limit.

# Default: 5M

#ClamukoMaxFileSize 10M

User avatar
jholder
Ambassador
Ambassador
Posts: 4824
Joined: Fri Sep 12, 2014 10:00 pm

[SOLVED] Antivirus not running, sending all mail to deferred queue

Post by jholder »

Can you see if there's a Stale PID file in

/opt/zimbra/log/clamd.pid
User avatar
jholder
Ambassador
Ambassador
Posts: 4824
Joined: Fri Sep 12, 2014 10:00 pm

[SOLVED] Antivirus not running, sending all mail to deferred queue

Post by jholder »

Also, it would be helpful if you could update your profile with your Zimbra version :)
jh
Ron Haines
Posts: 15
Joined: Fri Sep 12, 2014 10:37 pm

[SOLVED] Antivirus not running, sending all mail to deferred queue

Post by Ron Haines »

[quote user="jholder"]Can you see if there's a Stale PID file in

/opt/zimbra/log/clamd.pid[/QUOTE]
No, cannot see any pid file at /opt/zimbra/log/clamd.pid

What I can see is clamd starting every few minutes, taking up all available cpu (jumps to the start in 'top -u') then vanishing. Even when clamd is running there is no pid file.
Have updated my profile. Apologies, should have done that when I signed up to the forums.
User avatar
jholder
Ambassador
Ambassador
Posts: 4824
Joined: Fri Sep 12, 2014 10:00 pm

[SOLVED] Antivirus not running, sending all mail to deferred queue

Post by jholder »

Rerun the installer as an upgrade. Let's see if that fixes the issue.
Also, run the disk utility and verify your disk is running good.

Once the installer is complete, run the ./zmfixperms script located in the /opt/zimbra/libexec dir to see if that resolves the issue.
Ron Haines
Posts: 15
Joined: Fri Sep 12, 2014 10:37 pm

[SOLVED] Antivirus not running, sending all mail to deferred queue

Post by Ron Haines »

Thanks, will check the disk first, then run the installer as an upgrade. Was hoping to avoid anything too invasive, but this looks like a sensible way to go and I have a low usage window coming up within the next day.
Post Reply