dspam_clean locking up

[jdossey7]

I've had dspam enabled on our system for about 2 months now. It has been running fine, up until last week. dspam_clean has locked up 3 times over the last week. I don't know if it has really locked up or what is wrong, but on each of those 3 occasions I would get complaints from our users that they were not getting their email. Each time I found that dspam_clean was still running at 7:30am even though it was started around midnight. Killing dspam_clean would restart the flow of email. But then it would have to process the backlog of messages that had come in since midnight and that would take about 4 hours.
I know nothing about dspam, so I don't know where to start looking. Anybody have any suggestions?
This is on version 5.0.14 FOSS, running on Centos 5.

[jdossey7]

I did some looking around and found that the /opt/zimbra/data/dspam/z/i/zimbra/zimbra.sig dir is huge - > 17MB. I can't even do an 'ls' on it due to it's size. I think that might be the problem.

[jdossey7]

I think I've found the problem. I found some other posts related to the zimbra.sig directory and a cron job that cleans it out. So I looked at 'crontab -l' and found this entry:

0 1 * * * [ -d /opt/zimbra/data/dspam/data/z/i/zimbra/zimbra.sig ] && find /opt/zimbra/dspam/var/dspam/data/z/i/zimbra/zimbra.sig/ -type f -name *sig -mtime +7 -exec rm {} ; > /dev/null 2>&1
The 2 paths are not the same. And the path on the find doesn't even exist! So I think I'll try changing the path on the find to match the path on the '-d' test.
Maybe a bug?

[wfweaver]

I too have had issues with dspam. Ran great for quite a long time then started having issues with dspam_clean chewing up all of memory and eventually causing swapping. Now it fills up spamtrain.log with "Can't allocate memory" errors. Opened a case and was told the issue was dspam_clean with hashes. The solution I was given was to run zmamavisdctl stop, and then run cssclean and csscompress on the zimbra.css file and then zmamavisdctl start once it finished. This worked well for a while. Now whenever I run cssclean I get a segfault and csscompress gives me a File size limit exceeded error - so I'm back to square one! Zimbra tells me it isn't really supported (though provided - they turned it off for performance reasons), but I've found that not running it provides unacceptable levels of spam so I must have it. The only other option I was given was to convert from the hash driver to a SQL driver - something I'd rather not do as it would break on every upgrade. Anyone have any other solution they can provide?:(

[luka123]

Hi,
I was just searching forum about DSPAM clean locking up and found this threat.

We are having similar issue at zimbra network edition 5.0.23. After upgrade from 5.0.11 I started DSPAM and every thing was working well for 2 months . Then suddenly zimbra MTA started to deferred mails during night. I found out that the process which was using almost 100% of CPU was /opt/zimbra/dspam/bin/dspam_clean . Then I disable DSPAM end everything was working well. Now I'm looking for solution how to resolve this issue.
Have you been able to resolve your DSPAM issue ? I would be grateful if you will respond.
Best regards Luka

[LaFong]

I have had dspam enabled for a year or so. Twice since I upgraded to 6.0.x, dspam began failing with segfaults. The first time, I reran zmtrainsa manually. Then, that didn't work. I tried cssclean, but it either did nothing, or gave a "cssclean failed on error -5" message. In /var/log/maillog, there was this error.

Jun 26 18:10:01 mail dspam[8943]: Unable to open file for reading: /usr/local/etc/dspam.conf: No such file or directory

I do not see an option to specify config file for cssclean, so it must be compiled in.
csscompress also failed to work. I tried the cssclean contained in the dspam 3.9.0 final release included with 6.0.7.1. Same error. (It looks like 6.0.8 is the same too.) I downloaded and manually compiled dspam 3.9.0, same problem. Finally, I got the head revisions for cssclean.c and csscompress.c and recompiled. Those versions worked. And fixed the segfaults. They are newer than the versions contained in 3.9.0, and are available here and here.
I then made an entry in the Zimbra crontab, to run dspam css maintenance monthly:

# Monthly maintenance of dspam hash db

0 4 * * 7 [ $(date +%e) -le 7 ] && /opt/zimbra/libexec/dspam_css_maintenance.sh

This runs the dspam maintenance script on the first Sunday of every month. Here is that script:



#!/bin/bash

# DSPAM maintenance script. It cleans and compresses the DSPAM hash db.

MAIL="it@somewhere.com"
# Disable dspam, temporarily

zmlocalconfig -u amavis_dspam_enabled

#zmamavisdctl restart
# Run cssclean and csscompress

/opt/zimbra/dspam/bin/cssclean /opt/zimbra/data/dspam/data/z/i/zimbra/zimbra.css > /var/log/dspam_css_maintenance.log

echo >> /var/log/dspam_css_maintenance.log

echo Compress >> /var/log/dspam_css_maintenance.log

echo -------- >> /var/log/dspam_css_maintenance.log

/opt/zimbra/dspam/bin/csscompress /opt/zimbra/data/dspam/data/z/i/zimbra/zimbra.css >> /var/log/dspam_css_maintenance.log
# Reenable dspam

zmlocalconfig -e amavis_dspam_enabled=true

#zmamavisdctl restart
# Run cssstats, and mail results

echo >> /var/log/dspam_css_maintenance.log

echo Stats >> /var/log/dspam_css_maintenance.log

echo ----- >> /var/log/dspam_css_maintenance.log

/opt/zimbra/dspam/bin/cssstat /opt/zimbra/data/dspam/data/z/i/zimbra/zimbra.css >> /var/log/dspam_css_maintenance.log

mail -s "Monthly dspam maintenance" "$MAIL"


I know dspam is still not officially supported, but it seems like maintenance of the Zimbra dspam hash db's should be included in the Zimbra cronjob (commented out if necessary), and that the css hash utilities should work. There are already a few RFEs concerning dspam, not sure if I want to add one.