Rewrote zmtrainsa for DSPAM

Have a great idea for extending Zimbra? Share ideas, ask questions, contribute, and get feedback.
unilogic
Advanced member
Advanced member
Posts: 51
Joined: Fri Sep 12, 2014 10:01 pm

Rewrote zmtrainsa for DSPAM

Post by unilogic »

I rewrote zmtrainsa to make it a bit more scalable as a training tool for spam filters. My version now grabs the spam or ham messages, and stores them temporarily so more then one spam trainer can take a look at it. Currently I have to split these functions up into two files because of fetchmail's limitations. Right now SA Bayes and DSPAM will be trained. If DSPAM is not installed it will ignore training DSPAM. Also I assumed that dspam was installed in /opt/dspam/bin. If your install is elsewhere change the DSPAMDIR variable in each file. Also, the temp location used is /tmp/sa. That can be changed with the TEMPDIR variable in zmtrainfilter. Run zmtrainsa with its normal usage, zmtrainfilter is really just a support file. Take a look at the files if you need them. The zmtrainsa script now needs zmtrainfilter to run. Although zmtrainfilter doesn't necessarily need zmtrainsa for single file input. I'll comment the code better when I get time. If you have any problems, or don't know why I did something let me know.
-Ben
Download latest fix on the second page of posts.

The files are compressed and attached as zmtrainsa.tgz
zmtrainsa.tgz
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

Rewrote zmtrainsa for DSPAM

Post by phoenix »

Hi
Unfortunately your scripts don't work for me. the first problem is that zmtrainsa doesn't recognise that zmtrainfilter is in the same directory and I've previously had a problem with zmtrainsa that was resolved by a new script from marcmac in this thread zmtrainsa
Although the new zmtrainsa script was for a problem with spaces in folder names your script exhibits the same problem even if the folder has a single name i.e. Junk.
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
unilogic
Advanced member
Advanced member
Posts: 51
Joined: Fri Sep 12, 2014 10:01 pm

Rewrote zmtrainsa for DSPAM

Post by unilogic »

Try out this fix, it should fix the above problems. Although I have no idea why it can't filter zmtrainfilter, but I did change that around a bit so hopfully it'll work. The folders with spaces problem carried over from the orignal zmtrainsa that came with M2. Sorry about that. Let me know if it still doesn't work.
Ben
zmtrainsa.tgz
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

Rewrote zmtrainsa for DSPAM

Post by phoenix »

Tried that fix and it now finds the zmtrainfilter but gives the following error:
$ zmtrainsa server user password spam junk

/opt/zimbra/bin/zmtrainsa: line 70: /opt/zimbra/.fetchmailrc: No such file or directory

chmod: cannot access `/opt/zimbra/.fetchmailrc': No such file or directory

Enter password for bpye@chinook:

fetchmail: lock creation failed.
Syncing SA bayes database
Cleaning up DSpam, this may take a while

dspam_clean starting

PROCESSING USER: zimbra

Processing sigs; age: 14

Signature: 438f67a5116903652140007

Created: Thu Dec 1 00:00:00 2005
Signature: 438f67a5116911847784999

Created: Thu Dec 1 00:00:00 2005
...... plus a lot more of the above messages.......



I did enter the correct user password when asked.
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
unilogic
Advanced member
Advanced member
Posts: 51
Joined: Fri Sep 12, 2014 10:01 pm

Rewrote zmtrainsa for DSPAM

Post by unilogic »

Ah yes, the problem is that its trying to create a file as the zimbra user in the /opt/zimbra folder which zimbra actually has no rights to. So I can either put the .fetchmailrc file in a different place say $TMPDIR or you can chown zimbra:zimbra to /opt/zimbra. Sorry about these mistakes. I've done alot to my install so alot of things aren't at default anymore and I overlook them. I've attached the fixed scripts. Let me know if you have any problems.
-Ben
zmtrainsa.tgz
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

Rewrote zmtrainsa for DSPAM

Post by phoenix »

Hi
I'm not sure if you wanted me to make any changes for this test but I ran it 'as is' and got these results:
rm: cannot get current directory: Permission denied

mkdir: cannot create directory `/tmp/sa': File exists

fetchmail: lock creation failed.
Syncing SA bayes database
Cleaning up DSpam, this may take a while

dspam_clean starting

PROCESSING USER: zimbra

Processing sigs; age: 14

Signature: 438f67a5116903652140007

Created: Thu Dec 1 00:00:00 2005
.................. lots more of the above and then this
Processing probabilities; age: 0

Processing unused; any: 90 quota: 30 nospam: 15 onehit: 15
Cleaning up temp files

rm: cannot get current directory: Permission denied

Done!


Thanks for your effort with this. :)
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
unilogic
Advanced member
Advanced member
Posts: 51
Joined: Fri Sep 12, 2014 10:01 pm

Rewrote zmtrainsa for DSPAM

Post by unilogic »

You've run the script as another user then zimbra at some point and its left /tmp/sa. Simply remove that directory manually and all should work. I'll write into the script later today checking that you have permissions for $TMPDIR and throwing an error if you don't. Thanks for being my beta test :)
-Ben
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

Rewrote zmtrainsa for DSPAM

Post by phoenix »

I don't think I've run this as anything other than the zimbra user, the /tmp/sa file was still there with zimbra as the owner/group so I deleted it and ran the test again. This is the results I got this time:
fetchmail: lock creation failed.
Syncing SA bayes database

bayes: synced databases from journal in 2 seconds: 237 unique entries (531 total entries)
Cleaning up DSpam, this may take a while

dspam_clean starting

PROCESSING USER: zimbra

Processing sigs; age: 14

Signature: 438f67a5116903652140007

Created: Thu Dec 1 00:00:00 2005
----- lots more of the above
Signature: 4395997a87243611876707

Created: Tue Dec 6 00:00:00 2005
Processing probabilities; age: 0

Processing unused; any: 90 quota: 30 nospam: 15 onehit: 15
Cleaning up temp files

rm: cannot get current directory: Permission denied

Done!



The /tmp/sa directory also still exists and the owner/group is zimbra.
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
unilogic
Advanced member
Advanced member
Posts: 51
Joined: Fri Sep 12, 2014 10:01 pm

Rewrote zmtrainsa for DSPAM

Post by unilogic »

Give this a try. I think what was happening by chance did you su to zimbra, and you ran zmtrainsa. But the current working directory you are still in zimbra doesn't have rights to. I made the scripts change to zimbra's home dir so give it a try. Also a 6th option of where you want the temp directory to be was added. Note: it will run without this option being supplied.
-Ben
zmtrainsa.tgz
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

Rewrote zmtrainsa for DSPAM

Post by phoenix »

Yes, you're correct I was in a different directory when I ran your zmtrainsa. I wasn't bothering to change as the original zmtrainsa worked. OK, new run and another error I'm afraid:
not found using default location /tmp/sa

fetchmail: lock creation failed.
Syncing SA bayes database
Cleaning up DSpam, this may take a while

dspam_clean starting

PROCESSING USER: zimbra

Processing sigs; age: 14

Signature: 438f67a5116903652140007

Created: Thu Dec 1 00:00:00 2005

--------------------

Processing probabilities; age: 0

Processing unused; any: 90 quota: 30 nospam: 15 onehit: 15
Cleaning up temp files

Done!
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
Post Reply