Rspamd: Fast, free and open-source spam filtering system

[chris_60]

Rspamd is working great with Zimbra.

Is there a way to have spam delivered to the user's junk mailbox in order to allow the user to review the mail in case it is not spam?

Chris

[phoenix]

You set the score for what is classified as spam, that will then get rejected if it exceeds the score you've defined but some spam will still (most likely) arrive in the users inboxes otherwise they will be in their Junk folder. If you are receiving mail from reputable senders then they should not be classified as spam and then arrive in the Inbox.

[chris_60]

Let me clarify my question somewhat.

Rspamd rejects scores over 15 (default). Those emails never reach the inbox or junk folder as far as I can tell.

Rspamd flags scores over 4 (default) as greylist. Those emails end up in the inbox.

Rspamd adds headers to scores over 6 (default). Those emails end up in the inbox.

What I am trying to do is to have mails scored over, say, 4 by Rspamd be directed to the junk folder.

It appears that with a basic configuration of Rspamd and Zimbra that an email incorrectly scored as >15 could never be seen by the user which presents a problem.

Thanks for the help!

Chris

[phoenix]

Take a look at the log files and see why those mails with a high score are rejected. All those settings you've mentioned can be modified by you. My question would be: why install an anti-spam system then cripple it by accepting mail into a users inbox?

[chris_60]

Take a look at the log files and see why those mails with a high score are rejected. All those settings you've mentioned can be modified by you.

I have been grepping through the logs and adjusting the settings. For the most part things work great. Very little is scored incorrectly.

My question would be: why install an anti-spam system then cripple it by accepting mail into a users inbox?

The goal is not to cripple it, but to render an experience similar to mainstream email providers where the user can view all spam and mark/un-mark accordingly. If a mail is flagged spam incorrectly and there is no way for the user to see that email (ie it never lands in any folder) then the user cannot "check the spam/junk folder" to correct the issue.

We definitely don't want anything scored over the absolute "spam" score in the inbox. We want it in the junk folder. Right now it is sent into the bit bucket.

[sangamc]

Rspamd is working great with Zimbra.

Is there a way to have spam delivered to the user's junk mailbox in order to allow the user to review the mail in case it is not spam?

Chris

I used the following:

Create filters for moving email with marked headers to the users Junk Folder
Add Filter Rule

Code: Select all

su - zimbra
all_accounts=`zmprov -l gaa| egrep -v 'admin|wiki|galsync|spam|ham|virus'`; for account in $all_accounts; do zmmailbox -z -m  $account afrl -f 'RSPAMD' active any header 'X-Spam' is 'Yes' fileinto "/Junk" stop; echo "Created filter for $account"; done ;

[chris_60]

I used the following:

Create filters for moving email with marked headers to the users Junk Folder
Add Filter Rule

Code: Select all

su - zimbra
all_accounts=`zmprov -l gaa| egrep -v 'admin|wiki|galsync|spam|ham|virus'`; for account in $all_accounts; do zmmailbox -z -m  $account afrl -f 'RSPAMD' active any header 'X-Spam' is 'Yes' fileinto "/Junk" stop; echo "Created filter for $account"; done ;

That works a trick!

SA must have some integration in Zimbra which handles this.

Kind regards,
Chris

[MisterM74]

I know, laughing while putting this message.

Drop the previous sentence.

Do you have a functional tutorial to integrate RSPAMD on the new version Zimbra Communautaire, because I would like in the same time, integrated ZEXTRAS, possible?

Yours truly
Mz

[phoenix]

I assume you're talking about the upcoming Zextras Carbonio? If you are, it hasn't been release yet and we'll have to see what's necessary to install Rspamd in there. FWIW, I wouldn't expect it to be too different than using ZCS.

[yawarniazi]

I am new to Zimbra and Rspamd world.

In a test environment I have just installed Zimbra on two VMs (mail.test1.com and mail.test2.com) for testing all its functionalities before going Live.

I followed Phoenix guide and successfully installed Rspamd, went through all settings, Rspamd is running fine without any error.

From userA@test2.com I sent an email to userB@test1.com. In user inbox I marked that email as Spam. email moved to the Junk folder.
At this stage I was expecting that next email from userA@test2.com to userB@test1.com should be marked as Spam and instead of appearing in Inbox it should directly go to the Junk/Spam folder of userB@test1.com.

But the said email is still coming in inbox and not marked as Spam/Junk.

rspamd.log result when I clicked on "Mark as Spam" button:

Code: Select all

2021-11-18 16:16:41 #2521(rspamd_proxy) <745876>; proxy; proxy_accept_socket: accepted milter connection from 127.0.0.1 port 40794
2021-11-18 16:16:41 #2521(rspamd_proxy) <745876>; milter; rspamd_milter_process_command: got connection from [b]172.17.48.4[/b]:44674
2021-11-18 16:16:42 #2521(rspamd_proxy) <745876>; proxy; rspamd_message_parse: loaded message; id: <1626800502.1.1637234201871.JavaMail.zimbra@test1.com>; queue-id: <EF894803D138A>; size: 5423; checksum: <5cd45641748f262d021809848909ebc9>
2021-11-18 16:16:42 #2521(rspamd_proxy) <745876>; proxy; rspamd_mime_part_detect_language: detected part language: en
2021-11-18 16:16:42 #2521(rspamd_proxy) <745876>; proxy; rspamd_mime_part_detect_language: detected part language: en
2021-11-18 16:16:42 #2521(rspamd_proxy) <745876>; proxy; dkim_symbol_callback: skip DKIM checks for local networks and authorized users
2021-11-18 16:16:42 #2521(rspamd_proxy) <745876>; lua; spf.lua:186: skip SPF checks for local networks and authorized users
2021-11-18 16:16:42 #2521(rspamd_proxy) <745876>; lua; dmarc.lua:349: skip DMARC checks as either SPF or DKIM were not checked
2021-11-18 16:16:42 #2521(rspamd_proxy) <745876>; proxy; dkim_module_load_key_format: cannot load dkim key /var/lib/rspamd/dkim/.dkim.key: cannot stat key file: '/var/lib/rspamd/dkim/.dkim.key' No such file or directory
2021-11-18 16:16:42 #2521(rspamd_proxy) <745876>; lua; once_received.lua:99: Skipping once_received for authenticated user or local network
2021-11-18 16:16:46 #2521(rspamd_proxy) <745876>; proxy; fuzzy_check_timer_callback: got IO timeout with server fuzzy2.rspamd.com:11335(xxx.xxx.xxx.xxx:11335), after 1/1 retransmits
2021-11-18 16:16:46 #2521(rspamd_proxy) <745876>; proxy; rspamd_stat_classifiers_process: skip statistics as SPAM class is missing
2021-11-18 16:16:46 #2521(rspamd_proxy) <745876>; proxy; rspamd_task_write_log: id: <1626800502.1.1637234201871.JavaMail.zimbra@test1.com>, qid: <EF894803D138A>, ip: 172.17.48.4, (default: F (no action): [2.90/15.00] [FROM_INVALID(2.00){},MID_RHS_MATCH_TO(1.00){},MIME_GOOD(-0.10){multipart/mixed;text/plain;},ARC_NA(0.00){},FROM_EQ_ENVFROM(0.00){},FROM_HAS_DN(0.00){},HAS_ATTACHMENT(0.00){},HAS_XOIP(0.00){},MIME_TRACE(0.00){0:+;1:+;2:~;3:+;},RCPT_COUNT_ONE(0.00){1;},RCVD_COUNT_ZERO(0.00){0;},TO_DN_NONE(0.00){},TO_MATCH_ENVRCPT_ALL(0.00){}]), len: 5423, time: 4006.968ms, dns req: 16, digest: <5cd45641748f262d021809848909ebc9>, rcpts: <spam@test1.com>, mime_rcpts: <spam@test1.com>
2021-11-18 16:16:46 #2521(rspamd_proxy) <745876>; proxy; rspamd_protocol_http_reply: regexp statistics: 0 pcre regexps scanned, 3 regexps matched, 176 regexps total, 42 regexps cached, 0B scanned using pcre, 1.50KiB scanned total
2021-11-18 16:16:46 #2521(rspamd_proxy) <dfd39d>; proxy; proxy_milter_finish_handler: finished milter connection

The next email still coming to inbox instead of Junk folder. Here is the rspamd.log for next email:

Code: Select all

2021-11-18 16:21:46 #2521(rspamd_proxy) <aae30d>; proxy; proxy_accept_socket: accepted milter connection from 127.0.0.1 port 40834
2021-11-18 16:21:47 #2521(rspamd_proxy) <aae30d>; milter; rspamd_milter_process_command: got connection from [b]172.18.75.61[/b]:50528
2021-11-18 16:21:48 #2521(rspamd_proxy) <aae30d>; proxy; rspamd_message_parse: loaded message; id: <fd84c31e26004ad9d54a0516279c3b41@test2.com>; queue-id: <6273D803D138A>; size: 3332; checksum: <5242c989b42488452926203b0576cede>
2021-11-18 16:21:48 #2521(rspamd_proxy) <aae30d>; proxy; rspamd_mime_part_detect_language: detected part language: en
2021-11-18 16:21:48 #2521(rspamd_proxy) <aae30d>; proxy; dkim_symbol_callback: skip DKIM checks for local networks and authorized users
2021-11-18 16:21:48 #2521(rspamd_proxy) <aae30d>; lua; spf.lua:186: skip SPF checks for local networks and authorized users
2021-11-18 16:21:48 #2521(rspamd_proxy) <aae30d>; lua; dmarc.lua:349: skip DMARC checks as either SPF or DKIM were not checked
2021-11-18 16:21:48 #2521(rspamd_proxy) <aae30d>; proxy; dkim_module_load_key_format: cannot load dkim key /var/lib/rspamd/dkim/test2.com.dkim.key: cannot stat key file: '/var/lib/rspamd/dkim/test2.com.dkim.key' No such file or directory
2021-11-18 16:21:48 #2521(rspamd_proxy) <aae30d>; lua; once_received.lua:99: Skipping once_received for authenticated user or local network
2021-11-18 16:21:49 #2521(rspamd_proxy) <aae30d>; proxy; rspamd_symcache_finalize_item: slow rule: URIBL_MULTI(257): 1003.19 ms; enable slow timer delay
2021-11-18 16:21:50 #2521(rspamd_proxy) <aae30d>; proxy; rspamd_symcache_finalize_item: slow rule: SEM_URIBL_FRESH15_UNKNOWN(254): 2268.30 ms; enable slow timer delay
2021-11-18 16:21:50 #2521(rspamd_proxy) <aae30d>; proxy; rspamd_symcache_finalize_item: slow rule: SEM_URIBL_UNKNOWN(250): 2269.30 ms
2021-11-18 16:21:50 #2521(rspamd_proxy) <aae30d>; [b]proxy; rspamd_stat_classifiers_process: skip statistics as SPAM class is missing[/b]
2021-11-18 16:21:50 #2521(rspamd_proxy) <aae30d>; proxy; rspamd_task_write_log: id: <fd84c31e26004ad9d54a0516279c3b41@test2.com>, qid: <6273D803D138A>, ip: 172.18.75.61, from: <userA@test2.com>, (default: F (no action): [-0.09/15.00] [MIME_GOOD(-0.10){text/plain;},XM_UA_NO_VERSION(0.01){},ARC_NA(0.00){},FROM_EQ_ENVFROM(0.00){},FROM_NO_DN(0.00){},MID_RHS_MATCH_FROM(0.00){},MIME_TRACE(0.00){0:+;},PREVIOUSLY_DELIVERED(0.00){userB@test.com;},RCPT_COUNT_ONE(0.00){1;},RCVD_COUNT_THREE(0.00){4;},RCVD_TLS_LAST(0.00){},RCVD_VIA_SMTP_AUTH(0.00){},TO_DN_ALL(0.00){},TO_MATCH_ENVRCPT_ALL(0.00){}]), len: 3332, time: 2372.586ms, dns req: 16, digest: <5242c989b42488452926203b0576cede>, rcpts: <userB@test1.com>, mime_rcpts: <userB@test1.com>
2021-11-18 16:21:50 #2521(rspamd_proxy) <aae30d>; proxy; rspamd_protocol_http_reply: regexp statistics: 0 pcre regexps scanned, 3 regexps matched, 176 regexps total, 68 regexps cached, 0B scanned using pcre, 2.05KiB scanned total
2021-11-18 16:21:50 #2521(rspamd_proxy) <06b048>; proxy; proxy_milter_finish_handler: finished milter connection

Here is the rusult of zmtrainsa command:

Code: Select all

20211118162804 Starting spam/ham extraction from system accounts.
[] INFO: Total messages processed: 4
[] INFO: Total messages processed: 1
20211118162809 Finished extracting spam/ham from system accounts.
20211118162809 Starting rspamd system accounts training.
20211118162809 List rspam stats before training.
Results for command: stat (0.177 seconds)
Messages scanned: 131
Messages with action reject: 0, 0.00%
Messages with action soft reject: 0, 0.00%
Messages with action rewrite subject: 0, 0.00%
Messages with action add header: 0, 0.00%
Messages with action greylist: 0, 0.00%
Messages with action no action: 131, 100.00%
Messages treated as spam: 0, 0.00%
Messages treated as ham: 131, 100.00%
Messages learned: 0
Connections count: 0
Control connections count: 0
Pools allocated: 217
Pools freed: 192
Bytes allocated: 26.80MiB
Memory chunks allocated: 133
Shared chunks allocated: 15
Chunks freed: 0
Oversized chunks: 2
Fuzzy hashes in storage "rspamd.com": 4224031428
Fuzzy hashes stored: 4224031428
Total learns: 0

Results for file: /tmp/spam.tQyc2pT/17d32cdc2dd-3 (0.284 seconds)
HTTP error: 404, all learn conditions denied learning spam in default classifier

Results for file: /tmp/spam.tQyc2pT/17d32cdc2dd-2 (0.284 seconds)
HTTP error: 404, all learn conditions denied learning spam in default classifier

Results for file: /tmp/spam.tQyc2pT/17d32cdc2dd-1 (0.284 seconds)
HTTP error: 404, all learn conditions denied learning spam in default classifier

Results for file: /tmp/spam.tQyc2pT/17d32cdc2dd-0 (0.284 seconds)
HTTP error: 404, all learn conditions denied learning spam in default classifier

Results for file: /tmp/ham.1UcxFcU/17d32cdcc6f-0 (0.001 seconds)
HTTP error: 404, all learn conditions denied learning ham in default classifier

20211118162810 List rspam stats after training.
Results for command: stat (0.231 seconds)
Messages scanned: 131
Messages with action reject: 0, 0.00%
Messages with action soft reject: 0, 0.00%
Messages with action rewrite subject: 0, 0.00%
Messages with action add header: 0, 0.00%
Messages with action greylist: 0, 0.00%
Messages with action no action: 131, 100.00%
Messages treated as spam: 0, 0.00%
Messages treated as ham: 131, 100.00%
Messages learned: 0
Connections count: 0
Control connections count: 6
Pools allocated: 223
Pools freed: 198
Bytes allocated: 26.80MiB
Memory chunks allocated: 133
Shared chunks allocated: 15
Chunks freed: 0
Oversized chunks: 2
Fuzzy hashes in storage "rspamd.com": 4224031428
Fuzzy hashes stored: 4224031428
Total learns: 0

20211118162810 Finished rspamd training.

In spam@test1.com and ham@test1.com I am getting reports on clicking "Mark as Spam" and "Mark not as Spam", but new emails from userA@test2.com are still appearing in inbox.

I searched many forums but unable to find the culprit of "HTTP error: 404, all learn conditions denied learning spam in default classifier" error after running zmtrainsa command.

I am near to finish the testing of all functionalities of Zimbra and ready to go Live. Stuck at "Mark as Spam" button.

Kindly assist me if I am doing anything wrong. Waiting for a quick help.

Thanks in advance.

yawarniazi