Custom SpamAssassin Rules doesn't affect after upgrade to 8.8.12

sergetsp · Post by **sergetsp** » Sun Apr 07, 2019 3:54 pm

Hello, all,

I have used custom rules of SpamAssassin in my /opt/zimbra/data/spamassassin/localrules/sauser.cf
file.

After upgrade Zimbra from version 8.6.0 to 8.8.12 the rules from this sauser.cf file no longer work.

I have read release notes, but information of new custom rules location not found.

Can you help me, where is the new location of custom SA rules,
or how I can specify my own rules?

Thank you in advance.

pup_seba · Post by **pup_seba** » Tue Apr 09, 2019 7:29 pm

Hi,

There is no new location for the rules. The path is the same /opt/zimbra/data/spamassassin/localrules

I've recently done several upgrades to 8.8.12 and in all cases, rules were still applying with no further change needed. The rules remained intact after the upgrades were done.

So I guess it must be something specific to your zimbra? Can you check the file permissions on that file? During start of your services, can you see if any error shows in your /var/log/zimbra.log? (something like cannot read file or bad format or something?)

sergetsp · Post by **sergetsp** » Fri Apr 12, 2019 3:24 pm

Thank you for reply. All pertittion for file is good:

[root@zmail localrules]# pwd
/opt/zimbra/data/spamassassin/localrules

[root@zmail localrules]# ls -la
total 60
drwxr-xr-x. 3 zimbra zimbra 4096 Apr 12 18:07 .
drwxr-xr-x. 5 zimbra zimbra 4096 Apr 12 18:07 ..
-rw-r--r--. 1 zimbra zimbra 1289 Jul 10 2017 init.pre
-rw-r--r--. 1 zimbra zimbra 2379 Jul 10 2017 local.cf
-r--r-----. 1 zimbra zimbra 4322 Apr 12 17:29 salocal.cf
drwx------. 2 zimbra zimbra 4096 Apr 12 18:07 sa-update-keys
-rw-r--r--. 1 zimbra zimbra 6846 Apr 2 13:52 sauser.cf
-rw-r--r--. 1 zimbra zimbra 2523 Jul 10 2017 v310.pre
-rw-r--r--. 1 zimbra zimbra 1194 Jul 10 2017 v312.pre
-rw-r--r--. 1 zimbra zimbra 2414 Jul 10 2017 v320.pre
-rw-r--r--. 1 zimbra zimbra 1237 Jul 10 2017 v330.pre
-rw-r--r--. 1 zimbra zimbra 1020 Jul 10 2017 v340.pre
-rw-r--r--. 1 zimbra zimbra 1309 Jul 10 2017 v341.pre

After "zmamavisdctl stop/start" Amavis starting correctly:

Apr 12 17:29:55 zmail zmconfigd[2106]: Rewrote: /opt/zimbra/conf/dspam.conf with mode 440 (0.03 sec)
Apr 12 17:29:55 zmail zmconfigd[2106]: Rewrote: /opt/zimbra/conf/amavisd.conf with mode 440 (0.03 sec)
Apr 12 17:29:55 zmail zmconfigd[2106]: Rewrote: /opt/zimbra/data/spamassassin/localrules/salocal.cf with mode 440 (0.01 sec)

It seems the main problem is amavis not checking mail at all, no spam hits calculating, Hits always is absent on *all* email messages (after upgrade),
here is the log:

Apr 12 18:09:53 zmail amavis[18067]: (18067-03-2) ESMTP [127.0.0.1]:10026 /opt/zimbra/data/amavisd/tmp/amavis-20190412T180811-18067-WpCh9x9W: <user@gmail.com> ->
<tsp@user.ru> SIZE=2731 Received: from zmail.user.ru ([127.0.0.1]) by localhost (zmail.user.ru [127.0.0.1]) (amavisd-new, port 10026) with ESMTP for <tsp@user.ru>; Fri, 12
Apr 2019 18:09:53 +0300 (MSK)
Apr 12 18:09:53 zmail amavis[18067]: (18067-03-2) Checking: JPGKUUK92fvp ORIGINATING/MYNETS [88.201.160.50] <user@gmail.com> -> <tsp@user.ru>
Apr 12 18:09:53 zmail amavis[18067]: (18067-03-2) JPGKUUK92fvp FWD from <user@gmail.com> -> <tsp@user.ru>, BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250
2.0.0 Ok: queued as 889EC7FF5E
Apr 12 18:09:53 zmail amavis[18067]: (18067-03-2) Passed CLEAN {RelayedInternal}, ORIGINATING/MYNETS LOCAL [88.201.160.50]:46454 [209.85.166.66] <user@gmail.com> ->
<tsp@user.ru>, Queue-ID: 49AD47FF72, Message-ID: <CAPjB2-3=f0PfWs8bxyTakUSMwdJ_=E7pj3uBZGVpu0foesuGhw@mail.gmail.com>, mail_id: JPGKUUK92fvp, Hits: -, size: 2731, queued_as:
889EC7FF5E, dkim_sd=20161025:gmail.com, 268 ms

How I can check main Spamassassin setting?

ps.

[zimbra@zmail conf]$ zmlocalconfig --show| fgrep antisp

antispam_backup_retention = 0
antispam_enable_restarts = true
antispam_enable_rule_compilation = true
antispam_enable_rule_updates = true

antispam_mysql_data_directory = ${zimbra_home}/data/amavisd/mysql/data
antispam_mysql_enabled = false
antispam_mysql_errlogfile = ${zimbra_home}/log/antispam-mysqld.log
antispam_mysql_host = 127.0.0.1
antispam_mysql_mycnf = ${zimbra_home}/conf/antispam-my.cnf
antispam_mysql_password =
antispam_mysql_pidfile = ${zimbra_home}/data/amavisd/mysql/mysql.pid
antispam_mysql_port = 7308
antispam_mysql_root_password =
antispam_mysql_socket = ${zimbra_home}/data/amavisd/mysql/mysql.sock
antispam_mysql_user = zimbra

JDunphy · Post by **JDunphy** » Fri Apr 12, 2019 8:24 pm

Very odd. Starting amavisd(SA) is guarded by the statement below looking to see if it's enabled. Perhaps verify it is still enabled. Note: I am running 8.7+ so my output could be slightly different.

Code: Select all

#su - zimbra
% [zimbra@tmail ~]$ /opt/zimbra/bin/zmprov -l gs `/opt/zimbra/bin/zmhostname` zimbraServiceEnabled | grep  antispam
zimbraServiceEnabled: antispam

Maybe also check /opt/zimbra/data/spamassassin/localrules/salocal.cf in case there is something odd.

amavis is started with zmamavisdctl
antivirus is started with zmantivirusctl
antispam is started with zmantispamctl

zmcontrol start will call zmantispamctl which will call antispam-mysql.server and zmanvisdctl

You mentioned that it started but only show the rewrote lines and not the amavis from your grep of /var/log/zimbra.log. There is going to be a lot of information including the calling SA parse statements, connections to ldap, new sockets, etc, etc.

Code: Select all

% [zimbra@tmail log]$ grep amavi /var/log/zimbra.log
Apr 12 03:09:11 tmail zmconfigd[31449]: Rewrote: /opt/zimbra/conf/amavisd.conf with mode 440 (0.05 sec)
Apr 12 03:09:24 tmail zmconfigd[32588]: Rewrote: /opt/zimbra/conf/amavisd.conf with mode 440 (0.07 sec)
Apr 12 04:00:56 tmail amavis[26594]: SA rundown_child (0)
Apr 12 04:44:57 tmail amavis[27516]: SA rundown_child (0)
Apr 12 04:45:07 tmail amavis[27524]: SA rundown_child (0)
Apr 12 04:45:17 tmail amavis[27528]: SA rundown_child (0)
Apr 12 05:09:30 tmail zmconfigd[32588]: Tracking service amavis 
Apr 12 05:09:30 tmail zmconfigd[32588]: Watchdog: service amavis now available for watchdog.
Apr 12 08:00:58 tmail amavis[2727]: SA rundown_child (0)
Apr 12 08:44:59 tmail amavis[3645]: SA rundown_child (0)
Apr 12 08:45:09 tmail amavis[3655]: SA rundown_child (0)
Apr 12 08:45:19 tmail amavis[3659]: SA rundown_child (0)
...
Apr 12 11:42:40 tmail amavis[17626]: SpamControl: init_pre_fork on SpamAssassin done
Apr 12 11:42:40 tmail amavis[17626]: extra modules loaded after daemonizing/chrooting: /usr/share/perl5/Net/libnet.cfg, Mail/SpamAssassin/Plugin/Attachments.pm, Mail/SpamAssassin/Plugin/FreeMail.pm, Mail/SpamAssassin/Plugin/HashBL.pm, Mail/SpamAssassin/Plugin/Image.pm, Mail/SpamAssassin/Plugin/OLEMacro.pm, Mail/SpamAssassin/Plugin/SpamCop.pm, Net/Cmd.pm, Net/Config.pm, Net/SMTP.pm

BTW, A command to watch amavisd processing through your system. Helpful for stuck or long lived email checks that are cpu bound.

Code: Select all

% amavisd-status --help
States legend:
  A  accepted a connection
  b  begin with a protocol for accepting a request
  m  'MAIL FROM' smtp command started a new transaction in the same session
  d  transferring data from MTA to amavisd
  =  content checking just started
  G  generating and verifying unique mail_id
  D  decoding of mail parts
  V  virus scanning
  S  spam scanning
  P  pen pals database lookup and updates
  r  preparing results
  Q  quarantining and preparing/sending notifications
  F  forwarding mail to MTA
  .  content checking just finished
  sp space indicates idle (elapsed time bar is showing dots)

Usage: /opt/zimbra/common/sbin/amavisd-status [-c <count>] [-w <wait-interval>]
exited
% amavisd-status -c 5
...

sergetsp · Post by **sergetsp** » Sat Apr 13, 2019 7:32 pm

Thank you,

I have checked mailsystem, SpamAssassin seems working fine.

My zimbra server is not a primary MX relay and incoming mail comes from my other mail server
(it is a primary MX).

If I try to send mail directly to my Zimbra server (from Internet to 25 port Zimbra) the system full checking
is completed and I see the ratio of email message (Hits: 2.98, for example).

But if email messages comes from my primary MX server
the SpamAssassin is not checked messages (Hits: - ).

Perhaps after upgrade I have new configuration of SpamAssassin (may be other trusted_network or internal_network setting ? ).
Now I have not access to my old spamassassin configuration (before upgrade Zimbra) and can't check old and new settings.

Here is zimbra log messages from my primary mx (Hits: -)

Apr 13 21:34:57 zmail amavis[8333]: (08333-05) Passed CLEAN {RelayedInternal}, ORIGINATING/MYNETS LOCAL [88.201.160.50]:40647 [209.85.166.175] <serge@gmail.com> -> <tsp@users.pm>, Queue-ID: DD2857FF09, Message-ID: <CAPjB2-1M6dAb5K+VE+m5twY8a37r+SMNbWJaG2qax2cy34QMkw@mail.gmail.com>, mail_id: OUqGCzliSe7g, Hits: -, size: 2502, queued_as: 06E027FF55, dkim_sd=20161025:gmail.com, 125 ms

Code: Select all

[root@zmail localrules]# f 08333-05 /var/log/zimbra.log
Apr 13 21:34:56 zmail amavis[8333]: (08333-05) ESMTP [127.0.0.1]:10026 /opt/zimbra/data/amavisd/tmp/amavis-20190413T213108-08333-ZN1a9bzJ: <serge@gmail.com> -> <tsp@users.pm> SIZE=2502 Received: from zmail.users.com ([127.0.0.1]) by localhost (zmail.users.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP for <tsp@users.com>; Sat, 13 Apr 2019 21:34:56 +0300 (MSK)
Apr 13 21:34:57 zmail amavis[8333]: (08333-05) dkim: VALID Author+Sender+MailFrom signature by d=gmail.com, From: <serge@gmail.com>, a=rsa-sha256, c=relaxed/relaxed, s=20161025, i=@gmail.com, ORIG [88.201.160.50]:40647
Apr 13 21:34:57 zmail amavis[8333]: (08333-05) Checking: OUqGCzliSe7g ORIGINATING/MYNETS [88.201.160.50] <serge@gmail.com> -> <tsp@users.pm>
Apr 13 21:34:57 zmail amavis[8333]: (08333-05) p001 1 Content-Type: text/plain, size: 19 B, name:
Apr 13 21:34:57 zmail amavis[8333]: (08333-05) OUqGCzliSe7g FWD from <serge@gmail.com> -> <tsp@users.pm>, BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 06E027FF55
Apr 13 21:34:57 zmail amavis[8333]: (08333-05) Passed CLEAN {RelayedInternal}, ORIGINATING/MYNETS LOCAL [88.201.160.50]:40647 [209.85.166.175] <serge@gmail.com> -> <tsp@users.pm>, Queue-ID: DD2857FF09, Message-ID: <CAPjB2-1M6dAb5K+VE+m5twY8a37r+SMNbWJaG2qax2cy34QMkw@mail.gmail.com>, mail_id: OUqGCzliSe7g, Hits: -, size: 2502, queued_as: 06E027FF55, dkim_sd=20161025:gmail.com, 125 ms
Apr 13 21:34:57 zmail amavis[8333]: (08333-05) size: 2502, TIMING [total 128 ms, cpu 53 ms] - SMTP greeting: 2.0 (2%)2, SMTP EHLO: 0.6 (0%)2, SMTP pre-MAIL: 0.4 (0%)2, lookup_ldap: 6 (5%)7, SMTP pre-DATA-flush: 0.9 (1%)8, SMTP DATA: 32 (25%)33, check_init: 0.2 (0%)34, digest_hdr: 4.9 (4%)37, digest_body_dkim: 12 (9%)47, collect_info: 3.2 (2%)49, mime_decode: 6 (5%)54, get-file-type1: 1.2 (1%)55, parts_decode: 0.1 (0%)55, check_header: 0.4 (0%)55, AV-scan-1: 4.2 (3%)59, decide_mail_destiny: 0.8 (1%)59, notif-quar: 0.2 (0%)59, fwd-connect: 3.8 (3%)62, fwd-mail-pip: 1.1 (1%)63, fwd-rcpt-pip: 0.1 (0%)63, fwd-data-chkpnt: 0.0 (0%)63, write-header: 0.6 (0%)64, fwd-data-contents: 0.0 (0%)64, fwd-end-chkpnt: 37 (29%)92, prepare-dsn: 0.7 (1%)93, report: 2.2 (2%)95, main_log_entry: 4.7 (4%)98, update_snmp: 0.4 (0%)99, SMTP pre-response: 0.2 (0%)99, SMTP response: 0.4 (0%)99, unlink-2-files: 0.3 (0%)99, rundown: 0.8 (1%)100
Apr 13 21:34:57 zmail amavis[8333]: (08333-05) size: 2502, RUSAGE minflt=420+0, majflt=0+0, nswap=0+0, inblock=0+0, oublock=32+0, msgsnd=0+0, msgrcv=0+0, nsignals=0+0, nvcsw=33+0, nivcsw=37+0, maxrss=116172+0, ixrss=0+0, idrss=0+0, isrss=0+0, utime=0.047+0.000, stime=0.006+0.000
Apr 13 21:34:57 zmail amavis[8333]: (08333-05) extra modules loaded: unicore/lib/gc_sc/SpacePer.pl

And here is log messages, when mail comes directly to Zimbra sever

Apr 13 21:41:52 zmail amavis[8337]: (08337-15) Passed CLEAN {RelayedInbound}, [82.202.216.59]:57603 [82.202.216.59] <user@user.com> -> <tsp@users.pm>, Queue-ID: ECF707FF09, Message-ID: <20190413184131.ECF707FF09@zmail.users.com>, mail_id: tpR1UjD_56aX, Hits: 2.899, size: 769, queued_as: A736F7FF55, 2487 ms

Code: Select all

Apr 13 21:41:50 zmail amavis[8337]: (08337-15) ESMTP [127.0.0.1]:10024 /opt/zimbra/data/amavisd/tmp/amavis-20190413T213141-08337-6dbJBvuj: <user@user.com> -> <tsp@users.pm> SIZE=770 Received: from zmail.users.com ([127.0.0.1]) by localhost (zmail.users.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP for <tsp@users.com>; Sat, 13 Apr 2019 21:41:50 +0300 (MSK)
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) Checking: tpR1UjD_56aX [82.202.216.59] <user@user.com> -> <tsp@users.pm>
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) p001 1 Content-Type: text/plain, size: 368 B, name:
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: config: time limit 300.0 s
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: message: main message type: text/plain
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: check: pms new, time limit in 299.996 s
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: check: adding caller rule hits, 0 rules
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: received-header: parsed as [ ip=82.202.216.59 rdns=listmaster.dneprovoi.ru helo=my.om by=zmail.users.com ident= envfrom= intl=0 id=ECF707FF09 auth= msa=0 ]
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: netset: trusted_networks lookup on 82.202.216.59, 14 networks, result: 0, 1.725 ms
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: received-header: relay 82.202.216.59 trusted? no internal? no msa? no
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: metadata: X-Spam-Relays-Trusted:
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: metadata: X-Spam-Relays-Untrusted: [ ip=82.202.216.59 rdns=listmaster.dneprovoi.ru helo=my.om by=zmail.users.com ident= envfrom= intl=0 id=ECF707FF09 auth= msa=0 ]
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: metadata: X-Spam-Relays-Internal:
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: metadata: X-Spam-Relays-External: [ ip=82.202.216.59 rdns=listmaster.dneprovoi.ru helo=my.om by=zmail.users.com ident= envfrom= intl=0 id=ECF707FF09 auth= msa=0 ]
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: check: tagrun - tag RELAYSUNTRUSTEDREVIP is now ready, value: 59.216.202.82
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: check: tagrun - tag RELAYSEXTERNALREVIP is now ready, value: 59.216.202.82
Apr 13 21:41:50 zmail amavis[8337]: (08337-15) SA dbg: askdns: depend on tags SENDERDOMAIN, rules: T_FROM_FMBLA_NEWDOM, T_FROM_FMBLA_NEWDOM14, T_FROM_FMBLA_NEWDOM28, T_FROM_FMBLA_NDBLOCKED

[ ... skip ... ]

Apr 13 21:41:52 zmail amavis[8337]: (08337-15) SA dbg: check: subtests=__BODY_TEXT_LINE,__BODY_TEXT_LINE,__BODY_TEXT_LINE,__DKIM_DEPENDABLE,__DOS_DIRECT_TO_MX,__DOS_RCVD_SAT,__DOS_SINGLE_EXT_RELAY,__ENV_AND_HDR_FROM_MATCH,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__E_LIKE_LETTER,__FORGED_SENDER,__HAS_DATE,__HAS_FROM,__HAS_MESSAGE_ID,__HAS_MSGID,__HAS_RCVD,__HAS_SUBJECT,__HAS_TO,__KAM_BODY_LENGTH_LT_1024,__KAM_BODY_LENGTH_LT_512,__KHOP_NO_FULL_NAME,__LAST_EXTERNAL_RELAY_NO_AUTH,__LAST_UNTRUS...
Apr 13 21:41:52 zmail amavis[8337]: (08337-15) ...TED_RELAY_NO_AUTH,__LCL__ENV_AND_HDR_FROM_MATCH,__LCL__KAM_BODY_LENGTH_LT_1024,__LCL__KAM_BODY_LENGTH_LT_512,__LONGWORDS_C,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__LOWER_E,__MISSING_REF,__MISSING_REPLY,__MSGID_OK_DIGITS,__MSGID_OK_HOST,__MSOE_MID_WRONG_CASE,__NONEMPTY_BODY,__RCVD_IN_DNSWL,__SANE_MSGID,__SUBJ_NOT_SHORT,__TOCC_EXISTS,__TO_NO_ARROWS_R,__YOU_WON,__YOU_WON_01
Apr 13 21:41:52 zmail amavis[8337]: (08337-15) SA dbg: timing: total 2375 ms - parse: 3.2 (0.1%), extract_message_metadata: 24 (1.0%), get_uri_detail_list: 0.59 (0.0%), tests_pri_-1000: 13 (0.5%), tests_pri_-950: 2.7 (0.1%), tests_pri_-900: 2.7 (0.1%), tests_pri_-90: 213 (9.0%), check_bayes: 200 (8.4%), b_tokenize: 6 (0.2%), b_tok_get_all: 173 (7.3%), b_comp_prob: 3.1 (0.1%), b_tok_touch_all: 0.17 (0.0%), b_finish: 10 (0.4%), tests_pri_0: 325 (13.7%), check_spf: 182 (7.7%), poll_dns_idle: 1887 (79.5%), check_dkim_adsp: 13 (0.6%), tests_pri_10: 3.2 (0.1%), tests_pri_20: 2.7 (0.1%), tests_pri_30: 2.9 (0.1%), check_pyzor: 0.63 (0.0%), tests_pri_500: 1755 (73.9%)
Apr 13 21:41:52 zmail amavis[8337]: (08337-15) SA dbg: check: tagrun - tag DKIMDOMAIN is still blocking action 1
Apr 13 21:41:52 zmail amavis[8337]: (08337-15) spam-tag, <user@user.com> -> <tsp@users.pm>, No, score=2.899 required=6.6 tests=[BAYES_50=0.8, RCVD_IN_DNSWL_NONE=-0.0001, TO_MALFORMED=2.099] autolearn=no autolearn_force=no
Apr 13 21:41:52 zmail amavis[8337]: (08337-15) tpR1UjD_56aX FWD from <user@user.com> -> <tsp@users.pm>, BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as A736F7FF55


Apr 13 21:41:52 zmail amavis[8337]: (08337-15) Passed CLEAN {RelayedInbound}, [82.202.216.59]:57603 [82.202.216.59] <user@user.com> -> <tsp@users.pm>, Queue-ID: ECF707FF09, Message-ID: <20190413184131.ECF707FF09@zmail.users.com>, mail_id: tpR1UjD_56aX, Hits: 2.899, size: 769, queued_as: A736F7FF55, 2487 ms
Apr 13 21:41:52 zmail amavis[8337]: (08337-15) TIMING-SA [total 2382 ms, cpu 292 ms] - parse: 3.2 (0.1%), extract_message_metadata: 24 (1.0%), get_uri_detail_list: 0.59 (0.0%), tests_pri_-1000: 13 (0.5%), tests_pri_-950: 2.7 (0.1%), tests_pri_-900: 2.7 (0.1%), tests_pri_-90: 213 (8.9%), check_bayes: 200 (8.4%), b_tokenize: 6 (0.2%), b_tok_get_all: 173 (7.3%), b_comp_prob: 3.1 (0.1%), b_tok_touch_all: 0.17 (0.0%), b_finish: 10 (0.4%), tests_pri_0: 325 (13.7%), check_spf: 182 (7.7%), poll_dns_idle: 1887 (79.2%), check_dkim_adsp: 13 (0.6%), tests_pri_10: 3.2 (0.1%), tests_pri_20: 2.7 (0.1%), tests_pri_30: 2.9 (0.1%), check_pyzor: 0.63 (0.0%), tests_pri_500: 1755 (73.7%), get_report: 1.27 (0.1%)
Apr 13 21:41:52 zmail amavis[8337]: (08337-15) size: 769, TIMING [total 2490 ms, cpu 328 ms, AM-cpu 36 ms, SA-cpu 292 ms] - SMTP greeting: 1.2 (0%)0, SMTP EHLO: 0.4 (0%)0, SMTP pre-MAIL: 0.4 (0%)0, lookup_ldap: 4.9 (0%)0, SMTP pre-DATA-flush: 0.8 (0%)0, SMTP DATA: 34 (1%)2, check_init: 0.2 (0%)2, digest_hdr: 0.4 (0%)2, digest_body_dkim: 0.1 (0%)2, collect_info: 1.1 (0%)2, mime_decode: 3.4 (0%)2, get-file-type1: 1.4 (0%)2, parts_decode: 0.1 (0%)2, check_header: 0.3 (0%)2, AV-scan-1: 3.6 (0%)2, spam-wb-list: 1.5 (0%)2, SA msg read: 1.0 (0%)2, SA parse: 5 (0%)2, SA check: 2375 (95%)98, decide_mail_destiny: 6 (0%)98, notif-quar: 0.2 (0%)98, fwd-connect: 4.5 (0%)98, fwd-mail-pip: 3.7 (0%)98, fwd-rcpt-pip: 0.1 (0%)98, fwd-data-chkpnt: 0.0 (0%)98, write-header: 0.2 (0%)98, fwd-data-contents: 0.0 (0%)98, fwd-end-chkpnt: 30 (1%)100, prepare-dsn: 0.9 (0%)100, report: 1.9 (0%)100, main_log_entry: 4.2 (0%)100, update_snmp: 0.5 (0%)100, SMTP pre-response: 0.1 (0%)100, SMTP response: 0.4 (0%)100, unlink-2-files:...
Apr 13 21:41:52 zmail amavis[8337]: (08337-15) ... 0.2 (0%)100, rundown: 1.6 (0%)100
Apr 13 21:41:52 zmail amavis[8337]: (08337-15) size: 769, RUSAGE minflt=4329+0, majflt=0+0, nswap=0+0, inblock=1160+0, oublock=40+0, msgsnd=0+0, msgrcv=0+0, nsignals=0+0, nvcsw=107+0, nivcsw=643+0, maxrss=117484+0, ixrss=0+0, idrss=0+0, isrss=0+0, utime=0.261+0.000, stime=0.067+0.000

JDunphy · Post by **JDunphy** » Sat Apr 13, 2019 8:41 pm

My environment is similar to yours. Double check @mynetworks in amavisd.conf that you don't have your MX listed there ... Sounds like policy_bank{'ORIGINATING'} is matching your MX. You can add your MX's in SA for trusted_networks in your salocal.cf ... While you are in there take a look at bypass_spam_checks_maps ... I'll take a closer look tomorrow how all this works because I am just guessing at cause.

sergetsp · Post by **sergetsp** » Sun Apr 14, 2019 7:41 am

Hello,

I have checked the /opt/zimbra/data/spamassassin/localrules/salocal.cf config file and it contains the option:

trusted_networks my-primary_MX-ip-subnet/30 10.32.0.0/16 10.37.0.0/16 10.33.0.0/16 127.0.0.0/8 192.168.122.0/24 192.168.255.0/24
lock_method flock
etc.

After "zmamavisdctl restart" main SpamAssassin config files seems reassembled, included "trusted_network".
But early (before update) I have other behavior, email messages from my primary MX are always checked.

Imho, after upgrade because of "trusted_networks" with IP of my primary MX is present in salocal.cf
the spam cheking is disabled.

What do you think about how is it possible to change trusted_networks for spam checking from primary MX server?

I know, that the settings are located in Zimbra WEB Administration at: Home - Configure - Servers - domain - MTA - MTA trusted networks
But before upgrade it is worked correctly.

Thank you in advance!

ps. in amavisd.conf I have not found the trusted_networks option (only in salocal.sf)

JDunphy · Post by **JDunphy** » Sun Apr 14, 2019 9:56 pm

I am starting to think the following happened... but you haven't confirmed amavisd.conf yet... So here is a guess in the absence of that.

- 8.7 introduced https://wiki.zimbra.com/wiki/Zimbra_Col ... Postscreen
- You probably had zimbraMtaMyNetworks configured with your MX... meaning that you could relay through your zimbra server from your MX. You can verify this by telnetting from your MX to your Zimbra server and issue a 'rcpt to' that is say a gmail address. On our Zimbra server we would deny that request from our MX's because while we trust them we don't trust them that much.

That ldap variable is also used in amavisd.conf to populate the mynetworks in amavisd.conf according to amavisd.conf.in

So why did it work previously on 8.6 and then break during upgrade? Perhaps related to postscreen or a newer version of SA being introduced with 8.7 and 8.8+ that changed some behavior you were expecting... I have been using the same configuration since 2005 and the upgrade didn't break us but I don't trust our MX's to relay through Zimbra and/or have them in our zimbraMtaMyNetworks... Our front facing MX's are not zimbra but roll your own MTA's. In fact our Zimbra servers don't even have MX records associated with them.

It is a little difficult to give advice here because your policy probably determined the method you chose. We have zimbra connect directly for outgoing delivery and all our MX's can delivery directly to Zimbra regardless of their MX weighting. We don't pass on MX requests to next MX in the chain for example but use the weighting for extra checks given some bots will always connect to the last MX...even if you have 5 MX's all with different weights... they start with the last one. We welcome that behavior.

If you are required to have your MX in zimbraMtaMyNetworks, perhaps investigate internal_networks which is like trusted_networks in your salocal.cf. Ref: https://spamassassin.apache.org/full/3. ... _Conf.html

BTW, very good catch on your part to have identified that your delivery by your MX is behaving like an internal delivery.

Maybe others have a better solution.

HTH,

Jim

JDunphy · Post by **JDunphy** » Sun Apr 14, 2019 11:32 pm

Note: if you run spamassasin from the command line with the -D option with any email (ie. copy from show original and create a text file to use as input)... Look for received-header to see how trusted_networks is being used by SA and it will tell you exactly what is happening and why the choices it is making. So think of trusted_hosts as the last trusted hop before it gets serious about those ip's in the next received header in this context.

I was thinking that adding your MX's to trusted_networks in your local salocal.cf and removing the MX from mynetworks if my earlier posts were not clear would fix this for you. Probably simplest way is to set the ldap variable I mentioned earlier and restart to get all those changes and update everything. If you go that route, don't forget to add your MX to your salocal.cf trusted_networks before restarting since it won't be propagated any longer.

If you don't have your MX in mynetworks inside amavisd.conf, I am missing something pretty major from your description of the problem.

sergetsp · Post by **sergetsp** » Mon Apr 15, 2019 8:56 am

Thank you very much, Jim,

I thinking about your anser, and here is my amavids.conf regarding your notes about rules for mynetworks:

(just fast answer)

The @mynetworks option include my localnets and public subnet of my primary MX.

Is it possible to exclude IP address of my primary MX from "exclide list" of SPAM chechikg?

The settings below:

Code: Select all

@mynetworks = qw( 88.210.150.0/30 10.32.0.0/16 10.37.0.0/16 );

and

$policy_bank{'ORIGINATING'} = {  # mail supposedly originating from our users
  originating => 1,  # declare that mail was submitted by our smtp client
  allow_disclaimers => 0,  # enables disclaimer insertion if available
  # notify administrator of locally originating malware
  virus_admin_maps => ['admin@zmail.mydomain.com'],
  spam_admin_maps  => ['admin@zmail.mydomain.com'],
  warnbadhsender   => 0,
  bypass_spam_checks_maps   => [1], # don't spam-check internal mail
  # forward to a smtpd service providing DKIM signing service
#  forward_method => 'smtp:[127.0.0.1]:10030',
  # force MTA conversion to 7-bit (e.g. before DKIM signing)
#  smtpd_discard_ehlo_keywords => ['8BITMIME'],
  bypass_banned_checks_maps => [0],  # allow sending any file names and types
  terminate_dsn_on_notify_success => 0,  # don't remove NOTIFY=SUCCESS option
};

$policy_bank{'ORIGINATING_POST'} = { # Post DKIM we need to run SA
  originating => 0,
  # notify administrator of locally originating malware
  virus_admin_maps => ['admin@zmail.mydomain.com'],
  spam_admin_maps  => ['admin@zmail.mydomain.com'],
  warnbadhsender   => 0,
  #bypass_spam_checks_maps   => [1], # don't spam-check internal mail if desired
  bypass_virus_checks_maps => [1], # Don't check AV a second time
  # force MTA conversion to 7-bit (e.g. before DKIM signing)
#  smtpd_discard_ehlo_keywords => ['8BITMIME'],
  bypass_banned_checks_maps => [0],  # allow sending any file names and types
  terminate_dsn_on_notify_success => 0,  # don't remove NOTIFY=SUCCESS option
  archive_quarantine_method => undef, # Don't run archiving a second time
};

Zimbra Forums

Custom SpamAssassin Rules doesn't affect after upgrade to 8.8.12

Custom SpamAssassin Rules doesn't affect after upgrade to 8.8.12

Re: Custom SpamAssassin Rules doesn't affect after upgrade to 8.8.12

Re: Custom SpamAssassin Rules doesn't affect after upgrade to 8.8.12

Re: Custom SpamAssassin Rules doesn't affect after upgrade to 8.8.12

Re: Custom SpamAssassin Rules doesn't affect after upgrade to 8.8.12

Re: Custom SpamAssassin Rules doesn't affect after upgrade to 8.8.12

Re: Custom SpamAssassin Rules doesn't affect after upgrade to 8.8.12

Re: Custom SpamAssassin Rules doesn't affect after upgrade to 8.8.12

Re: Custom SpamAssassin Rules doesn't affect after upgrade to 8.8.12

Re: Custom SpamAssassin Rules doesn't affect after upgrade to 8.8.12