SPF checks not being made?

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
cronos
Posts: 17
Joined: Sat Sep 13, 2014 3:06 am

SPF checks not being made?

Post by cronos »

Hi guys
vanilla installs of 8.0.2. on both CentOS (unsupported) and Ubuntu (supported).
Although there is lots of google history showing that SPF has to be installed/enabled in SpamAssassin, I believe that was for the older versions of Zimbra, and that recent versions have SPF checking enabled by default.
We were first alerted to SPF not working correctly when we received a load of spam from "ourselves" ;-)
Having looked into it further, we *never* see any SPF_FAIL or SPF_PASS tests in the X-Spam headers.
I have spent all day looking into this and can see that SPF is installed:
Feb 20 17:36:22 zimbra-2 amavis[3377]: Module Mail::SPF 2.008

Feb 20 17:36:22 zimbra-2 amavis[3377]: SA dbg: config: read file /opt/zimbra/conf/spamassassin/25_spf.cf

Feb 20 17:36:22 zimbra-2 amavis[3377]: SA dbg: config: read file /opt/zimbra/conf/spamassassin/60_whitelist_spf.cf
Its a vanilla ZCS install with no options changed, so AntiSpam (SpamAssassin) is enabled in the admin UI.
Nothing fancy on the networking side either, its a VM running on a single interface with internet IP. Its not being proxied to or anything. Its MTA trusted networks is default ie itself (via 127.0.0.0/8 and its own IP address).
I don't know why but just cannot see any evidence of SPF checking being made. Ideas ?
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

SPF checks not being made?

Post by phoenix »

It works for me with a standard ZCS install and no modifications:
X-Spam-Status: No, score=-3.541 tagged_above=-10 required=5

tests=[BAYES_00=-1.9, LOTS_OF_MONEY=0.001, RCVD_IN_DNSWL_LOW=-0.7,

RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1,

SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_BIG_HEADERS_2K=0.01,

T_FSL_HAS_TINYURL=0.01, T_FSL_HELO_NON_FQDN_2=0.01,

T_HEADER_FROM_DIFFERENT_DOMAINS=0.01, T_HK_MUCHMONEY=0.01,

T_LONG_HEADER_LINE_80=0.01, T_NOT_A_PERSON=-0.01,

T_TVD_PH_BODY_ACCOUNTS_POST=0.01, T_TVD_PH_BODY_META_ALL=0.01,

T_URL_SHORTENER=0.01] autolearn=ham
Do you have SPF records for your own server & domain?
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
cronos
Posts: 17
Joined: Sat Sep 13, 2014 3:06 am

SPF checks not being made?

Post by cronos »

Hi Bill
fresh install, test email to the default admin@host account created and...
X-Spam-Status: No, score=-2.987 tagged_above=-10 required=6.6

tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,

DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001,

RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01,

RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1, TVD_SPACE_RATIO=0.001,

T_BIG_HEADERS_2K=0.01, T_FSL_HELO_NON_FQDN_2=0.01,

T_LONG_HEADER_LINE_160=0.01, T_LONG_HEADER_LINE_400=0.01,

T_LONG_HEADER_LINE_80=0.01, T_RCD_RDNS_SERVER=-0.01,

T_RCD_RDNS_SERVER_MESSY=-0.01] autolearn=ham
different tests to yours though, why is that ?
SPF records are indeed in place for our own domains, but you mention "server". Do we need to create an SPF record for the server itself ? (its a meaningless infrastructure hostname of zimbra-x.somedomain.net) ?
cronos
Posts: 17
Joined: Sat Sep 13, 2014 3:06 am

SPF checks not being made?

Post by cronos »

So - anyone any ideas on how to even begin debugging this ?
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

SPF checks not being made?

Post by phoenix »

[quote user="cronos"]fresh install, test email to the default admin@host account created and...[/QUOTE]A 'test email' from where, internal or external?
[quote user="cronos"]X-Spam-Status: No, score=-2.987 tagged_above=-10 required=6.6

tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,

DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001,

RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01,

RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1, TVD_SPACE_RATIO=0.001,

T_BIG_HEADERS_2K=0.01, T_FSL_HELO_NON_FQDN_2=0.01,

T_LONG_HEADER_LINE_160=0.01, T_LONG_HEADER_LINE_400=0.01,

T_LONG_HEADER_LINE_80=0.01, T_RCD_RDNS_SERVER=-0.01,

T_RCD_RDNS_SERVER_MESSY=-0.01] autolearn=ham
different tests to yours though, why is that ?[/QUOTE]That would be because they're different emails.
[quote user="cronos"]SPF records are indeed in place for our own domains, but you mention "server". Do we need to create an SPF record for the server itself ? (its a meaningless infrastructure hostname of zimbra-x.somedomain.net) ?[/QUOTE]I meant for the server that's hosting your domain.
Have you actually tried any of the many SPF checking services available on the internet to verify the validity of your SPF records?
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
cronos
Posts: 17
Joined: Sat Sep 13, 2014 3:06 am

SPF checks not being made?

Post by cronos »

The test email was from an external source. As for SPF checking services, these would be for the sender, not our own domain or receiving mailserver.
The bottom line is that it shouldn't matter whether you have SPF in place or not for our domain - the server should be performing SPF checks on incoming email, which it isn't. I have no idea why and putting spamassassin into debug mode shows us nothing either. Most odd.
cronos
Posts: 17
Joined: Sat Sep 13, 2014 3:06 am

SPF checks not being made?

Post by cronos »

Right, just done a vanilla install of ubuntu.
1. installed the package dependencies (netcat, sqlite etc etc)

2. Downloaded zcs-NETWORK-8.0.2_GA_5569.UBUNTU12_64.20121210115144.tgz

3. Run installer, defaults chosen. Set admin password and license file.
at which point the server is then ready to receive emails for the default admin@hostname user thats setup. So I send in an email from an external personal account to here, plus also our exchange box as I know that does SPF checks.
The exchange box (obviously I've changed the details):

Received-SPF: Pass (mx-1.ourdomain.xxx: domain of lee@mydomain designates xx.xx.xx.xx as permitted sender)
The zimbra box

X-Spam-Status: No, score=-2.541 tagged_above=-10 required=6.6

tests=[ALL_TRUSTED=-1, BAYES_00=-1.9, DKIM_SIGNED=0.1,

DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001,

MIME_HTML_MOSTLY=0.428, T_BIG_HEADERS_3K=0.01,

T_LONG_HEADER_LINE_80=0.01, T_UNKNOWN_ORIGIN=0.01]
No SPF results ?
cronos
Posts: 17
Joined: Sat Sep 13, 2014 3:06 am

SPF checks not being made?

Post by cronos »

Finally tracked it down. Its all to do with the MTA trusted networks setting.
Now don't even get me started on the admin UI which is unable to understand a /32, or the IPv6 stuff, making any change in that screen impossible, not like its been outstanding as a bug for all time eh vmware......
anyhow - that screen insists that the servers' local IP be present along with 127.0.0.0/8. Unfortunately in our case the "local IP" is actually a public internet IP as the servers are behind a routed firewall connection. Having the public internet IP in the trusted MTA (trusted_networks I presume) results in SPF checks being bypassed.
So - fixed that by running our old friend zmprov to set the ZimbraMta setting.
Next up - we've noticed that the score assigned to an SPF fail in 50_scores.cf is pitifully low. Bill - I note from a lot of threads you are running with 66/25 as the AS/AV kill/tag percentages, but have you changed any of the spamassassin scoring ?
My business partner is of the opinion that an SPF_FAIL (hard fail) is a delete on sight, I'm a bit more forgiving ;-) but certainly 0.001 seems daft. For now we've set SPF_FAIL to 5.
Comments ?
sadiq007
Advanced member
Advanced member
Posts: 104
Joined: Sat Sep 13, 2014 12:27 am

SPF checks not being made?

Post by sadiq007 »

Bill or anyone else there for reply?
If someone send me mail from external domain then my zimbra sometime checking SPF and sometimes it will not

why ????


Mail with SPF test

===============================================
Return-Path: DannyKenely@tele2.no

Received: from mail.mydomain.com (LHLO mail.mydomain.com)

(192.168.0.200) by mail.mydomain.com with LMTP; Thu, 28 Feb 2013
04:28:04 +0530 (IST)

Received: from localhost (localhost.localdomain [127.0.0.1])

by mail.mydomain.com (Postfix) with ESMTP id 369D21A90023

for ; Thu, 28 Feb 2013 04:28:04 +0530 (IST)

X-Virus-Scanned: amavisd-new at mydomain.com

X-Spam-Flag: NO

X-Spam-Score: 2.818

X-Spam-Level: **

X-Spam-Status: No, score=2.818 tagged_above=-10 required=6.6

tests=[AM:BOOST=-10, BAYES_99=3.5, HTML_MESSAGE=0.001,

RCVD_IN_BRBL_LASTEXT=1.449, RCVD_IN_PBL=3.335, RCVD_IN_PSBL=2.7,

RCVD_IN_XBL=0.375, RDNS_NONE=0.793, SPF_SOFTFAIL=0.665] autolearn=no

Received: from mail.mydomain.com ([127.0.0.1])

by localhost (mail.mydomain.com [127.0.0.1]) (amavisd-new, port 10024)

with ESMTP id 7lB+jlPaP8gF for ;

Thu, 28 Feb 2013 04:28:03 +0530 (IST)

Received: from [200.4.178.243] (unknown [200.4.178.243])

by mail.mydomain.com (Postfix) with ESMTP id 18D3E1A90021

for ; Thu, 28 Feb 2013 04:28:01 +0530 (IST)

Received: from mailout-us.gmx.com ([74.208.5.67]) by mailgw.swip.net;Wed, 27 Feb 2013 07:58:01 -0800

Received: (qmail 9741 invoked by uid 0); Wed, 27 Feb 2013 07:58:01 -0800

Received: from 192.154.146.220 by rms-us059.v300.gmx.net with HTTP

Content-Type: multipart/mixed;boundary="========GMXBoundary837441695531696615208"


Mail without SPF test

====================================
Return-Path: sender@senderdomain.com

Received: from mail.mydomain.com (LHLO mail.mydomain.com)

(192.168.0.200) by mail.mydomain.com with LMTP; Thu, 28 Feb 2013

13:16:00 +0530 (IST)

Received: from localhost (localhost.localdomain [127.0.0.1])

by mail.mydomain.com (Postfix) with ESMTP id AFF271AB005C

for ; Thu, 28 Feb 2013 13:16:00 +0530 (IST)

X-Virus-Scanned: amavisd-new at mydomain.com

X-Spam-Flag: NO

X-Spam-Score: -2.599

X-Spam-Level:

X-Spam-Status: No, score=-2.599 tagged_above=-10 required=6.6

tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7]

autolearn=ham

Received: from mail.mydomain.com ([127.0.0.1])

by localhost (mail.mydomain.com [127.0.0.1]) (amavisd-new, port 10024)

with ESMTP id 5B1kkU9VX3qi for ;

Thu, 28 Feb 2013 13:15:58 +0530 (IST)

Received: from mail-ve0-f169.google.com (mail-ve0-f169.google.com [209.85.128.169])

by mail.mydomain.com (Postfix) with ESMTPS id 7ECA91AB0056

for ; Thu, 28 Feb 2013 13:15:57 +0530 (IST)

Received: by mail-ve0-f169.google.com with SMTP id 15so1480777vea.14

for ; Wed, 27 Feb 2013 23:45:53 -0800 (PST)

X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;

d=google.com; s=20120113;

h=mime-version:x-received:date:message-id:subject:from:to

:content-type:x-gm-message-state;

bh=Lfg2FkwhrMjHzL6RZ/pWN3kdiDGPFsoIQhM+X6SgOPo=;

b=belXd5Txn6OvGP2LhjG9aAW4eWn7+n65kf1NFqRTcQNnEnbMG8bsxRHZO15HmeDUbb

ga8IUALyFvBJk1ebFEbRLu+7AcgjvHXHjNuiJxRaWOBQ1Y9GdkV73/wBrkPNKBJLBX6o

+NII2mRF+aCdGlj42G5ZyeibTri3j8kHesvOWLHX70MKxDMl9iX9UzJpYeox6nLqVSaO

KTe07Pi6DfpXnYzhHRPOtwnQpx3St9gSUAvaqn2pocMrLu1Iaf7ZbIVN5QeDaf8lK1WS

E61gaO5lJ54J8O7r9RSK0DbuKI+rALFKSXS59vHUREEQ4qoFEASyUoOkywBQ6f7DTqLC

tRyA==
cronos
Posts: 17
Joined: Sat Sep 13, 2014 3:06 am

SPF checks not being made?

Post by cronos »

Hi
does the sender have an SPF record setup? Zimbra cannot check what isn't there.
In the case of your second example the email has come from Google and is DKIM signed. I may be wrong, but I think if the email is DKIM signed SPF is bypassed (after all whats the point of making an older, often inaccurate DNS based check when you have one which is far stronger)
Zimbra, or rather SpamAssassin - seems to be quite random in the checks it applies to email. I've seen some blatant spam come in the front door with hardly any checks made and so it ends up in the inbox. 15 years experience of running mailservers with integrated AV/AS tells me to simply turn off AV/AS and use something else. In our case we'll probably use a MailFoundry appliance, ran 4 of these for years and they are pretty damn good for the money (and the only viable option if you're a hoster with lots of mailboxes)
Post Reply