8.7.x spam DMARC_FAIL_REJECT=9 for aol.com and zimbra.com

[saschadd]

Hi,

i noticed that from version 8.7.x the spam check gives very often DMARC_FAIL_REJECT=9

By now i noticed aol.com and zimbra.com mails marked like that.
As the result all emails from these domains are marked as spam which isnt correct.

By searching the forums and the web i havent found a way to correct this but maybe someone knows how to do.
Thanks in advance for any guidances.

sascha

[saschadd]

Am i really the only one facing this problem?

[saschadd]

Okay problem solved.

After switching from

automatic forwarding email from external account to zimbra

to

zimbra collecting emails from external account via pop3

the problem is gone.
Therefore this must have been an forwarding issue which i dont understand completely.

I found out that the external account forwards email from an central emailserver for example mail.isp.net and not via mail.mydomain.com.
But what causes the DMARC_FAIL_REJECT=9 then?
Maybe someone can explain?

[JDunphy]

I now have this problem also. It doesn't seem reasonable to ask AOL users not to forward email.

This meta rule is the cause of the problem and unique to Zimbra I believe. From salocal.cf:

Code: Select all

ifplugin Mail::SpamAssassin::Plugin::AskDNS

  askdns   __DMARC_POLICY_REJECT _dmarc._AUTHORDOMAIN_ TXT /^v\s*=DMARC1 (?=\s*;) .* ;\s* p\s*=\s*reject     \s*(?:;|\z)/x

  meta     DMARC_FAIL_REJECT !(DKIM_VALID_AU || SPF_PASS) && __DMARC_POLICY_REJECT
  describe DMARC_FAIL_REJECT DMARC validation failed and policy is to reject
  score    DMARC_FAIL_REJECT 9.0

Some headers:

Code: Select all

X-Spam-Status: Yes, score=12.782 required=4.8 tests=[BAYES_50=0.8,
	DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DMARC_FAIL_REJECT=9,
	DOMAIN_SPAM_TLD=2.5, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001,
	HTTP_IN_BODY=0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01,
	RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, RP_MATCHES_RCVD=-0.1]
	autolearn=no autolearn_force=no
Authentication-Results: mail.example.com (amavisd-new);
	dkim=pass (1024-bit key) header.d=mx.aol.com
....

Received: from omr-a019e.mx.aol.com (omr-a019e.mx.aol.com [204.29.186.67])
	by mail.example.com (8.14.4/8.14.6) with ESMTP id v4VE8VhO016894
	for <user@example.com>; Wed, 31 May 2017 07:08:33 -0700
Received: from mtaomg-mcc01.mx.aol.com (mtaomg-mcc01.mx.aol.com [172.26.253.85])
	by omr-a019e.mx.aol.com (Outbound Mail Relay) with ESMTP id C460E38001A1;
	Wed, 31 May 2017 10:08:30 -0400 (EDT)
Received: from core-mad01g.mail.aol.com (core-mad01.mail.aol.com [172.27.61.11])
	by mtaomg-mcc01.mx.aol.com (OMAG/Core Interface) with ESMTP id 10D7838000087;
	Wed, 31 May 2017 10:08:30 -0400 (EDT)

....
X-Mailer: AOL 9.8 sub 2019
X-Originating-IP: [12.207.165.194]
x-aol-global-disposition: G
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mx.aol.com;
	s=20150623; t=1496239710;
	bh=DgMwjmie0Nwcou8hSxuUTizNH5nfeCA59xLzVr3ieD8=;
	h=From:To:Subject:Message-ID:Date:MIME-Version:Content-Type;
	b=UNeCTyr5FeOXrtoxCX560gUaRNLrYoAaEoprnj+4B4/vOlXdUTg0QDh+oUBDOH3g1
	 txw+Brj81YB55XP0IRX3iDmMf48+CdD6aMXy7eVBeeAzfUAGlOPI1D+TWdWsTTreaG
	 yqME9onFZWEspDRxsEEquOyjzb4SMllESdROEmt8=
x-aol-sid: 3039ac1afd55592ece5e479b

My case is a little odd because I am forwarding from an inbound mail relay (sendmail) but do have trusted_networks defined for spamassasin so it knows the ip of those servers. I had tested this and it seemed to work vs not having those ip addresses listed for SPF.

From what I could see in my headers, DKIM_VALID_AU and SPF_PASS were not set for this valid aol message. I am thinking of && __FROM_AOL_COM to that DMARC_FAIL_REJECT rule if I can't understand why.

Maybe it just because forwarding breaks with DMARC but that is a false positive. What are others doing for aol users? I think it fails the SPF test because AOL dosn't list all their outbound addresses and spamassasin doesn't really trust received headers and reads all of them from what I can tell. AOL might list them all but it's really convoluted and I lost track have going down some include paths when manually looking up the SPF addresses.

I am interested what others think is the correct option to do. While we are on spamassasin, it would have been great if zimbra did the rule updates themselves since the spamaassasin server is currently down and to be replaced by new hardware and yahoo email are being flagged with false positives for about 3 weeks unless you do this because they recently changed one of their headers.

# TO REMOVE (and also see 20_meta_tests.cf)
header __MOZILLA_MUA User-Agent =~ /^mozilla\b/i

It's been a bad few weeks for false positives with spamassasin rules.

[L. Mark Stone]

Automatic forwarding of all inbound email to another account can be leveraged by spammers. Indeed, we have seen where Gmail rejects outright an onslaught of forwarded email from a specific account.

Much better to have Zimbra, Gmail, AOL etc. collect email from other accounts, rather than have other accounts forward email.

Hope that helps,
Mark

[JDunphy]

Thanks Mark,

It wasn't done like that but an unsophisticated aol user attempting to forward an email to a zimbra inbox to share some information. I am not sure the AOL users have the ability to 'edit as new' like zimbra users do. Its a sticky problem because AOL is the one asking for the reject in their policy. So we are left trying to fix the problem since in a paid email corporate solution the users generally want all their email and no spam. Yea how hard can that be. LOL

Given that and I realize this is a sticky problem, I'm leaning toward and'ing aol.com to the rule (ie. && __FROM_AOL_COM) for this since that is a rather large group of potential false positives. Hopefully, the other rules in aggregate would catch real junk as spam if that rule fails to fire for DMARC and aol. I know this particular zimbra customer has a filter rule for all their employees that says if the sender isn't in their contacts and the email is html or an attachment to mark it as spam so hopefully it won't be too bad for them. In any event, I know they will tell me.

That I haven't see this before on my own zimbra server tells you how many aol users send me email... or should I say forward email to me. <grin>

Jim

[L. Mark Stone]

Thanks Mark,

It wasn't done like that but an unsophisticated aol user attempting to forward an email to a zimbra inbox to share some information. I am not sure the AOL users have the ability to 'edit as new' like zimbra users do. Its a sticky problem because AOL is the one asking for the reject in their policy. So we are left trying to fix the problem since in a paid email corporate solution the users generally want all their email and no spam. Yea how hard can that be. LOL

Given that and I realize this is a sticky problem, I'm leaning toward and'ing aol.com to the rule (ie. && __FROM_AOL_COM) for this since that is a rather large group of potential false positives. Hopefully, the other rules in aggregate would catch real junk as spam if that rule fails to fire for DMARC and aol. I know this particular zimbra customer has a filter rule for all their employees that says if the sender isn't in their contacts and the email is html or an attachment to mark it as spam so hopefully it won't be too bad for them. In any event, I know they will tell me.

That I haven't see this before on my own zimbra server tells you how many aol users send me email... or should I say forward email to me. <grin>

Jim

FWIW we advise our clients that most people strongly prefer a few spams over false positives (legitimate email incorrectly identified as spam). We've seen a lot of false positives with SPF and DKIM, nothing so far from DMARC fortunately.

About two years ago we put a Barracuda in front of our Zimbra systems and couldn't be happier. Now, when a sender is blocked by the Barracuda, we get to say to the client that "yes, that's a problem on the sender's side, and oh by the way, we won't be the only ones blocking their email." It helps the client to understand the responsibility in most cases is on the sender to make sure their email is configured correctly.

All the best,
Mark

[JDunphy]

As a followup of what I ended up doing.

I went with a different solution for this rule instead of mitigating the problem for just AOL as I had initially planned.
After more thought, I am staying with the approach that I have responsibility for email once the sending MTA hands off the email... no matter where that is in our mail delivery network. Users establish trust in their mail system earned over time and usage/experience ... but any reproducible false positives and/or non delivery chip away at that trust even if its not our fault. Mark makes a good point that users tolerate spam in their inbox more than they tolerate false positives.

So if that is the current prime directive, I don't see how I can enforce DMARC reject policy on our end if a simple issue like forwarding an email is guaranteed to be a false positive with some email systems like aol because they requested a reject policy. I like DMARC but things sometimes break and training users that it is the senders responsibility is a challenge especially since they use forward themselves and don't get enough feedback that it failed or went to junk.

I have reduced the score for that meta rule because I believe in the aggregate of the system that it can discern from additional factors what is junk then simply trust the reject policy. This will at least reduce false positives because someone needed to share an email they had by forwarding it to a zimbra mailbox. It could also increase junk but that isn't what we are seeing after 24 hours. I have the rule scored at 1 and it can always go up but when you are at 9, it's going to be false positive every time. I can imagine the internal debates going around Zimbra when they chose to force DMARC reject policy to junk. Not an easy answer what is correct for every environment.

I might be revisiting this but that is what is happening now.

[L. Mark Stone]

Sounds like a pragmatic plan!

It's unfortunate that many of the spam solutions to be effective require pretty much all of the legitimate email players to conform.

Several times now we've blocked senders for hard fails on their SPF record. The senders used a hyphen instead of a tilde to prepend "all" at the end of the SPF record, then changed email providers but didn't update their SPF record.

Our clients were mad at us first of course, but once we explained that their their critical supplier/vendor dropped the ball with the sender's mission-critical email, our clients "got it." Still, after that happened for the Nth time, we changed the settings in the Barracuda for "SPF Hard Fail Action" from Quarantine to Tag. At least that way, our clients get the email right away (which is indeed sometimes spoofed and true spam) but with a big warning tag prepended to the Subject.

Some things, like Email, are just (and will always be) messy...

All the best,
Mark

P.S. During the Cold War, the USA employed scores of trucks and airmen to keep runways clean. Recall the F-16 has its engine intake scoop on the very bottom of the fuselage, right above the runway. Russian fighters like the MiG-25 Foxbat fitted big doors to their engine intakes that during takeoff and landing could block the forward entrance in favor of an opening on the top of the fuselage, where less FOD (Foreign Objects and Debris) were likely to enter the engines. We use the old Russian approach to email spam warfare!

[DavidMerrill]

I'm running into this too (with one of our clients, they're running 8.7.11.GA.1854.UBUNTU16.64 NE).

I've been trolling the forum & wiki (for details on the two files I mention below) and trying to parse what I'm seeing (while filtering out documentation for older versions of Zimbra). From what I can see here:

• https://wiki.zimbra.com/wiki/Improving_Anti-spam_system

One makes edits here:

• /opt/zimbra/conf/salocal.cf.in

and then after a Zimbra services restart THIS file gets regenerated:

• /opt/zimbra/data/spamassassin/localrules/salocal.cf

is that correct (the regeneration of salocal.cf)?