Block SPAM email from new domains?
Block SPAM email from new domains?
A recent problem seems to be caused by domains maliciously registered very recently, looking very similar to real online banking domains, often registered the the same day as the received SPAM, with all the correct DKIM keys etc (often at Godaddy), then used to SPAM/Phish. Issues are:
By default Zimbra (8.8.9) seems to auto white-list mail that passes a DKIM test so no other spamassassin rules are applied (where is this - can I remove it easily ?)
Is it possible to create a rule in spamassassin or postfix that gets the registered date from DNS and rejects anything newer than say 3 days?
Usually these guys get shutdown the same day, but not until a flood of spam has gone out.
Thanks
By default Zimbra (8.8.9) seems to auto white-list mail that passes a DKIM test so no other spamassassin rules are applied (where is this - can I remove it easily ?)
Is it possible to create a rule in spamassassin or postfix that gets the registered date from DNS and rejects anything newer than say 3 days?
Usually these guys get shutdown the same day, but not until a flood of spam has gone out.
Thanks
- DualBoot
- Elite member
- Posts: 1326
- Joined: Mon Apr 18, 2016 8:18 pm
- Location: France - Earth
- ZCS/ZD Version: ZCS FLOSS - 8.8.15 Mutli servers
- Contact:
Re: Block SPAM email from new domains?
Hello,
Regards,
What are the elements which make you thinking that ?By default Zimbra (8.8.9) seems to auto white-list mail that passes a DKIM test so no other spamassassin rules are applied (where is this - can I remove it easily ?)
Regards,
Re: Block SPAM email from new domains?
Obviously not whitelisted by me - domain created yesterday, yet in the headers:
X-Spam-Status: No, score=x required=9.4 WHITELISTED tests=[]
autolearn=unavailable
and DKIM sig:
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=key; d=lloydsbankonline.uk;
h=Mime-Version:Date:To:Subject:From:Content-Type:Message-ID;
bh=g8CBLIxIV95qev5ISXClU7XCK54=;
b=SIMR5cxJejaySutGSdbnu2bDvaQhT9IyBxqrzfri4XHMATWnK9UoT95OGWBaxRkOTPRae9g7kegW
BkOh7H8c1MwGfH/ubQxMZXIS6xzIcy/32Fb+Kb6FINQOaqXzOidD7lw54j+n+aOLpvy5CKzkBNz6
fvM9ea6gdMC9klmS6p4=
dkim=pass (1024-bit key) header.d=lloydsbankonline.uk
Domain was suspended by godaddy before I read the email in this case...
X-Spam-Status: No, score=x required=9.4 WHITELISTED tests=[]
autolearn=unavailable
and DKIM sig:
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=key; d=lloydsbankonline.uk;
h=Mime-Version:Date:To:Subject:From:Content-Type:Message-ID;
bh=g8CBLIxIV95qev5ISXClU7XCK54=;
b=SIMR5cxJejaySutGSdbnu2bDvaQhT9IyBxqrzfri4XHMATWnK9UoT95OGWBaxRkOTPRae9g7kegW
BkOh7H8c1MwGfH/ubQxMZXIS6xzIcy/32Fb+Kb6FINQOaqXzOidD7lw54j+n+aOLpvy5CKzkBNz6
fvM9ea6gdMC9klmS6p4=
dkim=pass (1024-bit key) header.d=lloydsbankonline.uk
Domain was suspended by godaddy before I read the email in this case...
Re: Block SPAM email from new domains?
Another example - danskebankcom.uk - registered with Godaddy.com today, spammed a lot, then suspended today
-
- Advanced member
- Posts: 51
- Joined: Fri Mar 16, 2018 5:25 pm
- Location: USA
- ZCS/ZD Version: 8.8.12.GA.3794.UBUNTU18.64 FOSS
- Contact:
Re: Block SPAM email from new domains?
I was intrigued by your inquiry here. I feel like my hands are tied when it comes to AS/AV.
I believe the answer to extend beyond SA regex rules is to create a custom plugin.
I'm an experienced developer but I know nothing about SpamAssassin rules. However with a little searching I found some hits that could be of use to both/all of us.
I can't actually do anything with this now, so I'm posting here for this inquiry, for my own reference later, and for posterity.
https://wiki.zimbra.com/wiki/Improving_Anti-spam_system
https://wiki.zimbra.com/wiki/Anti-spam_Strategies
Basics: https://wiki.apache.org/spamassassin/WritingRules
https://wiki.apache.org/spamassassin/UsingDcc
https://wiki.apache.org/spamassassin/HashSharingSystem
https://wiki.apache.org/spamassassin/UsingNetworkTests
https://stackoverflow.com/questions/261 ... ll-command
Solution!! >> Use Perl : https://wiki.apache.org/spamassassin/CustomPlugins << lots of examples
https://spamassassin.apache.org/full/3. ... Plugin.txt
Another great/complete example of a plugin: https://metacpan.org/pod/Mail::SpamAssa ... entPresent
Tutorial, might not be too helpful: https://www.perlmonks.org/?node_id=133023
Example .cf file: https://www.pccc.com/downloads/SpamAssa ... rib/KAM.cf
Note that with a Perl/X language bridge it should be possible to write rules in JavaScript, Java, and other languages. While not very performant, async processing and allowance for delivery delays make that a non-issue.
HTH
I believe the answer to extend beyond SA regex rules is to create a custom plugin.
I'm an experienced developer but I know nothing about SpamAssassin rules. However with a little searching I found some hits that could be of use to both/all of us.
I can't actually do anything with this now, so I'm posting here for this inquiry, for my own reference later, and for posterity.
https://wiki.zimbra.com/wiki/Improving_Anti-spam_system
https://wiki.zimbra.com/wiki/Anti-spam_Strategies
Basics: https://wiki.apache.org/spamassassin/WritingRules
https://wiki.apache.org/spamassassin/UsingDcc
https://wiki.apache.org/spamassassin/HashSharingSystem
https://wiki.apache.org/spamassassin/UsingNetworkTests
https://stackoverflow.com/questions/261 ... ll-command
Solution!! >> Use Perl : https://wiki.apache.org/spamassassin/CustomPlugins << lots of examples
https://spamassassin.apache.org/full/3. ... Plugin.txt
Another great/complete example of a plugin: https://metacpan.org/pod/Mail::SpamAssa ... entPresent
Tutorial, might not be too helpful: https://www.perlmonks.org/?node_id=133023
Example .cf file: https://www.pccc.com/downloads/SpamAssa ... rib/KAM.cf
Note that with a Perl/X language bridge it should be possible to write rules in JavaScript, Java, and other languages. While not very performant, async processing and allowance for delivery delays make that a non-issue.
HTH
- JDunphy
- Outstanding Member
- Posts: 901
- Joined: Fri Sep 12, 2014 11:18 pm
- Location: Victoria, BC
- ZCS/ZD Version: 9.0.0_P39 NETWORK Edition
Re: Block SPAM email from new domains?
Another option is to created your own black list as SA has plugins ready to use.
Add this to your local rules (ie. sauser.cf)
Then update your dnsbl.example.com zone whenever you need to blacklist someone... BIND syntax here:
Just update your DNS and reload. Your changes will be reflected instantly for zimbra without touching or refreshing any rules on your zimbra hosts. You can test your configuration by:
If an entry returns 127.0.0.1 then your rule will score them 5 points and send it to spam.
Given that most of the DNS providers now have API's, one could add these domains dynamically or build your own with nsupdate for BIND.
Note: there are variations to this built-in rule above... check_rbl, check_envfrom, check_txt, etc, etc. see: /opt/zimbra/common/lib/perl5/Mail/SpamAssassin/Plugin/DNSEval.pm and for usage of these ... see: /opt/zimbra/data/spamassassin/state/3.004001/updates_spamassassin_org and grep for rule usage from other rules before adding them to your salocal.cf
Now the real problem... bulk discovery of new domains is becoming really hard. In the old days, we would parse whois output and see when it was created. It would appear most registrar's sell this data so it has become valuable and really difficult to do this in a bulk and automated fashion that I know of. The few lists that were targeting this missed many of these new domains when I checked against some spam that we saw. You might be better off using something like invaluement's URI bl and score it or target some of the spam senders and see what domains they have registered.
Add this to your local rules (ie. sauser.cf)
Code: Select all
# conference attendee leads spammers
header J_FRM_IN_BL eval:check_rbl_from_domain('example, 'dnsbl.example.com', '127.0.0.1')
describe J_FRM_IN_BL listed in dnsbl.example.com
tflags J_FRM_IN_BL net
score J_FRM_IN_BL 5
Code: Select all
;
; $id$
;
; 127.0.0.2 --- open relays
; 127.0.0.3 --- dial-up/dynamic IP ranges
; 127.0.0.4 --- Spam Sources
; 127.0.0.5 --- multi-stage open relays
; 127.0.0.8 --- insecure or similar CGI scripts that become open relays
; 127.0.0.9 --- open proxy servers
; 127.0.0.25 --- bad helo header
; 127.0.0.26 --- troll addresses
$TTL 8H; Min TTL
@ IN SOA relay2.example.com. abuse.example.com. (
2002042337 ; serial
10800 ; Refresh every 2 days
3600 ; Retry every hour
604800 ; Expire every XX days
600 ) ; Minimum XX days
@ IN NS NS2.example.com.
@ IN NS ns3.example.com.
; Lead Generator domain names. Known to spam w/ attendee lists, hotels, etc.
; LAST UPDATED: 11/13/2018
;$ORIGIN .dnsbl.example.com.
affiniquedata.com IN A 127.0.0.1
agilityb2binfo.com IN A 127.0.0.1
...
Code: Select all
% dig +short affiniquedata.com.dnsbl.example.com
127.0.0.1
Given that most of the DNS providers now have API's, one could add these domains dynamically or build your own with nsupdate for BIND.
Note: there are variations to this built-in rule above... check_rbl, check_envfrom, check_txt, etc, etc. see: /opt/zimbra/common/lib/perl5/Mail/SpamAssassin/Plugin/DNSEval.pm and for usage of these ... see: /opt/zimbra/data/spamassassin/state/3.004001/updates_spamassassin_org and grep for rule usage from other rules before adding them to your salocal.cf
Now the real problem... bulk discovery of new domains is becoming really hard. In the old days, we would parse whois output and see when it was created. It would appear most registrar's sell this data so it has become valuable and really difficult to do this in a bulk and automated fashion that I know of. The few lists that were targeting this missed many of these new domains when I checked against some spam that we saw. You might be better off using something like invaluement's URI bl and score it or target some of the spam senders and see what domains they have registered.
Re: Block SPAM email from new domains?
Thanks for all the feedback.
When I run a 'whois' on my own domain, I can immediately see "Creation Date: ...." in standard ISO 8601 format.
So I should be able to check that in an SA rule ...
I was hoping someone might have written the code in the dim-distant-past ?
When I run a 'whois' on my own domain, I can immediately see "Creation Date: ...." in standard ISO 8601 format.
So I should be able to check that in an SA rule ...
I was hoping someone might have written the code in the dim-distant-past ?
- JDunphy
- Outstanding Member
- Posts: 901
- Joined: Fri Sep 12, 2014 11:18 pm
- Location: Victoria, BC
- ZCS/ZD Version: 9.0.0_P39 NETWORK Edition
Re: Block SPAM email from new domains?
I am not a fan of this method for lots of reasons mostly due to the non deterministic high latency costs associated with rwhois real-time lookups. SA has had a few plugins over the years... I think 10 years ago we had URIwhois??? and more recently rules like: https://wiki.apache.org/spamassassin/Rules/URIBL_REDkeynet wrote: When I run a 'whois' on my own domain, I can immediately see "Creation Date: ...." in standard ISO 8601 format.
So I should be able to check that in an SA rule ...
If you want a quick solution that you can put it into your local sauser.cf and try... perhaps this:
https://spameatingmonkey.com/services
Code: Select all
# SEM-FRESH
urirhssub SEM_FRESH fresh.spameatingmonkey.net. A 2
body SEM_FRESH eval:check_uridnsbl('SEM_FRESH')
describe SEM_FRESH Contains a domain registered less than 5 days ago
tflags SEM_FRESH net
score SEM_FRESH 0.5
Code: Select all
SEM-FRESH — Domains registered in the last 5 days
SEM-FRESH10 — Domains registered in the last 10 days
SEM-FRESH15 — Domains registered in the last 15 days
SEM-FRESH30 — Domains registered in the last 30 days
SEM-NETBLACK — Networks identified as having a low reputation
...
Re: Block SPAM email from new domains?
Thanks.
One does at least know a DNS record exists if SPAM comes from that domain and passes DKIM. The example I gave lloydsbankonline.uk was registered (and suspended - the record is still there) on 14-Nov-2018, but doesn't appear in SEM-FRESH15 or SEM-FRESH30, or SEM-URIRED (and also no history) which is disappointing, though perhaps filtered because the domain is suspended, I don't know.
Perhaps a hybrid approach, if DKIM passes (which filters the majority of SPAM), then the more expensive check of registrar records?
One does at least know a DNS record exists if SPAM comes from that domain and passes DKIM. The example I gave lloydsbankonline.uk was registered (and suspended - the record is still there) on 14-Nov-2018, but doesn't appear in SEM-FRESH15 or SEM-FRESH30, or SEM-URIRED (and also no history) which is disappointing, though perhaps filtered because the domain is suspended, I don't know.
Perhaps a hybrid approach, if DKIM passes (which filters the majority of SPAM), then the more expensive check of registrar records?
- JDunphy
- Outstanding Member
- Posts: 901
- Joined: Fri Sep 12, 2014 11:18 pm
- Location: Victoria, BC
- ZCS/ZD Version: 9.0.0_P39 NETWORK Edition
Re: Block SPAM email from new domains?
That is a great use of meta rules...keynet wrote:Thanks.
One does at least know a DNS record exists if SPAM comes from that domain and passes DKIM. The example I gave lloydsbankonline.uk was registered (and suspended - the record is still there) on 14-Nov-2018, but doesn't appear in SEM-FRESH15 or SEM-FRESH30, or SEM-URIRED (and also no history) which is disappointing, though perhaps filtered because the domain is suspended, I don't know.
Perhaps a hybrid approach, if DKIM passes (which filters the majority of SPAM), then the more expensive check of registrar records?
Code: Select all
META KEY_FRESH (!DKIM_VALID_AU && SEM_FRESH)
score KEY_FRESH 3
describe KEY_FRESH new domain in last 5 days and not DKIM valid author
https://spamassassin.apache.org/full/3. ... _DKIM.html ... You can also OR the rules ... so this would be !SIGNED or !VALID in addition to being fresh in the last 5 days.
Code: Select all
META KEY_FRESH ((!DKIM_VALID_AU || !DKIM_SIGNED) && SEM_FRESH)
...
Adjust scoring as I am just showing examples of usage. Total score over 5 then it goes to junk and over 15 and not delivered to user junk folder are the standard defaults with Zimbra.
Note: I use a common prefix for any custom rules so using KEY_ given your username above... Mine start with J_ This allows one to observe the X-Spam-Status in every email and know which rules were yours and how they contributed to the score. Check out spamassassin -D so you can cut/paste mail and verify your rules before having zimbra test them for you in production. For example, if you paste a spam message into /tmp/b1.txt:
Code: Select all
su - zimbra
% spamassassin -D < /tmp/b1.txt > /dev/null 2> /tmp/3.err
I am interested to see what new rules you create.
Jim