I'm getting a lot of spam for a multitude of sources, all with the .icu extension on the domain name, eg werstupid.icu, werspammers.icu, etc.
Is there a way to blacklist an entire extension like *.icu?
blacklist a domain extension
Re: blacklist a domain extension
FYI I'm on v 8.7.6GA_1776.
I tried
However this doesn't work. I get:
I tried
Code: Select all
cd ~/conf
echo 'icu REJECT' >postfix_reject_sender
postmap postfix_reject_sender
zmprov ms `zmhostname` +zimbraMtaSmtpdSenderRestrictions "check_sender_access lmdb:/opt/zimbra/conf/postfix_reject_sender"
zmmtactl restartCode: Select all
zimbra@Mail1:~/common/conf$ zmprov ms 'mail1.ourdomain.com' +zimbraMtaSmtpdSenderRestrictions "check_sender_access lmdb:/opt/zimbra/conf/postfix_reject_sender"
ERROR: account.INVALID_ATTR_VALUE (invalid attr value: invalid attr value - unable to modify attributes: ldap host=mail1.ourdomain.com:389: attribute 'zimbraMtaSmtpdSenderRestrictions' cannot have multiple values)
Re: blacklist a domain extension
Success! I found a solution by using amavis.
In the past I've been blacklisting a specific domain using this command:
Thought about it and I tried this:
I've been monitoring the zimbra.log and I can see where emails coming in from various icu domains and all are getting log entries with "Sender address triggers FILTER" and they're not being delivered.
BTW - you can see what filters have been added to Amavis with the command:
In the past I've been blacklisting a specific domain using this command:
Code: Select all
zmprov md ourdomain.com +amavisBlacklistSender badspammer.orgCode: Select all
zmprov md ourdomain.com +amavisBlacklistSender icuBTW - you can see what filters have been added to Amavis with the command:
Code: Select all
zmprov gd ourdomain.com-
JDunphy
- Outstanding Member
- Posts: 901
- Joined: Fri Sep 12, 2014 11:18 pm
- Location: Victoria, BC
- ZCS/ZD Version: 9.0.0_P39 NETWORK Edition
Re: blacklist a domain extension
Fantastic that you show how amavisd would do it... here is yet another way with SA which is part of amavisd and would bring the full SA rules and spam engine into play. If the total spam score is over 15, it isn't delivered and behaves the same way as having amavis blacklist it.
So if you want more granularity and trust your scoring engine to add more points and want to eliminate some false positives, you might to do something like this with SA in /opt/zimbra/data/spamassassin/localrules/sauser.cf
If you don't want it delivered, score it 15 vs 2.5 as shown above or score it past 5 to put it in your users junk folder but not 15 ... The above rule looks at both the Reply-To and the Received lines... Maybe add 'Return-Path' to that 'OR' clause if you want the SMTP envelope from also checked.
So now you have 3 methods... postfix could do it and block at the incoming delivery at various places including the helo/mail to part of the smtp protocol but you could have false positives and would never give your spam training system a shot to evaluate it. Amavisd can do it by not delivering it to the user but you have already accepted it in contrast to the postfix method ... and finally SA can offer you finer granularity to put it in the users junk folder, not deliver it, or accept it for delivery to the inbox by adjusting the score. The SA method also allows you to desk test your rules from the command line with real spam before putting it into production.
Jim
So if you want more granularity and trust your scoring engine to add more points and want to eliminate some false positives, you might to do something like this with SA in /opt/zimbra/data/spamassassin/localrules/sauser.cf
Code: Select all
header __J_DOMAIN_RCVD_TLD Received =~ /\.(online|click|science|it|ru|space|rocks|xyz|me|ec|eu|links|id|in|work|ninja|asia|mx|racing|faith|br|top|email|date|trade|bid|stream|club|loan|win|review|press|fun|mk|icu|gallery|host|cf|lt|it|my|io)\s+/si
header __J_DOMAIN_RPLTO_TLD Reply-To =~ /\.(online|click|science|it|ru|space|rocks|xyz|me|ec|eu|links|id|in|work|ninja|asia|mx|racing|faith|br|top|email|date|trade|bid|stream|club|loan|win|review|press|fun|mk|icu|gallery|host|cf|lt|it)(?:\s|>)+/si
meta J_DOMAIN_SPAM_TLD (__J_DOMAIN_RCVD_TLD || __J_DOMAIN_RPLTO_TLD)
score J_DOMAIN_SPAM_TLD 2.5
describe J_DOMAIN_SPAM_TLD Prevalent use of .info|.links|.rocks in spam/malware
So now you have 3 methods... postfix could do it and block at the incoming delivery at various places including the helo/mail to part of the smtp protocol but you could have false positives and would never give your spam training system a shot to evaluate it. Amavisd can do it by not delivering it to the user but you have already accepted it in contrast to the postfix method ... and finally SA can offer you finer granularity to put it in the users junk folder, not deliver it, or accept it for delivery to the inbox by adjusting the score. The SA method also allows you to desk test your rules from the command line with real spam before putting it into production.
Jim
Re: blacklist a domain extension
Is sauser.cf a manually created file? On my system (Ubuntu 16.04.6 LTS, Zimbra 8.7.6) that file isn't there.
-
JDunphy
- Outstanding Member
- Posts: 901
- Joined: Fri Sep 12, 2014 11:18 pm
- Location: Victoria, BC
- ZCS/ZD Version: 9.0.0_P39 NETWORK Edition
Re: blacklist a domain extension
It was named something else in the past and I think the move from 8.0 to 8.6 it was renamed to sauser.cf ... It is where all your local rules go. I think it use to be /opt/zimbra/conf/sa/salocal.cf and /opt/zimbra/conf/sa/sauser.cf in the past... long long time ago. During one of the upgrades, it went to /opt/zimbra/data/spamassassin/localrules/sauser.cf for us. I run 8.7.11+ now.
Manually create it and that should work if it is missing. You can verify it is being called by invoking SA from the command line. There should be some wiki's links discussing how this all fits together and how to test with SA which I include in the link below.
Ref: https://wiki.zimbra.com/wiki/JDunphy-SA-RuleWriting
Manually create it and that should work if it is missing. You can verify it is being called by invoking SA from the command line. There should be some wiki's links discussing how this all fits together and how to test with SA which I include in the link below.
Ref: https://wiki.zimbra.com/wiki/JDunphy-SA-RuleWriting
Re: blacklist a domain extension
I do have local.cf and salocal.cf in /opt/zimbra/data/spamassassin/localrules so it looks like I have some leftovers/etc from a prior version. salocal.cf looks to be the correct place on my system as it is being updated regularly. I also mange a 8.8.15 system and it looks the same with salocal.cf being updated regularly.
I think your advice about creating it is right. Since the systems are updating salocal.cf that probably shouldn't be touched and spamassasin is using salocal.cf and sauser.cf (local filters). For now I'm going to stay with the amavisd solution but I'll keep the spamassassin solution in mind. I'm not seeing much spam that could be blocked at the TLD other than icu but if it starts to increase the spamassasin you posted will be quicker to implement for multiple TLDs.
I am a little surprised Zimbra doesn't have a GUI interface that allows one to set this up. The ability to blacklist/whitelist seems to me to be a core feature for a mail server. The ability to block by region would also be handy. Regardless, I'm glad I was able to do this even if it was from the command line.
I think your advice about creating it is right. Since the systems are updating salocal.cf that probably shouldn't be touched and spamassasin is using salocal.cf and sauser.cf (local filters). For now I'm going to stay with the amavisd solution but I'll keep the spamassassin solution in mind. I'm not seeing much spam that could be blocked at the TLD other than icu but if it starts to increase the spamassasin you posted will be quicker to implement for multiple TLDs.
I am a little surprised Zimbra doesn't have a GUI interface that allows one to set this up. The ability to blacklist/whitelist seems to me to be a core feature for a mail server. The ability to block by region would also be handy. Regardless, I'm glad I was able to do this even if it was from the command line.
-
tonyg
- Advanced member
- Posts: 51
- Joined: Fri Mar 16, 2018 5:25 pm
- Location: USA
- ZCS/ZD Version: 8.8.12.GA.3794.UBUNTU18.64 FOSS
- Contact:
Re: blacklist a domain extension
I STRONGLY request that the list of TLDs suggested by @JDunphy be reconsidered. For the last few years I've been encouraging site owners to venture away from the classic TLDs, .com and .net, and to consider new domains from within the hundreds of TLDs now available.
I personally run sites with email under .rocks, .club, and .online. I just retired a couple .work sites and will probably have a couple .ninja sites soon. My primary email server is a .rocks domain. These and some of the others in the sample (?) list provided by @JDunphy are completely legitimate and now in common use. Tarnishing entire TLDs is not a good approach to the problems we are trying to solve.
Some are obviously subject to scrutiny, like .click. I didn't think about .links, but seeing it here, if I had one I'd actually consider retiring it because it really does look spammy. Others like .fun were created for a specific purpose which we shouldn't take it upon ourselves to judge or condemn. Of course in a business context, filtering .fun seems appropriate. But knowing that people are filtering my .rocks email server causes me great concern as that is nothing but a name serving several domains including .com. If nothing else, the .email TLD was created in part for those of US (oh yeah, I have .me and .us domains too) who host email servers and those of you who provide related services. Don't block your own industry or support channels!
I do agree with regional blocking, and personally I'm working toward blocking inbound connections at the network level with IPTables, soon NFTAbles. Some would say, in a global economy, that's just as unfair as non-regional TLD blocking. My response is that for some regional TLDs I have only had, literally, less than 10 legitimate contacts over the last 20 years. I can justify regional blocking with a 99.999% chance of accuracy. That high level of certainty does NOT apply to the TLDs which have emerged over the last few years, and we will continue to see more over the coming years.
As a pharmaceutical company needs to be judicious about not blocking emails that contain keywords for their industry, we all need to personally evaluate vectors of abuse to our own servers. In this specific case, I urge you to be more judicious about blocking out top-level "anything", whether top-level domains or IP blocks. We have wiggle room with spam scores. I understand the score of 2.5. Well, try 1.5, certainly not 15. Greylist unknown servers matching the TLD criteria. Check the SPF and DKIM. Check the RBLs. Allow legitimate TLD owners like me an opportunity to allay your concerns.
Thanks for your indulgence.
I personally run sites with email under .rocks, .club, and .online. I just retired a couple .work sites and will probably have a couple .ninja sites soon. My primary email server is a .rocks domain. These and some of the others in the sample (?) list provided by @JDunphy are completely legitimate and now in common use. Tarnishing entire TLDs is not a good approach to the problems we are trying to solve.
Some are obviously subject to scrutiny, like .click. I didn't think about .links, but seeing it here, if I had one I'd actually consider retiring it because it really does look spammy. Others like .fun were created for a specific purpose which we shouldn't take it upon ourselves to judge or condemn. Of course in a business context, filtering .fun seems appropriate. But knowing that people are filtering my .rocks email server causes me great concern as that is nothing but a name serving several domains including .com. If nothing else, the .email TLD was created in part for those of US (oh yeah, I have .me and .us domains too) who host email servers and those of you who provide related services. Don't block your own industry or support channels!
I do agree with regional blocking, and personally I'm working toward blocking inbound connections at the network level with IPTables, soon NFTAbles. Some would say, in a global economy, that's just as unfair as non-regional TLD blocking. My response is that for some regional TLDs I have only had, literally, less than 10 legitimate contacts over the last 20 years. I can justify regional blocking with a 99.999% chance of accuracy. That high level of certainty does NOT apply to the TLDs which have emerged over the last few years, and we will continue to see more over the coming years.
As a pharmaceutical company needs to be judicious about not blocking emails that contain keywords for their industry, we all need to personally evaluate vectors of abuse to our own servers. In this specific case, I urge you to be more judicious about blocking out top-level "anything", whether top-level domains or IP blocks. We have wiggle room with spam scores. I understand the score of 2.5. Well, try 1.5, certainly not 15. Greylist unknown servers matching the TLD criteria. Check the SPF and DKIM. Check the RBLs. Allow legitimate TLD owners like me an opportunity to allay your concerns.
Thanks for your indulgence.
-
tonyg
- Advanced member
- Posts: 51
- Joined: Fri Mar 16, 2018 5:25 pm
- Location: USA
- ZCS/ZD Version: 8.8.12.GA.3794.UBUNTU18.64 FOSS
- Contact:
Re: blacklist a domain extension
As to the location for rules, from this Zimbra wiki page and the notes by Mark Stone, I have been under the impression that any .cf file under /opt/zimbra/data/spamassassin/localrules will be incorporated into processing when we 'zmamavisdctl restart'. So it's no longer one file like sauser.cf. You can categorize all of your rules into as many files as you want, which I think is great for selective tweaks if required.
Related, from that wiki we see the suggestion to use KAM.cf. In the same KAM site folder, there is nonKAMrules.cf, the two of which I have saved as sakam.cf and sakamnon.cf. I'm about to set cron to update those scripts every few days since it looks like they are still actively maintained.
HTH
Related, from that wiki we see the suggestion to use KAM.cf. In the same KAM site folder, there is nonKAMrules.cf, the two of which I have saved as sakam.cf and sakamnon.cf. I'm about to set cron to update those scripts every few days since it looks like they are still actively maintained.
HTH
-
JDunphy
- Outstanding Member
- Posts: 901
- Joined: Fri Sep 12, 2014 11:18 pm
- Location: Victoria, BC
- ZCS/ZD Version: 9.0.0_P39 NETWORK Edition
Re: blacklist a domain extension
Thanks @tonyg and I agree with your comments ... That example showing various TLD's should NOT be used without understanding your local sender reputation components for your scoring model. It was posted as an example of another way and not a recommendation of TLD's to use nor an appropriate score. That was not the intention of the above post as everyone should have a different scoring model based on rules you use. It was to show another method of finer granularity and hopefully lead to more effective rules vs outright blocking at the amavis or postfix layer. For example, if some of those TLD's also exist in various blacklists you are scoring AND are from a country you don't normally have in your email mix than you might ladder the scoring higher with a meta rule to get more points for that particular email. If you block based on blacklist or TLD or domain, you will have false positives and will not know it. Your users will eventually have less trust in the email system you manage should they discover expected email isn't arriving.
@tony - Here is your processing order from some notes I had around which I think I copied from the output from the SA makefile when I installed 4.3.2. Not all directory paths are populated so in theory you could populate outside of zimbra to retain your customization's I would think and still keep your customization upgrade proof. Easy enough to test with SA in debug mode to see if your custom rules load from a non zimbra path.
Now a general comment if anyone is still reading this thread. 
As with anything spam, unless you have tuned it to your email mix for your users and company, you can expect a less than optimized experience. For example, some of the default SA rules can score 1.5 points for mentioning money which is pretty crazy if you host businesses. Likewise, not overriding some of the DMARC scoring rules can score an email forwarded by one of your users to their zimbra account to a tune of 7.5 points and always land in their junk folder. If possible, looking at the trained spam/ham every day would allow you to quickly tune your system and see the problem rules and scores for your email system. Adjust your rules and scoring until the accuracy increases. Sure you will see some mistakes by your users but there is really valuable information if you focus on the headers and html structure of the email and rules that are problematic. Along the way, you will discover lots of obfuscation and tracking methods such as making bayes busting text invisible by moving it off the visual screen. Much easier to target that to classify email as potentially junk versus sending it to the junk folder because it contained the word "money" or a TLD like .rocks was used.
Finally, enable some optional plugin's for problem spam can often help. For example, a modern TextCat will allow one to list your expected languages otherwise you can score UNWANTED_LANGUAGE_BODY fairly high to land that type of spam into the junk folder or higher still and not delivered. Not every plugin is enabled that is shipped with Zimbra's SA but that is an example of the many plugin's that are available.
HTH,
Jim
@tony - Here is your processing order from some notes I had around which I think I copied from the output from the SA makefile when I installed 4.3.2. Not all directory paths are populated so in theory you could populate outside of zimbra to retain your customization's I would think and still keep your customization upgrade proof. Easy enough to test with SA in debug mode to see if your custom rules load from a non zimbra path.
Code: Select all
#Default configuration data is loaded from the first existing directory in:
#
# /opt/zimbra/data/spamassassin/state/3.004002
# /opt/zimbra/data/spamassassin/rules
# /opt/zimbra/common/share/spamassassin
# /usr/local/share/spamassassin
# /usr/share/spamassassin
# Site-specific configuration data is used to override any values which
# had already been set. This is loaded from the first existing directory in:
#
# /opt/zimbra/data/spamassassin/localrules
# /opt/zimbra/common/etc/mail/spamassassin
# /opt/zimbra/common/etc/spamassassin
# /usr/local/etc/spamassassin
# /usr/pkg/etc/spamassassin
# /usr/etc/spamassassin
# /etc/mail/spamassassin
# /etc/spamassassin
# From those 3 directories, SpamAssassin will first read fiel ending in ".pre" in lexical order
# and then read files ending in ".cf" in lexical order.
# In other words, it will read init.pre first, then 10_default_prefs.cf
# before 50_scores.cf and 20_body_tests.cf before 20_head_tests.cf.
# Options in later files will override earlier files.
As with anything spam, unless you have tuned it to your email mix for your users and company, you can expect a less than optimized experience. For example, some of the default SA rules can score 1.5 points for mentioning money which is pretty crazy if you host businesses. Likewise, not overriding some of the DMARC scoring rules can score an email forwarded by one of your users to their zimbra account to a tune of 7.5 points and always land in their junk folder. If possible, looking at the trained spam/ham every day would allow you to quickly tune your system and see the problem rules and scores for your email system. Adjust your rules and scoring until the accuracy increases. Sure you will see some mistakes by your users but there is really valuable information if you focus on the headers and html structure of the email and rules that are problematic. Along the way, you will discover lots of obfuscation and tracking methods such as making bayes busting text invisible by moving it off the visual screen. Much easier to target that to classify email as potentially junk versus sending it to the junk folder because it contained the word "money" or a TLD like .rocks was used.
Finally, enable some optional plugin's for problem spam can often help. For example, a modern TextCat will allow one to list your expected languages otherwise you can score UNWANTED_LANGUAGE_BODY fairly high to land that type of spam into the junk folder or higher still and not delivered. Not every plugin is enabled that is shipped with Zimbra's SA but that is an example of the many plugin's that are available.
Code: Select all
% cd /opt/zimbra/common/lib/perl5/Mail/SpamAssassin
% ls Plugin
AccessDB.pm Attachments.pm BodyEval.pm DKIM.pm Hashcash.pm ImageInfo.pm OLEMacro.pm Pyzor.pm ReplaceTags.pm SpamCop.pm TxRep.pm URILocalBL.pm
AntiVirus.pm AutoLearnThreshold.pm BodyRuleBaseExtractor.pm DNSEval.pm HeaderEval.pm Image.pm OneLineBodyRuleType.pm Razor2.pm Reuse.pm SPF.pm URIDetail.pm VBounce.pm
AskDNS.pm AWL.pm Check.pm FreeMail.pm HTMLEval.pm MIMEEval.pm PDFInfo.pm RelayCountry.pm Rule2XSBody.pm Test.pm URIDNSBL.pm WhiteListSubject.pm
ASN.pm Bayes.pm DCC.pm HashBL.pm HTTPSMismatch.pm MIMEHeader.pm PhishTag.pm RelayEval.pm Shortcircuit.pm TextCat.pm URIEval.pm WLBLEval.pm
Jim