Page 14 of 24

Re: Rspamd: A replacement for Spamassassin & Postscreen

Posted: Wed Jan 10, 2018 9:09 pm
by vstakhov
I think it's quite clear that no generic purposes spam filter can deal with spam in all languages all over the world out of the box. You need to customise both Rspamd and SA for your particular environment. Both Rspamd and SA have similar techniques to filter spam (Rspamd can even use SA rules). Therefore, I'm not surprised that your customised SA deals with *your* mail traffic better than uncustomised Rspamd. Furthermore, Rspamd has many modules disabled by default providing, generally speaking, personal or small company spam filtering functions out-of-the-box. For everything more than that you need to add your custom intelligence to improve spam filtering (custom rules, corpus training, complaints processing, etc).

With regard to the performance and CPU usage spikes: I'm pretty sure that there was something special about your usage patterns. Unfortunately, you have not provided information about this issue so I'm totally lost what was wrong in your case: some of Rspamd users have really highly loaded systems with more than 1000 messages per second in peak times. And you were the first who reported about weird cpu usage (even on CentOS 6), so I can conclude that you were doing something wrong (or, at least, unexpected and thus untested).

Re: Rspamd: A replacement for Spamassassin & Postscreen

Posted: Thu Jan 11, 2018 9:06 am
by bunny
Hello Sir,
Thanks to the author for providing the updated version of rspamd-1.6.5-4.x86_64 with which rspamd is starting without any problem.

I have temporarily setup and enabled rspamd on our production server which is in DMZ behind the Firewall. When rspamd service is started, mails-both inbound and outbound are not passing through the server. Some are getting rejected with “query refused” error and some are found in the deferred queue with error “connection refused”.

Code: Select all

Error logs from zimbra.log:

-warning: <IPAddress>.relays.mail-abuse.org: RBL lookup error: Host or domain name not found. Name service error for name=<IPAddress>.relays.mail-abuse.org type=A: Host not found, try again
-to=<userID@ourdomain>, relay=none, delay=48, delays=48/0.01/0/0, dsn=4.4.1, status=deferred (connect to 127.0.0.1[127.0.0.1]:10024: Connection refused)
-warning: connect to Milter service inet:localhost:11332: Connection refused 
Also, I have noticed the following error messages in /var/log/messages
Jan 11 14:16:31 primary named-sdb[8510]: error (network unreachable) resolving '1.1.2.144.zen.spamhaus.org/A/IN': 2a00:1a28:1251:178:73:210:119:fa53#53
Jan 11 14:16:31 primary named-sdb[8510]: error (network unreachable) resolving '26.221.168.184.sbl.spamhaus.org/A/IN': 2a03:b0c0:1:d0::257b:e00e#53
Jan 11 14:42:40 primary named-sdb[8510]: error (host unreachable) resolving '26.189.93.201.in-addr.arpa/PTR/IN': 189.19.56.230#53
My named.conf installed in the server itself is as follows:

Code: Select all

options {
        listen-on port 53 { 127.0.0.1; <serverIP>; };
        listen-on-v6 port 53 { ::1; };
        directory       "/var/named";
        dump-file       "/var/named/data/cache_dump.db";
        statistics-file "/var/named/data/named_stats.txt";
        memstatistics-file "/var/named/data/named_mem_stats.txt";
        allow-query     { localhost; any; };
        recursion yes;

        dnssec-enable yes;
        dnssec-validation yes;
        dnssec-lookaside auto;

        /* Path to ISC DLV key */
        bindkeys-file "/etc/named.iscdlv.key";

        managed-keys-directory "/var/named/dynamic";
// configured paths to named.pid and session.key files in next 2 lines
//      pid-file "/var/named/chroot/var/run/named/named.pid";
//      session-keyfile "/var/named/chroot/var/run/named/session.key";
};

logging {
        channel default_debug {
                file "data/named.run";
                severity dynamic;
        };
};

zone "." IN {
        type hint;
        file "named.ca";
};

include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";
In our server, I have also enabled zimbra-milter to implement sendToDistList restrictions and cbpolicyd for ratelimit.

May I know where I have mis-configured the system.

Thanks & Regards,

Re: Rspamd: A replacement for Spamassassin & Postscreen

Posted: Thu Jan 11, 2018 9:52 am
by phoenix
It looks like those errors are related to DNS, I'd start by checking if you can test those RBL lookups from the command line on your ZCS server and also check if your firewall is ok and/or your network.

Re: Rspamd: A replacement for Spamassassin & Postscreen

Posted: Thu Jan 11, 2018 11:29 am
by bunny
Hello Sir,

If I dig from the prompt, main domains are responding, but not sub-domains. For eg:
From
Jan 11 17:05:33 primary named-sdb[3781]: error (network unreachable) resolving 'bondedsender.org/DS/IN': 2001:500:f::1#53
Bondsender.org is responding

From
Jan 11 17:05:25 primary named-sdb[3781]: error (network unreachable) resolving 'x.ns.spamhaus.org/AAAA/IN': 2400:cb00:2049:1::a29f:1823#53
spamhaus.org is responding whereas sbl.spamhaus.org and zen.spamhaus.org

From
Jan 11 16:54:12 primary named-sdb[27291]: error (connection refused) resolving '197.123.75.208.b.barracudacentral.org/A/IN': 64.235.145.15#53
barracudacentral.org works

Thanks & Regards,

Re: Rspamd: A replacement for Spamassassin & Postscreen

Posted: Thu Jan 11, 2018 2:25 pm
by phoenix
I didn't ask you to check the domains, I asked you to check the RBL entries: https://www.startpage.com/do/dsearch?qu ... ge=english I also mentioned that it might be a problem with your firewall or network and you need to check those as well.

Re: Rspamd: A replacement for Spamassassin & Postscreen

Posted: Sat Feb 03, 2018 10:26 pm
by sangamc
bunny wrote:Hello Sir,

Migration of my server to production mode is delayed due to some policy decisions. Till I saw the post by 10424bofh I was very confident that I could configure the server with antispam solution. But, now I lost my confidence. I am not an expert in this, just maintaining the server with online documentation and discussions in forums. After reading the suggestion of 10424bofh I am thinking whether to move the server to production mode or not since all our users are verymuch dependent on mail correspondence.
sangamc wrote:I am using Rspamd on production servers. Switched 2 servers this week and plan to switch them all over the next few days. Of the 4 email servers we have. The largest has approx 500 users and recieves approx 10 to 15K messages a day. The smallest is my office server with 30 users and about 3k messages a day. I dont mind testing features or running benchmarks so if you have any questions let me know. All servers are hosted by centurylink running Zcs 8.7.11 on centos 6.9 with 2 cpu and 4GB ram and from 300gb to 1TB hdd space.
May I know whether you have switched all your 4 email servers to production. I am eager to know about the performance of the server that is handling 500 users. Can you please give your feedback on the configuration / post configuration changes made in rspamd.

Thanks & Regards
Sorry for the late response. I didn't get the notification. I did finish 3 out of 4 servers. The largest one rpsmad plans are on hold but I have been running Rspamd on a server with 300 users.

SO far things have been working great but over the last couple of days i have a strange issue with postfix user runs cleanup -z -t -unix -u every 40 mins and CPU hits 100%!! I am still investigating and NOT sure if its rspamd since I have 2 other servers where it runs perfectly. I included a link to the CPU stats Image
https://ibb.co/nx54G6

Users have seen a dramatic reduction in SPAM. (some users miss it :),)
Server performance did not change much for me. Rspamd fits right into the gap Spamassasin left as far as server resources.
Customizing your SPAM protection takes a quantum leap using the web interface to whitelist / blacklist IP, domain name, subject, header, recipient, etc. ALL on the fly. Making life super easy when it comes to combating spam as well as dealing with false positives or emails from 'that company that can't configure their email server properly'
The web interface is fantastic (especially when I have to show the Bigwigs what they are paying me for LOL)

Here is a typical day for us
Image
https://ibb.co/mdpd9R

Re: Rspamd: A replacement for Spamassassin & Postscreen

Posted: Tue Feb 06, 2018 7:39 am
by bunny
Hello Sir,

Thank you for your reply. I could rectify the errors which were in DNS configuration as pointed out by Phoenix and enabled rspamd in our production server for about 10days. But once it started working, the other policies I have implemented like zmmilter (for sendtodistlist), cbpolicyd (ratelimit) are getting disabled. These features are very essential in our environment. Can we configure the same features in rspamd?

Thanks & Regards,

Re: Rspamd: A replacement for Spamassassin & Postscreen

Posted: Tue Feb 06, 2018 7:43 am
by phoenix
Ratelimiting can be done by rspamd but I don't understand why any of the features you mention are are being disabled, can you explain in a bit more detail exactly what's happening?

Re: Rspamd: A replacement for Spamassassin & Postscreen

Posted: Tue Feb 06, 2018 11:57 am
by bunny
Hello Sir,

Initially, when the mail server was installed and configured, we have configured
- zmcbpolicyd - for ratelimit - due to which a user can send only to 15 recipients in 60sec.
- zmmilter - for group mailing - due to which sending mails to distribution lists is restricted to only some of the internal users.
The above were implemented inorder to handle compromised accounts, if any.

Now, after installing rspamd I noticed that these features are not working when sudeenly one of the users has sent mail to all the users in our institute and permissions to him are denied in milter settings. Samething happened with ratelimiting.

zmmilter works on port no. 7026 and that we are replacing with 11332. Is this the reason that milter is not working.

Thanks & Regards

Re: Rspamd: A replacement for Spamassassin & Postscreen

Posted: Tue Feb 06, 2018 12:55 pm
by phoenix
I don't use either of those features on my server but give me a while and I'll check on a test server to see what happens with them and rspamd.

BTW, for compromised accounts did you implement a better and stronger password policy on your server?