spamassassin not working after update to 8.7.11

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
manspan
Posts: 9
Joined: Fri Jan 23, 2015 2:08 am

spamassassin not working after update to 8.7.11

Postby manspan » Sat Jun 10, 2017 3:38 pm

hello everybody,
we upgraded our Zimbra Open Source installation to ver. 8.7.11 and found out that spamassassin was no longer filtering mails.
We have tried a lot of things to fix the issue but things got worst: Running "sa-update -D" we found missing Perl modules (Net::SSLeay, Digest::SHA1, Razor2::Client::Agent, DBI, Enode::Detect, Geo::IP, InstallFiles::Temp, IO::Socket::IP, Mail::DKIM, Net::Patricia, ExtUtils::MakeMaker, Test::More) so we installed them. After that, mails got stuck in the queue because of "lost connection with 127.0.0.1[127.0.0.1] while sending RCPT TO". To make things work again we had to disable spamassassin from amavisd.conf. Could anyone help?

regards Manolis


User avatar
JDunphy
Advanced member
Advanced member
Posts: 121
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: Release 8.7.9_GA_1794.RHEL6_64_2017
Contact:

Re: spamassassin not working after update to 8.7.11

Postby JDunphy » Sun Jun 11, 2017 4:45 pm

Can you verify any of the following?

1) Did you run ' sa-update -D' as the zimbra user?
2) What OS are you running?
3) output of perl -V | grep -A 10 INC
4) What previous version was the zimbra upgrade from?
5) What lines did you change in the configuration files as mail was flowing and now it isn't by your description.
6) output of: su - zimbra -c "/opt/zimbra/common/bin/spamassassin --lint"
7) Look at /opt/zimbra/log/zmsetup* for your upgrade to see if anything is listed there

example:

Code: Select all

su - zimbra
[zimbra@mail ~]$ perl -V |grep -A 10 INC
  @INC:
    /opt/zimbra/common/lib/perl5/x86_64-linux-thread-multi
    /opt/zimbra/common/lib/perl5/x86_64-linux-thread-multi
    /opt/zimbra/common/lib/perl5
    /usr/local/lib64/perl5
    /usr/local/share/perl5
    /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib64/perl5
    /usr/share/perl5


Might dig deeper into the installation by:

Code: Select all

% cd /opt/zimbra/common/lib/perl5/Mail
% ls SpamAssassin
AICache.pm          Dns.pm            PerMsgStatus.pm
ArchiveIterator.pm  DnsResolver.pm    PersistentAddrList.pm
AsyncLoop.pm        HTML.pm           Plugin
AutoWhitelist.pm    Locales.pm        Plugin.pm
Bayes               Locker            PluginHandler.pm
Bayes.pm            Locker.pm         RegistryBoundaries.pm
BayesStore          Logger            Reporter.pm
BayesStore.pm       Logger.pm         SQLBasedAddrList.pm
Client.pm           MailingList.pm    SpamdForkScaling.pm
Conf                Message           SubProcBackChannel.pm
Conf.pm             Message.pm        Timeout.pm
Constants.pm        NetSet.pm         Util
DBBasedAddrList.pm  PerMsgLearner.pm  Util.pm

etc, etc, etc


Perhaps this will help you determine the root cause.
manspan
Posts: 9
Joined: Fri Jan 23, 2015 2:08 am

Re: spamassassin not working after update to 8.7.11

Postby manspan » Mon Jun 19, 2017 5:12 am

1) I run sa-update -D' as the zimbra user
2) Ubuntu 14.04.5 LTS
3) perl -V | grep -A 10 INC
DEBPKG:debian/enc2xs_inc - http://bugs.debian.org/290336 Tweak enc2xs to follow symlinks and ignore missing @INC directories.
DEBPKG:debian/errno_ver - http://bugs.debian.org/343351 Remove Errno version check due to upgrade problems with long-running processes.
DEBPKG:debian/libperl_embed_doc - http://bugs.debian.org/186778 Note that libperl-dev package is required for embedded linking
DEBPKG:fixes/respect_umask - Respect umask during installation
DEBPKG:debian/writable_site_dirs - Set umask approproately for site install directories
DEBPKG:debian/extutils_set_libperl_path - EU:MM: Set location of libperl.a to /usr/lib
DEBPKG:debian/no_packlist_perllocal - Don't install .packlist or perllocal.pod for perl or vendor
DEBPKG:debian/prefix_changes - Fiddle with *PREFIX and variables written to the makefile
DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the binary targets.
DEBPKG:debian/instmodsh_doc - Debian policy doesn't install .packlist files for core or vendor.
DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH as per Debian policy.
--
DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian
DEBPKG:debian/module_build_man_extensions - http://bugs.debian.org/479460 Adjust Module::Build manual page extensions for the Debian Perl policy
DEBPKG:debian/prune_libs - http://bugs.debian.org/128355 Prune the list of libraries wanted to what we actually need.
DEBPKG:fixes/net_smtp_docs - [rt.cpan.org #36038] http://bugs.debian.org/100195 Document the Net::SMTP 'Port' option
DEBPKG:debian/perlivp - http://bugs.debian.org/510895 Make perlivp skip include directories in /usr/local
DEBPKG:debian/cpanplus_definstalldirs - http://bugs.debian.org/533707 Configure CPANPLUS to use the site directories by default.
DEBPKG:debian/cpanplus_config_path - Save local versions of CPANPLUS::Config::System into /etc/perl.
DEBPKG:debian/deprecate-with-apt - http://bugs.debian.org/702096 Point users to Debian packages of deprecated core modules
DEBPKG:debian/squelch-locale-warnings - http://bugs.debian.org/508764 Squelch locale warnings in Debian package maintainer scripts
DEBPKG:debian/skip-upstream-git-tests - Skip tests specific to the upstream Git repository
DEBPKG:debian/patchlevel - http://bugs.debian.org/567489 List packaged patches for 5.18.2-2ubuntu1.1 in patchlevel.h
--
@INC:
/etc/perl
/usr/local/lib/perl/5.18.2
/usr/local/share/perl/5.18.2
/usr/lib/perl5
/usr/share/perl5
/usr/lib/perl/5.18
/usr/share/perl/5.18
/usr/local/lib/site_perl

4) previous version 8.7.7
5) In order to make Zimbra working I uncommented @bypass_spam_checks_maps = (1); in /opt/zimbra/conf/amavisd.conf (to DISABLE anti-spam code)
6) output of: su - zimbra -c "/opt/zimbra/common/bin/spamassassin --lint":
Jun 19 07:52:52.146 [5628] warn: netset: cannot include 127.0.0.0/8 as it has already been included
Jun 19 07:52:52.149 [5628] warn: netset: cannot include 10.100.100.0/23 as it has already been included
Jun 19 07:52:52.149 [5628] warn: netset: cannot include 127.0.0.0/8 as it has already been included
Jun 19 07:52:52.150 [5628] warn: netset: cannot include 10.34.152.32/27 as it has already been included
Jun 19 07:52:52.150 [5628] warn: netset: cannot include 0:0:0:0:0:0:0:1/128 as it has already been included
7) didn't find anything strange in /opt/zimbra/log/zmsetup.20170609-173635.log
User avatar
JDunphy
Advanced member
Advanced member
Posts: 121
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: Release 8.7.9_GA_1794.RHEL6_64_2017
Contact:

Re: spamassassin not working after update to 8.7.11

Postby JDunphy » Mon Jun 19, 2017 11:02 am

manspan wrote:--
@INC:
/etc/perl
/usr/local/lib/perl/5.18.2
/usr/local/share/perl/5.18.2
/usr/lib/perl5
/usr/share/perl5
/usr/lib/perl/5.18
/usr/share/perl/5.18
/usr/local/lib/site_perl

That isn't what I was expecting. Observe:

Code: Select all

mail:~:41> which perl
/usr/bin/perl
mail:~:42> perl -V |grep -A 10 INC
  @INC:
    /usr/local/lib64/perl5
    /usr/local/share/perl5
    /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib64/perl5
    /usr/share/perl5
    .
mail:~:43> su - zimbra
[zimbra@mail ~]$ which perl
/usr/bin/perl
[zimbra@mail ~]$ perl -V |grep -A 10 INC
  @INC:
    /opt/zimbra/common/lib/perl5/x86_64-linux-thread-multi
    /opt/zimbra/common/lib/perl5/x86_64-linux-thread-multi
    /opt/zimbra/common/lib/perl5
    /usr/local/lib64/perl5
    /usr/local/share/perl5
    /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib64/perl5
    /usr/share/perl5
    .
   


Interesting that you don't have the /opt/zimbra paths included in your @INC. Without those, I would think most things would be broken and spam filtering would be just a small part. Any explanation why running perl -V is lacking those paths as the zimbra user? You might want to verify the following is set and that those directories exist and are populated with perl modules:

Code: Select all

[zimbra@mail ~]$ su - zimbra
[zimbra@mail ~]$ env |grep -i perl
PERL5LIB=/opt/zimbra/common/lib/perl5/x86_64-linux-thread-multi:/opt/zimbra/common/lib/perl5
PERLLIB=/opt/zimbra/common/lib/perl5/x86_64-linux-thread-multi:/opt/zimbra/common/lib/perl5


I am running centos but you would think for ubuntu/debian, the installation is similar with the required perl modules located under /opt/zimbra/common/lib/perl5
gb3
Posts: 14
Joined: Sat Sep 13, 2014 12:46 am

Re: spamassassin not working after update to 8.7.11

Postby gb3 » Mon Jun 19, 2017 4:25 pm

Just to add my 2 cents.... I'm having the exact same issue and I have a similar perl -V | grep -A 10 INC output.

Code: Select all

perl -V | grep -A 10 INC
   DEBPKG:debian/enc2xs_inc - http://bugs.debian.org/290336 Tweak enc2xs to follow symlinks and ignore missing @INC directories.
   DEBPKG:debian/errno_ver - http://bugs.debian.org/343351 Remove Errno version check due to upgrade problems with long-running processes.
   DEBPKG:debian/libperl_embed_doc - http://bugs.debian.org/186778 Note that libperl-dev package is required for embedded linking
   DEBPKG:fixes/respect_umask - Respect umask during installation
   DEBPKG:debian/writable_site_dirs - Set umask approproately for site install directories
   DEBPKG:debian/extutils_set_libperl_path - EU:MM: Set location of libperl.a to /usr/lib
   DEBPKG:debian/no_packlist_perllocal - Don't install .packlist or perllocal.pod for perl or vendor
   DEBPKG:debian/prefix_changes - Fiddle with *PREFIX and variables written to the makefile
   DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the binary targets.
   DEBPKG:debian/instmodsh_doc - Debian policy doesn't install .packlist files for core or vendor.
   DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH as per Debian policy.
--
   DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian
   DEBPKG:debian/module_build_man_extensions - http://bugs.debian.org/479460 Adjust Module::Build manual page extensions for the Debian Perl policy
   DEBPKG:debian/prune_libs - http://bugs.debian.org/128355 Prune the list of libraries wanted to what we actually need.
   DEBPKG:fixes/net_smtp_docs - [rt.cpan.org #36038] http://bugs.debian.org/100195 Document the Net::SMTP 'Port' option
   DEBPKG:debian/perlivp - http://bugs.debian.org/510895 Make perlivp skip include directories in /usr/local
   DEBPKG:debian/cpanplus_definstalldirs - http://bugs.debian.org/533707 Configure CPANPLUS to use the site directories by default.
   DEBPKG:debian/cpanplus_config_path - Save local versions of CPANPLUS::Config::System into /etc/perl.
   DEBPKG:debian/deprecate-with-apt - http://bugs.debian.org/702096 Point users to Debian packages of deprecated core modules
   DEBPKG:debian/squelch-locale-warnings - http://bugs.debian.org/508764 Squelch locale warnings in Debian package maintainer scripts
   DEBPKG:debian/skip-upstream-git-tests - Skip tests specific to the upstream Git repository
   DEBPKG:debian/patchlevel - http://bugs.debian.org/567489 List packaged patches for 5.18.2-2ubuntu1.1 in patchlevel.h
--
  @INC:
    /opt/zimbra/common/lib/perl5/x86_64-linux-gnu-thread-multi
    /opt/zimbra/common/lib/perl5/x86_64-linux-gnu-thread-multi
    /opt/zimbra/common/lib/perl5
    /etc/perl
    /usr/local/lib/perl/5.18.2
    /usr/local/share/perl/5.18.2
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.18
    /usr/share/perl/5.18
User avatar
JDunphy
Advanced member
Advanced member
Posts: 121
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: Release 8.7.9_GA_1794.RHEL6_64_2017
Contact:

Re: spamassassin not working after update to 8.7.11

Postby JDunphy » Mon Jun 19, 2017 4:57 pm

Interesting that you have same problem. Yours is different than the OP because you have the zimbra perl paths in your @INC you provided.

Did you check this to make sure the code is there?

Code: Select all

% cd /opt/zimbra/common/lib/perl5/Mail
% ls SpamAssassin
AICache.pm          Dns.pm            PerMsgStatus.pm
ArchiveIterator.pm  DnsResolver.pm    PersistentAddrList.pm
AsyncLoop.pm        HTML.pm           Plugin
AutoWhitelist.pm    Locales.pm        Plugin.pm
Bayes               Locker            PluginHandler.pm
Bayes.pm            Locker.pm         RegistryBoundaries.pm
BayesStore          Logger            Reporter.pm
BayesStore.pm       Logger.pm         SQLBasedAddrList.pm
Client.pm           MailingList.pm    SpamdForkScaling.pm
Conf                Message           SubProcBackChannel.pm
Conf.pm             Message.pm        Timeout.pm
Constants.pm        NetSet.pm         Util
DBBasedAddrList.pm  PerMsgLearner.pm  Util.pm

etc, etc, etc


Keep in mind, I am referencing a centos 6.8 network install (commercial version) but I would think it should be similar even for different OS's and open source.
manspan
Posts: 9
Joined: Fri Jan 23, 2015 2:08 am

Re: spamassassin not working after update to 8.7.11

Postby manspan » Tue Jun 20, 2017 7:22 pm

I've run /opt/zimbra/common/sbin/amavisd - debug and found some missing perl modules. I've done the necessary installations and now I get:

1)
% cd /opt/zimbra/common/lib/perl5/Mail
% ls SpamAssassin
AICache.pm Dns.pm PerMsgStatus.pm
ArchiveIterator.pm DnsResolver.pm PersistentAddrList.pm
AsyncLoop.pm TML.pm Plugin
AutoWhitelist.pm Locales.pm PluginHandler.pm
Bayes Locker Plugin.pm
Bayes.pm Locker.pm RegistryBoundaries.pm
BayesStore Logger Reporter.pm
BayesStore.pm Logger.pm SpamdForkScaling.pm
Client.pm MailingList.pm SQLBasedAddrList.pm
Conf Message SubProcBackChannel.pm
Conf.pm Message.pm Timeout.pm
Constants.pm NetSet.pm Util
DBBasedAddrList.pm PerMsgLearner.pm Util.pm

2)
% which perl
/usr/bin/perl

% perl -V | grep -A 10 INC
@INC:
/etc/perl
/usr/local/lib/perl/5.18.2
/usr/local/share/perl/5.18.2
/usr/lib/perl5
/usr/share/perl5
/usr/lib/perl/5.18
/usr/share/perl/5.18
/usr/local/lib/site_perl

3)
% su - zimbra
% env |grep -i perl
PERL5LIB=/opt/zimbra/common/lib/perl5/x86_64-linux-gnu-thread-multi:/opt/zimbra/common/lib/perl5
PERLLIB=/opt/zimbra/common/lib/perl5/x86_64-linux-gnu-thread-multi:/opt/zimbra/common/lib/perl5

After that amavisd complained that Mail::Spamassassin could not be found, so I executed PERL5LIB=/opt/zimbra/common/lib/perl5 and added the path.

Then I got "Config file "/etc/amavisd.conf" does not exist, at ./amavisd line 2152."

a)I've tried amavisd -c /opt/zimbra/conf/amavisd.conf showkeys
and i got Config file "/opt/zimbra/conf/amavisd.conf" should not be owned by EUID 999, at /opt/zimbra/common/sbin/amavisd line 2152.

b)I've changed amavisd.conf owenership to root:root and try again amavisd -c /opt/zimbra/conf/amavisd.conf showkeys
and i got Error reading config file "/opt/zimbra/conf/amavisd.conf": Permission denied at /opt/zimbra/common/sbin/amavisd line 2165.

Now I'm stuck!!!
Antispam and antivirus service is not running...
User avatar
JDunphy
Advanced member
Advanced member
Posts: 121
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: Release 8.7.9_GA_1794.RHEL6_64_2017
Contact:

Re: spamassassin not working after update to 8.7.11

Postby JDunphy » Tue Jun 20, 2017 7:44 pm

Did you happen to update the default shell for zimbra? I noticed that ~zimbra/.bashrc has this:

Code: Select all

eval `/usr/bin/perl -V:archname`
PERLLIB=/opt/zimbra/common/lib/perl5/$archname:/opt/zimbra/common/lib/perl5
export PERLLIB

PERL5LIB=$PERLLIB
export PERL5LIB


Note: that PERLLIB is 2 paths... /opt/zimbra/common/lib/perl5 was after the first one.

Given that zimbra has it's own tree, I can't explain why your commands continue to yield references outside the /opt/zimbra installation. The biggest thing for me is that you don't have perl looking at @INC

Code: Select all

    @INC
    /opt/zimbra/common/lib/perl5/x86_64-linux-thread-multi
    /opt/zimbra/common/lib/perl5


It will always look in order which is outside /opt/zimbra from the output you are providing? The other thing is after changing your environment, you need to bounce the software so any children can inherit those environmental settings. I am not being much help here so hopefully someone else has an idea what went wrong for your update.
gb3
Posts: 14
Joined: Sat Sep 13, 2014 12:46 am

Re: spamassassin not working after update to 8.7.11

Postby gb3 » Thu Jun 22, 2017 1:38 am

After looking into this issue further I found that there is an open ticket for Ubuntu 12 (possibly 14) where AMAVISD just stops working properly. This may be the issue you were having as well.

https://forums.zimbra.org/viewtopic.php?t=60179

A corresponding bug has been reported as well here: https://bugzilla.zimbra.com/show_bug.cgi?id=106361

I believe this workaround has solved my issue and may do the same for you.

Return to “Administrators”

Who is online

Users browsing this forum: No registered users and 7 guests