[SOLVED] Zimbra Migration hits "undefined symbol: Perl_Istack_sp_ptr" Centos6 to Centos7

[mzietlow]

Been pulling my few remaining hairs out the last several hours...

I'm having trouble migrating Zimbra 8.8.15_GA_3980 from Centos/6 VM to a Centos/7 VM.

Not sure if I can get support here but I'm Hitting the following Perl Error after running ./install.sh
"/usr/bin/perl: symbol lookup error: /opt/zimbra/common/lib/perl5//x86_64-linux-thread-multi/auto/Socket/Socket.so: undefined symbol: Perl_Istack_sp_ptr"

Was Following the Guide Here:
https://wiki.zimbra.com/wiki/How_to_mov ... her_server

Any insight into my steps would sure be appreciated.
The Centos/6 server I'm migrating from uses Perl5.10.1 so I'm not sure if this means I need to downgrade?
I'll try anything at this point.

Code: Select all

[root@mx1 zcs-8.8.15_GA_3869.RHEL7_64.20190918004220]# uname -a
Linux mx1.mydomainl.com 3.10.0-1160.15.2.el7.x86_64 #1 SMP Wed Feb 3 15:06:38 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

[root@mx1 zcs-8.8.15_GA_3869.RHEL7_64.20190918004220]# ./install.sh -s

Operations logged to /tmp/install.log.Hkjpockj
Checking for existing installation...
    zimbra-drive...NOT FOUND
    zimbra-imapd...NOT FOUND
    zimbra-patch...NOT FOUND
...blah...blah...blah...
ZCS upgrade from 8.8.15 to 8.8.15 will be performed.
...
Software Installation complete!

[root@mx1 zcs-8.8.15_GA_3869.RHEL7_64.20190918004220]# rm -rf /opt/zimbra
[root@mx1 zcs-8.8.15_GA_3869.RHEL7_64.20190918004220]# mv  ~/zimbra.backup /opt/zimbra
[root@mx1 zcs-8.8.15_GA_3869.RHEL7_64.20190918004220]# /opt/zimbra/libexec/zmfixperms -e -v
...
[root@mx1 zcs-8.8.15_GA_3869.RHEL7_64.20190918004220]# ./install.sh

Operations logged to /tmp/install.log.VMCdYtk8
Checking for existing installation...
    zimbra-drive...FOUND zimbra-drive-1.0.13.1576152256-1
    zimbra-imapd...NOT FOUND
    zimbra-patch...FOUND zimbra-patch-8.8.15.1611217815
    zimbra-mta-patch...FOUND zimbra-mta-patch-8.8.15.1610596132
    zimbra-proxy-patch...FOUND zimbra-proxy-patch-8.8.15.1604495621
    zimbra-license-tools...NOT FOUND
    zimbra-license-extension...NOT FOUND
    zimbra-network-store...NOT FOUND
    zimbra-network-modules-ng...NOT FOUND
    zimbra-chat...NOT FOUND
    zimbra-talk...NOT FOUND
    zimbra-ldap...FOUND zimbra-ldap-8.8.15_GA_3869
    zimbra-logger...FOUND zimbra-logger-8.8.15_GA_3869
    zimbra-mta...FOUND zimbra-mta-8.8.15_GA_3869
    zimbra-dnscache...NOT FOUND
    zimbra-snmp...FOUND zimbra-snmp-8.8.15_GA_3869
    zimbra-store...FOUND zimbra-store-8.8.15_GA_3869
    zimbra-apache...FOUND zimbra-apache-8.8.15_GA_3869
    zimbra-spell...FOUND zimbra-spell-8.8.15_GA_3869
    zimbra-convertd...NOT FOUND
    zimbra-memcached...FOUND zimbra-memcached-1.4.37-2
    zimbra-proxy...FOUND zimbra-proxy-8.8.15_GA_3869
    zimbra-archiving...NOT FOUND
    zimbra-core...FOUND zimbra-core-8.8.15_GA_3869
ZCS upgrade from 8.8.15 to 8.8.15 will be performed.
Validating ldap configuration
/usr/bin/perl: symbol lookup error: /opt/zimbra/common/lib/perl5//x86_64-linux-thread-multi/auto/Socket/Socket.so: undefined symbol: Perl_Istack_sp_ptr
Unknown Error: It should be impossible to reach this statement.
[root@mx1 zcs-8.8.15_GA_3869.RHEL7_64.20190918004220]#

[root@mx1 zcs-8.8.15_GA_3869.RHEL7_64.20190918004220]# perl -v

This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi
(with 44 registered patches, see perl -V for more detail)

Copyright 1987-2012, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

[root@mx1 zcs-8.8.15_GA_3869.RHEL7_64.20190918004220]# su - zimbra
[zimbra@mx1 ~]$ perl -MCPAN -e shell
perl: symbol lookup error: /opt/zimbra/common/lib/perl5/x86_64-linux-thread-multi/auto/Socket/Socket.so: undefined symbol: Perl_Istack_sp_ptr

[Klug]

You have ran "./install.sh -s" on the destination server once to set it up, this is OK.

But after you've done it, you have replaced all the binaries by CentOS6 binaries (rm /opt/zimbra and move files over).

You should be running "./install.sh -s" again and not "./install.sh"
"-s" is for "software only".

If you run "./install.sh", you're going through a full installation/upgrade and the script tries to use the files that are already on the server.
As you have a CentOS7 server and you copied a CentOS6 version of ZCS, libraries and binaries are CentoS6 version and the don't match the CentOS7 OS: you get the error.

So, what I'd do is a "./install.sh -s" to reinstall the proper libraries/binaries.
And that's all.

I've been doing this a lot lately and this is what I do:
. ./install.sh -s on the new server to setup the new environment
. stop old server, dump LDAP databases (the three of them)
. rsync everything but LDAP sparse files
. copy LDAP dumps to new server
. ./install.sh -s on new server to reinstall the new binaries/libraries
. import LDAP dumps on new server

The new server has to have the name name than the old one.
/etc/hosts and DNS have to be correct.

LDAP export/import : https://wiki.zimbra.com/wiki/LDAP_data_import_export

[mzietlow]

If you run "./install.sh", you're going through a full installation/upgrade and the script tries to use the files that are already on the server.
As you have a CentOS7 server and you copied a CentOS6 version of ZCS, libraries and binaries are CentoS6 version and the don't match the CentOS7 OS: you get the error.

So, what I'd do is a "./install.sh -s" to reinstall the proper libraries/binaries.
And that's all.

Hi Klug,
Thanks for the quick reply.
I realized last night that my arch was too different and probably the cause.
What sent me down this unnecessary long rabbit hole was this:

Code: Select all

Do you want to verify message store database integrity? [Y]
Verifying integrity of message store databases.  This may take a while.
Starting mysqld...done.
mysqld is alive
perl: symbol lookup error: /opt/zimbra/common/lib/perl5/x86_64-linux-thread-multi/auto/Socket/Socket.so: undefined symbol: Perl_Istack_sp_ptr

I'm used to solving errors not ignoring them...
Running install -s TWICE and DO NOT-DO NOT-DO NOT-DO NOT "verify message store database integrity? [Y]" (because it's supposed to fail in this test case) addressed my Perl_Istack_sp_ptr error!


The only Issues I need to resolve now are
1. Why does proxy keep asking me for sudo pass when I have suders set to "%zimbra ALL=NOPASSWD:/opt/zimbra/common/sbin/nginx"
EDIT, RESOLVED: sudo chgrp zimbra /opt/zimbra/common/sbin/nginx
2. Somehow I missed libpcre prereq'...
EDIT, RESOLVED: yum -y install pcre-devel;ln -s /usr/lib64/libpcre.so.1.2.0 /usr/lib64/libpcre.so.0
3. postfix failed to start.
EDIT RESOLVED: netstat -anp | grep :25;service postfix stop;systemctl disable postfix

EDIT: ALL BELOW ERRORS RESOLVED!!!!

Code: Select all

[zimbra@mx1 ~]$ zmcontrol start
Host mx1.zmdomain.com
        Starting ldap...Done.
        Starting zmconfigd...Done.
        Starting logger...Done.
        Starting mailbox...Done.
        Starting memcached...Done.
        Starting proxy...[sudo] password for zimbra:
Failed.
Starting proxy.../opt/zimbra/common/sbin/nginx: error while loading shared libraries: libpcre.so.0: cannot open shared object file: No such file or directory
failed.


        Starting amavis...Done.
        Starting antispam...Done.
        Starting antivirus...Done.
        Starting opendkim...Done.
        Starting snmp...Done.
        Starting spell...Done.
        Starting mta...Failed.
Starting saslauthd...done.
postfix failed to start


        Starting stats...Done.
        Starting service webapp...Done.
        Starting zimbra webapp...Done.
        Starting zimbraAdmin webapp...Done.
        Starting zimlet webapp...Done.

Thank you!
-Z