Zimbra 10 FOSS Installation Guide

[adrian.gibanel.btactic]

Anyway, in /opt/zimbra/libs there's BOTH a commons-io-2.6.jar and a commons-io-1.4.jar.
Is there somewhere else I need to look, or something else I need to do to ensure Zimbra's looking at the right version of commons-io?

List every jar in /opt/zimbra/lib/jars.

Code: Select all

ls -lh /opt/zimbra/lib/jars/*jar | less

There might be another repeated jar that messes up everything (by trying to use an obsolete or future call from commons) .
The usual suspect for me would be and slapd or ldap jar, however given the problem you describe it might be another jar.

[adrian.gibanel.btactic]

Is there somewhere else I need to look, or something else I need to do to ensure Zimbra's looking at the right version of commons-io?

Just yesterday a commit appeared towards develop branch regarding some old jars not being removed prior to the upgrade.

ZCS-14910 : delete old SMIME jar before upgrade (#274)

Why don't you check:

Code: Select all

ls -lh /opt/zimbra/lib/ext/com_zimbra_ssdb_ephemeral_store/*jar
ls -lh /opt/zimbra/lib/ext/smime/*jar
ls -lh /opt/zimbra/extensions-extra/openidconsumer/*jar

to see if there are repeated jars there?

If you are trying to send attachments it might be related to smime being turned on your system and not on the other tested system. Also combined to the fact that you upgrade from 8.8.15 which it's quite old. Just a wild guess.

[bjquinn2]

Anyway, in /opt/zimbra/libs there's BOTH a commons-io-2.6.jar and a commons-io-1.4.jar.
Is there somewhere else I need to look, or something else I need to do to ensure Zimbra's looking at the right version of commons-io?

List every jar in /opt/zimbra/lib/jars.

Code: Select all

ls -lh /opt/zimbra/lib/jars/*jar | less

There might be another repeated jar that messes up everything (by trying to use an obsolete or future call from commons) .
The usual suspect for me would be and slapd or ldap jar, however given the problem you describe it might be another jar.

Since the error I'm getting is:

msg: system failure: java.lang.NoSuchMethodError: 'int org.apache.commons.io.IOUtils.read(java.io.InputStream, byte[])'

And there are duplicate jars for commons-io in /opt/zimbra/libs, and the older jar was v1.4, which was prior to the implementation of IOUtils.read in v2.0, that just HAD to be the solution. But I removed the old v1.4 jar, and then zmcontrol restart, and even rebooted, and the exact same error remained. That's why I'm thinking there's got to be something else I need to do - a cache somewhere, or a different folder with jars in it or something. I don't think it's a difference in smime, as it's the same machine (I'm snapshotting it, upgrading it, and then reverting back when it doesn't work).

Also, for what it's worth, both of those common-io jars were there in 8.8.15 as well. Maybe 10.x started using IOUtils.read, whereas 8.8.15 didn't.

Anyway, to answer your other question, I have a TON of duplicated jars. I put asterisks next to the ones that appear duplicated to me. Is the right thing to do to just pick the latest version of each one and get rid of the other one?

Code: Select all

UserAgentUtils-1.21.jar
activation-1.1.1.jar
***ant-1.6.5.jar
***ant-1.7.0-ziputil-patched.jar
ant-contrib-1.0b2.jar
ant-tar-patched.jar
***antisamy-1.5.8z.jar
***antisamy-1.5.8z2.jar
antlr-3.2.jar
apache-jsieve-core-0.5.jar
***apache-jsp-9.4.18.v20190429.jar
***apache-jsp-9.4.46.v20220331.jar
apache-log4j-extras-1.0.jar
apache-mime4j-core-0.8.7.jar
***asm-3.3.1.jar
***asm-8.0.1.jar
batik-css-1.7.jar
***batik-i18n-1.14.jar
***batik-i18n-1.9.jar
***batik-util-1.14.jar
***batik-util-1.8.jar
bcprov-jdk15on-1.64.jar
commons-cli-1.2.jar
***commons-codec-1.14.jar
***commons-codec-1.7.jar
commons-collections-3.2.2.jar
***commons-compress-1.10.jar
***commons-compress-1.20.jar
commons-csv-1.2.jar
commons-dbcp-1.4.jar
commons-fileupload-1.4.jar
***commons-io-1.4.jar
***commons-io-2.6.jar
commons-lang-2.6.jar
commons-logging.jar
commons-net-3.3.jar
commons-pool-1.6.jar
commons-pool2-2.4.2.jar
concurrentlinkedhashmap-lru-1.3.1.jar
curator-client-2.0.1-incubating.jar
curator-framework-2.0.1-incubating.jar
curator-recipes-2.0.1-incubating.jar
curator-x-discovery-2.0.1-incubating.jar
cxf-2.7.18.jar
cxf-core-3.5.5.jar
dom4j-2.1.1.jar
ehcache-3.1.2.jar
freemarker-2.3.19.jar
gifencoder-0.9.jar
gmbal-api-only-2.2.6.jar
guava-28.1-jre.jar
helix-core-0.6.1-incubating.jar
httpasyncclient-4.1.4.jar
httpclient-4.5.8.jar
httpcore-4.4.11.jar
httpcore-nio-4.4.11.jar
httpmime-4.3.1.jar
ical4j-0.9.16-patched.jar
icu4j-4.8.1.1.jar
istack-commons-runtime-3.0.8.jar
jackson-annotations-2.10.1.jar
jackson-core-2.10.1.jar
jackson-databind-2.10.1.jar
jackson-dataformat-smile-2.9.2.jar
jackson-module-jaxb-annotations-2.8.9.jar
jamm-0.2.5.jar
javax.annotation-api-1.2.jar
javax.servlet-api-3.1.0.jar
javax.ws.rs-api-2.0-m10.jar
jaxb-api-2.3.1.jar
jaxb-impl-2.3.1.jar
jaxen-1.1.3.jar
jaxws-api-2.3.1.jar
jaxws-rt-4.0.0.jar
jcharset-2.0.jar
jcommon-1.0.21.jar
jcs-1.3.jar
jdom-1.1.jar
jedis-2.9.0.jar
jersey-client-1.11.jar
jersey-core-1.11.jar
jersey-json-1.11.jar
jersey-multipart-1.11.jar
jersey-server-1.11.jar
jersey-servlet-1.11.jar
jfreechart-1.0.15.jar
jline-0.9.93.jar
jna-3.4.0.jar
json.jar
jsr181-api-1.0-MR1.jar
jsr311-api-1.1.1.jar
junixsocket-common-2.3.2.jar
junixsocket-demo-2.3.2.jar
junixsocket-mysql-2.3.2.jar
junixsocket-native-common-2.3.2.jar
junixsocket-rmi-2.3.2.jar
jython-standalone-2.5.2.jar
jzlib-1.0.7.jar
libidn-1.24.jar
log4j-1.2.16.jar
log4j-api-2.17.1.jar
log4j-core-2.17.1.jar
lucene-analyzers-3.5.0.jar
lucene-core-3.5.0.jar
lucene-smartcn-3.5.0.jar
mail-1.4.7.jar
mariadb-java-client-2.4.3.jar
mina-core-2.0.4.jar
mina-core-2.1.6.jar
native-lib-loader-2.3.5.jar
neethi-3.0.2.jar
nekohtml-1.9.13.1z.jar
oauth-20100527.jar
policy-2.3.jar
resolver-20050927.jar
sac-1.3.jar
***slf4j-api-1.7.30.jar
***slf4j-api-1.7.36.jar
slf4j-log4j12-1.7.30.jar
slf4j-simple-1.7.36.jar
***spring-aop-3.0.7.RELEASE.jar
***spring-aop-6.0.8.jar
***spring-beans-3.0.7.RELEASE.jar
***spring-beans-6.0.8.jar
***spring-context-3.0.7.RELEASE.jar
***spring-context-6.0.8.jar
***spring-core-3.0.7.RELEASE.jar
***spring-core-6.0.8.jar
***spring-expression-3.0.7.RELEASE.jar
***spring-expression-6.0.8.jar
spymemcached-2.12.1.jar
sqlite-jdbc-3.7.15-M1.jar
stax-ex-1.7.7.jar
stax2-api-3.1.1.jar
streambuffer-2.2.6.jar
syslog4j-0.9.46.jar
tika-core-1.24.1.jar
unboundid-ldapsdk-2.3.5.jar
woodstox-core-asl-4.2.0.jar
wsdl4j-1.6.3.jar
xercesImpl-2.9.1-patch-01.jar
xmlschema-core-2.0.3.jar
yuicompressor-2.4.2-zimbra.jar
zimbra-charset.jar
zimbra-native.jar
zimbraclient.jar
zimbracommon.jar
zimbrasoap.jar
zimbrastore.jar
zkclient-0.1.0.jar
***zm-ews-stub-2.0.jar
***zm-ews-stub-4.0.jar
zmzimbratozimbramig.jar
zookeeper-3.4.5.jar

I have duplicated jars in jetty_home as well.

EDIT: I moved out all of the duplicates in libs and jetty_home, and restarted, but it still gave me the same error.

[adrian.gibanel.btactic]

or a different folder with jars in it or something.

As per my previous comment:

Why don't you check:

Code: Select all

ls -lh /opt/zimbra/lib/ext/com_zimbra_ssdb_ephemeral_store/*jar
ls -lh /opt/zimbra/lib/ext/smime/*jar
ls -lh /opt/zimbra/extensions-extra/openidconsumer/*jar

to see if there are repeated jars there?

[bjquinn2]

or a different folder with jars in it or something.

As per my previous comment:

Why don't you check:

Code: Select all

ls -lh /opt/zimbra/lib/ext/com_zimbra_ssdb_ephemeral_store/*jar
ls -lh /opt/zimbra/lib/ext/smime/*jar
ls -lh /opt/zimbra/extensions-extra/openidconsumer/*jar

to see if there are repeated jars there?

Apologies, I did check there. No duplicate jars in those locations.

[adrian.gibanel.btactic]

Apologies, I did check there. No duplicate jars in those locations.

If you have already checked every jar thanks to something similar to:

Code: Select all

cd /opt/zimbra
find -type f -iname '*jar'

you might want to give a shot to the Maldua's 10.0.7 release even if I'm not confident it will solve your problem when you upgrade from 8.8.15.

If I am being honest I have always suspected that this was a problem about Ianw's build having those additional 10.1 repos (because of the build being so develop branch dependent). The fact that he and another guy have tried that release and they have not found your problem mostly discard this hypothesis.

If you believe in your compromised scenario you should probably check your jar files md5sum towards another VPS with a clean 10.0.7 install and compare. Something like, get all of the jars of the installation, order them, get their md5sum and save the result onto a file. Do the same thing onto the clean installed server and then compare the two files. You might then spot some additional or modified jar files that your cracker puts onto your zimbra thanks to leftover cron entry.

Your problem seems more and more offtopic towards this thread original subject.

[halfgaar]

FIY, my Zimbra installation using Ian's build from techfines.online (10.0.6-ish, Ubuntu 18.04) is presenting me with apt updates of several support packages. For instance zimbra-php is being upgraded from 7.4 to 8.3, which is a major release change. Running 'apt-cache policy' shows the packages come from https://repo.zimbra.com/apt/1010, which are the 10.1 release. It was mentioned here on the forums by a Zimbra employee (can't remember who), that 1010 is being added because the build is from the development branch.

For now, I've disabled the said repo in apt. Just mentioning, as it may need addressing.

[JDunphy]

Speaking of version 10 only. What are the advantages of building from the development branch currently? For a while it was the only method that seemed to work especially for version 8 and 9.

They appear to be using a more sane approach to building FOSS releases starting with version 10. One advantage might be to have the latest security fixes and features at all times but is that an approach that production servers would want? I like that Ian's script can do this but it's also trivial to add building by the latest tag at this point after some initial guessing to the branch for zm-build. The build.pl just iterates through the tags so it probably isn't as complex as we once thought.

Part of me wonders if it's even worth guessing and generating the tags if they continue to be disciplined about the version 10 tags. Below are 2 versions of bash scripts I use to build by tags while looking into this last month to see if we could automate the guessing and builds. The last example was a demonstration to show the insanity having to guess the build arguments before other tools like Maldua's helper scripts became available around the same time and taking a fairly deep dive into build.pl

Code: Select all

% ./build_zimbra.sh --version 10 --dry-run
#!/bin/sh

git clone --depth 1 --branch "10.0.6" "git@github.com:Zimbra/zm-build.git"
cd zm-build
ENV_CACHE_CLEAR_FLAG=true ./build.pl --ant-options -DskipTests=true --git-default-tag="10.0.7,10.0.6,10.0.5,10.0.4,10.0.2,10.0.1,10.0.0-GA,10.0.0" --build-release-no="10.0.7" --build-type=FOSS --build-release="DAFFODIL" --build-thirdparty-server=files.zimbra.com --no-interactive --build-release-candidate=GA

Ref: https://wiki.zimbra.com/wiki/JDunphy-Co ... mbraScript

Example of believing they will stay disciplined with their FOSS check-ins and simply iterate from a starting argument representing release we want to build for version 10.

Code: Select all

% ./show-me-the-branch.sh 7
Successfully cloned git@github.com:Zimbra/zm-build.git with tag 10.0.6.
_____________ Zimbra build script for Release: 10.0.7 ________________
#!/bin/sh

export PATH=/usr/local/bin:/usr/sbin:/usr/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/home/jad/bin:/usr/sbin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

cd zm-build

# Build the source tree with the specified parameters
ENV_CACHE_CLEAR_FLAG=true ./build.pl --ant-options -DskipTests=true --git-default-tag="10.0.7,10.0.6,10.0.5,10.0.4,10.0.3,10.0.2,10.0.1,10.0.0-GA,10.0.0" --build-release-no="10.0.7" --build-type=FOSS --build-release=DAFFODIL --build-release-candidate=GA --build-thirdparty-server=files.zimbra.com --no-interactive

Disclaimer: I do not run OSS versions in production of Zimbra other than an interested party on how to build these releases and hopefully gain a deeper understanding of what goes into a zimbra release.

These example scripts are possible because of Ian's zimbra-build-helper.sh which establishes the correct build environment and Maldua's tag generator scripts for the first script above.

Jim

[jeastman]

Hi Jim,

You are correct that the build numbers starting with Zimbra 10 should be more sane. Starting (resuming?) with Zimbra 10, the product is adhering to proper semantic versioning. The version number is broken into 3 parts: part 1 is the major version, part 2 is the minor version, part 3 is the patch (... so no more patches of patch versions ...).

The Zimbra engineering team continues to monitor these forums [ when not blocked by 502s... still working on that ] and should be able to provide insights / hints on how to overcome any issues with the build.

[adrian.gibanel.btactic]

You are correct that the build numbers starting with Zimbra 10 should be more sane. Starting (resuming?) with Zimbra 10, the product is adhering to proper semantic versioning. The version number is broken into 3 parts: part 1 is the major version, part 2 is the minor version, part 3 is the patch (... so no more patches of patch versions ...).

1) Well, that's nice to know. It could make more easy to read the different 10.X versions.

2) I am currently encoding the git tag onto the --build-release-candidate switch so that we know where the build has come from:

• 10.1.0-beta: BETAz10x1x0xbetaZacme
• 10.0.7: GAz10x0x7Zacme
• 9.0.0.p36: GAz9x0x0xp36Zacme

I might end up removing the z10x0x7Z bit if tag matches the --build-no bit even if it makes 8.8.x and 9.x builds naming not consistent with 10.x builds naming.

3) As you might have seen I am also experimenting on adding there what I call the builder id, in this example: acme. In other software this is often called as vendor id.

The idea is that when you run zmcontrol -v you then know: Which tag the build has been built from and what company/user has built it.
Otherwise when reporting problems using the FOSS builds you have to show your version plus stating where you downloaded it from.

Any comments about it?
I mean... Would we ever see in the build.pl roadmap any additional switches that later on can be read from zmcontrol -v such as --builder-id or --extra-id ?
I mean, now it sort of works using --build-release-candidate but it should comply with RHEL packages versioning scheme and Debian packages versioning scheme and it's a bit difficult to read sometimes.