Zimbra 9 - FOSS

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
bulletxt
Advanced member
Advanced member
Posts: 81
Joined: Sat Sep 13, 2014 1:08 am

Re: Zimbra 9 - FOSS

Post by bulletxt »

ianw1974 wrote:
bmunger wrote:
ianw1974 wrote:Hi,
I'm unsure to really understand the implications. Is this going to be a path for upgrades once the support for 8.8.15 from Synacor expires?

I'm running a small server at home with only my family using accounts so far and do not want to subscribe to a network edition for home use. I used to have a simple email server before Zimbra and can return to that with other package(s) to complement, but I'd rather stick to the same path and avoid all the config headaches.

If I understand, you are alone, no support from the programmers' community to build Zimbra 9 binaries out of the source code?

It seems odd that there is no gathering of forces to branch an open source Zimbra for the future...

It looked really busy on the forums and I'm surprised that this Open Source project goes to a dead end...

Bernard
Check Zimbra EOL here: https://www.zimbra.com/support/support- ... lifecycle/ both 8.8.15 and 9.0 end on the same date. That said, Zimbra 10 shows as being available, although there are no Network Edition downloads for it yet, so looks like perhaps it hasn't been released yet. Either way, it should also be possible to build that was well sometime soon.

There are other versions, Zextras has Carbonio Community Edition which has a team of people behind it. My builds were purely to offer Zimbra as it was previously when Zimbra provided the OSE/FOSS edition. I have no wish to fork it, rebrand it or whatever like Zextras have done since I am just one person and I simply don't have the time for such a workload. How long my builds are available depends on the Zimbra Build repositories being accessible, and versions being published via that. If that stops, then my builds stop also. People can also use my scripts and build for themselves, but again, it relies on the Zimbra build repositories for it to work.

There are quite a few options, depending on how you want to proceed, be it a paid version of Zimbra, Zextras Carbonio, or one of my Zimbra OSE-type builds. There might even be other people doing the same that I personally don't know about.

Hi, what is the situation about Zimbra 10? Are you still able to build the OSS version?
User avatar
ianw1974
Outstanding Member
Outstanding Member
Posts: 229
Joined: Sat Sep 13, 2014 12:45 am
Location: UK and Poland
Contact:

Re: Zimbra 9 - FOSS

Post by ianw1974 »

Will soon see, just attempting a build now.
User avatar
ianw1974
Outstanding Member
Outstanding Member
Posts: 229
Joined: Sat Sep 13, 2014 12:45 am
Location: UK and Poland
Contact:

Re: Zimbra 9 - FOSS

Post by ianw1974 »

Zimbra 10 builds now available on my site.
phoenix
Ambassador
Ambassador
Posts: 27272
Joined: Fri Sep 12, 2014 9:56 pm
Location: Liverpool, England

Re: Zimbra 9 - FOSS

Post by phoenix »

Hi Ian

Thanks for your builds, I'll have to give it a go when I've a bit of spare time. :)
Regards

Bill

Rspamd: A high performance spamassassin replacement

Per ardua ad astra
bulletxt
Advanced member
Advanced member
Posts: 81
Joined: Sat Sep 13, 2014 1:08 am

Re: Zimbra 9 - FOSS

Post by bulletxt »

ianw1974 wrote:Zimbra 10 builds now available on my site.
Great. Good to know this is still possibile. So zimbra 10 still has "classic" interface
User avatar
ianw1974
Outstanding Member
Outstanding Member
Posts: 229
Joined: Sat Sep 13, 2014 12:45 am
Location: UK and Poland
Contact:

Re: Zimbra 9 - FOSS

Post by ianw1974 »

Yes, it's the same as what Zimbra would have provided, like they did when they made OSE builds for Zimbra 8. No Modern UI, since that's in Network Edition.
User avatar
JDunphy
Outstanding Member
Outstanding Member
Posts: 889
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: 9.0.0_P39 NETWORK Edition

Re: Zimbra 9 - FOSS

Post by JDunphy »

Hi Ian,

Ran your script on Oracle 8 for 8.8.15 with no problems this afternoon. You have made the process very simple. Excellent job!

My very high level of understanding of the process is the following:

* update config.build
* run your build program
* We are left with a tar image that we can extract for 8.8.15 or 9 or 10 depending on what was in config.build. That worked REALLY well.
* once extracted, we do our normal install.sh that came from this build tar image.
* that in turn brings in some 3rd party builds from a zimbra repository that we did not build. examples would be nginx, clamav, etc.
* patches might also be applied again by zimbra as the final part of the update process running install.sh or manually

Is that your understanding? I am a little confused how patches are handled unless we are always pulling the latest development branch. For 8.8.15 and 9 that is probably ok but I wonder about 10.

Code: Select all

% cat config.build
# Zimbra 8 Build Info
######### config.build
BUILD_NO                = 3953
BUILD_RELEASE           = JOULE
BUILD_RELEASE_NO        = 8.8.15
BUILD_RELEASE_CANDIDATE = GA
BUILD_TYPE              = FOSS
BUILD_THIRDPARTY_SERVER = files.zimbra.com
INTERACTIVE             = 0
The 3rd party build is where I am having some difficulty.

I am attempting to build nginx where they appear to have their own copy in addition to a zimbra extension... I am building a modsecurity 3+ dynamic module (ngx_http_modsecurity_module.so) for nginx but ideally need the nginx tree to build that from. I failed on the 3rd party build following along with their Zimbra build instructions but was able to use their provided spec file, grab the exact same version of nginx source and finally build a module that I could load with their provided zimbra nginx.

While that works, I would like to build the entire system including all 3rd party modules and come up with a scheme to automate how we handle patches ourselves vs running install.sh and perhaps building individual components should we find that we need a newer version of clamav for example.

Have you ever built any of the 3rd party modules or have any ideas?

Jim
User avatar
ianw1974
Outstanding Member
Outstanding Member
Posts: 229
Joined: Sat Sep 13, 2014 12:45 am
Location: UK and Poland
Contact:

Re: Zimbra 9 - FOSS

Post by ianw1974 »

Hi Jim,

To be honest, I haven't tried. Someone did ask me once if I could provide a script for building those components (https://github.com/ianw1974/zimbra-buil ... /issues/14) but I just don't have the time to delve that deeply into it and try to figure out what all the dependencies are that need to be installed. There's like 185-190 packages, and the dependencies for each would take a fair amount of time to figure out. There is an example though on the link in that issue back to the Zimbra third-party repository which does show at least how to build nginx. Although I'm guessing you probably saw that already though. In the thirdparty directory on that repo linked in the issue, it could just be a case of doing similar to the nginx example, and loop through each of the packages to build. Obviously the first time round is going to be a bit time-consuming as you'd need to work through each package, figure out what the dependencies are, install them, then build it, and repeat if some other deps are missing. But again, there is a lack of info, since it only gives info for the nginx build. I believe I did get the nginx build to work, but started getting a dependency nightmare for all the others. If I ever get a huge amount of spare time to work on it I might try, but I'm somewhat time-limited at the moment so not something I'm able to take on right now.

My scripts tend to build the latest from what is available on the develop branch. I guess in most cases this is sufficient for most peoples needs, but the special cases for other integrations that require recompiling to integrate modules is kind of outside what I was trying to achieve.

My builds were to replace the hole that was left when the ZCS builds were no longer made by Synacor.

I have now created like config.build.9 and config.build.10 so it can be simplified by copying that to config.build for whichever version you are building. I mainly created these because I wanted to use them in the Dockerfile for building Zimbra with docker/podman. I also use these on my build system, so that I can build all versions that I provide for download, it means I can just literally loop through the distributions, and with the supplied version, build all of them. Then repeat for the next version. Helps free up a bit of my time :)
User avatar
jeastman
Zimbra Employee
Zimbra Employee
Posts: 82
Joined: Tue Mar 29, 2016 1:36 pm

Re: Zimbra 9 - FOSS

Post by jeastman »

Hi Jim and Ian,

I've been working with the team to break out the 3rd party components from the Zimbra build system as much as possible. This will take a little bit of time, but we are beginning the process. This should make it much easier for a complete build.

The idea is to ensure we have each 3rd party component broken out into its own "project", with its own lifecycle. The intent is to stop building the 3rd party components we do not have to (rely on the OS distribution where possible). This should help to us out of being a bottleneck for security updates, minor patches, etc. For those components where we have some kind of customization (patch, build configuration, etc.), the idea will be to host the modified version and provide packaging. There is a long history where Zimbra build everything associated with the product and simply laid down binary versions of it all. We began getting out of that process (sometime around 8.7, I believe), but the full work was never completed. I'm attempting to get us to finally complete that work.

Ultimately, this would provide us with a means to distribute the 3rd party components independently of the Zimbra Product, meaning if someone wanted to make use of just the Zimbra-modified version of Nginx, for instance, they could. It would also mean that you would not need to independently build those 3rd party packages as part of the Zimbra build (unless you wanted to do so).

As Ian mentioned, it is a lot of packages and I don't expect we will change it all overnight, but you should see some incremental changes here. In the mean time, the dependencies are already worked out for you if you care to build everything. Check the https://github.com/Zimbra/packages project for the build of these components. Each component in the "thirdparty" folder has a makefile, associated patches, and a package spec file (deb or RPM) which lays out everything needed to build that component.

Please let me know if you have additional questions or if I can be of further assistance.
John Eastman
User avatar
JDunphy
Outstanding Member
Outstanding Member
Posts: 889
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: 9.0.0_P39 NETWORK Edition

Re: Zimbra 9 - FOSS

Post by JDunphy »

Thanks for the reply Ian and John,

That is really encouraging news about where we are headed.

I was following this documentation how to build it from this link: https://github.com/Zimbra/packages

There was a missing and critical step in the documentation which was make getsrc before make build.

For others interested. Ian's script does most of the work. make build does bring in the dependencies which was zimbra-base zimbra-openssl-devel zimbra-cyrus-sasl-devel so it is as simple as this if you have built the FOSS tar image with his script:

Code: Select all

% cd /home/git/zimbra
% git clone https://github.com/zimbra/zimbra-package-stub
% git clone https://github.com/zimbra/packages.git
% cd packages
% git submodule update --init --recursive --remote
% cd packages/thirdparty/nginx
% make getsrc   # should be added to https://github.com/Zimbra/packages
% make build
% ls
build  docs  extras  Makefile  nginx  patches  README  zimbra-nginx  zmmodules
% tar tvf build/RHEL8_64/src/zimbra-nginx-1.20.0.tar.gz
Amazing how something as simple as building a tar image for FOSS and then building nginx gives me new hope that we can enhance this to ride out active attacks before patching is available.

Why would we want to do this building and/or extend supplied zimbra's nginx modules? Example rule from GBT-4 showing possible concept but not tested yet.

Code: Select all

SecRule REQUEST_URI|REQUEST_HEADERS|REQUEST_BODY "(mboximport|wp-login)" \
    "id:12347,phase:2,t:block,log,deny,status:403,\
    msg:'Blocked request: %{MATCHED_VAR}', \
    chain, \
    setenv:ip.blocked=1, \
    exec:/usr/bin/ipset add blacklist24hr %{REMOTE_ADDR}"
    
Given this could have been dynamically loaded without a poxy restart when the rule file changed with modSecurity 3. The community could share rules and/or Zimbra could monetize vetted and trusted rules as part of a nightly update similarly to how SA rules are updated. That would provide another layer to safeguard against active attacks against zimbra before patches are released or tested and help those stuck on older version before then can upgrade to supported releases. Another idea is to put up a captcha to slow bots down that have not authenticated with suspicious activity against mailboxd.
Post Reply