Hi folks,
I'm really getting fed up w/ your way of doing security "patches".
Things like this:
We">https://www.zimbra.com/forums/announcem ... ility.html
We already had the same mess w/ the heartbleed fix. A script which downloads some tarball and replaces individual files,
completely outside the package manager, so defeating the whole purpose of package management.
First of all, that script is totally insecure. Okay, you've now learned to use https instead, but w/ explicitly switching off
certificate check, this becomes nothing but a bad joke. Oh, and why that md5 file ? I hope you dont think it should
give you more security, because it does NOT.
You really wanna invite the NSA to abuse your "patching" for injecting backdoors ?!
And now for the approach of replacing single files, instead of just shipping new packages:
Do you understand what the concept of package management it is meant for ? Because I really doubt that
Package management isn't just a fancy way for transporting bundles of files to some target system, it is
the *CORE* of software deployment on modern *nix platforms, which operators use all the day.. The main
purpose is CONSISTENCY and easy tracking of installed files and package versions (of course, including
_fully automatic_ checking for available packages and updates, downloads, signature checking, deployment,
and removal, etc).
And it also adds another - fully automatic - security layer, as the package manager checks package signatures
against the repo and validates the repo against the authorized keys. If done correctly (which is pretty easy),
it automatically makes sure that only properly authenticated packages are installed (assuming you've told
it to trust the repo in question).
Another _vital_ aspect:
Operators usually don't have the time for reading tons upgade notifications on a daily basis and run upgrades
for dozens of packages on each single machine manually. That's what we have distros for (and for professional
commercial software, we just expect the vendor to do his homework properly!). And when it comes to security
fixes, that's what mechanisms like 'unattended-upgrades' are for, which deploy security fixes fully-automatically,
w/o having to wait for an operator to trigger the upgrade manually.
This is a pretty typical problem of many commercial vendors (also seen it at a lot of my clients - many even don't
know how to write proper Makefiles or use a decent SCM): they might produce pretty good software, but
deployment tends to be a total mess.
This mess is getting worse and worse over the years (well, I'm pretty curious how the situation becomes w/ JP,
haven't got access to the source for a review yet ...), and it _really_ should be cleaned up soon, otherwise I
see even more people switching to other (really opensource!) solutions like Kolab !
I can understand that such fundamental infrastructure things don't easily fit into product plans, and hilevel
decision makes usually don't know anything about that. But these things are really importan for stability
as well as TCO. It's like trying to build a skyscraper on thin sand ... either it's extremly expensive or it
will fall apart sooner or later ... ;-o
In case you're lacking proper resources (which is my _strong_ impression for years now), don't hesistate
to call me. I can arrange a whole team w/ very deep Zimbra knowledge (we're probably the ones who
do the deepest / most complex Zimbra customizations in whole Europe, if not world wide). Besides the
architect, who eg. developed zmpkg (which would by myself :p), we have a whole team of very
Zimbra-experienced developers, several system integrators / operators, lots of testers, etc - everything
that's needed to roll the whole show on our own
After all these years, I still belive that Zimbra has great potential, but it needs a change in development
management approaches. The mentioned deployment issues is just one part, the lack of a _real_
OSS community is another one (I guess, I already mentioned the problems w/ your SCM infrastructure
and workflows, which is really expensive ;-o)
cu
Horrible security "patching"
-
- Advanced member
- Posts: 75
- Joined: Sat Sep 13, 2014 2:29 am
Horrible security "patching"
Instead of ranting in the forums you'd be far better providing a well described description and possible solution to the 'problem' in bugzilla, possibly an RFE.
Horrible security "patching"
I agree with Bill here. Zimbra being an open source product, we would love to leverage the experience and commitment of the community for the benefit of everyone. Obviously 'metux' has good ideas and passion for improvement, so I would like to work with these ideas if anyone is willing to start a discussion and help with implementation.
Horrible security "patching"
This is already tracked under https://bugzilla.zimbra.com/show_bug.cgi?id=87788
--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
-
- Advanced member
- Posts: 75
- Joined: Sat Sep 13, 2014 2:29 am
Horrible security "patching"
[quote user="10330phoenix"]Instead of ranting in the forums you'd be far better providing a well described description and possible solution to the 'problem' in bugzilla, possibly an RFE.[/QUOTE]
Also did so quite some time ago.
There's not much to explain, as it's pretty trivial and obvious:
in case of such an hotfix case occours, fork off at the latest release, fix the bug, then rebuild and publish that.
And, of course, for automatic updates (which should be the standard for such immediate hotfixes),
put the packages into an repo and tell the world it's url, so package manager and usual tools
like `unattended-upgrades` (which every decent operator activates) will automatically handle
the security fix upgrade.
What is so complicated here ?
Also did so quite some time ago.
There's not much to explain, as it's pretty trivial and obvious:
in case of such an hotfix case occours, fork off at the latest release, fix the bug, then rebuild and publish that.
And, of course, for automatic updates (which should be the standard for such immediate hotfixes),
put the packages into an repo and tell the world it's url, so package manager and usual tools
like `unattended-upgrades` (which every decent operator activates) will automatically handle
the security fix upgrade.
What is so complicated here ?
Horrible security "patching"
As you yourself note, you've already asked this, and it has been answered, in the bug. It's also been answered offline for you in direct email as well. I don't see much point to your posting here other than some need to grandstand. But just in case a 3rd time is a charm:
The installer is monolithic and ancient (virtually unchanged since about 2005). It's not currently possible with the way it is designed to use repositories. This has already been identified as an issue that needs immediate attention, as noted in https://bugzilla.zimbra.com/show_bug.cgi?id=87788. The plan is to rework the installer as part of the ZCS 8.6 release. One issue using repositories does not address is our clients that have servers with no external internet access. The current installer resolves this by including everything it needs as part of the package. I.e., we are going to likely still need to bundle a "complete" installer vs one that relies on repositories. Either way, you are already quite aware of the fact that Zimbra knows this is an issue, and that Zimbra already has plans to address it. Ranting on the forums does nothing in terms of resolving an already identified issue.
--Quanah
The installer is monolithic and ancient (virtually unchanged since about 2005). It's not currently possible with the way it is designed to use repositories. This has already been identified as an issue that needs immediate attention, as noted in https://bugzilla.zimbra.com/show_bug.cgi?id=87788. The plan is to rework the installer as part of the ZCS 8.6 release. One issue using repositories does not address is our clients that have servers with no external internet access. The current installer resolves this by including everything it needs as part of the package. I.e., we are going to likely still need to bundle a "complete" installer vs one that relies on repositories. Either way, you are already quite aware of the fact that Zimbra knows this is an issue, and that Zimbra already has plans to address it. Ranting on the forums does nothing in terms of resolving an already identified issue.
--Quanah
--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
-
- Advanced member
- Posts: 75
- Joined: Sat Sep 13, 2014 2:29 am
Horrible security "patching"
[quote user="quanah"]As you yourself note, you've already asked this, and it has been answered, in the bug.
[/QUOTE]
Not completely answered.
But that's not the main issue here. You already have a build system, which creates packages.
Even it's not yet suited for initial deployment or larger upgrades (like from helix to ironmaiden),
but certainly enough for such relatively small things.
In this case, you just had to run a usual rebuild and publish the packages to some repo,
and point people to it. And at this point you also optionally could even have added your repo url
and key to the system.
Having to run the installer on initial deployments and major upgrades, is still acceptable.
But such security-fixes should really run automatically.
Instead of writing a script that downloads some tarball and directly unpacks it completely
outside the package management, you could instead at least have it download upgraded
.deb/.rpm and install it - this at least would have left the package manager database
in an consistent state.
> It's not currently possible with the way it is designed to use repositories.
Simply put the upgraded .deb's / .rpm's in a repository and point people to it.
The installer doesn't have much to do with that. (in the case in question, it didn't
even have to be run)
After that point, the actual packages can be removed from the installer tarball - the installer
then just needs to add the repositories, and change certain calls (eg. on Debian, call apt
instead of dpkg).
Yet again: I'm currently just talking about upgrades within an major release line.
[QUOTE]
The plan is to rework the installer as part of the ZCS 8.6 release.
[/QUOTE]
So, in several years ? ;-o
[QUOTE]
One issue using repositories does not address is our clients that have servers with no external internet access.
[/QUOTE]
Mirroring package repositories is rather trivial. Can be done w/ some lines of shell script.
Same for automatically generating an installer tarball that includes the repo mirror.
(local mirrors aren't that unusual, in fact quite standard for decades, eg. w/ CD installations)
[QUOTE]
I.e., we are going to likely still need to bundle a "complete" installer vs one that relies on repositories.
[/QUOTE]
Pretty trivial, can be automatically generated, pretty easily.
[/QUOTE]
Not completely answered.
But that's not the main issue here. You already have a build system, which creates packages.
Even it's not yet suited for initial deployment or larger upgrades (like from helix to ironmaiden),
but certainly enough for such relatively small things.
In this case, you just had to run a usual rebuild and publish the packages to some repo,
and point people to it. And at this point you also optionally could even have added your repo url
and key to the system.
Having to run the installer on initial deployments and major upgrades, is still acceptable.
But such security-fixes should really run automatically.
Instead of writing a script that downloads some tarball and directly unpacks it completely
outside the package management, you could instead at least have it download upgraded
.deb/.rpm and install it - this at least would have left the package manager database
in an consistent state.
> It's not currently possible with the way it is designed to use repositories.
Simply put the upgraded .deb's / .rpm's in a repository and point people to it.
The installer doesn't have much to do with that. (in the case in question, it didn't
even have to be run)
After that point, the actual packages can be removed from the installer tarball - the installer
then just needs to add the repositories, and change certain calls (eg. on Debian, call apt
instead of dpkg).
Yet again: I'm currently just talking about upgrades within an major release line.
[QUOTE]
The plan is to rework the installer as part of the ZCS 8.6 release.
[/QUOTE]
So, in several years ? ;-o
[QUOTE]
One issue using repositories does not address is our clients that have servers with no external internet access.
[/QUOTE]
Mirroring package repositories is rather trivial. Can be done w/ some lines of shell script.
Same for automatically generating an installer tarball that includes the repo mirror.
(local mirrors aren't that unusual, in fact quite standard for decades, eg. w/ CD installations)
[QUOTE]
I.e., we are going to likely still need to bundle a "complete" installer vs one that relies on repositories.
[/QUOTE]
Pretty trivial, can be automatically generated, pretty easily.
Horrible security "patching"
It would take distributing zimbra-core, which is a massive package full of a million different things. Installing a new version of this on top of an existing install could wreak havoc for customers that apply their own custom patches on top of what Zimbra ships (and there are plenty that do that). It is not a workable solution. Again, the current system is perfect, but it also isolates the the update to what's required without touching large portions of the product unnecessarily.
--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
-
- Advanced member
- Posts: 75
- Joined: Sat Sep 13, 2014 2:29 am
Horrible security "patching"
> It would take distributing zimbra-core, which is a massive package full of a million different things.
In case of such an security fix, most of these things would remain the same. Where's the problem ?
> Installing a new version of this on top of an existing install could wreak havoc for customers that
> apply their own custom patches on top of what Zimbra ships (and there are plenty that do that).
If course, they'll have to rebase their customizations onto the new version (which in that case
would be an no-op).
By the way: we're one of these people doing pretty many core customizations.
And we have pretty much automatized the whole process. (on minor upgrades, it's just a
matter of a few minutes manual stuff)
Oh, and why havent't you split it already in several smaller packages ?
Even with the horribly broken IronMaiden buildsystem, it's not that difficult (once you've got
the build system working at all ;-o). You're already building several packages - just add
add another one ...
EDIT: with some package manager magic, you could even split it out within an minor
upgrade - without overwriting the other stuff ...
In case of such an security fix, most of these things would remain the same. Where's the problem ?
> Installing a new version of this on top of an existing install could wreak havoc for customers that
> apply their own custom patches on top of what Zimbra ships (and there are plenty that do that).
If course, they'll have to rebase their customizations onto the new version (which in that case
would be an no-op).
By the way: we're one of these people doing pretty many core customizations.
And we have pretty much automatized the whole process. (on minor upgrades, it's just a
matter of a few minutes manual stuff)
Oh, and why havent't you split it already in several smaller packages ?
Even with the horribly broken IronMaiden buildsystem, it's not that difficult (once you've got
the build system working at all ;-o). You're already building several packages - just add
add another one ...
EDIT: with some package manager magic, you could even split it out within an minor
upgrade - without overwriting the other stuff ...