ldap database went from 97meg to 86gig

Discuss your pilot or production implementation with other Zimbra admins or our engineers.
User avatar
quanah
Zimbra Alumni
Zimbra Alumni
Posts: 1667
Joined: Fri Sep 12, 2014 10:33 pm
Contact:

ldap database went from 97meg to 86gig

Postby quanah » Thu May 09, 2013 12:56 pm

mdb.prev is a folder created as part of an upgrade process in case the upgrade fails. If the upgrade was successful, then yes, you can delete it.


--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
User avatar
quanah
Zimbra Alumni
Zimbra Alumni
Posts: 1667
Joined: Fri Sep 12, 2014 10:33 pm
Contact:

ldap database went from 97meg to 86gig

Postby quanah » Thu May 09, 2013 12:57 pm

Sure. Lets say you flip a coin, and it comes up heads. And then you decide because of this, it will always come up heads. That's essentially what you are doing. I.e., just because "cp" worked for you this time is no guarantee it'll work for you next time. The only officially supported method for backing up BDB based databases with OpenLDAP (this is not Zimbra specific) is slapcat/slapadd.
There are scenarios in which you can safely copy the BDB DB, which I've covered before.
--Quanah
--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
thesubmitter
Posts: 34
Joined: Fri Sep 12, 2014 11:35 pm

ldap database went from 97meg to 86gig

Postby thesubmitter » Thu May 09, 2013 2:00 pm

[quote user="quanah"]Another note -- in ZCS 8.0.2 and later, when you resize the DB via localconfig, that is all you have to do. As soon as the modifictaion is made to slapd, the data.mdb file will automatically resize to whatever the new maxsize value is. No need to slapcat & slapadd to change the size of the file.[/QUOTE]
This didn't seem accurate for me in 8.0.2 GA OS
acesuares
Posts: 39
Joined: Sat Sep 13, 2014 12:21 am

ldap database went from 97meg to 86gig

Postby acesuares » Tue May 14, 2013 10:48 pm

[quote user="quanah"]Sure. Lets say you flip a coin, and it comes up heads. And then you decide because of this, it will always come up heads.

--Quanah[/QUOTE]
Well, this little change did have a big impact. cold backups that took 4 minutes now took 58 minutes and diskspace on the backup storage was running out fast. Let alone almost one hour downtime.
I followed this convesation with amazement. End good all good but before that, there where some weird paths followed.
It really does make me wonder if Zimbra is really the Open Source Solution that I feel confident with. At that time I choose between Zarafa, Scalix and ZImbra and Zimbra came out by far superior for end users. But then I discovered the cumbersome backup procedures, that have once more bitten me, and the total lack of per-domain administrator possibilities.
For now I am definately stuck with Zimbra, and to be fair, this was the first hickup in over two years, and I do hope Zimbra will evolve as a real open source product. Because I also fear, that without a backing company, something as complex as Zimbra will easily fall apart by lack of a solid open source developer base.
Thank you.
acesuares
Posts: 39
Joined: Sat Sep 13, 2014 12:21 am

ldap database went from 97meg to 86gig

Postby acesuares » Tue May 14, 2013 11:02 pm

[quote user="thesubmitter"]This didn't seem accurate for me in 8.0.2 GA OS[/QUOTE]
I stopped zimbra, changed the values with zmlocalconfig, and nothing happened. Then restarted zimbra and the data.mdb file was immediately resized.
This works for me.
User avatar
quanah
Zimbra Alumni
Zimbra Alumni
Posts: 1667
Joined: Fri Sep 12, 2014 10:33 pm
Contact:

ldap database went from 97meg to 86gig

Postby quanah » Wed May 15, 2013 8:57 am

[quote user="acesuares"]I stopped zimbra, changed the values with zmlocalconfig, and nothing happened. Then restarted zimbra and the data.mdb file was immediately resized.
This works for me.[/QUOTE]
Yes, if you stop zimbra, it won't automatically resize, because for automatic resize to happen, zmconfigd has to be running...
--Quanah
--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
User avatar
quanah
Zimbra Alumni
Zimbra Alumni
Posts: 1667
Joined: Fri Sep 12, 2014 10:33 pm
Contact:

ldap database went from 97meg to 86gig

Postby quanah » Wed May 15, 2013 12:32 pm

[quote user="acesuares"]Well, this little change did have a big impact. cold backups that took 4 minutes now took 58 minutes and diskspace on the backup storage was running out fast. Let alone almost one hour downtime.
I followed this convesation with amazement. End good all good but before that, there where some weird paths followed.
It really does make me wonder if Zimbra is really the Open Source Solution that I feel confident with. At that time I choose between Zarafa, Scalix and ZImbra and Zimbra came out by far superior for end users. But then I discovered the cumbersome backup procedures, that have once more bitten me, and the total lack of per-domain administrator possibilities.
For now I am definately stuck with Zimbra, and to be fair, this was the first hickup in over two years, and I do hope Zimbra will evolve as a real open source product. Because I also fear, that without a backing company, something as complex as Zimbra will easily fall apart by lack of a solid open source developer base.
Thank you.[/QUOTE]
I really don't get what this has to do with Zimbra, really. OpenLDAP is an open source project that has no relation to Zimbra. You implemented a backup solution that was never valid as far as OpenLDAP is concerned. When Zimbra leveraged new technology available from OpenLDAP, you got bit by the fact your backup procedure has been wrong for years. If you had implemented a proper valid backup procedure that was safe to use with OpenLDAP, which is what zimbra NE does, you never would never have run into this issue.
--Quanah
--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
Chicken76
Posts: 21
Joined: Sat Sep 13, 2014 2:28 am

ldap database went from 97meg to 86gig

Postby Chicken76 » Thu May 16, 2013 4:14 am

As a conclusion to this unnecessarily long thread quanah, which method of backing up the LDAP databases would you recommend for ZCS 8.0.3 GA, slapcat or mdb_copy?

Does any of them work if the LDAP server is stopped?
User avatar
quanah
Zimbra Alumni
Zimbra Alumni
Posts: 1667
Joined: Fri Sep 12, 2014 10:33 pm
Contact:

ldap database went from 97meg to 86gig

Postby quanah » Mon Jun 03, 2013 8:29 am

[quote user="Chicken76"]As a conclusion to this unnecessarily long thread quanah, which method of backing up the LDAP databases would you recommend for ZCS 8.0.3 GA, slapcat or mdb_copy?

Does any of them work if the LDAP server is stopped?[/QUOTE]
As noted in the wiki, you can use either method, and both methods work regardless of whether or not slapd is running or stopped.
--Quanah
--
Quanah Gibson-Mount
Product Architect, Symas http://www.symas.com/
OpenLDAP Core team http://www.openldap.org/project/
CISX
Posts: 3
Joined: Sat Sep 13, 2014 3:14 am

ldap database went from 97meg to 86gig

Postby CISX » Thu Jun 13, 2013 6:42 am

[quote user="quanah"]It is provisioned at 80GB because MDB has a maxsize limit. To allow upgrading from an unknown size, I had to choose some number that I was sure would allow anyone to upgrade without issue. I know there is no one who uses zimbra with an 80GB database.

[/QUOTE]
Why was upgrading an unknown size?

IT is an engineering discipline, it should not resolve around guess work!

If you are upgrading from version 8.0.0 to 8.0.1, there is nothing unknown about the database size. It can be found out by many ways. To pick an arbitrary figure that covers all cases is simply poor engineering.

Sadly we are getting use to the "Zimbra" wayÂ… :(

Return to “Administrators”

Who is online

Users browsing this forum: No registered users and 9 guests