hi all,
I have single server deployment of zcs-7.1.1_GA_3196.F11.20110527000857
I had hardware issue with power break and when server booted back, my csfk corrupted these files:
- config db for slapd
- binary files of LDAP
- text files of LDAP log
- text files of LDAP backup
I corrected config db for slapd by copying from config files, that works. but when I tried to start ldap service, I get error from slapd with description about unparseable database file.
What I tried is googling and searchiong here, but all I found was based on ldap backup files or log files, which I can't use.
So what I need is some hint how to rebuild LDAP content or how to delete ldap content and create new one to get it working back.
many thanks to everyone helping..
broken LDAP
broken LDAP
[quote user="jajapaja28"]nobody is able to help?[/QUOTE]
Two suggestions, but they're not going to help you much now. In the future, you should do regular backups. If your database is corrupted, the answer is to restore from a known good backup. That's the best way to do it.
Second, you need a UPS to prevent power interruptions, if you don't already have one.
Hate to tell you this, unless someone here knows some really deep LDAP/database recovery guru-stuff, you'll probably have to reinstall and start all over.
Two suggestions, but they're not going to help you much now. In the future, you should do regular backups. If your database is corrupted, the answer is to restore from a known good backup. That's the best way to do it.
Second, you need a UPS to prevent power interruptions, if you don't already have one.
Hate to tell you this, unless someone here knows some really deep LDAP/database recovery guru-stuff, you'll probably have to reinstall and start all over.
-
jajapaja28
- Posts: 6
- Joined: Sat Sep 13, 2014 2:16 am
broken LDAP
first, thanks for reaction..
[quote]Two suggestions, but they're not going to help you much now. In the future, you should do regular backups. If your database is corrupted, the answer is to restore from a known good backup. That's the best way to do it.[/QUOTE]
of course, but it is too late now.
[quote]
Second, you need a UPS to prevent power interruptions, if you don't already have one.
[/QUOTE]
I have UPS, it was fail of my motherboard..
[quote]
Hate to tell you this, unless someone here knows some really deep LDAP/database recovery guru-stuff, you'll probably have to reinstall and start all over.[/QUOTE]
I see, I have no problem to make new instalation of whole server, question is then how to migrate data (emails, contacts, calendars) to new instalation when old server is not able to be up and I don§t know how to export data
[quote]Two suggestions, but they're not going to help you much now. In the future, you should do regular backups. If your database is corrupted, the answer is to restore from a known good backup. That's the best way to do it.[/QUOTE]
of course, but it is too late now.
[quote]
Second, you need a UPS to prevent power interruptions, if you don't already have one.
[/QUOTE]
I have UPS, it was fail of my motherboard..
[quote]
Hate to tell you this, unless someone here knows some really deep LDAP/database recovery guru-stuff, you'll probably have to reinstall and start all over.[/QUOTE]
I see, I have no problem to make new instalation of whole server, question is then how to migrate data (emails, contacts, calendars) to new instalation when old server is not able to be up and I don§t know how to export data
broken LDAP
[quote user="jajapaja28"]
I see, I have no problem to make new instalation of whole server, question is then how to migrate data (emails, contacts, calendars) to new instalation when old server is not able to be up and I don§t know how to export data[/QUOTE]
That's just it. You can't trust your database, so you couldn't trust the exported data, anyway. When I say you'll have to start over from scratch, that's just what you'll have to do.
Now, if it's just usernames, if you can get LDAP running, you could possibly use an LDAP browser. There are plenty of posts here about LDAP browsing; do a search. But if you want to recover that old email, that's a different matter entirely.
Don't take this the wrong way, because I've been where you are. I suspect that the reason no one else posted a reply is because the answer is obvious: no backup, you have a ton of work to do.
You could possibly hire a data recovery expert to see what he/she could recover. That will be very expensive. But this is just a classic case of, "things blew up and I don't have a backup." That's how it has to be treated.
I see, I have no problem to make new instalation of whole server, question is then how to migrate data (emails, contacts, calendars) to new instalation when old server is not able to be up and I don§t know how to export data[/QUOTE]
That's just it. You can't trust your database, so you couldn't trust the exported data, anyway. When I say you'll have to start over from scratch, that's just what you'll have to do.
Now, if it's just usernames, if you can get LDAP running, you could possibly use an LDAP browser. There are plenty of posts here about LDAP browsing; do a search. But if you want to recover that old email, that's a different matter entirely.
Don't take this the wrong way, because I've been where you are. I suspect that the reason no one else posted a reply is because the answer is obvious: no backup, you have a ton of work to do.
You could possibly hire a data recovery expert to see what he/she could recover. That will be very expensive. But this is just a classic case of, "things blew up and I don't have a backup." That's how it has to be treated.
broken LDAP
Let me explain a little further. Nothing that I say here is anything that you couldn't find in a more detailed search here and elsewhere; but maybe it bears repeating.
First, I emphasize that I've been in your place and I know how you feel. You have my sympathy. But just like a patient with cancer, would rather I lie to you and give you false hope, or get to the bottom line?
The bottom line is that yours is the worst kind of failure: the hardware apparently stopped right in the middle of major server operations. You don't know the state of that data. If it was in the middle of rearranging a bunch of records, or something like that, you will NOT be able to recover much, if anything, even if you use a professional recovery service. You'll STILL have a ton of work to do, because YOU will have to go through that recovered data and decide what's good and what's bad, one record at a time.
Again: I speak from experience. For example, even if you recover some of those calendar records, you won't be able to trust them. They're liable to say that your daughter's birthday party is in 2014 at 73:00AM. See what I'm saying?
And to myself and anyone else who hits this post in the dark, distant future: KEEP BACKUPS. IT'S NOT OPTIONAL.
Bad news all around. But the quickest answer, and the only one I can give you with a clear conscience, is to apologize to your users, build a new server and start entering the data again from a fresh start.
First, I emphasize that I've been in your place and I know how you feel. You have my sympathy. But just like a patient with cancer, would rather I lie to you and give you false hope, or get to the bottom line?
The bottom line is that yours is the worst kind of failure: the hardware apparently stopped right in the middle of major server operations. You don't know the state of that data. If it was in the middle of rearranging a bunch of records, or something like that, you will NOT be able to recover much, if anything, even if you use a professional recovery service. You'll STILL have a ton of work to do, because YOU will have to go through that recovered data and decide what's good and what's bad, one record at a time.
Again: I speak from experience. For example, even if you recover some of those calendar records, you won't be able to trust them. They're liable to say that your daughter's birthday party is in 2014 at 73:00AM. See what I'm saying?
And to myself and anyone else who hits this post in the dark, distant future: KEEP BACKUPS. IT'S NOT OPTIONAL.
Bad news all around. But the quickest answer, and the only one I can give you with a clear conscience, is to apologize to your users, build a new server and start entering the data again from a fresh start.
-
L. Mark Stone
- Ambassador
- Posts: 2800
- Joined: Wed Oct 09, 2013 11:35 am
- Location: Portland, Maine, US
- ZCS/ZD Version: 10.0.7 Network Edition
- Contact:
broken LDAP
[quote]
I SEE, I HAVE NO PROBLEM TO TAKE NEW INSTALATION OF WHOLE SERVER, QUESTION IS THEN HOW TO MIGRATE DATA (EMAILS, CONTACTS, CALENDARS) TO NEW INSTALATION WHEN OLD SERVER IS NOT ABLE TO BE UP AND I DON§T KNOW HOW TO EXPORT DATA[/QUOTE]
IF LDAP IS TOTALLY GONE THEN YOU HAVE NO MAILBOX USER-TO-UID MAPPING. MYSQL STORES THE UID-TO-FILE_DIRECTORY INFORMATION, SO UNLESS YOU HAVE WORKING LDAP, MYSQL AND THE ACTUAL MAIL BLOBS, IT'S IMPOSSIBLE TO RECOVER EMAIL, CONTACTS AND CALENDARS VIA AN EXPORT OF SOME SORT.
NEVERTHELESS, IF ONLY THE MAIL BLOBS ARE INTACT, YOU CAN AT LEAST RECOVER ALL OF THE EMAIL -- BUT IT'S A VERY MANUAL PROCESS:
COPY /OPT/ZIMBRA/STORE SOMEPLACE SAFE.
CREATE A LIST OF ALL OF THE MAILBOXES ON THE SYSTEM IN EXCEL OR A TEXT FILE.
EXPLORE THE SUBDIRECTORIES IN YOUR COPY OF /OPT/ZIMBRA/STORE, OPENING MAILBLOBS IN A TEXT EDITOR TO GET THE RECIPIENT.
ONCE YOU HAVE IDENTIFIED WHICH SUBDIRECTORY BELONGS TO WHICH USER, UPDATE YOUR TEXT FILE OR SPREADSHEET.
COPY THE IDENTIFIED DIRECTORIES TO A NEW TREE, WITH EACH DIRECTORY NAMED AS THE USER (DOESN'T HAVE TO BE EXACT, JUST HUMAN-READABLE).
DEPLOY A COURIER IMAP SERVER AND CONFIGURE USER ACCOUNTS TO MATCH WHAT WAS IN ZIMBRA AND POINT EACH ACCOUNT TO THE RECOVERED DIRECTORIES. COURIER CAN READ ZIMBRA MAILBLOBS DIRECTLY.
DEPLOY THE REPLACEMENT ZIMBRA SERVER AND CREATE THE MAILBOXES.
USING AN EMAIL CLIENT OF YOUR CHOICE, FOR EACH USER'S MAILBOX CONNECT THE EMAIL CLIENT TO BOTH THE NEW ZIMBRA SERVER AND THE COURIER IMAP SERVER, AND THEN DRAG 'N DROP THE EMAILS FROM THE COURIER IMAP SERVER INTO THE ZIMBRA MAILBOX.
WHEN FINISHED YOU CAN DESTROY THE COURIER ACCOUNTS IN THE EMAIL CLIENT AND THEN DESTROY THE COURIER SERVER.
USERS WILL NEED TO RECREATE CONTACTS AND CALENDAR ENTRIES, BUT IF THEY KEPT A LOCAL COPY THEY CAN JUST IMPORT INTO ZIMBRA.
ALL OF THE ZIMBRA SHARING WILL NEED TO BE RECREATED, AS WELL AS ADMIN ACCOUNTS AND OTHER ZIMBRA-SPECIFIC CONFIGURATIONS.
IT IS FOR THESE KINDS OF SITUATIONS THAT COMPANIES PAY FOR THE PROFESSIONAL EDITION, IF ONLY JUST TO GET SUPPORTED BACKUPS (NOT TO MENTION MOBILE INTEGRATION).
IF YOU HAVE A FEW DOZEN MAILBOXES, THE RECOVERY PROCESS IS NOT SO BAD. IF YOU HAVE A FEW THOUSAND MAILBOXES...
ANYWAY, HOPE THAT HELPS, AND BEST OF LUCK WITH YOUR RECOVERY!
MARK
I SEE, I HAVE NO PROBLEM TO TAKE NEW INSTALATION OF WHOLE SERVER, QUESTION IS THEN HOW TO MIGRATE DATA (EMAILS, CONTACTS, CALENDARS) TO NEW INSTALATION WHEN OLD SERVER IS NOT ABLE TO BE UP AND I DON§T KNOW HOW TO EXPORT DATA[/QUOTE]
IF LDAP IS TOTALLY GONE THEN YOU HAVE NO MAILBOX USER-TO-UID MAPPING. MYSQL STORES THE UID-TO-FILE_DIRECTORY INFORMATION, SO UNLESS YOU HAVE WORKING LDAP, MYSQL AND THE ACTUAL MAIL BLOBS, IT'S IMPOSSIBLE TO RECOVER EMAIL, CONTACTS AND CALENDARS VIA AN EXPORT OF SOME SORT.
NEVERTHELESS, IF ONLY THE MAIL BLOBS ARE INTACT, YOU CAN AT LEAST RECOVER ALL OF THE EMAIL -- BUT IT'S A VERY MANUAL PROCESS:
COPY /OPT/ZIMBRA/STORE SOMEPLACE SAFE.
CREATE A LIST OF ALL OF THE MAILBOXES ON THE SYSTEM IN EXCEL OR A TEXT FILE.
EXPLORE THE SUBDIRECTORIES IN YOUR COPY OF /OPT/ZIMBRA/STORE, OPENING MAILBLOBS IN A TEXT EDITOR TO GET THE RECIPIENT.
ONCE YOU HAVE IDENTIFIED WHICH SUBDIRECTORY BELONGS TO WHICH USER, UPDATE YOUR TEXT FILE OR SPREADSHEET.
COPY THE IDENTIFIED DIRECTORIES TO A NEW TREE, WITH EACH DIRECTORY NAMED AS THE USER (DOESN'T HAVE TO BE EXACT, JUST HUMAN-READABLE).
DEPLOY A COURIER IMAP SERVER AND CONFIGURE USER ACCOUNTS TO MATCH WHAT WAS IN ZIMBRA AND POINT EACH ACCOUNT TO THE RECOVERED DIRECTORIES. COURIER CAN READ ZIMBRA MAILBLOBS DIRECTLY.
DEPLOY THE REPLACEMENT ZIMBRA SERVER AND CREATE THE MAILBOXES.
USING AN EMAIL CLIENT OF YOUR CHOICE, FOR EACH USER'S MAILBOX CONNECT THE EMAIL CLIENT TO BOTH THE NEW ZIMBRA SERVER AND THE COURIER IMAP SERVER, AND THEN DRAG 'N DROP THE EMAILS FROM THE COURIER IMAP SERVER INTO THE ZIMBRA MAILBOX.
WHEN FINISHED YOU CAN DESTROY THE COURIER ACCOUNTS IN THE EMAIL CLIENT AND THEN DESTROY THE COURIER SERVER.
USERS WILL NEED TO RECREATE CONTACTS AND CALENDAR ENTRIES, BUT IF THEY KEPT A LOCAL COPY THEY CAN JUST IMPORT INTO ZIMBRA.
ALL OF THE ZIMBRA SHARING WILL NEED TO BE RECREATED, AS WELL AS ADMIN ACCOUNTS AND OTHER ZIMBRA-SPECIFIC CONFIGURATIONS.
IT IS FOR THESE KINDS OF SITUATIONS THAT COMPANIES PAY FOR THE PROFESSIONAL EDITION, IF ONLY JUST TO GET SUPPORTED BACKUPS (NOT TO MENTION MOBILE INTEGRATION).
IF YOU HAVE A FEW DOZEN MAILBOXES, THE RECOVERY PROCESS IS NOT SO BAD. IF YOU HAVE A FEW THOUSAND MAILBOXES...
ANYWAY, HOPE THAT HELPS, AND BEST OF LUCK WITH YOUR RECOVERY!
MARK
___________________________________
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
AWS Certified Solutions Architect-Associate
L. Mark Stone
Mission Critical Email - Zimbra VAR/BSP/Training Partner https://www.missioncriticalemail.com/
AWS Certified Solutions Architect-Associate
-
jajapaja28
- Posts: 6
- Joined: Sat Sep 13, 2014 2:16 am
broken LDAP
[quote]
First, I emphasize that I've been in your place and I know how you feel. You have my sympathy.
[/QUOTE]
thanks for that!
[quote]
The bottom line is that yours is the worst kind of failure: the hardware apparently stopped right in the middle of major server operations. You don't know the state of that data. If it was in the middle of rearranging a bunch of records, or something like that, you will NOT be able to recover much, if anything, even if you use a professional recovery service. You'll STILL have a ton of work to do, because YOU will have to go through that recovered data and decide what's good and what's bad, one record at a time.
[/QUOTE]
have you ever heard about transactional access? I have no problem with mysql data, only LDAP database is stored in binary file! binary! OMG..
[quote]
Again: I speak from experience. For example, even if you recover some of those calendar records, you won't be able to trust them. They're liable to say that your daughter's birthday party is in 2014 at 73:00AM. See what I'm saying?
[/QUOTE]
this is general problem, do you trust your data whenever you need it? bad way. The same for car navigation, are you still driving ahead when your navi says that? of course not.. so this is tool only and I'm using it as a tool, everybody should accept that tis CAN FAIL FOR SOME REASON.
[quote]
And to myself and anyone else who hits this post in the dark, distant future: KEEP BACKUPS. IT'S NOT OPTIONAL.
[/QUOTE]
Sure, I know. Next level of your sentence is - keep backup on different storage/filesystem.
[quote]
Bad news all around. But the quickest answer, and the only one I can give you with a clear conscience, is to apologize to your users, build a new server and start entering the data again from a fresh start.[/QUOTE]
My idea is somehow create new LDAP database and join new records to some IDs of zimbra backend. Of course I will make completely new instance when it is not reasonable way..
First, I emphasize that I've been in your place and I know how you feel. You have my sympathy.
[/QUOTE]
thanks for that!
[quote]
The bottom line is that yours is the worst kind of failure: the hardware apparently stopped right in the middle of major server operations. You don't know the state of that data. If it was in the middle of rearranging a bunch of records, or something like that, you will NOT be able to recover much, if anything, even if you use a professional recovery service. You'll STILL have a ton of work to do, because YOU will have to go through that recovered data and decide what's good and what's bad, one record at a time.
[/QUOTE]
have you ever heard about transactional access? I have no problem with mysql data, only LDAP database is stored in binary file! binary! OMG..
[quote]
Again: I speak from experience. For example, even if you recover some of those calendar records, you won't be able to trust them. They're liable to say that your daughter's birthday party is in 2014 at 73:00AM. See what I'm saying?
[/QUOTE]
this is general problem, do you trust your data whenever you need it? bad way. The same for car navigation, are you still driving ahead when your navi says that? of course not.. so this is tool only and I'm using it as a tool, everybody should accept that tis CAN FAIL FOR SOME REASON.
[quote]
And to myself and anyone else who hits this post in the dark, distant future: KEEP BACKUPS. IT'S NOT OPTIONAL.
[/QUOTE]
Sure, I know. Next level of your sentence is - keep backup on different storage/filesystem.
[quote]
Bad news all around. But the quickest answer, and the only one I can give you with a clear conscience, is to apologize to your users, build a new server and start entering the data again from a fresh start.[/QUOTE]
My idea is somehow create new LDAP database and join new records to some IDs of zimbra backend. Of course I will make completely new instance when it is not reasonable way..
-
jajapaja28
- Posts: 6
- Joined: Sat Sep 13, 2014 2:16 am
broken LDAP
[quote user="LMStone"]If LDAP is totally gone then you have no mailbox user-to-UID mapping. MySQL stores the UID-to-file_directory information, so unless you have working LDAP, MySQL and the actual mail blobs, it's impossible to recover email, contacts and calendars via an export of some sort.
Nevertheless, if only the mail blobs are intact, you can at least recover all of the email -- but it's a very manual process:
[/QUOTE]
Thanks Mark for this, it is usable for emails only, right? what about contacts, calendars?
Nevertheless, if only the mail blobs are intact, you can at least recover all of the email -- but it's a very manual process:
[/QUOTE]
Thanks Mark for this, it is usable for emails only, right? what about contacts, calendars?
-
jajapaja28
- Posts: 6
- Joined: Sat Sep 13, 2014 2:16 am
broken LDAP
and for better idea about how big it is.. this is private server for about 5 domains with few users and about 5GB total mailboxes/calendars/tasks/contacts size.
And because I have only few users (plus some system accounts), it should not be complicated to create new ldap database with similar entries and remap them to other storages.. but what I miss here is knowhow about ldap structure, system accounts and joining to other internal components.
And because I have only few users (plus some system accounts), it should not be complicated to create new ldap database with similar entries and remap them to other storages.. but what I miss here is knowhow about ldap structure, system accounts and joining to other internal components.