BackupNG fails on restore

Forum to discuss, report issues, or provide feedback about Zimbra Admin Plus
User avatar
JDunphy
Outstanding Member
Outstanding Member
Posts: 889
Joined: Fri Sep 12, 2014 11:18 pm
Location: Victoria, BC
ZCS/ZD Version: 9.0.0_P39 NETWORK Edition

Re: BackupNG fails on restore

Post by JDunphy »

I decided to look at some source I had here to see how it was calculated and why you didn't get a bigger value. It's not very sophisticated. :-)

Code: Select all

sub mailboxdMemoryMB {
  my $system_mem = shift;
  my $memory;
  if ($system_mem > 16) {
    $memory = 0.2*$system_mem;
  } else {
    $memory = 0.25*$system_mem;
  }
  return int($memory*1024);
}

sub getSystemMemory {
  my $os = lc qx(uname -s);
  chomp($os);
  return "unknown" unless $os;
  my $mem;
  if ($os eq "linux") {
    $mem = qx(cat /proc/meminfo | grep ^MemTotal: | awk '{print \$2}');
    chomp($mem);
    $mem = sprintf "%0.1f", $mem/(1024*1024);
  } elsif ($os eq "darwin") {
    $mem = qx(sysctl hw.memsize | awk '{print \$NF}');
    chomp($mem);
    $mem = sprintf "%0.1f", $mem/(1024*1024*1024);
  }
  return $mem;
}
$config{SYSTEMMEMORY} = getSystemMemory();
$config{MAILBOXDMEMORY} = mailboxdMemoryMB($config{SYSTEMMEMORY});
Breaking this down ... it appears this is what we have left.

Code: Select all

#!/usr/bin/perl

$mem = qx(cat /proc/meminfo | grep ^MemTotal: | awk '{print \$2}');
chomp($mem);
$mem = sprintf "%0.1f", $mem/(1024*1024);
print "mem is $mem\n";

print 0.25*$mem*1024 . "\n";
You can run this and see how close it gets to your 2048 for 8GB allocation.
Perhaps it is different now with 8.8+ but once set, its up to the user to update the value.

PS. I wonder if there is more granularity for larger memory (ie. 16GB+) than 0.2*$mem*1024 :-)
yellowhousejake
Advanced member
Advanced member
Posts: 129
Joined: Tue Sep 09, 2014 9:57 am
ZCS/ZD Version: Release 10.0.1.GA.4518.UBUNTU20_64

Re: BackupNG fails on restore

Post by yellowhousejake »

Other projects came to the front and this had to take a backseat, I am back to working with this now.

JDunphy, here is the output of your script.

zimbra@zimbra2:~$ perl memtest.pl
mem is 7.8
1996.8

DAve
yellowhousejake
Advanced member
Advanced member
Posts: 129
Joined: Tue Sep 09, 2014 9:57 am
ZCS/ZD Version: Release 10.0.1.GA.4518.UBUNTU20_64

Re: BackupNG fails on restore

Post by yellowhousejake »

Thank you everyone, I believe we are were we need to be now with the migration. I am still running into some issues with the installation, but that is for another thread on another board. For now, the output of the successful migration backup import is below. I see nothing in there that alarms me, and checks of several accounts seem to have gone well.

I intend to remove and reinstall Zimbra, then do another full backup import again. I want to have a solid process down and tested without error before doing the production migration.

Unless someone spots an issue in the import notification below, I think we are good.

DAve

----------------------------------------------------------------------------------------------------------------------------------------------------

This is an automated notification from ZxBackup about External Restore.

Operation External Restore Completed.

Operation Id: ab1ed3f2-9fd5-4f9b-b9df-c69442dada28
Operation Host: zimbra2.XXXXX.org
Operation Log path: /opt/zimbra/log/op_ExternalRestore_ab1ed3f2-9fd5-4f9b-b9df-c69442dada28.log
Operation Start Time: 28/11/2018 10:16:07
Operation Duration: 14 hours, 17 minutes, 39 seconds

Operation requested by: admin@zimbra2.XXXXX.org

Network Modules NG Version: 2.9.0
commit: 423b5076c550c71168b23f8b8cc649363bea2d78

Zal Version: 2.4.0
Zal commit: 93807f76531868754de673a94725162b2b2a0edc
Zimbra version: 8.8.9_GA_3019 20180809160254 20180823-0614 NETWORK


Warning: the following admin accounts have been restored: dgoodrich@XXXXX.org, pbyerley@XXXXX.org, nriedman@XXXXX.org, rsouchon@XXXXX.org, admin@zimbra.XXXXX.org
Warning: Backup detected a potential issue with the following items (digest mismatch).
Account adehoney@XXXXX.org, items id: 6368, 307, 4103
Account mschwamberger@XXXXX.org, items id: 25827
Account igalbraith@XXXXX.org, items id: 2211
Account mripley@XXXXX.org, items id: 4442, 5722, 5839
Account efields@XXXXX.org, items id: 6891
Account lelmore@XXXXX.org, items id: 38033, 32474
Account nwright@XXXXX.org, items id: 2678
Account rsouchon@XXXXX.org, items id: 188060
Account jschroeder@XXXXX.org, items id: 32388, 32389, 33405
Account iernst@XXXXX.org, items id: 3003
Account bpetrey@XXXXX.org, items id: 612
Account ngilbert@XXXXX.org, items id: 45864, 5554, 29190
Account mfruth@XXXXX.org, items id: 64737, 70856, 262582, 196952, 181982, 187999
Account jwertman@XXXXX.org, items id: 36292, 36956, 1915
While such items have been restored, a manual check is highly suggested in order to verify the correctness of the items themselves .


Warnings: 5
Errors: 0


- stats -
Restored Items: 611179
Skipped Items: 1474946
Unrestored Items: 2

- ignored ldap attributes -
zimbraDomainName, zimbraId, zimbraDomainType, zimbraCreateTimestamp

- unrestorable items -
For more informations about unrestorable items see

account: ghall@XXXXX.org
unrestored items list: 20648,20650

- debug info -
-- parameters --
skipAliases = false
origin = SOAP
filterDeleteAccount = true
humanDate = last available
accountNameList = adehoney@XXXXX.org, admin@zimbra.XXXXX.org, aevans@XXXXX.org, aforshey@XXXXX.org, afranklin@XXXXX.org, agraham@XXXXX.org, aharlow@XXXXX.org, alewman@XXXXX.org, amilke@XXXXX.org, anoack@XXXXX.org, apaul@XXXXX.org, apfaff@XXXXX.org, aroberts@XXXXX.org, astrickland@XXXXX.org, atroxell@XXXXX.org, awoodsmall@XXXXX.org ...
maxFileSize = -1
obeyHSM = false
dataPath = /opt/migrate
skipSystemAccount = true
onlyProvisioning = false
skipAccountProvisioning = false
ConcurrentAccountsRestore = 1
OpStartTime = 1543418167619
notificationeMails = admin@zimbra2.XXXXX.org
skipDistList = false
skipCos = false
domainNameList = XXXXX.org, zimbra.XXXXX.org
restoreChatBuddies = true
-- context --
ContactGroupMap = {7bddb1cc-f9df-47a4-abde-4c72b0a0d33e=[756, 748, 765], 795eeaba-3ed7-465f-9387-007012709633=[385], 5288c80e-67aa-47be-9b50-643b7886dc33=[46849, 23201, 36339, 47940, 35528, 35979, 35516, 44861], 20940f8b-a9d9-4ba7-b64b-a07bd13daabf=[8471], 30bef5a3-c736-41cb-838d-45de3ffc36b8=[16097, 14473], 6a4076e4-b31b-43cf-b47c-68dd358f3011=[11621, 16456], ad7e97a0-6ed9-41a5-a778-dd8d5d05946b=[204419, 90753, 90752, 90755, 90756, 90759, 180037, 90761, 90760, 90765, 143695, 143823, 90764, 129612, 147788, 90767, 90769, 174419, 90768, 90771, 199504, 90770, 152806, 103973, 103975, 90662, 103977, 199146, 90664, 103976, 90667, 149033, 103917, 90668, 90675, 152308, 90680, 90747, 90682, 170046, 90749, 207742, 90748, 90751, 208380, 90750], 6d0a194d-031d-452a-9683-d0a80fed87a3=[96180, 96182], 81365f16-3a33-4d33-b50c-0596bb9cb577=[11281, 11282, 11283, 11236, 11242], 6041b800-becb-4590-8024-ec528e4cfa69=[45746, 23437], ffc078e4-4893-412f-ab10-daeae488168e=[2865], 0889cd29-79d2-4f83-a40f-4ee7154337f9=[3619, 3684, 3690], 6511fb58-fbf2-40ee-b891-7801e454c6d4=[38530, 30402, 30360, 30381], 12a1bb6c-67d3-42e5-97cc-42225709febd=[3402, 13882, 13883, 13884, 13885, 13886], e0e74a12-a548-491a-a4b5-3bffd5a6164c=[21912]}
restored_hack_7_8_list = null
lastItemId = 122712
restoredAdminList = dgoodrich@XXXXX.org, pbyerley@XXXXX.org, nriedman@XXXXX.org, rsouchon@XXXXX.org, admin@zimbra.XXXXX.org
numItemsSkipped = 1474946
backup_version = 8.0.7
lastFakeItemId = 122712
ignoredLdapAttributes = zimbraDomainName, zimbraId, zimbraDomainType, zimbraCreateTimestamp
completed = 1
tag_to_rename = {}
numTotalAccounts = 268
extMap = com.zextras.modules.backup.ZEBackupExtMap@5d0e97d9
usedCOSList = a14ecb61-bcb1-48d3-adc7-8579a37ff1d1, 8a4f9dfe-ea9d-4161-8aab-92922cd02556
numItemsRestored = 611179
lastAccountId = a1cf2146-3e99-492d-a53a-43cf1c4c0a82
domainsWithCos = XXXXX.org
currentAccount = mfruth@XXXXX.org
warningCount = 5
numAccountsRestoredPhase3 = 268
unrestorableAccountItems = {(ae2ec45e-3a3b-4e3c-92e9-7a94dbae264b, ghall@XXXXX.org)=[20648, 20650]}
numAccountsRestoredPhase2 = 268
numAccountsRestoredPhase1 = 268
Post Reply