Deleting ALL accounts doesn't free up disk space

[connordon]

Hello,

I'm testing the migration to a new zimbra and decided to delete all the accounts (that was created on the test server on last external restore launch) (except the admin one that has no email inside) to launch the external restore again with a "clean" env

But the disk space is still used

--- /opt/zimbra ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
35.6 GiB [##########] /store
35.2 GiB [######### ] /redolog
2.0 GiB [ ] /db
927.7 MiB [ ] /data
725.2 MiB [ ] /common


Restarting zimbra didn't solved

Is there anything i need to do?

Cheers

[connordon]

Update:

I ended up optimizing all mysql tables

Solved

[liverpoolfcfan]

How did you do that? We have deleted many accounts on a server and are seeing the same issue that the backups have not reduced in size.

[connordon]

1) Retrieve your MYSQL passwords

Code: Select all

zmlocalconfig -s | grep mysql | grep password

2) Launch mysqlcheck with optimize option ("-o") and provide via stdin the root password you got (I used "-S" because it didn't find the right socket)

Code: Select all

mysqlcheck -o --all-databases -S /opt/zimbra/data/tmp/mysql/mysql.sock -u root -p

3) Make sure to launch this on a idle situation, because mysqlcheck will lock tables, you can't just shutdown zimbra services because trying to start only the MYSQL server was a pain for me, there is not "zmmysqlctl" command

It's going to find any table that contains free space (they are like a container that can only grow and no resize) and it's going to recreate them from scratch (with the minimum size of the actual data), make sure you have some free space, in a specific moment the old and the new table will exists at the same time, if you have a table of 12 GB with actually 5GB of data, the OS must have at least 5 GB of free space (for the new one) before deleting the old 12 GB table

PS: i don't think, depending on what backup situation you use (legacy or zxsuite backup) that the deleted account data, outside of the trash retention will still be backupped

After running these command you're probably going to see less used space from the "/opt/zimbra/store" folder, there is no backup data i remember on that path

And I always recommend a backup, you never know...

[ghen]

you can't just shutdown zimbra services because trying to start only the MYSQL server was a pain for me, there is not "zmmysqlctl" command

There's "mysql.server start|stop" (not consistenly named indeed).

[connordon]

you can't just shutdown zimbra services because trying to start only the MYSQL server was a pain for me, there is not "zmmysqlctl" command

There's "mysql.server start|stop" (not consistenly named indeed).

lol, forgot to try the obvious one

[L. Mark Stone]

<snip>
After running these command you're probably going to see less used space from the "/opt/zimbra/store" folder, there is no backup data i remember on that path

And I always recommend a backup, you never know...

MariaDB stores nothing in /opt/zimbra/store; that's just for the mail blobs. After you delete an account from the Admin Console or via "zmprov da <account>", Zimbra dispatches a thread to delete the mail blobs. The mail blob deletion happens in a "lazy" way, so as not to use too much I/O. Depending on how busy the server is and how much data needs to get deleted, this "lazy" deletion can take anywhere from less than a minute to more than an hour. If you restart Zimbra before the deletion finishes, you wind up with "orphaned" mail blobs in /opt/zimbra/store that still consume disk space.

The space consumed by the redologs is another matter. Current NG backups no longer rely on Zimbra's native redologs. Only if you are running the Network Edition Classic backups are the redologs required. And, it takes an NE Classic Incremental backup to clean out the redolog archive directory. Without that, over time the redolog consumption will fill your entire disk. If you are not using the NE Classic backups, you may want to consider doing like:

Code: Select all

zmprov mcf zimbraRedoLogDeleteOnRollover TRUE
zmprov mcf zimbraRedoLogEnabled FALSE
zmmailboxdctl restart

Hope that helps,
Mark

[connordon]

MariaDB stores nothing in /opt/zimbra/store; that's just for the mail blobs. After you delete an account from the Admin Console or via "zmprov da <account>", Zimbra dispatches a thread to delete the mail blobs. The mail blob deletion happens in a "lazy" way, so as not to use too much I/O. Depending on how busy the server is and how much data needs to get deleted, this "lazy" deletion can take anywhere from less than a minute to more than an hour. If you restart Zimbra before the deletion finishes, you wind up with "orphaned" mail blobs in /opt/zimbra/store that still consume disk space.

Thanks for the info,

One question: is possible to see progress/force this blob deletion ?

zmblobchk is not supported anymore and this docs: https://wiki.zimbra.com/wiki/Zimbra_NG_Blob_Check , doesn't work as expected:

Code: Select all

2022-09-20 09:46:15,084 WARN Unable to find the account related to mailboxid 131: it is not present in the local database. Path in store: /opt/zimbra/store/0/131
2022-09-20 09:46:15,085 WARN Unable to find the account related to mailboxid 133: it is not present in the local database. Path in store: /opt/zimbra/store/0/133
2022-09-20 09:46:15,085 WARN Unable to find the account related to mailboxid 134: it is not present in the local database. Path in store: /opt/zimbra/store/0/134
2022-09-20 09:46:15,085 WARN Unable to find the account related to mailboxid 138: it is not present in the local database. Path in store: /opt/zimbra/store/0/138
2022-09-20 09:46:15,085 WARN Unable to find the account related to mailboxid 139: it is not present in the local database. Path in store: /opt/zimbra/store/0/139
2022-09-20 09:46:15,086 WARN Unable to find the account related to mailboxid 78: it is not present in the local database. Path in store: /opt/zimbra/store/0/78

Because the accounts are already deleted