Very slow getRestURL "//?fmt=tgz

[maxxer]

Hi.
We're currently making backup using getRestURL, but since some time (sadly I don't know exactly when, we took over management of this server only later) it has become dead slow! The IT manager says nothing has been changed on the hardware, on the software side we installed ActiveSync service with Z-Push, and they have around 20 user connected.
RAM is not totally full (even tough 90% is used) and swap is at 50%, but there's no big iowait.

Since the deployment of z-push we increased the soap session limit to 12 (zimbra_session_limit_soap=12) in order to handle the syncing users.

What could cause such slowness?
We run backups during the night, so when most clients are not doing activity, but could z-push be slowing down export?

Thanks

[L. Mark Stone]

Hi.
We're currently making backup using getRestURL, but since some time (sadly I don't know exactly when, we took over management of this server only later) it has become dead slow! The IT manager says nothing has been changed on the hardware, on the software side we installed ActiveSync service with Z-Push, and they have around 20 user connected.
RAM is not totally full (even tough 90% is used) and swap is at 50%, but there's no big iowait.

Since the deployment of z-push we increased the soap session limit to 12 (zimbra_session_limit_soap=12) in order to handle the syncing users.

What could cause such slowness?
We run backups during the night, so when most clients are not doing activity, but could z-push be slowing down export?

Thanks

Has your Zimbra server been around for a few years?

If so, check to make sure the size of your InnoDB buffer pool is larger (by at least 20%) than the size of your InnoDB database. You can use a tool like mysqltuner.pl to check if you are not facile with MySQL command line work.

If the database is bigger than the buffer pool, then there will be a lot of thrashing of MySQL data between RAM and disk -- at the same time that you are trying to read and compress blobs from disk to export the mailbox.

Hint: If you run top while doing a mailbox export and the %wa is above 20, you definitely have a disk I/O issue.

Hope that helps,
Mark

[maxxer]

Thank you very much but it doesn't seem a IOwait issue, the value in top is below 1% all the time.

I have some vague suspects that it could be related to SOAP sessions (if possible): I'm currently investigating some z-push issues where I have mail.NO_SUCH_ITEM errors in z-push-error.log. I've increased the LC zimbra_session_limit_soap param and effectively I'm not seeing SOAP errors anymore, but from time to time I get those waitset messages...

[Klug]

Quick reminder (not linked to your issue): the REST interface is broken.

It's totally broken through a zimbra-proxy and partly broken if you access the mailbox directly.
It should not be used for backup, you'll never be sure your dump contains the data you want.
https://bugzilla.zimbra.com/show_bug.cgi?id=101760

[maxxer]

Quick reminder (not linked to your issue): the REST interface is broken.

thanks for the reminder, unfortunately I know it very well, but for the time being I'm still using it in some places and doing a cold rsync from time to time. While we're OT, is there any other backup solution (other than NE and Zextras)? thanks