QRESYNC issues

Have a great idea for extending Zimbra? Share ideas, ask questions, contribute, and get feedback.
Post Reply
gds
Posts: 3
Joined: Tue Nov 16, 2021 6:58 pm

QRESYNC issues

Post by gds »

I've been researching the addition of imap QRESYNC to Thunderbird. I've noticed that Zimbra has some issues that I don't see with other servers like Dovecot or Cyrus which also support QRESYNC.

A problem with Zimbra is that when a lot of messages are imap APPENDED to a mailbox, the UIDs assigned to them are not always consecutive. This causes zimbra to report the missing UIDs as VANISHED on a new imap SELECT of the mailbox due to missing or skipped UIDs. With a large append of 1000's of messages, it appears that Zimbra does not use about every 7th UID, leaving lots of 1 message (1 UID) gaps.

As another example, say the UIDs in mailbox A are initially 1 to 100 and 100 is the highest UID in any mailbox. Then if I APPEND 100 messages to mailbox B, it will assign UIDs 101 to 200 to the messages. Then if I APPEND 100 more messages to mailbox A, they will be assigned UIDs 201 to 300, leaving a "gap" of 100 UIDs in mailbox A. Messages 101:200 in mailbox A will be reported as VANISHED on a new SELECT.

Also, when messages are truly EXPUNGED, it appears the Zimbra does not really respect the MODSEQ supplied to the SELECT command and just reports all missing UIDs as VANISHED and not just those expunged after the supplied MODSEQ.
gds
Posts: 3
Joined: Tue Nov 16, 2021 6:58 pm

Re: QRESYNC issues

Post by gds »

I can fix the small gaps in UID assignment with a change to Thunderbird. It seems that Zimbra does not like email headers sent via imap APPEND that begin with "From ". It causes Zimbra to sometimes ignore the "Date:" header and and also skip UIDs so they are not consecutively assigned. The proposed tb fix can be found here: bug 1741748

I still tend to think that a leading "From " in the imap APPEND messages should just be ignored by Zimbra. I've not noticed this problem with other servers.
gds
Posts: 3
Joined: Tue Nov 16, 2021 6:58 pm

Re: QRESYNC issues

Post by gds »

I went back and checked this again after over a year. Actually, the change I made to thunderbird didn't really affect the UIDs assigned by zimbra that occur when imap appending a fairly long list of messages. For unknown reasons, zimbra just randomly skips over a UID value and leaves gaps of one UID value in the increasing sequence. Technically this doesn't violate the base imap spec rfc3501 but it causes problems with qresync extension in that all the gaps are reported as vanished messages which they are not.

Also, since zimbra assigns UIDs from an account global counter, this also produces even larger gaps when messages are copied to different folders. These large gaps are also reported as vanished with qresync enabled. This wouldn't be a problem if zimbra used a separate UID assignment counter for each mailbox/folder like other qresync compliant servers or if zimbra just didn't report the gaps it creates as vanished UIDs.
Post Reply