Dovecot 2.2.16: disappearing messages, mismatched summaries, duplicated messages, excessive full re-downloads

[David Gessel]

Hi Timo, 

Thank you for the reply.  It is very much appreciated.

-------- Original Message --------
Subject: Re: Dovecot 2.2.16: disappearing messages, mismatched summaries, duplicated messages, excessive full re-downloads
From: Timo Sirainen <tss at iki.fi>
To: David Gessel <gessel at blackrosetech.com>
Date: Thu Apr 16 2015 21:31:30 GMT+0300 (Arabic Standard Time)

> On 16 Apr 2015, at 04:45, David Gessel <gessel at blackrosetech.com> wrote:
>>
>> My Dovecot 2.2.16 server is interacting poorly with a variety of mail clients.  The symptoms include disappearing messages, mismatched summaries, duplicated messages, and excessive full re-downloads.
> 
> What were you running before Dovecot v2.2.16? An earlier Dovecot? A whole different system?

I migrated from Cyrus to Dovecot in July of last year on version 2.2.13.   The issues started then and haven't really ever mitigated, though my impression (without any real data to back it up) is that it is more of a problem now, particularly mail appear/disappear issues.  The first issue I noticed was unexpectedly frequent full resyncs in TB.

> 
>> dovecot.log shows runs of this error:
>> Apr 15 09:02:37 imap(user at domain.com): Warning: Maildir /mail/domain.com/user//.Sent: Expunged message reappeared, giving a new UID (old uid=306286, file=1429102024.M826331P1156.host,S=1678,W=1724:2,S)
>>
>> or this error:
>> Apr 15 10:04:00 indexer-worker(user at domain.com): Warning: Maildir /mail/domain.com/user//.Sent: Expunged message reappeared, giving a new UID (old uid=306803, file=1428785463.M645546P27215.host,S=5389,W=5442:2,S)
> ..
>> The first entry in dovecot.log.0 is at Apr 15 10:00:12, and the last entry at Apr 15 12:00:13; Dovecot is logging 4,490 expunged message reappeared messages per hour.  Note the server has 4 accounts set up and very limited traffic, so 4,490 per hour would seem like a lot of anything to be logged.
> 
> These really shouldn't be happening..
> 

I didn't think so.  I've contemplated migrating to sdbox, but Maildir shouldn't be a problem, I don't think.  Things that strike me as possibly moving my config out of the well-tested regime include (not that anything is unique, but there may not be that many people testing all of the following):

FreeBSD 10.1
ZFS
mail_location = maildir:~/:CONTROL=/var/no-quota/%u
FTS-Lucene
Sieve/lmtp
Quota

Not that anything looks suspicious.  The logs don't seem to indicate any reason for the "Expunged message reappeared, giving a new UID" errors.  Is there someplace else the cause might be logged?