The update to dovecot2-2.2.18/dovecot-pigeonhole-0.4.8 seems to have made problems worse. Whole directories don't load at all - or they may load a weird segment - like mail from a few months ago back and mail from the last two days, but nothing in between.
I did a # find /mail -name "dovecot.index*" -type f -delete
and then on the client side (K9) things clear up, on TB a Right click on folder -> properties -> repair folder cleans things up. Of course all messages have to be redownloaded so it isn't exactly a performant option.
Is there a way to completely disable caching on dovecot? I know it is sort of the raison d'être, but if I can't solve the problems with dovecot, I'll have to try a different, non-caching/indexing server anyway and at least it would help narrow down the issues.
-------- Original Message -------- Subject: Re: Dovecot 2.2.16: disappearing messages, mismatched summaries, duplicated messages, excessive full re-downloads From: David Gessel <gessel@blackrosetech.com> To: Timo Sirainen <tss@iki.fi> Date: Thu Apr 23 2015 18:39:02 GMT+0400 (Arabian Standard Time)
very_dirty_syncs helped a lot - the problems still happen but only about every 30 minutes. It seems this flag prevents re-indexing on whatever was constantly inducing it (with every client-check, if there was a change written to the directory, is my best guess watching the logs go).
I still get all the same symptoms, but far less frequently.
I'm inclined to believe, as trivial as it may be to enumerate, that:
Something is triggering dovecot to believe the indexes need to be rebuilt.
When checking mail during the rebuild, clients get confused by UIDs in transition.
Today I had to quit TB and restart because duplicates filled my preview pane. Mulberry has been better, when it gets confused now, it is usually sufficient to initiate a manual check or expunge to get the list to update.
I would think that sdbox would alleviate these issues, no?
-------- Original Message -------- Subject: Re: Dovecot 2.2.16: disappearing messages, mismatched summaries, duplicated messages, excessive full re-downloads From: Timo Sirainen <tss@iki.fi> To: David Gessel <gessel@blackrosetech.com> Date: Sat Apr 18 2015 17:38:02 GMT+0300 (Arabic Standard Time)
On 18 Apr 2015, at 16:00, David Gessel <gessel@blackrosetech.com> wrote:
No. My best guess is that (your) ZFS+FreeBSD is simply not behaving the way Dovecot wants filesystems to behave (readdir() sometimes skipping over files). I don't have any other suggestion than trying a different filesystem or OS. I haven't heard ZFS having these problems before though.
Or migrating to sdbox. Absent any further suggestions, that will be my next step. My commitment to Maildir is far lower than my commitment to FreeBSD/ZFS. It is just a historical preference coming from a decade+ of courier IMAP.
Actually as a workaround you could try maildir_very_dirty_syncs=yes which gets rid of most of the readdir()s.