[Dovecot] mbox sync/lock issue, rc28 and later

[Jeff A. Earickson]

On Fri, 6 Apr 2007, Timo Sirainen wrote:

> Date: Fri, 6 Apr 2007 19:50:07 +0300
> From: Timo Sirainen <tss at iki.fi>
> Reply-To: Dovecot Mailing List <dovecot at dovecot.org>
> To: Jeff A. Earickson <jaearick at colby.edu>
> Cc: dovecot at dovecot.org
> Subject: Re: [Dovecot] mbox sync/lock issue, rc28 and later
> 
> On 6.4.2007, at 15.59, Jeff A. Earickson wrote:
>
>> Apr  6 07:22:52 karst dovecot: [ID 107833 mail.error] IMAP(jaearick): mbox 
>> file /var/mail/j/jaearick was modified while we were syncing, check your 
>> locking settings
>> 
>> Pine coughed up an internal error, no core dump, and dovecot
>> basically doubled the size of my mailbox by duplicating all of
>> my messages again.  What changed with file locking between rc27 and rc28 to 
>> cause this?  How to fix?
>
> I added this check to rc28. Hmm. I guess it's possible that the check doesn't 
> work that great with NFS.

I went back and read the wiki on mbox and locking to see what I might have
done wrong, eg: http://wiki.dovecot.org/MailboxFormat/mbox.  Procmail
on my mail server (the LDA for sendmail there) uses dotlocking, fcntl, lockf
for its lock setting -- this hasn't been changed in a looooong time.

>
> But what do you mean duplicated the messages? Same messages exist multiple 
> times in the mbox file? The only reason I can think of how that could happen 
> is if you expunged a lot of data from the middle of the mbox, and then in the 
> middle of the move that external modification check cancelled the sync.

That seems to be what happened.  I had marked a bunch of messages as deleted,
and had done "expunge" in Pine to trash them.  Then pine gave its internal
error.  Afterwords, most of the deleted messages were gone, some at the
end were not (not marked as deleted), and about 20 messages in the middle
of the mailbox were duplicated again at the end.  I think a new message came
in too.  Except for the duplication and the undeletes, no messages were
mangled.  I just had to re-delete some messages.

After this odd behavior, I went looking for a core file (none), and I checked
my syslogs.  That is when I found the pile of messages about "modified while
syncing".  I found this message 1161 times in my syslogs since March 28, 50
times today.  Nobody has complained about anything though.

>
> If that's not the case, maybe it really was something else than Dovecot that 
> went and duplicated the messages?
>
I only read my email via an IMAP connection with Pine.  Thats it.

Jeff Earickson
Colby College