[Dovecot] Still weird UID ordering issues with maildir

[Timo Sirainen]

On 11.9.2005, at 18:22, Todd Vierling wrote:

>>> repeated every time I try to login.  I had to nuke the dovecot 
>>> indexes and
>>> let it reindex (which throws off the ordering of some messages).  It 
>>> had
>>> been working without incident since updating to 20050829, until now.
>>
>> Actually deleting indexes shouldn't matter at all, since Dovecot does
>> that by itself after writing that error message.
>
> No it doesn't, at least not in nightly 20050829.  The imap process 
> exits and
> I have to delete them manually; they're still sitting around.  An 
> artifact
> of this is the fact that if I don't delete the indexes manually before
> reopening the mailbox, the error happens again with exactly the same 
> UIDs:

Hmm. Well, it doesn't exactly delete them but it marks them corrupted, 
which should make it rebuild them when opening the next time. Guess 
I'll have to look at that too.

>> The error message means that Dovecot saw a message 1187 that wasn't in
>> index, but index said it already had seen message 1188. Probably means
>> that the file was temporarily lost in some sync, but later seen again 
>> in
>> another sync.. Dovecot isn't very forgiving if readdir() skips files.
>
> I don't see how skipping files like that should happen -- HOWEVER, you 
> do
> realize that readdir(3) has no guarantee of ordering, right?  (Meaning 
> that
> two successive calls to readdir(3) can, per POSIX, return the same 
> list of
> files in a completely different order.)

Yes, the order isn't relied on. The filenames are all read into a hash 
table which is used.

>> Are there other maildir clients than Dovecot updating the maildir? 
>> That
>> can cause those problems.
>
> The only other mailbox touching process is procmail, which is putting 
> mail
> into the folders.  However, this happens even when procmail is not
> *actively* putting mail into the folder (it could be that mail had been
> added between the last time the mailbox was closed, and when it was now
> reopened).

OK. That shouldn't matter since it's touching only new/ dir.

> But isn't collision-free access supposed to be a feature of using 
> maildir --
> even if I were using nfs?  :)

That's the theory, but in practise if files are being deleted or 
renamed (I'm not sure about adding) inside a directory while another 
process is reading its contents, some files might get skipped. That's 
why Dovecot keeps dovecot-uidlist file locked while it's reading 
maildir, so the skipping shouldn't happen if only Dovecot is accessing 
the maildir.

But if there aren't other maildir clients accessing the maildir, do you 
then use some IMAP client that opens multiple connections to same 
mailbox? Just wondering if there's a way I could reproduce this 
problem..
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://dovecot.org/pipermail/dovecot/attachments/20050911/8ef11492/PGP.pgp