On Oct 19, 2009, at 9:35 AM, Thomas Hummel wrote:
Dovecot documentation states that the "random redirects to multiple
servers" NFS solution is to be avoided and I'm investigation the actual risks
of it and a way to put it to test.
If you do it, you'll most likely see some random index related errors.
Documentation says : "Dovecot locks the maildir while doing
modifications to it or while looking for new messages in it" and then mention
the "dovecot-uidlist.lock" dotlock file.a) if that file fcntl'ed in addition (i.e. dotlocked + fcntled
or just dotlocked) ?
No fcntl.
b) is that file THE way to lock the Maildir mentionned above oris it just something else used only for updating that particular
file (i.e. is Maildir locked in some way + dovecot-uidlist.lock created or
just dovecot-uidlist.lock created) ?
Maildir is then locked for Dovecot against doing any modifications.
Procmail will ignore it, and that's fine.
Documentation says : "NFS caching is a big problem when multiple
computers are accessing the same mailbox simultaneously"I guess it's because of the dotlocks files and not fcntl locking ?
No, it's because of NFS caching.. Like if Dovecot writes to
dovecot.index.log first and then dovecot.index, it expects that it's
not possible that a later read sees the dovecot.index change, but not
the dovecot.index.log change. But because of NFS caching this is
possible, and "corrupted index files" errors will occur. There are
many other similar issues.
Documentation says : "Dovecot v1.1 flushes NFS caches when
needed if you set mail_nfs_storage=yes"How can a program flush the NFS caches ? By which (system) call ?
That's the main problem. There is no reliable way to flush NFS
caches.. So I've added some highly OS-specific code that works most of
the time, but not perfectly. It works best with Linux.
- Documentation says : "Besides the NFS cache problems described
above, mailbox contents can't be cached as well in the memory either."Is it about in-memory indexes or part of indexes loaded into
memory ?
I was thinking about OS's filesystem caching in general, so all mail
files and such.
- How can I torture-test concurrent access to the same mailbox
through 2 dovecot servers ? I don't see imaptest.c anymore on http://dovecot.org/tools but I see 2 nfs_test ? Is there some command line options I
should use ?
imaptest exists now in http://imapwiki.org/ImapTest
- when and why can a random redirect to multiple server can cause
mailbox corruption ?
Pretty much randomly..
On flags only or on content as well ?
Actual mail content should be safe. Unless you just happen to get such
a cache file corruption that Dovecot doesn't notice it and sends some
broken headers to IMAP client.
The long term fix for this is http://www.dovecot.org/list/dovecot/2009-August/041983.html