[Dovecot] mbox locking

[Timo Sirainen]

On Tue, 2006-06-27 at 20:43 +0200, Thomas Hummel wrote:
> On Tue, Jun 27, 2006 at 03:58:28PM +0300, Timo Sirainen wrote:
> 
> Thanks Timo,
> 
> Those questions are because I had some issues (with 0.9x which I know
> isn't maintained anymore) with dead process ('D' flag in BSD ps) and
> fcntl locks that wouldn't go (unless of course I'd reboot or change
> the mbox inode). I'm testing each 1.0beta since I don't want to invest
> effort in 0.9x and would like to have some clear understanding of mbox
> locking strategy in case some similar issue arrise and to be able to
> choose the best locking strategy.

I don't think D means dead even in BSDs? Usually it means
non-interruptible sleep in kernel. I think it could mean that there's a
deadlock between processes (due to different lock ordering). If there's
only one process for the user in that state it means the kernel is
broken. If you're using NFS it probably means your lockd got broken.

> So, one more thing :
> 
> "mbox_read_locks" defaults to fcntl and
> "mbox_write_locks" defaults to dotlock fcntl
> 
> 1. Doesn't this assume that non-dovecot programs do use fcntl (maybe
>   among additional methods) ? In that case, why wouldn't you assume the
>   same for writes and default mbox_write_locks to fcntl only ?

Because often Dovecot is the only one reading the mboxes, but there may
be multiple different mail delivery agents each using their own locking
scheme, but pretty much all of them use dotlock.

If a mail delivery agent uses only dotlocking, then it means that
Dovecot's read locking doesn't work, so Dovecot could see half-written
mails at the end of the mbox. But it still protects against mbox getting
completely corrupted.

> 2. Obviously, fcntl in write_locks should be exclusive, but is the
> lock set by fcntl in mbox_read_lock exclusive or shared ?

It's shared.

> In other words, do you separate read and write locks :
> 
>   . to allow multiple simultneous reads taking the risk to read
>     inconsitent data (if so how do you recognize the situation to re-read
>     the mbox) ?
> 
>   . or because there are moments in a write "session" when the write
>     dotlock is set (but the write fcntl not set yet) and reads are still
>     allowed (because of your writing strategy) ?

Writers should do also the fcntl locking. As long as that's done,
multiple shared fcntl locks can be acquired so it's then safe to read
the mbox.

> Also should a mbox read block a mbox write ? Or should a UA trying to
> get a lock to read a mbox block or just return, sleep a try a bit
> later ?

Well, I don't really see much difference. If you want to do it anyway,
you might as well wait in blocking fcntl call so you don't waste CPU.

If using flock or lockf instead of fcntl Dovecot does non-blocking call
and sleeps a random time. I'm not actually sure why. I think I couldn't
get the call to break after a given time with alarm() but the man page
looks like that should have worked..

> 3. when, as written on the Wiki procmail -v reports 
> 
>  dotlocking, fcntl(), lockf(), flock()
> 
> does it mean that he uses both dotlocking and fcntl (like dovecot) or
> one *or* the other ?

It means it uses all the four in that order.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
Url : http://dovecot.org/pipermail/dovecot/attachments/20060628/cda6e87c/attachment.pgp