On Tue, 2006-06-27 at 20:43 +0200, Thomas Hummel wrote:
On Tue, Jun 27, 2006 at 03:58:28PM +0300, Timo Sirainen wrote:
Thanks Timo,
Those questions are because I had some issues (with 0.9x which I know isn't maintained anymore) with dead process ('D' flag in BSD ps) and fcntl locks that wouldn't go (unless of course I'd reboot or change the mbox inode). I'm testing each 1.0beta since I don't want to invest effort in 0.9x and would like to have some clear understanding of mbox locking strategy in case some similar issue arrise and to be able to choose the best locking strategy.
I don't think D means dead even in BSDs? Usually it means non-interruptible sleep in kernel. I think it could mean that there's a deadlock between processes (due to different lock ordering). If there's only one process for the user in that state it means the kernel is broken. If you're using NFS it probably means your lockd got broken.
So, one more thing :
"mbox_read_locks" defaults to fcntl and "mbox_write_locks" defaults to dotlock fcntl
- Doesn't this assume that non-dovecot programs do use fcntl (maybe among additional methods) ? In that case, why wouldn't you assume the same for writes and default mbox_write_locks to fcntl only ?
Because often Dovecot is the only one reading the mboxes, but there may be multiple different mail delivery agents each using their own locking scheme, but pretty much all of them use dotlock.
If a mail delivery agent uses only dotlocking, then it means that Dovecot's read locking doesn't work, so Dovecot could see half-written mails at the end of the mbox. But it still protects against mbox getting completely corrupted.
- Obviously, fcntl in write_locks should be exclusive, but is the lock set by fcntl in mbox_read_lock exclusive or shared ?
It's shared.
In other words, do you separate read and write locks :
. to allow multiple simultneous reads taking the risk to read inconsitent data (if so how do you recognize the situation to re-read the mbox) ?
. or because there are moments in a write "session" when the write dotlock is set (but the write fcntl not set yet) and reads are still allowed (because of your writing strategy) ?
Writers should do also the fcntl locking. As long as that's done, multiple shared fcntl locks can be acquired so it's then safe to read the mbox.
Also should a mbox read block a mbox write ? Or should a UA trying to get a lock to read a mbox block or just return, sleep a try a bit later ?
Well, I don't really see much difference. If you want to do it anyway, you might as well wait in blocking fcntl call so you don't waste CPU.
If using flock or lockf instead of fcntl Dovecot does non-blocking call and sleeps a random time. I'm not actually sure why. I think I couldn't get the call to break after a given time with alarm() but the man page looks like that should have worked..
- when, as written on the Wiki procmail -v reports
dotlocking, fcntl(), lockf(), flock()
does it mean that he uses both dotlocking and fcntl (like dovecot) or one *or* the other ?
It means it uses all the four in that order.