[Dovecot] Locking /var/mail/user issue with postfix and dovecot

[Stan Hoeppner]

On 10/25/2012 11:16 PM, Ben Morrow wrote:
> At  1AM +0300 on 26/10/12 you (Robert JR) wrote:
>> On 2012-10-26 00:15, Ben Morrow wrote:
>>
>>> As Stan said earlier, this is a Postfix question. The rule for
> 
> [Looking back at the thread it wasn't Stan, it was Dennis Guhl. Sorry
> about that.]

I prodded him a second time, might have been off-list, and he finally
posted there.  So call it a team effort. ;)

Wietse has already replied, and in typical fashion, asked for "concrete"
evidence that Postfix was performing fcntl before dotlock, because he
obviously knows better than anyone that Postfix applies a dotlock first,
which you already explained here.

>>> dotlocking is that you must create the .lock *before* opening the
>>> file, in case whoever has it locked will be replacing the file
>>> altogether; but with fcntl locking you must acquire the lock *after*
>>> opening the file, since that's the way the syscall works. This means
>>> that if Postfix is going to use both forms of lock, it has to
>>> acquire a dotlock before it can look for a fcntl lock.
>>>
>>> In other words: the methods in mailbox_delivery_lock are *not* tried
>>> in order, because they can't be. Dotlock is always tried first.
>>>
>>> You should have compatible locking settings for all your programs
>>> accessing your mboxes. If Postfix is using dotlock, Dovecot should be
>>> using dotlock as well. If you don't have any local programs (mail
>>> clients, for instance) which require dotlocks, you should probably
>>> change Postfix to just use fcntl locks.

<doc stuff snipped>

> but if that is the case you might as well configure everything to just
> use fcntl locks, and forget dotlocks altogether.

Yep.  Postfix can use either or both.  And, surprise, recommends using
maildir to avoid mailbox locking entirely.

> Stan's earlier point is fundamentally correct: if you can treat the
> Dovecot mailstore as a black box, with mail going in through the LDA and
> LMTP and mail coming out through POP and IMAP, your life will be much
> easier. Traditional Unix mailbox locking strategies are *completely*
> insane, and if all you are doing is delivering mail from Postfix and
> reading it from Dovecot it would be better to avoid them altogether, and
> switch to dbox if you can. However, if you have any other programs which
> touch the mail spool (local or NFS mail clients, deliveries through
> procmail) this may not be possible.

And since this is a POP only server, users' MUAs should be deleting
after download, so there shouldn't be much mail in these mbox files at
any given time, making migration to maildir or dbox relatively simple.

When using Dovecot LDA you'll eliminate the filesystem level locking
problems with mbox.  However, you may still have read/write contention
within Dovecot, such as in your 20MB download as new mail arrives
example, especially if the new message has an xx MB attachment.  I don't
believe Dovecot is going to start appending a new message while it's
still reading out the existing 20MB of emails.  Depending on how long
this takes Dovecot may still issue a 4xx to Postfix, which will put the
new message in the deferred queue.  With maildir or dbox, reading
existing mail and writing new messages occurs concurrently, as each
message is a different file.

-- 
Stan