dovecot-lda without starting dovecot?

[Stephan von Krawczynski]

On Tue, 7 Nov 2017 19:19:23 +0200
Timo Sirainen <tss at iki.fi> wrote:

> On 7 Nov 2017, at 9.15, Stephan von Krawczynski <skraw.ml at ithnet.com> wrote:
> > 
> > On Tue, 07 Nov 2017 13:19:12 +1000
> > Noel Butler <noel.butler at ausics.net> wrote:
> >   
> >> mail_location   Optionally disable indexes using   :INDEX=MEMORY  
> >> 
> >> don't use this on IMAP boxes, but is safe to use on SMTP and POP3's
> >> boxes though 
> >> 
> >> eg: 
> >> 
> >> mail_location =
> >> maildir:/var/vmail/%Ld/%1Ln/%1.1Ln/%2.1Ln/%Ln/Maildir:INDEX=MEMORY 
> >>   
> > Hello Noel,
> > 
> > this sounds interesting. Can you please elaborate why you think this is no
> > good idea for IMAP?
> > We used a different LDA-scheme before (simply created the mail in
> > maildir/tmp, then renamed it to maildir/new, just like Bernstein suggests
> > to do) and it worked very well, no matter if the box was used whith IMAP
> > or POP3. Why should there be any difference in using dovecot-lda without
> > indexes? Does dovecot-lda "create" the new mail by atomic rename from tmp,
> > too?  
> 
> Although the above disables updating indexes, there's still dovecot-uidlist
> file that is always updated by dovecot-lda. It might also cause corruption
> problems when multiple servers are accessing it at the same time. There's
> some old code that attempts to avoid updating the uidlist when it's not
> necessary (MAILBOX_TRANSACTION_FLAG_ASSIGN_UIDS==0), but I don't know if
> that works. There's also .dovecot.lda-dupes file that is written. And the
> Sieve scripts' compiled versions that you wanted to start using. In short:
> This might work, but I have a feeling you'll run into random corruption
> problems. I gave up trying to support simultaneous access in multiple
> servers via NFS long time ago, because no matter what I did kernel always
> cached too much and caused corruption (or alternatively it didn't cache
> enough and caused performance problems).

Hello Timo,

Me again have to stress that our former implementation of the lda process did
do exactly nothing to all the dovecot files, and everything worked pretty
well. We had no problems in years. So I really wonder if it wouldn't be the
best way to simply cut away all the heavy dovecot magic from the lda as an
option. All it really needs to do is pipe the mail into a temp file, do the
sieve stuff and learn from it where to rename this temp file to, basically.
On the other hand you have lots of config parameters in dovecot dealing with
different approaches to lock files (from fcntl to dotlock). I would expect at
least one of them to work over NFS.
And we do see stuff like:

2017-11-07T00:46:59.102961+01:00 mail-a05 dovecot: lda(somebody at somedomain.com):
Warning: Locking transaction log
file <nfs-path>/dovecot.index.log took 40 seconds (appending)

So some locking seems to work.
-- 
Regards,
Stephan