[Dovecot] A large number of imap processes

[Claude Frantz]

Timo Sirainen wrote:

>> I think that the large number of now running processes is the rebuilding 
>> of the index. I am right here, Timo ? 
> 
> Index updates typically don't take long. Although if client fetches some
> metadata that isn't in cache file, Dovecot needs to open the messages
> and parse them. That takes some I/O. This can be avoided with v1.1 by
> using deliver, since it updates the cache file immediately.

I have tried it using deliver in the version dovecot-1.0.10-0_66.fc7. 
Each delivery run was longer when using maildrop, but the load was too
high too. Do you think that in the version 1.1 the behavior would be 
different ?

> Although I'm still not sure if that would help, because a lot of clients
> just download the entire message body automatically, which in any case
> causes I/O.

Most users here use thunderbird, outlook (and express) or a webmailer 
using squirrelmail.

>> At the present time, the mailfilter language of 
>> maildrop (and Courier) is used here in place of procmail used in the 
>> past. I ignore if using dovecot's LDA could be a solution in this 
>> environment. But the mailfilter language could be a problem. A migration 
>> to sieve is not really a choice, in my opinion. I have never seen a good 
>> sieve script testing software.
> 
> Can you tell maildrop to deliver the message to a specific mailbox using
> an external command (i.e. calling deliver) instead of delivering itself?

Yes, it is possible and I have tried it. It is then necessary to include 
the code in a mailfilter which finally calls deliver when the decision 
has been made. Here again, the result was not really better. In fact, it 
is difficult to make such a delivery via maidrop calling deliver as a 
"standard" in a productive environment.

In order to circumvent the problem, I have now configured sendmail, the 
MTA, in a little bit special manner: Incoming messages are stored in the 
queue first and the delivery occurs only in the queue run. These are run 
every 2 minutes, each run processes max. 150 messages. Incoming messages 
are split in multiple ones having 4 recipients max. In this manner the 
load can be limited at the expense of pass through time, if necessary. 
Further, there are more (split) message in the queue. This is not really 
a good "solution", but it will limit the problems a little bit. I'm 
still looking for a better solution. NFS is still a problem.

Thanks a lot !

Claude