[Dovecot] Indexing Performance Question (was tpop3d vs dovecot)

[Timo Sirainen]

On 2.2.2007, at 1.29, Jakob Hirsch wrote:

> Quoting Timo Sirainen:
>
>> The reason why the messages are read in the first place is because  
>> the
>> message sizes need to be returned so that linefeeds are counted in  
>> CR+LF
>> format, while they're typically stored in LF-only format. So if  
>> Dovecot
>> just stat()ed the file and returned that as the message's size, it  
>> would
>> be violating the POP3 spec. tpop3d seems to be doing that.
>
> Oh. I would suggest a config switch (yes, yet another one) that tells
> dovecot to do the same. Most clients don't care about it, anyway,  
> and if
> you tell your LDA to use crlf, dovecot would even do the right thing.

Maybe.. I think plugin would be best though, since I don't really  
like adding options to intentionally break RFC.

Anyway I don't think it should be that problematic if the cache file  
works properly. It probably doesn't even add all that much extra disk  
I/O since the client will just download the new mails anyway, so as  
long as they stay in kernel's buffer cache long enough there's no  
problem. Well, except for a bit of extra CPU usage.

Oh and CVS HEAD deliver stores the message sizes directly into cache,  
so it won't be a problem at all with it. :)

> Hm, how does dovecot know that a message contains crlf, so it can use
> sendfile instead of read/write? If you have some kind of simple
> autodetection, like "read the first line, if it contains crlf, assume
> all other lines also do", we already have some (non-optimized) form  
> of that.
> I guess this also affects IMAP, right?

There's no simple autodetection. There's a "message is binary safe"  
flag in cache file that gets set after the message has once been  
parsed and only CRLF-linefeeds were seen.

With IMAP the message's size is needed to be sent before the actual  
message contents, and it must be exactly correct or the whole session  
will break.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://dovecot.org/pipermail/dovecot/attachments/20070202/37e12671/attachment.pgp