speed Re: [Dovecot] New index / mailbox API

[Farkas Levente]

hi,
it's an interesting read, but go back to a higher level for a moment.
I like dovecot since it's use simple files as mail storage (not like 
cyrus). in case of dovecot if something happend I can easily switch to 
another imap server, which is one of the most important feature in a 
production enviroment (there should have to be a way of escape). other 
important feature is the speed (when we have a few hundred of 
mailboxes). some kind of indexing seems to be a good way of to do this. 
but I always feel that dovecot reinvent the wheel, since there is a 
dozen of database system which has nothing else to do just indexing (ok 
it's not true, but..). they probably do it in the right way (or at least 
we can find some) and they has a few years of experience. they do right 
the indexing, the locking, the transactions, etc..
so why we not use one realy fast and good database engine to index our 
mail storage?
the only reason what I can accept in this case, that this is some very 
special type of database and dovecot can use such algorithm which suited 
to this problem better then a general indexing algorithms. is this true?

another thing which always come to my mind when think about speed:
why we do the indexing when we look into the folders, since IMHO it'd be 
more efficient if we do it at the mail delivery time. the mail arrival 
is more balanced during the time, so the system load is more balanced. 
so there can be
- one optional deivery helper application which can do the indexing 
during the deivery time,
- indexing during remove, move, copy etc. imap operations,
- and the current (eg, the new indexing engine) if someone do not use 
the delivery helper apps.

just my 2c.

-- 
   Levente                               "Si vis pacem para bellum!"