[Dovecot] Full text search improvements

[Timo Sirainen]

On 2.12.2013, at 20.50, Mike Abbott <michael.abbott at apple.com> wrote:

>> how [FTS indexing] could be improved for everyone in future
> 
> For sites which set client_limit > 1 it would help performance not to stall for INDEXER_WAIT_MSECS when polling the indexer for input.  Currently dovecot unwinds back out to the main command loop repeatedly to allow other clients to use the process but it also stalls the whole process for INDEXER_WAIT_MSECS every time it finds no input from the indexer, which hurts responsiveness for those other clients.  This can be avoided by removing the client's I/O from the main ioloop and adding the indexer's instead, or perhaps by leveraging CLIENT_COMMAND_STATE_WAIT_EXTERNAL.

Gets a bit tricky to implement, at least without changing the lib-storage API. I did have some plans for this earlier where lib-storage could call some callback when there is more data available for search/fetch/mailbox_open/etc functions. Currently I’m thinking that most of the reasons for client_limit>1 can be avoided just by moving IMAP IDLE connections to a separate imap-idle process where they wait until they have more work to do. Do you think that would work for you also?