[Dovecot] Search over big folder hierarchy (was: Global FTS index?)

[Timo Sirainen]

On Tue, 2008-06-24 at 15:29 +0800, Patrick Nagel wrote:
> On Tuesday 24 June 2008, Patrick Nagel wrote:
> > Hi,
> >
> > I tried the FTS (and FTS Squat) plugin today, and it works as advertised.
> >
> > But: On my 13000 folders with 160000 mails maildir I use for testing, the
> > speed increase is not as big as one would wish (it still takes several
> > minutes to complete a search).
> >
> > Is my assumption correct, that there is no way to do a search over a big
> > IMAP folder hierarchy in a reasonable amount of time, because each folder
> > has to be 'selected', and only one folder can be selected at once?

Yes, they have to be selected. There isn't any way currently in IMAP to
search from multiple mailboxes using a single command, so even if
Dovecot implemented a Squat index that indexed mails from all mailboxes,
you'd still have to implement a non-standard extension to use that.

Hmm. Or v1.2 has virtual mailboxes - you could create a single virtual
mailbox from all your other mailboxes and then search it. I think if
Squat is enabled it'll create a single index from all the mails. I'm not
sure if I want to leave it like that though..

I have also been thinking about making Squat indexes global for all
mailboxes. If done well it should reduce disk space as well as enable
fast multi-mailbox searches, but I'm a bit worried about memory usage
and other slowness when updating the index. The Squat building/updating
could use more work, but I haven't yet figured out a great solution for
it.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://dovecot.org/pipermail/dovecot/attachments/20080624/c3f3d32a/attachment.bin