[Dovecot] virtual mailbox / INTHREAD use case, issues, questions

[Timo Sirainen]

On Wed, 2010-01-06 at 11:50 -0500, David Abrahams wrote:
> At Wed, 6 Jan 2010 14:17:40 +0200,
> Timo Sirainen wrote:
> > 
> > On 4.1.2010, at 20.47, David Abrahams wrote:
> > 
> > > 1. I had to manually create the virtual folder for all my users or
> > >   they couldn't access their mail at all.  Is this fixed in 2.0?
> > 
> > No.
> 
> OK, is it considered a bug that should be addressed?  IMO there should
> at least be a way to tell dovecot to ignore the non-existence of a
> particular folder.

Actually it looks like v2.0 just creates the directory if it's missing.
I'm not really sure if it's a good thing or not, but I guess I moved
that part of the code to be common across all storages, so maybe it's
good. :)

> > "lots of mailboxes" should cause error to be logged.
> 
> What does the error look like?  I could search my logs for it.

Easiest would be to make Dovecot log errors to a different file
(log_path to different than info_log_path). Then you wouldn't have to
search for errors, simply see if anything shows up in the error log. If
you want to grep, I suppose anything with Error: or Fatal: or Panic:
prefix would do.

> > > 7. I tried to create an IMAP search, rather than a virtual folder,
> > >   that looked for x-mailbox INBOX header like the virtual folder
> > >   does.  It too came up empty.  It doesn't exactly surprise me
> > >   because I don't see an x-mailbox header in any of these messages.
> > 
> > x-mailbox doesn't use a header, it uses the actual mailbox name
> > where the message exists.
> 
> Oh... what if the message exists in multiple mailboxes?  Typically
> anything in INBOX can also be found in my "all mail" archive.
..
> I guess that means it's crucial that, whatever else I do, INBOX should
> be part of the "all mail" virtual folder in order for this to work.  I
> didn't quite understand that before, and I think it's important to
> have a description somewhere of how x-mailbox works that would help me
> to get to that conclusion.  Certainly the existing description at
> http://wiki.dovecot.org/Plugins/Virtual that it represents the
> "original" mailbox isn't adequate for that purpose.

Hmm. If you're making a virtual mailbox out of another virtual mailbox,
I'm not entirely sure what X-MAILBOX should do. Currently it returns the
parent virtual mailbox's name. But should it instead return the mailbox
name that physically contains the mails? I'm beginning to think that it
probably should. I'll change that in v2.0.

> > > Lastly, I want to report that for me, the speed of these IMAP searches
> > > doesn't begin to approach what I can do with mairix if I want a
> > > message's entire thread.  I'm happy to use mairix, although it would
> > > be nicer not to duplicate that capability if it's already in the
> > > server.  But it needs to be fast.  Is there something I can do to make
> > > it go faster?
> > 
> > How slow is it? How many messages do you have? I think it should be pretty fast.
> 
> 150523 messages in 2.6G
> 
> > Anyway, dovecot -n output and contents of dovecot-virtual file could help figuring out what your problem is.
> 
> Having got the x-mailbox insight (I think---thanks), I have tried 
> the following combinations:
> 
>   # ~/Maildir/virtual/INBOX/dovecot-virtual
>   INBOX
>   zz_archive*
>     inthread refs x-mailbox inbox
> 
> shows only messages in INBOX. THe archives are actually in folders
> like zz_archive.2010.01

Is this fast or slow then?

>   # ~/Maildir/virtual/all/dovecot-virtual
>   *
>     all
> 
>   # ~/Maildir/virtual/INBOX/dovecot-virtual
>   virtual.all
>     inthread refs x-mailbox inbox
> 
> Appears to hang

It's probably going to take a while to build the thread index. After
that it should be pretty fast. It would be easier to see how far it gets
by talking imap:

a login username password
b select virtual.all
c thread refs all
d select virtual.INBOX

After the c) command is finished, the thread thread index is built and
d) command should be fast enough. In my tests with 40k messages the d)
command takes two seconds. Most of that time goes to opening
virtual.all, which in turn is slow because of the number of mailboxes
and could be made faster by mailbox list indexes.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
Url : http://dovecot.org/pipermail/dovecot/attachments/20100119/41bde49c/attachment.bin