At Tue, 19 Jan 2010 16:51:04 +0200, Timo Sirainen wrote:
On Wed, 2010-01-06 at 11:50 -0500, David Abrahams wrote:
At Wed, 6 Jan 2010 14:17:40 +0200, Timo Sirainen wrote:
On 4.1.2010, at 20.47, David Abrahams wrote:
- I had to manually create the virtual folder for all my users or they couldn't access their mail at all. Is this fixed in 2.0?
No.
OK, is it considered a bug that should be addressed? IMO there should at least be a way to tell dovecot to ignore the non-existence of a particular folder.
Actually it looks like v2.0 just creates the directory if it's missing. I'm not really sure if it's a good thing or not, but I guess I moved that part of the code to be common across all storages, so maybe it's good. :)
Well, my inclination would be to ignore missing directories until you try to do something with them. If I set up a fancy virtual folder arrangement for advanced users I'd like ordinary users not to ever have to encounter it.
"lots of mailboxes" should cause error to be logged.
What does the error look like? I could search my logs for it.
Easiest would be to make Dovecot log errors to a different file (log_path to different than info_log_path). Then you wouldn't have to search for errors, simply see if anything shows up in the error log.
I'll try that, thanks.
If you want to grep, I suppose anything with Error: or Fatal: or Panic: prefix would do.
Nothing in the current logs, but they may have been rotated out.
- I tried to create an IMAP search, rather than a virtual folder, that looked for x-mailbox INBOX header like the virtual folder does. It too came up empty. It doesn't exactly surprise me because I don't see an x-mailbox header in any of these messages.
x-mailbox doesn't use a header, it uses the actual mailbox name where the message exists.
Oh... what if the message exists in multiple mailboxes? Typically anything in INBOX can also be found in my "all mail" archive. .. I guess that means it's crucial that, whatever else I do, INBOX should be part of the "all mail" virtual folder in order for this to work. I didn't quite understand that before, and I think it's important to have a description somewhere of how x-mailbox works that would help me to get to that conclusion. Certainly the existing description at http://wiki.dovecot.org/Plugins/Virtual that it represents the "original" mailbox isn't adequate for that purpose.
Hmm. If you're making a virtual mailbox out of another virtual mailbox, I'm not entirely sure what X-MAILBOX should do. Currently it returns the parent virtual mailbox's name. But should it instead return the mailbox name that physically contains the mails? I'm beginning to think that it probably should.
Why? Because fetching from there will be faster?
Having got the x-mailbox insight (I think---thanks), I have tried the following combinations:
# ~/Maildir/virtual/INBOX/dovecot-virtual INBOX zz_archive* inthread refs x-mailbox inbox
shows only messages in INBOX. THe archives are actually in folders like zz_archive.2010.01
Is this fast or slow then?
Oh, perfectly fast, but also perfectly ineffective :-)
# ~/Maildir/virtual/all/dovecot-virtual * all
# ~/Maildir/virtual/INBOX/dovecot-virtual virtual.all inthread refs x-mailbox inbox
Appears to hang
It's probably going to take a while to build the thread index. After that it should be pretty fast. It would be easier to see how far it gets by talking imap:
a login username password b select virtual.all c thread refs all d select virtual.INBOX
a login <username> <pw> a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH] Logged in b select virtual.all closed
(boom!)
I guess that explains my problem... after a fashion.
Ah, my error file contains lots of these:
Jan 21 18:06:01 IMAP(dave): Error: mmap() failed with index cache file /home/dave/Maildir/.zz_archive.2009.06/dovecot.index.cache: Cannot allocate memory Jan 21 18:06:01 IMAP(dave): Fatal: pool_system_realloc(1048576): Out of memory
Okay, I bumped up mail_process_size from 256 to 1024 and it seems not to be bailing out quite so soon.
b select virtual.all
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft unknown-3 unknown-5 unknown-6 $Forwarded $NotJunk unknown-0 unknown-2 NonJunk NotJunk gnus-forward $Junk Junk gnus-download gnus-save unknown-1 $Label1 JunkRecorded $MDNSent unknown-4 gnus-expire unknown-8 receipt-handled)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft unknown-3 unknown-5 unknown-6 $Forwarded $NotJunk unknown-0 unknown-2 NonJunk NotJunk gnus-forward $Junk Junk gnus-download gnus-save unknown-1 $Label1 JunkRecorded $MDNSent unknown-4 gnus-expire unknown-8 receipt-handled \*)] Flags permitted.
- 149262 EXISTS
- 0 RECENT
- OK [UNSEEN 1] First unseen.
- OK [UIDVALIDITY 1264126364] UIDs valid
- OK [UIDNEXT 149263] Predicted next UID
- OK [HIGHESTMODSEQ 24] Highest b OK [READ-WRITE] Select completed. c thread refs all c BAD Error in IMAP command THREAD: Missing search parameters
But, whatever is wrong with command c, it does seem to be working!
After the c) command is finished, the thread thread index is built and d) command should be fast enough. In my tests with 40k messages the d) command takes two seconds. Most of that time goes to opening virtual.all, which in turn is slow because of the number of mailboxes and could be made faster by mailbox list indexes.
Thanks for your help; I'll be playing with this and seeing how well it performs in practice.
-- Dave Abrahams Meet me at BoostCon: http://www.boostcon.com BoostPro Computing http://www.boostpro.com