[Dovecot] Using tags instead of folders?
Hi,
currently most people store their messages in some sort of folder hierarchy, so that they can find them later easily. Here, for example, mails that are connected to a project are being stored in /CompanyName/ProjectNumber folders.
Now I was wondering if there is any reason against using tags instead of a folder hierarchy, because tags are better when it comes to mails that belong to multiple projects (maybe even belonging to multiple companies) at once. All mails could then be stored in one folder/mailbox, fulltext search indexing would work well. You could search for a tag that consists of the ProjectNumber and would get all mails that belong to that project, just as if you would have opened that folder in the old hierarchy. You could save this search as a 'Search Folder' (talking in Thunderbird terms).
Then I looked at how Thunderbird implements those tags when using IMAP. It stores them as IMAP keywords. Dovecot stores the first 26 flags in the filename, because it will only use one letter. Mapping between client's tag name and the letter is being done in dovecot-keywords files (one for each mailbox). If there are more IMAP keywords than 26, they will be stored in the dovecot.index file. (http://wiki.dovecot.org/MailboxFormat/Maildir)
Now imagine we want to have 1000s (maybe 10000s) of different tags (=IMAP keywords) - instead of folders. Can this be recommended? What happens if the index file gets corrupted for some reason? (I had to delete dovecot.index files before...). What happens if the Thunderbird profile breaks? What happens if we want to move on to another client?
Are there other ways to implement 'tagging' instead of a hierarchy?
Patrick.
-- STAR Software (Shanghai) Co., Ltd. http://www.star-group.net/ Phone: +86 (21) 3462 7688 x 826 Fax: +86 (21) 3462 7779
PGP key: https://stshacom1.star-china.net/keys/patrick_nagel.asc Fingerprint: E09A D65E 855F B334 E5C3 5386 EF23 20FC E883 A005
On Wed, 2008-07-02 at 15:56 +0800, Patrick Nagel wrote:
Then I looked at how Thunderbird implements those tags when using IMAP. It stores them as IMAP keywords. Dovecot stores the first 26 flags in the filename, because it will only use one letter. Mapping between client's tag name and the letter is being done in dovecot-keywords files (one for each mailbox). If there are more IMAP keywords than 26, they will be stored in the dovecot.index file. (http://wiki.dovecot.org/MailboxFormat/Maildir)
Now imagine we want to have 1000s (maybe 10000s) of different tags (=IMAP keywords) - instead of folders. Can this be recommended?
Dovecot is currently optimized to handle a rather small number of keywords. They're stored as a bitmask, so using 1000 different keywords would use 125 bytes per each mail. If the keyword usage gets this high it would be a good idea to change the implementation to store them differently.
You're also concentrating on only maildir format. Yes, there the limit is 26 keywords and I don't have plans to improve it. But for example with dbox format there is no such limit.
What happens if the index file gets corrupted for some reason? (I had to delete dovecot.index files before...).
v1.1 tries very hard to fix corrupted dovecot.index files and it also ignores errors in dovecot.index.log file. Also there have been rarely problems with those files. dovecot.index.cache file has been nearly always the problem and it can always be safely deleted.
What happens if the Thunderbird profile breaks? What happens if we want to move on to another client?
AFAIK Thunderbird currently supports only 5 keywords and it names them $Lablel1 .. $Label5. Not very portable.
Are there other ways to implement 'tagging' instead of a hierarchy?
With v1.2 it'll be possible to create virtual mailboxes. Each keyword could have its corresponding virtual mailbox. Pretty much like in gmail.
Hi,
On Wednesday 02 July 2008, Timo Sirainen wrote:
On Wed, 2008-07-02 at 15:56 +0800, Patrick Nagel wrote:
Then I looked at how Thunderbird implements those tags when using IMAP. It stores them as IMAP keywords. Dovecot stores the first 26 flags in the filename, because it will only use one letter. Mapping between client's tag name and the letter is being done in dovecot-keywords files (one for each mailbox). If there are more IMAP keywords than 26, they will be stored in the dovecot.index file. (http://wiki.dovecot.org/MailboxFormat/Maildir)
Now imagine we want to have 1000s (maybe 10000s) of different tags (=IMAP keywords) - instead of folders. Can this be recommended?
Dovecot is currently optimized to handle a rather small number of keywords. They're stored as a bitmask, so using 1000 different keywords would use 125 bytes per each mail. If the keyword usage gets this high it would be a good idea to change the implementation to store them differently.
You're also concentrating on only maildir format. Yes, there the limit is 26 keywords and I don't have plans to improve it. But for example with dbox format there is no such limit.
I see. I kind of like maildir (and thought it was "recommended").
What happens if the index file gets corrupted for some reason? (I had to delete dovecot.index files before...).
v1.1 tries very hard to fix corrupted dovecot.index files and it also ignores errors in dovecot.index.log file. Also there have been rarely problems with those files. dovecot.index.cache file has been nearly always the problem and it can always be safely deleted.
Ok. I don't remember the details - maybe deleting dovecot.index.cache would have been enough.
What happens if the Thunderbird profile breaks? What happens if we want to move on to another client?
AFAIK Thunderbird currently supports only 5 keywords and it names them $Lablel1 .. $Label5. Not very portable.
No, those $label1...5 are the default five tags. You can add more (although I didn't check if there is actually a (low) limit. May well be. One of our users already has something like 20 IMAP keywords in various folders/mailboxes, and she uses Thunderbird and its tags. Tags that are created by the user get a name that somehow resembles the tag's name in Thunderbird. Example: a tag named "Content Check" in Thunderbird will end up as "content_check" in dovecot-keywords.
Are there other ways to implement 'tagging' instead of a hierarchy?
With v1.2 it'll be possible to create virtual mailboxes. Each keyword could have its corresponding virtual mailbox. Pretty much like in gmail.
Yeah, this is the second time those virtual mailboxes would come in handy for me ;) Do you have a roadmap for 1.2? :D
Patrick.
-- STAR Software (Shanghai) Co., Ltd. http://www.star-group.net/ Phone: +86 (21) 3462 7688 x 826 Fax: +86 (21) 3462 7779
PGP key: https://stshacom1.star-china.net/keys/patrick_nagel.asc Fingerprint: E09A D65E 855F B334 E5C3 5386 EF23 20FC E883 A005
On 7/2/2008, Timo Sirainen (tss@iki.fi) wrote:
You're also concentrating on only maildir format. Yes, there the limit is 26 keywords and I don't have plans to improve it. But for example with dbox format there is no such limit.
Cool! But three dbox questions:
Do you consider dbox to be ready for prime time, in a production environment?
Related to first question - do you foresee any major changes to its internals, that would result in potentially painful updates in the future, for anyone desiring to use it in production now?
and
- Where is documentation on configuration settings? There is nothing on the wiki under Mailbox location configuration where maildir and mbox are specifically discussed, I only found one page where it is mentioned (Mailbox formats), and its own page, but there is nothing about actual configuration settings.
The only caveat I can surmise is that, since nothing else [currently] supports dbox format except dovecot, you'd have to use dovecot's LDA - which is not a problem, actually desirable for index updates, but it would be good to see some official docs on how to set it [dbox] up correctly.
Tia Timo,
--
Best regards,
Charles
On Wed, 2008-07-02 at 07:41 -0400, Charles Marcus wrote:
On 7/2/2008, Timo Sirainen (tss@iki.fi) wrote:
You're also concentrating on only maildir format. Yes, there the limit is 26 keywords and I don't have plans to improve it. But for example with dbox format there is no such limit.
Cool! But three dbox questions:
- Do you consider dbox to be ready for prime time, in a production environment?
I'm not aware of any bugs, but I'm also not aware of any real users..
- Related to first question - do you foresee any major changes to its internals, that would result in potentially painful updates in the future, for anyone desiring to use it in production now?
No.
- Where is documentation on configuration settings? There is nothing on the wiki under Mailbox location configuration where maildir and mbox are specifically discussed, I only found one page where it is mentioned (Mailbox formats), and its own page, but there is nothing about actual configuration settings.
The rotation settings are currently do nothing, since currently it supports only one file = one message. So there are currently no dbox settings.
The only caveat I can surmise is that, since nothing else [currently] supports dbox format except dovecot, you'd have to use dovecot's LDA - which is not a problem, actually desirable for index updates, but it would be good to see some official docs on how to set it [dbox] up correctly.
I'm not really sure what it would say. :)
Timo Sirainen schrieb:
On Wed, 2008-07-02 at 07:41 -0400, Charles Marcus wrote:
On 7/2/2008, Timo Sirainen (tss@iki.fi) wrote:
You're also concentrating on only maildir format. Yes, there the limit is 26 keywords and I don't have plans to improve it. But for example with dbox format there is no such limit. Cool! But three dbox questions:
- Do you consider dbox to be ready for prime time, in a production environment?
I'm not aware of any bugs, but I'm also not aware of any real users..
- Related to first question - do you foresee any major changes to its internals, that would result in potentially painful updates in the future, for anyone desiring to use it in production now?
No.
- Where is documentation on configuration settings? There is nothing on the wiki under Mailbox location configuration where maildir and mbox are specifically discussed, I only found one page where it is mentioned (Mailbox formats), and its own page, but there is nothing about actual configuration settings.
The rotation settings are currently do nothing, since currently it supports only one file = one message. So there are currently no dbox settings.
The only caveat I can surmise is that, since nothing else [currently] supports dbox format except dovecot, you'd have to use dovecot's LDA - which is not a problem, actually desirable for index updates, but it would be good to see some official docs on how to set it [dbox] up correctly.
I'm not really sure what it would say. :)
Hi Timo, perhaps it would be helpfull to have more examples with dbox in the wiki, where examples with maildir are stated too even if there is no difference about use , that just for keep away some setup unsureness, and does the wiki more complete if using dbox by the users
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
On 7/12/2008, Timo Sirainen (tss@iki.fi) wrote:
The only caveat I can surmise is that, since nothing else [currently] supports dbox format except dovecot, you'd have to use dovecot's LDA - which is not a problem, actually desirable for index updates, but it would be good to see some official docs on how to set it [dbox] up correctly.
I'm not really sure what it would say. :)
How about something like:
"Use me! Abuse me! I need some testing!" ;)
--
Best regards,
Charles
On 7/12/2008, Charles Marcus (CMarcus@Media-Brokers.com) wrote:
The only caveat I can surmise is that, since nothing else [currently] supports dbox format except dovecot, you'd have to use dovecot's LDA - which is not a problem, actually desirable for index updates, but it would be good to see some official docs on how to set it [dbox] up correctly.
I'm not really sure what it would say. :)
How about something like:
"Use me! Abuse me! I need some testing!" ;)
Oh.. and thanks for the answers, and welcome back - hope you had an enjoyable trip to India...
--
Best regards,
Charles
On 7/2/2008 4:23 AM, Timo Sirainen wrote:
With v1.2 it'll be possible to create virtual mailboxes. Each keyword could have its corresponding virtual mailbox. Pretty much like in gmail.
I'm curious about how this will be controlled from the users/client standpoint?
I'm guessing there would have to be some kind of support in the client to be able to create/define/manipulate the virtual folders?
Hopefully an extension could be created in Thunderbird to do this... maybe even with a little help from Grand Master Timo (since you are most familiar with the best way for a client to talk to dovecot)? ;)
Thanks Timo... 1.1.x is really working well... so much better than 1.0.x
- which was already so much better than courier-imap. I'm still trying to convince my biggest client to let me change them over to dovecot - but when they finally let me, they're gonna love the difference...
--
Best regards,
Charles
participants (4)
-
Charles Marcus
-
Patrick Nagel
-
Robert Schetterer
-
Timo Sirainen