[Dovecot] IMAP - many folders?
Hi,
I've been reading the list and it seems the answer to my (soon to be asked question) is no, at least with 0.99, but I'll ask to be sure.
I have following setup: new mail goes to ~/Mail/Maildir, except for mailing lists which are moved by procmail to ~/Mail/mailing_lists/name, spam which goes to ~/Mail/spam. Read mail is moved by mutt to ~/Mail/read. Everything is in maildir format.
Is it possibile to configure dovecot in such a way to allow user to access all those mail folders thorough imap/pop? User can access ~/Mail/Maildir by setting default_mail_env, but so far nothing else.
Piotr Krukowiecki wrote:
Hi,
I've been reading the list and it seems the answer to my (soon to be asked question) is no, at least with 0.99, but I'll ask to be sure.
I have following setup: new mail goes to ~/Mail/Maildir, except for mailing lists which are moved by procmail to ~/Mail/mailing_lists/name, spam which goes to ~/Mail/spam. Read mail is moved by mutt to ~/Mail/read. Everything is in maildir format.
Is it possibile to configure dovecot in such a way to allow user to access all those mail folders thorough imap/pop? User can access ~/Mail/Maildir by setting default_mail_env, but so far nothing else.
Piotr,
do you want your users to see their own directory heirarchy as explained above? or to see your folders specifically?
if you want them to see their own folders in their home mail heirarchy, then you need to make sure your mail client is configured to access them. This is handled by subscriptions and some mail clients will auto-subscribe to all existing folders the first time it connects to a server (i.e. you setup the account) and some don't and you need to specifically tell the client to subscribe.
This sounds like what your problem may be. which mail client are you using?
hope this helps
alan
On Thu, Nov 18, 2004 at 12:29:13PM +0900, alan premselaar wrote:
Piotr Krukowiecki wrote:
I have following setup: new mail goes to ~/Mail/Maildir, except for mailing lists which are moved by procmail to ~/Mail/mailing_lists/name, spam which goes to ~/Mail/spam. Read mail is moved by mutt to ~/Mail/read. Everything is in maildir format.
do you want your users to see their own directory heirarchy as explained above? or to see your folders specifically?
if you want them to see their own folders in their home mail heirarchy, then you need to make sure your mail client is configured to access them. This is handled by subscriptions and some mail clients will auto-subscribe to all existing folders the first time it connects to a server (i.e. you setup the account) and some don't and you need to specifically tell the client to subscribe.
This sounds like what your problem may be. which mail client are you using?
Personally I'm using mutt. Some users use Mozilla, some pine, some MS something. I'll tell you how I "use" my mail.
When I use mutt from the computer which holds the mail (thorough ssh), I usually use "c<tab>" and get a listing of ~/Mail/ directory, which looks e.g. like this: (cut to fit the screen) [...] 2 drwxr-xr-x [...] 168 Nov 18 08:47 mailing_lists/ 3 drwxr-xr-x [...] 224 Nov 18 08:06 debian_mailing_list 4 drwx------ [...] 120 Nov 17 18:36 Maildir/ [...]
I can see the last changed date, which allows me to see when was the last mail delivered.
Then I can e.g. change to do mailing_lists dir and see directories there etc. untill i choose a maildir or mbox item.
I haven't been using imap/pop at all yet, but I suppose when using it I'd like to be able to do the same, but from home. I.e. using mutt to list directories on remote imap server. I'm not sure if it can be done?
If not, then at least I'd like to be able to access e.g. Maildir, mailing_lists/dovecot ;) and some others configured folders. The problem here is that even if I pass mailing_lists/dovecot as accessed folder the dovecot(?) changes "/" to ".". I've read that I'd have to change the slash to dot in dovecot sources - is is very hard to do?
I think that other users will use other means of reading their mail, but I still belive that "/" approach is better then the "." one.
Piotr Krukowiecki wrote:
On Thu, Nov 18, 2004 at 12:29:13PM +0900, alan premselaar wrote:
Piotr Krukowiecki wrote:
I have following setup: new mail goes to ~/Mail/Maildir, except for mailing lists which are moved by procmail to ~/Mail/mailing_lists/name, spam which goes to ~/Mail/spam. Read mail is moved by mutt to ~/Mail/read. Everything is in maildir format.
do you want your users to see their own directory heirarchy as explained above? or to see your folders specifically?
if you want them to see their own folders in their home mail heirarchy, then you need to make sure your mail client is configured to access them. This is handled by subscriptions and some mail clients will auto-subscribe to all existing folders the first time it connects to a server (i.e. you setup the account) and some don't and you need to specifically tell the client to subscribe.
This sounds like what your problem may be. which mail client are you using?
Personally I'm using mutt. Some users use Mozilla, some pine, some MS something. I'll tell you how I "use" my mail.
When I use mutt from the computer which holds the mail (thorough ssh), I usually use "c<tab>" and get a listing of ~/Mail/ directory, which looks e.g. like this: (cut to fit the screen) [...] 2 drwxr-xr-x [...] 168 Nov 18 08:47 mailing_lists/ 3 drwxr-xr-x [...] 224 Nov 18 08:06 debian_mailing_list 4 drwx------ [...] 120 Nov 17 18:36 Maildir/ [...]
I can see the last changed date, which allows me to see when was the last mail delivered.
Then I can e.g. change to do mailing_lists dir and see directories there etc. untill i choose a maildir or mbox item.
I haven't been using imap/pop at all yet, but I suppose when using it I'd like to be able to do the same, but from home. I.e. using mutt to list directories on remote imap server. I'm not sure if it can be done?
If not, then at least I'd like to be able to access e.g. Maildir, mailing_lists/dovecot ;) and some others configured folders. The problem here is that even if I pass mailing_lists/dovecot as accessed folder the dovecot(?) changes "/" to ".". I've read that I'd have to change the slash to dot in dovecot sources - is is very hard to do?
I think that other users will use other means of reading their mail, but I still belive that "/" approach is better then the "." one.
Piotr,
the '.' character is used as an IMAP delimiter used by dovecot (and some other IMAP servers, although it's been awhile since i've used others so I don't remember which off the top of my head)
so, if you want sub folders to show up in your IMAP client, you'll need a heirarchy similar to (assuming you're using Maildir format, each of these directories will have 3 additional sub-directories called cur/ new/ and tmp/):
~/Maildir/.INBOX/ ~/Maildir/.some_folder/ ~/Maildir/.some_folder.sub_folder/
etc.
this will show up as:
INBOX some_folder _sub_folder
in your IMAP client (i'm not familiar with mutt or its IMAP capabilities, but for example thunderbird will work this way)
hope this helps
alan
On Thu, Nov 18, 2004 at 05:53:49PM +0900, alan premselaar wrote:
the '.' character is used as an IMAP delimiter used by dovecot (and some other IMAP servers, although it's been awhile since i've used others so I don't remember which off the top of my head)
so, if you want sub folders to show up in your IMAP client, you'll need a heirarchy similar to (assuming you're using Maildir format, each of these directories will have 3 additional sub-directories called cur/ new/ and tmp/):
~/Maildir/.INBOX/ ~/Maildir/.some_folder/ ~/Maildir/.some_folder.sub_folder/
No. I want ~/Mail/Maildir/ as a folder for new, incomming mail, and "/" as a delimiter. So my question is "how hard would it be to change sources to make '/' to be default delimiter"?
this will show up as:
INBOX some_folder _sub_folder
in your IMAP client (i'm not familiar with mutt or its IMAP capabilities, but for example thunderbird will work this way)
Yes, mutt as IMAP client will work with this as well (or any other delimiter - it's configurable iirc).
But if I use this approach I'll have 20+ entries in ~/Mail/ directory and that would inconvinient to use with mutt as a local mail (not IMAP) reader (with 'c<tab>').
That's why I want my mail nicely sorted in subdirs, not everything in one directory.
Piotr Krukowiecki wrote:
So my question is "how hard would it be to change sources to make '/' to be default delimiter"?
Very.
Maildir specifically uses .folder.folder names so that the IMAP server can find _all_ the folders by reading a single top-level directory. If sub-folders were stored ../folder/folder, then the IMAP server would have to scan the entire mail directory (in my case, several gigabytes) every time it started up, so it could find the list of folders.
On Mon, Nov 22, 2004 at 11:49:27AM +1100, Gregory Bond wrote:
Piotr Krukowiecki wrote:
So my question is "how hard would it be to change sources to make '/' to be default delimiter"?
Maildir specifically uses .folder.folder names so that the IMAP server can find _all_ the folders by reading a single top-level directory. If sub-folders were stored ../folder/folder, then the IMAP server would have to scan the entire mail directory (in my case, several gigabytes) every time it started up, so it could find the list of folders.
I see the point...
Scanning only top level directory and not scanning sub directories, but allowing user to access them if they specify them directly would be some kind of solution.
Is it possibile to set default maildir to ~/Mail/Maildir/ and to use ~/Mail as a containter for all .folder directories? I couldn't achieve it. That would help me a little.
On Mon, Nov 22, 2004 at 11:49:27AM +1100, Gregory Bond wrote:
Piotr Krukowiecki wrote:
So my question is "how hard would it be to change sources to make '/' to be default delimiter"?
Very.
Maildir specifically uses .folder.folder names so that the IMAP server can find _all_ the folders by reading a single top-level directory. If sub-folders were stored ../folder/folder, then the IMAP server would have to scan the entire mail directory (in my case, several gigabytes) every time it started up, so it could find the list of folders.
Then again, the imap delimeter and how the files are stored are two different things. One could map a different delimiter into the same storage layout, except that you'd then have to deal with the case of the storage delimeter appearing in folder names. But there are various ways to deal with that.
mm
Mark E. Mallett wrote:
Then again, the imap delimeter and how the files are stored are two different things. One could map a different delimiter into the same storage layout,
True - but the OP was asking about the actual physical directories, because he wants to access them from outside IMAP (mutt & procmail).
On Mon, 2004-11-22 at 11:49 +1100, Gregory Bond wrote:
Piotr Krukowiecki wrote:
So my question is "how hard would it be to change sources to make '/' to be default delimiter"?
Very.
Maildir specifically uses .folder.folder names so that the IMAP server can find _all_ the folders by reading a single top-level directory. If sub-folders were stored ../folder/folder, then the IMAP server would have to scan the entire mail directory (in my case, several gigabytes) every time it started up, so it could find the list of folders.
The overhead isn't fantastic.
Maildir/.foo.Baz/ {new,cur,tmp}
Means being unable to rename folders (and all children) atomically.
Maildir/.foo/.Baz/ {new,cur,tmp}
Means a few more opendir/closedir operations.
Because messages are still in cur/new/tmp, even if you have gigabytes of email dovecot (nor your email client) doesn't have to scan any of it to get your folder hierarchy.
Furthermore, if your IMAP client is smart (and won't send LSUB "" *) then splitting them up could actually be faster, as you don't have to load a list of ALL folders to find out what the children folders of INBOX.Work.Office13 are.
On 23.11.2004, at 00:00, Geo Carncross wrote:
The overhead isn't fantastic.
Maildir/.foo.Baz/ {new,cur,tmp}
Means being unable to rename folders (and all children) atomically.
Yep, annoying that.
Maildir/.foo/.Baz/ {new,cur,tmp}
Means a few more opendir/closedir operations.
At some point I'll probably implement mailbox list indexes which could reduce them.
I've also some plans to separate the directory layout and mailbox formats so you could use either one, or do some stranger things such as mboxes in maildir++ layout.
Furthermore, if your IMAP client is smart (and won't send LSUB "" *) then splitting them up could actually be faster, as you don't have to load a list of ALL folders to find out what the children folders of INBOX.Work.Office13 are.
Actually with LSUB Dovecot doesn't scan the directories at all (or 1.0-tests at least won't), it just lists what is in subscriptions file.
Hi,
At 04:55 p.m. 3/12/2004, Timo Sirainen wrote:
Means a few more opendir/closedir operations.
At some point I'll probably implement mailbox list indexes which could reduce them.
I've also some plans to separate the directory layout and mailbox formats so you could use either one, or do some stranger things such as mboxes in maildir++ layout.
What I would muchly like to see soon, is a means of forcing the re-indexing to occur at a convenient time, rather than when the Maildir is first read. Ideally also as part of this, a way of manually checking the validity of the index files and then recreating them as required.
It'd make testing of the -test releases much easier, as I could wipe all the indexes and recreate them straight away without having the performance hit when the users first re-read their mailboxes.
Reuben
On 3.12.2004, at 07:19, Reuben Farrelly wrote:
What I would muchly like to see soon, is a means of forcing the re-indexing to occur at a convenient time, rather than when the Maildir is first read. Ideally also as part of this, a way of manually checking the validity of the index files and then recreating them as required.
It'd make testing of the -test releases much easier, as I could wipe all the indexes and recreate them straight away without having the performance hit when the users first re-read their mailboxes.
Is the performance hit so large with 1.0-test anymore? 0.99.x would re-read all messages and cause lots of I/O, but 1.0-tests try to delay reading as long as possible.
Yes - every folder that I then go into after wiping the indexes, requires a full resync between client (Eudora) and dovecot of the headers (not downloading the message bodies). The hard drive on server works pretty hard for a while to resync up especially if there are a fair few messages, but after that initial performance hit the performance is good. Obviously it only reindexes the folders as they are opened.
It could just be a slow client, but I doubt it given the level of hard drive activity that goes on on the server. At a guess, it would be re-indexing perhaps, 30 mails/second, and it takes a while with 3000 messages in a (trash) folder. I hate to think how long it would take with 100 clients all resyncing at the same time after an index wipe or migration from an existing system to a new dovecot installation...
reuben
At 04:36 p.m. 5/12/2004, you wrote:
On 3.12.2004, at 07:19, Reuben Farrelly wrote:
What I would muchly like to see soon, is a means of forcing the re-indexing to occur at a convenient time, rather than when the Maildir is first read. Ideally also as part of this, a way of manually checking the validity of the index files and then recreating them as required.
It'd make testing of the -test releases much easier, as I could wipe all the indexes and recreate them straight away without having the performance hit when the users first re-read their mailboxes.
Is the performance hit so large with 1.0-test anymore? 0.99.x would re-read all messages and cause lots of I/O, but 1.0-tests try to delay reading as long as possible.
On 5.12.2004, at 10:11, Reuben Farrelly wrote:
Yes - every folder that I then go into after wiping the indexes, requires a full resync between client (Eudora) and dovecot of the headers (not downloading the message bodies). The hard drive on server works pretty hard for a while to resync up especially if there are a fair few messages, but after that initial performance hit the performance is good. Obviously it only reindexes the folders as they are opened.
Do you remove dovecot-uidlist as well? If you don't, clients shouldn't do a full resync because UIDs don't change then.
Ah. Yes that explains it, I've been removing ~/Maildir/.*/dovecot* :(
Thanks for the tip!
reuben
At 05:09 a.m. 6/12/2004, Timo Sirainen wrote:
On 5.12.2004, at 10:11, Reuben Farrelly wrote:
Yes - every folder that I then go into after wiping the indexes, requires a full resync between client (Eudora) and dovecot of the headers (not downloading the message bodies). The hard drive on server works pretty hard for a while to resync up especially if there are a fair few messages, but after that initial performance hit the performance is good. Obviously it only reindexes the folders as they are opened.
Do you remove dovecot-uidlist as well? If you don't, clients shouldn't do a full resync because UIDs don't change then.
Gregory Bond wrote:
Piotr Krukowiecki wrote:
So my question is "how hard would it be to change sources to make '/' to be default delimiter"?
Very.
Maildir specifically uses .folder.folder names so that the IMAP server can find _all_ the folders by reading a single top-level directory.
This is not how standard Maildir works. The .folder.folder notation is an extension introduced by Courier-IMAP.
Lars Hansson
On Nov 23, 2004, at 13:44, Lars Hansson wrote:
Gregory Bond wrote:
Piotr Krukowiecki wrote:
So my question is "how hard would it be to change sources to make '/' to be default delimiter"?
Very. Maildir specifically uses .folder.folder names so that the IMAP server can find _all_ the folders by reading a single top-level directory.
This is not how standard Maildir works. The .folder.folder notation is an extension introduced by Courier-IMAP.
Courier-IMAP introduced Maildir++ which enhance Maildir with folders and quotas. url:http://www.inter7.com/courierimap/README.maildirquota.html
Is it possible to create folders using Maildir?
participants (9)
-
alan premselaar
-
Bertrand LUPART
-
Geo Carncross
-
Gregory Bond
-
Lars Hansson
-
Mark E. Mallett
-
piotr＠mallorn.ii.uj.edu.pl
-
Reuben Farrelly
-
Timo Sirainen