[Dovecot] Question about mbox_snarf and dovecot2.0
Hello, I'm running revision 10622:de9d6dae7fe5 on AIX 5.3 with some local mods for our inbox hash function. I'm having a problem doing a "select inbox" when I use the mbox_snarf plugin. When I run truss on the process I see the following stat calls:
614528: kread(9, " 1 s e l e c t i n b".., 4096) = 15 614528: statx("/gpfs/inbox/14/tstem38", 0x0FFFFFFFFFFFF2F8, 176, 0) = 0 614528: statx("/gpfs/users/t/s/tstem38/IMAP//gpfs/inbox/14/.imap/tstem38", 0x0FFFFFFFFFFFED98, 176, 0) Err#2 ENOENT 614528: statx("/gpfs/users/t/s/tstem38/IMAP//gpfs/inbox/14/tstem38", 0x0FFFFFFFFFFFEDD8, 176, 0) Err#2 ENOENT 614528: statx("/gpfs/users/t/s/tstem38/IMAP", 0x0FFFFFFFFFFFEC88, 176, 0) = 0 614528: statx("/gpfs/users/t/s/tstem38/IMAP//gpfs/inbox/14/tstem38", 0x0FFFFFFFFFFFEDD8, 176, 0) Err#2 ENOENT 614528: statx("/gpfs/users/t/s/tstem38/IMAP", 0x0FFFFFFFFFFFEC88, 176, 0) = 0
and the client gets: 1 NO [CANNOT] Invalid mailbox name
For pathing I have: plugin.conf: mbox_snarf = /gpfs/inbox/%Ju/%u and mail.conf:mail_location = mbox:%h:INBOX=/var/spool/mail/%Ju/%u
I verified that not using the mbox_snarf works just fine. I have no namespaces enabled.
Thanks, Jonathan
On Tue, 2010-02-02 at 12:33 -0500, Jonathan Siegle wrote:
I verified that not using the mbox_snarf works just fine. I have no namespaces enabled.
The code contains:
//FIXME:storage->set->mail_full_filesystem_access = TRUE;
IIRC I couldn't figure out a nice and easy way to actually make that work. Maybe the whole plugin could be redesigned. Probably by creating a configuration like:
namespace { location = mbox:~/mail .. } namespace { prefix = ~~MboxSnarf~~/ # or whatever unique string you can think of location = mbox:~/mail:INBOX=/var/mail/%u list = no hidden = yes } plugin { mbox_snarf = ~~MboxSnarf~~/ }
On Feb 2, 2010, at 12:47 PM, Timo Sirainen wrote:
On Tue, 2010-02-02 at 12:33 -0500, Jonathan Siegle wrote:
I verified that not using the mbox_snarf works just fine. I have no namespaces enabled.
The code contains:
//FIXME:storage->set->mail_full_filesystem_access = TRUE;
IIRC I couldn't figure out a nice and easy way to actually make that work. Maybe the whole plugin could be redesigned. Probably by creating a configuration like:
namespace { location = mbox:~/mail .. } namespace { prefix = ~~MboxSnarf~~/ # or whatever unique string you can think of location = mbox:~/mail:INBOX=/var/mail/%u list = no hidden = yes } plugin { mbox_snarf = ~~MboxSnarf~~/ }
First, I haven't found a working config to use for dovecot2.0 and mbox_snarf.
As I need this to function, I've been thinking about your words above and been reading http://wiki.dovecot.org/Design/Storage/Plugins. The mentality of it reminds me of the lazy-expunge-plugin, but this API is taking some time getting used to. Any pointers would be appreciated.
Thanks, Jonathan
On Thu, 2010-04-29 at 11:11 -0400, Jonathan Siegle wrote:
namespace { location = mbox:~/mail .. } namespace { prefix = ~~MboxSnarf~~/ # or whatever unique string you can think of location = mbox:~/mail:INBOX=/var/mail/%u list = no hidden = yes } plugin { mbox_snarf = ~~MboxSnarf~~/ }
I guess my idea above was that mbox_snarf would:
- Do nothing special when opening the mailbox (unlike now), except add
hooks to sync_init and mailbox_close.
- when mbox_snarf_sync_init() is called the first time:
- find mbox_snarf namespace
- open INBOX from that namespace and keep it open
- goto 3)
- every time mbox_snarf_sync_init() is called:
- call mbox_snarf()
- on mailbox_close() close the snarf mailbox
As I need this to function, I've been thinking about your words above and been reading http://wiki.dovecot.org/Design/Storage/Plugins. The mentality of it reminds me of the lazy-expunge-plugin, but this API is taking some time getting used to. Any pointers would be appreciated.
Maybe the above helps? That Plugins wiki page probably won't help you much, since the current mbox_snarf already sets up the plugin stuff correctly. "Overriding methods" is probably the only important part there. The other Design/* pages could be more helpful.
On Apr 29, 2010, at 1:27 PM, Timo Sirainen wrote:
On Thu, 2010-04-29 at 11:11 -0400, Jonathan Siegle wrote:
namespace { location = mbox:~/mail .. } namespace { prefix = ~~MboxSnarf~~/ # or whatever unique string you can think of location = mbox:~/mail:INBOX=/var/mail/%u list = no hidden = yes } plugin { mbox_snarf = ~~MboxSnarf~~/ }
I guess my idea above was that mbox_snarf would:
- Do nothing special when opening the mailbox (unlike now), except add
hooks to sync_init and mailbox_close.
Ok gotcha.
- when mbox_snarf_sync_init() is called the first time:
- find mbox_snarf namespace
Know how to do that.
- open INBOX from that namespace and keep it open
Ok sync init would open /var/mail/%u
- goto 3)
- every time mbox_snarf_sync_init() is called:
- call mbox_snarf()
Oh Ok. So mbox_snarf would open the SNARF FILE and do the move and then return it like it does now.
- on mailbox_close() close the snarf mailbox
As I need this to function, I've been thinking about your words above and been reading http://wiki.dovecot.org/Design/Storage/Plugins. The mentality of it reminds me of the lazy-expunge-plugin, but this API is taking some time getting used to. Any pointers would be appreciated.
Maybe the above helps? That Plugins wiki page probably won't help you much, since the current mbox_snarf already sets up the plugin stuff correctly. "Overriding methods" is probably the only important part there. The other Design/* pages could be more helpful.
Yeah your thoughts help here. Thanks. Do you have a working example of mailbox_opened ? Would that solve my IDLE problem(doesn't stat /var/spool at all so no new messages)?
-Jonathan
On Thu, 2010-04-29 at 14:00 -0400, Jonathan Siegle wrote:
- open INBOX from that namespace and keep it open
Ok sync init would open /var/mail/%u
Well, you would simply call mailbox_open(ns->list, "INBOX"). Dovecot internally then figures out that INBOX is /var/mail/%u based on the namespace's configuration.
- every time mbox_snarf_sync_init() is called:
- call mbox_snarf()
Oh Ok. So mbox_snarf would open the SNARF FILE and do the move and then return it like it does now.
I'm not sure what you mean. I was thinking it would do exactly what it does now. You give it source and dest mailboxes and it moves messages from one to the other. (Although one minor optimization could be done: If no messages are moved, don't call the final mailbox_sync()).
Also note that nothing here now assumes anything about what mailbox format is being used. The whole plugin should perhaps be renamed to "snarf", since it would work with any mailbox formats (source/destination could even be different formats).
Yeah your thoughts help here. Thanks. Do you have a working example of mailbox_opened ?
You probably want to hook into mailbox_allocated and do the method overrides there. For that there are many examples in most existing plugins, like lazy_expunge.
Would that solve my IDLE problem(doesn't stat /var/spool at all so no new messages)?
You can implement that by overriding mailbox.notify_changes method and calling mailbox_notify_changes(snarf_mailbox) and implementing a callback handler that does the snarfing whenever it's called. (And also implement the notify_changes removal code.)
On Apr 29, 2010, at 2:42 PM, Timo Sirainen wrote:
On Thu, 2010-04-29 at 14:00 -0400, Jonathan Siegle wrote:
- open INBOX from that namespace and keep it open
Ok sync init would open /var/mail/%u
Well, you would simply call mailbox_open(ns->list, "INBOX"). Dovecot internally then figures out that INBOX is /var/mail/%u based on the namespace's configuration.
Ok. I couldn't find a mailbox_open that takes a list and char* as arguments, but I found a mail_namespace_find_inbox function which works well.
Also note that nothing here now assumes anything about what mailbox format is being used. The whole plugin should perhaps be renamed to "snarf", since it would work with any mailbox formats (source/destination could even be different formats).
Where do I specify the Destination mailbox in the config file? I have a source of INBOX(/var/spool). Should I just tack on SNARF on the var list?
namespace foo { prefix = ~~Snarf/ location = mbox~/mail:INBOX=/var/mail/%u:SNARF=/some/other/file list = no hidden = yes inbox = yes }
Thanks, Jonathan
On 5.5.2010, at 20.27, Jonathan Siegle wrote:
- open INBOX from that namespace and keep it open
Ok sync init would open /var/mail/%u
Well, you would simply call mailbox_open(ns->list, "INBOX"). Dovecot internally then figures out that INBOX is /var/mail/%u based on the namespace's configuration.
Ok. I couldn't find a mailbox_open that takes a list and char* as arguments, but I found a mail_namespace_find_inbox function which works well.
Oh, I meant mailbox_alloc() actually. mail_namespace_find_inbox() returns the INBOX's namespace, but I think above I meant opening the hidden snarf-INBOX. But you probably do need mail_namespace_find*() to find the snarf namespace by its configured prefix.
Also note that nothing here now assumes anything about what mailbox format is being used. The whole plugin should perhaps be renamed to "snarf", since it would work with any mailbox formats (source/destination could even be different formats).
Where do I specify the Destination mailbox in the config file? I have a source of INBOX(/var/spool). Should I just tack on SNARF on the var list?
namespace foo { prefix = ~~Snarf/ location = mbox~/mail:INBOX=/var/mail/%u:SNARF=/some/other/file list = no hidden = yes inbox = yes }
Destination mailbox is always the default INBOX (and that you can find with the mail_namespace_find_inbox()). So something like:
src = mailbox_alloc(snarf_ns->list, "INBOX"); dest = mailbox_alloc(inbox_ns->list, "INBOX");
On Thu, 2010-04-29 at 11:11 -0400, Jonathan Siegle wrote:
As I need this to function, I've been thinking about your words above and been reading http://wiki.dovecot.org/Design/Storage/Plugins. The mentality of it reminds me of the lazy-expunge-plugin, but this API is taking some time getting used to. Any pointers would be appreciated.
So I guess you never got around to implementing it? I finally did: http://dovecot.org/list/dovecot/2010-November/055020.html
On Nov 18, 2010, at 2:08 PM, Timo Sirainen wrote:
On Thu, 2010-04-29 at 11:11 -0400, Jonathan Siegle wrote:
As I need this to function, I've been thinking about your words above and been reading http://wiki.dovecot.org/Design/Storage/Plugins. The mentality of it reminds me of the lazy-expunge-plugin, but this API is taking some time getting used to. Any pointers would be appreciated.
So I guess you never got around to implementing it? I finally did: http://dovecot.org/list/dovecot/2010-November/055020.html
I did do it. I've been testing it for a few months now. Sorry. Should have said something.
participants (2)
-
Jonathan Siegle
-
Timo Sirainen