[Dovecot] Proprietary mail storage.
I have a large existing read-only collection of mails packaged in individual zip files as <rfc822 file>+<some additional info>. Is it possible (and how difficult) to create a proprietary plugin (like gzip) which will open a zip file, extract mail and pass it back to Dovecot? Where do I start? If plugin is not the right approach, what is? Another question is how will this affect Dovecot performance and how to avoid any significant degradation. Any help is appreciated.Thanks.
Am 31.08.2011 16:24, schrieb Alex Cherniak:
I have a large existing read-only collection of mails packaged in individual zip files as <rfc822 file>+<some additional info>. Is it possible (and how difficult) to create a proprietary plugin (like gzip) which will open a zip file, extract mail and pass it back to Dovecot? Where do I start? If plugin is not the right approach, what is? Another question is how will this affect Dovecot performance and how to avoid any significant degradation. Any help is appreciated.Thanks.
did you looked at http://wiki2.dovecot.org/Plugins/Zlib ?
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
On 31.8.2011, at 17.24, Alex Cherniak wrote:
I have a large existing read-only collection of mails packaged in individual zip files as <rfc822 file>+<some additional info>. Is it possible (and how difficult) to create a proprietary plugin (like gzip) which will open a zip file, extract mail and pass it back to Dovecot? Where do I start? If plugin is not the right approach, what is?
Is it otherwise a Maildir? If yes, you could base your code on the zlib plugin, or perhaps more easily you could use mail-filter plugin: http://dovecot.org/patches/2.0/mail-filter.tar.gz
With mail-filter you can basically just put the messages through whatever program/script you want which gets the mail as input and outputs the wanted message body. I think the v2.0 mail-filter had some (potential?) bug, v2.1 mail-filter is anyway redesigned and should work perfectly.
Another question is how will this affect Dovecot performance and how to avoid any significant degradation.
I doubt it's going to be a problem.
Am 31.08.2011 16:52, schrieb Timo Sirainen:
On 31.8.2011, at 17.24, Alex Cherniak wrote:
I have a large existing read-only collection of mails packaged in individual zip files as <rfc822 file>+<some additional info>. Is it possible (and how difficult) to create a proprietary plugin (like gzip) which will open a zip file, extract mail and pass it back to Dovecot? Where do I start? If plugin is not the right approach, what is?
Is it otherwise a Maildir? If yes, you could base your code on the zlib plugin, or perhaps more easily you could use mail-filter plugin: http://dovecot.org/patches/2.0/mail-filter.tar.gz
are there any examples or how too online for mail-filter ?
With mail-filter you can basically just put the messages through whatever program/script you want which gets the mail as input and outputs the wanted message body. I think the v2.0 mail-filter had some (potential?) bug, v2.1 mail-filter is anyway redesigned and should work perfectly.
Another question is how will this affect Dovecot performance and how to avoid any significant degradation.
I doubt it's going to be a problem.
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
On 31.8.2011, at 18.04, Robert Schetterer wrote:
Is it otherwise a Maildir? If yes, you could base your code on the zlib plugin, or perhaps more easily you could use mail-filter plugin: http://dovecot.org/patches/2.0/mail-filter.tar.gz
are there any examples or how too online for mail-filter ?
For compiling look at the beginning of mail-filter-plugin.c
For using add it to mail_plugins and:
plugin { mail_filter_executable = /path/to/your/script.sh }
A script could be for example (totally insecure and broken):
#!/bin/sh
sed s/Hello/Hi/ > /tmp/foo cat /tmp/foo rm /tmp/foo
Unfortunately you can't both read stdin and write to stdout at the same time because of some internal Dovecot problems with it. So you'll have to write it to a temp file and then output that after the entire input is read.
Am 31.08.2011 17:14, schrieb Timo Sirainen:
On 31.8.2011, at 18.04, Robert Schetterer wrote:
Is it otherwise a Maildir? If yes, you could base your code on the zlib plugin, or perhaps more easily you could use mail-filter plugin: http://dovecot.org/patches/2.0/mail-filter.tar.gz
are there any examples or how too online for mail-filter ?
For compiling look at the beginning of mail-filter-plugin.c
For using add it to mail_plugins and:
plugin { mail_filter_executable = /path/to/your/script.sh }
A script could be for example (totally insecure and broken):
#!/bin/sh
sed s/Hello/Hi/ > /tmp/foo cat /tmp/foo rm /tmp/foo
Unfortunately you can't both read stdin and write to stdout at the same time because of some internal Dovecot problems with it. So you'll have to write it to a temp file and then output that after the entire input is read.
sorry for silly question is there any known typical usage for that , or was this on the wishlist to solve some stuff ?
Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
On 31.8.2011, at 18.17, Robert Schetterer wrote:
Is it otherwise a Maildir? If yes, you could base your code on the zlib plugin, or perhaps more easily you could use mail-filter plugin: http://dovecot.org/patches/2.0/mail-filter.tar.gz
are there any examples or how too online for mail-filter ? .. sorry for silly question is there any known typical usage for that , or was this on the wishlist to solve some stuff ?
It was originally written to be used with imapc ("imap proxy") backend. You could use it for stuff like decode encrypted PGP mails or scan for viruses and drop them if found.
Am 31.08.2011 17:21, schrieb Timo Sirainen:
On 31.8.2011, at 18.17, Robert Schetterer wrote:
Is it otherwise a Maildir? If yes, you could base your code on the zlib plugin, or perhaps more easily you could use mail-filter plugin: http://dovecot.org/patches/2.0/mail-filter.tar.gz
are there any examples or how too online for mail-filter ? .. sorry for silly question is there any known typical usage for that , or was this on the wishlist to solve some stuff ?
It was originally written to be used with imapc ("imap proxy") backend. You could use it for stuff like decode encrypted PGP mails
that sounds like a very good idea
or scan for viruses and drop them if found.
ok , thats solved i another way at my setup, but nice to have
great idea anyway, thx for coding
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
Thanks, Timo. Technically, it's not a Maildir, but my plan is to re-create one with folders containing hard or symbolic links pointing to the real storage. Is it going to be a problem?
On Wed, Aug 31, 2011 at 10:52 AM, Timo Sirainen tss@iki.fi wrote:
On 31.8.2011, at 17.24, Alex Cherniak wrote:
I have a large existing read-only collection of mails packaged in individual zip files as <rfc822 file>+<some additional info>. Is it possible (and how difficult) to create a proprietary plugin (like gzip) which will open a zip file, extract mail and pass it back to Dovecot? Where do I start? If plugin is not the right approach, what is?
Is it otherwise a Maildir? If yes, you could base your code on the zlib plugin, or perhaps more easily you could use mail-filter plugin: http://dovecot.org/patches/2.0/mail-filter.tar.gz
With mail-filter you can basically just put the messages through whatever program/script you want which gets the mail as input and outputs the wanted message body. I think the v2.0 mail-filter had some (potential?) bug, v2.1 mail-filter is anyway redesigned and should work perfectly.
Another question is how will this affect Dovecot performance and how to avoid any significant degradation.
I doubt it's going to be a problem.
Should work fine.
On 31.8.2011, at 22.19, Alex Cherniak wrote:
Thanks, Timo. Technically, it's not a Maildir, but my plan is to re-create one with folders containing hard or symbolic links pointing to the real storage. Is it going to be a problem?
On Wed, Aug 31, 2011 at 10:52 AM, Timo Sirainen tss@iki.fi wrote:
On 31.8.2011, at 17.24, Alex Cherniak wrote:
I have a large existing read-only collection of mails packaged in individual zip files as <rfc822 file>+<some additional info>. Is it possible (and how difficult) to create a proprietary plugin (like gzip) which will open a zip file, extract mail and pass it back to Dovecot? Where do I start? If plugin is not the right approach, what is?
Is it otherwise a Maildir? If yes, you could base your code on the zlib plugin, or perhaps more easily you could use mail-filter plugin: http://dovecot.org/patches/2.0/mail-filter.tar.gz
With mail-filter you can basically just put the messages through whatever program/script you want which gets the mail as input and outputs the wanted message body. I think the v2.0 mail-filter had some (potential?) bug, v2.1 mail-filter is anyway redesigned and should work perfectly.
Another question is how will this affect Dovecot performance and how to avoid any significant degradation.
I doubt it's going to be a problem.
participants (3)
-
Alex Cherniak
-
Robert Schetterer
-
Timo Sirainen