[Dovecot] Server-side mail archiving - possible dovecot lda feature?
Hi Timo,
I have a question on the feasibility of adding the capability directly in dovecot LDA (or maybe this wouldn't be the correct place?) to provide a server-side mail archiving feature.
What I'd like is a simple way to define a maildir, ie:
vmail/archive/$user/Inbox and vmail/archive/$user/Inbox.Sent
And then have the dovecot LDA deliver a copy of all mail (incoming and outgoing) to this archive folder.
Is the dovecot LDA the best place for this to go? Or would an entirely new plug-in be more appropriate?
My boss may be willing to pay to have this capability added. I have investigated other third party solutions, but I would prefer something that is tightly integrated with dovecot.
I look forward to your response...
--
Best regards,
Charles
On Fri, 2006-11-03 at 08:30 -0500, Charles Marcus wrote:
Is the dovecot LDA the best place for this to go? Or would an entirely new plug-in be more appropriate?
Neither?
For archive.inbox, simply make your MTA deliver a second copy to there, or configure the dovecot LDA to put a copy into it. That should be trivial.
For archive.outbox, you need to make your MTA deliver a copy of the message to the right folder, with exim that's easy with the unseen router option (IIRC, but I do know that the exim FAQ contains an entry answering exactly your question)
johannes
Johannes Berg wrote:
On Fri, 2006-11-03 at 08:30 -0500, Charles Marcus wrote:
Is the dovecot LDA the best place for this to go? Or would an entirely new plug-in be more appropriate?
Neither?
Why not? Have you ever googled on what is available for an integrated email archiving solution? There are no (good) OSS solutions available, and there are only a few commercial ones, most of those aimed at Exchange servers. I did quite a bit of googling, and there doesn't seem to be a good OSS solution to archiving all email on a server, which is why I am asking about this here.
For archive.inbox, simply make your MTA deliver a second copy to there,
Postfix has an 'always_bcc' option, but it does not preserve the message with all of the headers intact, so this is not an option.
or configure the dovecot LDA to put a copy into it. That should be trivial.
For a programmer, maybe - but sadly I am not. But at least you have partially answered my question - apparently it *is* possible to do this with dovecots lda. Thanks...
anmar@gmx.net said:
dovecot LDA has already -m that allows what you're looking for, just make your MTA do a second delivery adjusting LDA arguments.
Well, since I'm not a programmer, I wouldn't know where to begin, which, again, is why I asked.
What I'd like is something that is easily configurable *and* *fully* *supported* (ie, no 'hacks' required). Something like:
archive_incoming= archive_sent= archive_directory=
Hacking is fine for someone with the requisite skills to dig themselves out of the holes they inevitably dig for themselves.
Why would you object to some extra options to the LDA that would support feeding Postfix (or other MTA's via a documented list of arguments for each supported MTA) the required command(s) to accomplish the delivery to the correct maildir?
For archive.outbox, you need to make your MTA deliver a copy of the message to the right folder, with exim that's easy with the unseen router option (IIRC, but I do know that the exim FAQ contains an entry answering exactly your question)
Again, Postfix's 'always_bcc' option won't work for us because it doesn't preserve the headers. I did quite a bit of googling, and there doesn't seem to be a good OSS solution to this, which is why I asked. I want an *exact* *duplicate* of all messages.
It does indeed sound like dovecots LDA is fully capable of doing this, at least for the 'incoming' mail, and it sounds like it would also work for outgoing mail if you have your MTA configured to use the dovecot LDA. What I'd like to know is, is it *feasible* to consider hiring someone to modify the LDA to fully support this using documented switches? Or via a new plug-in?
--
Best regards,
Charles
On Fri, 2006-11-03 at 09:29 -0500, Charles Marcus wrote:
Why not? Have you ever googled on what is available for an integrated email archiving solution? There are no (good) OSS solutions available, and there are only a few commercial ones, most of those aimed at Exchange servers. I did quite a bit of googling, and there doesn't seem to be a good OSS solution to archiving all email on a server, which is why I am asking about this here.
Well, you see, there is no OSS solution to email like exchange that provides all of MTA, LDA and IMAP/POP server. Hence, there's no way to have an integrated archiving solution. Outgoing mail is only visible to the MTA and incoming mail is visible to all three. Hence, the most logical place to put archiving would be the MTA (not dovecot but postfix for you).
But then you'll probably complain that the archive isn't visible in IMAP.
For archive.inbox, simply make your MTA deliver a second copy to there,
Postfix has an 'always_bcc' option, but it does not preserve the message with all of the headers intact, so this is not an option.
Use a saner MTA then.
For a programmer, maybe - but sadly I am not. But at least you have partially answered my question - apparently it *is* possible to do this with dovecots lda. Thanks...
Ho humm, last I checked dovecot lda supported mail filtering by way of config files. So you can always force a config file that first drops a copy to the archive and then invokes the user's filter file.
It does indeed sound like dovecots LDA is fully capable of doing this, at least for the 'incoming' mail,
Yes, and I'm confident that no hacking is required for that. I personally use maildrop, and it initially always reads /etc/maildroprc before the user's ~/.mailfilter. I could, if I wanted to, force it to copy the message to the archive in that case.
and it sounds like it would also work for outgoing mail if you have your MTA configured to use the dovecot LDA.
No, you obviously haven't understood mail flow yet. Read up on what MTAs do. For outgoing mail, no LDA ("*local* delivery agent" after all) is involved at all.
What I'd like to know is, is it *feasible* to consider hiring someone to modify the LDA to fully support this using documented switches? Or via a new plug-in?
No. See above. The LDA never gets a copy of outgoing mail.
For the MTA, I can only point to how exim can do it: http://www.exim.org/eximwiki/FAQ/Delivery/Q0622 http://www.exim.org/eximwiki/FAQ/Miscellaneous/Q5033
Has it ever occurred to you that a product as you want is not even possible because if written for postfix it won't work on another system that uses exim or sendmail or ....? And if it serves the archive with cyrus and uses maildrop, then it won't work with dovecot/dovecot-lda?
johannes
On Fri, November 3, 2006 10:00 am, Johannes Berg wrote:
On Fri, 2006-11-03 at 09:29 -0500, Charles Marcus wrote:
Why not? Have you ever googled on what is available for an integrated email archiving solution? There are no (good) OSS solutions available, and there are only a few commercial ones, most of those aimed at Exchange servers. I did quite a bit of googling, and there doesn't seem to be a good OSS solution to archiving all email on a server, which is why I am asking about this here.
Well, you see, there is no OSS solution to email like exchange that provides all of MTA, LDA and IMAP/POP server.
Technically, that's not true -- Courier MTA includes all of those (Courier MTA, Maildrop, and Courier IMAP). Not that I'm recommending it -- I find Courier MTA's configuration ... peculiar, and its documentation is severely lacking.
Jim Trigg
On Fri, 2006-11-03 at 10:45 -0500, Jim Trigg wrote:
Technically, that's not true -- Courier MTA includes all of those (Courier MTA, Maildrop, and Courier IMAP).
Oh, I wasn't aware that it had an MTA too. There's also hula.
johannes
the most logical place to put archiving would be the MTA (not dovecot but postfix for you).
I agree - but was under the impression that, if Postfix is using the dovecot LDA, it might be easy to do this with the dovecot lda. Obviously I was not thinking clearly - thanks for straightening me out....
But then you'll probably complain that the archive isn't visible in IMAP.
? I would share the maildir, and only those who had access could see the archives.
For archive.inbox, simply make your MTA deliver a second copy to there,
Postfix has an 'always_bcc' option, but it does not preserve the message with all of the headers intact, so this is not an option.
Use a saner MTA then.
There are more than a few people out there who might take issue with your intimation that postfix is not a sane mta. That said...
<snip>
For the MTA, I can only point to how exim can do it: http://www.exim.org/eximwiki/FAQ/Delivery/Q0622 http://www.exim.org/eximwiki/FAQ/Miscellaneous/Q5033
Thanks for the pointers. I've heard of exim, but never played with it... maybe I'll give it a look...
Has it ever occurred to you that a product as you want is not even possible because if written for postfix it won't work on another system that uses exim or sendmail or ....?
So, why exactly do you think I made reference to doing this via a plug-in?
And if it serves the archive with cyrus and uses maildrop, then it won't work with dovecot/dovecot-lda?
I don't get your point? What difference does it make if a plug-in for postfix doesn't work for exim?
Anyway, I've learned enough to know that I should definitely be looking at an MTA level solution, so thanks at least for getting me this far, even if you did so with a bit of an attitude...
--
Best regards,
Charles
Charles Marcus escribió:
I did quite a bit of googling, and there doesn't seem to be a good OSS solution to archiving all email on a server, which is why I am asking about this here.
Maybe because they're not needed? Any decent mta will allow doing 'shadow' deliveries given a set of conditions, so all archiving needs can be met by configuring the mta (in exim land this can be done as an unseen router or in a system filter, neither of which require programming skills).
anmar@gmx.net said:
dovecot LDA has already -m that allows what you're looking for, just make your MTA do a second delivery adjusting LDA arguments.
Well, since I'm not a programmer, I wouldn't know where to begin, which, again, is why I asked.
Well, it's not a programming issue, it's a MTA configuration one.
Why would you object to some extra options to the LDA that would support feeding Postfix (or other MTA's via a documented list of arguments for each supported MTA) the required command(s) to accomplish the delivery to the correct maildir?
Let's say you're delivering using dovecot lda as '/path/to/lda/deliver -d $user', then you need your mta do a second delivery invoking '/path/to/lda/deliver -d $user -m Archive'. That's all. Both for 'incoming' (two LDA deliveries) or 'outgoing' (one smtp and one LDA delivery) mail.
Then if you're mta does not allow that sort of configuration, you need a more flexible mta :)
Regards,
Angel Marin http://anmar.eu.org/
Charles Marcus escribió:
And then have the dovecot LDA deliver a copy of all mail (incoming and outgoing) to this archive folder.
dovecot LDA has already -m that allows what you're looking for, just make your MTA do a second delivery adjusting LDA arguments.
Regards,
Angel Marin http://anmar.eu.org/
On Friday 03 November 2006 08:30, Charles Marcus wrote:
Hi Timo,
I have a question on the feasibility of adding the capability directly in dovecot LDA (or maybe this wouldn't be the correct place?) to provide a server-side mail archiving feature.
What I'd like is a simple way to define a maildir, ie:
vmail/archive/$user/Inbox and vmail/archive/$user/Inbox.Sent
And then have the dovecot LDA deliver a copy of all mail (incoming and outgoing) to this archive folder.
Is the dovecot LDA the best place for this to go? Or would an entirely new plug-in be more appropriate?
My boss may be willing to pay to have this capability added. I have investigated other third party solutions, but I would prefer something that is tightly integrated with dovecot.
I look forward to your response...
As discussed elsewhere in this thread, the LDA is not the appropriate place for this feature, the MTA is. I do something similar already with sendmail using milters. As I understand it, postfix also has milter capability, so a postfix/milter guru is likely what you're looking for.
hth
Mike
participants (5)
-
Angel Marin
-
Charles Marcus
-
Jim Trigg
-
Johannes Berg
-
Michael Surette