[Dovecot] Getting the SMTP envelope through IMAP?
Hi,
just a simple question about dovecot:
The mbox file format contains the SMTP envelope sender and transmission date in a FROM ... line preceding the RFC822 mail header and body.
As far as I see at the moment, the IMAP protocol does not support transmission of that SMTP envelope data in either way, but confusingly uses the term „envelope” for a structure describing the RFC822 mail header.
My question is, whether there is any way to retrieve the SMTP envelope through dovecot's IMAP anyway, maybe through some extension.
regards Hadmut
Hadmut Danisch hadmut@danisch.de wrote:
just a simple question about dovecot:
The mbox file format contains the SMTP envelope sender and transmission date in a FROM ... line preceding the RFC822 mail header and body.
As far as I see at the moment, the IMAP protocol does not support transmission of that SMTP envelope data in either way, but confusingly uses the term „envelope” for a structure describing the RFC822 mail header.
My question is, whether there is any way to retrieve the SMTP envelope through dovecot's IMAP anyway, maybe through some extension.
Almost all MTA copy "(SMTP) envelope sender" to Return-Path: header before delivery to the final mailbox.
If you do not get Return-Path: header in messages delivered to your (dovecot) mailbox then name your MTA (exim/postfix/sendmail/.....).
-- [pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu You are absolute plate-glass. I see to the very back of your mind. -- Sherlock Holmes
On Fri, Oct 21, 2011 at 05:34:49PM +0200, Hadmut Danisch wrote:
Hi,
just a simple question about dovecot:
The mbox file format contains the SMTP envelope sender and transmission date in a FROM ... line preceding the RFC822 mail header and body.
The line 'From foo@bar [..]' is only a seperation for different emails in the mbox (in contrast, an RFC 822 From: is followed by a colon).
As far as I see at the moment, the IMAP protocol does not support transmission of that SMTP envelope data in either way, but confusingly
The SMTP envelope does only exist within the involved MTAs and only as long as the message is not finally delivered.
And if the MTA does not reveal the value of MAIL FROM: there will only be the address from the Return-Path: or From: header fields to populate the From line.
uses the term „envelope” for a structure describing the RFC822 mail header.
My question is, whether there is any way to retrieve the SMTP envelope through dovecot's IMAP anyway, maybe through some extension.
Hardly. Even something like FETCH 1 (BODY[HEADER.FIELDS (FROM)]) would not reveal the mbox message seperator but the RFC 822 From: header.
Dennis
On 21.10.2011 20:02, Dennis Guhl wrote:
The SMTP envelope does only exist within the involved MTAs and only as long as the message is not finally delivered.
The intended use is to create a backup from a mailbox through IMAP, which is as close as possible to the original mbox file, thus resembling the FROM lines as well.
The envelope sender address does not drop it's meaning after delivery. It's required for debugging purposes, writing mail filters, create blacklist filters from detected spam, etc.
regards
On Sat, Oct 22, 2011 at 12:00:34AM +0200, Hadmut Danisch wrote:
On 21.10.2011 20:02, Dennis Guhl wrote:
The SMTP envelope does only exist within the involved MTAs and only as long as the message is not finally delivered.
The intended use is to create a backup from a mailbox through IMAP, which is as close as possible to the original mbox file, thus resembling the FROM lines as well.
I don't think it will be possible. The tool for this would be rsync (wich might be difficult if you only have IMAP access to the machine in question).
The envelope sender address does not drop it's meaning after delivery.
This I never claimed.
It's required for debugging purposes, writing mail filters, create blacklist filters from detected spam, etc.
Disregarding end users mail filters this are all tasks for mailadmins which can tell thier MTA to write a reliable Return-Path: header and which have access to the corresponding maillog.
Dennis
On 22.10.2011 10:15, Dennis Guhl wrote:
It's required for debugging purposes, writing mail filters, create
blacklist filters from detected spam, etc. Disregarding end users mail filters this are all tasks for mailadmins which can tell thier MTA to write a reliable Return-Path: header and which have access to the corresponding maillog.
Definitely wrong.
Writing mail filters like ~/.mailfilter is a user's task. Training their individual spam filters as well.
And whether you're even permitted to keep the maillog that long depends on your local data protection laws.
And neither the Return-Path nor the Received-Lines reveal the precise date of delivery as the From line contains.
regards Hadmut
On Sat, Oct 22, 2011 at 11:09:28AM +0200, Hadmut Danisch wrote:
On 22.10.2011 10:15, Dennis Guhl wrote:
It's required for debugging purposes, writing mail filters, create
blacklist filters from detected spam, etc. Disregarding end users mail filters this are all tasks for mailadmins
^^^^^^^^^^^^^^^^^^^^^^^which can tell thier MTA to write a reliable Return-Path: header and which have access to the corresponding maillog.
Definitely wrong.
Writing mail filters like ~/.mailfilter is a user's task. Training their individual spam filters as well.
. o O ( at least he didn't truncate the relevant quote this time )
And whether you're even permitted to keep the maillog that long depends on your local data protection laws.
How long do you think a responsible mailadmin need to train filter.
Btw. even here in Germany, where we most likely have the most restrictive laws regarding the protection of personal data, I am allowed to keep the maillog as long as I can show that the log is needed to satisfy user requests.
And neither the Return-Path nor the Received-Lines reveal the precise date of delivery as the From line contains.
Nonsense.
Every RFC conformant received line contains the full date-time as specified in RFC 5322. Nothing else is taken into account for the mbox
From line.
Dennis
I don't believe it does make much sense to ask technical questions if this ends in silly discussions about whether an admin should do something this or that way or how long logfiles should be kept. This is not related to the technical question anymore and completely useless.
I was looking for a simple yes or no, not for fruitless debates.
I hate it if one is asking a pure technical question and in response gets lessons in what people consider as a correct behaviour.
On Sat, 22 Oct 2011, Hadmut Danisch wrote:
I don't believe it does make much sense to ask technical questions if this ends in silly discussions about whether an admin should do something this or that way or how long logfiles should be kept. This is not related to the technical question anymore and completely useless.
I was looking for a simple yes or no, not for fruitless debates.
I hate it if one is asking a pure technical question and in response gets lessons in what people consider as a correct behaviour.
This is a generic problem with technicians, always having tons of arguments to support their statements ;-)
On the other hand, the question was a bit broad as a starting point. The SMTP envelope is nothing more than SMTP protocol and is not in itself part of the email format RFC. If you want to have this information saved in the email message, then it is the task of the SMTP-server to add this in the headers of the message. (Hardly parseable in Received headers, probably better when also included in things like Return-path, Delivery-date and Envelope-to.) Whether or not one should apply any filtering, or when, or where, may be related to this topic but I'd say that's the freedom of the user. Or, the arbitrary choice of some manager ;-)
Just my 2 cents..
-- Maarten
On 22.10.2011 18:56, Maarten Bezemer wrote:
On the other hand, the question was a bit broad as a starting point. The SMTP envelope is nothing more than SMTP protocol and is not in itself part of the email format RFC. If you want to have this information saved in the email message, then it is the task of the SMTP-server to add this in the headers of the message. (Hardly parseable in Received headers, probably better when also included in things like Return-path, Delivery-date and Envelope-to.) Whether or not one should apply any filtering, or when, or where, may be related to this topic but I'd say that's the freedom of the user. Or, the arbitrary choice of some manager ;-)
Again, this discussion is nuts. If this is supposed to be some support mailing list (as the dovecot web page suggests) people should take care to focus on the question rather than taking a question as an opportunity for telling their individual opinion.
The main question is whether I can draw a precise copy of a mailfolder through IMAP without any loss of data.
It does not make any sense to discuss what that information could be used for, especially nobody on that list is familiar with the local requirements I have to fulfill.
(BTW, I am familiar with the SMTP envelope, I was working more than two years at the IRTF and IETF about treatment of the SMTP envelope, and doing mail system administration since around 1989. I don't need any introduction or further discussion about that.)
Please understand that I do not want to waste any more time in this discussion that completely misses the point and the initial question.
On Sat, 22 Oct 2011, Hadmut Danisch wrote:
Again, this discussion is nuts. If this is supposed to be some support mailing list (as the dovecot web page suggests) people should take care to focus on the question rather than taking a question as an opportunity for telling their individual opinion.
The main question is whether I can draw a precise copy of a mailfolder through IMAP without any loss of data.
My last reply to this thread, then.
The answer to your main question would be 'yes', since Dovecot's IMAP interface supports retrieving the entire email message including all its meta-data (aka headers or envelope).
The mbox from-line, not to be confused with the From: header, is simply not part of the email message but used only as a separator. Heck, it need not even contain valid information, but only proper formatting to satisfy picky/"smart" MUA's. Usually, it conveniently does carry some useful information, but e.g. when the SMTP envelope-from is '<>' then the mbox from-line usually contains something like MAILER-DAEMON to stay within the formatting specification.
So, instead of blaming others of derailing a discussion and/or not simply answering a question, it might be an equally good idea to think twice about how you asked the question.
I'm actually happy that 'simple' questions are handled the way they are. Most people do not want an answer to the question they asked, but want a solution to their problem. Which they may have described in the question but often is left as an exercise for the reader. ;-)
But then again, I'm a technician, not a shrink, so my apologies for being so unfriendly...
-- Maarten
The mbox from-line, not to be confused with the From: header, is simply not part of the email message but used only as a separator. Heck, it need not even contain valid information, but only proper formatting to satisfy picky/"smart" MUA's. Usually, it conveniently does carry some useful information, but e.g. when the SMTP envelope-from is '<>' then the mbox from-line usually contains something like MAILER-DAEMON to stay within the formatting specification.
Once again: I've never asked for an explanation about what that From-Line is.
I did not ask whether it contains useful information either.
I've asked whether dovecot allows to retrieve it over IMAP.
Why is it impossible for you to understand the question before answering?
Most people do not want an answer to the question they asked, but want a solution to their problem.
I did not ask for a solution of a problem, either.
I've asked for a precise answer exactly to the question I've asked.
That's why you fail to focus on the question and to answer it. Because you want to sell your solution for what you believe the problem is - or what you like it to be. But your assumption about the problem is completely wrong. So is your answer.
Stop obtruding solutions that nobody has asked for.
On Sat, 2011-10-22 at 11:09 +0200, Hadmut Danisch wrote:
And neither the Return-Path nor the Received-Lines reveal the precise date of delivery as the From line contains.
The delivery date in From_ line is available in IMAP INTERNALDATE. The address part isn't available without patching Dovecot (mail_get_special(mail, MAIL_FETCH_FROM_ENVELOPE, &value))
participants (5)
-
Andrzej Adam Filip
-
Dennis Guhl
-
Hadmut Danisch
-
Maarten Bezemer
-
Timo Sirainen