[Dovecot] ENVELOPE mishandling of Subject lines with quote(s)?
Hi,
I am running dovecot version 1.2.9 on a Ubuntu 10.04 server. To obtain the message's header info from this dovecot imap server, I run the following imap command on a client side's imap session:
. fetch 88:90 (envelope)
which returns (in this case)
88 FETCH (ENVELOPE ("Tue, 28 Oct 2008 15:45:35 -0500" "Linear Actuators Bring Flexibility to Packaging" (("Design World" NIL "subscription" "designworldonline.com")) (("Design World" NIL "subscription" "designworldonline.com")) (("Design World" NIL "subscription" "designworldonline.com")) ((NIL NIL "dinh" "flic.net")) NIL NIL NIL "<490779CF.00000E7D@ak119.mta.exacttarget.com>"))
89 FETCH (ENVELOPE ("29 Oct 2008 15:44:03 -0500" {65} 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available (("Insight Media" NIL "administrator" "insightmedia.info")) (("Insight Media" NIL "administrator" "insightmedia.info")) (("Insight Media" NIL "administrator" "insightmedia.info")) ((NIL NIL "dinh" "flic.net")) NIL NIL NIL "<20081029154358.0C359AE169595F1B@insightmedia.info>"))
90 FETCH (ENVELOPE ("Thu, 30 Oct 2008 04:24:31 +0800 (SGT)" "6 new products Now Online!" (("Global Sources Product Alert" NIL "paservices" "globalsources.com")) (("Global Sources Product Alert" NIL "paservices" "globalsources.com")) (("Global Sources Product Alert" NIL "paservices" "globalsources.com")) (("Dinh Le" NIL "dinh" "flic.net")) NIL NIL NIL "<352725277.1225311871753.JavaMail.web@fuchsia3.globalsources.com>"))
. OK Fetch completed.
Notice that message 89's Subject line
Subject: 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available
has quotes and this dovecot's ENVELOPE response failed to get parsed by a popular Perl module (Mail::IMAPClient::BodyStructure::Envelope). I have a hunch that this is dovecot's fault but I could be wrong. How should dovecot (or any imap server) deal with Subject lines with quotes inside ENVELOPE?
ENVELOPE seems to encapsulate most of the important information and its format can be parsed cleanly and quickly. I prefer to fetch ENVELOP, at this point, rather than the other alternatives.
Thanks for your help,
Dinh
Quoting Dinh Le <dinh@flic.net>:
- 89 FETCH (ENVELOPE ("29 Oct 2008 15:44:03 -0500" {65} 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available (("Insight Media" NIL "administrator" "insightmedia.info"))
(("Insight Media" NIL "administrator" "insightmedia.info"))
(("Insight Media" NIL "administrator" "insightmedia.info")) ((NIL
NIL "dinh" "flic.net")) NIL NIL NIL
"<20081029154358.0C359AE169595F1B@insightmedia.info>"))
[snip]
Notice that message 89's Subject line
Subject: 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available
has quotes and this dovecot's ENVELOPE response failed to get parsed by a popular Perl module (Mail::IMAPClient::BodyStructure::Envelope).
I have a hunch that this is dovecot's fault but I could be wrong.
How should dovecot (or any imap server) deal with Subject lines with
quotes inside ENVELOPE?
The subject, because it contains quotes, can't be expressed in a
quoted string so it is instead sent in a literal string. In a literal
string, the data is exactly what appears in the original message. No
post-processing mechanism should be stripping quotes or anything from
this data - it is already in its canonical state.
Dovecot is 100% correct in its return.
michael
On 6.2.2012, at 22.45, Michael M Slusarz wrote:
The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string.
Small correction: Quotes can be escaped in a quoted-string, but Dovecot just doesn't want to do it. Many other servers behave this way as well.
In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state.
Dovecot is 100% correct in its return.
Yes. Looks like Perl's IMAP client hasn't been built properly to expect literals in places where they are allowed. My guess is that this is only one of the 100 other places where a literal is allowed by RFC but not by Perl..
Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :)
On 6.2.2012, at 22.55, Timo Sirainen wrote:
Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :)
BTW. If anyone wants to try, the attached patch makes Dovecot return (almost) all possible fields as literals. You'll need to delete dovecot.index.cache files to get the full effect.
Quoting Timo Sirainen <tss@iki.fi>:
On 6.2.2012, at 22.55, Timo Sirainen wrote:
Maybe I should add some evil setting to Dovecot where it would use
literals everywhere it can, or perhaps randomly send
atoms/strings/literals. :)BTW. If anyone wants to try, the attached patch makes Dovecot return
(almost) all possible fields as literals. You'll need to delete
dovecot.index.cache files to get the full effect.
Very useful. Turns out the only thing I wasn't handling correctly was
the MIME types/subtypes in BODYSTRUCTURE.
This is a great tool for debugging client code. Not sure the
difficulty to add as a config (or even a hidden config). Being able
to easily turn on/off would increase my usage of the tool.
michael
On Mon, 6 Feb 2012, Timo Sirainen wrote:
On 6.2.2012, at 22.45, Michael M Slusarz wrote:
The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string.
Small correction: Quotes can be escaped in a quoted-string, but Dovecot just doesn't want to do it. Many other servers behave this way as well.
In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state.
Dovecot is 100% correct in its return.
Yes. Looks like Perl's IMAP client hasn't been built properly to expect literals in places where they are allowed. My guess is that this is only one of the 100 other places where a literal is allowed by RFC but not by Perl..
Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :)
Thanks, I will try to come up
On Mon, 6 Feb 2012, Timo Sirainen wrote:
On 6.2.2012, at 22.45, Michael M Slusarz wrote:
The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string.
Small correction: Quotes can be escaped in a quoted-string, but Dovecot just doesn't want to do it. Many other servers behave this way as well.
In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state.
Dovecot is 100% correct in its return.
Yes. Looks like Perl's IMAP client hasn't been built properly to expect literals in places where they are allowed. My guess is that this is only one of the 100 other places where a literal is allowed by RFC but not by Perl..
Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :)
Oops, my happy Alpine ^X finger accidently sent the previous reply. I meant to say thanks and will try to come up with a fix to this literal string problem for the Perl IMAPClient module.
Don't want Timo's evil config setting to wreak havocs :-).
On 6 Feb 2012, at 16:34, Dinh Le wrote:
On Mon, 6 Feb 2012, Timo Sirainen wrote:
On 6.2.2012, at 22.45, Michael M Slusarz wrote:
The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string.
Small correction: Quotes can be escaped in a quoted-string, but Dovecot just doesn't want to do it. Many other servers behave this way as well.
In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state.
Dovecot is 100% correct in its return.
Yes. Looks like Perl's IMAP client hasn't been built properly to expect literals in places where they are allowed. My guess is that this is only one of the 100 other places where a literal is allowed by RFC but not by Perl..
Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :)
Oops, my happy Alpine ^X finger accidently sent the previous reply. I meant to say thanks and will try to come up with a fix to this literal string problem for the Perl IMAPClient module.
Don't want Timo's evil config setting to wreak havocs :-).
The best solution for you may be to to update the version of Mail::IMAPClient you are using. There were fixes for its handling of literals claimed in the CHANGES file of v3.26 (2011-01-31). I don't know if that will fix this particular problem, but if you are running an older version then it would be worth trying the current version.
participants (4)
-
Bill Cole
-
Dinh Le
-
Michael M Slusarz
-
Timo Sirainen