<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
h1
{mso-style-priority:9;
mso-style-link:"T\00EDtulo 1 Car";
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:24.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML con formato previo Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Texto de globo Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EstiloCorreo17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.TextodegloboCar
{mso-style-name:"Texto de globo Car";
mso-style-priority:99;
mso-style-link:"Texto de globo";
font-family:"Tahoma","sans-serif";}
span.Ttulo1Car
{mso-style-name:"T\00EDtulo 1 Car";
mso-style-priority:9;
mso-style-link:"T\00EDtulo 1";
font-family:"Times New Roman","serif";
font-weight:bold;}
span.HTMLconformatoprevioCar
{mso-style-name:"HTML con formato previo Car";
mso-style-priority:99;
mso-style-link:"HTML con formato previo";
font-family:"Courier New";}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 3.0cm 70.85pt 3.0cm;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1837764113;
mso-list-template-ids:-888390722;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="ES" link="blue" vlink="purple">
<div class="WordSection1">
<pre><span lang="EN-US" style="color:black">Hello:<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black"> I have found the same problem reported above by Patrick Cernko affecting our system and corrupting our messages. Even worse, Outlook 2016 will no synchronize and the clients cannot see any message, even if there is only one corrupted mail per mailbox.<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black"> I cannot figure out a feasible workaround for our system, and I can see that in new version 2.38 the bug is not fixed.<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black"> Will this issue be treated soon?<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black"> Thanks a lot<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black"> Maren Zubizarreta<o:p></o:p></span></pre>
<h1><span lang="EN-US" style="color:black"><o:p> </o:p></span></h1>
<h1><span lang="EN-US" style="color:black">WARNING: using attachment_dir with plugin zlib can corrupt mails<o:p></o:p></span></h1>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:13.5pt;color:black">Patrick Cernko</span></b><span lang="EN-US" style="font-size:13.5pt;color:black;background:white"> </span><a href="mailto:dovecot%40dovecot.org?Subject=Re:%20Re%3A%20WARNING%3A%20using%20attachment_dir%20with%20plugin%20zlib%20can%20corrupt%20mails&In-Reply-To=%3C1ab8edcc-bef4-3b41-d09a-c523674a5031%40mpi-klsb.mpg.de%3E" title="WARNING: using attachment_dir with plugin zlib can corrupt mails"><span lang="EN-US" style="font-size:13.5pt">pcernko
at mpi-klsb.mpg.de</span></a><span lang="EN-US" style="font-size:13.5pt;color:black"><br>
<i>Fri Jul 19 17:52:37 EEST 2019</i></span><span lang="EN-US"><o:p></o:p></span></p>
<ul type="disc">
<li class="MsoNormal" style="color:black;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span lang="EN-US" style="font-size:13.5pt">Previous message: </span><span style="font-size:13.5pt"><a href="https://dovecot.org/list/dovecot/2019-July/116617.html"><span lang="EN-US">index worker 2.3.7 undefined symbol errors</span></a></span><span lang="EN-US" style="font-size:13.5pt"><o:p></o:p></span></li><li class="MsoNormal" style="color:black;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<span lang="EN-US" style="font-size:13.5pt">Next message: </span><span style="font-size:13.5pt"><a href="https://dovecot.org/list/dovecot/2019-July/116571.html"><span lang="EN-US">Address family not supported by protocol</span></a></span><span lang="EN-US" style="font-size:13.5pt"><o:p></o:p></span></li><li class="MsoNormal" style="color:black;mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
<b><span lang="EN-US" style="font-size:13.5pt">Messages sorted by:</span></b><span lang="EN-US" style="font-size:13.5pt"> </span><span style="font-size:13.5pt"><a href="https://dovecot.org/list/dovecot/2019-July/date.html#116570"><span lang="EN-US">[ date ]</span></a></span><span lang="EN-US" style="font-size:13.5pt"> </span><span style="font-size:13.5pt"><a href="https://dovecot.org/list/dovecot/2019-July/thread.html#116570"><span lang="EN-US">[
thread ]</span></a></span><span lang="EN-US" style="font-size:13.5pt"> </span><span style="font-size:13.5pt"><a href="https://dovecot.org/list/dovecot/2019-July/subject.html#116570"><span lang="EN-US">[ subject ]</span></a></span><span lang="EN-US" style="font-size:13.5pt"> </span><span style="font-size:13.5pt"><a href="https://dovecot.org/list/dovecot/2019-July/author.html#116570"><span lang="EN-US">[
author ]</span></a></span><span lang="EN-US" style="font-size:13.5pt"><o:p></o:p></span></li></ul>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" noshade="" style="color:black" align="center">
</div>
<pre style="white-space:pre-wrap;font-variant-ligatures: normal;font-variant-caps: normal;orphans: 2;text-align:start;widows: 2;-webkit-text-stroke-width: 0px;text-decoration-style: initial;text-decoration-color: initial;word-spacing:0px"><span lang="EN-US" style="color:black">Hello list, hello Dovecot developers,<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">this week, I discovered a serious bug in Dovecot, that lead to several <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">broken mails on our servers. The bug corrupts the first few characters <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">of the mail header during saving. On our setup, it was almost always <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">only the very first line of text, that was corrupted.<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">Depending on the IMAP client (they seem to request different header <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">fields, ... during mail access), the bug causes the imap process to hang <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">up the TCP connection and log errors like this:<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">><i> imap(USERNAME)<4767><TeQP4ASOTK5/AAAB>: Error: Corrupted record in index cache file /IMAP/mail/mailboxes/USERNAME/mdbox/mailboxes/Trash/dbox-Mails/dovecot.index.cache: UID 489113: Broken fields in mailbox Trash: read(attachments-connector(zlib(/IMAP/mail/mailboxes/USERNAME/mdbox/storage/m.813))): FETCH BODY[HEADER.FIELDS (RETURN-PATH SUBJECT)] got too little data: 2 vs 122<o:p></o:p></i></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">In our case that finally grabbed my attention, the client was the users <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">iphone that did not display any new messages but his Thunderbird did.<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">The bug seems to be triggered by a bad "interaction" of attachment_dir <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">option and zlib plugin. If you use both, you most likely are affected, <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">too, except you only use zlib plugin for reading previously compressed <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">stored mails. That's also the workaround we use now: zlib plugin only <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">enabled in mail_plugins but no plugin/zlib_save set.<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">The bug occurs on very specific mails. Due to privacy reasons I could <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">not provide sample mails here. Storing such mails seems to trigger the <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">bug reproducible.<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">I attached a very minimal doveconf -n config, that can be used to <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">trigger the bug. If one of the developers is interested, I can try to <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">generate an "anonymized" version of such a specific mail that still <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">causes the issue. I discovered the bug on our productive systems, <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">running latest Dovecot 2.2 release, but the latest 2.3 I used during <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">debugging is affected, too.<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">During debugging, I also found one hint, that might help find the bug: <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">If you store a problematic mail with zlib_save=gz (or zlib_save=bz2) and <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">then disable the zlib plugin in mail_plugins, you can call<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">doveadm fetch -u test hdr all | grep -v ^hdr: | gzip --decompress<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">on test's mailbox with only that one broken mail.<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">This will display the beginning of the rfc822 mail text until gzip <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">terminates with "gzip: stdin: unexpected end of file", approximately <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">after twice the length of the mail HEADER. This might indicate, that <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">dovecot stores the uncompressed size of the header in it's data <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">structures although the mail is stored compressed.<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">I also found a very efficient way to find all affected mails in our setup:<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">doveadm -f flow fetch -A 'user guid mailbox uid seq flags hdr' all | \<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"> grep -a "^[^ ]+ user=" | \<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"> grep -avF ' hdr=Return-path: ' | \<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"> grep -av '.* hdr=[[:print:][:space:]]*$'<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">(runtime for ~6M mails on our servers was 20-30min)<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">This can be even more optimized if you have a powerful storage system <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">with GNU parallel:<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">><i> doveadm user '*' | parallel "doveadm -f flow fetch -u '{}' 'user guid mailbox uid seq flags hdr' all | grep -a '^user=' | grep -avF ' hdr=Return-path: ' | grep -av '.* hdr=[[:print:][:space:]]*$' || true"<o:p></o:p></i></span></pre>
<pre><span lang="EN-US" style="color:black">(runtime for ~6M mails on our servers was ~4min)<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">The command will give you a list of mails that possibly are affected, <o:p></o:p></span></pre>
<pre><span style="color:black">check the full output of<o:p></o:p></span></pre>
<pre><span style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">doveadm fetch -u USERNAME hdr guid GUID | less<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">to verify that the header is really broken.<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">On our systems I found 39 mails within ~12M mails.<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">I was able to recover these mails "manually" by reconstructing the <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">Return-Path header line, importing the fixed mails and expunging the <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">corrupt ones. Before importing, I had to disable zlib_save option obviously.<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"><o:p> </o:p></span></pre>
<pre><span lang="EN-US" style="color:black">Best regards,<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">-- <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">Patrick Cernko <</span><span style="color:black"><a href="https://dovecot.org/mailman/listinfo/dovecot"><span lang="EN-US">pcernko at mpi-klsb.mpg.de</span></a></span><span lang="EN-US" style="color:black">> +49 681 9325 5815<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">Joint Administration: Information Services and Technology<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">Max-Planck-Institute fuer Informatik & Softwaresysteme<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">-------------- next part --------------<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"># 2.3.6.1 (d124cc84b): /etc/dovecot/dovecot.conf<o:p></o:p></span></pre>
<pre><span style="color:black"># OS: Linux 4.14.127.1.amd64-smp x86_64 Debian 9.9 <o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"># Hostname: adove.mpi-klsb.mpg.de<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">listen = *<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">mail_attachment_dir = /var/vmail/attachments<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">mail_attachment_fs = posix<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">mail_gid = nogroup<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">mail_home = /var/vmail/%u<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">mail_location = mdbox:~/mdbox<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">mail_plugins = " zlib"<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">mail_uid = nobody<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">passdb {<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"> args = /etc/dovecot/userdb<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"> driver = passwd-file<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">}<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">plugin {<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"> zlib_save = gz<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">}<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">protocols = imap<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">userdb {<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"> args = /etc/dovecot/userdb<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black"> driver = passwd-file<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">}<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">-------------- next part --------------<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">A non-text attachment was scrubbed...<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">Name: smime.p7s<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">Type: application/pkcs7-signature<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">Size: 5324 bytes<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">Desc: S/MIME Cryptographic Signature<o:p></o:p></span></pre>
<pre><span lang="EN-US" style="color:black">URL: <</span><span style="color:black"><a href="https://dovecot.org/pipermail/dovecot/attachments/20190719/3da620ad/attachment-0001.p7s"><span lang="EN-US">https://dovecot.org/pipermail/dovecot/attachments/20190719/3da620ad/attachment-0001.p7s</span></a></span><span lang="EN-US" style="color:black">><o:p></o:p></span></pre>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
</div>
</body>
</html>