[Dovecot] empty emails with 2.2.13/pigeonhole

[Stephan Bosch]

On 6/4/2014 1:26 PM, Cor Bosman wrote:
> I recently upgraded from 2.2.10 to 2.2.13, and also upgraded pigeonhole. Ever since im seeing some empty emails appear in my inbox.
>
> Return-Path: <cricket at stats2.xs4all.net>
> Delivered-To: <cor>
> Received: from imapdirector1.xs4all.net ([194.109.26.173])
> 	by userimap9.xs4all.nl (Dovecot) with LMTP id CHu0MUDDjlOBdwAAwvsqNQ
> 	for <cor>; Wed, 04 Jun 2014 08:57:04 +0200
>
> Thats all this mail contains. This only happens using LMTP/Sieve, and with many different senders/recipients. Debug log shows nothing out of the ordinary, except that the mail seems to be corrupted by the time it hits sieve. It's very hard to diagnose. Our path is kind of lengthy, and in theory it could happen everywhere.
>
> Mail first arrives on incoming MX server which is sendmail. From there it's sent to a dovecot director using lmtp, which sends it to a server that does final delivery with sieve. Im not sure where the email gets corrupted. Any extra debugging I could turn on? 
>
> Two extra things to note. So far it seems to only happen to emails that would have hit a matching sieve filter rule.  I have asked a dozen people to verify this, and all emails would have matched a filter if all was well. Instead, it gets dropped into the inbox because pigeonhole seems to be working with the corrupt file and cant match any rules anymore. And second, 99% of our email is handled differently, from the sendmail server directly into a local mailer, and that never causes issues. We're in the process of switching to lmtp.

-> Can you tell me more about your config? Do you use any of the more
obscure features like extprograms?

-> You can use vnd.dovecot.debug in combination with the variables
extension to see whether Sieve even gets a sensible message or not:

http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/doc/rfc/spec-bosch-sieve-debug.txt

-> The thing I find strange is that the headers you show are all added
to the message in LMTP before it is fed to Sieve. It is unlikely that
Sieve somehow delivers half a message, so my current hunch is that it
happens before Sieve gets it.

An LMTP transaction like the following:

220 host Dovecot ready.
lhlo frop
250-host
250-8BITMIME
250-CHUNKING
250-DSN
250-ENHANCEDSTATUSCODES
250-PIPELINING
250 STARTTLS
mail from:<stephan at rename-it.nl>
250 2.1.0 OK
rcpt to:<stephan>
250 2.1.5 OK
data
354 OK
.
250 2.0.0 <stephan> t+tYKQ9Hj1MBEwAAg5GDIQ Saved
quit
221 2.0.0 Bye

Yields:

Return-Path: <stephan at rename-it.nl>
Delivered-To: <stephan>
Received: from frop ([127.0.0.1])
	by host (Dovecot) with LMTP id t+tYKQ9Hj1MBEwAAg5GDIQ
	for <stephan>; Wed, 04 Jun 2014 18:19:34 +0200


So, my guess is that the message is empty before it arrives at the final
LMTP. Can you sniff the traffic to that LMTP server?

Regards,

Stephan.