Hello Simone,

I've been trying for the past days to replicate the problem...

Since we're on a production environment, I can't put dovecot in debug mode and leave it in debug for an entire day, so I've been checking the queue every half an hour and as soon as a mail got stuck, I'd change mail_debug to yes and reload dovecot's config.

Well, everytime I've done this, the mail got delivered except once. Just one time.

On the back end I got this log:

Jun 23 12:17:00 dovecot: lmtp(10680): Connect from a.b.c.d Jun 23 12:17:00 dovecot: lmtp(10680): Debug: auth input: user@domain.it quota_rule=*:backend=104857600S home=/mnt/mbox1/mailusers/user@domain.it/ Jun 23 12:17:00 dovecot: lmtp(10680): Debug: Added userdb setting: plugin/quota_rule=*:backend=104857600S Jun 23 12:18:00 dovecot: lmtp(10680): Disconnect from a.b.c.d: Connection closed

I compared the log of the stuck mail with a mail that gets delivered, and I noticed that there are a few more lines of debug... in fact, after the "Debug: Added userdb setting" I see:

Debug: Effective uid Debug: Quota root Debug: Quota rule Debug: maildir++ saved mail to INBOX Connection closed

I hope this info can help.

Regards

CINZIA INNOCENZI Technical Consulting HP Enterprise Services Telephone +39 06.6542.5758 Email cinzia.innocenzi@hp.com    Via Achille Campanile, 85 / Rome, Italy  00143

-----Original Message----- From: Innocenzi, Cinzia Sent: giovedì 23 giugno 2011 11.06 To: 'Simone Caruso' Cc: dovecot@dovecot.org; Del Coiro, Maria Giacinta Subject: RE: [Dovecot] Dovecot 2.0.13 problem with LMTP

I'll give it a try as soon as I get another mail stuck in the queue. Unfortunately it doesn't happen too often.

Thank you. I'll keep you updated.

Regards

CINZIA INNOCENZI Technical Consulting HP Enterprise Services Telephone +39 06.6542.5758 Email cinzia.innocenzi@hp.com    Via Achille Campanile, 85 / Rome, Italy  00143

-----Original Message----- From: Simone Caruso [mailto:info@simonecaruso.com] Sent: mercoledì 22 giugno 2011 22.08 To: Innocenzi, Cinzia Cc: dovecot@dovecot.org; Del Coiro, Maria Giacinta Subject: Re: [Dovecot] Dovecot 2.0.13 problem with LMTP

if u already didnt, try setting 'mail_debug=yes' in config.

-- Simone Caruso IT Consultant p.iva: 03045250838

Hi Simone,

no, I don't think the problem is related to a firewall or to some limit exceeded because mails get stuck while others are being delivered at the same time. The only log I have is dovecot's log file. There's nothing on messages log file.

I checked all open fd's of all dovecot processes. Each process has max 50 open fd's. We've got a soft/hard limit of 1024 fd's.

Is there anything else I can check? Is there any other dovecot parameter that I can enable so that I can send you a more detailed debug output?

CINZIA INNOCENZI Technical Consulting HP Enterprise Services Telephone +39 06.6542.5758 Email cinzia.innocenzi@hp.com    Via Achille Campanile, 85 / Rome, Italy  00143

-----Original Message----- From: Simone Caruso [mailto:info@simonecaruso.com] Sent: lunedì 27 giugno 2011 13.03 To: dovecot@dovecot.org Cc: Innocenzi, Cinzia; Del Coiro, Maria Giacinta Subject: Re: [Dovecot] Dovecot 2.0.13 problem with LMTP

On 27/06/2011 11:23, Innocenzi, Cinzia wrote:

Hello Simone, Jun 22 11:57:53 front-end dovecot: lmtp(16397): Disconnect from a.b.c.d: Disconnected for inactivity

Logs don't say anything useful to me... :/ Maybe a firewall problem between fe and be !? There is no other logs in syslog that can be related to dovecot? file descriptors limit exceeded?

-- Simone Caruso IT Consultant +39 349 65 90 805 p.iva: 03045250838

Hello Timo,

so far I've seen this happening when proxying a mail for one recipient; I've never seen this happening for multiple recipients.

The other thing I've noticed is that if I reload dovecot, the mail usually gets delivered. Once, it remained stuck in the queue, but after a few hours, when I checked postfix's queue again, it got delivered.

Let me know if you need further information.

Regards,

CINZIA INNOCENZI Technical Consulting HP Enterprise Services Telephone +39 06.6542.5758 Email cinzia.innocenzi@hp.com    Via Achille Campanile, 85 / Rome, Italy  00143

-----Original Message----- From: Timo Sirainen [mailto:tss@iki.fi] Sent: martedì 28 giugno 2011 2.04 To: Innocenzi, Cinzia Cc: dovecot@dovecot.org; Del Coiro, Maria Giacinta Subject: Re: [Dovecot] Dovecot 2.0.13 problem with LMTP

On Wed, 2011-06-22 at 10:31 +0000, Innocenzi, Cinzia wrote:

Hello,

we have dovecot 2.0.13 installed in our RHEL5 servers. We use dovecot as a lmtp proxy on the front end mail relays and dovecot for back end lmtp/pop/imap. All our servers have RHEL5.6 64-bit and dovecot 2.0.13 installed, and the back end has also an ext3 mbox file-system mounted.

Sometimes we have mails that get stuck on our postfix queue with the following error: Remote server not answering (timeout in DATA input) (in reply to end of DATA command) .. It doesn't happen for every single mail, it's random.

My guess is there's a bug somewhere in the LMTP proxy code. It's a bit annoyingly complex code.. Do you see this happening when proxying only a single mail (one RCPT TO per DATA, not multiple)?

And I would guess that when attempting to deliver the same mail again it works fine (at least in a few tries)?

Hello Timo,

I'm sorry I haven't replied before, but I've been investigating as much as possible on this problem, and, for a moment, I thought I came up with something...

Thanks to Simone Caruso's hint, I've been doing some checks on LMTP-LDAP connections, since I would always get an LDAP error in the dovecot proxy's logfile.

Furthermore, I found out that the back end nodes didn't have resolv.conf aligned properly. So I blamed the errors to this misconfiguration, especially because as soon as I aligned resolv.conf, the three stuck mails I had in the queue this morning got delivered immediately!

Unfortunately, this is not the case.

I applied the three patches you provided to front end proxies and back end nodes.

I have a stuck mail that will not get delivered! I'm attaching the logs.

Thanks again for your help.

Regards,

CINZIA INNOCENZI Technical Consulting HP Enterprise Services Telephone +39 06.6542.5758 Email cinzia.innocenzi@hp.com    Via Achille Campanile, 85 / Rome, Italy  00143

-----Original Message----- From: Timo Sirainen [mailto:tss@iki.fi] Sent: giovedì 30 giugno 2011 7.36 To: Innocenzi, Cinzia Cc: Del Coiro, Maria Giacinta; dovecot@dovecot.org Subject: Re: [Dovecot] Dovecot 2.0.13 problem with LMTP

Try to apply these patches to both proxy and backend and see what it logs then?

http://hg.dovecot.org/dovecot-2.0/raw-rev/1b1fc681a277 http://hg.dovecot.org/dovecot-2.0/raw-rev/36e7ded2ef0b http://hg.dovecot.org/dovecot-2.0/raw-rev/537d4b6d9a7a

On Tue, 2011-06-28 at 08:38 +0000, Innocenzi, Cinzia wrote:

Hello Timo,

so far I've seen this happening when proxying a mail for one recipient; I've never seen this happening for multiple recipients.

The other thing I've noticed is that if I reload dovecot, the mail usually gets delivered. Once, it remained stuck in the queue, but after a few hours, when I checked postfix's queue again, it got delivered.

Let me know if you need further information.

Regards,

CINZIA INNOCENZI Technical Consulting HP Enterprise Services Telephone +39 06.6542.5758 Email cinzia.innocenzi@hp.com
Via Achille Campanile, 85 / Rome, Italy 00143

-----Original Message----- From: Timo Sirainen [mailto:tss@iki.fi] Sent: martedì 28 giugno 2011 2.04 To: Innocenzi, Cinzia Cc: dovecot@dovecot.org; Del Coiro, Maria Giacinta Subject: Re: [Dovecot] Dovecot 2.0.13 problem with LMTP

On Wed, 2011-06-22 at 10:31 +0000, Innocenzi, Cinzia wrote:

...

Hello,

we have dovecot 2.0.13 installed in our RHEL5 servers. We use dovecot as a lmtp proxy on the front end mail relays and dovecot for back end lmtp/pop/imap. All our servers have RHEL5.6 64-bit and dovecot 2.0.13 installed, and the back end has also an ext3 mbox file-system mounted.

Sometimes we have mails that get stuck on our postfix queue with the following error: Remote server not answering (timeout in DATA input) (in reply to end of DATA command) .. It doesn't happen for every single mail, it's random.

My guess is there's a bug somewhere in the LMTP proxy code. It's a bit annoyingly complex code.. Do you see this happening when proxying only a single mail (one RCPT TO per DATA, not multiple)?

And I would guess that when attempting to deliver the same mail again it works fine (at least in a few tries)?

Actually this doesn't happen every time there's inactivity, but the following message appears often in dovecot proxy's logfile:

auth: Error: LDAP: Connection lost to LDAP server, reconnecting

I thought this information could be of use to you.

Please let me know if there's anything else I can check.

Regards,

Cinzia

-----Original Message----- From: Timo Sirainen [mailto:tss@iki.fi] Sent: martedì 5 luglio 2011 16.26 To: Innocenzi, Cinzia Cc: Del Coiro, Maria Giacinta; dovecot@dovecot.org Subject: RE: [Dovecot] Dovecot 2.0.13 problem with LMTP

On Mon, 2011-07-04 at 18:58 +0000, Innocenzi, Cinzia wrote:

Thanks to Simone Caruso's hint, I've been doing some checks on LMTP-LDAP connections, since I would always get an LDAP error in the dovecot proxy's logfile.

You mean every time when LMTP gets disconnected because of inactivity, the LDAP gets also disconnected? This is beginning to sound like it's not a Dovecot bug..