[Dovecot] Overloading IMAP Server
Hi All, I am having an issue with dovecot IMAP reading and writing to one specific mailbox. I have a check email delivery script that sends out a email and then logs into dovecot to check if the mailbox that the message has been delivered to. However, the servers that send these messages send them in bunches depending on how many postfix spools it is checking. So I could get 60 messages all being sent to 1 mailbox and then having to read that same mailbox to check if the message actually got delivered.
I am using postfix as the MTA and dovecot is actually delivering the message to the mailbox
I am getting a lot of errors such as the following:
Could not connect to IMAP server imap failed: IMAP RECEIVE CRITICAL - Could not select INBOX
Any ideas as to what I can tweak in dovecot to allow these messages to be verified.
Thanks for any information or help you could provide!!
Ken
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, 21 Jan 2013, Harford, Ken wrote:
I am having an issue with dovecot IMAP reading and writing toone specific mailbox. I have a check email delivery script that sends
What's the purpose of the verification:
The prompt delivery?
The delivery at all?
The correct delivery?
out a email and then logs into dovecot to check if the mailbox that the message has been delivered to. However, the servers that send these messages send them in bunches depending on how many postfix spools it is checking. So I could get 60 messages all being sent to 1 mailbox and then having to read that same mailbox to check if the message actually got delivered.
I am getting a lot of errors such as the following:
Could not connect to IMAP server imap failed: IMAP RECEIVE CRITICAL - Could not select INBOX
Is this an error logged by Dovecot?
Any ideas as to what I can tweak in dovecot to allow these messages to be verified.
a) Wait a bit before logging into the IMAP server and check the messages sequentionally, not in parallel. E.g. your script could synchronize itself with other parallel instances by a shared lock.
b) Use mail_log and a script on the mail server monitoring Dovecot logs and register delivered message ids and/or recipient. Then figure out a good way to probe, if one pariticular message has been seen. script verifies that data.
- -or- make note of message id/recipient on the sender side, somehow push (ssh or something) that information on regular basis to the mail server, where a
c) To verify the correct delivery: use some sort of checksumming.
Kind regards,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux)
iQEVAwUBUP5WEF3r2wJMiz2NAQIatAgAjwZR1VGnyURuncLRNeeY0dBaqWzAUcNK g4YoiXB6NLxP6Xr/QO6Pw+6Q3sECJ0CJdwImOz56FVFNuiOygUMQ1dCdZ+Tpy9Wu xeAdobrkOo2uzgRnUh7+WLeZ8k32gZZztCAQA90uTvB3MBox+7J/P8/mApp+8/tr JKyhUy8RJIYXH+yrw/ZkbjmOSEEkf/wTPnBQcMKw5/4odha4lu+yEKx766mm0YCe 3USL7eVqN0Q9FwUH+iCjnRJAdKRETP4CnfQpjVp7eiJX7HD0iNwhucDwf037Ev2N PG173M40pMllRyesjiV8gwOu8mAHu6V3QUiFJkYAKFOotUVO6xMrNQ== =SeaP -----END PGP SIGNATURE-----
Am 22.01.2013 10:04, schrieb Steffen Kaiser:
On Mon, 21 Jan 2013, Harford, Ken wrote:
I am having an issue with dovecot IMAP reading and writingto one specific mailbox. I have a check email delivery script that sends
What's the purpose of the verification:
The prompt delivery?
The delivery at all?
The correct delivery?
out a email and then logs into dovecot to check if the mailbox that the message has been delivered to. However, the servers that send these messages send them in bunches depending on how many postfix spools it is checking. So I could get 60 messages all being sent to 1 mailbox and then having to read that same mailbox to check if the message actually got delivered.
I am getting a lot of errors such as the following:
Could not connect to IMAP server imap failed: IMAP RECEIVE CRITICAL - Could not select INBOX
Is this an error logged by Dovecot?
Any ideas as to what I can tweak in dovecot to allow these messages to be verified.
a) Wait a bit before logging into the IMAP server and check the messages sequentionally, not in parallel. E.g. your script could synchronize itself with other parallel instances by a shared lock.
b) Use mail_log and a script on the mail server monitoring Dovecot logs and register delivered message ids and/or recipient. Then figure out a good way to probe, if one pariticular message has been seen. -or- make note of message id/recipient on the sender side, somehow push (ssh or something) that information on regular basis to the mail server, where a script verifies that data.
c) To verify the correct delivery: use some sort of checksumming.
Kind regards,
-- Steffen Kaiser
Hi , i think you like to do some monitoring ?
you may do it like this
if you have i.e xymon client on the mailserver and a central xymon monitoring server , write a server mailscript which sends some check mail i.e all 5 minutes from the monitoring server to the mailserver
then on the mailserver write some simple mailcheck monitoring client script if the mail arrived i.e in .new of the maildir mailbox i.e all 5 mins , go yellow if it failed for two times, go red if this fails for three times simply delete the test mail after check, use a special check mailbox which is not accessable from outbound ( postfix access table etc )
if you want additional imap/pop3 check with the test mail the check script could use i.e some imap/pop3 console mail reader perhaps mutt or pine etc for reading and deleting the test mail etc, you may integrate checksumming at some stage
i wouldnt deal with logs for this kind of monitoring, or you may do it addtional
i think there are nagios scripts allready ,which might work this way
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Am 22.01.2013 10:20, schrieb Robert Schetterer:
Am 22.01.2013 10:04, schrieb Steffen Kaiser:
On Mon, 21 Jan 2013, Harford, Ken wrote:
I am having an issue with dovecot IMAP reading and writingto one specific mailbox. I have a check email delivery script that sends
What's the purpose of the verification:
The prompt delivery?
The delivery at all?
The correct delivery?
out a email and then logs into dovecot to check if the mailbox that the message has been delivered to. However, the servers that send these messages send them in bunches depending on how many postfix spools it is checking. So I could get 60 messages all being sent to 1 mailbox and then having to read that same mailbox to check if the message actually got delivered.
I am getting a lot of errors such as the following:
Could not connect to IMAP server imap failed: IMAP RECEIVE CRITICAL - Could not select INBOX
Is this an error logged by Dovecot?
Any ideas as to what I can tweak in dovecot to allow these messages to be verified.
a) Wait a bit before logging into the IMAP server and check the messages sequentionally, not in parallel. E.g. your script could synchronize itself with other parallel instances by a shared lock.
b) Use mail_log and a script on the mail server monitoring Dovecot logs and register delivered message ids and/or recipient. Then figure out a good way to probe, if one pariticular message has been seen. -or- make note of message id/recipient on the sender side, somehow push (ssh or something) that information on regular basis to the mail server, where a script verifies that data.
c) To verify the correct delivery: use some sort of checksumming.
Kind regards,
-- Steffen Kaiser
Hi , i think you like to do some monitoring ?
you may do it like this
if you have i.e xymon client on the mailserver and a central xymon monitoring server , write a server mailscript which sends some check mail i.e all 5 minutes from the monitoring server to the mailserver
then on the mailserver write some simple mailcheck monitoring client script if the mail arrived i.e in .new of the maildir mailbox i.e all 5 mins , go yellow if it failed for two times, go red if this fails for three times simply delete the test mail after check, use a special check mailbox which is not accessable from outbound ( postfix access table etc )
if you want additional imap/pop3 check with the test mail the check script could use i.e some imap/pop3 console mail reader perhaps mutt or pine etc for reading and deleting the test mail etc, you may integrate checksumming at some stage
i wouldnt deal with logs for this kind of monitoring, or you may do it addtional
i think there are nagios scripts allready ,which might work this way
Best Regards MfG Robert Schetterer
however all monitoring server have simple imap and pop3 checks at default so the will alarm you when there get no login slot at the mailserver
real time log monitoring is more complex, and mostly event related which might different from server to server
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
The correct delivery.
The error message is generated by the script. This is a set of perl scripts from Nagios
check_email_delivery check_imap_receive check_smtp_send
I am going to see if I can delay the IMAP portion of the main script,
Thanks for the suggestion Steffen
Ken
On 1/22/13 4:04 AM, "Steffen Kaiser" skdovecot@smail.inf.fh-brs.de wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, 21 Jan 2013, Harford, Ken wrote:
I am having an issue with dovecot IMAP reading and writingto one specific mailbox. I have a check email delivery script that sends
What's the purpose of the verification:
The prompt delivery?
The delivery at all?
The correct delivery?
out a email and then logs into dovecot to check if the mailbox that the message has been delivered to. However, the servers that send these messages send them in bunches depending on how many postfix spools it is checking. So I could get 60 messages all being sent to 1 mailbox and then having to read that same mailbox to check if the message actually got delivered.
I am getting a lot of errors such as the following:
Could not connect to IMAP server imap failed: IMAP RECEIVE CRITICAL - Could not select INBOX
Is this an error logged by Dovecot?
Any ideas as to what I can tweak in dovecot to allow these messages to be verified.
a) Wait a bit before logging into the IMAP server and check the messages sequentionally, not in parallel. E.g. your script could synchronize itself with other parallel instances by a shared lock.
b) Use mail_log and a script on the mail server monitoring Dovecot logs and register delivered message ids and/or recipient. Then figure out a good way to probe, if one pariticular message has been seen. script verifies that data.
- -or- make note of message id/recipient on the sender side, somehow push (ssh or something) that information on regular basis to the mail server, where a
c) To verify the correct delivery: use some sort of checksumming.
Kind regards,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux)
iQEVAwUBUP5WEF3r2wJMiz2NAQIatAgAjwZR1VGnyURuncLRNeeY0dBaqWzAUcNK g4YoiXB6NLxP6Xr/QO6Pw+6Q3sECJ0CJdwImOz56FVFNuiOygUMQ1dCdZ+Tpy9Wu xeAdobrkOo2uzgRnUh7+WLeZ8k32gZZztCAQA90uTvB3MBox+7J/P8/mApp+8/tr JKyhUy8RJIYXH+yrw/ZkbjmOSEEkf/wTPnBQcMKw5/4odha4lu+yEKx766mm0YCe 3USL7eVqN0Q9FwUH+iCjnRJAdKRETP4CnfQpjVp7eiJX7HD0iNwhucDwf037Ev2N PG173M40pMllRyesjiV8gwOu8mAHu6V3QUiFJkYAKFOotUVO6xMrNQ== =SeaP -----END PGP SIGNATURE-----
Okay, I set a wait period of 30 seconds and I am still having issues.
I am sending about 50 messages from 6 servers (300 messages) to the same IMAP mailbox at the same time and dovecot doesn't seem to be able handle that kind of message count.
I am using dovecot delivery instead of postfix delivery
Any other ideas?
Thanks
Ken
On 1/22/13 4:04 AM, "Steffen Kaiser" skdovecot@smail.inf.fh-brs.de wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, 21 Jan 2013, Harford, Ken wrote:
I am having an issue with dovecot IMAP reading and writingto one specific mailbox. I have a check email delivery script that sends
What's the purpose of the verification:
The prompt delivery?
The delivery at all?
The correct delivery?
out a email and then logs into dovecot to check if the mailbox that the message has been delivered to. However, the servers that send these messages send them in bunches depending on how many postfix spools it is checking. So I could get 60 messages all being sent to 1 mailbox and then having to read that same mailbox to check if the message actually got delivered.
I am getting a lot of errors such as the following:
Could not connect to IMAP server imap failed: IMAP RECEIVE CRITICAL - Could not select INBOX
Is this an error logged by Dovecot?
Any ideas as to what I can tweak in dovecot to allow these messages to be verified.
a) Wait a bit before logging into the IMAP server and check the messages sequentionally, not in parallel. E.g. your script could synchronize itself with other parallel instances by a shared lock.
b) Use mail_log and a script on the mail server monitoring Dovecot logs and register delivered message ids and/or recipient. Then figure out a good way to probe, if one pariticular message has been seen. script verifies that data.
- -or- make note of message id/recipient on the sender side, somehow push (ssh or something) that information on regular basis to the mail server, where a
c) To verify the correct delivery: use some sort of checksumming.
Kind regards,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux)
iQEVAwUBUP5WEF3r2wJMiz2NAQIatAgAjwZR1VGnyURuncLRNeeY0dBaqWzAUcNK g4YoiXB6NLxP6Xr/QO6Pw+6Q3sECJ0CJdwImOz56FVFNuiOygUMQ1dCdZ+Tpy9Wu xeAdobrkOo2uzgRnUh7+WLeZ8k32gZZztCAQA90uTvB3MBox+7J/P8/mApp+8/tr JKyhUy8RJIYXH+yrw/ZkbjmOSEEkf/wTPnBQcMKw5/4odha4lu+yEKx766mm0YCe 3USL7eVqN0Q9FwUH+iCjnRJAdKRETP4CnfQpjVp7eiJX7HD0iNwhucDwf037Ev2N PG173M40pMllRyesjiV8gwOu8mAHu6V3QUiFJkYAKFOotUVO6xMrNQ== =SeaP -----END PGP SIGNATURE-----
participants (3)
-
Harford, Ken
-
Robert Schetterer
-
Steffen Kaiser