[Dovecot] Sieve problem. Timo, is this mbox file size limitation hard coded? If so, why?
I would not have expected this upon implementing sieve. And I really need to get around this limitation. LDA has no problem writing to these large mbox files. Why does sieve have a problem with them? This is very odd.
May 10 17:45:04 greer dovecot: deliver(stan): write() failed with mbox file /home/stan/mail/1-Debian-Users: File too large May 10 17:45:04 greer dovecot: deliver(stan): write() failed with mbox file /home/stan/mail/1-Debian-Users: File too large May 10 17:45:04 greer dovecot: deliver(stan): sieve: msgid=20100510224422.GA4293@heima.gjk.dk: failed to store into mailbox '1-Debian-Users': Internal error occurred. Refer to server log for more information. [2010-05-10 17:45:04] May 10 17:45:04 greer dovecot: deliver(stan): sieve: msgid=20100510224422.GA4293@heima.gjk.dk: stored mail into mailbox 'INBOX' May 10 17:45:04 greer dovecot: deliver(stan): sieve: execution of script /home/stan/.dovecot.sieve failed, but implicit keep was successful
May 10 17:47:41 greer dovecot: deliver(stan): write() failed with mbox file /home/stan/mail/1-Spam-l: File too large May 10 17:47:41 greer dovecot: deliver(stan): sieve: msgid=AANLkTinOywSh2KzgJhfHEihRSqKQn_Tg9KWnOwazmVmp@mail.gmail.com: failed store into mailbox '1-Spam-l': Internal error occurred. Refer to server log for more information. [2010-05-10 17:47:41] May 10 17:47:41 greer dovecot: deliver(stan): sieve: msgid=AANLkTinOywSh2KzgJhfHEihRSqKQn_Tg9KWnOwazmVmp@mail.gmail.com: stored il into mailbox 'INBOX' May 10 17:47:41 greer dovecot: deliver(stan): sieve: execution of script /home/stan/.dovecot.sieve failed, but implicit keep was scessful
[05:52:24][root@greer]/usr/bin$ la /home/stan/mail/ total 301M drwx------ 5 stan stan 4.0K May 7 04:33 . drwxr-xr-x 10 stan stan 4.0K May 10 16:18 .. drwx------ 37 stan stan 4.0K May 7 04:33 .imap -rw------- 1 stan stan 463 May 7 04:33 .subscriptions -rw------- 1 stan stan 66M May 10 17:45 1-Debian-Users -rw------- 1 stan stan 14M May 10 17:16 1-Dovecot -rw------- 1 stan stan 3.6M May 10 16:05 1-Linux-IDE -rw------- 1 stan stan 30M May 10 17:48 1-Postfix-Users -rw------- 1 stan stan 1.4M May 10 12:33 1-Roundcube -rw------- 1 stan stan 17M May 10 15:55 1-Samba -rw------- 1 stan stan 56M May 10 17:47 1-Spam-l -rw------- 1 stan stan 6.7M May 10 17:45 1-XFS -rw------- 1 stan stan 18M May 10 16:32 SpamTrap
Is there any reason we have a fixed ceiling on the size of an mbox sieve will write to? Man this is disappointing. What is the ceiling? It appears to be somewhere between 30MB and 56MB as writes to 1-Postfix-Users are successful but writes to 1-Spam-l fail.
Is there a fix/patch for this?
Thanks.
-- Stan
Apologies for pointing the finger at lda/sieve. After much digging and Googling I found that the problem actually lies with a Postfix default setting, specifically:
"mailbox_size_limit (default: 51200000)
The maximal size of any local(8) individual mailbox or maildir file, or
zero (no limit). In fact, this limits the size of any file that is written to upon local delivery, including files written by external commands that are executed by the local(8) delivery agent.
This limit must not be smaller than the message size limit."
I set it to zero and that fixed the problem. This is indeed one strange situation. The error was reported in the log by dovecot, but the setting that fixes the problem is actually in Postfix. It would be nice if there was a short heads up regarding this in the dovecot wiki postfix LDA and/or sieve sections. I'll mention this on postfix-user as well.
-- Stan
Stan Hoeppner put forth on 5/10/2010 6:03 PM:
I would not have expected this upon implementing sieve. And I really need to get around this limitation. LDA has no problem writing to these large mbox files. Why does sieve have a problem with them? This is very odd.
May 10 17:45:04 greer dovecot: deliver(stan): write() failed with mbox file /home/stan/mail/1-Debian-Users: File too large May 10 17:45:04 greer dovecot: deliver(stan): write() failed with mbox file /home/stan/mail/1-Debian-Users: File too large May 10 17:45:04 greer dovecot: deliver(stan): sieve: msgid=20100510224422.GA4293@heima.gjk.dk: failed to store into mailbox '1-Debian-Users': Internal error occurred. Refer to server log for more information. [2010-05-10 17:45:04] May 10 17:45:04 greer dovecot: deliver(stan): sieve: msgid=20100510224422.GA4293@heima.gjk.dk: stored mail into mailbox 'INBOX' May 10 17:45:04 greer dovecot: deliver(stan): sieve: execution of script /home/stan/.dovecot.sieve failed, but implicit keep was successful
May 10 17:47:41 greer dovecot: deliver(stan): write() failed with mbox file /home/stan/mail/1-Spam-l: File too large May 10 17:47:41 greer dovecot: deliver(stan): sieve: msgid=AANLkTinOywSh2KzgJhfHEihRSqKQn_Tg9KWnOwazmVmp@mail.gmail.com: failed store into mailbox '1-Spam-l': Internal error occurred. Refer to server log for more information. [2010-05-10 17:47:41] May 10 17:47:41 greer dovecot: deliver(stan): sieve: msgid=AANLkTinOywSh2KzgJhfHEihRSqKQn_Tg9KWnOwazmVmp@mail.gmail.com: stored il into mailbox 'INBOX' May 10 17:47:41 greer dovecot: deliver(stan): sieve: execution of script /home/stan/.dovecot.sieve failed, but implicit keep was scessful
[05:52:24][root@greer]/usr/bin$ la /home/stan/mail/ total 301M drwx------ 5 stan stan 4.0K May 7 04:33 . drwxr-xr-x 10 stan stan 4.0K May 10 16:18 .. drwx------ 37 stan stan 4.0K May 7 04:33 .imap -rw------- 1 stan stan 463 May 7 04:33 .subscriptions -rw------- 1 stan stan 66M May 10 17:45 1-Debian-Users -rw------- 1 stan stan 14M May 10 17:16 1-Dovecot -rw------- 1 stan stan 3.6M May 10 16:05 1-Linux-IDE -rw------- 1 stan stan 30M May 10 17:48 1-Postfix-Users -rw------- 1 stan stan 1.4M May 10 12:33 1-Roundcube -rw------- 1 stan stan 17M May 10 15:55 1-Samba -rw------- 1 stan stan 56M May 10 17:47 1-Spam-l -rw------- 1 stan stan 6.7M May 10 17:45 1-XFS -rw------- 1 stan stan 18M May 10 16:32 SpamTrap
Is there any reason we have a fixed ceiling on the size of an mbox sieve will write to? Man this is disappointing. What is the ceiling? It appears to be somewhere between 30MB and 56MB as writes to 1-Postfix-Users are successful but writes to 1-Spam-l fail.
Is there a fix/patch for this?
Thanks.
On 11/05/10 04:06, Stan Hoeppner wrote:
It would be nice if there was a short heads up regarding this in the dovecot wiki postfix LDA and/or sieve sections. I'll mention this on postfix-user as well.
From http://wiki.dovecot.org/LDA:
"Note that Postfix's mailbox_size_limit setting applies to all files that are written to. So if you have a limit of 50 MB, deliver can't write to log files larger than 50 MB and you'll start getting temporary failures."
Bob
B. Johannessen put forth on 5/11/2010 1:31 AM:
On 11/05/10 04:06, Stan Hoeppner wrote:
It would be nice if there was a short heads up regarding this in the dovecot wiki postfix LDA and/or sieve sections. I'll mention this on postfix-user as well.
From http://wiki.dovecot.org/LDA:
"Note that Postfix's mailbox_size_limit setting applies to all files that are written to. So if you have a limit of 50 MB, deliver can't write to log files larger than 50 MB and you'll start getting temporary failures."
No wonder I didn't find it. It's buried in the "logging" section of the main LDA page instead of the Postfix section, where one would assume to find _Postfix specific_ LDA information.
Timo, any chance we could get this text moved into an appropriate place on the Postfix LDA page where it belongs? I'm sure it would be very helpful to future Postfix/LDA/mbox users. And also add the error message they'll see in the logs when they run into this problem, something like:
Jan 1 00:00:00 hostname dovecot: deliver(user): write() failed with mbox file /home/user/mail/foo: File too large
Thanks for pointing this out Bob.
-- Stan
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi,
On 2010-05-12 11:31, Stan Hoeppner wrote:
From http://wiki.dovecot.org/LDA:
"Note that Postfix's mailbox_size_limit setting applies to all files that are written to. So if you have a limit of 50 MB, deliver can't write to log files larger than 50 MB and you'll start getting temporary failures."
No wonder I didn't find it. It's buried in the "logging" section of the main LDA page instead of the Postfix section, where one would assume to find _Postfix specific_ LDA information.
Timo, any chance we could get this text moved into an appropriate place on the Postfix LDA page where it belongs? I'm sure it would be very helpful to future Postfix/LDA/mbox users. And also add the error message they'll see in the logs when they run into this problem, something like:
Jan 1 00:00:00 hostname dovecot: deliver(user): write() failed with mbox file /home/user/mail/foo: File too large
It's a Wiki, why don't you just change it? :)
Patrick.
Key ID: 0x86E346D4 http://patrick-nagel.net/key.asc Fingerprint: 7745 E1BE FA8B FBAD 76AB 2BFC C981 E686 86E3 46D4 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org/
iEYEARECAAYFAkvqJE0ACgkQyYHmhobjRtRKAQCfbO1/sHTsuO6yfbLz1XDRQZ7y crIAoM7qH+ItVgFAUicDCkRYeVxmTYGT =QbSU -----END PGP SIGNATURE-----
On 2010-05-10 10:06 PM, Stan Hoeppner wrote:
I set it to zero and that fixed the problem. This is indeed one strange situation. The error was reported in the log by dovecot, but the setting that fixes the problem is actually in Postfix. It would be nice if there was a short heads up regarding this in the dovecot wiki postfix LDA and/or sieve sections. I'll mention this on postfix-user as well.
Bob already pointed you to the wiki entry, but it was also *telling* you to look in the server (postfix) logs, where the error is most likely much clearer:
May 10 17:45:04 greer dovecot: deliver(stan): sieve: msgid=20100510224422.GA4293@heima.gjk.dk: failed to store into mailbox '1-Debian-Users': Internal error occurred. Refer to server log for more information. [2010-05-10 17:45:04]
:)
--
Best regards,
Charles
Charles Marcus put forth on 5/11/2010 7:53 AM:
On 2010-05-10 10:06 PM, Stan Hoeppner wrote:
I set it to zero and that fixed the problem. This is indeed one strange situation. The error was reported in the log by dovecot, but the setting that fixes the problem is actually in Postfix. It would be nice if there was a short heads up regarding this in the dovecot wiki postfix LDA and/or sieve sections. I'll mention this on postfix-user as well.
Bob already pointed you to the wiki entry,
You're reading messages out of order Charles. Bob didn't post his message until 4.5 hours _after_ I'd already solved the problem and posted what you quote above.
but it was also *telling* you to look in the server (postfix) logs, where the error is most likely much clearer:
May 10 17:45:04 greer dovecot: deliver(stan): sieve: msgid=20100510224422.GA4293@heima.gjk.dk: failed to store into mailbox '1-Debian-Users': Internal error occurred. Refer to server log for more information. [2010-05-10 17:45:04]
:)
Charles, first off, that _IS_ the "server" log you're looking at, from both mail.err and mail.log. Second, Postfix doesn't log _anything_ relating this error. That is the entirety of this problem. I made that abundantly clear in my previous two emails on this subject. The fact that Postfix doesn't log anything, but that LDA does log it on behalf of Postfix, but _without telling the OP this is the case_, makes standard troubleshooting procedure useless.
Postfix generates the error condition but Dovecot LDA reports it. _This should be fixed_. If Postfix generates the error it should be Postfix stamping the log with the error, not Dovecot. If for some reason changing it so that Postfix logs the error is difficult or can't be done, then at the very least Dovecot should add information to the logged error telling the OP to troubleshoot in the direction of Postfix, _not_ Dovecot. From all I've found to this point, this situation _ONLY_ occurs with Postfix. So log it that way for $deity's sake. Anyway...
I used the Dovecot Wiki search function to find the error message and it returned nothing, zilch: http://wiki.dovecot.org/LDA?action=fullsearch&context=180&value=write%28%29+failed+with+mbox+file&titlesearch=Titles
I then reread the Postfix/LDA section of the wiki and found nothing. I then turned to Google and found the answer via a couple of archived list/forum posts a couple of years old from OPs that ran into the same situation I did. Thank $deity for Google. The posts I found were to neither this list nor the Postfix list. IIRC they were on Linux distro list archives, CentOS maybe.
Most people managing computer systems are logical thinkers. If we weren't, we'd be doing something else for a living. It's _not_ logical to:
- Have one software report errors on behalf of another software
- Then give no clue that this is what's happening in the log
- Have the documentation of the issue buried in an illogical location
I took all the logical steps to troubleshoot and resolve this. My efforts were thwarted by:
- An illogical error reporting mechanism
- Documentation located in an illogical place
Charles, you make it sound as if I'm some nitwit of an OP who can't read logs properly. I hope you now understand that the problem is the opposite. The problem was my thorough, _logical_ reading of the logs. The logs said look at Dovecot LDA, but the problem was Postfix.
In closing, there are currently _5_ MTA specific subsections of the LDA wiki. The documentation describing this _Postfix specific_ issue is in the main LDA page under "Logging". There is no other MTA specific information in the "Logging" section but for Postfix. This information belongs in the Postfix section. In addition, Deovecot needs to add additional text to the error message sent to the log somehow identifying that the error is related to Postfix.
-- Stan
On Mon, 2010-05-10 at 21:06 -0500, Stan Hoeppner wrote:
May 10 17:45:04 greer dovecot: deliver(stan): write() failed with mbox file /home/stan/mail/1-Debian-Users: File too large
http://hg.dovecot.org/dovecot-2.0/rev/f7527971fed0 will probably help in future too.
participants (5)
-
B. Johannessen
-
Charles Marcus
-
Patrick Nagel
-
Stan Hoeppner
-
Timo Sirainen