[Dovecot] Possible mbox corruption if mail is saved with wrong Content-Length header (2.1.6)
One of our users likes to delete attachments from his mails. He is using roundcube and we installed a plugin for that.
He reports that from time to time one of his mails gets corrupted (cut off in the middle).
I could verify there is something wrong in the logs:
imap(x1234567): Error: Cached message size larger than expected (6796 > 2844) imap(x1234567): Error: Corrupted index cache file /net/mbx3/disc2/home/85/x1234567/Index/.imap/Sent/dovecot.index.cache: Broken physical size for mail UID 5001 imap(x1234567): Error: read(/net/mbx3/disc2/home/85/x1234567/Mail/Sent) failed: Input/output error (FETCH for mailbox Sent UID 5001) imap(x1234567): Info: Disconnected: Internal error occurred. Refer to server log for more information. [2012-08-23 00:00:49] in=1124145 out=1324791
I had a look at the plugin and it works by duplicating the message and leaving out the attachment(s). The headers are also duplicated, including Content-Length. I've fixed that in the plugin, (don't know yet if it helps), but I think it shouldn't cause such breakage in dovecot.
# doveconf -n # 2.1.6: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.9-103.ELsmp i686 Red Hat Enterprise Linux AS release 4 (Nahant Update 9) doveconf: Warning: Dovecot was last started using /usr/local/dovecot/etc/dovecot.conf, but this config is /usr/local/dovecot/etc/dovecot/dovecot.conf auth_master_user_separator = * disable_plaintext_auth = no doveadm_password = (hidden) lda_mailbox_autocreate = yes log_path = /dev/stderr log_timestamp = mail_location = mbox:~/Mail:INBOX=~/Mailbox:INDEX=~/Index mail_plugins = " quota zlib mail_log notify virtual" mdbox_rotate_size = 5 M namespace { hidden = no inbox = yes location = prefix = separator = / type = private } namespace { hidden = yes inbox = no list = children location = maildir:~/Maildir prefix = "#MD/" separator = / type = private } passdb { driver = pam } passdb { args = /usr/local/dovecot/etc/passwd.dovcot.master driver = passwd-file master = yes } plugin { acl = vfile mail_log_events = undelete mailbox_delete mailbox_rename mailbox_create mail_log_fields = uid box msgid size quota = fs:Disk quota } protocols = pop3 imap service auth { client_limit = 4300 user = root } service doveadm { inet_listener { port = 542 } } service imap-login { inet_listener imap { address = 134.147.64.42 port = 143 } service_count = 0 } service imap { drop_priv_before_exec = yes process_limit = 4096 vsz_limit = 1 G } service pop3-login { inet_listener pop3 { address = 134.147.64.42 port = 110 } service_count = 0 } service pop3 { drop_priv_before_exec = yes process_limit = 2048 vsz_limit = 512 M } ssl = no userdb { driver = passwd } protocol imap { mail_max_userip_connections = 50 mail_plugins = " quota zlib mail_log notify virtual imap_quota imap_zlib" }
Thanks Jost Krieger
| Helft Spam ausrotten! HTML in Mail ist unhöflich. | | Postmaster, JAPH, manchmal Wahrsager am RZ der RUB | | Wahre Worte sind nicht gefällig, gefällige Worte sind nicht wahr.| | Lao Tse, Tao Te King 81 |
On 31.8.2012, at 11.42, Jost Krieger wrote:
One of our users likes to delete attachments from his mails. He is using roundcube and we installed a plugin for that.
Does the plugin change the mbox file directly?
imap(x1234567): Error: Cached message size larger than expected (6796 > 2844)
This looks like it does. Existing emails must not be changed, IMAP protocol forbids it (and Dovecot internals don't support it either). The only safe way to drop attachments is to assign a new UID for the mail, which basically means saving a new message without attachments and deleting the old one. The Content-Length header doesn't really have anything to do with this error.
[Sorry about the personal reply.]
On Sun, Sep 02, 2012 at 05:28:28PM +0300, Timo Sirainen wrote:
On 31.8.2012, at 11.42, Jost Krieger wrote:
One of our users likes to delete attachments from his mails. He is using roundcube and we installed a plugin for that.
Does the plugin change the mbox file directly?
No, it just uses IMAP.
imap(x1234567): Error: Cached message size larger than expected (6796 > 2844)
This looks like it does. Existing emails must not be changed, IMAP protocol forbids it (and Dovecot internals don't support it either). The only safe way to drop attachments is to assign a new UID for the mail, which basically means saving a new message without attachments and deleting the old one. The Content-Length header doesn't really have anything to do with this error.
Sorry if I was unclear. This is exactly what the plugin does. It builds a new message, starting with the old message and leaving out the unwanted attchment(s). Then it saves the new message, deletes the old message and finally copies over the flags.
I had a look at the handling of "Content-Length" in dovecot and had the impression that a new header was added but the old was not removed (if formally ok) and on reading the first header wins. But I'm not sure, of course.
Yours Jost Krieger
| Jost.Krieger+sig@ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 |
On 3.9.2012, at 17.11, Jost Krieger wrote:
imap(x1234567): Error: Cached message size larger than expected (6796 > 2844)
This looks like it does. Existing emails must not be changed, IMAP protocol forbids it (and Dovecot internals don't support it either). The only safe way to drop attachments is to assign a new UID for the mail, which basically means saving a new message without attachments and deleting the old one. The Content-Length header doesn't really have anything to do with this error.
Sorry if I was unclear. This is exactly what the plugin does. It builds a new message, starting with the old message and leaving out the unwanted attchment(s). Then it saves the new message, deletes the old message and finally copies over the flags.
I had a look at the handling of "Content-Length" in dovecot and had the impression that a new header was added but the old was not removed (if formally ok) and on reading the first header wins. But I'm not sure, of course.
Any Content-Length header sent by IMAP client or via LDA input should always get dropped (among some other headers) and replaced with the correct value. So if there's a bug, I doubt it has anything to do with that. But if you can easily reproduce this failure it would be nice to get it fixed..
On Tue 11 Sep 2012 05:33:10 PM GMT, Timo Sirainen wrote:
Any Content-Length header sent by IMAP client or via LDA input should always get dropped (among some other headers) and replaced with the correct value. So if there's a bug, I doubt it has anything to do with that. But if you can easily reproduce this failure it would be nice to get it fixed..
You're right, of course. And I cannot reproduce the problem. The user that could reproduce it has somehow given up.
I'v experimented and read the source, and there is no way a "Content-Length" could be saved by dovecot :-)
But ...
This is on a system where delivery is not (yet) via dovecot, but directly by qmail. Two possibilities come to mind:
We have a locking problem that only surfaces when qmail and dovecot try to write at the same time to the end of the mbox. Most people (and clients) hardly ever do an APPEND to the INBOX.
Something qmail writes to the INBOX makes dovecot hiccup (like a wrong Content-Length header :-;).
Which would anyone think more likely?
Yours Jost Krieger
| Jost.Krieger+sig@ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 |
participants (2)
-
Jost Krieger
-
Timo Sirainen