[Dovecot] Dovecot crashes with malformed Qpopper's mbox
I am testing migration from Qpopper 4.0.5 and have come across a problem.
Dovecot dies while processing UIDL command when accessing malformed Qpopper's mbox which has duplicated "From " separator. Qpopper occasionally makes such header while rewriting mbox.
In the attached file, unnecessary "From " at line 3.
Dovecot 1.0.0 on Debian 4.0(x86)
ext3 filesystem(not NFS)
description Dovecot dies while processing UIDL command with malformed Qpopper's mbox which has duplicated "From " separator.
This occurs when "pop3_reuse_xuidl" is "yes".
conversation with Dovecot
$ telnet localhost 110 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. +OK Dovecot ready. user xxxxxxxx +OK pass xxxxxxxx +OK Logged in. uidl Connection closed by foreign host.Dovecot configuration # /usr/local/etc/dovecot.conf protocols: imap pop3 ssl_disable: yes disable_plaintext_auth: no login_dir: /usr/local/var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login mail_extra_groups: mail mail_location: mbox:~/:INBOX=/var/mail/%u mail_full_filesystem_access: yes mbox_write_locks: fcntl mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 pop3_reuse_xuidl(default): no pop3_reuse_xuidl(imap): no pop3_reuse_xuidl(pop3): yes pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv auth default: passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master:
how to reproduce
- set "pop3_reuse_xuidl = yes".
- replace test user's mailbox with attached file.
- pop3 login and exec "UIDL". POP3 process will die.
-- // -------------------------------------------------------------- // MAEDA, Go <maeda-g@secom-sanin.co.jp>
POP3(test01): file ../../../src/lib/array.h: line 157 (_array_idx): assertion failed: (idx * array->element_size < array->buffer->used) POP3(test01): Raw backtrace: pop3 [0x80a9fab] -> pop3 [0x80a99a9] -> pop3 [0x807ec2c] -> pop3(index_mail_get_headers+0x254) [0x807eeb4] -> pop3(index_mail_get_first_header+0x18) [0x807ef48] -> pop3(mail_get_first_header+0x16) [0x809b746] -> pop3 [0x8057ae9] -> pop3 [0x8057c7f] -> pop3 [0x805674e] -> pop3(io_loop_handler_run+0x128) [0x80b03e8] -> pop3(io_loop_run+0x28) [0x80af858] -> pop3(main+0x82) [0x8058792] -> /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xc8) [0xb7dffea8] -> pop3 [0x8055c21] child 4788 (pop3) killed with signal 6
From xxx@xx.xxxxxxx.xx.xx Sat Feb 17 19:27:47 2007 X-UIDL: i[f"!$h*!!S67!!/RW!! From xxx@xx.xxxxxxx.xx.xx Sat Feb 17 19:27:47 2007 Return-Path: <xxx@xx.xxxxxxx.xx.xx> X-Original-To: xxxxxxxx@xx.xxxxxxx.xx.xx Delivered-To: xxxxxxxx@xx.xxxxxxx.xx.xx Received: xxxx xxxx-xxxx-xxxxxx.xxxxxxx.xx.xx (xx [xxx.x.x.x]) xx xx-xxxxxxxx.xxxxxxx.xx.xx (xxxxxxx) xxxx xxxx xx xxxxxxxxxx xxx <xxxxxxxx@xx.xxxxxxx.xx.xx>; xxx, xx xxx xxxx xx:xx:xx +xxxx (xxx) Received: xx xx.xxxxxxx.xx.xx (xxxxxxx, xxxx xxxxxx xx) xx xxxxxxxxxx; xxx, xx xxx xxxx xx:xx:xx +xxxx (xxx) X-Mailer: xxxxxxx xx.x xx xxx.xxxxxx.xx.xx X-HTTP_REFERER: xxxx://xxx.xx-xxxxxxx.xx.xx/xxxx/xxxxx.xxxx Errors-To: xxxxxxxxx@xx-xxxxxxx.xx.xx To: xxxxxxxxx@xx-xxxxxxx.xx.xx From: x.xxxxxxxxx@xxxxxxx.xxx Subject: $x<xxx$x$?$$(xx-xxxx Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="ISO-2022-JP" Message-Id: <xxxxxxxxxxxxxx.xxxxxxxxxx@xx.xxxxxxx.xx.xx> Date: Sat, 17 Feb 2007 19:27:47 +0900 (JST) Status: RO
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
From xxx@xx.xxxxxxx.xx.xx Tue Feb 20 14:06:25 2007 X-UIDL: 7;j"!L;8"!b\%#!pH5!! Return-Path: <xxx@xx.xxxxxxx.xx.xx> X-Original-To: xxxxxxxx@xx.xxxxxxx.xx.xx Delivered-To: xxxxxxxx@xx.xxxxxxx.xx.xx Received: xxxx xxxx-xxxx-xxxxxx.xxxxxxx.xx.xx (xx [xxx.x.x.x]) xx xx-xxxxxxxx.xxxxxxx.xx.xx (xxxxxxx) xxxx xxxx xx xxxxxxxxxx xxx <xxxxxxxx@xx.xxxxxxx.xx.xx>; xxx, xx xxx xxxx xx:xx:xx +xxxx (xxx) Received: xx xx.xxxxxxx.xx.xx (xxxxxxx, xxxx xxxxxx xx) xx xxxxxxxxxx; xxx, xx xxx xxxx xx:xx:xx +xxxx (xxx) X-Mailer: xxxxxxx xx.x xx xxx.xxxxxx.xx.xx X-HTTP_REFERER: xxxx://xxx.xx-xxxxxxx.xx.xx/xxxx/xxxxx.xxxx Errors-To: xxxxxxxxx@xx-xxxxxxx.xx.xx To: xxxxxxxxx@xx-xxxxxxx.xx.xx From: xxxxx@xxxxx-xxx.xx.xx Subject: $x<xxx$x$?$$(xx-xxxx Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="ISO-2022-JP" Message-Id: <xxxxxxxxxxxxxx.xxxxxxxxxx@xx.xxxxxxx.xx.xx> Date: Tue, 20 Feb 2007 14:06:25 +0900 (JST) Status: RO
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
On Thu, 2007-05-24 at 16:51 +0900, 前田 剛 wrote:
Dovecot dies while processing UIDL command when accessing malformed Qpopper's mbox which has duplicated "From " separator. Qpopper occasionally makes such header while rewriting mbox.
Thanks, fixed: http://hg.dovecot.org/dovecot-1.0/rev/89aa4c7f40b7
Dovecot dies while processing UIDL command when accessing malformed Qpopper's mbox which has duplicated "From " separator. Qpopper occasionally makes such header while rewriting mbox.
Thanks, fixed: http://hg.dovecot.org/dovecot-1.0/rev/89aa4c7f40b7
It works. Thanks you.
Although the malformed mbox I sent is consists of 2 messages, Dovecot recognizes 3 messages.
+OK Dovecot ready.
user xxxxxxxx
+OK
pass xxxxxxxx
+OK Logged in.
list
+OK 3 messages:
1 23
2 940
3 968
.
retr 1
+OK 23 octets
X-UIDL: i[f"!$h*!!S67!!/RW!!
.In the above pop3 session, message #1 and #2 is originally one message, so Dovecot should recognize that the whole mbox contains 2 messages, not 3 messages.
I would like you to consider improving Dovecot to skip second (and duplicated, unnecessary) "From" separator.
-- // -------------------------------------------------------------- // MAEDA, Go <maeda-g@secom-sanin.co.jp>
On Thu, 2007-05-24 at 19:31 +0900, 前田 剛 wrote:
I would like you to consider improving Dovecot to skip second (and duplicated, unnecessary) "From" separator.
That won't be happening for v1.0.x and not really a priority to me for v1.1 either. Related to this in TODO:
- always add empty line. make the parser require it too? syncing should
make sure there always exists two LFs at end of file. raw-mbox-stream
should make sure the last message ends with LF even if it doesn't exist
in the fileIf you're interested in fixing it yourself I can give a bit of help :)
That won't be happening for v1.0.x and not really a priority to me for v1.1 either. Related to this in TODO:
I see. I know that few people encounter with this problem.
If you're interested in fixing it yourself I can give a bit of help :)
Thanks. I'll try to fix it myself. Please give me advice. The function I have to modify is _read() in istream-raw-mbox.c, isn't it? And need to add some code to skip duplicated "From " separator, I guess.
-- // -------------------------------------------------------------- // MAEDA, Go <maeda-g@secom-sanin.co.jp>
On 24.5.2007, at 17.53, MAEDA, Go wrote:
If you're interested in fixing it yourself I can give a bit of
help :)Thanks. I'll try to fix it myself. Please give me advice. The function I have to modify is _read() in istream-raw-mbox.c,
isn't it? And need to add some code to skip duplicated "From " separator,
I guess.
Yep. I think the way to do this right is:
- Instead of requiring "\n[\r]From ", require "\n[\r]\n[\r]From "
- in mbox-save.c make sure that the extra empty line is always
added (I'm not sure if this already is done, I think it isn't) - preferrably also fix broken mboxes in mbox-sync.c and mbox-
save.c: If you notice that it doesn't end with two linefeeds, add one
or two so it does
The optional CRs are a bit annoying to handle correctly..
Timo Sirainen wrote:
- Instead of requiring "\n[\r]From ", require "\n[\r]\n[\r]From "
- in mbox-save.c make sure that the extra empty line is always added (I'm not sure if this already is done, I think it isn't)
- preferrably also fix broken mboxes in mbox-sync.c and mbox-save.c: If you notice that it doesn't end with two linefeeds, add one or two so it does
At first, I have modified istream-raw-mbox.c and now my Dovecot stops mistaking the secound "From " line for a message separator.
But RETRed message contains unnecessary (may be harmful for some pop3 clients) "From " line.
+OK Dovecot ready.
user xxxxxxxx
+OK
pass xxxxxxxx
+OK Logged in.
top 1 0
+OK
X-UIDL: i[f"!$h*!!S67!!/RW!!
From xxx@xx.xxxxxxx.xx.xx Sat Feb 17 19:27:47 2007
Return-Path: <xxx@xx.xxxxxxx.xx.xx>
:
:
:I have thought that save_header_callback() in mbox-save.c should drop the "From " line but it still left in the message.
Timo, please give me more advice. What should I do to workaround this?
-- // -------------------------------------------------------------- // MAEDA, Go <maeda-g@secom-sanin.co.jp>
*** dovecot-1.0.0/src/lib-storage/index/mbox/istream-raw-mbox.c 2007-03-14 00:32:37.000000000 +0900 --- dovecot-1.0.0-fix/src/lib-storage/index/mbox/istream-raw-mbox.c 2007-05-25 20:23:07.000000000 +0900
*** 243,255 **** i++; from_after_pos = i; from_start_pos = i - 6; ! if (from_start_pos > 0 && ! buf[from_start_pos-1] == '\r') { ! /* CR also belongs to it. */ ! crlf_ending = TRUE; from_start_pos--; ! } else { crlf_ending = FALSE; } } fromp = mbox_from; --- 243,260 ---- i++; from_after_pos = i; from_start_pos = i - 6; ! ! if (from_start_pos > 2 && ! memcmp(buf + from_start_pos - 3, ! "\r\n\r", 3) == 0) { from_start_pos--; ! crlf_ending = TRUE; ! } else if (from_start_pos > 0 && ! buf[from_start_pos - 1] == '\n') { crlf_ending = FALSE;
} } fromp = mbox_from;
On Fri, 2007-05-25 at 22:09 +0900, MAEDA, Go wrote:
At first, I have modified istream-raw-mbox.c and now my Dovecot stops mistaking the secound "From " line for a message separator.
But RETRed message contains unnecessary (may be harmful for some pop3 clients) "From " line. .. I have thought that save_header_callback() in mbox-save.c should drop the "From " line but it still left in the message.
That's only when saving the message. If you want to filter it when reading the mailbox, you'll have to add a similar callback to mbox-mail.c:mbox_mail_get_stream().
plain text document attachment (patch.txt)
And could you please use diff -u format when sending patches. The default format is horrible to read. :)
participants (3)
-
MAEDA, Go
-
Timo Sirainen
-
前田 剛