[Dovecot] bug uni_utf8_str_is_valid(vname)
Heya,
We are expiriencing issues with dovecot 2.1.1 on Linux with weird filenames in home directory of username. We are using mbox IMAP folders, with no special changes (mail_location = mbox:~/:INBOX=%h/.mailbox).
Mar 6 13:37:17 machine dovecot: imap(username): Panic: file mail-storage.c: line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname)) Mar 6 13:37:17 machine dovecot: imap(username): Error: Raw backtrace: /opt/dovecot-2.1.1/lib/dovecot/libdovecot.so.0 [0x2ba41cb79450] -> /opt/dovecot-2.1.1/lib/dovecot/libdovecot.so.0 [0x2ba41cb794a6] -> /opt/dovecot-2.1.1/lib/dovecot/libdovecot.so.0 [0x2ba41cb78963] -> /opt/dovecot-2.1.1/lib/dovecot/libdovecot-storage.so.0 [0x2ba41c87ebd5] -> /opt/dovecot-2.1.1/lib/dovecot/libdovecot-storage.so.0 [0x2ba41c88c12c] -> /opt/dovecot-2.1.1/lib/dovecot/libdovecot-storage.so.0(fs_list_iter_next+0x1b4) [0x2ba41c88c494] -> /opt/dovecot-2.1.1/lib/dovecot/libdovecot-storage.so.0 [0x2ba41c885342] -> /opt/dovecot-2.1.1/lib/dovecot/libdovecot-storage.so.0(mailbox_list_iter_next+0x234) [0x2ba41c885604] -> dovecot/imap [0x40b2d1] -> dovecot/imap(cmd_list_full+0x520) [0x40c1f0] -> dovecot/imap(cmd_list+0xb) [0x40c3eb] -> dovecot/imap(command_exec+0x37) [0x410427] -> dovecot/imap [0x40f4cd] -> dovecot/imap [0x40f582] -> dovecot/imap(client_handle_input+0x3f) [0x40f6cf] -> dovecot/imap(client_input+0x62) [0x410052] -> /opt/dovecot Mar 6 13:37:17 machine dovecot: imap(username): Fatal: master: service(imap): child 20873 killed with signal 6 (core dumps disabled)
The bug is reproducible by using home folder structure available from: http://bit.ly/x8pTXS
AFAIK, the problem lies in processing the file list of home folder, which can contain filenames that do not have proper UTF-8 encoding of filenames, which causes dovecot to crash. On the other hand, UTF-8 filenames created on the system by hand (using touch), are not displayed in IMAP LIST command (sample is included in the folder structure; single letter file).
Cheers, Jernej
On Tue, 2012-03-06 at 14:28 +0100, Jernej Porenta wrote:
Heya,
We are expiriencing issues with dovecot 2.1.1 on Linux with weird filenames in home directory of username. We are using mbox IMAP folders, with no special changes (mail_location = mbox:~/:INBOX=% h/.mailbox).
Mar 6 13:37:17 machine dovecot: imap(username): Panic: file mail-storage.c: line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname)) .. AFAIK, the problem lies in processing the file list of home folder, which can contain filenames that do not have proper UTF-8 encoding of filenames, which causes dovecot to crash.
Yes, Dovecot shouldn't crash even if there are non-UTF8 mailboxes. This should fix it by renaming such mailboxes: http://hg.dovecot.org/dovecot-2.1/rev/c077ca9bc306
On the other hand, UTF-8 filenames created on the system by hand (using touch), are not displayed in IMAP LIST command (sample is included in the folder structure; single letter file).
This is a bit trickier problem. The mailbox names are currently stored in filesystem as IMAP's modified-UTF7. So it's not really even currently supposed to work, although it's not very nice that the mailboxes aren't visible either. Maybe I'll do something smart in future for this, like allowing both mUTF-7 and UTF-8 and remembering per-mailbox which formatting it is in.
On Mar 14, 2012, at 3:29 PM, Timo Sirainen wrote:
On Tue, 2012-03-06 at 14:28 +0100, Jernej Porenta wrote:
Heya,
We are expiriencing issues with dovecot 2.1.1 on Linux with weird filenames in home directory of username. We are using mbox IMAP folders, with no special changes (mail_location = mbox:~/:INBOX=% h/.mailbox).
Mar 6 13:37:17 machine dovecot: imap(username): Panic: file mail-storage.c: line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname)) .. AFAIK, the problem lies in processing the file list of home folder, which can contain filenames that do not have proper UTF-8 encoding of filenames, which causes dovecot to crash.
Yes, Dovecot shouldn't crash even if there are non-UTF8 mailboxes. This should fix it by renaming such mailboxes: http://hg.dovecot.org/dovecot-2.1/rev/c077ca9bc306
We tried version 2.1.2, which unfortunately does not fix the issue with weird characters.
Whenever . LIST "" "*" is issued, dovecot crashes: Error: Raw backtrace: /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b54671eb870] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b54671eb8c6] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b54671ead83] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b5466f2a0e5] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b5466f376cc] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b5466f37846] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(fs_list_iter_init+0x4b1) [0x2b5466f38241] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(mailbox_list_iter_init_multiple+0xec) [0x2b5466f3119c] -> dovecot/imap [0x40bbb6] -> dovecot/imap(cmd_list_full+0x520) [0x40c1f0] -> dovecot/imap(cmd_list+0xb) [0x40c3eb] -> dovecot/imap(command_exec+0x37) [0x410497] -> dovecot/imap [0x40f4ed] -> dovecot/imap [0x40f5a2] -> dovecot/imap(client_handle_input+0x3f) [0x40f6ef] -> dovecot/imap(client_input+0x62) [0x410072] -> /opt
Any clues?
On the other hand, UTF-8 filenames created on the system by hand (using touch), are not displayed in IMAP LIST command (sample is included in the folder structure; single letter file).
This is a bit trickier problem. The mailbox names are currently stored in filesystem as IMAP's modified-UTF7. So it's not really even currently supposed to work, although it's not very nice that the mailboxes aren't visible either. Maybe I'll do something smart in future for this, like allowing both mUTF-7 and UTF-8 and remembering per-mailbox which formatting it is in.
I think we can leave this issue out, since I don't believe the users will be creating folders directly from interactive SSH sessions and rather use IMAP to create folders. So, this use-case is not very likely to occur.
Thank you for your help...
Cheers, Jernej
On 16.3.2012, at 11.09, Jernej Porenta wrote:
Mar 6 13:37:17 machine dovecot: imap(username): Panic: file mail-storage.c: line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname)) .. We tried version 2.1.2, which unfortunately does not fix the issue with weird characters.
Whenever . LIST "" "*" is issued, dovecot crashes: Error: Raw backtrace: /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b54671eb870] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b54671eb8c6] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b54671ead83] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b5466f2a0e5] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b5466f376cc] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b5466f37846] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(fs_list_iter_init+0x4b1) [0x2b5466f38241] -> /opt/dovecot
I don't think this is the same Panic as the original one? What is the Panic message now?
On Mar 16, 2012, at 2:14 PM, Timo Sirainen wrote:
On 16.3.2012, at 11.09, Jernej Porenta wrote:
Mar 6 13:37:17 machine dovecot: imap(username): Panic: file mail-storage.c: line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname)) .. We tried version 2.1.2, which unfortunately does not fix the issue with weird characters.
Whenever . LIST "" "*" is issued, dovecot crashes: Error: Raw backtrace: /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b54671eb870] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b54671eb8c6] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b54671ead83] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b5466f2a0e5] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b5466f376cc] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b5466f37846] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(fs_list_iter_init+0x4b1) [0x2b5466f38241] -> /opt/dovecot
I don't think this is the same Panic as the original one? What is the Panic message now?
Mar 19 10:56:35 server dovecot: imap-login: Login: user=<user>, method=PLAIN, rip=193.2.1.110, lip=193.2.1.83, mpid=14732, secured Mar 19 10:56:40 server dovecot: imap(user): Panic: file mail-storage.c: line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname)) Mar 19 10:56:40 server dovecot: imap(user): Error: Raw backtrace: /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b7a91610870] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b7a916108c6] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b7a9160fd83] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b7a9134f0e5] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b7a9135c6cc] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b7a9135c846] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(fs_list_iter_init+0x4b1) [0x2b7a9135d241] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(mailbox_list_iter_init_multiple+0xec) [0x2b7a9135619c] -> dovecot/imap [0x40bbb6] -> dovecot/imap(cmd_list_full+0x520) [0x40c1f0] -> dovecot/imap(cmd_list+0xb) [0x40c3eb] -> dovecot/imap(command_exec+0x37) [0x410497] -> dovecot/imap [0x40f4ed] -> dovecot/imap [0x40f5a2] -> dovecot/imap(client_handle_input+0x3f) [0x40f6ef] -> dovecot/imap(client_input+0x62) [0x410072] -> /opt Mar 19 10:56:40 server dovecot: imap(user): Fatal: master: service(imap): child 14732 killed with signal 6 (core dumps disabled)
It is the same. We will try 2.1.3 today and report the results...
Regards, Jernej
On Mar 19, 2012, at 10:58 AM, Jernej Porenta wrote:
Mar 19 10:56:35 server dovecot: imap-login: Login: user=<user>, method=PLAIN, rip=193.2.1.110, lip=193.2.1.83, mpid=14732, secured Mar 19 10:56:40 server dovecot: imap(user): Panic: file mail-storage.c: line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname)) Mar 19 10:56:40 server dovecot: imap(user): Error: Raw backtrace: /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b7a91610870] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b7a916108c6] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b7a9160fd83] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b7a9134f0e5] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b7a9135c6cc] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b7a9135c846] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(fs_list_iter_init+0x4b1) [0x2b7a9135d241] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(mailbox_list_iter_init_multiple+0xec) [0x2b7a9135619c] -> dovecot/imap [0x40bbb6] -> dovecot/imap(cmd_list_full+0x520) [0x40c1f0] -> dovecot/imap(cmd_list+0xb) [0x40c3eb] -> dovecot/imap(command_exec+0x37) [0x410497] -> dovecot/imap [0x40f4ed] -> dovecot/imap [0x40f5a2] -> dovecot/imap(client_handle_input+0x3f) [0x40f6ef] -> dovecot/imap(client_input+0x62) [0x410072] -> /opt Mar 19 10:56:40 server dovecot: imap(user): Fatal: master: service(imap): child 14732 killed with signal 6 (core dumps disabled)
It is the same. We will try 2.1.3 today and report the results...
Same thing with 2.1.3 (. LIST "" "*"): Mar 19 14:08:59 server dovecot: imap-login: Login: user=<username>, method=PLAIN, rip=193.2.1.110, lip=193.2.1.83, mpid=28438, secured Mar 19 14:09:04 server dovecot: imap(username): Panic: file mail-storage.c: line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname)) Mar 19 14:09:04 server dovecot: imap(username): Error: Raw backtrace: /opt/dovecot-2.1.3/lib/dovecot/libdovecot.so.0 [0x2ae071811870] -> /opt/dovecot-2.1.3/lib/dovecot/libdovecot.so.0 [0x2ae0718118c6] -> /opt/dovecot-2.1.3/lib/dovecot/libdovecot.so.0 [0x2ae071810d83] -> /opt/dovecot-2.1.3/lib/dovecot/libdovecot-storage.so.0 [0x2ae0715500c5] -> /opt/dovecot-2.1.3/lib/dovecot/libdovecot-storage.so.0 [0x2ae07155d6ac] -> /opt/dovecot-2.1.3/lib/dovecot/libdovecot-storage.so.0 [0x2ae07155d826] -> /opt/dovecot-2.1.3/lib/dovecot/libdovecot-storage.so.0(fs_list_iter_init+0x4b1) [0x2ae07155e221] -> /opt/dovecot-2.1.3/lib/dovecot/libdovecot-storage.so.0(mailbox_list_iter_init_multiple+0xec) [0x2ae07155717c] -> dovecot/imap [0x40bbb6] -> dovecot/imap(cmd_list_full+0x520) [0x40c1f0] -> dovecot/imap(cmd_list+0xb) [0x40c3eb] -> dovecot/imap(command_exec+0x37) [0x410497] -> dovecot/imap [0x40f4ed] -> dovecot/imap [0x40f5a2] -> dovecot/imap(client_handle_input+0x3f) [0x40f6ef] -> dovecot/imap(client_input+0x62) [0x410072] -> /opt Mar 19 14:09:04 server dovecot: imap(username): Fatal: master: service(imap): child 28438 killed with signal 6 (core dumps disabled)
The home directory of the username is tar.gzipped here: http://www2.arnes.si/~krklubsls13/username.tar.gz
# dovecot -n # 2.1.3: /opt/dovecot-2.1.3/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.17.1.el5 x86_64 CentOS release 5.7 (Final) default_login_user = nobody disable_plaintext_auth = no login_greeting = Server ready. login_trusted_networks = x.y.z.p/32 mail_location = mbox:~/:INBOX=%h/.mailbox:INDEX=/opt/dovecot2-indexes/%1u/%u mail_plugins = quota mail_privileged_group = dovecot mbox_write_locks = fcntl namespace inbox { inbox = yes location = prefix = separator = / type = private } passdb { args = session=yes dovecot driver = pam } plugin { quota = fs } protocols = imap pop3 service imap-login { inet_listener imap { port = 143 } inet_listener imaps { ssl = no } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { ssl = no } } ssl = no userdb { driver = passwd } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_max_userip_connections = 15 mail_plugins = quota imap_quota } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh }
Regards, Jernej
On Mon, 2012-03-19 at 14:27 +0100, Jernej Porenta wrote:
Mar 19 10:56:40 server dovecot: imap(user): Panic: file mail-storage.c: line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname))
It is the same. We will try 2.1.3 today and report the results...
The home directory of the username is tar.gzipped here: http://www2.arnes.si/~krklubsls13/username.tar.gz
Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/c77fbfce438d
On Mar 19, 2012, at 2:53 PM, Timo Sirainen wrote:
On Mon, 2012-03-19 at 14:27 +0100, Jernej Porenta wrote:
Mar 19 10:56:40 server dovecot: imap(user): Panic: file mail-storage.c: line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname))
It is the same. We will try 2.1.3 today and report the results...
The home directory of the username is tar.gzipped here: http://www2.arnes.si/~krklubsls13/username.tar.gz
Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/c77fbfce438d
Confirmed working…
Thank you again, cheers, Jernej
participants (2)
-
Jernej Porenta
-
Timo Sirainen