May 2012 - dovecot - dovecot.org

Re: [Dovecot] Dovecot/doveadm crash
by Daniel L. Miller 01 May '12

01 May '12

On 4/30/2012 1:16 PM, Daniel L. Miller wrote: > Having a problem with a mailbox. I've been trying to rebuild - but > doveadm force-resync crashes. This is mdbox with sis. > Here's a backtrace - gdb --args doveadm force-resync -u msiegel(a)amfes.com Sent GNU gdb (Ubuntu/Linaro 7.2-1ubuntu11) 7.2 Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/local/bin/doveadm...done. (gdb) run Starting program: /usr/local/bin/doveadm force-resync -u msiegel(a)amfes.com Sent [Thread debugging using libthread_db enabled] doveadm(msiegel(a)amfes.com): Warning: mdbox /var/mail/amfes.com/msiegel/mdbox/storage: rebuilding indexes doveadm(msiegel(a)amfes.com): Panic: file istream.c: line 466 (i_stream_grow_buffer): assertion failed: (stream->max_buffer_size > 0) doveadm(msiegel(a)amfes.com): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x4167a) [0x7ffff765467a] -> /usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7ffff7654762] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7ffff762c2a0] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4949b) [0x7ffff765c49b] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_get_buffer_space+0x82) [0x7ffff765c522] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x4b1c8) [0x7ffff765e1c8] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_read+0x69) [0x7ffff765bb69] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_read_next_line+0x1d) [0x7ffff765c04d] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(dbox_file_metadata_read+0xd0) [0x7ffff78dd870] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x3b958) [0x7ffff78da958] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x2a9) [0x7ffff78daff9] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x6b4) [0x7ffff78d97a4] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mdbox_sync+0x34) [0x7ffff78d98d4] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x87) [0x7ffff78d99b7] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) [0x7ffff7918871] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x7ffff7918987] -> /usr/local/bin/doveadm() [0x40e2ff] -> /usr/local/bin/doveadm() [0x40e018] -> /usr/local/bin/doveadm() [0x40ea83] -> /usr/local/bin/doveadm(doveadm_mail_try_run+0x141) [0x40eef1] -> /usr/local/bin/doveadm(main+0x3c1) [0x415911] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff) [0x7ffff729deff] -> /usr/local/bin/doveadm() [0x40de39] Program received signal SIGABRT, Aborted. 0x00007ffff72b2d05 in raise () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt full #0 0x00007ffff72b2d05 in raise () from /lib/x86_64-linux-gnu/libc.so.6 No symbol table info available. #1 0x00007ffff72b6ab6 in abort () from /lib/x86_64-linux-gnu/libc.so.6 No symbol table info available. #2 0x00007ffff7654688 in default_fatal_finish (type=<value optimized out>, status=0) at failures.c:191 backtrace = 0x648140 "/usr/local/lib/dovecot/libdovecot.so.0(+0x4167a) [0x7ffff765467a] -> /usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7ffff7654762] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fat"... #3 0x00007ffff7654762 in default_fatal_handler (ctx=0x7fffffffdc80, format=<value optimized out>, args=<value optimized out>) at failures.c:205 status = 0 #4 0x00007ffff762c2a0 in i_panic (format=0x62b6 <Address 0x62b6 out of bounds>) at failures.c:263 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fffffffdd50, reg_save_area = 0x7fffffffdc90}} #5 0x00007ffff765c49b in i_stream_grow_buffer (stream=0x6c7590, bytes=<value optimized out>) at istream.c:466 old_size = <value optimized out> __FUNCTION__ = "i_stream_grow_buffer" #6 0x00007ffff765c522 in i_stream_get_buffer_space (stream=0x6c7590, wanted_size=<value optimized out>, size_r=<value optimized out>) at istream.c:500 __FUNCTION__ = "i_stream_get_buffer_space" #7 0x00007ffff765e1c8 in i_stream_file_read (stream=0x6c7590) at istream-file.c:58 fstream = 0x6c7590 size = <value optimized out> ret = <value optimized out> __FUNCTION__ = "i_stream_file_read" #8 0x00007ffff765bb69 in i_stream_read (stream=0x6c75f0) at istream.c:130 _stream = 0x6c7590 old_size = 8192 ret = <value optimized out> __FUNCTION__ = "i_stream_read" #9 0x00007ffff765c04d in i_stream_read_next_line (stream=0x6c75f0) at istream.c:382 line = <value optimized out> #10 0x00007ffff78dd870 in dbox_file_metadata_read_at (file=0x6c74a0) at dbox-file.c:661 line = 0x6c98d0 "BSent" buf_size = 8192 ret = 0 #11 dbox_file_metadata_read (file=0x6c74a0) at dbox-file.c:688 metadata_offset = <value optimized out> ret = 0 __FUNCTION__ = "dbox_file_metadata_read" ---Type <return> to continue, or q <return> to quit--- #12 0x00007ffff78da958 in rebuild_file_mails (ctx=0x6b6b90, storage_dir=0x674420 "/var/mail/amfes.com/msiegel/mdbox/storage", alt=<value optimized out>) at mdbox-storage-rebuild.c:139 offset = 16 last = false fixed = false first = <value optimized out> guid = <value optimized out> prev_offset = 0 ret = <value optimized out> rec = 0x7ffff7f99210 old_rec = <value optimized out> #13 rebuild_add_file (ctx=0x6b6b90, storage_dir=0x674420 "/var/mail/amfes.com/msiegel/mdbox/storage", alt=<value optimized out>) at mdbox-storage-rebuild.c:269 file = 0x6c74a0 ext = <value optimized out> deleted = false ret = <value optimized out> file_id = 558 id_str = <value optimized out> #14 mdbox_storage_rebuild_scan_dir (ctx=0x6b6b90, storage_dir=0x674420 "/var/mail/amfes.com/msiegel/mdbox/storage", alt=<value optimized out>) at mdbox-storage-rebuild.c:807 _data_stack_cur_id = 4 dir = <value optimized out> d = <value optimized out> ret = 0 #15 0x00007ffff78daff9 in mdbox_storage_rebuild_scan (storage=0x674200, atomic=<value optimized out>) at mdbox-storage-rebuild.c:855 data = 0x6baed0 data_size = 8 #16 mdbox_storage_rebuild_in_context (storage=0x674200, atomic=<value optimized out>) at mdbox-storage-rebuild.c:888 ctx = 0x6b6b90 ret = 0 #17 0x00007ffff78d97a4 in mdbox_sync_begin (mbox=0x6aec20, flags=MDBOX_SYNC_FLAG_FORCE_REBUILD, atomic=0x6b6b60, ctx_r=0x7fffffffe1b8) at mdbox-sync.c:233 storage = 0x674200 ctx = <value optimized out> sync_flags = <value optimized out> ret = 4 rebuild = true storage_rebuilt = false ---Type <return> to continue, or q <return> to quit--- #18 0x00007ffff78d98d4 in mdbox_sync (mbox=0x6aec20, flags=MDBOX_SYNC_FLAG_FORCE_REBUILD) at mdbox-sync.c:311 sync_ctx = 0x0 atomic = 0x6b6b60 ret = <value optimized out> #19 0x00007ffff78d99b7 in mdbox_storage_sync_init (box=0x6aec20, flags=320) at mdbox-sync.c:341 mbox = 0x6aec20 mdbox_sync_flags = <value optimized out> ret = <value optimized out> #20 0x00007ffff7918871 in mailbox_sync_init (box=0x6aec20, flags=320) at mail-storage.c:1298 _data_stack_cur_id = 3 ctx = <value optimized out> #21 0x00007ffff7918987 in mailbox_sync (box=<value optimized out>, flags=<value optimized out>) at mail-storage.c:1344 ctx = <value optimized out> status = {sync_delayed_expunges = 1} #22 0x000000000040e2ff in cmd_force_resync_box (ctx=0x65e000, user=<value optimized out>) at doveadm-mail.c:211 box = 0x6aec20 ret = 0 #23 cmd_force_resync_run (ctx=0x65e000, user=<value optimized out>) at doveadm-mail.c:240 _data_stack_cur_id = 2 iter = 0x6ad8c0 info = 0x6add38 ret = 0 #24 0x000000000040e018 in doveadm_mail_next_user (ctx=0x65e000, input=<value optimized out>, error_r=0x7fffffffe300) at doveadm-mail.c:311 error = <value optimized out> ret = 0 __FUNCTION__ = "doveadm_mail_next_user" #25 0x000000000040ea83 in doveadm_mail_cmd (cmd=0x6550e0, argc=4, argv=0x64f390) at doveadm-mail.c:518 input = {module = 0x0, service = 0x433222 "doveadm", username = 0x64f3c7 "msiegel(a)amfes.com", session_id = 0x0, local_ip = { family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} ctx = 0x65e000 getopt_args = 0x43147d "AS:u:" wildcard_user = 0x0 error = <value optimized out> ret = <value optimized out> c = <value optimized out> ---Type <return> to continue, or q <return> to quit--- #26 0x000000000040eef1 in doveadm_mail_try_run (cmd_name=0x64f3b7 "force-resync", argc=4, argv=0x64f378) at doveadm-mail.c:577 cmd__foreach_end = 0x655380 cmd = 0x6550e0 cmd_name_len = 12 __FUNCTION__ = "doveadm_mail_try_run" #27 0x0000000000415911 in main (argc=4, argv=0x64f378) at doveadm.c:374 cmd_name = 0x64f3b7 "force-resync" i = <value optimized out> quick_init = false c = <value optimized out> -- Daniel

1 0

[Dovecot] SETANNOTATION for Dovecot
by Dieter Knopf 01 May '12

01 May '12

Hello, i'm using kmail2 and for Folder Settings the SETANNOATION command is needed. Is there any way to add this command to Dovecot? I already found a dovecot-metadata-plugin but there is no Package available for Debian. Thanks Dieter

2 1

[Dovecot] 2.1.5: Panic: file mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: (mailbox_list_is_valid_pattern(_list, name))
by Nick Urbanik 01 May '12

01 May '12

Dear Folks, We had this assertion failure on Dovecot 2.1.1 on a particular mailbox. I upgraded to 2.1.5, and we have the same assertion failure, on the same mailbox: Apr 27 16:02:03 imap(nu-imaptest00258(a)example.com): Panic: file mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: (mailbox_list_is_valid_pattern(_list, name)) Apr 27 16:02:03 imap(nu-imaptest00258(a)example.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x4246a) [0x7f310c28146a] -> /usr/lib64/dovecot/libdovecot.so.0(+0x424b6) +[0x7f310c2814b6] -> /usr/lib64/dovecot/libdovecot.so.0(+0x1950a) [0x7f310c25850a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x889b5) [0x7f310c5539b5] -> +/usr/lib64/dovecot/lib10_quota_plugin.so(+0x8a30) [0x7f3109ccea30] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0x8cdf) [0x7f3109ccecdf] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0x93c8) +[0x7f3109ccf3c8] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0x9436) [0x7f3109ccf436] -> /usr/lib64/dovecot/lib10_quota_plugin.so(quota_get_resource+0x72) [0x7f3109cca792] -> +/usr/lib64/dovecot/lib10_quota_plugin.so(quota_test_alloc+0x1bd) [0x7f3109ccaaed] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0xa870) [0x7f3109cd0870] -> +/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_save_begin+0x4d) [0x7f310c544b9d] -> dovecot/imap [nu-imaptest00258(a)example.com 127.0.0.1 APPEND]() [0x408ac4] -> dovecot/imap +[nu-imaptest00258(a)example.com 127.0.0.1 APPEND](cmd_append+0x139) [0x408d69] -> dovecot/imap [nu-imaptest00258(a)example.com 127.0.0.1 APPEND](command_exec+0x3d) [0x410aed] -> dovecot/imap +[nu-imaptest00258(a)example.com 127.0.0.1 APPEND]() [0x40fa2e] -> dovecot/imap [nu-imaptest00258(a)example.com 127.0.0.1 APPEND]() [0x40fb1a] -> dovecot/imap [nu-imaptest00258(a)example.com +127.0.0.1 APPEND](client_handle_input+0x135) [0x40fd45] -> dovecot/imap [nu-imaptest00258(a)example.com 127.0.0.1 APPEND](client_input+0x5f) [0x41066f] -> +/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f310c28d9d6] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f310c28ea5f] -> +/usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f310c28d978] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f310c279c73] -> dovecot/imap [nu-imaptest00258(a)example.com +127.0.0.1 APPEND](main+0x29d) [0x418b3d] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f310bebccdd] -> dovecot/imap [nu-imaptest00258(a)example.com 127.0.0.1 APPEND]() [0x408449] Apr 27 16:02:03 imaptest01 perdition.imap4[15893]: Closing session: 211.29.132.209:42922->211.29.132.245:143 authorisation_id=NONE authentication_id="nu-imaptest00258" received=67 sent=53 Apr 27 16:02:03 imap(nu-imaptest00258(a)example.com): Fatal: master: service(imap): child 15894 killed with signal 6 (core not dumped) Any suggestions welcome. I guess enabling core dumps is a good start, I'll do that, but has anyone seen this before? -- Nick Urbanik http://nicku.org 808-71011 nick.urbanik(a)optusnet.com.au GPG: 7FFA CDC7 5A77 0558 DC7A 790A 16DF EC5B BB9D 2C24 ID: BB9D2C24 I disclaim, therefore I am.

2 1

[Dovecot] [PATCH] Dovecot 2.1 compiling fix for AIX
by SATOH Fumiyasu 01 May '12

01 May '12

Hi, I'm trying to build Dovecot 2.1.5 on AIX 6.1 and got the following build error: $ ./configure ... $ make ... In file included from stats-connection.c:9: stats-plugin.h:14: error: field 'user_cpu' has incomplete type stats-plugin.h:14: error: field 'sys_cpu' has incomplete type ... The attached patch fixes this problem. -- -- Name: SATOH Fumiyasu (fumiyas @ osstech co jp) -- Business Home: http://www.OSSTech.co.jp/ -- Personal Home: https://github.com/fumiyas/

2 2

[Dovecot] restoring mails and directories
by oni-neko＠gmx.net 01 May '12

01 May '12

Good day! is there an easy way to restore e.g. deleted mails and/or mail directories? let me specify: I'm running dovecot 1.2.9 on a current ubuntu lts. It is set to Maildir and I backup the folder with the users maildata nightly via rsync. now, when someone deletes a mail that they still need, and I copy it back from the backup, it will not be visible to mail clients. When I change the mailfile by a letter or so, it shows. that's easy enough, if not exactly awesome ;-) But, if said person deletes a whole folder of mails, this way to make restored mails show up becomes rather unreasonable. surely there is a better way to do this? I'd be glad for any and all hints! =) best regards silvana -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de

3 2

[Dovecot] Dovecot dsync mail replication issues
by Reuben Farrelly 01 May '12

01 May '12

Hi, I'm trying to set up some dsync based replication between two hosts on my network. The current topology is a single server running Postfix with a single dovecot installation with a Maildir per user (only 4 users including myself). No NFS, just local system users on ext4. I am only using system users, ie no virtual users. I am using dovecot deliver to deliver mails into the Maildir's. This - being a very standard Maildir installation - all works just fine. I'm running dovecot-2.1.5 (release versions) x86_64 with Gentoo on both hosts and using key based ssh to transport the data using the root login. What I would like to do is extend the design so as to replicate the Maildirs across a second machine and in the future be able to connect via IMAP into either machine to sync mail. Two way dsync would be rather cool because I could then deliver via SMTP to either system and have changes automagically propagate - but initially just simple replication would be a good start. I was hoping that dsync would allow me to do this, but I've run into quite a number of problems while getting this to work. http://dovecot.org/list/dovecot/2012-March/064243.html ...was very useful and I've based my config on that. Initially I've tried to sync up the user Maildirs, and this has more or less worked: doveadm sync -u lyn remote:root@dustbowl.reub.net This succeeds without error on the initial sync. However if I try to run the re-sync again (such as a use case of if the sync of another Maildir takes 4 hours so I want to resync up the earlier ones again) I end up with a mysteriously named INBOX folder in both the source and destination Maildirs: drwx------ 5 lyn lyn 4096 Apr 30 19:32 .INBOX_7a86a62d465a974fb92f00003b258734 It has the basic structure of a Maildir but is empty in terms of mails: drwx------ 2 lyn lyn 4096 Apr 30 19:32 cur -rw------- 1 lyn lyn 220 Apr 30 19:32 dovecot.index.log -rw------- 1 lyn lyn 51 Apr 30 19:32 dovecot-uidlist -rw------- 1 lyn lyn 0 Apr 30 19:32 maildirfolder drwx------ 2 lyn lyn 4096 Apr 30 19:32 new drwx------ 2 lyn lyn 4096 Apr 30 19:32 tmp First question: why is this random named directory being created in the origin Maildir? Shouldn't the replication be more or less read-only in the origin Maildir? Second question: If I re-attempt a doveadm sync a second time I get this error: tornado Maildir # doveadm sync -u lyn remote:root@dustbowl.reub.net dsync-local(lyn): Error: Can't rename mailbox INBOX_7a86a62d465a974fb92f00003b258734 to INBOX: Target mailbox already exists dsync-local(lyn): Error: Can't rename mailbox INBOX to INBOX_eb15f30ea563be4b703200002bd68bb1: Renaming INBOX isn't supported. tornado Maildir # It's not clear if the second attempt has failed or succeeded, and it's a bit odd that it errors out on a directory that the dovecot sync process itself has created. Third question: Upon starting Dovecot lots of ugliness is logged in the mail log: Apr 30 19:44:59 tornado dovecot: master: Dovecot v2.1.5 starting up (core dumps disabled) Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Error: user mozsync: Initialization failed: Namespace '': mkdir(/var/www/xxx/server-full/Maildir) failed: Permission denied (euid=1016(mozsync) egid=1016(<unknown>) missing +w perm: /var/www/xxx/server-full, dir owned by 0:0 mode=0755) Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Error: sync: User init failed Apr 30 19:44:59 tornado dovecot: doveadm(mozsync): Warning: I/O leak: 0x414190 (line 102, fd 16) Apr 30 19:44:59 tornado dovecot: dsync-local(cisco): Error: remote: doveadm(cisco): Fatal: User doesn't exist Apr 30 19:44:59 tornado dovecot: dsync-local(cisco): Error: read() from worker server failed: EOF Users mozsync and cisco are not valid mail users and it's not appropriate that Dovecot tries to create a Maildir for either of them. The users are system unprivileged users only, and do not ever send or receive mail. And - I/O leak? ;) Also, user cisco is local to one box only, and does not exist (nor does it need to) on the remote host. So any complaints about this user are invalid and dovecot needs to ignore replication for this user anyway. Fourth question, upon starting dovecot again, mail.log then spews several hundred of these messages: Apr 30 19:45:06 tornado dovecot: dsync-local(reuben): Error: msg-get failed: box=Trash uid=114863 guid=1335382569.M98089P29952.tornado,S=6479,W=6625 Before aborting entirely with: Apr 30 19:45:09 tornado dovecot: imap: Server shutting down. in=328 out=2042 It seems to me that a a few of those problems logged could be solved by being able to specify which system users to synchronise, rather than Dovecot making a blind assumption that all users actually have valid Maildirs that need to be created, and all need to be sync'd between two hosts. Subsequent delivery based sync'ing fails silently (pending more investigation) but I'd like to try and fix the above listed problems first before worrying about the later stages of the sync not working. Has anyone else run into these problems or found better ways of making this type of setup all work? Thanks, Reuben

3 2

[Dovecot] Single Instance Storage management
by Daniel L. Miller 01 May '12

01 May '12

Was there a doveadm command that checks the SIS files and removes any not referenced? Will the "doveadm purge -A" catch that? -- Daniel

2 1