September 2015 - dovecot

imap segfaults when using antispam plugin, and moving mail to an 'unsure' folder.
by Andrew McN 02 Sep '15

02 Sep '15

I put up a question on serverfault first, which has the details, including a stack trace: http://serverfault.com/questions/713687/dovecot-antispam-error-storing-mail . Details also copied below. Is this a known bug? If I built dovecot from source rather than using the Ubuntu 14.04 packages, would I expect that to help? Are there any configuration details that might be relevant? (dovecot -n output shown Any help much appreciated. Andrew McNaughton Details on the serverfault page repeated below, plus a little more. ----------------------------------------------------- Ubuntu 14.04.3 x86_64 Running inside lxc-docker 1.7.0 dovecot-imapd 1:2.2.9-1ubuntu2 dovecot-antispam 2.0+20130822-2build1 Filesystem is via docker, so aufs is involved, and under that on the host systems is ext4 ----------------------------------------------------- I know mail is not supposed to be moved into a mailbox of type 'unsure'. The server should refuse this action. What actually happens though is that the imap process throws an error and dies: Aug 12 17:53:07 255c7fc2d2f0 imap: antispam: mailbox_is_unsure(Review-Classification): 1 Aug 12 17:53:07 255c7fc2d2f0 dovecot: imap(user(a)example.com): Panic: file mail-storage.c: line 2141 (mailbox_copy): assertion failed: (!ctx->unfinished) Aug 12 17:53:07 255c7fc2d2f0 dovecot: imap(user(a)example.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x5e271) [0x7f718e7dd271] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e34e) [0x7f718e7dd34e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f718e798a9e] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x7196f) [0x7f718eab896f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_move+0x15) [0x7f718eab8985] -> dovecot/imap(+0xded5) [0x7f718ef84ed5] -> dovecot/imap(command_exec+0x3c) [0x7f718ef8f6dc] -> dovecot/imap(+0x176cf) [0x7f718ef8e6cf] -> dovecot/imap(+0x17785) [0x7f718ef8e785] -> dovecot/imap(client_handle_input+0x125) [0x7f718ef8eaa5] -> dovecot/imap(client_input+0x75) [0x7f718ef8ee35] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x27) [0x7f718e7ed247] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd7) [0x7f718e7edfd7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f718e7ecde8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f718e79dc93] -> dovecot/imap(main+0x2c4) [0x7f718ef832f4] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f718e3dbec5] -> dovecot/imap(+0xc460) [0x7f718ef83460] Aug 12 17:53:07 255c7fc2d2f0 dovecot: imap(user(a)example.com): Fatal: master: service(imap): child 58 killed with signal 6 (core dumped) ----------------------------------------------------------------------- root@8ec9579667fc:/# gdb /usr/lib/dovecot/imap /tmp/61 GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1 Copyright (C) 2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/lib/dovecot/imap...Reading symbols from /usr/lib/debug//usr/lib/dovecot/imap...done. done. [New LWP 61] Core was generated by `dovecot/imap'. Program terminated with signal SIGABRT, Aborted. #0 0x00007f5dd5394cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt full #0 0x00007f5dd5394cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 resultvar = 0 pid = 61 selftid = 61 #1 0x00007f5dd53980d8 in __GI_abort () at abort.c:89 save_stage = 2 act = {__sigaction_handler = {sa_handler = 0x800, sa_sigaction = 0x800}, sa_mask = {__val = {140041026109520, 178, 140041026077952, 512, 140040990095251, 140041026077952, 140041026077952, 512, 140040990087565, 140734823456136, 140040990231826, 140041026077952, 140734823456040, 0, 140040990231929, 140041026077952}}, sa_flags = -713567062, sa_restorer = 0x7fff6128dd01} sigs = {__val = {32, 0 <repeats 15 times>}} #2 0x00007f5dd5781298 in default_fatal_finish (type=<optimized out>, status=status@entry=0) at failures.c:192 backtrace = 0x7f5dd79d0538 "/usr/lib/dovecot/libdovecot.so.0(+0x5e271) [0x7f5dd5781271] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e34e) [0x7f5dd578134e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f5dd573ca9e] -> /usr/lib/d"... #3 0x00007f5dd578134e in i_internal_fatal_handler (ctx=0x7fff6128dd70, format=<optimized out>, args=<optimized out>) at failures.c:653 status = 0 #4 0x00007f5dd573ca9e in i_panic (format=format@entry=0x7f5dd5ab1e68 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:264 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fff6128de60, reg_save_area = 0x7fff6128dda0}} #5 0x00007f5dd5a5c96f in mailbox_copy (_ctx=<optimized out>, mail=mail@entry=0x7f5dd79ffbc0) at mail-storage.c:2141 ctx = 0x7f5dd7a043e0 t = 0x7f5dd79fdf00 keywords = 0x0 pvt_flags = 0 real_mail = <optimized out> ret = -1 __FUNCTION__ = "mailbox_copy" #6 0x00007f5dd5a5c985 in mailbox_move (_ctx=<optimized out>, mail=0x7f5dd79ffbc0) at mail-storage.c:2153 ctx = 0x7f5dd7a043e0 #7 0x00007f5dd5f28ed5 in fetch_and_copy (copy_count_r=<synthetic pointer>, src_uidset_r=<synthetic pointer>, search_args=<optimized out>, src_trans_r=0x7fff6128df18, t=<optimized out>, move=true, client=0x7f5dd79ee440) at cmd-copy.c:67 search_ctx = 0x7f5dd79ff920 src_trans = 0x7f5dd79fec10 srcset_ctx = {str = 0x7f5dd79d0350, first_uid = 0, last_uid = 4294967295} ret = 1 save_ctx = 0x0 mail = 0x7f5dd79ffbc0 copy_count = 1 src_uidset = 0x7f5dd79d0350 #8 cmd_copy_full (cmd=0x7f5dd79eee70, move=<optimized out>) at cmd-copy.c:123 client = 0x7f5dd79ee440 dest_storage = <optimized out> destbox = 0x7f5dd79f8d10 t = 0x7f5dd79fdf00 src_trans = 0x7f5dd573d029 <t_pop_check+9> search_args = 0x7f5dd79f73a0 messageset = 0x7f5dd79dbc10 "76" mailbox = 0x7f5dd79dbc18 "Review-Classification" src_uidset = <optimized out> sync_flags = (unknown: 0) imap_flags = (unknown: 0) changes = {pool = 0x7f5dd79d0350, uid_validity = 0, saved_uids = {arr = {buffer = 0x1, element_size = 140040986301132}, v = 0x1, v_modifiable = 0x1}, ignored_modseq_changes = 3617438728, changed = 93, no_read_perm = 127} msg = <optimized out> ret = <optimized out> __FUNCTION__ = "cmd_copy_full" #9 0x00007f5dd5f336dc in command_exec (cmd=0x7f5dd79eee70) at imap-commands.c:158 hook = 0x7f5dd79d90d0 ret = <optimized out> #10 0x00007f5dd5f326cf in client_command_input (cmd=0x7f5dd79eee70) at imap-client.c:780 client = 0x7f5dd79ee440 command = <optimized out> __FUNCTION__ = "client_command_input" #11 0x00007f5dd5f32785 in client_command_input (cmd=0x7f5dd79eee70) at imap-client.c:841 client = 0x7f5dd79ee440 command = <optimized out> __FUNCTION__ = "client_command_input" #12 0x00007f5dd5f32aa5 in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0x7f5dd79ee440) at imap-client.c:879 No locals. #13 client_handle_input (client=client@entry=0x7f5dd79ee440) at imap-client.c:891 _data_stack_cur_id = 3 remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #14 0x00007f5dd5f32e35 in client_input (client=0x7f5dd79ee440) at imap-client.c:933 cmd = 0x7f5dd79e5ae0 output = 0x7f5dd79eeca0 bytes = 39 __FUNCTION__ = "client_input" #15 0x00007f5dd5791247 in io_loop_call_io (io=0x7f5dd79eed90) at ioloop.c:388 ioloop = 0x7f5dd79d8720 t_id = 2 #16 0x00007f5dd5791fd7 in io_loop_handler_run (ioloop=ioloop@entry=0x7f5dd79d8720) at ioloop-epoll.c:220 ctx = 0x7f5dd79d93b0 list = 0x7f5dd79dade0 io = <optimized out> tv = {tv_sec = 1799, tv_usec = 999178} events_count = <optimized out> msecs = <optimized out> ret = 1 i = 0 call = <optimized out> __FUNCTION__ = "io_loop_handler_run" #17 0x00007f5dd5790de8 in io_loop_run (ioloop=0x7f5dd79d8720) at ioloop.c:412 __FUNCTION__ = "io_loop_run" #18 0x00007f5dd5741c93 in master_service_run (service=0x7f5dd79d85b0, callback=<optimized out>) at master-service.c:566 No locals. #19 0x00007f5dd5f272f4 in main (argc=1, argv=0x7f5dd79d8390) at main.c:400 set_roots = {0x7f5dd614aa00 <imap_setting_parser_info>, 0x0} login_set = {auth_socket_path = 0x7f5dd79d0048 "\001", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x7f5dd5f3c0f0 <login_client_connected>, failure_callback = 0x7f5dd5f3be20 <login_client_failed>, request_auth_token = 1} service_flags = <optimized out> storage_service_flags = <optimized out> username = 0x0 c = <optimized out> ------------------------------------------------------------------------------- root@8ec9579667fc:/# dovecot -n # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-43-generic x86_64 Ubuntu 14.04.3 LTS ext4 auth_debug = yes auth_verbose = yes disable_plaintext_auth = no first_valid_gid = 150 first_valid_uid = 150 last_valid_gid = 150 last_valid_uid = 150 mail_debug = yes mail_gid = vmail mail_location = maildir:/var/vmail/%d/%n/Maildir:INDEX=/var/vmail/%d/%n/Maildir/indexes mail_uid = vmail namespace inbox { inbox = yes location = mailbox Attention-Not-Required { auto = subscribe special_use = \Junk } mailbox Attention { auto = subscribe } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Review-Classification { auto = subscribe } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { antispam_backend = dspam antispam_debug_target = syslog antispam_dspam_args = --deliver=;--user;%u; antispam_dspam_binary = /usr/bin/dspam antispam_signature = X-DSPAM-Signature antispam_signature_missing = error antispam_spam = Attention-Not-Required antispam_trash = Trash antispam_unsure = Review-Classification antispam_verbose_debug = 1 mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster(a)example.net protocols = " imap lmtp" service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } user = vmail } ssl_cert = </etc/dovecot/dovecot.crt ssl_key = </etc/dovecot/private/dovecot.key ssl_protocols = TLSv1.2 TLSv1.1 TLSv1 !SSLv3 !SSLv2 userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = " sieve" } protocol lda { mail_plugins = " sieve" } protocol imap { mail_plugins = " antispam" }

1 1

doveadm-server hangs with 100% cpu usage
by Dennis Kuhn 02 Sep '15

02 Sep '15

we have some replication issues. From time to time a doveadm-server process takes 100% cpu in the state recv_mailbox_tree_deletes on the replica. The process runs forever until it is manually killed. Strace on this process doesn't show anything. Sometimes we have several doveadm-server processes in this state, all for the same account, all with 100% CPU Load. The logfile on the master says: Error: Timeout during state=recv_mailbox_tree Error: dsync(dobby5.heinlein-support.de): I/O has stalled, no activity for 600 seconds My workaround is to delete the user directory on the replica so that the whole account is replicated again. This solves the problem for this specific account. The dovecot version is 2.2.15 on the master and 2.2.16 on the replica. Dennis Kuhn -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-57 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

3 4

Moving mails from shared folder to shared folder doesn't work
by Peer Heinlein 02 Sep '15

02 Sep '15

Hi, having the attached configuration with Dovecot 2.2.9, Dovecot's crashing when moving a mail from a shared folder into another shared folder. Moving that mail into a "real" folder of the user or moving that mail from a real folder into the shared folder is always working. But moving from a shared folder into a shared folder doesn't work. Sep 2 11:47:42 mail02 dovecot: imap(peer.heinlein(a)example.com): Panic: file mail-index.c: line 380 (mail_index_keywords_unref): assertion failed: (keywords->refcount > 0) Sep 2 11:47:42 mail02 dovecot: imap(peer.heinlein(a)example.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x62792) [0x7f87e455f792] -> /usr/lib/dovecot/libdovecot.so.0(+0x627f6) [0x7f87e455f7f6] -> /usr/lib/dovecot/libdovecot.so.0(+0x1c5ca) [0x7f87e45195ca] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb032f) [0x7f87e487232f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x75) [0x7f87e4838315] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_move+0x1d) [0x7f87e48383dd] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](+0xec4a) [0x7f87e4cfbc4a] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](command_exec+0x3d) [0x7f87e4d06acd] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](+0x18b60) [0x7f87e4d05b60] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](+0x18c4d) [0x7f87e4d05c4d] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](client_handle_input+0x11d) [0x7f87e4d05edd] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](client_input+0x72) [0x7f87e4d06292] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f87e456f5f6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xaf) [0x7f87e45706cf] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f87e456f568] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f87e451e0c3] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](main+0x303) [0x7f87e4d10043] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f87e4192c8d] -> dovecot/imap [peer.heinlein(a)example.com 172.31.xxx.xxx UID move](+0xd0e9) [0x7f87e4cfa0e9] Sep 2 11:47:42 mail02 dovecot: imap(peer.heinlein(a)example.com): Fatal: master: service(imap): child 18707 killed with signal 6 (core dumps disabled) Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

1 0

Single Instance Storage after the fact
by Joe Beaubien 02 Sep '15

02 Sep '15

Hi everyone, I tried to find info about this online, but I couldn't find anything talking specifically about what I am trying to accomplish. I am sending and receiving the same attachments pretty often and as a result my email accounts are growing much more rapidly than I would like. *My goal* is to set up a second instance of Dovecot with SIS enabled and transfer everything over to this new Dovecot setup. I was thinking of simply running a dsync backup from the old instance and then a dsync backup -R on the new instance with SIS. Would this effectively create a new set of mailboxes without attachments (and a new folder full of attachments)? If not, can someone point me to information on how to accomplish this? Best regards, Joe B.

1 0