- dovecot - dovecot.org

[Dovecot] dsync - Panic: file mail-index-transaction-update.c: line 19
by Götz Reinicke - IT Koordinator 13 May '14

13 May '14

Hi, we have started to switch our users mails from mbox to maildir and see for some users panics. May be someone has a hint on whats causing that and how to fix it? It's dovecot 2.2.10 on Redhat. dsync(s109487): Error: Corrupted index cache file /srv/vmail/s109487/mail/.imap/GS Drehbuch 1/dovecot.index.cache: Broken physical size for mail UID 42 dsync(s109487): Error: mail parser: read(/srv/vmail/s109487/mail/GS Drehbuch 1, box=GS Drehbuch 1) failed: Invalid argument dsync(s109487): Panic: file mail-index-transaction-update.c: line 19 (mail_index_transaction_lookup): assertion failed: (seq >= t->first_new_seq && seq <= t->last_new_seq) dsync(s109487): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x332ea6722a] -> /usr/lib64/dovecot/ Thanks and regards . Götz -- Götz Reinicke IT-Koordinator Tel. +49 7141 969 82 420 Fax +49 7141 969 55 420 E-Mail goetz.reinicke(a)filmakademie.de Filmakademie Baden-Württemberg GmbH Akademiehof 10 71638 Ludwigsburg www.filmakademie.de Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: Jürgen Walter MdL Staatssekretär im Ministerium für Wissenschaft, Forschung und Kunst Baden-Württemberg Geschäftsführer: Prof. Thomas Schadt

1 0

[Dovecot] Released Pigeonhole v0.4.3 for Dovecot v2.2.13.
by Stephan Bosch 13 May '14

13 May '14

Hello Dovecot users, Now that Dovecot v2.2.13 is released, I can release a new Pigeonhole. Due to changes in Dovecot, this Pigeonhole release will not compile cleanly against older Dovecot releases. This release is mainly about bugfixes. One notable addition is that I have updated the "vnd.dovecot.duplicate" extension to the new IETF draft "duplicate" extension. Changelog v0.4.3: * Editheader extension: Made control characters allowed for editheader, except NUL. Before, this would cause a runtime error. + Upgraded Dovecot-specific Sieve "vnd.dovecot.duplicate" extension to match the new draft "duplicate" extension. - Fixed sieve_result_global_log_error to log only as i_info in administrator log (syslog) if executed from multiscript context. - Sieve redirect extension: Adjusted loop detection to show leniency to resent messages. - Sieve include extension: Fixed problem with handling of duplicate includes with different parameters :once or :optional. - Sieve spamtest/virustest extensions: Tests were erroneously performed against the original message. When used together with extprograms filter to add the spam headers, the changes were not being used by the spamtest and virustest extensions. - Deprecated Sieve notify extension: Fixed segfault problems in message string substitution. - ManageSieve: Fixed active link verification to handle redundant path slashes correctly. - Sieve vacation extension: - Fixed interaction of sieve_vacation_dont_check_recipient with sieve_vacation_send_from_recipient setting. - Fixed log message for discarded response. - Sieve extprograms plugin: - Forgot to disable the alarm() timeouts set for script execution. - Fixed fd leak and handling of output shutdown. - Fixed 'Bad filedescriptor' error occurring when disconnecting script client. - Made sure that programs are never forked with root privileges. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.3.tar… http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.3.tar… Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

1 0

[Dovecot] 2.2.12: Panic: file mail-index.c: line 380 (mail_index_keywords_unref): assertion failed: (keywords->refcount > 0)
by Leho Kraav 13 May '14

13 May '14

I upgraded 2.1 -> 2.2 something like a week ago because I needed INDEXPVT. Not sure if this crash started immediately or not, noticed it today looking at journalctl. Backtrace http://bpaste.net/raw/181944/ and pasted below. This seems to crash on every IMAP connection made, so any ideas for a possible client-level workaround are quite welcome until dovecot code improves here. root@server ~ $ dovecot -n # 2.2.12: /etc/dovecot/dovecot.conf # OS: Linux 3.4.54-vs2.3.3.5+pf64 x86_64 Gentoo Base System release 2.2 ext4 hostname = *hidden* listen = 192.168.1.2 log_path = /dev/stderr log_timestamp = mail_gid = mail mail_home = /secure/Maildir/%n mail_location = Maildir:/secure/Maildir/%n mail_plugins = acl mail_privileged_group = mail mail_uid = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { list = children location = maildir:/secure/Maildir/%%n:INDEXPVT=/secure/Maildir/%n/shared/%%n prefix = Shared.%%n. separator = . subscriptions = no type = shared } namespace { location = maildir:/secure/Maildir/projekt:INDEX=/secure/Maildir/%n/projekt prefix = Projekt. separator = . subscriptions = no type = public } namespace { location = maildir:/secure/Maildir/rss:INDEX=/secure/Maildir/%n/rss prefix = RSS. separator = . subscriptions = no type = public } namespace inbox { hidden = no inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_shared_dict = file:/secure/Maildir/shared-mailboxes.db mail_log_events = mailbox_delete sieve = /secure/Maildir/%n/dovecot-sieve sieve_dir = /secure/Maildir/%n/sieve sieve_global_path = /etc/dovecot/sieve/default.sieve } postmaster_address = postmaster protocols = imap lmtp sieve service auth { unix_listener auth-userdb { mode = 0600 user = mail } user = mail } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = </etc/ssl/server.crt ssl_key = </etc/ssl/server.key userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lda { mail_plugins = acl sieve acl } protocol imap { mail_max_userip_connections = 20 mail_plugins = acl imap_acl } protocol sieve { managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date } #0 0x00007f64da799535 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f64da79a9b8 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007f64dab76dc5 in default_fatal_finish (type=<optimized out>, status=status@entry=0) at failures.c:193 backtrace = 0xae54a8 "/usr/lib64/dovecot/libdovecot.so.0(+0x6adcf) [0x7f64dab76dcf] -> /usr/lib64/dovecot/libdovecot.so.0(+0x6ae2e) [0x7f64dab76e2e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f64dab30d4e] -> /usr"... #3 0x00007f64dab76e2e in i_internal_fatal_handler (ctx=0x7fffb9d88d00, format=<optimized out>, args=<optimized out>) at failures.c:657 status = 0 #4 0x00007f64dab30d4e in i_panic (format=format@entry=0x7f64dae79848 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:267 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fffb9d88df0, reg_save_area = 0x7fffb9d88d30}} #5 0x00007f64dae59fbc in mail_index_keywords_unref (_keywords=<optimized out>) at mail-index.c:380 keywords = <optimized out> __FUNCTION__ = "mail_index_keywords_unref" #6 0x00007f64dae22657 in mailbox_copy (_ctx=<optimized out>, mail=mail@entry=0xbcf260) at mail-storage.c:2140 ctx = 0xbd3a80 t = 0xbcd580 keywords = 0xbd4a50 pvt_flags = 0 real_mail = <optimized out> ret = -1 __FUNCTION__ = "mailbox_copy" #7 0x00007f64dae2270d in mailbox_move (_ctx=<optimized out>, mail=0xbcf260) at mail-storage.c:2153 ctx = 0xbd3a80 #8 0x000000000040e72d in fetch_and_copy (copy_count_r=<synthetic pointer>, src_uidset_r=<synthetic pointer>, search_args=<optimized out>, src_trans_r=0x7fffb9d88e88, t=0xbcd580, move=true, client=0xb0de50) at cmd-copy.c:67 search_ctx = 0xbcefe0 src_trans = 0xbce2c0 srcset_ctx = {str = 0xae5320, first_uid = 0, last_uid = 4294967295} ret = 1 save_ctx = 0x0 mail = 0xbcf260 copy_count = 1 src_uidset = 0xae5320 #9 cmd_copy_full (cmd=0xb0ea30, move=true) at cmd-copy.c:123 client = 0xb0de50 dest_storage = <optimized out> destbox = 0xb87bf0 t = 0xbcd580 src_trans = <optimized out> search_args = 0xb86be0 messageset = 0xaf0b60 "2631" mailbox = 0xaf0b68 "Trash" src_uidset = <optimized out> sync_flags = (unknown: 0) imap_flags = (unknown: 0) changes = {pool = 0x7fffb9d88fb0, uid_validity = 3117977312, saved_uids = {arr = {buffer = 0xd, element_size = 140071145425344}, v = 0xd, v_modifiable = 0xd}, ignored_modseq_changes = 11471704, changed = false, no_read_perm = false} copy_count = <optimized out> msg = <optimized out> ret = <optimized out> __FUNCTION__ = "cmd_copy_full" #10 0x0000000000418e6c in command_exec (cmd=cmd@entry=0xb0ea30) at imap-commands.c:158 hook = 0xaee0f0 ret = <optimized out> #11 0x0000000000417ed0 in client_command_input (cmd=0xb0ea30) at imap-client.c:780 client = 0xb0de50 command = <optimized out> __FUNCTION__ = "client_command_input" #12 0x0000000000417f8a in client_command_input (cmd=0xb0ea30) at imap-client.c:841 client = 0xb0de50 command = <optimized out> __FUNCTION__ = "client_command_input" #13 0x0000000000418245 in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0xb0de50) at imap-client.c:879 No locals. #14 client_handle_input (client=client@entry=0xb0de50) at imap-client.c:891 _data_stack_cur_id = 3 ret = 64 remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #15 0x00000000004185f2 in client_input (client=0xb0de50) at imap-client.c:933 cmd = <optimized out> output = 0xb0e880 bytes = 26 __FUNCTION__ = "client_input" #16 0x00007f64dab873b6 in io_loop_call_io (io=0xb0e950) at ioloop.c:388 ioloop = 0xaed740 t_id = 2 #17 0x00007f64dab88237 in io_loop_handler_run (ioloop=ioloop@entry=0xaed740) at ioloop-epoll.c:220 ctx = 0xaee3d0 events = 0x0 event = 0xaef1f0 list = 0xaefdc0 io = <optimized out> tv = {tv_sec = 4, tv_usec = 995201} events_count = <optimized out> msecs = <optimized out> ret = 1 i = <optimized out> call = <optimized out> __FUNCTION__ = "io_loop_handler_run" #18 0x00007f64dab86ec8 in io_loop_run (ioloop=0xaed740) at ioloop.c:412 __FUNCTION__ = "io_loop_run" #19 0x00007f64dab36033 in master_service_run (service=0xaed5d0, callback=callback@entry=0x4216a0 <client_connected>) at master-service.c:566 No locals. #20 0x000000000040cc87 in main (argc=1, argv=0xaed390) at main.c:400 set_roots = {0x429180 <imap_setting_parser_info>, 0x0} login_set = {auth_socket_path = 0xae5048 "\001", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x421550 <login_client_connected>, failure_callback = 0x421260 <login_client_failed>, request_auth_token = 1} service_flags = <optimized out> storage_service_flags = <optimized out> username = 0x0 c = <optimized out> #0 0x00007f64da799535 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f64da79a9b8 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007f64dab76dc5 in default_fatal_finish (type=<optimized out>, status=status@entry=0) at failures.c:193 backtrace = 0xae54a8 "/usr/lib64/dovecot/libdovecot.so.0(+0x6adcf) [0x7f64dab76dcf] -> /usr/lib64/dovecot/libdovecot.so.0(+0x6ae2e) [0x7f64dab76e2e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f64dab30d4e] -> /usr"... #3 0x00007f64dab76e2e in i_internal_fatal_handler (ctx=0x7fffb9d88d00, format=<optimized out>, args=<optimized out>) at failures.c:657 status = 0 #4 0x00007f64dab30d4e in i_panic (format=format@entry=0x7f64dae79848 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:267 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fffb9d88df0, reg_save_area = 0x7fffb9d88d30}} #5 0x00007f64dae59fbc in mail_index_keywords_unref (_keywords=<optimized out>) at mail-index.c:380 keywords = <optimized out> __FUNCTION__ = "mail_index_keywords_unref" #6 0x00007f64dae22657 in mailbox_copy (_ctx=<optimized out>, mail=mail@entry=0xbcf260) at mail-storage.c:2140 ctx = 0xbd3a80 t = 0xbcd580 keywords = 0xbd4a50 pvt_flags = 0 real_mail = <optimized out> ret = -1 __FUNCTION__ = "mailbox_copy" #7 0x00007f64dae2270d in mailbox_move (_ctx=<optimized out>, mail=0xbcf260) at mail-storage.c:2153 ctx = 0xbd3a80 #8 0x000000000040e72d in fetch_and_copy (copy_count_r=<synthetic pointer>, src_uidset_r=<synthetic pointer>, search_args=<optimized out>, src_trans_r=0x7fffb9d88e88, t=0xbcd580, move=true, client=0xb0de50) at cmd-copy.c:67 search_ctx = 0xbcefe0 src_trans = 0xbce2c0 srcset_ctx = {str = 0xae5320, first_uid = 0, last_uid = 4294967295} ret = 1 save_ctx = 0x0 mail = 0xbcf260 copy_count = 1 src_uidset = 0xae5320 #9 cmd_copy_full (cmd=0xb0ea30, move=true) at cmd-copy.c:123 client = 0xb0de50 dest_storage = <optimized out> destbox = 0xb87bf0 t = 0xbcd580 src_trans = <optimized out> search_args = 0xb86be0 messageset = 0xaf0b60 "2631" mailbox = 0xaf0b68 "Trash" src_uidset = <optimized out> sync_flags = (unknown: 0) imap_flags = (unknown: 0) changes = {pool = 0x7fffb9d88fb0, uid_validity = 3117977312, saved_uids = {arr = {buffer = 0xd, element_size = 140071145425344}, v = 0xd, v_modifiable = 0xd}, ignored_modseq_changes = 11471704, changed = false, no_read_perm = false} copy_count = <optimized out> msg = <optimized out> ret = <optimized out> __FUNCTION__ = "cmd_copy_full" #10 0x0000000000418e6c in command_exec (cmd=cmd@entry=0xb0ea30) at imap-commands.c:158 hook = 0xaee0f0 ret = <optimized out> #11 0x0000000000417ed0 in client_command_input (cmd=0xb0ea30) at imap-client.c:780 client = 0xb0de50 command = <optimized out> __FUNCTION__ = "client_command_input" #12 0x0000000000417f8a in client_command_input (cmd=0xb0ea30) at imap-client.c:841 client = 0xb0de50 command = <optimized out> __FUNCTION__ = "client_command_input" #13 0x0000000000418245 in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0xb0de50) at imap-client.c:879 No locals. #14 client_handle_input (client=client@entry=0xb0de50) at imap-client.c:891 _data_stack_cur_id = 3 ret = 64 remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #15 0x00000000004185f2 in client_input (client=0xb0de50) at imap-client.c:933 cmd = <optimized out> output = 0xb0e880 bytes = 26 __FUNCTION__ = "client_input" #16 0x00007f64dab873b6 in io_loop_call_io (io=0xb0e950) at ioloop.c:388 ioloop = 0xaed740 t_id = 2 #17 0x00007f64dab88237 in io_loop_handler_run (ioloop=ioloop@entry=0xaed740) at ioloop-epoll.c:220 ctx = 0xaee3d0 events = 0x0 event = 0xaef1f0 list = 0xaefdc0 io = <optimized out> tv = {tv_sec = 4, tv_usec = 995201} events_count = <optimized out> msecs = <optimized out> ret = 1 i = <optimized out> call = <optimized out> __FUNCTION__ = "io_loop_handler_run" #18 0x00007f64dab86ec8 in io_loop_run (ioloop=0xaed740) at ioloop.c:412 __FUNCTION__ = "io_loop_run" #19 0x00007f64dab36033 in master_service_run (service=0xaed5d0, callback=callback@entry=0x4216a0 <client_connected>) at master-service.c:566 No locals. #20 0x000000000040cc87 in main (argc=1, argv=0xaed390) at main.c:400 set_roots = {0x429180 <imap_setting_parser_info>, 0x0} login_set = {auth_socket_path = 0xae5048 "\001", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x421550 <login_client_connected>, failure_callback = 0x421260 <login_client_failed>, request_auth_token = 1} service_flags = <optimized out> storage_service_flags = <optimized out> username = 0x0 c = <optimized out> #0 0x00007f64da799535 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f64da79a9b8 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007f64dab76dc5 in default_fatal_finish (type=<optimized out>, status=status@entry=0) at failures.c:193 backtrace = 0xae54a8 "/usr/lib64/dovecot/libdovecot.so.0(+0x6adcf) [0x7f64dab76dcf] -> /usr/lib64/dovecot/libdovecot.so.0(+0x6ae2e) [0x7f64dab76e2e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f64dab30d4e] -> /usr"... #3 0x00007f64dab76e2e in i_internal_fatal_handler (ctx=0x7fffb9d88d00, format=<optimized out>, args=<optimized out>) at failures.c:657 status = 0 #4 0x00007f64dab30d4e in i_panic (format=format@entry=0x7f64dae79848 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:267 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fffb9d88df0, reg_save_area = 0x7fffb9d88d30}} #5 0x00007f64dae59fbc in mail_index_keywords_unref (_keywords=<optimized out>) at mail-index.c:380 keywords = <optimized out> __FUNCTION__ = "mail_index_keywords_unref" #6 0x00007f64dae22657 in mailbox_copy (_ctx=<optimized out>, mail=mail@entry=0xbcf260) at mail-storage.c:2140 ctx = 0xbd3a80 t = 0xbcd580 keywords = 0xbd4a50 pvt_flags = 0 real_mail = <optimized out> ret = -1 __FUNCTION__ = "mailbox_copy" #7 0x00007f64dae2270d in mailbox_move (_ctx=<optimized out>, mail=0xbcf260) at mail-storage.c:2153 ctx = 0xbd3a80 #8 0x000000000040e72d in fetch_and_copy (copy_count_r=<synthetic pointer>, src_uidset_r=<synthetic pointer>, search_args=<optimized out>, src_trans_r=0x7fffb9d88e88, t=0xbcd580, move=true, client=0xb0de50) at cmd-copy.c:67 search_ctx = 0xbcefe0 src_trans = 0xbce2c0 srcset_ctx = {str = 0xae5320, first_uid = 0, last_uid = 4294967295} ret = 1 save_ctx = 0x0 mail = 0xbcf260 copy_count = 1 src_uidset = 0xae5320 #9 cmd_copy_full (cmd=0xb0ea30, move=true) at cmd-copy.c:123 client = 0xb0de50 dest_storage = <optimized out> destbox = 0xb87bf0 t = 0xbcd580 src_trans = <optimized out> search_args = 0xb86be0 messageset = 0xaf0b60 "2631" mailbox = 0xaf0b68 "Trash" src_uidset = <optimized out> sync_flags = (unknown: 0) imap_flags = (unknown: 0) changes = {pool = 0x7fffb9d88fb0, uid_validity = 3117977312, saved_uids = {arr = {buffer = 0xd, element_size = 140071145425344}, v = 0xd, v_modifiable = 0xd}, ignored_modseq_changes = 11471704, changed = false, no_read_perm = false} copy_count = <optimized out> msg = <optimized out> ret = <optimized out> __FUNCTION__ = "cmd_copy_full" #10 0x0000000000418e6c in command_exec (cmd=cmd@entry=0xb0ea30) at imap-commands.c:158 hook = 0xaee0f0 ret = <optimized out> #11 0x0000000000417ed0 in client_command_input (cmd=0xb0ea30) at imap-client.c:780 client = 0xb0de50 command = <optimized out> __FUNCTION__ = "client_command_input" #12 0x0000000000417f8a in client_command_input (cmd=0xb0ea30) at imap-client.c:841 client = 0xb0de50 command = <optimized out> __FUNCTION__ = "client_command_input" #13 0x0000000000418245 in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0xb0de50) at imap-client.c:879 No locals. #14 client_handle_input (client=client@entry=0xb0de50) at imap-client.c:891 _data_stack_cur_id = 3 ret = 64 remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #15 0x00000000004185f2 in client_input (client=0xb0de50) at imap-client.c:933 cmd = <optimized out> output = 0xb0e880 bytes = 26 __FUNCTION__ = "client_input" #16 0x00007f64dab873b6 in io_loop_call_io (io=0xb0e950) at ioloop.c:388 ioloop = 0xaed740 t_id = 2 #17 0x00007f64dab88237 in io_loop_handler_run (ioloop=ioloop@entry=0xaed740) at ioloop-epoll.c:220 ctx = 0xaee3d0 events = 0x0 event = 0xaef1f0 list = 0xaefdc0 io = <optimized out> tv = {tv_sec = 4, tv_usec = 995201} events_count = <optimized out> msecs = <optimized out> ret = 1 i = <optimized out> call = <optimized out> __FUNCTION__ = "io_loop_handler_run" #18 0x00007f64dab86ec8 in io_loop_run (ioloop=0xaed740) at ioloop.c:412 __FUNCTION__ = "io_loop_run" #19 0x00007f64dab36033 in master_service_run (service=0xaed5d0, callback=callback@entry=0x4216a0 <client_connected>) at master-service.c:566 No locals. #20 0x000000000040cc87 in main (argc=1, argv=0xaed390) at main.c:400 set_roots = {0x429180 <imap_setting_parser_info>, 0x0} login_set = {auth_socket_path = 0xae5048 "\001", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x421550 <login_client_connected>, failure_callback = 0x421260 <login_client_failed>, request_auth_token = 1} service_flags = <optimized out> storage_service_flags = <optimized out> username = 0x0 c = <optimized out> #0 0x00007f64da799535 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f64da79a9b8 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007f64dab76dc5 in default_fatal_finish (type=<optimized out>, status=status@entry=0) at failures.c:193 backtrace = 0xae54a8 "/usr/lib64/dovecot/libdovecot.so.0(+0x6adcf) [0x7f64dab76dcf] -> /usr/lib64/dovecot/libdovecot.so.0(+0x6ae2e) [0x7f64dab76e2e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f64dab30d4e] -> /usr"... #3 0x00007f64dab76e2e in i_internal_fatal_handler (ctx=0x7fffb9d88d00, format=<optimized out>, args=<optimized out>) at failures.c:657 status = 0 #4 0x00007f64dab30d4e in i_panic (format=format@entry=0x7f64dae79848 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:267 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fffb9d88df0, reg_save_area = 0x7fffb9d88d30}} #5 0x00007f64dae59fbc in mail_index_keywords_unref (_keywords=<optimized out>) at mail-index.c:380 keywords = <optimized out> __FUNCTION__ = "mail_index_keywords_unref" #6 0x00007f64dae22657 in mailbox_copy (_ctx=<optimized out>, mail=mail@entry=0xbcf260) at mail-storage.c:2140 ctx = 0xbd3a80 t = 0xbcd580 keywords = 0xbd4a50 pvt_flags = 0 real_mail = <optimized out> ret = -1 __FUNCTION__ = "mailbox_copy" #7 0x00007f64dae2270d in mailbox_move (_ctx=<optimized out>, mail=0xbcf260) at mail-storage.c:2153 ctx = 0xbd3a80 #8 0x000000000040e72d in fetch_and_copy (copy_count_r=<synthetic pointer>, src_uidset_r=<synthetic pointer>, search_args=<optimized out>, src_trans_r=0x7fffb9d88e88, t=0xbcd580, move=true, client=0xb0de50) at cmd-copy.c:67 search_ctx = 0xbcefe0 src_trans = 0xbce2c0 srcset_ctx = {str = 0xae5320, first_uid = 0, last_uid = 4294967295} ret = 1 save_ctx = 0x0 mail = 0xbcf260 copy_count = 1 src_uidset = 0xae5320 #9 cmd_copy_full (cmd=0xb0ea30, move=true) at cmd-copy.c:123 client = 0xb0de50 dest_storage = <optimized out> destbox = 0xb87bf0 t = 0xbcd580 src_trans = <optimized out> search_args = 0xb86be0 messageset = 0xaf0b60 "2631" mailbox = 0xaf0b68 "Trash" src_uidset = <optimized out> sync_flags = (unknown: 0) imap_flags = (unknown: 0) changes = {pool = 0x7fffb9d88fb0, uid_validity = 3117977312, saved_uids = {arr = {buffer = 0xd, element_size = 140071145425344}, v = 0xd, v_modifiable = 0xd}, ignored_modseq_changes = 11471704, changed = false, no_read_perm = false} copy_count = <optimized out> msg = <optimized out> ret = <optimized out> __FUNCTION__ = "cmd_copy_full" #10 0x0000000000418e6c in command_exec (cmd=cmd@entry=0xb0ea30) at imap-commands.c:158 hook = 0xaee0f0 ret = <optimized out> #11 0x0000000000417ed0 in client_command_input (cmd=0xb0ea30) at imap-client.c:780 client = 0xb0de50 command = <optimized out> __FUNCTION__ = "client_command_input" #12 0x0000000000417f8a in client_command_input (cmd=0xb0ea30) at imap-client.c:841 client = 0xb0de50 command = <optimized out> __FUNCTION__ = "client_command_input" #13 0x0000000000418245 in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0xb0de50) at imap-client.c:879 No locals. #14 client_handle_input (client=client@entry=0xb0de50) at imap-client.c:891 _data_stack_cur_id = 3 ret = 64 remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #15 0x00000000004185f2 in client_input (client=0xb0de50) at imap-client.c:933 cmd = <optimized out> output = 0xb0e880 bytes = 26 __FUNCTION__ = "client_input" #16 0x00007f64dab873b6 in io_loop_call_io (io=0xb0e950) at ioloop.c:388 ioloop = 0xaed740 t_id = 2 #17 0x00007f64dab88237 in io_loop_handler_run (ioloop=ioloop@entry=0xaed740) at ioloop-epoll.c:220 ctx = 0xaee3d0 events = 0x0 event = 0xaef1f0 list = 0xaefdc0 io = <optimized out> tv = {tv_sec = 4, tv_usec = 995201} events_count = <optimized out> msecs = <optimized out> ret = 1 i = <optimized out> call = <optimized out> __FUNCTION__ = "io_loop_handler_run" #18 0x00007f64dab86ec8 in io_loop_run (ioloop=0xaed740) at ioloop.c:412 __FUNCTION__ = "io_loop_run" #19 0x00007f64dab36033 in master_service_run (service=0xaed5d0, callback=callback@entry=0x4216a0 <client_connected>) at master-service.c:566 No locals. #20 0x000000000040cc87 in main (argc=1, argv=0xaed390) at main.c:400 set_roots = {0x429180 <imap_setting_parser_info>, 0x0} login_set = {auth_socket_path = 0xae5048 "\001", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x421550 <login_client_connected>, failure_callback = 0x421260 <login_client_failed>, request_auth_token = 1} service_flags = <optimized out> storage_service_flags = <optimized out> username = 0x0 c = <optimized out> #0 0x00007f64da799535 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f64da79a9b8 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007f64dab76dc5 in default_fatal_finish (type=<optimized out>, status=status@entry=0) at failures.c:193 backtrace = 0xae54a8 "/usr/lib64/dovecot/libdovecot.so.0(+0x6adcf) [0x7f64dab76dcf] -> /usr/lib64/dovecot/libdovecot.so.0(+0x6ae2e) [0x7f64dab76e2e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f64dab30d4e] -> /usr"... #3 0x00007f64dab76e2e in i_internal_fatal_handler (ctx=0x7fffb9d88d00, format=<optimized out>, args=<optimized out>) at failures.c:657 status = 0 #4 0x00007f64dab30d4e in i_panic (format=format@entry=0x7f64dae79848 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:267 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fffb9d88df0, reg_save_area = 0x7fffb9d88d30}} #5 0x00007f64dae59fbc in mail_index_keywords_unref (_keywords=<optimized out>) at mail-index.c:380 keywords = <optimized out> __FUNCTION__ = "mail_index_keywords_unref" #6 0x00007f64dae22657 in mailbox_copy (_ctx=<optimized out>, mail=mail@entry=0xbcf260) at mail-storage.c:2140 ctx = 0xbd3a80 t = 0xbcd580 keywords = 0xbd4a50 pvt_flags = 0 real_mail = <optimized out> ret = -1 __FUNCTION__ = "mailbox_copy" #7 0x00007f64dae2270d in mailbox_move (_ctx=<optimized out>, mail=0xbcf260) at mail-storage.c:2153 ctx = 0xbd3a80 #8 0x000000000040e72d in fetch_and_copy (copy_count_r=<synthetic pointer>, src_uidset_r=<synthetic pointer>, search_args=<optimized out>, src_trans_r=0x7fffb9d88e88, t=0xbcd580, move=true, client=0xb0de50) at cmd-copy.c:67 search_ctx = 0xbcefe0 src_trans = 0xbce2c0 srcset_ctx = {str = 0xae5320, first_uid = 0, last_uid = 4294967295} ret = 1 save_ctx = 0x0 mail = 0xbcf260 copy_count = 1 src_uidset = 0xae5320 #9 cmd_copy_full (cmd=0xb0ea30, move=true) at cmd-copy.c:123 client = 0xb0de50 dest_storage = <optimized out> destbox = 0xb87bf0 t = 0xbcd580 src_trans = <optimized out> search_args = 0xb86be0 messageset = 0xaf0b60 "2631" mailbox = 0xaf0b68 "Trash" src_uidset = <optimized out> sync_flags = (unknown: 0) imap_flags = (unknown: 0) changes = {pool = 0x7fffb9d88fb0, uid_validity = 3117977312, saved_uids = {arr = {buffer = 0xd, element_size = 140071145425344}, v = 0xd, v_modifiable = 0xd}, ignored_modseq_changes = 11471704, changed = false, no_read_perm = false} copy_count = <optimized out> msg = <optimized out> ret = <optimized out> __FUNCTION__ = "cmd_copy_full" #10 0x0000000000418e6c in command_exec (cmd=cmd@entry=0xb0ea30) at imap-commands.c:158 hook = 0xaee0f0 ret = <optimized out> #11 0x0000000000417ed0 in client_command_input (cmd=0xb0ea30) at imap-client.c:780 client = 0xb0de50 command = <optimized out> __FUNCTION__ = "client_command_input" #12 0x0000000000417f8a in client_command_input (cmd=0xb0ea30) at imap-client.c:841 client = 0xb0de50 command = <optimized out> __FUNCTION__ = "client_command_input" #13 0x0000000000418245 in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0xb0de50) at imap-client.c:879 No locals. #14 client_handle_input (client=client@entry=0xb0de50) at imap-client.c:891 _data_stack_cur_id = 3 ret = 64 remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #15 0x00000000004185f2 in client_input (client=0xb0de50) at imap-client.c:933 cmd = <optimized out> output = 0xb0e880 bytes = 26 __FUNCTION__ = "client_input" #16 0x00007f64dab873b6 in io_loop_call_io (io=0xb0e950) at ioloop.c:388 ioloop = 0xaed740 t_id = 2 #17 0x00007f64dab88237 in io_loop_handler_run (ioloop=ioloop@entry=0xaed740) at ioloop-epoll.c:220 ctx = 0xaee3d0 events = 0x0 event = 0xaef1f0 list = 0xaefdc0 io = <optimized out> tv = {tv_sec = 4, tv_usec = 995201} events_count = <optimized out> msecs = <optimized out> ret = 1 i = <optimized out> call = <optimized out> __FUNCTION__ = "io_loop_handler_run" #18 0x00007f64dab86ec8 in io_loop_run (ioloop=0xaed740) at ioloop.c:412 __FUNCTION__ = "io_loop_run" #19 0x00007f64dab36033 in master_service_run (service=0xaed5d0, callback=callback@entry=0x4216a0 <client_connected>) at master-service.c:566 No locals. #20 0x000000000040cc87 in main (argc=1, argv=0xaed390) at main.c:400 set_roots = {0x429180 <imap_setting_parser_info>, 0x0} login_set = {auth_socket_path = 0xae5048 "\001", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x421550 <login_client_connected>, failure_callback = 0x421260 <login_client_failed>, request_auth_token = 1} service_flags = <optimized out> storage_service_flags = <optimized out> username = 0x0 c = <optimized out>

3 2

[Dovecot] IMAP disconnect before greeting / banner
by Benoit Panizzon 13 May '14

13 May '14

Hello We have a server on which we run multiple instances of dovecot. Each instance serving a domain and binding to a specific IP. The whole thing on three servers using the proxy ability and replication of dovecot. Now one domain has more users and we start seeing very strange behaviour. As soon as about 45 users have connected to the imap service ip of that instance, the server accepts the connections and then disconnects without showing the IMAP banner. This happens on all of our three servers and only for that specific domain (which has most connections). While this occurs, there is no problem with the other instances for other domains on the same physical machine. So the database backend etc. are not the issue. While this occurs, there is no hint of what goes wrong in the logfiles. I did even set mail_debug=yes but that still does not produce any output of why clients get disconnected prior to the server displaying the banner. Any hints on where I could start looking for the error, or get more debug output as with mail_debug=yes ? -Benoit-

1 0

[Dovecot] ManageSieve OOM with Huge Folders
by Fabian Peter 12 May '14

12 May '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi all, we're experiencing "Out of Memory" Problems caused by Sieve-Filters delivering mail to huge IMAP folders. From the logs, I'm beginning to think the problem is that the sieve/lmtp process loads the "dovecot.index.cache" file apparently twice into memory to do something with it. We've set our memory limit per process to 256MB - the cache file of the folders we're having issues with is exactly 128MB. Mails coming in that should be filtered to these folders are being denied, an error is being logged. Apparently, there would be two easy solutions to this: increasing memory OR decreasing folder size. Both are not possible. Increasing memory just means we will have this issue again in some weeks. Deacreasing mailbox size isn't possible since the folders contain archived system mails from our backend software which need to be kept for a while. Currently, we're creating a new folder each time we reach this limit, but that isn't exactly the best way to go for the future. In the end, the folders are for archiving purposes and therefore should not be managed at all since all this is automated. I hope there's some obvious switch I've just not seen in the config. I actually can't imagine that this is default or wanted behaviour since there must be people with large inboxes and sieve filters that either also have this problem or don't have it since they fixed it. The problem clearly is reproducable. Each time the cache file hits 128MB - which happens at around 200k mails in the folder - (with our current setting of 256MB memory limit per process), things go down and our mailqueues reach 10k undelivered mails within a few hours. I hope you can help me Sincerly Fabian OS - -- Linux 2.6.32-16-pve #1 SMP Mon Oct 22 08:38:13 CEST 2012 i686 GNU/Linux (The IMAP server is running within a debian squeeze openVZ container on the Proxmox Virtual Environment Kernel; filesystem for the containers is ext3) VERSION - ------- 2.1.14 (bf80034a547d) CONFIG - ------ # 2.1.14 (bf80034a547d): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-16-pve i686 Debian 6.0.7 simfs auth_cache_size = 10 M mail_debug = yes mail_location = maildir:/var/mail/%u:LAYOUT=fs mail_max_userip_connections = 10000 maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Archive { auto = no special_use = \Archive } mailbox Archives { auto = subscribe special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = /var/mail/%u/.dovecot.sieve sieve_dir = /var/mail/%u/sieve } protocols = imap lmtp sieve service imap-login { client_limit = 5 process_limit = 0 process_min_avail = 4 service_count = 0 } service imap { client_limit = 10 process_limit = 1024 process_min_avail = 4 } service lmtp { client_limit = 20 inet_listener lmtp { address = *, [::] port = 24 } process_limit = 0 process_min_avail = 10 vsz_limit = 256 M } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { client_limit = 10 process_limit = 0 process_min_avail = 4 } ssl = required ssl_ca = </etc/ssl/certs/wildcard.hexonet.net.ca ssl_cert = </etc/ssl/certs/wildcard.hexonet.net.crt ssl_key = </etc/ssl/certs/wildcard.hexonet.net.key userdb { args = uid=vmail gid=vmail home=/var/mail/%u driver = static } protocol lmtp { mail_plugins = sieve } LOG - --- May 12 06:36:36: Debug: none: root=, index=, control=, inbox=, alt= May 12 06:36:36: Debug: auth input: USERNAME(a)hexonet.net uid=5000 gid=5000 home=/var/mail/USERNAME(a)hexonet.net May 12 06:36:36: Debug: Effective uid=5000, gid=5000, home=/var/mail/USERNAME(a)hexonet.net May 12 06:36:36: Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/var/mail/USERNAME@hexonet.net:LAYOUT=fs May 12 06:36:36: Debug: fs: root=/var/mail/USERNAME(a)hexonet.net, index=, control=, inbox=/var/mail/USERNAME(a)hexonet.net, alt= May 12 06:36:36: Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts. May 12 06:36:36: Debug: PJyQMOZrcFN7awAA9jyUPQ: sieve: using the following location for user's Sieve script: /var/mail/USERNAME(a)hexonet.net/.dovecot.sieve May 12 06:36:36: Debug: PJyQMOZrcFN7awAA9jyUPQ: sieve: opening script /var/mail/USERNAME(a)hexonet.net/.dovecot.sieve May 12 06:36:36: Debug: PJyQMOZrcFN7awAA9jyUPQ: sieve: script binary /var/mail/USERNAME(a)hexonet.net/.dovecot.svbin successfully loaded May 12 06:36:36: Debug: PJyQMOZrcFN7awAA9jyUPQ: sieve: binary save: not saving binary /var/mail/USERNAME(a)hexonet.net/.dovecot.svbin, because it is already stored May 12 06:36:36: Debug: PJyQMOZrcFN7awAA9jyUPQ: sieve: executing script from /var/mail/USERNAME(a)hexonet.net/.dovecot.svbin May 12 06:36:36: PJyQMOZrcFN7awAA9jyUPQ: sieve: msgid=unspecified: stored mail into mailbox 'IMAP-FOLDER' May 12 06:36:41 Debug: none: root=, index=, control=, inbox=, alt= May 12 06:36:41 Debug: auth input: USERNAME(a)hexonet.net uid=5000 gid=5000 home=/var/mail/USERNAME(a)hexonet.net May 12 06:36:41: Debug: Effective uid=5000, gid=5000, home=/var/mail/USERNAME(a)hexonet.net May 12 06:36:41: Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/var/mail/USERNAME@hexonet.net:LAYOUT=fs May 12 06:36:41: Debug: fs: root=/var/mail/USERNAME(a)hexonet.net, index=, control=, inbox=/var/mail/USERNAME(a)hexonet.net, alt= May 12 06:36:41: Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts. May 12 06:36:41: Debug: QJyQMOZrcFN7awAA9jyUPQ: sieve: using the following location for user's Sieve script: /var/mail/USERNAME(a)hexonet.net/.dovecot.sieve May 12 06:36:41: Debug: QJyQMOZrcFN7awAA9jyUPQ: sieve: opening script /var/mail/USERNAME(a)hexonet.net/.dovecot.sieve May 12 06:36:41: Debug: QJyQMOZrcFN7awAA9jyUPQ: sieve: script binary /var/mail/USERNAME(a)hexonet.net/.dovecot.svbin successfully loaded May 12 06:36:41: Debug: QJyQMOZrcFN7awAA9jyUPQ: sieve: binary save: not saving binary /var/mail/USERNAME(a)hexonet.net/.dovecot.svbin, because it is already stored May 12 06:36:41: Debug: QJyQMOZrcFN7awAA9jyUPQ: sieve: executing script from /var/mail/USERNAME(a)hexonet.net/.dovecot.svbin May 12 06:36:41: Fatal: pool_system_realloc(268435456): Out of memory May 12 06:36:41: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x45951) [0x219951] -> /usr/lib/dovecot/libdovecot.so.0(+0x459bf) [0x2199bf] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x1e8dc5] -> /usr/lib/dovecot/libdovecot.so.0(+0x5806e) [0x22c06e] -> /usr/lib/dovecot/libdovecot.so.0(+0x40ec8) [0x214ec8] -> /usr/lib/dovecot/libdovecot.so.0(buffer_get_space_unsafe+0x85) [0x2153b5] -> /usr/lib/dovecot/libdovecot.so.0(buffer_append_space_unsafe+0x2b) [0x2154cb] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_cache_map+0x219) [0x506cf9] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_cache_open_and_verify+0x75) [0x507245] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_cache_register_get_list+0x50) [0x5092a0] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_parse_header_init+0x15f) [0x4f6def] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_cache_parse_init+0x7c) [0x4f73fc] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_add+0x1d7) [0x4a0a87] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_begin+0x1d3) [0x4a0dd3] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_begin+0x58) [0x4d7988] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0xba) [0x4d163a] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_copy+0x4b) [0x49bacb] - -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x54) [0x4d78b4] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x35f5f) [0xc52f5f] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_result_execute+0x16b) [0xc49efb] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x69) [0xc5b139] - -> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x2892) [0x75e892] - -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x4f) [0x19641f] - -> dovecot/lmtp() [0x804c812] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x42) [0x227822] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd3) [0x2289e3] - -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x2277b0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x2d) [0x21050d] May 12 06:36:41: Fatal: master: service(lmtp): child 27515 returned error 83 (Out of memory (service lmtp { vsz_limit=256 MB }, you may need to increase it)) - -- Fabian Peter Systemadministrator HEXONET GmbH T: +49 6841 69 84 308 F: +49 6841 69 84 199 E: fpeter(a)hexonet.net W: http://www.hexonet.net HEXONET GmbH, Talstrasse 27, 66424 Homburg, Germany. CEO & General Manager: Jens Wagner, HRB 2839 (HOM), Amtsgericht Saarbrücken, VAT-ID: DE-138316882 HEXONET Services Inc., 104 - 7455 132nd Street, Surrey, B.C., V3W 1J8, Canada. CSO & General Manager: Robert Birkner This email and any files transmitted are confidential and intended only or the person(s) directly addressed. If you are not the intended recipient, any use, copying, transmission, distribution, or other forms of dissemination is strictly prohibited. If you have received this email in error, please notify the sender immediately and permanently delete this email with any files that may be attached. -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) Comment: GPGTools - https://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCgAGBQJTcHVYAAoJEPIZChNlZ+or5VQIAKegO6HwORlpEmP3YBkJnHXY sYsR1sP89Vm1Uzv4cVL30KF1P37NI9sZwHrO5H2aC5sqNSadaZ0lOUFncwlZwphh JsSS2Jye+56UW7XR92rUuwC8FPYRk6nOxTA0hHtAGW1IxCROz8P32V24cRlDE32k rcq2GT0V4laFlD5YNcqFWAClTqAholnHZCm/7VpBCevUTx0F+KVZxEth/xOf0ajY EmWoXat4Fihnly1YtdPqN5FqIyFdzBF8tyuWaUZN3BfO8QDso+tci/NRbe2yYkIK lhulr4PAvccsIqJ+u/9j+XN7SL9ZXDtU0w4xWQe+5SrGfFXJlXsuJoRgBdThn9I= =YP4k -----END PGP SIGNATURE-----

2 1

[Dovecot] v2.2.13.rc1 vs. Pigeonhole
by Tamsy 12 May '14

12 May '14

Server: Ubuntu 10.04 LTS 32bit There seems to be an incompatibility when compiling Pigeonhole against Dovecot v2.2.13.rc1. # make check with Pigeonhole throws up the following error message: <snip> Test case: ./tests/extensions/editheader/addheader.svtest: 1: Test 'Addheader - first' SUCCEEDED 2: Test 'Addheader - last' SUCCEEDED 3: Test 'Addheader - framed' SUCCEEDED 4: Test 'Addheader - folded' SUCCEEDED 5: Test 'Addheader - newlines' FAILED: wrong first header content in redirected mail 6: Test 'Addheader - implicit keep' SUCCEEDED 7: Test 'Addheader - UTF 8' SUCCEEDED 8: Test 'Addheader - devious characters' SUCCEEDED FAIL: 1 of 8 tests failed. I tried this with the latest Pigeonhole from Mercurial: - dovecot-2-2-pigeonhole-3b6917a4807c and also with the older: - dovecot-2-2-pigeonhole-e111a2393a92 Both versions are quitting during "make check" with the same error when compiling against dovecot-2.2.13.rc1. Compiling those versions against dovecot-2.2.12 is going without errors though. Cheers, Tamsy

3 4

[Dovecot] message-decoder bug for attachments with charset=binary attribute in content-type?
by Robert Wolf 12 May '14

12 May '14

Hello, I have configure dovecot with solr and I wanted to let solr index content of attachments. For testing I have used biabam command line tool to generate emails with attachments. I have found that dovecot with fts_decoder incorrectly decodes these attachments from biabam and therefore pdftotext has reported corrupted PDF. The problem is that biabam generates header with charset=binary and dovecot message decoder tries to process it as UTF8 or non-UTF8 data. ============================================================ --biabam.ZxWVLybiabam.ZxWVLy Content-Type: application/pdf; charset=binary Content-Disposition: attachment; filename="bacula-jobs.pdf" Content-Transfer-Encoding: base64 JVBERi0xLjQKJcfsj6IKNSAwIG9iago8PC9MZW5ndGggNiAwIFIvRmlsdGVy IC9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nF2PT0+EMBDF7/0U7yYYWdqFXdbe 1vgnMXpQezMeClSoQNltweh+egvLyczhN3kz703mCLpioFMtLDoSv2aoHKGo ..... ============================================================ This PDF begins orginal with ============================================================ 0000000: 2550 4446 2d31 2e34 0a25 c7ec 8fa2 0a35 %PDF-1.4.%.....5 0000010: 2030 206f 626a 0a3c 3c2f 4c65 6e67 7468 0 obj.<</Length 0000020: 2036 2030 2052 2f46 696c 7465 7220 2f46 6 0 R/Filter /F 0000030: 6c61 7465 4465 636f 6465 3e3e 0a73 7472 lateDecode>>.str ============================================================ But the dovecot pass following data to fts_decoder script: ============================================================ 0000000: 2550 4446 2d31 2e34 0a25 c3a4 c3bc c3b6 %PDF-1.4.%...... 0000010: c39f 0a32 2030 206f 626a 0a3c 3c2f 4c65 ...2 0 obj.<</Le 0000020: 6e67 7468 2033 2030 2052 2f46 696c 7465 ngth 3 0 R/Filte 0000030: 722f 466c 6174 6544 6563 6f64 653e 3e0a r/FlateDecode>>. ============================================================ As you can see binary data are mangled. Alpine and Thunderbird do not write charset=binary to content-type header and searching works perfect. I have searched in source code and I have found one place. If I replace the following code in file dovecot-2.1.7/src/lib-mail/message-decoder.c on line 241 with new one, the dovecots message decoder decodes message correctly and pdftotext can convert attached PDF. Original code: ============================================================ 241: ctx->binary_input = ctx->content_charset == NULL && 242: (ctx->flags & MESSAGE_DECODER_FLAG_RETURN_BINARY) != 0 && 243: (part->flags & (MESSAGE_PART_FLAG_TEXT | 244: MESSAGE_PART_FLAG_MESSAGE_RFC822)) == 0; ============================================================ My update: ============================================================ 241 ctx->binary_input = ((ctx->content_charset != NULL) && (strcasecmp(ctx->content_charset, "binary") == 0)) || (ctx->content_charset == NULL && 242 (ctx->flags & MESSAGE_DECODER_FLAG_RETURN_BINARY) != 0 && 243 (part->flags & (MESSAGE_PART_FLAG_TEXT | 244 MESSAGE_PART_FLAG_MESSAGE_RFC822)) == 0); ============================================================ This will set ctx->binary_input for the attachment with charset set to "binary". I don't know if this is correct update, but the searching works with this update for biabam binary attachments too. Could you please verify this problem and maybe update the code? Thank you very much. # dovecot --version 2.1.7 Config: plugin { fts = solr fts_solr = url=http://localhost:8080/solr/ fts_decoder = decode2text } service decode2text { executable = script /etc/dovecot/scripts/decode2text.sh user = dovecot unix_listener decode2text { mode = 0666 } } Regards, Robert Wolf.

1 0

[Dovecot] questions about process_limit
by Andreas Meyer 12 May '14

12 May '14

Hello all! # dovecot --version 2.1.17 Sometimes I have this in the logfile: May 11 16:55:52 master: Warning: service(imap-login): process_limit (100) reached, client connections are being dropped May 11 17:35:03 master: Warning: service(imap-login): process_limit (100) reached, client connections are being dropped May 11 17:36:27 master: Warning: service(imap-login): process_limit (100) reached, client connections are being dropped I then put this in the dovecot.conf service imap-login { service_count = 1 process_min_avail = 50 #process_limit = $default_process_limit process_limit = 10000 vsz_limit = 64M } and get this: May 11 18:19:36 master: Info: Dovecot v2.1.17 starting up (core dumps disabled) May 11 18:19:36 config: Warning: service auth { client_limit=1000 } is lower than required under max. load (10300) May 11 18:19:36 config: Warning: service anvil { client_limit=1000 } is lower than required under max. load (10203) Puting this in: service auth { client_limit = 10300 unix_listener auth-userdb { mode = 0666 user = vmail group = vmail } } service anvil { client_limit = 10300 } I get this: May 11 18:27:37 delta.bitcorner.eu dovecot[3989]: Warning: fd limit (ulimit -n) is lower than required under max. load (1024 < 10300), because of service auth { client_limit } I have also seen this today: May 11 13:14:05 log: Warning: Killed with signal 15 (by pid=1 uid=0 code=kill) May 11 13:14:05 auth: Error: read(anvil-auth-penalty) failed: EOF May 11 13:14:05 auth: Error: net_connect_unix(anvil-auth-penalty) failed: Permission denied What is the best way to avoid this process_limit errors and configure dovecot the best way? Greetings Andreas

3 2

[Dovecot] Segfault with passwd as a second userdb at auth_fields_rollback
by Jan Kończak 12 May '14

12 May '14

Hello, I've been trying to set up dovecot to support passwd-file + passwd/PAM auth. I made an unusual config (passwd-file as userdb failed to work for me), which uses static as userdb: =============== passdb { driver = passwd-file args = username_format=%u scheme=CRYPT /etc/dovecot/users } userdb { driver = static default_fields = uid=vmail gid=vmail home=/var/spool/mail/%d/%n } passdb { driver = pam } userdb { driver = passwd } =============== While passwd-file+static worked well, passwd/PAM caused trouble: $ doveadm user xxx doveadm(root): Error: userdb lookup(xxx): Disconnected unexpectedly field value If passwd-file had the user, check went fine; otherwise it segfaulted. The crash has been accompanied by a syslog message: kernel: auth[9788]: segfault at 18 ip 0000000000416391 sp 00007fffd11a3d90 error 4 in auth[400000+44000] Translating address lead me to: 0000000000416390 <auth_fields_rollback>: 416390: 53 push %rbx 416391: 48 83 7f 18 00 cmpq $0x0,0x18(%rdi) I was using packages from http://packages.atrpms.net/dist/el6/dovecot/ To be precise, dovecot-2.2.10-1_14.el6.x86_64.rpm I tampered with the src.rpm, upgraded spec&sources to match 2.2.12. It didn't help, so I tries to "workaround" this. The bug has gone when in src/auth/auth-fields.c:210 I added check if the field ptr is not null. As it's a userdb query, it seems that for some reason (static?) in src/auth/auth-request.c:1058 the function auth_fields_rollback is called with null request->userdb_reply Now it works fine: $ doveadm user xxx field value uid 1000 gid 100 home /home/xxx mail maildir:~/.maildir system_groups_user xxx $ doveadm user jkonczak@ast... field value uid 103 gid 103 home /var/spool/mail/ast.../jkonczak mail maildir:~/.maildir So now virtual users get vmail uid/gid, system users get their uid/gid, home is fine etc. Below I attach the patch + full config. Regards, Jan Kończak =============================================== $ cat SOURCES/dovecot-2.2.12-auth_fields_rollback-nullptr_check.patch --- src/auth/auth-fields.c.orig 2014-05-11 16:17:49.755258666 +0200 +++ src/auth/auth-fields.c 2014-05-11 16:18:24.704946586 +0200 @@ -207,6 +207,7 @@ void auth_fields_rollback(struct auth_fields *fields) { + if (fields==NULL) return; if (array_is_created(&fields->snapshot_fields)) { array_clear(&fields->fields); array_append_array(&fields->fields, &fields->snapshot_fields); =============================================== $ dovecot --version 2.2.12 $ doveconf # passdb/userdb/auth related passdb { args = username_format=%u scheme=CRYPT /etc/dovecot/users default_fields = deny = no driver = passwd-file master = no override_fields = pass = no result_failure = continue result_internalfail = continue result_success = return-ok skip = never } passdb { args = default_fields = deny = no driver = pam master = no override_fields = pass = no result_failure = continue result_internalfail = continue result_success = return-ok skip = never } service auth { chroot = client_limit = 0 drop_priv_before_exec = no executable = auth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-client { group = mode = 0600 user = $default_internal_user } unix_listener auth-login { group = mode = 0600 user = $default_internal_user } unix_listener auth-master { group = mode = 0600 user = } unix_listener auth-userdb { group = mode = 0666 user = $default_internal_user } unix_listener login/login { group = mode = 0666 user = } unix_listener token-login/tokenlogin { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } userdb { args = default_fields = uid=vmail gid=vmail home=/var/spool/mail/%d/%n driver = static override_fields = result_failure = continue result_internalfail = continue result_success = return-ok skip = never } userdb { args = default_fields = driver = passwd override_fields = result_failure = continue result_internalfail = continue result_success = return-ok skip = never } ===============================================

2 1

[Dovecot] "doveadm penalty -a" doesn't work?
by Peer Heinlein 12 May '14

12 May '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 We're running multiple instances on the same server. While doveadm penalty works: root@host:/etc/dovecot# doveadm penalty IP penalty last_penalty last_update 93.180.xxx.xxx 4 2014-05-10 12:21:08 12:21:08 80.153.xxx.xxx 0 2014-05-10 12:21:12 12:21:12 a seperate anvil-Socket doesn't work at all: root@host:/etc/dovecot# doveadm penalty -a /var/run/dovecot-example/anvil doveadm penalty [-a <anvil socket path>] [<ip/bits>] It just shows up the help message. (Dovecot 2.2.12) Peer - -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTbf41AAoJEAOLLpq5E82H/h4H/2VpFfrqG+dcW73dS8CAWXVM thWcj5/GKUDaE/MYKsATDmNenlPuKIHsC8+hM7pyB3kfA0d7EDP/aNcaOWdw16XE fq7PSoLWiPJcsTG6ArzyuJMYiCwb2TuFIimFjjlZWVqnB+iAVv0y7ESJ9De/dmzM eE3ct/JmuY+0MrRAKSNdDX/Dbaakh03/JQ3/GvAwYeTFgviCfa3D5hsYCfhDZbPC 2vE3A9D46SqhmrediuWdtnj6rAhYwfjS+77jXaK6NyYru0WtYf9ozgtt5pbb89Hd EKzdNyKdKsZEnQ1ubv7cj5671OwL1ikp/d8tJr3/l4tgRHWf2+BmuGJIoo8AFwE= =2gZD -----END PGP SIGNATURE-----

2 1