- dovecot - dovecot.org

Bug in dovecot 2.3 virtual plugin
by Jakobus Schürz 07 Feb '18

07 Feb '18

Hi there! I compiled dovecot 2.3 from git. Because there is already a bug in virtual-plugin, and i hoped, it get fixed... but it doesn't. So this is the error-message from the log Jän 03 16:27:08 aldebaran dovecot[26460]: indexer-worker(jakob)<26476><qQ6g1+BhIJvAqAAO:sjYhMTH2TFpsZwAAk1Mx3g>: Panic: file unichar.c: line 160 (uni_ucs4_to_utf8_c): assertion failed: (uni_is_valid_ucs4(chr)) Jän 03 16:27:08 aldebaran dovecot[26460]: indexer-worker(jakob)<26476><qQ6g1+BhIJvAqAAO:sjYhMTH2TFpsZwAAk1Mx3g>: Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0xc6021) [0x7f8299f7a021] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xc60ed) [0x7f8299f7a0ed] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8299eec481] -> /usr/local/lib/dovecot/libdovecot.so.0(uni_ucs4_to_utf8_c+0xa0) [0x7f8299fb1500] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xa75e0) [0x7f8299f5b5e0] -> /usr/local/lib/dovecot/libdovecot.so.0(mail_html2text_more+0xc5) [0x7f8299f5b775] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xcfcc) [0x7f82990aefcc] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(fts_parser_more+0x27) [0x7f82990aeca7] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x5e9) [0x7f82990acc39] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x1122d) [0x7f82990b322d] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x916a) [0x7f82958e316a] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x10f5d) [0x7f82990b2f5d] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x916a) [0x7f82958e316a] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x10f5d) [0x7f82990b2f5d] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x2e) [0x7f829a2641be] -> dovecot/indexer-worker [jakob Synoptic/AKTUELL](+0x2533) [0x562227882533] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7f8299f91bf9] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109) [0x7f8299f93499] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x52) [0x7f8299f91d02] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f8299f91f18] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f8299f0f1e3] -> dovecot/indexer-worker [jakob Synoptic/AKTUELL](main+0xe7) [0x562227881f47] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f8299b352b1] -> dovecot/indexer-worker [jakob Synoptic/AKTUELL](_start+0x2a) [0x56222788201a] Jän 03 16:27:08 aldebaran dovecot[26460]: indexer: Error: Indexer worker disconnected, discarding 1 requests for jakob Jän 03 16:27:08 aldebaran dovecot[26460]: imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Error: indexer failed to index mailbox Synoptic/AKTUELL Jän 03 16:27:08 aldebaran dovecot[26460]: indexer-worker(jakob)<26476><qQ6g1+BhIJvAqAAO:sjYhMTH2TFpsZwAAk1Mx3g>: Fatal: master: service(indexer-worker): child 26476 killed with signal 6 (core dumps disabled) Jän 03 16:27:09 aldebaran dovecot[26460]: indexer-worker(jakob)<26484><qQ6g1+BhIJvAqAAO:J6mUIEz2TFp0ZwAAk1Mx3g>: Error: lucene index /var/lib/dovecot/db/indexes/Maildir/jakob/lucene-indexes: IndexWriter() failed (#1): Lock obtain timed out Jän 03 16:27:10 aldebaran dovecot[26460]: indexer-worker(jakob)<26484><qQ6g1+BhIJvAqAAO:J6mUIEz2TFp0ZwAAk1Mx3g>: Error: Mailbox Synoptic/AKTUELL: Transaction commit failed: BUG: Unknown internal error (attempted to index 1488 messages (UIDs 15214..16775)) Jän 03 16:27:10 aldebaran dovecot[26460]: imap(jakob)<26480><lP5r2OBhKJvAqAAO>: Error: indexer failed to index mailbox Synoptic/AKTUELL Jän 03 16:27:11 aldebaran dovecot[26460]: imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Panic: file mail-index.c: line 793 (mail_index_close): assertion failed: (index->open_count > 0) Jän 03 16:27:11 aldebaran dovecot[26460]: imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0xc6021) [0x7fb0fbd3a021] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xc60ed) [0x7fb0fbd3a0ed] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fb0fbcac481] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xf39a8) [0x7fb0fc0d99a8] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xd92d1) [0x7fb0fc0bf2d1] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xd9363) [0x7fb0fc0bf363] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_storage_mailbox_free+0x9) [0x7fb0fc0b0f39] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xd05a4) [0x7fb0fc0b65a4] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_free+0x19) [0x7fb0fc030c99] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xabdc) [0x7fb0f729bbdc] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xacb9) [0x7fb0f729bcb9] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a) [0x7fb0fc030c0a] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xac60) [0x7fb0f729bc60] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xacb9) [0x7fb0f729bcb9] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a) [0x7fb0fc030c0a] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_free+0x13) [0x7fb0fc030c93] -> dovecot/imap [jakob 192.168.0.14 LOGOUT](imap_client_close_mailbox+0x5b) [0x56514d31305b] -> dovecot/imap [jakob 192.168.0.14 LOGOUT](cmd_logout+0x35) [0x56514d309485] -> dovecot/imap [jakob 192.168.0.14 LOGOUT](command_exec+0x64) [0x56514d312ca4] -> dovecot/imap [jakob 192.168.0.14 LOGOUT](+0x1b0b2) [0x56514d3110b2] -> dovecot/imap [jakob 192.168.0.14 LOGOUT](+0x1b13c) [0x56514d31113c] -> dovecot/imap [jakob 192.168.0.14 LOGOUT](client_handle_input+0x1a5) [0x56514d311565] -> dovecot/imap [jakob 192.168.0.14 LOGOUT](client_input+0x82) [0x56514d311be2] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7fb0fbd51bf9] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109) [0x7fb0fbd53499] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x52) [0x7fb0fbd51d02] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fb0fbd51f18] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fb0fbccf1e3] Jän 03 16:27:11 aldebaran dovecot[26460]: imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Fatal: master: service(imap): child 26472 killed with signal 6 (core dumps disabled) I get this errors when i open or search in a virtual folder. i have enabled fts_lucene. My config is: # dovecot -n # 2.3.devel (393bfbea8): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.0.rc1 (d68c23a1) # OS: Linux 4.13.0-trunk-amd64 x86_64 Debian 9.3 btrfs auth_mechanisms = plain login cram-md5 digest-md5 auth_socket_path = /usr/local/var/run/dovecot/auth-userdb first_valid_uid = 1000 imap_capability = +XDOVECOT imap_client_workarounds = tb-extra-mailbox-sep lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = *, ::1 login_trusted_networks = 127.0.0.1/8 192.168.0.0/24 192.168.1.0/24 172.17.0.0/24 172.18.0.0/24 mail_gid = vmail mail_home = /var/mail/%u mail_location = maildir:/var/mail/%u/Maildir:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/Maildir/%u:INDEXPVT=/var/lib/dovecot/db/indexes/Maildir/%u:CONTROL=/var/lib/dovecot/db/control/Maildir/%u mail_plugins = zlib quota acl listescape mail_log notify virtual fts fts_lucene mail_privileged_group = vmail mail_server_admin = mailto:jakob@xundeenergie.at mail_shared_explicit_inbox = yes mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { list = children location = maildir:/var/mail/public/:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/public/%u:INDEXPVT=/var/lib/dovecot/db/indexes/public/%u prefix = Roseggergasse/ separator = / subscriptions = no type = public } namespace Geteilt { hidden = no inbox = no list = children location = maildir:/var/mail/%%u/Maildir:LAYOUT=fs:INDEXPVT=/var/lib/dovecot/db/indexes/shared/%u/%%u:INDEX=/var/lib/dovecot/db/indexes/shared/%u/%%u prefix = Geteilt/%%n/ separator = / subscriptions = no type = shared } namespace Mailarchiv { hidden = no inbox = no list = children location = maildir:/var/mail/mailarchiv/%u/:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/mailarchiv/%u:INDEXPVT=/var/lib/dovecot/db/indexes/mailarchiv/%u mailbox incoming { auto = create } mailbox outgoing { auto = create } prefix = Mailarchiv/ separator = / subscriptions = no type = private } namespace Real { hidden = yes list = no location = virtual:/var/mail/real:INDEX=/var/lib/dovecot/db/indexes/real/%u prefix = Real/ separator = / subscriptions = no } namespace Synoptic { hidden = no list = children location = virtual:/var/mail/virtual:INDEX=/var/lib/dovecot/db/indexes/virtual/%u mailbox INBOX/Archives { auto = no special_use = \Archive } mailbox INBOX/Drafts { auto = no special_use = \Drafts } mailbox INBOX/Entwürfe { auto = no special_use = \Drafts } mailbox INBOX/Junk { auto = no special_use = \Junk } mailbox INBOX/Sent { auto = no special_use = \Sent } mailbox INBOX/Spam { auto = no special_use = \Junk } prefix = Synoptic/ separator = / subscriptions = no } namespace inbox { hidden = no inbox = yes location = mailbox Archiv { auto = no special_use = \Archive } mailbox Archive { auto = no special_use = \Archive } mailbox Archives { auto = no special_use = \Archive } mailbox "Deleted Messages" { auto = no autoexpunge = 30 days special_use = \Trash } mailbox Drafts { auto = no special_use = \Drafts } mailbox Entwürfe { auto = no special_use = \Drafts } mailbox "Gelöschte Elemente" { auto = no autoexpunge = 30 days special_use = \Trash } mailbox "Gelöschte Objekte" { auto = no autoexpunge = 30 days special_use = \Trash } mailbox Gesendet { auto = no special_use = \Sent } mailbox "Gesendete Elemente" { auto = no special_use = \Sent } mailbox "Gesendete Objekte" { auto = no special_use = \Sent } mailbox Important { auto = no } mailbox Junk { auto = subscribe autoexpunge = 30 days special_use = \Junk } mailbox Mistkübel { auto = no autoexpunge = 30 days special_use = \Trash } mailbox Papierkorb { auto = no autoexpunge = 30 days special_use = \Trash } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no autoexpunge = 30 days special_use = \Junk } mailbox Synoptic/Alle { auto = no comment = All my messages special_use = \All } mailbox Trash { auto = no autoexpunge = 30 days special_use = \Trash } mailbox Wichtig { auto = create } prefix = INBOX/ separator = / subscriptions = no type = private } namespace subscriptions { hidden = yes list = no location = prefix = subscriptions = yes } passdb { args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/users driver = passwd-file } plugin { acl = vfile:/etc/dovecot/dovecot-acl:cache_secs=300 acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db fts = lucene fts_autoindex = yes fts_autoindex_max_recent_msgs = 80 fts_index_timeout = 90s fts_lucene = whitespace_chars=@. normalize no_snowball mail_home = /var/mail/%u setting_name = sieve, managedsieve sieve = file:/var/mail/%u/sieve/;active=/var/mail/%u/sieve/%u.sieve } postmaster_address = postmaster@localhost protocols = imap pop3 lmtp imap lmtp sieve pop3 sieve service anvil { unix_listener anvil-auth-penalty { mode = 00 } } service auth { unix_listener auth-client { group = vmail mode = 0666 user = Debian-exim } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 1 service_count = 0 } service imap { executable = imap postlogin process_limit = 1024 vsz_limit = 400 M } service lmtp { executable = lmtp -L user = vmail vsz_limit = 400 M } service postlogin { executable = script-login -d rawlog } ssl = required ssl_cert = </usr/local/etc/dovecot/dovecot.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = username_format=%u /usr/local/etc/dovecot/users default_fields = home=/var/mail/%u driver = passwd-file } verbose_proctitle = yes protocol lmtp { auth_username_format = %n mail_plugins = zlib quota acl listescape mail_log notify virtual fts fts_lucene quota sieve acl } protocol lda { mail_plugins = zlib quota acl listescape mail_log notify virtual fts fts_lucene quota sieve acl } protocol imap { mail_max_userip_connections = 10 mail_plugins = zlib quota acl listescape mail_log notify virtual fts fts_lucene imap_quota imap_acl } The virtual Folder-definitions are: # find /var/mail/virtual/ /var/mail/real/ /var/mail/virtual/ /var/mail/virtual/GINBOX /var/mail/virtual/GINBOX/dovecot-virtual /var/mail/virtual/Neue /var/mail/virtual/Neue/dovecot-virtual /var/mail/virtual/Markierte /var/mail/virtual/Markierte/dovecot-virtual /var/mail/virtual/Ungelesene /var/mail/virtual/Ungelesene/dovecot-virtual /var/mail/virtual/Sent /var/mail/virtual/Sent/dovecot-virtual /var/mail/virtual/Drafts /var/mail/virtual/Drafts/dovecot-virtual /var/mail/virtual/Trash /var/mail/virtual/Trash/dovecot-virtual /var/mail/virtual/AKTUELL /var/mail/virtual/AKTUELL/dovecot-virtual /var/mail/virtual/Archives /var/mail/virtual/Archives/dovecot-virtual /var/mail/virtual/dovecot-acl-list /var/mail/virtual/WICHTIG /var/mail/virtual/WICHTIG/dovecot-virtual /var/mail/real/ /var/mail/real/Alle /var/mail/real/Alle/dovecot-virtual /var/mail/real/dovecot-acl-list /var/mail/real/einjahr /var/mail/real/einjahr/dovecot-virtual /var/mail/real/zweijahr /var/mail/real/zweijahr/dovecot-virtual /var/mail/real/dreijahr /var/mail/real/dreijahr/dovecot-virtual # cat /var/mail/virtual/AKTUELL/dovecot-virtual Real/Alle inthread refs younger 2592000 # cat /var/mail/real/Alle/dovecot-virtual !INBOX INBOX/* -INBOX/Trash -INBOX/Trash/* -INBOX/Archives -INBOX/Archives/* -INBOX/Drafts -INBOX/Drafts/* -INBOX/Junk -INBOX/Junk/* -INBOX/SYSTEM -INBOX/SYSTEM/* all The others are similar. Greez Jakob

5 12

Segmentation fault in "make check" test_istream_multiplex for 2.3.0 on Solaris Sparc
by Rainer Jung 07 Feb '18

07 Feb '18

Hi there, I get a crash when doing "make check" for version 2.3.0 on Solaris Sparc. The build was done with gcc 7.3.0. The crash happens in ioloop-poll.c line 181 when running test_istream_multiplex.c: #0 io_loop_handler_run_internal (ioloop=ioloop@entry=0x130dd0) at ioloop-poll.c:181 181 if (pollfd->revents != 0) { Due to the code 180 pollfd = &ctx->fds[ctx->fd_index[io->fd]]; 181 if (pollfd->revents != 0) { I can see in gdb, that io->fd is "-1", so using it as an array index doesn't make much sense: (gdb) print io->fd $2 = -1 and the resulting address of pollfd is not valid. The full io struct is: (gdb) print *io $3 = {io = {condition = IO_READ, source_filename = 0x8c680 "test-multiplex.c", source_linenum = 84, pending = false, callback = 0x33c14 <test_istream_multiplex_stream_read>, context = 0xc4b80 <test_channel+48>, ioloop = 0x130dd0, ctx = 0x0}, prev = 0xecfc0, next = 0xd3548, refcount = 1, fd = -1, istream = 0xd31e8} Full stack: [Current thread is 2 (Thread 1 (LWP 1))] (gdb) bt full #0 io_loop_handler_run_internal (ioloop=ioloop@entry=0x130dd0) at ioloop-poll.c:181 ctx = 0xd05b8 pollfd = 0x526cd8b0 tv = {tv_sec = 2147483, tv_usec = 0} io = 0xecd50 msecs = <optimized out> ret = 1 call = <optimized out> #1 0x000574a8 in io_loop_handler_run (ioloop=<optimized out>) at ioloop.c:666 No locals. #2 0x0005779c in io_loop_run (ioloop=0x130dd0) at ioloop.c:639 __func__ = "io_loop_run" #3 0x00033f7c in test_multiplex_stream () at test-multiplex.c:147 ioloop = 0x130dd0 fds = {6, 7} os = 0x0 is = 0x0 ichan1 = 0xd31e8 ochan1 = 0xd3470 io = 0xd34c8 ichan0 = 0x1b10e8 ochan0 = 0x1b1290 #4 test_multiplex () at test-multiplex.c:164 No locals. #5 0x0003e3d4 in test_run_named_funcs (tests=tests@entry=0x80444 <test_functions>, match=match@entry=0x7fd38 "") at test-common.c:255 _data_stack_cur_id = 2 i = <optimized out> #6 0x0003f1d8 in test_run_named_with_fatals (match=0x7fd38 "", tests=0x80444 <test_functions>, fatals=0x7fde0 <fatal_functions>) at test-common.c:337 No locals. #7 0x0001e024 in _start () I unfortunately don't know, why the fd field is "-1". Regards and thanks Rainer

2 1

New mail notification
by Matthew Allen 07 Feb '18

07 Feb '18

Hi, So I'm running dovecot 1.2.17 on Mac OSX using maildir in ~/Mail. After using getmail for a while to download my POP3 email and store it in the right place it started not working so I replaced it with a python script that I wrote. All was good for a while and then I stopped getting new mail notifications in my IMAP client. If I close the connection and restart the new mail shows up as !seen but no amount of re-selecting the folder will show the new mail while the connection is active. There is no untagged responses about new mail while the client is IDLEing. The client is InScribe (http://memecode.com/inscribe.php) but I doubt it's client specific. Seems to be server related. Now the python script in question just puts the .eml file in ~/Mail/tmp and then renames it to the ~/Mail/new folder. Does it need to do something else as well to make the notifications work? There aren't any error messages in the log. Just normal login history. If you can point me in the right direction that'd be appreciated. Regards Matthew Allen

1 0

Dovecot on GlusterFS via NFS is... strange.
by Frank Wall 07 Feb '18

07 Feb '18

Hi, I've discovered some interesting behaviour when running Dovecot on a GlusterFS volume (3.10) that is mounted via NFS 4.1 (ganesha NFS). First of all, using the usual settings for NFS storage seemed like an obvious choice for my setup (Dovecot 2.2.33.2, CentOS 7.4): mail_nfs_storage = yes mail_nfs_index = yes mail_fsync = always maildir_copy_with_hardlinks = yes mmap_disable = yes lock_method = dotlock (FWIW, I've been using these settings for many years on different NFS storages without any issues. Another instance of Dovecot 2.2.33 on FreeBSD 11.1 with a more traditional NFS storage is rock-solid.) SPOILER: These settings caused terrible issues. For example, when trying to clone a mailbox from an old server to this new Dovecot-on-GlusterFS setup (using imapsync), the following errors would show up frequently: Feb 6 16:44:53 dovecot: IMAP(user(a)example.com): Panic: file maildir-uidlist.c: line 1262 (maildir_uidlist_write_fd): assertion failed: (first_idx == 0) Feb 6 16:44:53 dovecot: IMAP(user(a)example.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x9f3de) [0x7f7b6b44f3de] -> /usr/lib64/dovecot/libdovecot.so.0(+0x9f4be) [0x7f7b6b44f4be] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f7b6b3e077c] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x79605) [0x7f7b6b75b605] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_uidlist_sync_finish+0x1ac) [0x7f7b6b75d61c] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_uidlist_sync_deinit+0x98) [0x7f7b6b75da48] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_transaction_save_commit_pre+0x41f) [0x7f7b6b75483f] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0xcccd0) [0x7f7b6b7aecd0] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit_full+0x9d) [0x7f7b6b7cb3cd] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_transaction_commit+0x107) [0x7f7b6b7af217] -> /usr/lib64/dovecot/lib10_quota_plugin.so(+0xe17c) [0x7f7b6a9ca17c] -> /usr/lib64/dovecot/lib01_acl_plugin.so(+0xda0a) [0x7f7b6abe0a0a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_transaction_commit_get_changes+0x51) [0x7f7b6b72c161] -> dovecot/imap [user(a)example.com 10.0.0.1 APPEND](+0xf255) [0x55ca8238f255] -> dovecot/imap [user(a)example.com 10.0.0.1 APPEND](command_exec+0x5c) [0x55ca8239cf0c] -> dovecot/imap [user(a)example.com 10.0.0.1 APPEND](+0xe771) [0x55ca8238e771] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f7b6b464cd2] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x10f) [0x7f7b6b4663bf] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7f7b6b464d6c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f7b6b464f28] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f7b6b3eafa3] -> dovecot/imap [user(a)example.com 10.0.0.1 APPEND](main+0x333) [0x55ca8238e2e3] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f7b6b00ec05] -> dovecot/imap [user(a)example.com 10.0.0.1 APPEND](+0xe476) [0x55ca8238e476] Feb 6 16:44:53 dovecot: IMAP(user(a)example.com): Fatal: master: service(imap): child 84782 killed with signal 6 (core dumps disabled) Feb 6 16:44:56 dovecot: IMAP(user(a)example.com): Error: readdir(/data/mail/abc/default/.Foldername 01_ Foo 1234/new) failed: Unknown error 523 Yes, the folder name is "special" in this example. (I'm not sure if this readdir() error is actually related to the other issue.) So, this caused the imap process to die repeatedly and the user got disconnected many many times. In this state the mail services were not really usable. Side note: The NFS share was mounted by using the most conservative moint options I could think of: vers=4.1,rsize=131072,wsize=131072,nosharecache,noac,nordirplus,proto=tcp And if you're wondering: Yes, I'm using Dovecot director for IMAP and LMTP (currently running 3 VMs for Dovecot director and 3 VMs for Dovecot IMAP). Then the fun began... I've decided to change the following settings: mail_nfs_storage = no mail_nfs_index = no Re-created the maildir and restarted the imapsync copy process. Out of sudden the errors are gone! Now Dovecot is performing pretty well on my GlusterFS volume. Does this make any sense to anyone? Ciao - Frank

1 0

"make check" test-hash-method bus error (Sparc alignment)
by Rainer Jung 06 Feb '18

06 Feb '18

Hi there, when doing "make check" for a 2.3.0 build on Solaris Sparc I get a bus error due to an alignment issue. It seems the error is only happening during testing, because in the official API it was fixed in https://github.com/dovecot/core/commit/d8361cc8576d9ede93a037f9b96f2a3f9b7e… But the tests use hand written test vectors, that are not correctly aligned: ... hash method md4 (test vectors) ....................................... : ok hash method md5 (test vectors) ....................................... : ok /bin/ksh[2]: 24287 Bus Error(coredump) make[4]: *** [Makefile:2655: check-local] Error 1 gdb output: Program terminated with signal SIGBUS, Bus error. #0 sha1_loop (ctxt=0xffbff724, input=0x83908, len=3) at sha1.c:224 224 ctxt->c.b64[0] += copysiz * 8; (gdb) print &ctxt->c.b64[0] $1 = (uint64_t *) 0xffbff73c So the address of the 8 byte type is not divisible by 8 (Sparc alignment requirement). Full stack: (gdb) bt full #0 sha1_loop (ctxt=0xffbff724, input=0x83908, len=3) at sha1.c:224 input_c = 0x83908 "abc" gaplen = <optimized out> gapstart = <optimized out> off = 0 copysiz = 3 #1 0x00024eb0 in test_hash_methods_fips () at test-hash-method.c:338 n = 0 method = 0xc4e68 <hash_method_sha1> context = 0xffbff724 "gE#\001▒ͫ\211\230▒▒▒\020\062Tv▒▒▒▒" result = "▒9▒▒^kK\r2U▒▒\225`\030\220▒▒\a\t" i = 7 last_method = 0x839d0 "sha1" test_vectors = {{method = 0x838e0 "md4", input = 0x838e8, ilen = 0, rounds = 1, output = 0x838f0, olen = 16}, {method = 0x838e0 "md4", input = 0x83908, ilen = 3, rounds = 1, output = 0x83910, olen = 16}, {method = 0x838e0 "md4", input = 0x83928, ilen = 62, rounds = 1, output = 0x83968, olen = 16}, {method = 0x83980 "md5", input = 0x838e8, ilen = 0, rounds = 1, output = 0x83988, olen = 16}, {method = 0x83980 "md5", input = 0x83908, ilen = 3, rounds = 1, output = 0x839a0, olen = 16}, { method = 0x83980 "md5", input = 0x83928, ilen = 62, rounds = 1, output = 0x839b8, olen = 16}, {method = 0x839d0 "sha1", input = 0x838e8, ilen = 0, rounds = 1, output = 0x839d8, olen = 20}, {method = 0x839d0 "sha1", input = 0x83908, ilen = 3, rounds = 1, output = 0x839f0, olen = 20}, {method = 0x839d0 "sha1", input = 0x83a08, ilen = 56, rounds = 1, output = 0x83a48, olen = 20}, {method = 0x83a60 "sha3-256", input = 0x838e8, ilen = 0, rounds = 1, output = 0x83a70, olen = 32}, {method = 0x83a60 "sha3-256", input = 0x83a98, ilen = 135, rounds = 1, output = 0x83b20, olen = 32}, {method = 0x83a60 "sha3-256", input = 0x83b48, ilen = 200, rounds = 1, output = 0x83c18, olen = 32}, {method = 0x83a60 "sha3-256", input = 0x83c40, ilen = 1, rounds = 200, output = 0x83c18, olen = 32}, { method = 0x83c48 "sha3-512", input = 0x838e8, ilen = 0, rounds = 1, output = 0x83c58, olen = 64}, {method = 0x83c48 "sha3-512", input = 0x83a98, ilen = 135, rounds = 1, output = 0x83ca0, olen = 64}, {method = 0x83c48 "sha3-512", input = 0x83b48, ilen = 200, rounds = 1, output = 0x83ce8, olen = 64}, { method = 0x83c48 "sha3-512", input = 0x83c40, ilen = 1, rounds = 200, output = 0x83ce8, olen = 64}} #2 0x0003e3e0 in test_run_named_funcs (tests=tests@entry=0x80434 <test_functions>, match=match@entry=0x7fd40 "") at test-common.c:255 _data_stack_cur_id = 2 i = <optimized out> #3 0x0003f1e4 in test_run_named_with_fatals (match=0x7fd40 "", tests=0x80434 <test_functions>, fatals=0x7fde8 <fatal_functions>) at test-common.c:337 No locals. #4 0x0001e024 in _start () Unaligned data comes from file src/lib/test-hash-method.c: ... 53 static void test_hash_methods_fips() { 54 const char *last_method = NULL; 55 56 struct { 57 const char *method; 58 const void *input; 59 size_t ilen; 60 size_t rounds; 61 const void *output; 62 size_t olen; 63 } test_vectors[] = 64 { ... 115 { "sha1", 116 "", 117 0, 118 1, 119 "\xda\x39\xa3\xee\x5e\x6b\x4b\x0d" 120 "\x32\x55\xbf\xef\x95\x60\x18\x90" 121 "\xaf\xd8\x07\x09", 122 160 / 8 123 }, ... etc. The input data needs to be 8 byte aligned but is just a void *. Thanks and regards, Rainer

1 0

Dovecot and /dev/rand
by Michael A. Peters 06 Feb '18

06 Feb '18

There's a (2014 I think) patch in dovecot rpm for Red Hat / Fedora ecosystem that was last update with 2.2.9 (patch last updated for that release) I want to know if the problem it solves still exists in 2.3.x branch. The patch is called "dovecot-2.2.9-nodevrand.patch" and is attached. It allegedly addresses an issue with installation in chroot that does not have /dev/random Red Hat bugzilla 1026790 I am apparently not authorized to see that bug so I don't know much about it, but with many code changes in 2.3.0 from 2.2.x I am wondering if the alleged issue was solved a different way or if the patch should be ported to 2.3.0 when packaged for people who might trigger it. Thanks.

1 0

Does Dovecot LDAP auth support LDAP referral
by Xuan Jia 06 Feb '18

06 Feb '18

We using Dovecot with LDAP. From the beginning, we using GC LDAP query with port 3268 for email accounts. For example, user1(a)our-organization.org (in the USA) with "base = dc=our-organization, dc=org" works fine. But refer to this document: https://wiki2.dovecot.org/AuthDatabase/LDAP When we change the LDAP from 3268 to 389 and with TLS, the base should be changed like this: "base = ou=usa, dc=our-organization, dc=org" But if the user (user2) located in United Kingdom (ou=gbr), the user can not login. When we debug with ldapsearch: ldapsearch -ZZ -v -h dc.our-organization.org -p 389 -D 'cn=auth_user,ou=usa,dc=our-organization,dc=org' -W -b 'dc=our-organization, dc=org' '(userPrincipalName=user2(a)our-organization.org)' It can return user2 information with some "numReferences". But in Dovecot, if "base = dc=our-organization, dc=org" it only reported auth error with timeout. Here are the config: # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 3.16.0-4-amd64 x86_64 Debian 9.3 auth_verbose = yes dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -l%{lock_timeout} -n%{namespace} imap_id_send = imapc_user = %u import_environment = TZ listen = * login_log_format_elements = user=u> method=%m rip=%r lip=%l mpid=%e %c mail_shared_explicit_inbox = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap-userdb.conf.ext driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap sieve" service managesieve-login { inet_listener sieve { port = 4190 } service_count = 0 } ssl_ca = /etc/ssl/certs/ca.pem ssl_cert = </etc/ssl/gameloft/our-organization.org.crt ssl_key = # hidden, use -P to show it ssl_parameters_regenerate = 1 weeks userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } So my question is: does Dovecot LDAP auth support LDAP referral? If Dovecot relies on OpenLDAP, it should be support. If not, why and what is the walkthrough? Xuan Jia

2 2

Dovecot 2.3.0, Panic: file mailbox-attribute.c: line 362 (mailbox_attribute_get_stream): assertion failed: (value_r->value != NULL || value_r->value_stream != NULL)
by Marco Giunta 05 Feb '18

05 Feb '18

Hi at all, I have a RHEL7 server with Dovecot 2.3.0 (new installation). I've a problem when trying to dsync from a Dovecot 2.2.24 server. If I try to sync any user with a folder with ACL, dsycn crash with panic: Source server: dsync-local(USERNAME): Debug: sieve: file storage: sync: Synchronization active dovecot: dsync-local(USERNAME): Debug: acl vfile: reading file /var/spool/mail/U/USERNAME/dovecot-acl dsync-local(USERNAME): Error: read(DEST_SERVER.example.com) failed: EOF (last sent=mail_change (EOL), last recv=mailbox) Destination server: Feb 2 14:15:23 DEST_SERVER dovecot: dsync-server(USERNAME): Panic: file mailbox-attribute.c: line 362 (mailbox_attribute_get_stream): assertion failed: (value_r->value != NULL || value_r->value_stream != NULL) Feb 2 14:15:23 DEST_SERVER dovecot: dsync-server(USERNAME): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xc8cc4) [0x7fa861bc0cc4] -> /usr/lib64/dovecot/libdovecot.so.0(+0xc8d7e) [0x7fa861bc0d7e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7fa861b34190] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x55cbc) [0x7fa861ec1cbc] -> dovecot/doveadm-server [10.0.11.137 USERNAME INBOX send:mail_requests recv:attributes](dsync_mailbox_import_attribute+0x4d) [0x55b9d4ce215d] -> dovecot/doveadm-server [10.0.11.137 USERNAME INBOX send:mail_requests recv:attributes](dsync_brain_sync_mails+0x2ef) [0x55b9d4cddbdf] -> dovecot/doveadm-server [10.0.11.137 USERNAME INBOX send:mail_requests recv:attributes](dsync_brain_run+0x2b0) [0x55b9d4cd93e0] -> dovecot/doveadm-server [10.0.11.137 USERNAME INBOX send:mail_requests recv:attributes](+0x43a10) [0x55b9d4cd9a10] -> dovecot/doveadm-server [10.0.11.137 USERNAME INBOX send:mail_requests recv:attributes](+0x5837f) [0x55b9d4cee37f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x65) [0x7fa861bd82b5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x10f) [0x7fa861bd9b5f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x52) [0x7fa861bd83b2] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fa861bd85d8] -> dovecot/doveadm-server [10.0.11.137 USERNAME INBOX send:mail_requests recv:attributes](+0x28369) [0x55b9d4cbe369] -> dovecot/doveadm-server [10.0.11.137 USERNAME INBOX send:mail_requests recv:attributes](+0x29c07) [0x55b9d4cbfc07] -> dovecot/doveadm-server [10.0.11.137 USERNAME INBOX send:mail_requests recv:attributes](+0x3f969) [0x55b9d4cd5969] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x65) [0x7fa861bd82b5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x10f) [0x7fa861bd9b5f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x52) [0x7fa861bd83b2] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fa861bd85d8] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fa861b56b23] -> dovecot/doveadm-server [10.0.11.137 USERNAME INBOX send:mail_requests recv:attributes](main+0x1b6) [0x55b9d4cb0536] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fa861756c05] -> dovecot/doveadm-server [10.0.11.137 USERNAME INBOX send:mail_requests recv:attributes](+0x1a5f5) [0x55b9d4cb05f5] Feb 2 14:15:23 DEST_SERVER dovecot: dsync-server(USERNAME): Fatal: master: service(doveadm): child 2149 killed with signal 6 (core dumped) Coredump and configuration attached. On source server I run this command: # doveadm -D backup -f -u USERNAME -x 'Archives*' tcp:DEST_SERVER.example.com but same panic if try to sync from destination server: # doveadm -D backup -fR -u USERNAME -x 'Archives*' tcp:SOURCE_SERVER.example.com Same panic also syncing any user with acl and using different acl_shared_dict (file or fs:posix) in dovecot configuration. Thanks, Marco -- Marco Giunta - ITCS SysAdmin Via Bonomea, 265 34136 - Trieste, Italy Tel: +39-040-3787-503 Fax: +39-040-3787-244

1 1

Re: How to grant user access to his .dovecot.sieve.log?
by Sergey Ivanov 04 Feb '18

04 Feb '18

Hi Gabriel, I'm trying to implement what you suggested using apache mod_userdir with === UserDir /srv/dovecot <DirectoryMatch "^/srv/dovecot/(?<WHICHUSER>[^/]+)"> Require user %{env:MATCH_WHICHUSER} DirectoryIndex .dovecot.sieve.log ... </DirectoryMatch> === I still need to find how to set up right permissions for these logs without too much brute force. It looks like this way I can get what I want, but I really hoped to find some better way: my Roundcube installation is on the load-balanced web farm and there is no and should not be a file access to these logs. I think such a feature as Sieve compiler must have some standard way to say what is wrong to the author of the Sieve script. -- Sergey. On Fri, Feb 2, 2018 at 2:29 PM, Gabriel Kaufmann <mailings(a)typoworx.com> wrote: > Alternatively use a web-interface with read-only access to the log? > > best regards > > Gabriel Kaufmann > >

1 0

Dual Dovecot setup
by dovecot.dirk＠o.banes.ch 03 Feb '18

03 Feb '18

Hi all, I´m using dovecot successfully since some years. My current setup is postfix and dovecot on one vserver. This works great so far. +--------------------+ eMail in | VSERVER1 | +---------->+ MX 1 (Master) | | POSTFIX MTA | | DOVECOT ALL Folders| +-----^--------------+ | |eMail Backup | | +------+---------+ | VSERVER2 | eMail in | MX 2 (Backup) | Backup | POSTFIX MTA | +---------> | +----------------+ However now I want to change this setup to have on box in my home which holds all eMails and Folders which are not in the Inbox. This is due to disk space and privacy concerns. Even if the home connection is very very stable for the case something fails while on vacation the content of the inbox of every user should always stay on the VSERVER1 as well - until it is moved to one subfolder. +---------------+ +-------------+ eMail in | VSERVER1 | | HOMEBOX | +---------->+ MX 1 (Master) | ?SYNC? | DOVECOT | | POSTFIX MTA +--------------> ALL FOLDERS | | DOVECOT INBOX | | | +-----^---------+ +-------------+ | |eMail Backup | | +------+---------+ | VSERVER2 | eMail in | MX 2 (Backup) | Backup | POSTFIX MTA | +---------> | +----------------+ I read into documentation and also did a fair share of googeling but I could only identify possible solutions which I´m not sure they will work. a) dsync - Can dsync only sync the Inbox ? b) fetchmail - Unsure how an eMail moved from Inbox (Homebox) to another folder would get removed from the inbox of the vserver1 Therefore I hope maybe someone of you has an idea how this can be archived. Thank you best regards Dirk

1 0