December 2018 - dovecot

2.3.3: Panic: file ostream-zlib.c: line 37 (o_stream_zlib_close): assertion failed
by Tom Sommer 13 Mar '19

13 Mar '19

I see this in my logs after 2.3.3: using zlib plugin, ofc. Oct 02 10:01:39 imap(user(a)example.com)<50643></2k4Sjp3vMqC496W>: Panic: file ostream-zlib.c: line 37 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0 || zstream->ostream.error_handling_disabled) Oct 02 10:01:39 imap(user(a)example.com)<50643></2k4Sjp3vMqC496W>: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xce56a) [0x7f442487556a] -> /usr/lib64/dovecot/libdovecot.so.0(+0xce5b1) [0x7f44248755b1] -> /usr/lib64/dovecot/libdovecot.so.0(+0x3d941) [0x7f44247e4941] -> /usr/lib64/dovecot/lib20_zlib_plugin.so(+0x5cdf) [0x7f44233c4cdf] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf4b46) [0x7f442489bb46] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_destroy+0x13) [0x7f442489be83] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x173) [0x7f4424b93973] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x36) [0x7f4424b68696] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](+0xede9) [0x55a9d7e86de9] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](command_exec+0x65) [0x55a9d7e956e5] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](+0xf9e6) [0x55a9d7e879e6] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x55) [0x7f442488c275] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbf) [0x7f442488e13f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x55) [0x7f442488c365] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f442488c588] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f4424808053] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](main+0x32d) [0x55a9d7ea20dd] -> /lib64/libc.so.6(__libc_start_main+0x100) [0x7f4424431d20] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](+0xe419) [0x55a9d7e86419] No clue how to reproduce -- Tom

3 2

imap-hibernate returned failure: Failed to parse client input: Invalid peer_dev_minor value
by Alex 18 Feb '19

18 Feb '19

Hello! I tried to use imap-hibernate. But errors in maillog: Oct 19 05:14:24 server dovecot: imap(email(a)example.com)<67125><Lu7ndYt4n8aASO/q>: Error: kevent(-1) for notify remove failed: Bad file descriptor Oct 19 05:14:24 server dovecot: imap(email(a)example.com)<67125><Lu7ndYt4n8aASO/q>: Error: close(-1) for notify remove failed: Bad file descriptor Oct 19 05:14:24 server dovecot: imap-hibernate: Error: Failed to parse client input: Invalid peer_dev_minor value: 18446744073709486335 Oct 19 05:14:24 server dovecot: imap(email(a)example.com)<67125><Lu7ndYt4n8aASO/q>: Error: /var/run/dovecot/imap-hibernate returned failure: Failed to parse client input: Invalid peer_dev_minor value: 18446744073709486335 Oct 19 05:14:24 server dovecot: imap: Error: dovecot-2.3.3, FreeBSD 10.4 dovenull 38990 0,0 0,0 43748 16308 - S 3:54 0:00,08 imap-login: (imap-login) dovenull 38998 0,0 0,1 47844 19536 - S 3:54 0:00,18 imap-login: [34.193.82.152 TLS proxy] (imap-login) dovenull 38999 0,0 0,1 47844 17180 - S 3:54 0:00,19 imap-login: (imap-login) dovenull 39000 0,0 0,0 35556 9584 - S 3:54 0:00,03 dovecot/imap-login dovenull 39001 0,0 0,0 43748 16320 - S 3:54 0:00,12 imap-login: (imap-login) dovenull 39002 0,0 0,0 43748 16320 - S 3:54 0:00,13 imap-login: (imap-login) dovenull 39003 0,0 0,0 43748 16328 - S 3:54 0:00,16 imap-login: (imap-login) dovenull 39004 0,0 0,1 47844 17948 - S 3:54 0:00,17 imap-login: [12.34.56.78 TLS proxy] (imap-login) vmail 39042 0,0 0,0 15936 3040 - I 3:54 0:00,01 imap-hibernate: [0 connections] (imap-hibernate) vmail 39076 0,0 0,0 78900 8984 - S 3:54 0:00,05 imap: [email(a)example.com 12.34.56.78 IDLE] (imap) vmail 44114 0,0 0,0 78900 8936 - I 4:06 0:00,05 imap: [email(a)example.com 34.193.82.152 IDLE] (imap) srw-rw---- 1 root dovecot 0 19 окт 03:54 imap-hibernate srw------- 1 root dovecot 0 19 окт 03:54 imap-master srw------- 1 root wheel 0 19 окт 03:54 imap-postlogin srw-rw-rw- 1 root wheel 0 19 окт 03:54 imap-urlauth srw------- 1 dovecot wheel 0 19 окт 03:54 imap-urlauth-worker imap_hibernate_timeout = 5 secs service imap-hibernate { extra_groups = $default_internal_group unix_listener imap-hibernate { group = dovecot } user = vmail } service imap { extra_groups = $default_internal_group unix_listener imap-master { group = $default_internal_group } } I tried to set user/group vmail, set mode = 0666, but the same errors. Can you help me with hibernate confiration ? Best regards, Alex

3 8

expunge not removing attachments?
by Ralf Hildebrandt 12 Feb '19

12 Feb '19

I have a large mail backup folder backup(a)backup.invalid; I'm cleaning up daily like this: infimum=`date -d "-4 day" +"%Y-%m-%d"` doveadm expunge -u backup(a)backup.invalid mailbox INBOX SAVEDBEFORE $infimum doveadm purge -u backup(a)backup.invalid yet I see this: # find attachments/ -type f -ctime +5 | wc -l 7522 # find attachments/ -type f | wc -l 127579 # find attachments/ -type f -mtime +5 | wc -l 14361 # find attachments/ -type f | wc -l 127793 About 5.9% of the files in attachments and below are older than 5 days. Why? Is that normal? using dovecot 2:2.3.1-1 from the official repos. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | https://www.charite.de

2 3

assertion failed: (srcleft <= CHARSET_MAX_PENDING_BUF_SIZE)
by Giacomo 11 Feb '19

11 Feb '19

I'm running the following: # 2.3.4 (0ecbaf23d): /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 11.2-RELEASE-p4 amd64 # FS: ZFS auth_mechanisms = plain login auth_username_format = %Ln listen = * mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { driver = pam } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } user = dovecot } service imap-login { inet_listener imap { port = 0 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service stats { unix_listener stats-reader { group = mode = 0600 user = } unix_listener stats-writer { group = $default_internal_group mode = 0666 user = } user = $default_internal_user } ssl_cert = </root/cert.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } dmesg shows: pid 74441 (imap), uid 1002: exited on signal 6 pid 74784 (imap), uid 1002: exited on signal 6 pid 21471 (imap), uid 1002: exited on signal 6 pid 21470 (imap), uid 1002: exited on signal 6 pid 21464 (imap), uid 1002: exited on signal 6 pid 21731 (imap), uid 1002: exited on signal 6 Sometimes the dovecot process serving an user crashes. He runs mostly airmail on macOS and samsung mail on Android. Happens only to that particular user. The log shows the following: (maillog) Dec 16 14:52:35 mail dovecot: imap-login: Login: user=<USER>, method=PLAIN, rip=79.12.201.157, lip=192.168.1.28, mpid=74779, TLS, session=<wMQI+iN9VtRPDMmd> Dec 16 14:52:39 mail dovecot: imap(USER)<74441><nLa+7SN9+fhPDMmd>: Panic: file charset-iconv.c: line 83 (charset_to_utf8_try): assertion failed: (srcleft <= CHARSET_MAX_PENDING_BUF_SIZE) Dec 16 14:52:39 mail dovecot: imap(USER)<74441><nLa+7SN9+fhPDMmd>: Fatal: master: service(imap): child 74441 killed with signal 6 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set service imap { drop_priv_before_exec=yes }) I'm here if more info is needed. Thanks for taking a look.

4 7

SELinux policy to allow Dovecot to connect to Mysql
by Robert Moskowitz 05 Feb '19

05 Feb '19

As I have noted in previous messages, I been getting the following on my new mailserver: Apr 7 10:17:27 z9m9z dovecot: dict: Error: mysql(localhost): Connect failed to database (postfix): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (13) - waiting for 25 seconds before retry They go away when I setenforce 0. It is not a timing issue as I earlier thought. So I googled dovecot mysql selinux and the only worthwhile hit was: http://zszsit.blogspot.com/2012/12/dovecot-mysql-selinux-issue-on-centos6.h… that provides a /etc/selinux/dovecot2mysql.te and other selinux stuff. Is there a simpler way like a setsbool option? With all the howtos on dovecot with mysql, it is interesting that none of them seem to have this problem. Maybe because they connect to mysql through TCP port 3306 which has ITS set of problems (like MariaDB defaults to not listening on TCP). thanks!

1 3

auth: Error - Request timed out
by Hajo Locke 01 Feb '19

01 Feb '19

Hello List, i use dovecot 2.2.22 and have the same problem described here: https://dovecot.org/pipermail/dovecot/2017-November/110020.html I can confirm that sometimes there is a problem with connection to mysql-db, but sometimes not. Reasons for failing are still under investigation by my mates. My current main problem is, that this fail seems to be a one way ticket for dovecot. Even if mysql is verifyable working again and waiting for connection dovecot stucks furthermore with errors like this: May 29 07:00:49 hostname dovecot: auth: Error: plain(mail(a)example.com,xxx.xxx.xx.xxx,<HGI4FVFt2Oy80j3P>): Request 999.7 timed out after 150 secs, state=1 When restarting dovecot all is immediately working again. Is there a way to tell dovecot to restart auth services or reinitialize mysql-connection after these hard fails? I could insert "idle_kill = 1 mins" into service auth and service auth-worker, but i dont know if this would work. Unfortunately i am not able to reproduce this error and there are always a couple of days between errors. Thanks, Hajo

4 6

v2.3.4 released
by Timo Sirainen 30 Jan '19

30 Jan '19

https://dovecot.org/releases/2.3/dovecot-2.3.4.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.4.tar.gz.sig Binary packages in https://repo.dovecot.org/ * The default postmaster_address is now "postmaster@<user domain or server hostname>". If username contains the @domain part, that's used. If not, then the server's hostname is used. * "doveadm stats dump" now returns two decimals for the "avg" field. + Added push notification driver that uses a Lua script + Added new SQL, DNS and connection events. See https://wiki2.dovecot.org/Events + Added "doveadm mailbox cache purge" command. + Added events API support for Lua scripts + doveadm force-resync -f parameter performs "index fsck" while opening the index. This may be useful to fix some types of broken index files. This may become the default behavior in a later version. - director: Kicking a user crashes if login process is very slow - pop3_no_flag_updates=no: Don't expunge DELEted and RETRed messages unless QUIT is sent. - auth: Fix crypt() segfault with glibc-2.28+ - imap: Running UID FILTER script with errors assert-crashes - dsync, pop3-migration: POP3 UIDLs weren't added to dovecot.index.cache while mails were saved. - dict clients may have been using 100% CPU while waiting for dict server to finish commands. - doveadm user: Fixed user listing via HTTP API - All levels of Cassandra log messages were logged as Dovecot errors. - http/smtp client may have crashed after SSL handshake - Lua auth converted strings that looked like numbers into numbers.

18 145

doveadm batch crash
by Jonathan Casiot 30 Jan '19

30 Jan '19

Hi I'm having an issue with doveadm batch - the following command always crashes: doveadm batch : mailbox status all inbox Whereas the following work as expected: doveadm mailbox status all inbox doveadm batch : mailbox list Any assistance gratefully received. Debug output from the crash: Debug: Loading modules from directory: /usr/lib64/dovecot Debug: Module loaded: /usr/lib64/dovecot/lib01_acl_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib10_mail_crypt_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib90_stats_plugin.so Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so doveadm(jc): Debug: Effective uid=1001, gid=100, home=/home/jc doveadm(jc): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled doveadm(jc): Debug: mail_crypt_plugin: mail_crypt_curve setting missing - generating EC keys disabled doveadm(jc): Debug: Namespace inbox: type=private, prefix=, sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:~/mdbox:ALT=~/Archive/tape/mdbox doveadm(jc): Debug: fs: root=/home/jc/mdbox, index=, indexpvt=, control=, inbox=, alt=/home/jc/Archive/tape/mdbox doveadm(jc): Debug: acl: initializing backend with data: vfile doveadm(jc): Debug: acl: acl username = jc doveadm(jc): Debug: acl: owner = 1 doveadm(jc): Debug: acl vfile: Global ACLs disabled doveadm(jc): Panic: file mail-storage.c: line 875 (mailbox_set_reason): assertion failed: (reason != NULL) doveadm(jc): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xa0e9e) [0x7fc993023e9e] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7fc993023f0a] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7fc992fb3c97] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x49856) [0x7fc993301856] -> /usr/bin/doveadm(+0x348e2) [0x5631dbb5f8e2] -> /usr/bin/doveadm(+0x2e1ad) [0x5631dbb591ad] -> /usr/bin/doveadm(+0x2bd2c) [0x5631dbb56d2c] -> /usr/bin/doveadm(+0x2c92a) [0x5631dbb5792a] -> /usr/bin/doveadm(doveadm_mail_try_run+0x215) [0x5631dbb58285] -> /usr/bin/doveadm(main+0x46a) [0x5631dbb473fa] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fc992bd83d5] -> /usr/bin/doveadm(+0x1c565) [0x5631dbb47565] Aborted (core dumped) And doveconf -n output: # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-957.1.3.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: ### redacted ### first_valid_gid = 100 first_valid_uid = 1000 last_valid_gid = 100 last_valid_uid = 1999 listen = * login_greeting = IMAP ready login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e session=<%{session}> %k mail_attribute_dict = file:%h/mdbox/dovecot-attributes mail_location = mdbox:~/mdbox:ALT=~/Archive/tape/mdbox mail_plugins = acl stats mail_crypt mailbox_list_index = yes mdbox_rotate_size = 16 M namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { driver = pam } plugin { acl = vfile mail_crypt_global_private_key = <### redacted ### mail_crypt_global_public_key = <### redacted ### mail_crypt_save_version = 2 stats_memory_limit = 16 M stats_refresh = 30 secs stats_track_cmds = yes } postmaster_address = ### redacted ### protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { process_limit = 200 process_min_avail = 4 service_count = 1 } service stats { fifo_listener stats-mail { group = users mode = 01224 } } ssl = required ssl_cert = <### redacted ### ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv2 !SSLv3 !TLSv1 userdb { driver = passwd } verbose_proctitle = yes protocol imap { imap_idle_notify_interval = 10 mins imap_metadata = yes mail_max_userip_connections = 20 mail_plugins = acl stats mail_crypt imap_acl imap_stats } Regards, -- Jonathan

2 2

doveadm proxy list repeats header line
by Michael Goth 20 Jan '19

20 Jan '19

Hi all, just a minor thing: Why does `doveadm proxy list` repeat the header line after each entry? Too me, that just makes the output harder to read. So instead of this: root@mail1 ~ # doveadm proxy list username proto src ip dest ip port info(a)domain1.de imap 80.152.196.98 222.133.127.53 993 username service src-ip dest-ip dest-port info(a)domain1.de imap 80.152.196.98 222.133.127.53 993 username service src-ip dest-ip dest-port info(a)domain1.de imap 80.152.196.98 222.133.127.53 993 username service src-ip dest-ip dest-port tj(a)domain1.de imap 83.149.131.71 222.133.127.53 993 username service src-ip dest-ip dest-port tj(a)domain1.de imap 83.149.131.71 222.133.127.53 993 I'd prefer this: root@mail1 ~ # doveadm proxy list username proto src ip dest ip port info(a)domain1.de imap 80.152.196.98 222.133.127.53 993 info(a)domain1.de imap 80.152.196.98 222.133.127.53 993 info(a)domain1.de imap 80.152.196.98 222.133.127.53 993 tj(a)domain1.de imap 83.149.131.71 222.133.127.53 993 tj(a)domain1.de imap 83.149.131.71 222.133.127.53 993 The repeated header lines also happen with other formatters. Regards, Michael

3 3

Panic…
by SH Development 20 Jan '19

20 Jan '19

I have started getting these in my log. What does this mean and what do I need to do? Panic: file mail-index-util.c: line 37 (mail_index_uint32_to_offset): assertion failed: (offset < 0x40000000) Dec 12 23:29:44 mail dovecot: imap(jeff(a)stariontech.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x3cafa) [0x7f38c5a2cafa] -> /usr/lib64/dovecot/libdovecot.so.0(+0x3cb46) [0x7f38c5a2cb46] -> /usr/lib64/dovecot/libdovecot.so.0(+0x16e2a) [0x7f38c5a06e2a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0x74) [0x7f38c5d0d3f4] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x81abd) [0x7f38c5cf3abd] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_cache_compress+0x77) [0x7f38c5cf4247] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x1d2) [0x7f38c5d08012] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x5fd88) [0x7f38c5cd1d88] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x570a0) [0x7f38c5cc90a0] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_storage_sync_force+0x52) [0x7f38c5cc94c2] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_file_do+0x99) [0x7f38c5cc9f49] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x59b7f) [0x7f38c5ccbb7f] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_mail_set_seq+0x112) [0x7f38c5cb64f2] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0x16b) [0x7f38c5cbae9b] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x20) [0x7f38c5ca2240] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_search_next+0x26) [0x7f38c5ca2296] -> dovecot/imap(imap_fetch_more+0x2bf) [0x41221f] -> dovecot/imap(cmd_fetch+0x42c) [0x40a33c] -> dovecot/imap() [0x40f90d] -> dovecot/imap() [0x40f9fa] -> dovecot/imap(client_handle_input+0x135) [0x40fc25] -> dovecot/imap(client_input+0x5f) [0x41054f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x48) [0x7f38c5a37e98] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f38c5a38e0f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f38c5a37e28] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f38c5a26293] -> dovecot/imap(main+0x2f9) [0x418849] -> /lib64/libc.so.6(__libc_start_main+0x10 Dec 12 23:29:44 mail dovecot: master: Error: service(imap): child 11157 killed with signal 6 (core dumps disabled)

6 5