January 2019 - dovecot

lmtp 2.3.2.1 segfault with backtrace
by Tom Sommer 13 Mar '19

13 Mar '19

lmtp on Director crash with 2.3.2.1 # gdb /usr/libexec/dovecot/lmtp /var/core/60174 GNU gdb (GDB) Red Hat Enterprise Linux (7.2-92.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/libexec/dovecot/lmtp...Reading symbols from /usr/lib/debug/usr/libexec/dovecot/lmtp.debug...done. done. [New Thread 60174] Reading symbols from /usr/lib64/dovecot/libdovecot-lda.so.0...Reading symbols from /usr/lib/debug/usr/lib64/dovecot/libdovecot-lda.so.0.0.0.debug...done. done. Loaded symbols for /usr/lib64/dovecot/libdovecot-lda.so.0 Reading symbols from /usr/lib64/dovecot/libdovecot-storage.so.0...Reading symbols from /usr/lib/debug/usr/lib64/dovecot/libdovecot-storage.so.0.0.0.debug...done. done. Loaded symbols for /usr/lib64/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/lib64/dovecot/libdovecot.so.0...Reading symbols from /usr/lib/debug/usr/lib64/dovecot/libdovecot.so.0.0.0.debug...done. done. Loaded symbols for /usr/lib64/dovecot/libdovecot.so.0 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /usr/lib64/dovecot/libssl_iostream_openssl.so...Reading symbols from /usr/lib/debug/usr/lib64/dovecot/libssl_iostream_openssl.so.debug...done. done. Loaded symbols for /usr/lib64/dovecot/libssl_iostream_openssl.so Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libssl.so.10 Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libcrypto.so.10 Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libgssapi_krb5.so.2 Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done. Loaded symbols for /lib64/libkrb5.so.3 Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libcom_err.so.2 Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols found)...done. Loaded symbols for /lib64/libk5crypto.so.3 Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libz.so.1 Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libkrb5support.so.0 Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libkeyutils.so.1 Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /lib64/libselinux.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libselinux.so.1 Core was generated by `dovecot/lmtp'. Program terminated with signal 11, Segmentation fault. #0 smtp_client_command_set_replies (cmd=0x0, replies=1) at smtp-client-command.c:401 401 i_assert(cmd->replies_expected == 1 || Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.212.el6.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-65.el6.x86_64 libcom_err-1.41.12-24.el6.x86_64 libselinux-2.0.94-7.el6.x86_64 openssl-1.0.1e-57.el6.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) bt full #0 smtp_client_command_set_replies (cmd=0x0, replies=1) at smtp-client-command.c:401 __func__ = "smtp_client_command_set_replies" #1 0x00007f5d14f40f3f in smtp_client_transaction_data_cb (reply=0x7ffe5cd19650, trans=0x7f5d176ae2b8) at smtp-client-transaction.c:658 conn = 0x7f5d176ade80 rcpt = 0x7f5d176ae560 i = <value optimized out> count = 1 #2 0x00007f5d14f3e941 in smtp_client_command_fail_reply (_cmd=<value optimized out>, reply=0x7ffe5cd19650) at smtp-client-command.c:299 cmd = 0x7f5d17600d18 tmp_cmd = <value optimized out> conn = 0x7f5d176ade80 state = <value optimized out> callback = 0x7f5d14f40e80 <smtp_client_transaction_data_cb> #3 0x00007f5d14f4113f in smtp_client_transaction_fail_reply (trans=0x7f5d176ae2b8, reply=0x7ffe5cd19650) at smtp-client-transaction.c:365 conn = 0x7f5d176ade80 rcpts = <value optimized out> i = <value optimized out> count = <value optimized out> __func__ = "smtp_client_transaction_fail_reply" #4 0x00007f5d14f41442 in smtp_client_transaction_timeout (trans=0x7f5d176ae2b8) at smtp-client-transaction.c:419 reply = {status = 451, enhanced_code = {x = 4, y = 4, z = 0}, text_lines = 0x7f5d1759c090} #5 0x00007f5d14fdbe7b in io_loop_handle_timeouts_real (ioloop=0x7f5d175a4c80) at ioloop.c:641 timeout = 0x7f5d176299d0 tv = {tv_sec = 0, tv_usec = 0} tv_call = {tv_sec = 1534202969, tv_usec = 770866} t_id = 3 item = 0x7f5d176299d0 #6 io_loop_handle_timeouts (ioloop=0x7f5d175a4c80) at ioloop.c:655 _data_stack_cur_id = 2 #7 0x00007f5d14fdd667 in io_loop_handler_run_internal (ioloop=0x7f5d175a4c80) at ioloop-epoll.c:195 ctx = 0x7f5d175af300 events = <value optimized out> event = <value optimized out> list = <value optimized out> io = <value optimized out> tv = {tv_sec = 34, tv_usec = 105308} events_count = <value optimized out> msecs = <value optimized out> ret = 0 i = <value optimized out> call = <value optimized out> __func__ = "io_loop_handler_run_internal" #8 0x00007f5d14fdb7c5 in io_loop_handler_run (ioloop=0x7f5d175a4c80) at ioloop.c:726 __func__ = "io_loop_handler_run" #9 0x00007f5d14fdb9e8 in io_loop_run (ioloop=0x7f5d175a4c80) at ioloop.c:699 __func__ = "io_loop_run" #10 0x00007f5d14f58963 in master_service_run (service=0x7f5d175a4b10, callback=<value optimized out>) at master-service.c:767 No locals. #11 0x00007f5d159eb584 in main (argc=1, argv=0x7f5d175a48a0) at main.c:158 set_roots = {0x7f5d15262e20, 0x7f5d157c2040, 0x7f5d15bf1540, 0x0} service_flags = <value optimized out> storage_service_flags = <value optimized out> tmp_base_dir = 0x7f5d1759c040 "erver not answering (transaction timed out while waiting for reply to DATA)" c = <value optimized out> error = <value optimized out> -- Tom

3 3

2.3.3: Panic: file ostream-zlib.c: line 37 (o_stream_zlib_close): assertion failed
by Tom Sommer 13 Mar '19

13 Mar '19

I see this in my logs after 2.3.3: using zlib plugin, ofc. Oct 02 10:01:39 imap(user(a)example.com)<50643></2k4Sjp3vMqC496W>: Panic: file ostream-zlib.c: line 37 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0 || zstream->ostream.error_handling_disabled) Oct 02 10:01:39 imap(user(a)example.com)<50643></2k4Sjp3vMqC496W>: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xce56a) [0x7f442487556a] -> /usr/lib64/dovecot/libdovecot.so.0(+0xce5b1) [0x7f44248755b1] -> /usr/lib64/dovecot/libdovecot.so.0(+0x3d941) [0x7f44247e4941] -> /usr/lib64/dovecot/lib20_zlib_plugin.so(+0x5cdf) [0x7f44233c4cdf] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf4b46) [0x7f442489bb46] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_destroy+0x13) [0x7f442489be83] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x173) [0x7f4424b93973] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x36) [0x7f4424b68696] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](+0xede9) [0x55a9d7e86de9] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](command_exec+0x65) [0x55a9d7e956e5] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](+0xf9e6) [0x55a9d7e879e6] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x55) [0x7f442488c275] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbf) [0x7f442488e13f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x55) [0x7f442488c365] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f442488c588] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f4424808053] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](main+0x32d) [0x55a9d7ea20dd] -> /lib64/libc.so.6(__libc_start_main+0x100) [0x7f4424431d20] -> dovecot/imap [user(a)example.com X.X.X.X APPEND](+0xe419) [0x55a9d7e86419] No clue how to reproduce -- Tom

3 2

imap-hibernate returned failure: Failed to parse client input: Invalid peer_dev_minor value
by Alex 18 Feb '19

18 Feb '19

Hello! I tried to use imap-hibernate. But errors in maillog: Oct 19 05:14:24 server dovecot: imap(email(a)example.com)<67125><Lu7ndYt4n8aASO/q>: Error: kevent(-1) for notify remove failed: Bad file descriptor Oct 19 05:14:24 server dovecot: imap(email(a)example.com)<67125><Lu7ndYt4n8aASO/q>: Error: close(-1) for notify remove failed: Bad file descriptor Oct 19 05:14:24 server dovecot: imap-hibernate: Error: Failed to parse client input: Invalid peer_dev_minor value: 18446744073709486335 Oct 19 05:14:24 server dovecot: imap(email(a)example.com)<67125><Lu7ndYt4n8aASO/q>: Error: /var/run/dovecot/imap-hibernate returned failure: Failed to parse client input: Invalid peer_dev_minor value: 18446744073709486335 Oct 19 05:14:24 server dovecot: imap: Error: dovecot-2.3.3, FreeBSD 10.4 dovenull 38990 0,0 0,0 43748 16308 - S 3:54 0:00,08 imap-login: (imap-login) dovenull 38998 0,0 0,1 47844 19536 - S 3:54 0:00,18 imap-login: [34.193.82.152 TLS proxy] (imap-login) dovenull 38999 0,0 0,1 47844 17180 - S 3:54 0:00,19 imap-login: (imap-login) dovenull 39000 0,0 0,0 35556 9584 - S 3:54 0:00,03 dovecot/imap-login dovenull 39001 0,0 0,0 43748 16320 - S 3:54 0:00,12 imap-login: (imap-login) dovenull 39002 0,0 0,0 43748 16320 - S 3:54 0:00,13 imap-login: (imap-login) dovenull 39003 0,0 0,0 43748 16328 - S 3:54 0:00,16 imap-login: (imap-login) dovenull 39004 0,0 0,1 47844 17948 - S 3:54 0:00,17 imap-login: [12.34.56.78 TLS proxy] (imap-login) vmail 39042 0,0 0,0 15936 3040 - I 3:54 0:00,01 imap-hibernate: [0 connections] (imap-hibernate) vmail 39076 0,0 0,0 78900 8984 - S 3:54 0:00,05 imap: [email(a)example.com 12.34.56.78 IDLE] (imap) vmail 44114 0,0 0,0 78900 8936 - I 4:06 0:00,05 imap: [email(a)example.com 34.193.82.152 IDLE] (imap) srw-rw---- 1 root dovecot 0 19 окт 03:54 imap-hibernate srw------- 1 root dovecot 0 19 окт 03:54 imap-master srw------- 1 root wheel 0 19 окт 03:54 imap-postlogin srw-rw-rw- 1 root wheel 0 19 окт 03:54 imap-urlauth srw------- 1 dovecot wheel 0 19 окт 03:54 imap-urlauth-worker imap_hibernate_timeout = 5 secs service imap-hibernate { extra_groups = $default_internal_group unix_listener imap-hibernate { group = dovecot } user = vmail } service imap { extra_groups = $default_internal_group unix_listener imap-master { group = $default_internal_group } } I tried to set user/group vmail, set mode = 0666, but the same errors. Can you help me with hibernate confiration ? Best regards, Alex

3 8

expunge not removing attachments?
by Ralf Hildebrandt 12 Feb '19

12 Feb '19

I have a large mail backup folder backup(a)backup.invalid; I'm cleaning up daily like this: infimum=`date -d "-4 day" +"%Y-%m-%d"` doveadm expunge -u backup(a)backup.invalid mailbox INBOX SAVEDBEFORE $infimum doveadm purge -u backup(a)backup.invalid yet I see this: # find attachments/ -type f -ctime +5 | wc -l 7522 # find attachments/ -type f | wc -l 127579 # find attachments/ -type f -mtime +5 | wc -l 14361 # find attachments/ -type f | wc -l 127793 About 5.9% of the files in attachments and below are older than 5 days. Why? Is that normal? using dovecot 2:2.3.1-1 from the official repos. -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | https://www.charite.de

2 3

assertion failed: (srcleft <= CHARSET_MAX_PENDING_BUF_SIZE)
by Giacomo 11 Feb '19

11 Feb '19

I'm running the following: # 2.3.4 (0ecbaf23d): /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 11.2-RELEASE-p4 amd64 # FS: ZFS auth_mechanisms = plain login auth_username_format = %Ln listen = * mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { driver = pam } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } user = dovecot } service imap-login { inet_listener imap { port = 0 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service stats { unix_listener stats-reader { group = mode = 0600 user = } unix_listener stats-writer { group = $default_internal_group mode = 0666 user = } user = $default_internal_user } ssl_cert = </root/cert.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } dmesg shows: pid 74441 (imap), uid 1002: exited on signal 6 pid 74784 (imap), uid 1002: exited on signal 6 pid 21471 (imap), uid 1002: exited on signal 6 pid 21470 (imap), uid 1002: exited on signal 6 pid 21464 (imap), uid 1002: exited on signal 6 pid 21731 (imap), uid 1002: exited on signal 6 Sometimes the dovecot process serving an user crashes. He runs mostly airmail on macOS and samsung mail on Android. Happens only to that particular user. The log shows the following: (maillog) Dec 16 14:52:35 mail dovecot: imap-login: Login: user=<USER>, method=PLAIN, rip=79.12.201.157, lip=192.168.1.28, mpid=74779, TLS, session=<wMQI+iN9VtRPDMmd> Dec 16 14:52:39 mail dovecot: imap(USER)<74441><nLa+7SN9+fhPDMmd>: Panic: file charset-iconv.c: line 83 (charset_to_utf8_try): assertion failed: (srcleft <= CHARSET_MAX_PENDING_BUF_SIZE) Dec 16 14:52:39 mail dovecot: imap(USER)<74441><nLa+7SN9+fhPDMmd>: Fatal: master: service(imap): child 74441 killed with signal 6 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set service imap { drop_priv_before_exec=yes }) I'm here if more info is needed. Thanks for taking a look.

4 7

mbox 2 Maildir
by Tamas Hegedus 08 Feb '19

08 Feb '19

Hi, I am not an professional system admin, but I am running a small debian mail server for many years. It is time to migrate from mbox to Maildir. There is a nice and clean description for doing this without service downtime at https://wiki.dovecot.org/Tools/Doveadm/Sync I have to set per-user config for mailbox type using userdb extra fields. However, I miss some information and I can not find useful info by google. 1. The main question: dovecot will know that for some users the mailbox format is still mbox, but others already have Maildir. I do not see any hint that I should setup postfix/procmail for handling the delivery per-user basis. But I think I have to set this, too. 2. I have not learned much about userdb and it seems that dovecot was written for and used by experts. I mean that the docs/wiki is not enough for me: from the wiki page of userdb it will be for me difficult to setup one. Can you suggest some docs for dummies? Thanks for your help, Tamas dovecot --version 2.2.27 (c0f36b0) postfix 3.1.8 procmail v3.23pre 2001/09/13 -- Tamas Hegedus, PhD Senior Research Fellow MTA-SE Molecular Biophysics Research Group Hungarian Academy of Sciences | phone: (36) 1-459 1500/60233 Semmelweis University | fax: (36) 1-266 6656 Tuzolto utca 37-47 | mailto:tamas@hegelab.org Budapest, 1094, Hungary | http://www.hegelab.org --- This email has been checked for viruses by AVG. https://www.avg.com

4 8

SELinux policy to allow Dovecot to connect to Mysql
by Robert Moskowitz 05 Feb '19

05 Feb '19

As I have noted in previous messages, I been getting the following on my new mailserver: Apr 7 10:17:27 z9m9z dovecot: dict: Error: mysql(localhost): Connect failed to database (postfix): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (13) - waiting for 25 seconds before retry They go away when I setenforce 0. It is not a timing issue as I earlier thought. So I googled dovecot mysql selinux and the only worthwhile hit was: http://zszsit.blogspot.com/2012/12/dovecot-mysql-selinux-issue-on-centos6.h… that provides a /etc/selinux/dovecot2mysql.te and other selinux stuff. Is there a simpler way like a setsbool option? With all the howtos on dovecot with mysql, it is interesting that none of them seem to have this problem. Maybe because they connect to mysql through TCP port 3306 which has ITS set of problems (like MariaDB defaults to not listening on TCP). thanks!

1 3

Error: User bob@aaa.bbb doesn't have home dir set, disabling duplicate database
by subscription1 02 Feb '19

02 Feb '19

I made a mistake when I moved dovecot to a new server and only specified mail_location instead of mail_home All I have in 10-mail.conf is --------- mail_location = maildir:/home/vmail/mailboxes/%d/%n ------------ All emails for the few accounts I have are in these mailboxes and I can get/see them via my mail client I do, however, get the following error -------------------- imap(bob(a)aaa.bbb): Error: User bob(a)aaa.bbb doesn't have home dir set, disabling duplicate database ------------------- After looking at https://wiki2.dovecot.org/VirtualUsers/Home I tried the following -------------------------------- mail_home = /home/vmail/mailboxes/%d/%n mail_location = maildir:~/mail ------------------------------ Problem with this is that I now can't see any of my emails in the client. Appreciate any help on how to fix this. Thanks

4 8

auth: Error - Request timed out
by Hajo Locke 01 Feb '19

01 Feb '19

Hello List, i use dovecot 2.2.22 and have the same problem described here: https://dovecot.org/pipermail/dovecot/2017-November/110020.html I can confirm that sometimes there is a problem with connection to mysql-db, but sometimes not. Reasons for failing are still under investigation by my mates. My current main problem is, that this fail seems to be a one way ticket for dovecot. Even if mysql is verifyable working again and waiting for connection dovecot stucks furthermore with errors like this: May 29 07:00:49 hostname dovecot: auth: Error: plain(mail(a)example.com,xxx.xxx.xx.xxx,<HGI4FVFt2Oy80j3P>): Request 999.7 timed out after 150 secs, state=1 When restarting dovecot all is immediately working again. Is there a way to tell dovecot to restart auth services or reinitialize mysql-connection after these hard fails? I could insert "idle_kill = 1 mins" into service auth and service auth-worker, but i dont know if this would work. Unfortunately i am not able to reproduce this error and there are always a couple of days between errors. Thanks, Hajo

4 6

Dovecot and FTS experiment
by Tomasz Nowak 30 Jan '19

30 Jan '19

Hello, I'm trying to experiment with Dovecot and Solr server. I have >30k email addresses that I want to index to speed up searching and save IOPS on mail servers. For now - I'm doing some experiments and I'm testing how it is working. I'm thinking about adding one additional server with Solr and configure all mail servers to use that server. I have some questions. 1. I have 15 mail servers. It will be good If I add new server with Solr and use it on all Dovecot servers? Or maybe I should install Solr on all mail servers? 2. I notice - I have mail account with 3GB of mail. Index files in mail dir has 5MB. After indexing mailbox in Solr - index files has 15MB. What changes in those files? FTS indexing adds something to that files - but what? 3. If I need to disconnect Dovecot from Solr server - simply Can I remove plugin configuration from dovecot.conf and everything would be working? Or maybe - I need to remove index files and recreate it? It is important for me because if some day I would decide to disconnect FTS server - I would have to recreate indexes for >30k mailboxes.. 4. What if some day I want to change Solr server to new server with empty Solr database? I can simply reindex mailboxes? Or I need to do something more? Thanks!

3 4