July 2019 - dovecot - dovecot.org

intermittant passdb error
by NeuralNET 23 Jan '20

23 Jan '20

I'm having great difficulty debugging this problem.. I'm having intermittant problems connecting to dovecot. This started a couple of days ago following an apt-get update on the server. I'm running Ubuntu 14.04, dovecot 2.2.9. Here's my dovecot -n --- # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.19.0-59-generic x86_64 Ubuntu 14.04.4 LTS auth_mechanisms = plain login mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve_dir = ~/sieve } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } user = $default_internal_user } ssl = required ssl_cert = </etc/dovecot/box.neuralnet.org.uk_crt.pem ssl_key = </etc/dovecot/box.neuralnet.org.uk_key.pem ssl_protocols = !SSLv2 !SSLv3 userdb { driver = passwd } --- Pretty standard stuff. In addition disable_plaintext_auth = yes is set in 10-auth.conf This showed up in the mail.err after the reboot; --- May 17 09:04:51 server dovecot: imap-login: Error: read(anvil) failed: EOF May 17 09:04:51 server dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF May 17 09:04:51 server dovecot: auth: Error: net_connect_unix(anvil-auth-penalty) failed: Permission denied --- At that point The following commands were run on the server a few minutes apart with no changes made in the interim; --- username@box /etc/dovecot $ doveadm user username field valueuserdb lookup: user username doesn't exist username@box /etc/dovecot $ doveadm user username field value uid 1000 gid 1000 home /home/username mail maildir:~/Maildir system_groups_user username --- so, passdb wasn't accessible, but now it is ? Imap logins are showing in mail.log as failing, but thunderbird is currently connected; --- May 18 12:00:58 box dovecot: imap-login: Disconnected (auth failed, 3 attempts in 19 secs): user=<username>, method=PLAIN, rip=10.0.0.3, lip=10.0.0.99, TLS, session=<********> May 18 12:08:27 box dovecot: imap-login: Disconnected (auth failed, 3 attempts in 20 secs): user=<username>, method=PLAIN, rip=10.0.0.3, lip=10.0.0.99, TLS, session=<********> May 18 12:08:50 box dovecot: imap-login: Disconnected (auth failed, 3 attempts in 12 secs): user=<username>, method=PLAIN, rip=10.0.0.3, lip=10.0.0.99, TLS, session=<********> --- do I conclude that method=plain is failing, whereas method=login succeeds ? I tried setting auth_debug_password=yes and get the following; --- May 18 12:26:42 box dovecot: auth: Debug: client in: CONT#0111#011A********== (previous base64 data may contain sensitive data) May 18 12:26:42 box dovecot: auth-worker(12214): Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth May 18 12:26:42 box dovecot: auth-worker(12214): Debug: pam(expert,10.0.0.3): lookup service=dovecot May 18 12:26:42 box dovecot: auth-worker(12214): Debug: pam(expert,10.0.0.3): #1/1 style=1 msg=Password: May 18 12:26:42 box dovecot: auth-worker(12211): pam(username,10.0.0.3): pam_authenticate() failed: Authentication failure (password mismatch?) (given password: ******) May 18 12:26:44 box dovecot: auth: Debug: client passdb out: FAIL#0112#011user=username --- Any assistance debugging this further is greatly recieved! Thanks.

2 2

CentOS x64 compilation fails.
by Mart Pirita 16 Dec '19

16 Dec '19

Hello. I have problems to compile Dovecot2.2.13 with rpmbuild using options: ./configure \ --prefix=/usr \ --with-ssl=openssl \ --with-ssldir=/etc/ssl \ --sysconfdir=/etc \ --without-vpopmail \ --with-pam \ --without-bsdauth \ --without-sql \ --without-nss \ --without-ldap \ --without-pgsql \ --without-mysql \ --without-sqlite \ --with-rundir=/var/run/dovecot \ --without-deliver \ --without-gssapi make Using CentOS 5.11, option - rpmbuild -ba dovecot2.2.spec fails with error: /bin/mkdir -p '/var/tmp/dovecot-2.2.13-mp-root/usr/lib64/dovecot' /bin/sh ../../libtool --mode=install /usr/bin/install -c libdovecot-storage.la '/var/tmp/dovecot-2.2.13-mp-root/usr/lib64/dovecot' libtool: install: error: cannot install `libdovecot-storage.la' to a directory not ending in /usr/lib/dovecot make[4]: *** [install-pkglibLTLIBRARIES] Error 1 make[4]: Leaving directory `/usr/src/redhat/BUILD/dovecot-2.2.13/src/lib-storage' make[3]: *** [install-am] Error 2 make[3]: Leaving directory `/usr/src/redhat/BUILD/dovecot-2.2.13/src/lib-storage' make[2]: *** [install-recursive] Error 1 make[2]: Leaving directory `/usr/src/redhat/BUILD/dovecot-2.2.13/src/lib-storage' make[1]: *** [install-recursive] Error 1 make[1]: Leaving directory `/usr/src/redhat/BUILD/dovecot-2.2.13/src' make: *** [install-recursive] Error 1 error: Bad exit status from /var/tmp/rpm-tmp.14076 (%install) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.14076 (%install) But using CentOS 5.11with option- rpmbuild --target=i686 -ba dovecot2.2.spec works fine. However, using CentOS 6.5, with option - rpmbuild -ba dovecot2.2.spec fails with error: /bin/mkdir -p '/root/rpmbuild/BUILDROOT/dovecot-2.2.13-mp.x86_64/usr/lib64/dovecot' /bin/sh ../../libtool --mode=install /usr/bin/install -c libdovecot-storage.la '/root/rpmbuild/BUILDROOT/dovecot-2.2.13-mp.x86_64/usr/lib64/dovecot' libtool: install: error: cannot install `libdovecot-storage.la' to a directory not ending in /usr/lib/dovecot make[4]: *** [install-pkglibLTLIBRARIES] Error 1 make[4]: Leaving directory `/root/rpmbuild/BUILD/dovecot-2.2.13/src/lib-storage' make[3]: *** [install-am] Error 2 make[3]: Leaving directory `/root/rpmbuild/BUILD/dovecot-2.2.13/src/lib-storage' make[2]: *** [install-recursive] Error 1 make[2]: Leaving directory `/root/rpmbuild/BUILD/dovecot-2.2.13/src/lib-storage' make[1]: *** [install-recursive] Error 1 make[1]: Leaving directory `/root/rpmbuild/BUILD/dovecot-2.2.13/src' make: *** [install-recursive] Error 1 error: Bad exit status from /var/tmp/rpm-tmp.HP3meN (%install) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.HP3meN (%install) But when CentOS 5.11with option rpmbuild --target=i686 -ba dovecot2.2.spec suceeds, CentOS 6.5still fails: cpio: dovecot-2.2.13/src/lib-compression/abspath.c: Cannot stat: No such file or directory cpio: dovecot-2.2.13/src/lib-compression/compat.h: Cannot stat: No such file or directory cpio: dovecot-2.2.13/src/lib-compression/lib.h: Cannot stat: No such file or directory 18096 blocks + /usr/lib/rpm/check-buildroot /root/rpmbuild/BUILDROOT/dovecot-2.2.13-mp.i386/usr/lib/dovecot/dovecot-config:LIBDOVECOT='-L/root/rpmbuild/BUILDROOT/dovecot-2.2.13-mp.i386/usr/lib/dovecot -ldovecot' /root/rpmbuild/BUILDROOT/dovecot-2.2.13-mp.i386/usr/lib/dovecot/dovecot-config:LIBDOVECOT_INCLUDE=-I/root/rpmbuild/BUILDROOT/dovecot-2.2.13-mp.i386/usr/include/dovecot Found '/root/rpmbuild/BUILDROOT/dovecot-2.2.13-mp.i386' in installed files; aborting error: Bad exit status from /var/tmp/rpm-tmp.69mGY0 (%install) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.69mGY0 (%install) How to fix these errors? -- Mart

5 8

BUG: panic when using fs:posix as dict for acl_shared_dict
by Marco Giunta 13 Dec '19

13 Dec '19

Hi, I'm using Dovecot 2.2.33.2 on a RHEL 7, new installation. When I use fs:posix as dict for acl_shared_dict, like in Dovecot wiki (https://wiki.dovecot.org/SharedMailboxes/ClusterSetup) doveadm-server crash with error: # doveadm acl set -u USERNAME FOLDER user=DEST_USERNAME lookup read write-seen doveadm(USERNAME): Panic: file dict-fs.c: line 127 (fs_dict_iterate_init): assertion failed: ((flags & DICT_ITERATE_FLAG_RECURSE) == 0) doveadm(USERNAME): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x9f3de) [0x7f0e4a4b23de] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7f0e4a4b244a] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0e4a44377c] -> /usr/lib64/dovecot/libdovecot.so.0(+0x31688) [0x7f0e4a444688] -> /usr/lib64/dovecot/libdovecot.so.0(dict_iterate_init_multiple+0x4d) [0x7f0e4a47cadd] -> /usr/lib64/dovecot/libdovecot.so.0(dict_iterate_init+0x29) [0x7f0e4a47cb89] -> /usr/lib64/dovecot/lib01_acl_plugin.so(acl_lookup_dict_rebuild+0x3e1) [0x7f0e49a40371] -> /usr/lib64/dovecot/lib01_acl_plugin.so(acl_backend_vfile_acllist_rebuild+0x488) [0x7f0e49a3dd18] -> /usr/lib64/dovecot/lib01_acl_plugin.so(acl_backend_vfile_object_update+0x3c7) [0x7f0e49a3e867] -> /usr/lib64/dovecot/lib01_acl_plugin.so(acl_mailbox_update_acl+0x68) [0x7f0e49a41e28] -> /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so(+0x2c11) [0x7f0e48da1c11] -> /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so(+0x3060) [0x7f0e48da2060] -> doveadm(+0x2b41c) [0x556f1280b41c] -> doveadm(+0x2c01a) [0x556f1280c01a] -> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x23b) [0x556f1280ce7b] -> doveadm(doveadm_cmd_run_ver2+0x50c) [0x556f1281c73c] -> doveadm(doveadm_cmd_try_run_ver2+0x37) [0x556f1281c7d7] -> doveadm(main+0x1e4) [0x556f127fb944] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f0e4a071c05] -> doveadm(+0x1bd35) [0x556f127fbd35] Aborted Attached coredump and configuration. I'm not a programmer, but seems that assert is raised by 'fs_dict_iterate_init' function inside 'src/lib-dict-extra/dict-fs.c': static struct dict_iterate_context * fs_dict_iterate_init(struct dict *_dict, const char *const *paths, enum dict_iterate_flags flags) { ... /* these flags are not supported for now */ i_assert((flags & DICT_ITERATE_FLAG_RECURSE) == 0); ... because it is called by 'acl_lookup_dict_iterate_read' function in file 'src/plugins/acl/acl-lookup-dict.c' static void acl_lookup_dict_iterate_read(struct acl_lookup_dict_iter *iter) { ... dict_iter = dict_iterate_init(iter->dict->dict, prefix, DICT_ITERATE_FLAG_RECURSE); ... with DICT_ITERATE_FLAG_RECURSE set. Same problem also with Dovecot 2.3.0. Thanks, Marco -- Marco Giunta - ITCS SysAdmin Via Bonomea, 265 34136 - Trieste, Italy Tel: +39-040-3787-503 Fax: +39-040-3787-244

4 6

Supporting RFC 5466 (IMAP4 Extension for Named Searches (Filters))
by Jesus Cea 06 Dec '19

06 Dec '19

I wonder if Dovecot supports RFC 5466 (IMAP4 Extension for Named Searches (Filters)) or if there is any plan about it. -- Jesús Cea Avión _/_/ _/_/_/ _/_/_/ jcea(a)jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ Twitter: @jcea _/_/ _/_/ _/_/_/_/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz

4 6

dovecot sieve duplicates detection
by André Rodier 04 Dec '19

04 Dec '19

Hello, I have tested the sieve duplicate script with success so far, but I have a question. I would like to know if the "duplicate" sieve flag in Dovecot is global to all folders, or specific to one folder only. For instance, if I copy an email from one folder to another, and I have a discard action on duplicate email, is this action will be applied (in this case, discard) or not. If the duplicate is global to all folders, is there a way to restrict the search in one folder only. Thanks for your help. André

3 10

Migration
by Hans Brage 01 Oct '19

01 Oct '19

Hi! I'm currently running an small imap-server on Dovecot 2.2.4 but will retire that server. I've set up an new server with Dovecot 2.3.4 and will migrate the mailboxes (maildir-format) from the old to the new server. I'm planning to restructure the accounts a bit when migrating so I need to move them one by one. Its only a few so thats not a big issue. Both servers are using virtual accounts (users file). I've understood that best method for migration is to run doveadm backup -R from the new server. But I really cant figure out what other parameters or settings that I need to perform that task and if it should be run as user vmail or root. Would be greatful for any tips, hints, links or similiar. Regards, Hans

6 9

Best way to move mail from one server to another
by James Brown 01 Oct '19

01 Oct '19

I’d like to migrate to a new server. I only need to transfer the mail store, have already done the users. I’ve seen different recommendations: IMAP-Sync Rsync Doveadmin backup and dsync What are the pros and cons of each? What would be best for me? I’m thinking that Doveadmin backup is the way to go. Both old and new servers are on same network. New server is running Dovecot 2.3.2.1, old server is 2.2.32. macOS X. Does anyone have any advice or examples? Thanks, James.

13 29

WARNING: using attachment_dir with plugin zlib can corrupt mails
by Patrick Cernko 12 Sep '19

12 Sep '19

Hello list, hello Dovecot developers, this week, I discovered a serious bug in Dovecot, that lead to several broken mails on our servers. The bug corrupts the first few characters of the mail header during saving. On our setup, it was almost always only the very first line of text, that was corrupted. Depending on the IMAP client (they seem to request different header fields, ... during mail access), the bug causes the imap process to hang up the TCP connection and log errors like this: > imap(USERNAME)<4767><TeQP4ASOTK5/AAAB>: Error: Corrupted record in index cache file /IMAP/mail/mailboxes/USERNAME/mdbox/mailboxes/Trash/dbox-Mails/dovecot.index.cache: UID 489113: Broken fields in mailbox Trash: read(attachments-connector(zlib(/IMAP/mail/mailboxes/USERNAME/mdbox/storage/m.813))): FETCH BODY[HEADER.FIELDS (RETURN-PATH SUBJECT)] got too little data: 2 vs 122 In our case that finally grabbed my attention, the client was the users iphone that did not display any new messages but his Thunderbird did. The bug seems to be triggered by a bad "interaction" of attachment_dir option and zlib plugin. If you use both, you most likely are affected, too, except you only use zlib plugin for reading previously compressed stored mails. That's also the workaround we use now: zlib plugin only enabled in mail_plugins but no plugin/zlib_save set. The bug occurs on very specific mails. Due to privacy reasons I could not provide sample mails here. Storing such mails seems to trigger the bug reproducible. I attached a very minimal doveconf -n config, that can be used to trigger the bug. If one of the developers is interested, I can try to generate an "anonymized" version of such a specific mail that still causes the issue. I discovered the bug on our productive systems, running latest Dovecot 2.2 release, but the latest 2.3 I used during debugging is affected, too. During debugging, I also found one hint, that might help find the bug: If you store a problematic mail with zlib_save=gz (or zlib_save=bz2) and then disable the zlib plugin in mail_plugins, you can call doveadm fetch -u test hdr all | grep -v ^hdr: | gzip --decompress on test's mailbox with only that one broken mail. This will display the beginning of the rfc822 mail text until gzip terminates with "gzip: stdin: unexpected end of file", approximately after twice the length of the mail HEADER. This might indicate, that dovecot stores the uncompressed size of the header in it's data structures although the mail is stored compressed. I also found a very efficient way to find all affected mails in our setup: doveadm -f flow fetch -A 'user guid mailbox uid seq flags hdr' all | \ grep -a "^[^ ]+ user=" | \ grep -avF ' hdr=Return-path: ' | \ grep -av '.* hdr=[[:print:][:space:]]*$' (runtime for ~6M mails on our servers was 20-30min) This can be even more optimized if you have a powerful storage system with GNU parallel: > doveadm user '*' | parallel "doveadm -f flow fetch -u '{}' 'user guid mailbox uid seq flags hdr' all | grep -a '^user=' | grep -avF ' hdr=Return-path: ' | grep -av '.* hdr=[[:print:][:space:]]*$' || true" (runtime for ~6M mails on our servers was ~4min) The command will give you a list of mails that possibly are affected, check the full output of doveadm fetch -u USERNAME hdr guid GUID | less to verify that the header is really broken. On our systems I found 39 mails within ~12M mails. I was able to recover these mails "manually" by reconstructing the Return-Path header line, importing the fixed mails and expunging the corrupt ones. Before importing, I had to disable zlib_save option obviously. Best regards, -- Patrick Cernko <pcernko(a)mpi-klsb.mpg.de> +49 681 9325 5815 Joint Administration: Information Services and Technology Max-Planck-Institute fuer Informatik & Softwaresysteme

4 4

pigeonhole/lda accessing -m folder
by matthias lay 12 Sep '19

12 Sep '19

Hi all, anybody knows, if there is a way to acces the folder from a call like this deovecot-lda -m destfolder from within a sieve script? thx in advance matze

5 9

Case sensitive problem for vacation with dovecot lmtp pigeonhole
by kadafax＠gmail.com 11 Sep '19

11 Sep '19

Hi list, This issue was raised here by John Fawcett on 12/02/19 but got no response. So, as I'm facing the same issue, I would like to try again: When using the vacation sieve extension, if I send an email to user.name(a)mydomain.com it will work, but not to User.Name(a)mydomain.com. Is there a way to make the user part case insensitive so that the vacation auto-reply is fired whatever the way user part is written ? doveconf -n output: # 2.3.5.1 (7ec6d0ade): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.5 (2483b085) # OS: Linux 3.10.0-957.5.1.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) ext4 # Hostname: myserver.mydomain.com auth_master_user_separator = * auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* auth_verbose = yes debug_log_path = /var/log/dovecot-debug.log default_client_limit = 10640 default_process_limit = 2128 default_vsz_limit = 0 hostname = mydomain.com info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot-error.log login_greeting = Welcome mail_gid = vmail mail_location = maildir:/var/vmail/%Ln mail_plugins = quota fts fts_lucene mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = maildir:/var/vmail/%Ln mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /usr/local/etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = /usr/local/etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. quota = maildir:User quota quota_rule = *:storage=30GB quota_rule2 = Trash:storage=+200M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/sieve-scripts/%Ln.sieve sieve_dir = /var/vmail/%Ln/home/sieve } postmaster_address = postmaster(a)mydomain.com protocols = imap pop3 lmtp sieve sieve quota_full_tempfail = yes service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0777 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_min_avail = 10 } service imap { process_limit = 2128 } service lmtp { executable = lmtp -L process_min_avail = 15 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = nobody } ssl_cert = </etc/letsencrypt/live/myserver.mydomain.com/fullchain.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /usr/local/etc/dovecot/dovecot-ldap-userdb.conf.ext driver = ldap override_fields = uid=vmail gid=vmail home=/var/vmail/%Ln/home } protocol imap { mail_max_userip_connections = 30 mail_plugins = quota fts fts_lucene imap_quota } protocol lmtp { info_log_path = /var/log/dovecot-lmtp.log log_path = /var/log/dovecot-lmtp-errors.log mail_plugins = quota fts fts_lucene sieve } protocol sieve { mail_max_userip_connections = 20 }

2 2