August 2019 - dovecot

User found but password failure
by Mauricio Tavares 11 Nov '19

11 Nov '19

Trying to figure out which step is causing me not to be able to login. I am using a password file, passdb { driver = passwd-file args = scheme=SHA512-CRYPT username_format=%u /etc/dovecot/passwd } We will assume that the pw I created using 'doveadm pw -s SHA512-CRYPT' matches the password I will be using to login below. I crank the debugging mode, auth_debug_passwords = yes auth_debug = yes And then try to login [root@mail ~]# nc -t localhost 143 * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN AUTH=GSSAPI] Howdy a login raub(a)example.com password [blank] which from what I gathered from /var/log/dovecot means it found matching username but did not match password: secured session=lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB lip=::1 rip=::1 lport=143 rport=36340 resp=base64-reply (previous base64 data may contain sensitive data) Aug 26 03:25:52 auth: Debug: passwd-file(raub@example.com,::1,<lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB>): lookup: user=raub(a)example.com file=/etc/dovecot/passwd Aug 26 03:25:52 auth: Debug: client passdb out: OK 1 user=raub(a)example.com Aug 26 03:25:52 auth: Debug: master in: REQUEST 1520959489 26560 1 63ee7f45236f85fd39573a5c8a2eb46a session_pid=26563 request_auth_token Aug 26 03:25:52 auth-worker(26562): Debug: passwd(raub@example.com,::1,<lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB>): lookup Aug 26 03:25:52 auth-worker(26562): Info: passwd(raub@example.com,::1,<lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB>): unknown user Aug 26 03:25:52 auth: Debug: passwd-file(raub@example.com,::1,<lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB>): lookup: user=raub(a)example.com file=/etc/dovecot/passwd Aug 26 03:25:52 auth: Error: plain(raub@example.com,::1,<lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB>): user not found from any userdbs Aug 26 03:25:52 auth: Debug: master userdb out: NOTFOUND 1520959489 Aug 26 03:25:52 imap: Error: Authenticated user not found from userdb, auth lookup id=1520959489 (auth connected 1 msecs ago, handshake 0 msecs ago, request took 1 msecs, client-pid=26560 client-id=1) Aug 26 03:25:52 imap-login: Info: Internal login failure (pid=26560 id=1) (internal failure, 1 successful auths): user=<raub(a)example.com>, method=PLAIN, rip=::1, lip=::1, mpid=26563, secured, session=<lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB> Per [1] I decided to see what the response (base64-reply) I am submitting to dovecot looks like: echo 'base64-reply' | base64 -d raub(a)example.compassword Which has the right username and password but follows the userpassword format, not useruserpassword as I was led to believe by [1]. Is that to be expected? If it is then the issue is further down the line. [1] https://wiki.dovecot.org/Debugging/Authentication

3 5

Migration
by Hans Brage 01 Oct '19

01 Oct '19

Hi! I'm currently running an small imap-server on Dovecot 2.2.4 but will retire that server. I've set up an new server with Dovecot 2.3.4 and will migrate the mailboxes (maildir-format) from the old to the new server. I'm planning to restructure the accounts a bit when migrating so I need to move them one by one. Its only a few so thats not a big issue. Both servers are using virtual accounts (users file). I've understood that best method for migration is to run doveadm backup -R from the new server. But I really cant figure out what other parameters or settings that I need to perform that task and if it should be run as user vmail or root. Would be greatful for any tips, hints, links or similiar. Regards, Hans

6 9

Best way to move mail from one server to another
by James Brown 01 Oct '19

01 Oct '19

I’d like to migrate to a new server. I only need to transfer the mail store, have already done the users. I’ve seen different recommendations: IMAP-Sync Rsync Doveadmin backup and dsync What are the pros and cons of each? What would be best for me? I’m thinking that Doveadmin backup is the way to go. Both old and new servers are on same network. New server is running Dovecot 2.3.2.1, old server is 2.2.32. macOS X. Does anyone have any advice or examples? Thanks, James.

13 29

doveadm / vsize
by Paul Macdonald 27 Sep '19

27 Sep '19

Hi, i want to keep track of user mailbox size, I'm using doveadm mailbox status -u <user> vsize INBOX for a given mailbox this is under reporting ( by a lot) is this recursive? what shoud i be using to get an accurate disk usage metric? thanks Paul. -- ------------------------- Paul Macdonald IFDNRG Ltd Web and video hosting ------------------------- t: 0131 5548070 m: 07970339546 e: paul(a)ifdnrg.com w: http://www.ifdnrg.com ------------------------- IFDNRG 40 Maritime Street Edinburgh EH6 6SA ---------------------------------------------------- Virtual Servers from £50.00pm High specification Dedicated Servers from £150.00pm ----------------------------------------------------

3 4

WARNING: using attachment_dir with plugin zlib can corrupt mails
by Patrick Cernko 12 Sep '19

12 Sep '19

Hello list, hello Dovecot developers, this week, I discovered a serious bug in Dovecot, that lead to several broken mails on our servers. The bug corrupts the first few characters of the mail header during saving. On our setup, it was almost always only the very first line of text, that was corrupted. Depending on the IMAP client (they seem to request different header fields, ... during mail access), the bug causes the imap process to hang up the TCP connection and log errors like this: > imap(USERNAME)<4767><TeQP4ASOTK5/AAAB>: Error: Corrupted record in index cache file /IMAP/mail/mailboxes/USERNAME/mdbox/mailboxes/Trash/dbox-Mails/dovecot.index.cache: UID 489113: Broken fields in mailbox Trash: read(attachments-connector(zlib(/IMAP/mail/mailboxes/USERNAME/mdbox/storage/m.813))): FETCH BODY[HEADER.FIELDS (RETURN-PATH SUBJECT)] got too little data: 2 vs 122 In our case that finally grabbed my attention, the client was the users iphone that did not display any new messages but his Thunderbird did. The bug seems to be triggered by a bad "interaction" of attachment_dir option and zlib plugin. If you use both, you most likely are affected, too, except you only use zlib plugin for reading previously compressed stored mails. That's also the workaround we use now: zlib plugin only enabled in mail_plugins but no plugin/zlib_save set. The bug occurs on very specific mails. Due to privacy reasons I could not provide sample mails here. Storing such mails seems to trigger the bug reproducible. I attached a very minimal doveconf -n config, that can be used to trigger the bug. If one of the developers is interested, I can try to generate an "anonymized" version of such a specific mail that still causes the issue. I discovered the bug on our productive systems, running latest Dovecot 2.2 release, but the latest 2.3 I used during debugging is affected, too. During debugging, I also found one hint, that might help find the bug: If you store a problematic mail with zlib_save=gz (or zlib_save=bz2) and then disable the zlib plugin in mail_plugins, you can call doveadm fetch -u test hdr all | grep -v ^hdr: | gzip --decompress on test's mailbox with only that one broken mail. This will display the beginning of the rfc822 mail text until gzip terminates with "gzip: stdin: unexpected end of file", approximately after twice the length of the mail HEADER. This might indicate, that dovecot stores the uncompressed size of the header in it's data structures although the mail is stored compressed. I also found a very efficient way to find all affected mails in our setup: doveadm -f flow fetch -A 'user guid mailbox uid seq flags hdr' all | \ grep -a "^[^ ]+ user=" | \ grep -avF ' hdr=Return-path: ' | \ grep -av '.* hdr=[[:print:][:space:]]*$' (runtime for ~6M mails on our servers was 20-30min) This can be even more optimized if you have a powerful storage system with GNU parallel: > doveadm user '*' | parallel "doveadm -f flow fetch -u '{}' 'user guid mailbox uid seq flags hdr' all | grep -a '^user=' | grep -avF ' hdr=Return-path: ' | grep -av '.* hdr=[[:print:][:space:]]*$' || true" (runtime for ~6M mails on our servers was ~4min) The command will give you a list of mails that possibly are affected, check the full output of doveadm fetch -u USERNAME hdr guid GUID | less to verify that the header is really broken. On our systems I found 39 mails within ~12M mails. I was able to recover these mails "manually" by reconstructing the Return-Path header line, importing the fixed mails and expunging the corrupt ones. Before importing, I had to disable zlib_save option obviously. Best regards, -- Patrick Cernko <pcernko(a)mpi-klsb.mpg.de> +49 681 9325 5815 Joint Administration: Information Services and Technology Max-Planck-Institute fuer Informatik & Softwaresysteme

4 4

pigeonhole/lda accessing -m folder
by matthias lay 12 Sep '19

12 Sep '19

Hi all, anybody knows, if there is a way to acces the folder from a call like this deovecot-lda -m destfolder from within a sieve script? thx in advance matze

5 9

Case sensitive problem for vacation with dovecot lmtp pigeonhole
by kadafax＠gmail.com 11 Sep '19

11 Sep '19

Hi list, This issue was raised here by John Fawcett on 12/02/19 but got no response. So, as I'm facing the same issue, I would like to try again: When using the vacation sieve extension, if I send an email to user.name(a)mydomain.com it will work, but not to User.Name(a)mydomain.com. Is there a way to make the user part case insensitive so that the vacation auto-reply is fired whatever the way user part is written ? doveconf -n output: # 2.3.5.1 (7ec6d0ade): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.5 (2483b085) # OS: Linux 3.10.0-957.5.1.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) ext4 # Hostname: myserver.mydomain.com auth_master_user_separator = * auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* auth_verbose = yes debug_log_path = /var/log/dovecot-debug.log default_client_limit = 10640 default_process_limit = 2128 default_vsz_limit = 0 hostname = mydomain.com info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot-error.log login_greeting = Welcome mail_gid = vmail mail_location = maildir:/var/vmail/%Ln mail_plugins = quota fts fts_lucene mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = maildir:/var/vmail/%Ln mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /usr/local/etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = /usr/local/etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. quota = maildir:User quota quota_rule = *:storage=30GB quota_rule2 = Trash:storage=+200M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/sieve-scripts/%Ln.sieve sieve_dir = /var/vmail/%Ln/home/sieve } postmaster_address = postmaster(a)mydomain.com protocols = imap pop3 lmtp sieve sieve quota_full_tempfail = yes service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0777 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_min_avail = 10 } service imap { process_limit = 2128 } service lmtp { executable = lmtp -L process_min_avail = 15 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = nobody } ssl_cert = </etc/letsencrypt/live/myserver.mydomain.com/fullchain.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /usr/local/etc/dovecot/dovecot-ldap-userdb.conf.ext driver = ldap override_fields = uid=vmail gid=vmail home=/var/vmail/%Ln/home } protocol imap { mail_max_userip_connections = 30 mail_plugins = quota fts fts_lucene imap_quota } protocol lmtp { info_log_path = /var/log/dovecot-lmtp.log log_path = /var/log/dovecot-lmtp-errors.log mail_plugins = quota fts fts_lucene sieve } protocol sieve { mail_max_userip_connections = 20 }

2 2

Dovecot Director upgrade from 2.2 to 2.3
by Alessio Cecchi 10 Sep '19

10 Sep '19

Hi, I have a setup with 3 Dovecot Director v2.2.36 and director_consistent_hashing = yes ;-) Now I would like to upgrade to 2.3.7, first only Director and after also Backend. Can works fine a ring of director with mixed 2.2 and 2.3 version? Mi idea is to setup a new Director server with 2.3, stop one server with 2.2 and insert the new 2.3 in the current ring to check if works fine. If all works fine I will replace all Director 2.2 with 2.3 version. Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 6

CVE-2019-11500: Critical vulnerability in Dovecot and Pigeonhole
by Aki Tuomi 04 Sep '19

04 Sep '19

Dear subscribers, we have been made aware of critical vulnerability in Dovecot and Pigeonhole. --- Open-Xchange Security Advisory 2019-08-14 Product: Dovecot Vendor: OX Software GmbH Internal reference: DOV-3278 Vulnerability type: Improper input validation (CWE-20) Vulnerable version: All versions prior to 2.3.7.2 and 2.2.36.4 Vulnerable component: IMAP and ManageSieve protocol parsers (before and after login) Report confidence: Confirmed Solution status: Fixed by Vendor Fixed version: 2.3.7.2, 2.2.36.4 Researcher credits: Nick Roessler and Rafi Rubin, University of Pennsylvania Vendor notification: 2019-04-13 Solution date: 2019-06-05 Public disclosure: 2019-08-28 CVE reference: CVE-2019-11500 CVSS: 8.1 (CVSS3.0/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H) Vulnerability Details: IMAP and ManageSieve protocol parsers do not properly handle NUL byte when scanning data in quoted strings, leading to out of bounds heap memory writes. Risk: This vulnerability allows for out-of-bounds writes to objects stored on the heap up to 8096 bytes in pre-login phase, and 65536 bytes post-login phase, allowing sufficiently skilled attacker to perform complicated attacks that can lead to leaking private information or remote code execution. Abuse of this bug is very difficult to observe, as it does not necessarily cause a crash. Attempts to abuse this bug are not directly evident from logs. Steps to reproduce: This bug is best observed using valgrind to see the out of bounds read with following snippet: perl -e 'print "a id (\"foo\" \"".("x"x1021)."\\A\" \"bar\" \"\000".("x"x1020)."\\A\")\n"' | nc localhost 143 Solution: Operators should update to the latest Patch Release. There is no workaround for the issue. --- Aki Tuomi Open-Xchange oy

8 14

Dovecot 2.3.7 - char "-" missing
by Domenico Pastore 03 Sep '19

03 Sep '19

Hello, i have update dovecot from version 2.2.15 to 2.3.7.2. I have a problem with mine java software because there is a different response when open connection to doveadm. I need open socket to doveadm for get imap quota of a mailbox. With version 2.2.15: # telnet 192.160.10.4 924 Trying 192.160.10.4... Connected to 192.160.10.4. Escape character is '^]'. - With version 2.3.7.2: # telnet 192.160.10.3 924 Trying 192.160.10.3... Connected to 192.160.10.3. Escape character is '^]'. The difference is "-" character. The version 2.3.7 not respond with "-" character after opening the connection. Is it possible to add the character again with a parameter? Why did doveadm's answer change? Br Domenico

3 2