- dovecot - dovecot.org

Occasional crash in db-auth.c (Valgrind: Invalid read of size 4 et al.), Dovecot 2.2.27+
by Jonas Wielicki 14 Sep '18

14 Sep '18

Dear list, We are encountering troubles with dovecot using LDAP userdbs on Debian stretch (but if I’m reading valgrind correctly, we can reproduce this with vanilla dovecot master). Minimal reproducer below. While testing an upgrade to Debian stretch (dovecot-core=1:2.2.27-3+deb9u1), auth-worker has stopped working. We are using two LDAP user databases; one which is iterable, and one which is not (for reasons; I think this is not relevant; if it is, we’re happy to elaborate). The issue seems to be present in any LDAP userdb iteration handling, but it only seems to cause a crash in certain conditions which seem to be reproducibly caused by our setup for some reason. We’re seeing one of two errors. Variant A: Jan 27 13:40:05 up2 dovecot: auth-worker: Error: *** Error in `dovecot/auth': free(): corrupted unsorted chunks: 0x000056553f0fcfa0 *** Jan 27 13:40:05 up2 dovecot: auth-worker: Error: ======= Backtrace: ========= Jan 27 13:40:05 up2 dovecot: auth-worker: Error: /lib/x86_64-linux-gnu/ libc.so.6(+0x70bcb)[0x7f78426adbcb] Jan 27 13:40:05 up2 dovecot: auth-worker: Error: /lib/x86_64-linux-gnu/ libc.so.6(+0x76f96)[0x7f78426b3f96] Jan 27 13:40:05 up2 dovecot: auth-worker: Error: /lib/x86_64-linux-gnu/ libc.so.6(+0x777de)[0x7f78426b47de] Jan 27 13:40:05 up2 dovecot: auth-worker: Error: /usr/lib/dovecot/modules/ auth/libauthdb_ldap.so(+0x5bde)[0x7f7842008bde] Jan 27 13:40:05 up2 dovecot: auth-worker: Error: /usr/lib/dovecot/ libdovecot.so.0(io_loop_call_io+0x52)[0x7f78430cfdd2] Jan 27 13:40:05 up2 dovecot: auth-worker: Error: /usr/lib/dovecot/ libdovecot.so.0(io_loop_handler_run_internal+0x109)[0x7f78430d1409] Jan 27 13:40:05 up2 dovecot: auth-worker: Error: /usr/lib/dovecot/ libdovecot.so.0(io_loop_handler_run+0x3c)[0x7f78430cfe6c] Jan 27 13:40:05 up2 dovecot: auth-worker: Error: /usr/lib/dovecot/ libdovecot.so.0(io_loop_run+0x38)[0x7f78430d0018] Jan 27 13:40:05 up2 dovecot: auth-worker: Error: /usr/lib/dovecot/ libdovecot.so.0(master_service_run+0x13)[0x7f7843057e93] Jan 27 13:40:05 up2 dovecot: auth-worker: Error: dovecot/auth(main+0x398) [0x56553da45f98] Jan 27 13:40:05 up2 dovecot: auth-worker: Error: /lib/x86_64-linux-gnu/ libc.so.6(__libc_start_main+0xf1)[0x7f784265d2b1] Jan 27 13:40:05 up2 dovecot: auth-worker: Error: dovecot/auth(_start+0x2a) [0x56553da461aa] Variant B: Jan 27 13:06:56 up2 dovecot: auth-worker(27495): Panic: file db-ldap.c: line 840 (db_ldap_result_unref): assertion failed: (res->refcount > 0) Jan 27 13:06:56 up2 dovecot: auth-worker(27495): Error: Raw backtrace: /usr/ lib/dovecot/libdovecot.so.0(+0x95272) [0x7f027a4fc272] -> /usr/lib/dovecot/ libdovecot.so.0(+0x9536d) [0x7f027a4fc36d] -> /usr/lib/dovecot/libdovecot.so. 0(i_fatal+0) [0x7f027a492951] -> /usr/lib/dovecot/modules/auth/ libauthdb_ldap.so(+0x3868) [0x7f0279447868] -> /usr/lib/dovecot/modules/auth/ libauthdb_ldap.so(+0x5d7c) [0x7f0279449d7c] -> /usr/lib/dovecot/libdovecot.so. 0(io_loop_call_io+0x52) [0x7f027a510dd2] -> /usr/lib/dovecot/libdovecot.so. 0(io_loop_handler_run_internal+0x109) [0x7f027a512409] -> /usr/lib/dovecot/ libdovecot.so.0(io_loop_handler_run+0x3c) [0x7f027a510e6c] -> /usr/lib/ dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f027a511018] -> /usr/lib/ dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f027a498e93] -> dovecot/ auth(main+0x398) [0x55b981cd2f98] -> /lib/x86_64-linux-gnu/libc.so. 6(__libc_start_main+0xf1) [0x7f0279a9e2b1] -> dovecot/auth(_start+0x2a) [0x55b981cd31aa] When running auth-worker within valgrind, we get more information (valgrind extract from mail.log attached for two versions of dovecot). We have attached mail.log contents for the respective valgrind runs. This seems to affect at least 2.2.27 onwards. We can reproduce the crash with our production data 100% of the time. The valgrind error is present even with very minimal setups (see below). Unfortunately, I haven’t been able so far to create an LDAP database which produces the crash as reliable as our production database does (which we cannot share for obvious reasons). Minimal Reproducer of Valgrind error: - Install dovecot 2.2.27 from debian stretch, or compile from master. - Install valgrind - In conf.d/10-auth.conf, disable auth-system.conf.ext - In conf.d/10-auth.conf, enable auth-ldap.conf.ext - Set contents of dovecot-ldap.conf.ext to: hosts = localhost base = dc=nodomain - In conf.d/10-master.conf, in section "service auth-worker", set executable = /usr/bin/valgrind /usr/lib/dovecot/auth -w (path may differ on your system) - Install an LDAP server with a database for dc=nodomain; this is trivial to do with debian: Simply install slapd and systemctl start slapd. - Start dovecot - Run doveadm user '*' - You should find the error in the mail.log. I hope this is somehow useful to fix our crash issue. We’ll be happy to provide more information as needed. kind regards, Jonas

4 4

Status of SMTPUTF8?
by Noah Tilton 14 Sep '18

14 Sep '18

I was wondering whether there is a roadmap for adding SMTPUTF8 support to Dovecot? My delivery pattern is Postfix -> Dovecot LMTP and it is choking on utf8 messages. I might be able to volunteer some of my time as a developer. Another thread about this seemed to go unanswered: http://dovecot.org/list/dovecot/2016-September/105474.html http://unix.stackexchange.com/questions/320091/configure-postfix-and-doveco… -Noah

4 3

LMTP tcp listener with auth?
by Andreas Thienemann 13 Sep '18

13 Sep '18

I am wondering if it's possible to have a tcp listener for LMTP do authentication and not accept any unauthenticated mail. My configuration looks like the following: service lmtp { inet_listener lmtp { address = 192.168.0.1 127.0.0.1 port = 24 } } This gives me a network reachable lmtp socket, but of course no autentication. When reading the docs, it seems though that only userdb lookup and no actual authentication is supported? Is that understanding correct? cheers, Andreas

4 3

Server certificate verification error with Dovecot 2.3.2.1
by Robert Gill 13 Sep '18

13 Sep '18

I'm attempting to upgrade my Dovecot installation to 2.3.2.1. My SSL certificate authority provides a bundle containing their CA, plus intermediate CAs, which I configure using the 'ssl_ca' option. The comments in the configuration file say to only set this when you're requiring client certificates, which I'm not, but fetchmail complains with a "Server certificate verification error, Broken certificate chain" error if that setting is not set. This works fine with Dovecot 2.2.34. After upgrading to 2.3.2.1, fetchmail throws that error whether 'ssl_ca' is set or not. Dovecot 2.3.2.1 reports the error SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca: SSL alert number 48 in the logs when attempting the TLS handshake. The permissions on the CA bundle haven't changed and should still be readable by Dovecot. I'm running Gentoo Linux on x86_64 and mail is stored on an ext4 file system. I'm attaching my config files for both Dovecot 2.2.34 and Dovecot 2.3.2.1.

3 3

dovecot + centos 7 + internal ca + hostname change
by Matt Bryant 13 Sep '18

13 Sep '18

Not sure if this is dovecot or not but can find very little ie no info around on this ... and added the pem file into /etc/pki/ca-trust/source/anchors and run udpate-ca-trust .. all works ok .. (this is on centos 7 btw) So wanted to change the hostname away from ip-x-x-x-x to something a little bit more descriptive .. but then kaboom .. doesnt work any more and the following errors are seen. Have created and internal CA for domain and added it to Sep 13 10:42:04 ip-10-0-40-230 dovecot: master: Dovecot v2.2.33.2 (d6601f4ec) starting up for imap, pop3, lmtp, sieve (core dumps disabled) Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: 'attr->pValue != NULL' not true at attrs_build Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: 'lexer->tok.field.name && lexer->tok.field.value' not true at p11_lexer_next Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: 'attrs != NULL' not true at attrs_build Sep 13 10:42:04 ip-10-0-40-230 dovecot: message repeated 16 times: [ auth: Error: p11-kit: 'attrs != NULL' not true at attrs_build] Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: 'new_memory != NULL' not true at maybe_expand_array Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: shouldn't be reached at p11_array_push Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: shouldn't be reached at sink_object Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: 'attrs != NULL' not true at attrs_build Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: 'new_memory != NULL' not true at maybe_expand_array Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: shouldn't be reached at p11_array_push Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: shouldn't be reached at sink_object ... ... Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: 'new_memory != NULL' not true at maybe_expand_array Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: shouldn't be reached at p11_array_push Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: shouldn't be reached at sink_object Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: 'attr->pValue != NULL' not true at attrs_build Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: 'new_memory != NULL' not true at maybe_expand_array Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: shouldn't be reached at p11_array_push Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: shouldn't be reached at sink_object Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: no CKA_CLASS attribute found Sep 13 10:42:04 ip-10-0-40-230 dovecot: auth: Error: p11-kit: couldn't load file into objects: /usr/share/pki/ca-trust-source/ca-bundle.trust.p11-kit Sep 13 10:42:05 ip-10-0-40-230 dovecot: auth-worker: Error: p11-kit: 'attrs != NULL' not true at attrs_build Sep 13 10:42:05 ip-10-0-40-230 dovecot: auth-worker: Fatal: master: service(auth-worker): child 14389 killed with signal 11 (core dumps disabled) Sep 13 10:42:05 ip-10-0-40-230 dovecot: auth-worker: Error: p11-kit: 'attrs != NULL' not true at attrs_build Sep 13 10:42:05 ip-10-0-40-230 dovecot: auth-worker: Fatal: master: service(auth-worker): child 14391 killed with signal 11 (core dumps disabled) Sep 13 10:42:05 ip-10-0-40-230 dovecot: auth-worker: Error: p11-kit: 'attrs != NULL' not true at attrs_build Sep 13 10:42:05 ip-10-0-40-230 dovecot: auth-worker: Fatal: master: service(auth-worker): child 14393 killed with signal 11 (core dumps disabled) why would a hostname change make any difference here .. the certs specified in dovecot config are all complete in their chain so not sure what its trying to do ... set hostname back to original works find .. so something is obviously tied or keyed to hostname though cant find anything specific anyone seen anything like this at all ?? rgds Matt

1 1

Re: make check (pigeonhole)
by Eric Broch 12 Sep '18

12 Sep '18

Must be non root user as well On 9/12/2018 1:05 PM, Aki Tuomi wrote: > Ok. We have that fix as well but for other reasons. Good to know this. > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: Eric Broch <ebroch(a)whitehorsetc.com> > Date: 12/09/2018 21:36 (GMT+02:00) > To: Aki Tuomi <aki.tuomi(a)dovecot.fi>, dovecot(a)dovecot.org > Subject: Re: make check (pigeonhole) > > Hi Aki, > > I needed this patch > <https://src.fedoraproject.org/cgit/rpms/dovecot.git/tree/dovecot-2.3.0.1-li…> > (fedora): > > |diff -up dovecot-2.3.0.1/src/auth/mycrypt.c.libxcrypt > dovecot-2.3.0.1/src/auth/mycrypt.c --- > dovecot-2.3.0.1/src/auth/mycrypt.c.libxcrypt 2018-02-28 > 15:28:58.000000000 +0100 +++ dovecot-2.3.0.1/src/auth/mycrypt.c > 2018-03-27 10:57:38.447769201 +0200 @@ -14,6 +14,7 @@ # define _XPG6 > /* Some Solaris versions require this, some break with this */ #endif > #include <unistd.h> +#include <crypt.h> #include "mycrypt.h" All works > well now. Thanks for your help Eric | > > On 9/12/2018 12:35 AM, Aki Tuomi wrote: >> >> I tried reproducing your problem and I only can make it happen if I >> run the test suite as root, which is not supported. >> >> Aki >> >> >> On 12.09.2018 06:49, Eric Broch wrote: >>> >>> I'll give those a look and make change accordingly. >>> >>> >>> On 9/11/2018 9:26 PM, Aki Tuomi wrote: >>>> You know we have rpm packages for 2.3.2.1 available at >>>> https://repo.dovecot.org if you are in a hurry? >>>> >>>> >>>> >>>> --- >>>> Aki Tuomi >>>> Dovecot oy >>>> >>>> -------- Original message -------- >>>> From: Eric Broch <ebroch(a)whitehorsetc.com> >>>> Date: 12/09/2018 00:35 (GMT+02:00) >>>> To: dovecot(a)dovecot.org >>>> Subject: Re: make check (pigeonhole) >>>> >>>> Any news on this error on CentOS 7 ? >>>> >>>> Help! >>>> >>>> >>>> On 9/10/2018 9:32 AM, Eric Broch wrote: >>>>> >>>>> I attempted as non-root user and got the exact same error. >>>>> >>>>> >>>>> On 9/10/2018 9:10 AM, Aki Tuomi wrote: >>>>>> Have you attempted this as non-root user? >>>>>> >>>>>> >>>>>> >>>>>> --- >>>>>> Aki Tuomi >>>>>> Dovecot oy >>>>> >>>>> -- >>>>> Eric Broch >>>>> White Horse Technical Consulting (WHTC) >>>> >>>> -- >>>> Eric Broch >>>> White Horse Technical Consulting (WHTC) >>> >>> -- >>> Eric Broch >>> White Horse Technical Consulting (WHTC) >> > > -- > Eric Broch > White Horse Technical Consulting (WHTC) -- Eric Broch White Horse Technical Consulting (WHTC)

1 0

Re: make check (pigeonhole)
by Eric Broch 12 Sep '18

12 Sep '18

I'll give those a look and make change accordingly. On 9/11/2018 9:26 PM, Aki Tuomi wrote: > You know we have rpm packages for 2.3.2.1 available at > https://repo.dovecot.org if you are in a hurry? > > > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: Eric Broch <ebroch(a)whitehorsetc.com> > Date: 12/09/2018 00:35 (GMT+02:00) > To: dovecot(a)dovecot.org > Subject: Re: make check (pigeonhole) > > Any news on this error on CentOS 7 ? > > Help! > > > On 9/10/2018 9:32 AM, Eric Broch wrote: >> >> I attempted as non-root user and got the exact same error. >> >> >> On 9/10/2018 9:10 AM, Aki Tuomi wrote: >>> Have you attempted this as non-root user? >>> >>> >>> >>> --- >>> Aki Tuomi >>> Dovecot oy >> >> -- >> Eric Broch >> White Horse Technical Consulting (WHTC) > > -- > Eric Broch > White Horse Technical Consulting (WHTC) -- Eric Broch White Horse Technical Consulting (WHTC)

2 3

Adding namespace alias_for causes index resync?
by Tom Sommer 12 Sep '18

12 Sep '18

I just added a new namespace-alias with alias_for. Apparently this causes all mailbox indexes to be resynced? Is this intentional and/or is there some way to avoid this? My NFS storage pretty much kills its elf when hundreds of thausands of users needs to resync indexes :) Thanks -- Tom

1 1

Re: LMTP tcp listener with auth?
by Andreas Thienemann 12 Sep '18

12 Sep '18

Hi Stephan, On Wed, 12 Sep 2018, Stephan Bosch wrote: > LMTP currently does not support AUTH. Bummer. Thought so. > What is your use case? Most people hide LMTP behind a firewall, or don't > expose it through TCP/IP in the first place. The port is currently only available to a few trusted machines, but having auth available would have been a nice benefit in case the firewall gets misconfigured. Cyrus for example will expose the LMTP port via TCP if configured, but expects valid user credentials. The larger use-case is a migration from cyrus to dovecot. As it's not just the MDA but also the MTA installation that is being migrated I need a way to deliver mail from the old MTA to the new MDA. LMTP comes in handy. https://github.com/ixs/lmtp_proxy/blob/master/README.md#minmal-downtime-mig… has a rough invocation of the steps needed for that. As I said, LMTP auth isn't critical for this, but would have been a nice addition. cheers, Andreas

1 0

Letsencrypt certificate for repo.dovecot.org expired May 14th..
by B. Reino 12 Sep '18

12 Sep '18

Dear all, Just in case you've missed it, the certificate for repo.dovecot.org just expired yesterday. This causes errors in e.g. apt-get update. Thanks in advance for fixing it, -- B. Reino

2 6