- dovecot - dovecot.org

Authentication/Penalty disabled (socket mode=0) introduces constant 5 sec delays (2.27 on debian 9)
by Ludovic Pouzenc 20 Jan '19

20 Jan '19

Hi, I hit a bizare problem with dovecot 2.2.7 on debian 9 with LMTP enabled and auth/penalty disabled as documented here : https://wiki.dovecot.org/Authentication/Penalty Use case : I run a swaks command to send an email to an exim4 that tries to make a callout to dovecot-lmtp. At RCPT TO: swaks hangs 5.0<something-small> seconds then process normally (exim is waiting for callout completion). with strace, I see 5 second delay with many tries to socket connection. (PID 9652 was an auth process freshly forked) [pid 9652] socket(AF_UNIX, SOCK_STREAM, 0) = 14 [pid 9652] fcntl(14, F_GETFL) = 0x2 (flags O_RDWR) [pid 9652] fcntl(14, F_SETFL, O_RDWR|O_NONBLOCK) = 0 [pid 9652] connect(14, {sa_family=AF_UNIX, sun_path="anvil-auth-penalty"}, 110) = -1 ECONNREFUSED (Connection refused) [pid 9652] close(14) = 0 [pid 9652] nanosleep({tv_sec=0, tv_nsec=20000000}, NULL) = 0 [pid 9652] socket(AF_UNIX, SOCK_STREAM, 0) = 14 [pid 9652] fcntl(14, F_GETFL) = 0x2 (flags O_RDWR) [pid 9652] fcntl(14, F_SETFL, O_RDWR|O_NONBLOCK) = 0 [pid 9652] connect(14, {sa_family=AF_UNIX, sun_path="anvil-auth-penalty"}, 110) = -1 ECONNREFUSED (Connection refused) [pid 9652] close(14) = 0 [pid 9652] nanosleep({tv_sec=0, tv_nsec=50000000}, NULL) = 0 [pid 9652] socket(AF_UNIX, SOCK_STREAM, 0) = 14 [pid 9652] fcntl(14, F_GETFL) = 0x2 (flags O_RDWR) [pid 9652] fcntl(14, F_SETFL, O_RDWR|O_NONBLOCK) = 0 [pid 9652] connect(14, {sa_family=AF_UNIX, sun_path="anvil-auth-penalty"}, 110) = -1 ECONNREFUSED (Connection refused) [pid 9652] close(14) = 0 [pid 9652] nanosleep({tv_sec=0, tv_nsec=90000000}, NULL) = 0 with ddd I get to: src/lib-master/anvil-client.c: int anvil_client_connect(struct anvil_client *client, bool retry) { //[...] fd = retry ? net_connect_unix_with_retries(client->path, 5000) : net_connect_unix(client->path); //[...] and retry is forced to TRUE by the caller and net_connect_unix_with_retries retries in case of ECONNREFUSED. How I get into it : ssh -X root@telegraphe-test apt install dovecot-dbg cd /dev/shm apt source dovecot cd dovecot-2.2.27 pidof dovecot ddd /usr/sbin/dovecot set follow-fork-mode child attach 11833 cont # Send an e-mail from swaks to exim that use transport=smtp in LMTP mode to dovecot-lmtp, press interrupt while exim is blocking after RCPT TO: (gdb) bt #0 0x00007f81528af270 in __nanosleep_nocancel () at ../sysdeps/unix/syscall-template.S:84 #1 0x00007f81528d8b84 in usleep (useconds=<optimized out>) at ../sysdeps/posix/usleep.c:32 #2 0x00007f815329368e in net_connect_unix_with_retries (path=0x562dc200ce10 <error: Cannot access memory at address 0x562dc200ce10>, msecs=msecs@entry=5000) at net.c:347 #3 0x00007f815320ce0d in anvil_client_connect (client=0x562dc200cda0, retry=retry@entry=true) at anvil-client.c:136 #4 0x0000562dc1429f5d in auth_penalty_init (path=0x562dc1451846 <error: Cannot access memory at address 0x562dc1451846>) at auth-penalty.c:37 #5 0x0000562dc1424166 in main_preinit () at main.c:202 #6 main (argc=<optimized out>, argv=<optimized out>) at main.c:396 (gdb) bt full #1 0x00007f81528d8b84 in usleep (useconds=<optimized out>) at ../sysdeps/posix/usleep.c:32 ts = {tv_sec = 0, tv_nsec = 80000000} #2 0x00007f815329368e in net_connect_unix_with_retries (path=0x562dc200ce10 <error: Cannot access memory at address 0x562dc200ce10>, msecs=msecs@entry=5000) at net.c:347 start = {tv_sec = 1545319486, tv_usec = 241864} now = {tv_sec = 1545319490, tv_usec = 198061} fd = -1 #3 0x00007f815320ce0d in anvil_client_connect (client=0x562dc200cda0, retry=retry@entry=true) at anvil-client.c:136 fd = <optimized out> __FUNCTION__ = <error reading variable __FUNCTION__ (Cannot access memory at address 0x7f81532a8260)> #4 0x0000562dc1429f5d in auth_penalty_init (path=0x562dc1451846 <error: Cannot access memory at address 0x562dc1451846>) at auth-penalty.c:37 No locals. #5 0x0000562dc1424166 in main_preinit () at main.c:202 mod_set = <error reading variable mod_set (Cannot access memory at address 0x7ffda885db80)> #6 main (argc=<optimized out>, argv=<optimized out>) at main.c:396 c = <optimized out> Backtrace stopped: Cannot access memory at address 0x7ffda885dbf8 (gdb) Code I've read : src/auth/main.c: static void main_preinit(void) if (!worker) auth_penalty = auth_penalty_init(AUTH_PENALTY_ANVIL_PATH); auth_request_stats_init(); src/auth/auth-penalty.c: struct auth_penalty *auth_penalty_init(const char *path) if (anvil_client_connect(penalty->client, TRUE) < 0) penalty->disabled = TRUE; else { src/lib-master/anvil-client.c: int anvil_client_connect(struct anvil_client *client, bool retry) fd = retry ? net_connect_unix_with_retries(client->path, 5000) : net_connect_unix(client->path); src/lib/net.c : int net_connect_unix_with_retries(const char *path, unsigned int msecs) { struct timeval start, now; int fd; if (gettimeofday(&start, NULL) < 0) i_panic("gettimeofday() failed: %m"); do { fd = net_connect_unix(path); if (fd != -1 || (errno != EAGAIN && errno != ECONNREFUSED)) // <------------------ HERE, I go into ECONNREFUSED case during 5 seconds break; /* busy. wait for a while. */ usleep(((rand() % 10) + 1) * 10000); if (gettimeofday(&now, NULL) < 0) i_panic("gettimeofday() failed: %m"); } while (timeval_diff_msecs(&now, &start) < (int)msecs); return fd; } Additionnal environment infos (mailboxes are NFS-backed) : lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 9.2 (stretch) Release: 9.2 Codename: stretch mount | grep ^/ /dev/sda1 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered) mount | grep nfs nfs1:/mail/maildir on /var/maildir type nfs (rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=172.16.2.28,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=172.16.2.28) nfs1:/mail on /var/mail type nfs (rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=172.16.2.28,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=172.16.2.28) cifs1:/homepermanents/info/lpouzenc on /home/lpouzenc type nfs (rw,nosuid,nodev,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=172.16.2.29,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=172.16.2.29) telegraphe-test:~# apt-cache policy dovecot-core dovecot-core: Installé : 1:2.2.27-3+deb9u2 Candidat : 1:2.2.27-3+deb9u2 Table de version : *** 1:2.2.27-3+deb9u2 500 500 http://miroir/debian stretch/main amd64 Packages 500 http://security.debian.org/debian-security stretch/updates/main amd64 Packages 100 /var/lib/dpkg/status telegraphe-test:~# dovecot -n # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) doveconf: Warning: service auth { client_limit=10000 } is lower than required under max. load (10240) # OS: Linux 4.9.0-4-amd64 x86_64 Debian 9.2 nfs auth_verbose = yes default_client_limit = 10000 default_process_limit = 2048 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_debug = yes mail_fsync = always mail_location = maildir:/var/mail/maildir/%u managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { driver = pam } plugin { sieve = file:/var/mail/mailconf/%1u/%u/sieve;active=/var/mail/mailconf/%1u/%u/.dovecot.sieve sieve_before = /var/mail/mailconf/global/sieve/before.d sieve_global = /var/mail/mailconf/global/sieve sieve_vacation_dont_check_recipient = yes } protocols = " imap lmtp sieve sieve" service anvil { unix_listener anvil-auth-penalty { mode = 00 } } service lmtp { inet_listener lmtp { port = 24 } } service managesieve { process_limit = 16 } ssl_cert = </etc/ssl/certs/telegraphe7_cert.pem ssl_key = # hidden, use -P to show it syslog_facility = local0 userdb { driver = passwd } verbose_proctitle = yes protocol lmtp { auth_username_format = %n mail_plugins = " sieve" } protocol imap { mail_max_userip_connections = 20 } Could you see a config tuning that keeps auth-penalty out of the way (it seems to behaves badly here, because we have perdition IMAP proxy in front, and it keeps track only of his IP and adds 2/4/8/15s delays to "random" legit clients. Could it be saved with a dedicated config entry to disable auth-penalty ? Is there solution to have auth-penalty working while having perdition IMAP proxy ? Regards, Ludovic Pouzenc -- Ludovic POUZENC - Ingénieur Système IMT Mines Albi (http://www.imt-mines-albi.fr) 81013 ALBI CT Cedex 09 - Tel : 05 63 49 33 56

2 3

panic when using dovecot master account
by André Rodier 20 Jan '19

20 Jan '19

Hello, I am running dovecot 2.2.34 (874deae), on Debian stable, from backports. I just tried the master account, and although everything worked in the email client, I had logs in the error logs: > imap(mirina): Panic: file mail-index-sync.c: line 413 > (mail_index_sync_begin_to2): assertion failed: (!index->syncing) > Nov 08 22:06:24 osaka dovecot[1450]: imap(mirina): Error: Raw > backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x9e412) [0x7f20ae813412] > -> /usr/lib/dovecot/libdovecot.so.0(+0x9e50d) [0x7f20ae81350d] -> > /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f20ae7a2c51] -> > /usr/lib/dovecot/libdovecot-storage.so.0(+0xe0fe4) [0x7f20aeb88fe4] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin_to+0x4f) > [0x7f20aeb890bf] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin+0x1c) > [0x7f20aeb8915c] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x218) > [0x7f20ad51f308] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) > [0x7f20aeaf02d4] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) > [0x7f20aeaf0387] -> > /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x31) > [0x7f20aeb6bbf1] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x936d) > [0x7f20ad51c36d] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8771) > [0x7f20aeb50771] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xfd46) [0x7f20adb8bd46] > -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x31) > [0x7f20aeaf0781] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x10b8) > [0x7f20ad5201a8] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) > [0x7f20aeaf02d4] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) > [0x7f20aeaf0387] -> > /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x31) > [0x7f20aeb6bbf1] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x936d) > [0x7f20ad51c36d] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8771) > [0x7f20aeb50771] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xfd46) [0x7f20adb8bd46] > -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x31) > [0x7f20aeaf0781] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x10b8) > [0x7f20ad5201a8] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) > [0x7f20aeaf02d4] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) > [0x7f20aeaf0387] -> dovecot/imap(cmd_select_full+0x16b) > [0x557de970cc3b] -> dovecot/imap(command_exec+0x5c) [0x557de971444c] -> > dovecot/imap(+0x1a912) [0x557de9712912] > Nov 08 22:06:24 osaka dovecot[1450]: imap(mirina): Fatal: master: > service(imap): child 4289 killed with signal 6 (core dumps disabled) I attach my dovecot configuration too. Thanks for your help.

3 6

Moving messages on test server, dovecot.index.log was locked for x seconds on cephfs
by Marc Roos 20 Jan '19

20 Jan '19

I am getting on a test environment were no users are logging in a dovecot.index.log was locked when moving messages with doveadm move -u testuser Archive/2012 mailbox INBOX/inbox BEFORE 2013-01-01. doveadm(testuser): Warning: Transaction log file /var/dovecot/testuser/index/.INBOX.inbox/dovecot.index.log was locked for 215 seconds (rotating while syncing) I have read about how this can be related to default locking fcntl and nfs, anyone having this solved with cephfs? https://www.dovecot.org/list/dovecot/2009-January/036194.html

1 0

Dovecot 2.3 no longer accepts ssl_key_password
by Chris Kiakas 20 Jan '19

20 Jan '19

Hit a little problem when I upgraded a system from FreeBSD 10.3 to 11.2. I did not receive any errors in the upgrade. The system is running 4 jails and everything seems to work except in Dovecot dovecot-2.3.4_5 where when using the exact same configuration which worked in 10.3 with the same password protected certificate key. (doveconf -n -P shows the correct password.) ssl_ca = </usr/local/etc/site.keys/name_com.ca-bundle ssl_cert = </usr/local/etc/site.keys/name_com.crt ssl_dh = </usr/local/etc/dovecot/dh.pem ssl_key = </usr/local/etc/site.keys/name.com.key ssl_key_password = keypassword The password works with openssl. Changing the password on the key has no effect. Removing the password on the cert with openssl and running dovecot with the new key works. I installed on another system and I am experiencing the same results. The issue persists whether I install dovecot from ports or pkg. I can't see where the problem is. It seems that Dovecot is unable to read the key when password protected even though it has the correct password. Has anyone experienced this? Chris

3 2

Re: Feature request SCRAM-SHA-256
by Stephan Bosch 20 Jan '19

20 Jan '19

Op 16/12/2018 om 10:06 schreef Tributh via dovecot: > > Am 16.12.18 um 09:42 schrieb Aki Tuomi: >>> On 16 December 2018 at 10:27 Tributh via dovecot <dovecot(a)dovecot.org> wrote: >>> >>> >>> Hi, >>> is that here the right place to make feature requests? >>> >>> dovecot supports as authentication mechanism >>> SCRAM-SHA-1 from RFC 5802 >>> which was updated to >>> SCRAM-SHA-256 in RFC 7677 >>> >>> Can SCRAM-SHA-256 be added to the authentication mechanisms? >>> >>> I would not like to request, that SCRAM-SHA-1 will be exchanged by >>> SCRAM-SHA-256, since several applications only support SCRAM-SHA-1 >>> >>> Regards >>> >>> Torsten >> Hi! >> >> Adding this is possible, it can even be done as a separate plugin. But I have to ask, why? Do you actually have clients that support this? >> >> Aki >> > Hi Aki, > let me first answer the second question. > Sadly I have no client which supports it, yet. > Here we have a chicken or the egg causality dilemma. > There was some communication with mail-client developers which stated > that they would start developing it, when they have a publicly usable > server to test against. > Now I hope that the most common IMAP server could be the one, which > gives this possibility. > Sadly, most communication is not publicly available. > > In the past CRAM-MD5 was very popular. When the insecurity came out, > everything just shifted to TLS, but that prevented not from sending a > plain password now. If a malicious actor is able to change DNS/TLS > endpoints, he will receive the plain passwords immediately. > I am not the expert in explaining how such an actor could do this. I > just wanted to have possibilities for everybody to prevent this possible > exposure of a plain password, which could than easily used abusively. > > I just hope for better security in the future. I looked a this a bit and since it is basically only a matter of replacing the hash algorithm, I created a quick implementation (after some refactoring): https://github.com/stephanbosch/dovecot-core/commits/auth-scram-sha-256 However, since there is no client that actually supports this, I cannot test this myself. I've briefly tested that the old SHA-1 still works (using mpop) and that the server properly announces the new mechanism when enabled, but that is it. It is based on the master branch. Configuration is identical to SCRAM-SHA-1, apart from the mechanism (and password scheme) name of course. Don't expect this to be released or even merged to the master branch any time soon: this is likely currently very low on our priority list. But, at least you can run your own server with SCRAM-SHA-256 support (and so can client developers). Maybe if this gets endorsed and supported by clients and gets some testing in the field, we can speed it along a bit, but that is not something I can promise. So, I hatched a chick for you. I hope you can make it lay a few eggs in the future... Regards, Stephan.

2 2

IMAP preauth and stats-writer
by Mark Hills 20 Jan '19

20 Jan '19

I use IMAP preauth; I connect with Alpine over SSH which is very useful. The last few upgrades this has become more difficult to to. Last time (moving 2.2 -> 2.3, I think) I had to put in a workaround: stats_writer_socket_path = It prevented /usr/local/libexec/dovecot/imap attempting to connect to a central stats service. As of an upgrade today (2.3.2.1_1 -> 2.3.4_3 on FreeBSD) it looks like that 'fix' stopped working, and I get: imap(mark,)Error: net_connect_unix() failed: Permission denied It goes to stderr, which breaks Alpine. "()" is actually the filename. It seems that the empty string is no longer an indication to disable it. Here it is with the default configuration: imap(mark,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied I do also have a dovecot running as a system daemon, and, interestingly, disable this and it's 'fixed'; no attempt to connect. However, disabling the service is not an option (needed for smartphone) The best I've come up with so far is when using preauth to hack it to send stderr to /dev/null. And yes, probably Alpine is at fault for interpreting stderr content (separate issue) * Is there a way to cleanly disable reporting to the stats service? Previously, running as preauth was all very clean. * Just wanted to highlight that IMAP preauth is really useful. Even though it might not be mainstream, it seems healthy to be able to easily install dovecot as an unprivilidged user in a "unixy" way. I'm on FreeBSD 11.2, with dovecot from ports. dovecot.conf below. -- Mark # # Dovecot configuration # mail_location = maildir:~/Maildir postmaster_address = postmaster namespace { inbox = yes } ssl = required ssl_cert = </etc/ssl/lets.crt ssl_key = </etc/ssl/local.key ssl_dh = </etc/ssl/dh.pem # # No writing of statistics to a priviledged service; # this is troublesome for a user to run the IMAP client # #stats_writer_socket_path = # # Authentication: only allow passwords sent over TLS, and # check against the system password database # passdb { driver = pam } userdb { driver = passwd } # # Disable all except standard IMAP with TLS logins # service imap-login { inet_listener imaps { port = 0 } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } } # # An authentication service to allow Exim to use the # same credentials as Dovecot # service auth { unix_listener auth-client { mode = 0600 user = mailnull } } protocol imap { mail_max_userip_connections = 40 } # END

4 9

BUG 'doveadm mailbox' list does not honour symlinks to (mbox) mailboxes
by Marc Roos 20 Jan '19

20 Jan '19

doveadm mailbox list does not honour symlinks to at least the mbox mailboxes I relocated temporarily. You would expect it to behave similar to the server and use the symlinks. dovecot-2.2.36-3.el7.x86_64 CentOS Linux release 7.6.1810 (Core)

1 0

Re: Feature request SCRAM-SHA-256
by Stephan Bosch 19 Jan '19

19 Jan '19

Hi, Op 13/01/2019 om 17:48 schreef Tributh via dovecot: > Hi, > sorry for my late reply. Was too busy during the week. > Thank you for your patches. I hope I will be able with them to get now > some client support for SCRAM-SHA-256. Will report how I succeed in the > future. I managed to test it successfully using MailKit: https://github.com/jstedfast/MailKit I used the IMAP demo in samples/ImapClientDemo. I limited the available mechanisms presented by the server to SCRAM-SHA-256 exclusively, so the demo needs no modification. So, from our end it seems to work fine. Regards, Stephan.

1 0

Reloading dovecot after certificate renewal?
by ＠lbutlr 17 Jan '19

17 Jan '19

Is it enough to issue a `service restart dovecot` when a certificate renews or do I need to service stop/start? -- "A thousand years ago we thought the world was a bowl. Five hundred years ago we knew it was a globe. Today we know it is flat and round carried through space on the back of a turtle. Don't you wonder what shape it will turn out to be tomorrow?" [Lord Vetinari]

1 1

Re: Authentication lost within session
by Sami Ketola 17 Jan '19

17 Jan '19

> On 17 Jan 2019, at 14.19, Marc Roos <M.Roos(a)f1-outsourcing.eu> wrote: > > > Same ip, connection, session happens after a few seconds, all on local > test network. Sorry I mistakenly overlooked at the ip:s. But anyway that is multiple connections and seems that PAM has some concurrency limit and refuses the third connection Thunderbird opens. so you probably have pam_limits with maxlogins=2 enabled. Sami

2 2