January 2019 - dovecot

IMAP preauth and stats-writer
by Mark Hills 20 Jan '19

20 Jan '19

I use IMAP preauth; I connect with Alpine over SSH which is very useful. The last few upgrades this has become more difficult to to. Last time (moving 2.2 -> 2.3, I think) I had to put in a workaround: stats_writer_socket_path = It prevented /usr/local/libexec/dovecot/imap attempting to connect to a central stats service. As of an upgrade today (2.3.2.1_1 -> 2.3.4_3 on FreeBSD) it looks like that 'fix' stopped working, and I get: imap(mark,)Error: net_connect_unix() failed: Permission denied It goes to stderr, which breaks Alpine. "()" is actually the filename. It seems that the empty string is no longer an indication to disable it. Here it is with the default configuration: imap(mark,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied I do also have a dovecot running as a system daemon, and, interestingly, disable this and it's 'fixed'; no attempt to connect. However, disabling the service is not an option (needed for smartphone) The best I've come up with so far is when using preauth to hack it to send stderr to /dev/null. And yes, probably Alpine is at fault for interpreting stderr content (separate issue) * Is there a way to cleanly disable reporting to the stats service? Previously, running as preauth was all very clean. * Just wanted to highlight that IMAP preauth is really useful. Even though it might not be mainstream, it seems healthy to be able to easily install dovecot as an unprivilidged user in a "unixy" way. I'm on FreeBSD 11.2, with dovecot from ports. dovecot.conf below. -- Mark # # Dovecot configuration # mail_location = maildir:~/Maildir postmaster_address = postmaster namespace { inbox = yes } ssl = required ssl_cert = </etc/ssl/lets.crt ssl_key = </etc/ssl/local.key ssl_dh = </etc/ssl/dh.pem # # No writing of statistics to a priviledged service; # this is troublesome for a user to run the IMAP client # #stats_writer_socket_path = # # Authentication: only allow passwords sent over TLS, and # check against the system password database # passdb { driver = pam } userdb { driver = passwd } # # Disable all except standard IMAP with TLS logins # service imap-login { inet_listener imaps { port = 0 } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } } # # An authentication service to allow Exim to use the # same credentials as Dovecot # service auth { unix_listener auth-client { mode = 0600 user = mailnull } } protocol imap { mail_max_userip_connections = 40 } # END

4 9

BUG 'doveadm mailbox' list does not honour symlinks to (mbox) mailboxes
by Marc Roos 20 Jan '19

20 Jan '19

doveadm mailbox list does not honour symlinks to at least the mbox mailboxes I relocated temporarily. You would expect it to behave similar to the server and use the symlinks. dovecot-2.2.36-3.el7.x86_64 CentOS Linux release 7.6.1810 (Core)

1 0

Re: Feature request SCRAM-SHA-256
by Stephan Bosch 20 Jan '19

20 Jan '19

Hi, Op 13/01/2019 om 17:48 schreef Tributh via dovecot: > Hi, > sorry for my late reply. Was too busy during the week. > Thank you for your patches. I hope I will be able with them to get now > some client support for SCRAM-SHA-256. Will report how I succeed in the > future. I managed to test it successfully using MailKit: https://github.com/jstedfast/MailKit I used the IMAP demo in samples/ImapClientDemo. I limited the available mechanisms presented by the server to SCRAM-SHA-256 exclusively, so the demo needs no modification. So, from our end it seems to work fine. Regards, Stephan.

1 0

Reloading dovecot after certificate renewal?
by ＠lbutlr 17 Jan '19

17 Jan '19

Is it enough to issue a `service restart dovecot` when a certificate renews or do I need to service stop/start? -- "A thousand years ago we thought the world was a bowl. Five hundred years ago we knew it was a globe. Today we know it is flat and round carried through space on the back of a turtle. Don't you wonder what shape it will turn out to be tomorrow?" [Lord Vetinari]

1 1

Re: Authentication lost within session
by Sami Ketola 17 Jan '19

17 Jan '19

> On 17 Jan 2019, at 14.19, Marc Roos <M.Roos(a)f1-outsourcing.eu> wrote: > > > Same ip, connection, session happens after a few seconds, all on local > test network. Sorry I mistakenly overlooked at the ip:s. But anyway that is multiple connections and seems that PAM has some concurrency limit and refuses the third connection Thunderbird opens. so you probably have pam_limits with maxlogins=2 enabled. Sami

2 2

Need help deduplicating messages fetched with getmail into dovecot mailbox
by Gabriel Kaufmann 17 Jan '19

17 Jan '19

Hello everyone, I'm using getmail to fetch some external mails from POP3 into a dovecot mailbox (providing IMAP). Getmail seems to have some serious problems keeping track of mails already fetched. I have every Mail exactly twice in my Mailbox delivered by Dovecot. Of course they are not in the original external POP3 Inbox. I hope I'm in right place here. I think the problem is not dovecot related, but getmail. As there is a huge number of user complains about getmail (and obviously no alternative?!) I hope to find some help here for a workaround. I noticed getmail can be configured for some kind of filtering to sort out Mails for local delivery to dovecot. http://pyropus.ca/software/getmail/configuration.html#filter-examples Does anyone know how to query the message-id (given by mail-headers) with doveadm? My first test was this: > doveadm fetch -u my-mailbox(a)domain.net "mailbox date.sent" message-id > "<369408722.286104911.1547114312259.Foo.root(a)someone.org>" But then I noticed this is only ment to be used with dovecot internal GUID - I think... Is it possible to query for the mail message-id also? Is it posible to reconfigure indexing to include this propery from the original message-file? I made a small perl-script to extract it using Perl-Lib Email::Simple. > To: my-mailbox(a)domain.net > *Message-ID: <369408722.286104911.1547114312259.Foo.root(a)someone.org>* > Subject: Some test I also already tried the doveadm deduplicate, but this also doesn't work as it is also basing on the internal GUID of dovecot and every duplicate-message seems to have a new, unique GUID. > doveadm deduplicate -u gabriel.kaufmann(a)gmx.net mailbox-guid May be anyone know a better solution. -- Best regards Gabriel Kaufmann

2 4

Accidentally deleted .imap folder, how can I reindex?
by Les 17 Jan '19

17 Jan '19

Hello, I have several users with Maildir format. After upgrading from 2.2 to 2.3, some directories could not be opened from thunderbird. The server log shows all kinds of errors including these: Corrupted record in index cache file /home/user01/Maildir/.Elk&APw-ld&APY-tt/dovecot.index.cache: UID 2802: Broken physical size in mailbox Cached message size larger than expected (18 > 17, box=Sent, UID=2802) While trying to fix the problem, I have accidentally deleted some of the index/cache directories. In particular, Maildir/.Sent/.imap and Maildir/.Sent/cur/.imap folders were deleted, but all of the mail message files are there. When I try to open the "Sent" folder from thunderbird, it does not show all of the messages. It only shows messages until 2019.01.12. If I send out an email from thunderbird, then a new message is copied into the Maildir/.Sent/ folder, but it is not visible from thunderbird. I have already tried to unsubscribe + subscribe to the Sent folder, but it did not work - it re-downloads all messages until 2019.01.12, but not after that. Can I somehow regenerate the index files, force dovecot to recreated .imap foder, and force thunderbird to re-download and show all of the files somehow? Thank you, Laszlo

2 2

Authentication lost within session
by Marc Roos 17 Jan '19

17 Jan '19

When I open the mail client (thunderbird), I can access the mailboxes and all of a sudden, I am loosing the authenticated session. Any idea's where to look? Jan 17 12:42:04 mail04 dovecot: imap-login: Login: user=<usertest>, method=PLAIN, rip=192.168.10.219, lip=192.168.10.44, mpid=13403, TLS, session=<NsYo4qV/CNfAqArb> Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Loading modules from directory: /usr/lib64/dovecot Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Module loaded: /usr/lib64/dovecot/lib20_listescape_plugin.so Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Effective uid=8267, gid=231, home=/home/popusers/usertest Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=/var/spool/mail/usertest:CONTROL=~/mail/contr ol:INDEX=/var/dovecot/usertest/index:LAYOUT=maildir++ Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: maildir++: root=/home/popusers/usertest/mail, index=/var/dovecot/usertest/index, indexpvt=, control=/home/popusers/usertest/mail/control, inbox=/var/spool/mail/usertest, alt= Jan 17 12:42:04 mail04 dovecot: imap-login: Login: user=<usertest>, method=PLAIN, rip=192.168.10.219, lip=192.168.10.44, mpid=13404, TLS, session=<yRkp4qV/CtfAqArb> Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Loading modules from directory: /usr/lib64/dovecot Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Module loaded: /usr/lib64/dovecot/lib20_listescape_plugin.so Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: INBOX: Mailbox opened because: SELECT Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Effective uid=8267, gid=231, home=/home/popusers/usertest Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=/var/spool/mail/usertest:CONTROL=~/mail/contr ol:INDEX=/var/dovecot/usertest/index:LAYOUT=maildir++ Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: maildir++: root=/home/popusers/usertest/mail, index=/var/dovecot/usertest/index, indexpvt=, control=/home/popusers/usertest/mail/control, inbox=/var/spool/mail/usertest, alt= Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: INBOX/test: Mailbox opened because: SELECT Jan 17 12:42:12 mail04 dovecot: imap-login: Login: user=<usertest>, method=PLAIN, rip=192.168.10.219, lip=192.168.10.44, mpid=13405, TLS, session=<MOKk4qV/C9fAqArb> Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: Loading modules from directory: /usr/lib64/dovecot Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: Module loaded: /usr/lib64/dovecot/lib20_listescape_plugin.so Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: Effective uid=8267, gid=231, home=/home/popusers/usertest Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=/var/spool/mail/usertest:CONTROL=~/mail/contr ol:INDEX=/var/dovecot/usertest/index:LAYOUT=maildir++ Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: maildir++: root=/home/popusers/usertest/mail, index=/var/dovecot/usertest/index, indexpvt=, control=/home/popusers/usertest/mail/control, inbox=/var/spool/mail/usertest, alt= Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: Drafts: Mailbox opened because: SELECT Jan 17 12:42:14 mail04 dovecot: auth-worker(13400): pam(usertest,192.168.10.219,<fLyv4qV/DNfAqArb>): pam_authenticate() failed: Authentication failure (password mismatch?) Jan 17 12:42:22 mail04 dovecot: auth-worker(13400): pam(usertest,192.168.10.219,<fLyv4qV/DNfAqArb>): pam_authenticate() failed: Authentication failure (password mismatch?) Jan 17 12:42:25 mail04 dovecot: auth-worker(13400): pam(usertest,192.168.10.219,<slYR46V/DdfAqArb>): pam_authenticate() failed: Authentication failure (password mismatch?)

2 1

Dovecot + Weakforced Policy server
by alberto bersol 16 Jan '19

16 Jan '19

Hi, I'm trying to set Weakforced with Dovecot and I cannot log in policy server. This is the config: /root/weakforced/wforce/wforce.conf ----------------------------------- ... webserver("0.0.0.0:8084", "super") ... /etc/dovecot/conf.d/95-policy.conf ---------------------------------- auth_policy_server_url = http://localhost:8084/ #auth_policy_hash_nonce = wforce:super auth_policy_hash_nonce = {SHA256-CRYPT}$5$Ue5UrToV.Bam02bQ$Bi9OJ62Mkgc20L2HnLVmD2OCHyXaKje6Hh7qNjnOkB9 I'm following the instructions of Dovecot's wiki: https://wiki.dovecot.org/Authentication/Policy ... "To generate the hash, you concatenate nonce, login name, nil byte, password and run it through the hash algorithm once. The hash is truncated when truncation is set to non-zero. The hash is truncated by first choosing bits from MSB to byte boundary (rounding up), then right-shifting the remainding bits. hash = H(nonce||user||'\x00'||password) bytes = round8(bits*8) hash = HEX(hash[0:bytes] >> (bytes-bits*8)) And I set hash with password (super) in this way: vm-weakforced:~# doveadm pw -p noncewforce\x00super -s SHA256-CRYPT {SHA256-CRYPT}$5$ZWIX2dnU7NJvGHgC$hYFbeCCaHYZv0yPP80GHygxQMPmI5BjMx2ttRe9zti2 But if I log in Dovecot Server: vm-weakforced:~# doveadm auth login usuario Password: passdb: usuario auth succeeded extra fields: user=usuario userdb extra fields: usuario system_groups_user=usuario uid=1000 gid=1000 home=/home/usuario Answer of Weakforced is always "...authentication failed": WforceWebserver: HTTP Request "/" from 127.0.0.1:39720: Web Authentication failed And Dovecot logs don't show anything else: ... Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: auth client connected (pid=967) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=doveadm#011resp=dXN1YXJpbwB1c3VhcmlvAHVzdWFyaW8= (previous base64 data may contain sensitive data) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: policy(usuario): Policy request http://localhost:8084/?command=allow Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: policy(usuario): Policy server request JSON: {"device_id":"","login":"usuario","protocol":"doveadm","pwhash":"0a00","remote":"","tls":false} Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: queue http://localhost:8084: Set request timeout to 2019-01-15 16:50:52.236 (now: 2019-01-15 16:50:50.236) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client: peer 127.0.0.1:8084 (shared): Peer created Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client: peer 127.0.0.1:8084: Peer pool created Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: Peer created Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: queue http://localhost:8084: Setting up connection to 127.0.0.1:8084 (1 requests pending) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: Linked queue http://localhost:8084 (1 queues linked) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: queue http://localhost:8084: Started new connection to 127.0.0.1:8084 Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: request [Req4: POST http://localhost:8084/?command=allow] Submitted (requests left=1) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: Creating 1 new connections to handle requests (already 0 usable, connecting to 0, closing 0) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: Making new connection 1 of 1 (0 connections exist, 0 pending) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: (127.0.0.1:8084): Connecting Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: (127.0.0.1:8084): Waiting for connect (fd=20) to finish for max 0 msecs Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: HTTP connection created (1 parallel connections exist) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: (127.0.0.1:8084): Client connected (fd=20) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: Connected Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: Ready for requests Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: Successfully connected (1 connections exist, 0 pending) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client: peer 127.0.0.1:8084: Successfully connected (1 connections exist, 0 pending) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: Using 1 idle connections to handle 1 requests (1 total connections ready) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: queue http://localhost:8084: Connection to peer 127.0.0.1:8084 claimed request [Req4: POST http://localhost:8084/?command=allow] Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: Claimed request [Req4: POST http://localhost:8084/?command=allow] Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: request [Req4: POST http://localhost:8084/?command=allow] Sent header Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: request [Req4: POST http://localhost:8084/?command=allow] Send more (sent 95, buffered=303) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: request [Req4: POST http://localhost:8084/?command=allow] Finished sending payload Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: No more requests to service for this peer (1 connections exist, 0 pending) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: Got 401 response for request [Req4: POST http://localhost:8084/?command=allow] (took 4 ms + 3 ms in queue) Jan 15 16:50:50 vm-weakforced dovecot: auth: Error: policy(usuario): Policy server HTTP error: 401 Unauthorized Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: Response payload stream destroyed (0 ms after initial response) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: request [Req4: POST http://localhost:8084/?command=allow] Finished Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: queue http://localhost:8084: Dropping request [Req4: POST http://localhost:8084/?command=allow] Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: request [Req4: POST http://localhost:8084/?command=allow] Free (requests left=1) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: No requests to service for this peer (1 connections exist, 0 pending) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: No more requests queued; going idle (timeout = 10000 msecs) ... Any idea? Thank you so much Regards,

2 4

doveadm neglecting to exit in failure?
by Felipe Gasper 16 Jan '19

16 Jan '19

----- cpssltest(a)cpanelssltest.org [/usr/local/cpanel]# doveadm -v expunge -u cpssltest -- mailbox-guid b8a359119e771b58484a0000a841250d savedbefore 365days; echo $? doveadm(cpssltest): Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied 0 ----- ^^ In the above, shouldn’t the “doveadm” command have exited nonzero to indicate a failure to connect? Thanks! -Felipe Gasper Mississauga, Ontario

3 2