January 2019 - dovecot

Reloading dovecot after certificate renewal?
by ＠lbutlr 17 Jan '19

17 Jan '19

Is it enough to issue a `service restart dovecot` when a certificate renews or do I need to service stop/start? -- "A thousand years ago we thought the world was a bowl. Five hundred years ago we knew it was a globe. Today we know it is flat and round carried through space on the back of a turtle. Don't you wonder what shape it will turn out to be tomorrow?" [Lord Vetinari]

1 1

Re: Authentication lost within session
by Sami Ketola 17 Jan '19

17 Jan '19

> On 17 Jan 2019, at 14.19, Marc Roos <M.Roos(a)f1-outsourcing.eu> wrote: > > > Same ip, connection, session happens after a few seconds, all on local > test network. Sorry I mistakenly overlooked at the ip:s. But anyway that is multiple connections and seems that PAM has some concurrency limit and refuses the third connection Thunderbird opens. so you probably have pam_limits with maxlogins=2 enabled. Sami

2 2

Need help deduplicating messages fetched with getmail into dovecot mailbox
by Gabriel Kaufmann 17 Jan '19

17 Jan '19

Hello everyone, I'm using getmail to fetch some external mails from POP3 into a dovecot mailbox (providing IMAP). Getmail seems to have some serious problems keeping track of mails already fetched. I have every Mail exactly twice in my Mailbox delivered by Dovecot. Of course they are not in the original external POP3 Inbox. I hope I'm in right place here. I think the problem is not dovecot related, but getmail. As there is a huge number of user complains about getmail (and obviously no alternative?!) I hope to find some help here for a workaround. I noticed getmail can be configured for some kind of filtering to sort out Mails for local delivery to dovecot. http://pyropus.ca/software/getmail/configuration.html#filter-examples Does anyone know how to query the message-id (given by mail-headers) with doveadm? My first test was this: > doveadm fetch -u my-mailbox(a)domain.net "mailbox date.sent" message-id > "<369408722.286104911.1547114312259.Foo.root(a)someone.org>" But then I noticed this is only ment to be used with dovecot internal GUID - I think... Is it possible to query for the mail message-id also? Is it posible to reconfigure indexing to include this propery from the original message-file? I made a small perl-script to extract it using Perl-Lib Email::Simple. > To: my-mailbox(a)domain.net > *Message-ID: <369408722.286104911.1547114312259.Foo.root(a)someone.org>* > Subject: Some test I also already tried the doveadm deduplicate, but this also doesn't work as it is also basing on the internal GUID of dovecot and every duplicate-message seems to have a new, unique GUID. > doveadm deduplicate -u gabriel.kaufmann(a)gmx.net mailbox-guid May be anyone know a better solution. -- Best regards Gabriel Kaufmann

2 4

Accidentally deleted .imap folder, how can I reindex?
by Les 17 Jan '19

17 Jan '19

Hello, I have several users with Maildir format. After upgrading from 2.2 to 2.3, some directories could not be opened from thunderbird. The server log shows all kinds of errors including these: Corrupted record in index cache file /home/user01/Maildir/.Elk&APw-ld&APY-tt/dovecot.index.cache: UID 2802: Broken physical size in mailbox Cached message size larger than expected (18 > 17, box=Sent, UID=2802) While trying to fix the problem, I have accidentally deleted some of the index/cache directories. In particular, Maildir/.Sent/.imap and Maildir/.Sent/cur/.imap folders were deleted, but all of the mail message files are there. When I try to open the "Sent" folder from thunderbird, it does not show all of the messages. It only shows messages until 2019.01.12. If I send out an email from thunderbird, then a new message is copied into the Maildir/.Sent/ folder, but it is not visible from thunderbird. I have already tried to unsubscribe + subscribe to the Sent folder, but it did not work - it re-downloads all messages until 2019.01.12, but not after that. Can I somehow regenerate the index files, force dovecot to recreated .imap foder, and force thunderbird to re-download and show all of the files somehow? Thank you, Laszlo

2 2

Authentication lost within session
by Marc Roos 17 Jan '19

17 Jan '19

When I open the mail client (thunderbird), I can access the mailboxes and all of a sudden, I am loosing the authenticated session. Any idea's where to look? Jan 17 12:42:04 mail04 dovecot: imap-login: Login: user=<usertest>, method=PLAIN, rip=192.168.10.219, lip=192.168.10.44, mpid=13403, TLS, session=<NsYo4qV/CNfAqArb> Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Loading modules from directory: /usr/lib64/dovecot Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Module loaded: /usr/lib64/dovecot/lib20_listescape_plugin.so Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Effective uid=8267, gid=231, home=/home/popusers/usertest Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=/var/spool/mail/usertest:CONTROL=~/mail/contr ol:INDEX=/var/dovecot/usertest/index:LAYOUT=maildir++ Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: maildir++: root=/home/popusers/usertest/mail, index=/var/dovecot/usertest/index, indexpvt=, control=/home/popusers/usertest/mail/control, inbox=/var/spool/mail/usertest, alt= Jan 17 12:42:04 mail04 dovecot: imap-login: Login: user=<usertest>, method=PLAIN, rip=192.168.10.219, lip=192.168.10.44, mpid=13404, TLS, session=<yRkp4qV/CtfAqArb> Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Loading modules from directory: /usr/lib64/dovecot Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Module loaded: /usr/lib64/dovecot/lib20_listescape_plugin.so Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: INBOX: Mailbox opened because: SELECT Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Effective uid=8267, gid=231, home=/home/popusers/usertest Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=/var/spool/mail/usertest:CONTROL=~/mail/contr ol:INDEX=/var/dovecot/usertest/index:LAYOUT=maildir++ Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: maildir++: root=/home/popusers/usertest/mail, index=/var/dovecot/usertest/index, indexpvt=, control=/home/popusers/usertest/mail/control, inbox=/var/spool/mail/usertest, alt= Jan 17 12:42:04 mail04 dovecot: imap(usertest): Debug: INBOX/test: Mailbox opened because: SELECT Jan 17 12:42:12 mail04 dovecot: imap-login: Login: user=<usertest>, method=PLAIN, rip=192.168.10.219, lip=192.168.10.44, mpid=13405, TLS, session=<MOKk4qV/C9fAqArb> Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: Loading modules from directory: /usr/lib64/dovecot Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: Module loaded: /usr/lib64/dovecot/lib20_listescape_plugin.so Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: Effective uid=8267, gid=231, home=/home/popusers/usertest Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=/var/spool/mail/usertest:CONTROL=~/mail/contr ol:INDEX=/var/dovecot/usertest/index:LAYOUT=maildir++ Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: maildir++: root=/home/popusers/usertest/mail, index=/var/dovecot/usertest/index, indexpvt=, control=/home/popusers/usertest/mail/control, inbox=/var/spool/mail/usertest, alt= Jan 17 12:42:12 mail04 dovecot: imap(usertest): Debug: Drafts: Mailbox opened because: SELECT Jan 17 12:42:14 mail04 dovecot: auth-worker(13400): pam(usertest,192.168.10.219,<fLyv4qV/DNfAqArb>): pam_authenticate() failed: Authentication failure (password mismatch?) Jan 17 12:42:22 mail04 dovecot: auth-worker(13400): pam(usertest,192.168.10.219,<fLyv4qV/DNfAqArb>): pam_authenticate() failed: Authentication failure (password mismatch?) Jan 17 12:42:25 mail04 dovecot: auth-worker(13400): pam(usertest,192.168.10.219,<slYR46V/DdfAqArb>): pam_authenticate() failed: Authentication failure (password mismatch?)

2 1

Dovecot + Weakforced Policy server
by alberto bersol 16 Jan '19

16 Jan '19

Hi, I'm trying to set Weakforced with Dovecot and I cannot log in policy server. This is the config: /root/weakforced/wforce/wforce.conf ----------------------------------- ... webserver("0.0.0.0:8084", "super") ... /etc/dovecot/conf.d/95-policy.conf ---------------------------------- auth_policy_server_url = http://localhost:8084/ #auth_policy_hash_nonce = wforce:super auth_policy_hash_nonce = {SHA256-CRYPT}$5$Ue5UrToV.Bam02bQ$Bi9OJ62Mkgc20L2HnLVmD2OCHyXaKje6Hh7qNjnOkB9 I'm following the instructions of Dovecot's wiki: https://wiki.dovecot.org/Authentication/Policy ... "To generate the hash, you concatenate nonce, login name, nil byte, password and run it through the hash algorithm once. The hash is truncated when truncation is set to non-zero. The hash is truncated by first choosing bits from MSB to byte boundary (rounding up), then right-shifting the remainding bits. hash = H(nonce||user||'\x00'||password) bytes = round8(bits*8) hash = HEX(hash[0:bytes] >> (bytes-bits*8)) And I set hash with password (super) in this way: vm-weakforced:~# doveadm pw -p noncewforce\x00super -s SHA256-CRYPT {SHA256-CRYPT}$5$ZWIX2dnU7NJvGHgC$hYFbeCCaHYZv0yPP80GHygxQMPmI5BjMx2ttRe9zti2 But if I log in Dovecot Server: vm-weakforced:~# doveadm auth login usuario Password: passdb: usuario auth succeeded extra fields: user=usuario userdb extra fields: usuario system_groups_user=usuario uid=1000 gid=1000 home=/home/usuario Answer of Weakforced is always "...authentication failed": WforceWebserver: HTTP Request "/" from 127.0.0.1:39720: Web Authentication failed And Dovecot logs don't show anything else: ... Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: auth client connected (pid=967) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=doveadm#011resp=dXN1YXJpbwB1c3VhcmlvAHVzdWFyaW8= (previous base64 data may contain sensitive data) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: policy(usuario): Policy request http://localhost:8084/?command=allow Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: policy(usuario): Policy server request JSON: {"device_id":"","login":"usuario","protocol":"doveadm","pwhash":"0a00","remote":"","tls":false} Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: queue http://localhost:8084: Set request timeout to 2019-01-15 16:50:52.236 (now: 2019-01-15 16:50:50.236) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client: peer 127.0.0.1:8084 (shared): Peer created Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client: peer 127.0.0.1:8084: Peer pool created Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: Peer created Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: queue http://localhost:8084: Setting up connection to 127.0.0.1:8084 (1 requests pending) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: Linked queue http://localhost:8084 (1 queues linked) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: queue http://localhost:8084: Started new connection to 127.0.0.1:8084 Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: request [Req4: POST http://localhost:8084/?command=allow] Submitted (requests left=1) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: Creating 1 new connections to handle requests (already 0 usable, connecting to 0, closing 0) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: Making new connection 1 of 1 (0 connections exist, 0 pending) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: (127.0.0.1:8084): Connecting Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: (127.0.0.1:8084): Waiting for connect (fd=20) to finish for max 0 msecs Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: HTTP connection created (1 parallel connections exist) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: (127.0.0.1:8084): Client connected (fd=20) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: Connected Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: Ready for requests Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: Successfully connected (1 connections exist, 0 pending) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client: peer 127.0.0.1:8084: Successfully connected (1 connections exist, 0 pending) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: Using 1 idle connections to handle 1 requests (1 total connections ready) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: queue http://localhost:8084: Connection to peer 127.0.0.1:8084 claimed request [Req4: POST http://localhost:8084/?command=allow] Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: Claimed request [Req4: POST http://localhost:8084/?command=allow] Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: request [Req4: POST http://localhost:8084/?command=allow] Sent header Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: request [Req4: POST http://localhost:8084/?command=allow] Send more (sent 95, buffered=303) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: request [Req4: POST http://localhost:8084/?command=allow] Finished sending payload Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: No more requests to service for this peer (1 connections exist, 0 pending) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: Got 401 response for request [Req4: POST http://localhost:8084/?command=allow] (took 4 ms + 3 ms in queue) Jan 15 16:50:50 vm-weakforced dovecot: auth: Error: policy(usuario): Policy server HTTP error: 401 Unauthorized Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: Response payload stream destroyed (0 ms after initial response) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: request [Req4: POST http://localhost:8084/?command=allow] Finished Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: queue http://localhost:8084: Dropping request [Req4: POST http://localhost:8084/?command=allow] Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: request [Req4: POST http://localhost:8084/?command=allow] Free (requests left=1) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: peer 127.0.0.1:8084: No requests to service for this peer (1 connections exist, 0 pending) Jan 15 16:50:50 vm-weakforced dovecot: auth: Debug: http-client[1]: conn 127.0.0.1:8084 [2]: No more requests queued; going idle (timeout = 10000 msecs) ... Any idea? Thank you so much Regards,

2 4

doveadm neglecting to exit in failure?
by Felipe Gasper 16 Jan '19

16 Jan '19

----- cpssltest(a)cpanelssltest.org [/usr/local/cpanel]# doveadm -v expunge -u cpssltest -- mailbox-guid b8a359119e771b58484a0000a841250d savedbefore 365days; echo $? doveadm(cpssltest): Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied 0 ----- ^^ In the above, shouldn’t the “doveadm” command have exited nonzero to indicate a failure to connect? Thanks! -Felipe Gasper Mississauga, Ontario

3 2

MySQL 8.0: Supported?
by Larry Rosenman 16 Jan '19

16 Jan '19

I got a complaint from a FreeBSD user that they couldn't compile dovecot against MySQL 8.0. Is MySQL 8.0 support with 2.3.4? they receive: checking for mysql_init in -lmysqlclient... no configure: error: Can't build with MySQL support: libmysqlclient not found ===> Script "configure" failed unexpectedly. Please report the problem to ler(a)FreeBSD.org [maintainer] and attach the "/wrkdirs/usr/ports/mail/dovecot/work/dovecot-2.3.4/config.log" including the output of the failure of your make command. Also, it might be a good idea to provide an overview of all packages installed on your system (e.g. a /usr/local/sbin/pkg-static info -g -Ea). *** Error code 1 thanks! -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx(a)gmail.com US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

3 11

Import mailbox from different domain
by Sergio Belkin 15 Jan '19

15 Jan '19

Hi folks, Let's say that I have on dovecot the domain example.net and on MS Exchange example.com. I've tried to import a mailbox from MS Exchange to Dovecot but it fails, I've run: dsync -Dv backup -R -u joe.doe imapc: dsync(joe.doe(a)example.net): Error: imapc(192.168.0.2:993): Authentication failed: AUTHENTICATE failed. Is there a way to submit different domains both on MS Exchange and Dovecot and to make everyone be happy :) ? Thanks in advance -- -- Sergio Belkin LPIC-2 Certified - http://www.lpi.org

2 1

mdbox + zlib performing less than just mdbox
by Marc Roos 14 Jan '19

14 Jan '19

I have test environment to determine what would be best settings. I have been told that enabling zlib compression would be good to save iops on storage. But doing the test now, I get worse results. [@test2 ~]# pr -m -t mail04-mdbox-vdb-append-64kb-6.log mail04-mdbox-vdb-append-64kb-8.log |less Logi Sele Appe Logi Sele Appe 100% 100% 100% 100% 100% 100% 1 1 6 1/ 1 1 1 6 1/ 1 0 0 9 1/ 1 0 0 8 1/ 1 0 0 7 1/ 1 0 0 8 1/ 1 0 0 10 1/ 1 0 0 9 1/ 1 0 0 7 1/ 1 0 0 6 1/ 1 0 0 8 1/ 1 0 0 8 1/ 1 0 0 9 1/ 1 0 0 8 1/ 1 0 0 10 1/ 1 0 0 8 1/ 1 0 0 9 1/ 1 0 0 7 1/ 1 0 0 11 1/ 1 0 0 7 1/ 1 29 3 1093 ms/cmd avg 43 2 1232 ms/cmd avg Logi Sele Appe Logi Sele Appe 100% 100% 100% 100% 100% 100% 0 0 10 1/ 1 0 0 9 1/ 1 0 0 9 1/ 1 0 0 7 1/ 1 0 0 10 1/ 1 0 0 7 1/ 1 0 0 9 1/ 1 0 0 6 1/ 1 0 0 11 1/ 1 0 0 8 1/ 1 0 0 9 1/ 1 0 0 7 1/ 1 0 0 8 1/ 1 0 0 6 1/ 1 0 0 11 1/ 1 0 0 7 1/ 1 0 0 9 1/ 1 0 0 5 1/ 1 0 0 9 1/ 1 0 0 9 1/ 1 0 0 1063 ms/cmd avg 0 0 1413 ms/cmd avg Logi Sele Appe Logi Sele Appe 100% 100% 100% 100% 100% 100% 0 0 8 1/ 1 0 0 8 1/ 1 0 0 10 1/ 1 0 0 8 1/ 1 0 0 7 1/ 1 0 0 8 1/ 1 0 0 10 1/ 1 0 0 6 1/ 1 0 0 10 1/ 1 0 0 7 1/ 1 0 0 11 1/ 1 0 0 8 1/ 1 0 0 7 1/ 1 0 0 8 1/ 1 0 0 11 1/ 1 0 0 6 1/ 1 0 0 8 1/ 1 0 0 7 1/ 1 0 0 10 1/ 1 0 0 8 1/ 1 0 0 1077 ms/cmd avg 0 0 1382 ms/cmd avg Logi Sele Appe Logi Sele Appe 100% 100% 100% 100% 100% 100% 0 0 7 1/ 1 0 0 6 1/ 1 0 0 9 1/ 1 0 0 4 1/ 1 0 0 6 1/ 1 0 0 7 1/ 1 0 0 9 1/ 1 0 0 7 1/ 1 0 0 8 1/ 1 0 0 7 1/ 1 0 0 10 1/ 1 0 0 5 1/ 1 0 0 9 1/ 1 0 0 8 1/ 1 0 0 7 1/ 1 0 0 6 1/ 1 0 0 9 1/ 1 0 0 6 1/ 1 0 0 7 1/ 1 0 0 8 1/ 1 0 0 1246 ms/cmd avg 0 0 1524 ms/cmd avg ... ... ... Logi Sele Appe Logi Sele Appe 100% 100% 100% 100% 100% 100% 0 0 8 1/ 1 0 0 5 1/ 1 0 0 6 1/ 1 0 0 4 1/ 1 0 0 6 1/ 1 0 0 4 1/ 1 0 0 7 1/ 1 0 0 4 1/ 1 0 0 5 1/ 1 0 0 5 1/ 1 0 0 5 1/ 1 0 0 4 1/ 1 0 0 6 1/ 1 0 0 4 1/ 1 0 0 7 1/ 1 0 0 5 1/ 1 0 0 6 1/ 1 0 0 4 1/ 1 0 0 7 1/ 1 0 0 5 1/ 1 0 0 1596 ms/cmd avg 0 0 2308 ms/cmd avg Logi Sele Appe Logi Sele Appe 100% 100% 100% 100% 100% 100% 0 0 7 1/ 1 0 0 4 1/ 1 0 0 1583 ms/cmd avg 0 0 4 1/ 1 0 0 2331 ms/cmd avg Totals: Logi Sele Appe Totals: 100% 100% 100% Logi Sele Appe 1 1 1978 100% 100% 100% 1 1 1145

2 2