- dovecot - dovecot.org

degfault imaptest
by Marc Roos 14 Jan '19

14 Jan '19

imaptest host=mail04.local port=143 user=xxx pass=xxx mbox=test.mbox [Fri Jan 11 15:09:14 2019] imaptest[7634]: segfault at 1354ff0 ip 0000000001354ff0 sp 00007ffed5d8f558 error 15 [Fri Jan 11 15:09:22 2019] imaptest[7635]: segfault at 2267ff0 ip 0000000002267ff0 sp 00007ffee5890308 error 15 [Fri Jan 11 15:10:47 2019] imaptest[7648]: segfault at 10e6ff0 ip 00000000010e6ff0 sp 00007ffedbb19f98 error 15 CentOS Linux release 7.6.1810 (Core) imaptest-20151005-1.el7.x86_64 Linux test2 3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 29 14:49:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

3 4

Indexing paralelism
by Joan Moreau 14 Jan '19

14 Jan '19

Hi Observing the processes of FTS, I observe the following: 1 - For one user, indexer-wroker does not start several threads for each request. On teh contrary, it waits for the first request to finish before starting the second. How to make sure all requests (or a limited number of it, for instance linked to the CPU number on the machine) are started asap ? 2 - If a IMAP query is received, the dovecot checks the last UID from FTS and launch a request of indexing to finish the index *before* running the search query . THis creates timeouts (and can take a while if many request are pending - see point 1) How to prevent that (i.e. the serach request is launched (read only) no matter what ? THe completeion of missing UIDs is launched in a separate thread ? Thank you

2 1

[FTS Xapian] Beta release
by Joan Moreau 13 Jan '19

13 Jan '19

Hi Please find attached the beta release of FTS Xapian, with the objective to replace fts_squat that is being deprecated. Configuration is exactly the same as for fts_squat: plugin { plugin = fts fts_xapian (...) fts = xapian fts_autoindex = yes fts_enforced = yes fts_xapian = partial=2 full=20 This is installed on my production server (>120Gb of mailboxes), and I will observe it during the coming days. I will definitely appreciate that this is added in the core git of docevot, in order to have a versionning of it, to remove squat and let basic users able to avoid Solr alternative as much as possible. Thanks JM

2 9

[FTS Xapian] Beta release
by Joan Moreau 13 Jan '19

13 Jan '19

Hi Please find attached the beta release of FTS Xapian, with the objective to replace fts_squat that is being deprecated. Configuration is exactly the same as for fts_squat: plugin { plugin = fts fts_xapian (...) fts = xapian fts_autoindex = yes fts_enforced = yes fts_xapian = partial=2 full=20 This is installed on my production server, and I will observe it during the coming days. I will definitely appreciate that this is added in the core git of docevot, in order to have a versionning of it, to remove squat and let basic users able to avoid Solr alternative as much as possible. THanks JM

1 0

Issue with LMTP proxying and port number
by Laz C. Peterson 13 Jan '19

13 Jan '19

Hello there, Everything was working fine on Dovecot 2.2.10 (on CentOS 7), but after updating to version 2.2.36, our director servers are not able to proxy LMTP. We are sending mail from exim to the local Dovecot LMTP socket, which then used to send it to our internal mail servers on port 24. Besides the update, nothing has changed in the config files. The configuration has been working great for months … Then we had a great idea of doing holiday maintenance. (Currently, we are working around this issue by having exim send mail to the IP and port, instead of the socket.) Doing a tcpdump capture shows no TCP traffic either. The logs seem to say that it is trying to go to port 0, so my guess is that it’s not getting the port directive from either static or sql — since the docs state that if these settings aren't specified, it will always use the same connection type for the proxy that it received the connection on. Any thoughts? I can share config if necessary. ~ Laz Peterson Paravis, LLC

3 5

Dovecot Submission Proxy Auth
by Paul Hecker 12 Jan '19

12 Jan '19

Hi, Dovecot 2.3.1 (8e2f634). Could not get Dovecot to forward the (plain) authentication to the SMTP server using submission. Reason why I need it is sender spoofing (do not want my employees to send messages in behalf of me). In exim I can disable sender spoofing with the authenticated user. When sending through dovecot, exim either does not accept the email (need auth) or relay every sender address (because relaying from localhost). Am I missing a setting or do I need any additional field in the (MySQL) user_query/password_query to forward the password? You can find my config here: https://gist.github.com/lluuaapp/7daddf761131da47237b0f45e6bab5a8 Thanks, Paul

5 12

Fatal: master: service(indexer-worker): child 493 killed with signal 11 (core dumped)
by Tim Mohlmann 12 Jan '19

12 Jan '19

Dear list, We been having some issues where the indexer-worker is crashing. This happens on production servers which are handling a slight amount of mail, but is also reproducible by moving messages. Also, users on my server are complaining about "Trashed" items coming back etc. Some details: - Dovecot 2.3.2.1 in alpine:3.8 based Docker container. As part of the Mailu distribution. (https://github.com/Mailu/Mailu) - I've first seen this issues on my production server, which stores mail on GlusterFS - I've been able to reproduce running the Docker container on a Virtual machine, using local storage. - There is another Mailu user reporting the same problem on a different VM provider / disk infrastructure: https://github.com/Mailu/Mailu/issues/751 - Libc: musl-1.1.19-r10 Output of dovecot -n: # dovecot -n # 2.3.2.1 (0719df592): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.2 (7704de5e) # OS: Linux 4.16.3-301.fc28.x86_64 x86_64 ext4 # Hostname: 98a2726271d3 auth_mechanisms = plain login disable_plaintext_auth = no first_valid_gid = 8 first_valid_uid = 8 hostname = mail.usrpro.io log_path = /dev/stderr mail_access_groups = mail mail_gid = mail mail_home = /mail/%u mail_location = maildir:/mail/%u mail_plugins = " fts fts_lucene quota quota_clone zlib" mail_privileged_group = mail mail_uid = mail mail_vsize_bg_after_count = 100 maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext spamtest spamtestplus editheader imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/auth.conf driver = dict } plugin { fts = lucene fts_autoindex = yes fts_autoindex_exclude = \Junk fts_lucene = whitespace_chars=@. imapsieve_mailbox1_before = file:/conf/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/conf/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * quota = count:User quota quota_clone_dict = proxy:/tmp/podop.socket:quota quota_vsizes = yes sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = dict:proxy:/tmp/podop.socket:sieve sieve_execute_bin_dir = /conf/bin sieve_extensions = +spamtest +spamtestplus +editheader sieve_global_extensions = +vnd.dovecot.execute sieve_plugins = sieve_imapsieve sieve_extprograms sieve_spamtest_max_value = 15 sieve_spamtest_status_header = X-Spam-Level sieve_spamtest_status_type = strlen sieve_vacation_dont_check_recipient = yes sieve_vacation_send_from_recipient = yes } postmaster_address = admin(a)usrpro.io protocols = imap pop3 lmtp sieve service auth-worker { unix_listener auth-worker { group = mail mode = 0660 user = dovecot } user = mail } service auth { user = dovecot } service imap-login { inet_listener imap { port = 143 } } service lmtp { inet_listener lmtp { port = 2525 } } service managesieve-login { inet_listener sieve { port = 4190 } } submission_host = 192.168.203.6 userdb { args = /etc/dovecot/auth.conf driver = dict } protocol imap { mail_plugins = " fts fts_lucene quota quota_clone zlib imap_quota imap_sieve" } protocol lmtp { mail_plugins = " fts fts_lucene quota quota_clone zlib sieve" recipient_delimiter = + } And the actual error log: imap_1 | Dec 19 16:31:08 indexer-worker(admin(a)usrpro.io)<490><m+t5VmJ93K7AqMsG:grc9HUxyGlzqAQAANEhNiw>: Fatal: master: service(indexer-worker): child 490 killed with signal 11 (core dumped) imap_1 | Dec 19 16:31:09 indexer-worker(admin(a)usrpro.io)<493><m+t5VmJ93K7AqMsG:HRLEK0xyGlztAQAANEhNiw>: Error: lucene index /mail/admin(a)usrpro.io/lucene-indexes: IndexWriter() failed (#1): Lock obtain timed out imap_1 | Dec 19 16:31:09 indexer-worker(admin(a)usrpro.io)<493><m+t5VmJ93K7AqMsG:HRLEK0xyGlztAQAANEhNiw>: Error: Mailbox INBOX: Mail search failed: Internal error occurred. Refer to server log for more information. [2018-12-19 16:31:08] imap_1 | Dec 19 16:31:09 indexer-worker(admin(a)usrpro.io)<493><m+t5VmJ93K7AqMsG:HRLEK0xyGlztAQAANEhNiw>: Error: Mailbox INBOX: Transaction commit failed: FTS transaction commit failed: backend deinit (attempted to index 1 messages (UIDs 1299..1299)) imap_1 | Dec 19 16:31:10 indexer-worker(admin(a)usrpro.io)<493><m+t5VmJ93K7AqMsG:GKWdMU1yGlztAQAANEhNiw>: Fatal: master: service(indexer-worker): child 493 killed with signal 11 (core dumped) imap_1 | Dec 19 16:31:10 indexer: Error: Indexer worker disconnected, discarding 1 requests for admin(a)usrpro.io imap_1 | Dec 19 16:31:11 indexer-worker(admin(a)usrpro.io)<494><m+t5VmJ93K7AqMsG:MRCzBE5yGlzuAQAANEhNiw>: Error: lucene index /mail/admin(a)usrpro.io/lucene-indexes: IndexWriter() failed (#1): Lock obtain timed out imap_1 | Dec 19 16:31:11 indexer-worker(admin(a)usrpro.io)<494><m+t5VmJ93K7AqMsG:MRCzBE5yGlzuAQAANEhNiw>: Error: Mailbox INBOX: Mail search failed: Internal error occurred. Refer to server log for more information. [2018-12-19 16:31:10] imap_1 | Dec 19 16:31:11 indexer-worker(admin(a)usrpro.io)<494><m+t5VmJ93K7AqMsG:MRCzBE5yGlzuAQAANEhNiw>: Error: Mailbox INBOX: Transaction commit failed: FTS transaction commit failed: backend deinit (attempted to index 1 messages (UIDs 1310..1310)) imap_1 | Dec 19 16:31:11 indexer: Error: Indexer worker disconnected, discarding 1 requests for admin(a)usrpro.io I managed to find a core dump file, which appeared outside of the container. So I copied it back in, installed and ran gdb: GNU gdb (GDB) 8.0.1 Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-alpine-linux-musl". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/libexec/dovecot/indexer-worker...(no debugging symbols found)...done. [New LWP 1075] warning: Can't read pathname for load map: No error information. Core was generated by `dovecot/indexer-worker'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007fbd9a31c11a in free () from /lib/ld-musl-x86_64.so.1 So this seems musl related. I installed musl-dbg and ran again: GNU gdb (GDB) 8.0.1 Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-alpine-linux-musl". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/libexec/dovecot/indexer-worker...(no debugging symbols found)...done. [New LWP 1075] warning: Can't read pathname for load map: No error information. Core was generated by `dovecot/indexer-worker'. Program terminated with signal SIGSEGV, Segmentation fault. #0 a_crash () at ./arch/x86_64/atomic_arch.h:108 108 ./arch/x86_64/atomic_arch.h: No such file or directory. Now I'm kinda lost in space. I don't know where that header file is. Tried running "find" on the filesystem and a google search. But nothing specific showed up. I am starting to feel this bug is more musl related than Dovecot. Since this has bitten our project more in the past, I'm considering to move the project to Debian based images. But I want to be 100% sure this is not a dovecot bug. Note, earlier I created an image with Alpine:edge with musl 1.1.20-r2 and Dovecot 2.3.3. Running that image I saw the same error symptoms (Was just a hopeful trail and error). I did not do any debugging on that one. Thanks in advance! Tim

5 11

Enable compression only for namespace with mdbox storage
by Marc Roos 11 Jan '19

11 Jan '19

I have a namespace with mbox and a namespace with mdbox mailboxes. How do I only enable the compression for the mdbox mailboxes?

2 3

repo.dovecot.org expired certificate
by Filipe Carvalho 10 Jan '19

10 Jan '19

Hello, Not sure if this is the right place to post this, but the ssl certificate of the repo.dovecot.org server expired on the 9th of January. It's giving an error via the browser and via the apt command in Debian: W: Failed to fetch https://repo.dovecot.org/ce-2.3-latest/debian/jessie/dists/jessie/main/bina… server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none Cheers! Filipe Carvalho -- UP Digital Filipe Carvalho Infraestruturas Tecnológicas / IT infrastructures filipec(a)uporto.pt <mailto:filipec@uporto.pt>

4 5

BUG: cannot move messages to root mailfolder in namespace "There can be only one namespace with inbox=yes"
by Marc Roos 09 Jan '19

09 Jan '19

I can only move messages to a root namespace folder when specifying inbox=yes. That is strange because it is not even the inbox. I don't even get why it is not by default possible, maybe bug there? You have made the mistake of overloading the use of a parameter inbox=yes (or not creating a parameter to tell folder can be used to move messages to), there is indeed only one folder that gets incoming messages delivered to, and that is the inbox=yes. You should have created a 2nd parameter specifying if a folder should be allowed to store mail. (I have been running this dovecot server for 20 years or so, and I am really surprised about still running into issues on such basic things as trying to put folders onto different storage. Not to mention the lack of help you can get here. Is there some roadmap, features implementation overview available? Can you make public issues on github?)

3 8

File permissions
by Rupert Gallagher 09 Jan '19

09 Jan '19

Hello, I am mosty done with the upgrade, but there is a problem that keeps me awake at night: file ownership and permissions. It would be most helpful to have the following from Aki's live system: ls -halFR /var/run/dovecot

2 2

RE: Dovecot (doveadm, ssl, sync) - SSL error
by Arkadiusz Majewski 08 Jan '19

08 Jan '19

Hello! dsync SSL still doesn't work for replication, so I've disabled it and tried to sync without. But I have a problem with temp directory. Is it possible to change path to temp folder? I don't want to set permissions but change temporary folder for replication. Thanks in advance. dovecot: doveadm: Error: safe_mkstemp(/tmp/dovecot.doveadm.) failed: Permission denied Arkadiusz Majewski From: Arkadiusz Majewski <majewsa(a)gmail.com> Sent: Sunday, December 3, 2017 7:38 PM To: 'dovecot(a)dovecot.org' <dovecot(a)dovecot.org> Subject: Dovecot (doveadm, ssl, sync) - SSL error Hello! I've got a problem to run syncing between both dovecot services on the separate servers. The error indicates to the problem with SSL. Directly using openssl command to connect from one server to other and vice versa is passed without any errors. OS: FreeBSD 11.1-RELEASE-p4 Dovecot: 2.2.33.2_2 and the older one dovecot-2.2.32.1_1 (or similar) - build by ports. OpenSSL: 1.0.2k-freebsd 26 Jan 2017 dovecot: doveadm(10.18.1.15): Error: doveadm client disconnected before handshake: SSL_accept() failed: error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol dovecot.conf (on both servers): mail_plugins = $mail_plugins notify replication service replicator { process_min_avail = 1 } service aggregator { fifo_listener replication-notify-fifo { user = dovecot } unix_listener replication-notify { user = dovecot } } service replicator { unix_listener replicator-doveadm { mode = 0600 } } replication_max_conns = 10 service doveadm { inet_listener { port = 12130 ssl = yes } } ssl = required ssl_protocols = SSLv3 TLSv1 TLSv1.1 TLSv1.2 ssl_cert = </usr/local/etc/dovecot/ssl/cert.crt ssl_key = </usr/local/etc/dovecot/ssl/cert.key ssl_client_ca_file = </usr/local/etc/dovecot/ssl/ca.pem ssl_client_ca_dir = /usr/local/etc/dovecot/ssl Thank you in advance for any help. Kind Regards, Arkadiusz Majewski

2 1

Permission error on stats writer
by Matt Simpson 08 Jan '19

08 Jan '19

I’m running Dovecot 2.3.4 on FreeBSD 11.2 with qmail. In my mail log, I see messages like this on every delivery Jan 8 10:30:02 v1 qmail: 1546961402.314222 delivery 533: success: lda(matt,)Error:_net_connect_unix(/var/run/dovecot/stats-writer)_failed:_Permission_denied/did_0+0+1/ Searching for this error, I found a message on a support forum for a different server that suggested adding this to dovecot.conf: service stats { unix_listener stats-reader { user = vmail group = vmail mode = 0660 } unix_listener stats-writer { user = vmail group = vmail mode = 0660 } } The person reporting the error said that this corrected it. But since I’m running qmail, I have no vmail user/group. I’m guessing that I need to replace user/group with the values that some mail delivery process is running with. I’ve tried a couple of things with no success. Any ideas? $ doveconf -n # 2.3.4 (0ecbaf23d): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 (60b0f48d) # OS: FreeBSD 11.2-RELEASE-p7 amd64 # Hostname: v1.redhorse.me auth_stats = yes auth_verbose = yes default_vsz_limit = 128 M lock_method = flock mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.pipe vnd.dovecot.execute namespace inbox { inbox = yes location = prefix = } passdb { args = imap driver = pam } plugin { recipient_delimiter = - sieve = file:~/sieve;active=~/.dovecot.sieve sieve_execute_bin_dir = /usr/local/lib/dovecot/sieve-pipe sieve_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve-pipe sieve_pipe_exec_timeout = 10s sieve_plugins = sieve_extprograms } postmaster_address = postmaster(a)redhorse.me protocols = imap service auth { unix_listener auth-master { group = qnofiles mode = 0660 user = alias } user = root } service imap-login { process_min_avail = 3 vsz_limit = 94 M } ssl_cert = </usr/local/etc/letsencrypt/live/bitchpack.us-0001/fullchain.pem ssl_cipher_list = EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes syslog_facility = local0 userdb { driver = passwd } verbose_proctitle = yes protocol lda { mail_plugins = " sieve" } ls -l /var/run/dovecot total 20 srw------- 1 root wheel 0 Jan 8 10:29 anvil srw------- 1 root wheel 0 Jan 8 10:29 anvil-auth-penalty srw------- 1 dovecot wheel 0 Jan 8 10:29 auth-client srw------- 1 dovecot wheel 0 Jan 8 10:29 auth-login srw-rw---- 1 alias qnofiles 0 Jan 8 10:29 auth-master -rw------- 1 root wheel 32 Jan 8 10:29 auth-token-secret.dat srw-rw-rw- 1 dovecot wheel 0 Jan 8 10:29 auth-userdb srw------- 1 dovecot wheel 0 Jan 8 10:29 auth-worker srw------- 1 root wheel 0 Jan 8 10:29 config srw-rw---- 1 root dovecot 0 Jan 8 10:29 dict srw-rw---- 1 root dovecot 0 Jan 8 10:29 dict-async srw------- 1 root wheel 0 Jan 8 10:29 director-admin srw-rw-rw- 1 root wheel 0 Jan 8 10:29 dns-client srw------- 1 root wheel 0 Jan 8 10:29 doveadm-server lrwx------ 1 root wheel 35 Jan 8 10:29 dovecot.conf -> /usr/local/etc/dovecot/dovecot.conf drwxr-xr-x 2 root wheel 512 Jan 8 10:29 empty srw-rw---- 1 root dovecot 0 Jan 8 10:29 imap-hibernate srw------- 1 root wheel 0 Jan 8 10:29 imap-master srw-rw-rw- 1 root wheel 0 Jan 8 10:29 imap-urlauth srw------- 1 dovecot wheel 0 Jan 8 10:29 imap-urlauth-worker srw-rw-rw- 1 root wheel 0 Jan 8 10:29 indexer srw------- 1 dovecot wheel 0 Jan 8 10:29 indexer-worker srw------- 1 dovecot wheel 0 Jan 8 10:29 ipc srw------- 1 root wheel 0 Jan 8 10:29 log-errors drwxr-x--- 2 root dovenull 512 Jan 8 10:29 login srw------- 1 root wheel 0 Jan 8 10:29 master -rw------- 1 root wheel 6 Jan 8 10:29 master.pid srw------- 1 root wheel 0 Jan 8 10:29 old-stats prw------- 1 root wheel 0 Jan 8 10:29 old-stats-mail prw------- 1 root wheel 0 Jan 8 10:29 old-stats-user srw------- 1 root wheel 0 Jan 8 10:29 replication-notify prw------- 1 root wheel 0 Jan 8 10:29 replication-notify-fifo srw------- 1 dovecot wheel 0 Jan 8 10:29 replicator srw------- 1 root wheel 0 Jan 8 10:29 stats-reader srw-rw---- 1 root dovecot 0 Jan 8 10:29 stats-writer drwxr-x--- 2 root dovenull 512 Jan 8 10:29 token-login

2 1

Apple mail fails with Submission
by Ruud Voorjans 08 Jan '19

08 Jan '19

Dear all, I'm running dovecot # 2.3.2.1 - Pigeonhole version 0.5.2 () - OS: Linux 4.18.0-12-generic x86_64 Ubuntu 18.10 with Submission. It works great except with apple mail (Iphone). I get an error with the MTA (postfix): ""postfix/submission/smtpd[32552]: warning: non-SMTP command from mail.example.org[xx.xx.xx.xx]: Content-Transfer-Encoding: 7bit"" with other mail-client(s) (Outlook (Desktop and Iphone app)) i have no problem and it proxy-sends the e-mail beautiful out to the recipient. I think it is some kind of bug or maybe it just isn't supported (yet), cause as you said it is still "pretty basic". Best Regards

5 10

Fwd: Re: gcc -> clang
by Rupert Gallagher 08 Jan '19

08 Jan '19

The compiler returns many warnings, and the test returns two IPv6-related errors. I am attaching both logs as reference. > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > On Thursday, January 3, 2019 9:53 AM, Aki Tuomi <aki.tuomi(a)open-xchange.com> wrote: > >> We compile all core code with both gcc and clang. What sort of interesting things did you find? >> >> Aki >> >>> On 03 January 2019 at 11:50 Rupert Gallagher via dovecot < dovecot(a)dovecot.org> wrote: >>> >>> Please, use clang instead of gcc. Code quality can only profit from it. I just compiled 2.3.4 and compiler stderr is full of interesting problems. >> >> --- >> Aki Tuomi

3 3

doveadm + HA
by Maciej Milaszewski IQ PL 07 Jan '19

07 Jan '19

Hi I have two server directors in ring and 5 dovecot servers (2.2.36) IP for IMAP and POP3 is a VIP (keepalived) What is the best solutions to get realy HA for 5 dovecot servers ? Maby corosync+pacemeker ? But this solution is too problematic and hardcore Why I need HA ? Doveadmin is too lazy and doveadm director does not know that one machine broke down and still sends traffic

2 1

Dovecot no logs by sieve user script
by Christian Wolf 07 Jan '19

07 Jan '19

Dear mailinglist, I have the problem that I had to migrate a setup and since then my user's sieve script does no longer log anything. I have sieve_user_log unset and sieve points to a folder (ready for ManageSievev service). In the old config I had sieve point to the symlink and sieve_dir point to the folder of all scripts. As sieve_dir is deprecated I use the sieve directive only (see below). According to my understanding of https://wiki2.dovecot.org/Pigeonhole/Sieve/ Configuration this should result in user logging in ~/.dovecot.sieve.log. This file is never generated by dovecot and if I create it manually, it is not filled. Can you tell me, what I am missing here in order to get the per-user logs running? Thank you very much Christian # dovecot --version # dovecot -n # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.6 auth_mechanisms = plain login disable_plaintext_auth = no mail_location = maildir:~/Mailbox managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { auto = create special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } postmaster_address = postmaster@wolf- stuttgart.net protocols = " imap sieve sieve" service auth { unix_listener /var/spool/postfix/ private/auth { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = required ssl_cert = </etc/letsencrypt/live/imap/fullchain.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lda { mail_plugins = " sieve" }

2 3

special_use folders still created
by Marc Roos 07 Jan '19

07 Jan '19

dovecot-2.2.36-3.el7.x86_64 The below configuration seems to work for Thunderbird and emclient, when moving a message to Junk. Emclient uses "Junk E-mail" and Thunderbird uses "Junk". On the server is only the file .Junk created as expected. When I move a message to Archive in Thunderbird. On the server the files .Archives and .Archives.2018 are created (Thunderbird creates a subfolder in Archives) How is this possible? On the server there is already an .Archive folder so it should create .Archive.2018 not? imap_capability = +SPECIAL-USE # drafts folders merging mailbox Drafts { special_use = \Drafts auto = create } # archive folders merging mailbox Archive { special_use = \Archive auto = create } mailbox Archives { special_use = \Archive auto = no } # spam folders merging mailbox Junk { special_use = \Junk auto = create } mailbox Spam { special_use = \Junk auto = no } mailbox "Junk E-mail" { special_use = \Junk auto = no }

4 3

reprocessing mail
by Marc Lucke 06 Jan '19

06 Jan '19

Hey guys, I had some corruptions whereby dovecot would bomb out on mail files in my maildir. I found that if piped the files to dovecot-lda the all email was clean &lost. I even retrieved email I thought might have been lost for good. I have some better ideas now, but I still processed a few thousand emails like that now & would like to do something like this: * stop dovecot * iterate over each file ** get date ** touch date (linux - to set the date/time) ** rename the file with the correct epoch ** update dovecot-uidlist with correct epoch * start dovecot Will that work? Or is there something I've missed or should do? - Marc

1 0

Sieve "OOO" configuration
by Jerry 05 Jan '19

05 Jan '19

I am able to get sieve issuing an "out of office"message correctly. However, I want to configure it to send an "OOO" message only during certain dates, say on weekends, or only between certain hours. I can do that manually; however, was wondering if there is any automatic method available that could handle this chore. Thanks! -- Jerry

2 2

zlib - some emails maybe double or partial gzipped
by admin 05 Jan '19

05 Jan '19

Hi, I found these in my logs: Jan 5 15:58:56 mail dovecot: imap-login: Login: user=<xy(a)z.com>, method=PLAIN, rip=1.2.3.4, lip=2.3.4.5, mpid=5852, TLS, session=<K2oaPLd+885fWs6U> Jan 5 15:58:56 mail dovecot: imap-login: Login: user=<xy(a)z.com>, method=PLAIN, rip=1.2.3.4, lip=2.3.4.5, mpid=5854, TLS, session=<uxAjPLd+ys5fWs6U> Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Error: Maildir filename has wrong S value, renamed the file from /var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/cur/1537260554.M80670P8164.mail,S=17247,W=17549:2,Sab,Z to /var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/cur/1537260554.M80670P8164.mail,S=4172:2,Sab,Z Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Error: Corrupted index cache file /var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/dovecot.index.cache: Broken physical size for mail UID 99 in mailbox Sent: read(zlib(/var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/cur/1537260554.M80670P8164.mail,S=17247,W=17549:2,Sab,Z)) failed: Cached message size larger than expected (17247 > 4268, box=Sent, UID=99) Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Panic: file istream.c: line 175 (i_stream_read): assertion failed: (old_size <= _stream->pos - _stream->skip) Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x95e92) [0x7f5068d3ee92] -> /usr/lib/dovecot/libdovecot.so.0(+0x95f8d) [0x7f5068d3ef8d] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f5068cd4a91] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x290) [0x7f5068d49f20] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f5068d4a72d] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header_next+0x72) [0x7f5068d25d12] -> /usr/lib/dovecot/libdovecot.so.0(+0x76fd1) [0x7f5068d1ffd1] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x53) [0x7f5068d49ce3] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f5068d4a72d] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header_next+0x72) [0x7f5068d25d12] -> /usr/lib/dovecot/libdovecot.so.0(+0x76fd1) [0x7f5068d1ffd1] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x53) [0x7f5068d49ce3] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f5068d4a72d] -> /usr/lib/dovecot/libdovecot.so.0(message_get_header_size+0x74) [0x7f5068d29904] -> /usr/lib/dovecot/libdovecot-storage.so.0(imap_msgpart_open+0x32f) [0x7f50690b52ef] -> dovecot/imap(+0x1ebbc) [0x556dd2cd7bbc] -> dovecot/imap(+0x1cfb6) [0x556dd2cd5fb6] -> dovecot/imap(imap_fetch_more+0x39) [0x556dd2cd70e9] -> dovecot/imap(cmd_fetch+0x33b) [0x556dd2cc8d9b] -> dovecot/imap(command_exec+0xa5) [0x556dd2cd4735] -> dovecot/imap(+0x199c2) [0x556dd2cd29c2] -> dovecot/imap(+0x19a4c) [0x556dd2cd2a4c] -> dovecot/imap(client_handle_input+0x1b5) [0x556dd2cd2e55] -> dovecot/imap(client_input+0x86) [0x556dd2cd33c6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f5068d539f2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109) [0x7f5068d55029] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7f5068d53a8c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f5068d53c38] Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Fatal: master: service(imap): child 5852 killed with signal 6 (core dumps disabled) # and again: Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Error: Maildir filename has wrong S value, renamed the file from /var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/cur/1537260554.M80670P8164.mail,S=4172:2,Sab,Z to /var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/cur/1537260554.M80670P8164.mail,S=4268:2,Sab,Z Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Error: Corrupted index cache file /var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/dovecot.index.cache: Broken physical size for mail UID 254 in mailbox Sent: read(zlib(/var/vmail/mailboxes/<xy(a)z.com>/mail/Sent/cur/1537260554.M80670P8164.mail,S=4172:2,Sab,Z)) failed: Cached message size smaller than expected (4172 < 4268, box=Sent, UID=254) Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Panic: file istream.c: line 212 (i_stream_read): assertion failed: (!i_stream_is_buffer_invalid(_stream)) Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x95e92) [0x7f1dd9f6be92] -> /usr/lib/dovecot/libdovecot.so.0(+0x95f8d) [0x7f1dd9f6bf8d] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f1dd9f01a91] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x190) [0x7f1dd9f76e20] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f1dd9f7772d] -> /usr/lib/dovecot/libdovecot.so.0(message_get_header_size+0x74) [0x7f1dd9f56904] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_init_stream+0x388) [0x7f1dda2a7538] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x66fc0) [0x7f1dda25efc0] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_stream_because+0x5d) [0x7f1dda22ee1d] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x66d87) [0x7f1dda25ed87] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_virtual_size+0x33) [0x7f1dda22ea83] -> dovecot/imap(+0x1eae3) [0x5595eb2dfae3] -> dovecot/imap(+0x1cfb6) [0x5595eb2ddfb6] -> dovecot/imap(imap_fetch_more+0x39) [0x5595eb2df0e9] -> dovecot/imap(cmd_fetch+0x33b) [0x5595eb2d0d9b] -> dovecot/imap(command_exec+0xa5) [0x5595eb2dc735] -> dovecot/imap(+0x199c2) [0x5595eb2da9c2] -> dovecot/imap(+0x19a4c) [0x5595eb2daa4c] -> dovecot/imap(client_handle_input+0x1b5) [0x5595eb2dae55] -> dovecot/imap(client_input+0x86) [0x5595eb2db3c6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f1dd9f809f2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109) [0x7f1dd9f82029] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7f1dd9f80a8c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f1dd9f80c38] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f1dd9f07fd3] -> dovecot/imap(main+0x328) [0x5595eb2cde68] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f1dd9b572e1] -> dovecot/imap(_start+0x2a) [0x5595eb2cdfea] Jan 5 15:58:56 mail dovecot: imap(<xy(a)z.com>): Fatal: master: service(imap): child 5854 killed with signal 6 (core dumps disabled) Trying to read them with zcat they turned out to be binary. I figured out they might accidentally be compressed twice - they are. How could this happen? I used this script: dovecot-maildir-compress.sh (https://gist.github.com/cviebrock/e37f80a3223dfee19051c936c5ebe509) which appends an additional flag 'Z' after processing/conmpressong the mail and only compresses those without the flag: > find=$(find "$maildir" -type f -name "*,S=*" -mtime +30 ! -name "*,*:2,*,*Z*" -printf "%f\n") ... > if echo $flags | grep ','; > then > newname=$filename"Z" > else > newname=$filename",Z" which is actually neat. Now two possible things: (1) I am not quite sure yet but my best guess is that moving messages from one folder to another (via mail client) the message looses it's 'Z' flag. (2) compressing might have failed to rename but successfully compressed the file and the another consecutive run of the script compressed the file again. How could I recursively (a) check each maildir file if e.g. the output of(zcat $file | zcat) is ASCII or !binary (b) find all mails which are compressed - i.e. valid gzip - but miss the 'Z' flag ? (c) find all mails that are valid gzip but dont contain ASCII if / dont uncompress to mail ? There are backups in place for all files I get a error for, but I want to recursively fix the problem. Any hint? Thanks in advance! -M -- info(a)awib.it

1 0

Searching small(ish) sites on ubuntu
by Dirk Koopman 05 Jan '19

05 Jan '19

I run a small dovecot site with ~5 users, some of whom have many large mailboxes. I used to use lucene until the packagers decided that it broke some licencing rules and stopped including it. I have dismally failed to configure Solr, plus it is huge and potentially exposes too many services as well. Squash is deprecated (but still used?) in 2.2+. I am currently on 2.2.33. So what can I use? Or do I have to bite the bullet and install from source, thus losing the advantages of packaging and automatic updates? Dirk

1 0

Compiling Dovecot on Solaris 11 fails
by Andrew Watkins 04 Jan '19

04 Jan '19

Hi, Tried to build dovecot-2.3.4 on Solaris 11 x86 and it fails at configure part. I just went checked and it last version it works on is v2.3.2.1 Version 2.3.2.1 (OK) # ./configure .... checking Linux compatible mremap()... no checking whether shared mmaps get updated by write()s... yes checking whether fd passing works... yes ... Version 2.3.4 (FAIL) .... checking Linux compatible mremap()... no checking whether shared mmaps get updated by write()s... no checking whether fd passing works... no configure: error: fd passing is required for Dovecot to work .... config.log shows: configure:22538: checking whether shared mmaps get updated by write()s configure:22585: gcc -o conftest -std=gnu99 -g -O2 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -mfunction-return=thunk -mindirect-branch=thunk -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 conftest.c >&5 configure:22585: $? = 0 configure:22585: ./conftest ./configure[2026]: eval: line 1: 29183: Memory fault(coredump) .... onfigure:22610: checking whether fd passing works configure:22685: gcc -o conftest -std=gnu99 -g -O2 -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -mfunction-return=thunk -mindirect-branch=thunk -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I./src/lib ./src/lib/fdpass.c conftest.c >&5 configure:22685: $? = 0 configure:22685: ./conftest ./configure[2026]: eval: line 1: 29206: Memory fault(coredump) ... $ ulimit -a core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited open files (-n) 512 pipe size (512 bytes, -p) 10 stack size (kbytes, -s) unlimited cpu time (seconds, -t) unlimited max user processes (-u) 29995 virtual memory (kbytes, -v) unlimited Any ideas and thanks. Andrew -- Andrew Watkins * Birkbeck, University of London * Computer Science * * http://notallmicrosoft.blogspot.com * * UKOUG Systems SIG Chair * * UKOUG Tech committee * * tel: 020 7631 6720 * ::e-mails are only read & actioned between 0900 and 1800 hours (Monday - Friday)

4 6

doveadm_allowed_commands doesn't work as expected
by Ronald Poon 04 Jan '19

04 Jan '19

Trying to limit the API calls to doveadm-http-api by configure allowed commands, but once the commands added to the list, the RestAPI no longer work. 1) Return correct reply when doveadm_allowed_commands is empty # curl -k -H "Content-Type: application/json" -H "Authorization: X-Dovecot-API <base64 api key>" https://localhost:9088/doveadm/v1 -d'[["quotaGet",{"user":"user1(a)mydomain.com"},"c01"]]' [["doveadmResponse",[{"root":"User quota","type":"STORAGE","value":"0","limit":1024","percent":"0"},{"root":"Userquota","type":"MESSAGE","value":"0","limit":"-","percent":"0"}],"c01"]] 2) Return unAuthorized when doveadm_allowed_commands = quotaGet,quotaRecalc,expunge # curl -k -H "Content-Type: application/json" -H "Authorization: X-Dovecot-API <base64 api key>" https://localhost:9088/doveadm/v1 -d'[["quotaGet",{"user":"user1(a)mydomain.com"},"c01"]]' [["error",{"type":"unAuthorized", "exitCode":0},"c01"]] Here is my config, # uname -a Linux ad92422d8e94 3.10.0-862.2.3.el7.x86_64 #1 SMP Wed May 9 18:05:47 UTC 2018 x86_64 Linux # free -m total used free shared buffers cached Mem: 15885 7133 8751 0 1 4374 -/+ buffers/cache: 2758 13126 Swap: 0 0 0 / # dovecot -n # 2.3.2.1 (0719df592): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.2 (7704de5e) # OS: Linux 3.10.0-862.2.3.el7.x86_64 x86_64 xfs # Hostname: ad92422d8e94 auth_mechanisms = plain login doveadm_allowed_commands = quotaGet,quotaRecalc,expunge doveadm_api_key = # hidden, use -P to show it hostname = mailhost.mydomain.com info_log_path = /dev/stdout lda_mailbox_autosubscribe = yes log_path = /dev/stderr login_greeting = Dovecot ready. mail_gid = vmail mail_home = /var/vmail/%d/%n mail_location = maildir:/var/vmail/%d/%n/Maildir mail_plugins = " quota zlib" mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { imapsieve_mailbox1_before = file:/etc/dovecot/sieve/global/learn-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/etc/dovecot/sieve/global/learn-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * quota = maildir:User quota quota_exceeded_message = User %u has exhausted allowed storage space. recipient_delimiter = - sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /etc/dovecot/sieve/global/spam-to-folder.sieve sieve_global_extensions = +vnd.dovecot.pipe sieve_pipe_bin_dir = /usr/lib/dovecot/sieve sieve_pipe_exec_timeout = 60s sieve_plugins = sieve_imapsieve sieve_extprograms zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster(a)mydomain.com protocols = lmtp imap pop3 sieve recipient_delimiter = - service auth { inet_listener { port = 9000 } } service doveadm { client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = $default_internal_group inet_listener http { port = 9088 ssl = yes } service_count = 1 } service lmtp { inet_listener lmtp { port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = </etc/tls/mailserver.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it submission_host = mta-host.mydomain.com userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = " quota zlib sieve" } protocol imap { mail_plugins = " quota zlib imap_sieve imap_quota imap_zlib" }

2 2

gcc -> clang
by Rupert Gallagher 03 Jan '19

03 Jan '19

Please, use clang instead of gcc. Code quality can only profit from it. I just compiled 2.3.4 and compiler stderr is full of interesting problems.

5 4