January 2020 - dovecot

Dovecot 2.3 repo for CentOS 8.
by Reio Remma 15 Feb '20

15 Feb '20

Hello! Are there any plans for an official Dovecot repo for CentOS 8? Thanks, Reio

7 10

auth-client via SSL?
by Heiko Schlittermann 05 Feb '20

05 Feb '20

Hello, does dovecot support tls-on-connect for AF INET based auth-client sockets? Rationale behind my question: Exim can use the Dovecot auth-client socket to delegate the SMTP-AUTH authentication to Dovecot. Currently Exim supports the AF UNIX only for this socket. Jeremy makes progress in extending this to use AF INET sockets too. While it works with clear text communication already, during testing I was to setup the auch-client socket as an TLS server (tls-on-connect). It doesn't seem to work as I'd expect. The socket still offers clear-text only. Here my configuration snippets regarding this socket ssl = yes ssl_cert = </etc/dovecot/private/server.pem ssl_key = </etc/dovecot/private/server.pem service auth { … unix_listener auth-client { group = _exim mode = 0660 } inet_listener auth-client { name = exim port = 4711 ssl = yes } } SSL connections to :993 work as expected. Best regards from Dresden/Germany Viele Grüße aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ -

2 3

Vacation use different SMTP server
by azurit＠pobox.sk 03 Feb '20

03 Feb '20

Hi, in our setup, we have two SMTP servers: - one for receiving messages (this one is using Dovecot's LMTP for delivery) - one for sending messages (this one is used by our users as outgoing SMTP server) Only the first one has list of local domains, local e-mail accoutns etc. so the second one will ALWAYS use MX records for delivery - with this setup, we can allow to our users to add any domain to their accounts (for example, if someone will add gmail.com and create e-mail accounts on it, it's complete safe because we will never deliver mail for gmail.com locally). The only problem seems to be vacation messages (maybe other messages send by sieve), as Dovecot is using the first SMTP server for sending. My questions: Can i set different SMTP server for sieve messages? How exactly is Dovecot/Sieve choosing which SMTP server will be used? Thanks. azur

3 6

mail_filter and mail_filter_out broken somewhere between 2.2.27 and 2.3.4, problem still exists in git
by Michael Fincham 03 Feb '20

03 Feb '20

Hi Dovecot folks, I recently reported this bug in Debian (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=939442) after a stretch -> buster upgrade, however since I can reproduce it using the latest 2.4 from git I thought perhaps I ought to take it straight upstream. Here's the text I provided Debian about the issue, and below that I've included a syslog snip showing the same issue in a fresh build of git master as well. mail_filter_out seems to fully panic (backtrace below), whereas mail_filter is just behaving oddly. Thanks :) ***************************** Package: dovecot-core Version: 2.3.4.1-5+deb10u1 Severity: important Hi Dovecot maintainers, After upgrading from the latest stretch version of dovecot-core (2.2.27-3+deb9u5) to buster (2.3.4.1-5+deb10u1) the "mail filter" interface seems to have broken. This problem is also present in the sid version (2.3.7.2-1). Mail filter is used to run the content of messages being stored or retrieved through an arbitrary process. The interface used for the external process is described here: https://wiki.dovecot.org/Plugins/MailFilter It is expected that the process will be called with the arguments specified in the configuration then supplied a message body on stdin, however since upgrading it seems that the arguments are not being correctly passed, instead being all jammed in to $1 and separated by tabs, rather than expanded in to argv. Even stranger the first few lines of the body of the message appear in argv rather than in stdin... These reproduction steps were carried out on a fresh install of Debian Buster with the "dovecot-core" package installed initially from stretch then upgraded to the buster version, however this behaviour has also been observed on stretch systems after upgrading to buster. To reproduce, drop this minimalised config in to /etc/dovecot/conf.d/repro.conf: # Minimal maildir setup for LDA first_valid_uid = 0 mail_location = maildir:~/Maildir userdb { driver = static args = allow_all_users=yes uid=mail gid=mail home=/tmp/%u } # Mail filter configuration mail_plugins = $mail_plugins mail_filter plugin { mail_filter = mail-filter read %u mail_filter_out = mail-filter write %u } service mail-filter { executable = script /usr/local/bin/mail-filter.sh user = dovecot unix_listener mail-filter { mode = 0666 } } And create the example script in /usr/local/bin/mail-filter.sh with executable permissions: #!/bin/bash echo "arguments were: $@" 1>&2 tempfile=$(mktemp) cat > "${tempfile}" cat "${tempfile}" rm "${tempfile}" This script simply logs the arguments it recieves, writes stdin to a file then retrieves it again from the file. Here's a sample session of me using dovecot-lda to pass a message through this script, then retrieve it again with doveadm: root@scratch-b6fd8bdd:~# /usr/lib/dovecot/dovecot-lda -d fincham@scratch-b6fd8bdd -f 'Test <scratch-b6fd8bdd>' < /root/test.eml root@scratch-b6fd8bdd:~# doveadm fetch -u 'fincham@scratch-b6fd8bdd' hdr subject Test hdr: To: Test <fincham@scratch-b6fd8bdd> From: Test <fincham@scratch-b6fd8bdd> Subject: This is the test e-mail root@scratch-b6fd8bdd:~# tail /var/log/syslog | grep mail-filter Sep 5 00:07:48 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: write fincham@scratch-b6fd8bdd Sep 5 00:07:50 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: read fincham@scratch-b6fd8bdd However after upgrading dovecot to the buster version this process no longer works, and the logged argv of the example script is very strange: root@scratch-b6fd8bdd:~# apt dist-upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following NEW packages will be installed: liblua5.3-0 libsodium23 ssl-cert The following packages will be upgraded: dovecot-core [...] root@scratch-b6fd8bdd:~# /usr/lib/dovecot/dovecot-lda -d fincham@scratch-b6fd8bdd -f scratch-b6fd8bdd < /root/test.eml Aborted root@scratch-b6fd8bdd:~# doveadm fetch -u 'fincham@scratch-b6fd8bdd' hdr subject Test hdr: Hello! doveadm(fincham@scratch-b6fd8bdd): Error: Mailbox INBOX: UID=1: read() failed: Cached message size larger than expected (115 > 7, box=INBOX, UID=1) (read reason=mail stream) doveadm(fincham@scratch-b6fd8bdd): Error: Corrupted record in index cache file /tmp/fincham(a)scratch-b6fd8bdd/Maildir/dovecot.index.cache: UID 1: Broken physical size in mailbox INBOX: read() failed: Cached message size larger than expected (115 > 7, box=INBOX, UID=1) doveadm(fincham@scratch-b6fd8bdd): Error: read() failed: Cached message size larger than expected (115 > 7, box=INBOX, UID=1) doveadm(fincham@scratch-b6fd8bdd): Error: fetch(hdr) failed for box=INBOX uid=1: read() failed: Cached message size larger than expected (115 > 7, box=INBOX, UID=1) root@scratch-b6fd8bdd:~# tail /var/log/syslog | grep mail-filter Sep 5 00:07:48 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: write fincham@scratch-b6fd8bdd Sep 5 00:07:50 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: read fincham@scratch-b6fd8bdd Sep 5 00:12:11 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: write#011fincham@scratch-b6fd8bdd#011 To: Test <fincham@scratch-b6fd8bdd> From: Test <fincham@scratch-b6fd8bdd> Subject: This is the test e-mail Sep 5 00:12:14 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: read#011fincham@scratch-b6fd8bdd#011 To: Test <fincham@scratch-b6fd8bdd> From: Test <fincham@scratch-b6fd8bdd> Subject: This is the test e-mail -- Package-specific info: dovecot configuration --------------------- # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-5-cloud-amd64 x86_64 Debian 10.0 # Hostname: scratch-b6fd8bdd first_valid_uid = 0 mail_location = maildir:~/Maildir mail_plugins = " mail_filter" mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { mail_filter = mail-filter read %u mail_filter_out = mail-filter write %u } service mail-filter { executable = script /usr/local/bin/mail-filter.sh unix_listener mail-filter { mode = 0666 } user = dovecot } ssl = no userdb { driver = passwd } userdb { args = allow_all_users=yes uid=mail gid=mail home=/tmp/%u driver = static } -- System Information: Debian Release: 10.0 APT prefers stable-updates APT policy: (500, 'stable-updates'), (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 4.19.0-5-cloud-amd64 (SMP w/2 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages dovecot-core depends on: ii adduser 3.118 ii libapparmor1 2.13.2-10 ii libbz2-1.0 1.0.6-9.1 ii libc6 2.28-10 ii libexttextcat-2.0-0 3.4.5-1 ii libicu63 63.1-6 ii liblua5.3-0 5.3.3-1.1 ii liblz4-1 1.8.3-1 ii liblzma5 5.2.4-1 ii libpam-runtime 1.3.1-5 ii libpam0g 1.3.1-5 ii libsodium23 1.0.17-1 ii libssl1.1 1.1.1c-1 ii libstemmer0d 0+svn585-1+b2 ii libwrap0 7.6.q-28 ii lsb-base 10.2019051400 ii openssl 1.1.1c-1 ii ssl-cert 1.0.39 ii ucf 3.0038+nmu1 ii zlib1g 1:1.2.11.dfsg-1 dovecot-core recommends no packages. Versions of packages dovecot-core suggests: pn dovecot-gssapi <none> pn dovecot-imapd <none> pn dovecot-ldap <none> pn dovecot-lmtpd <none> pn dovecot-lucene <none> pn dovecot-managesieved <none> pn dovecot-mysql <none> pn dovecot-pgsql <none> pn dovecot-pop3d <none> pn dovecot-sieve <none> pn dovecot-solr <none> pn dovecot-sqlite <none> pn dovecot-submissiond <none> ii ntp 1:4.2.8p12+dfsg-4 Versions of packages dovecot-core is related to: ii dovecot-core [dovecot-common] 1:2.3.4.1-5+deb10u1 pn dovecot-dev <none> pn dovecot-gssapi <none> pn dovecot-imapd <none> pn dovecot-ldap <none> pn dovecot-lmtpd <none> pn dovecot-managesieved <none> pn dovecot-mysql <none> pn dovecot-pgsql <none> pn dovecot-pop3d <none> pn dovecot-sieve <none> pn dovecot-sqlite <none> -- no debconf information ***************************** And here it is on 2.4 from git: Sep 5 02:18:10 scratch-b6fd8bdd dovecot: master: Dovecot v2.4.devel () starting up for imap, pop3, lmtp Sep 5 02:18:10 scratch-b6fd8bdd systemd[1]: Started LSB: Dovecot init script. Sep 5 02:18:12 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: write#011fincham@scratch-b6fd8bdd#011 To: Test <fincham@scratch-b6fd8bdd> From: Test <fincham@scratch-b6fd8bdd> Subject: This is the test e-mail Sep 5 02:18:12 scratch-b6fd8bdd dovecot: lda(fincham@scratch-b6fd8bdd)<4213><94f8BWRwcF11EAAA/w3Ckw>: Panic: file ostream.c: line 201 (o_stream_flush): assertion failed: (stream->stream_errno != 0) Sep 5 02:18:12 scratch-b6fd8bdd dovecot: lda(fincham@scratch-b6fd8bdd)<4213><94f8BWRwcF11EAAA/w3Ckw>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xef5fb) [0x7f85ed0715fb] -> /usr/lib/dovecot/libdovecot.so.0(+0xef661) [0x7f85ed071661] -> /usr/lib/dovecot/libdovecot.so.0(+0x4e808) [0x7f85ecfd0808] -> /usr/lib/dovecot/libdovecot.so.0(+0x52ec1) [0x7f85ecfd4ec1] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x96) [0x7f85ed1b24d6] -> /usr/lib/dovecot/libdovecot-lda.so.0(+0x4395) [0x7f85ed294395] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_finish+0x82) [0x7f85ed189e52] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0xd9) [0x7f85ed17c409] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_copy+0x52) [0x7f85ed1aea02] -> /usr/lib/dovecot/libdovecot-lda.so.0(+0x4221) [0x7f85ed294221] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x5c1bc) [0x7f85ed18a1bc] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver_save+0x1b0) [0x7f85ed295390] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x2f6) [0x7f85ed295ad6] -> /usr/libexec/dovecot/dovecot-lda(main+0x81b) [0x560cba869f4b] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f85ecdde09b] -> /usr/libexec/dovecot/dovecot-lda(_start+0x2a) [0x560cba86a65a]

2 3

Sieve prevents tigger of quota-warning
by Tom Sommer 31 Jan '20

31 Jan '20

Hi All I have a few reports from Sieve users, that they are not receiving quota warnings. It appears that when a mailbox is reaching quota, and sieve is responsible for moving mail the mail due to a rule, the quota-warning directive is not triggered? It works fine if Sieve is not part of the flow, and lmtp is alone in handling mail saving. Does anyone have a fix or insights into this? 2.3.9.2 Thanks -- Tom

2 1

Split storage & solr
by Daniel Miller 30 Jan '20

30 Jan '20

Is it possible to setup the following: 1. Primary mailserver with SMTP solutions and Dovecot. 2. Primary server will store recent mails. 3. Secondary server for archival storage and Solr. So I'm not looking for a distributed cluster - simply a splitting of designated functions. Clients would only be connecting to the primary server. I know Dovecot supports an alternate storage location - that's what made me think this was possible. What I'm more uncertain of is Solr. My (probably flawed) thinking is to take advantage of a low-cost high-availability cloud server to provide primary email services. Dovecot, Postfix, and even ASSP don't have significant memory or processor requirements by themselves - especially for my smaller user base. However - storage in the cloud can be at a premium. Therefore I'm thinking of continuing to self-host the archives. And my own server has the raw power & memory to handle Solr easily. What's triggering this is our ISP's quality has been deteriorating - and the alternates don't appear much better. Initial visualization would have a VPN/SSH connection between the servers, and NFS mounting my storage to the cloud server for archives. If our connection drops - in theory "current" mail handling is unaffected. My concerns/questions: 1. If Dovecot is unable to reach the remote Solr - upon re-connection will Solr be told about the new messages to index? Or do I need to setup a periodic re-scan? 2. Is there a "better" method of accessing the archive storage area than NFS? Either a different network file system or is there a way to do it with Dovecot directly? 3. What am I not taking into consideration in this setup that you think will be a problem? --- Daniel

2 1

IMAP compatible Notes app for Android
by Rupert Gallagher 29 Jan '20

29 Jan '20

Hello! Please share your favourite open-source Notes app for Android, as I am struggling to find one compatible with IMAP. Thank you, and happy 2020!

2 2

imap panic: file array.c: line 10 (array_idx_modifiable_i)
by mabi 29 Jan '20

29 Jan '20

Hello, I just migrated a few dovecot IMAP mailbox (Maildir format) from a Dovecot 2.1.7 (Debian 7) mailbox server to a Dovecot 2.3.4 (Debian 10) mailbox server and one particular mailbox does not work properly and generates the following error log entries: Jan 21 09:39:21 Panic: imap(EMAIL_ADDRESS)<7741><UTsrWKKcasoAAAAAAAAAAAAAAAAAAAAB>: file array.c: line 10 (array_idx_modifiable_i): assertion failed: (idx * array->element_size < array->buffer->used) Jan 21 09:39:21 Error: imap(EMAIL_ADDRESS)<7741><UTsrWKKcasoAAAAAAAAAAAAAAAAAAAAB>: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xdb15b) [0x7fa18def215b] -> /usr/lib/dovecot/libdovecot.so.0(+0xdb1f1) [0x7fa18def21f1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a001) [0x7fa18de61001] -> /usr/lib/dovecot/libdovecot.so.0(+0x493bf) [0x7fa18de603bf] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xde920) [0x7fa18e084920] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x22f) [0x7fa18e096a3f] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x85b4b) [0x7fa18e02bb4b] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x8862a) [0x7fa18e02e62a] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x88937) [0x7fa18e02e937] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_storage_sync_init+0xa3) [0x7fa18e02ece3] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x57) [0x7fa18e002497] -> dovecot/imap [EMAIL_ADDRESS ::1 UID STORE](imap_sync_init+0x91) [0x563d342e16e1] -> dovecot/imap [EMAIL_ADDRESS ::1 UID STORE](cmd_sync_delayed+0x236) [0x563d342e25f6] -> dovecot/imap [EMAIL_ADDRESS ::1 UID STORE](client_handle_input+0x268) [0x563d342d58f8] -> dovecot/imap [EMAIL_ADDRESS ::1 UID STORE](client_input+0x7e) [0x563d342d5d6e] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6f) [0x7fa18df0860f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x136) [0x7fa18df09c06] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7fa18df086ac] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7fa18df08810] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fa18de88fd3] -> dovecot/imap [EMAIL_ADDRESS ::1 UID STORE](main+0x325) [0x563d342c7bf5] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7fa18dc7309b] -> dovecot/imap [EMAIL_ADDRESS ::1 UID STORE](_start+0x2a) [0x563d342c7d8a] Jan 21 09:39:21 Fatal: imap(EMAIL_ADDRESS)<7741><UTsrWKKcasoAAAAAAAAAAAAAAAAAAAAB>: master: service(imap): child 7741 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) Does anyone have any idea what could be going wrong here? Maybe I need to run some kind of conversion or re-generate some indexes? Best regards, Mabi

3 10

Emails in Public folders aren't showing up in Outlook
by Barry Ralphs 29 Jan '20

29 Jan '20

Hi, I've inherited our Devocot setup from someone else & don't know much about it. I can view our Public email folders fine in Thunderbird, but when I try it with Outlook 2013, I can see the folder structure fine, but when I click on a folder it doesn't show any emails. Here's a screen shot of Outlook: https://imgur.com/CHIDJOr And Thunderbird: https://imgur.com/WxmsUTM Here's the version info & config output: # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-1062.1.1.el7.x86_64 x86_64 CentOS Linux release 7.7.1908 (Core) ext4 # Hostname: nikola auth_mechanisms = plain login auth_verbose = yes debug_log_path = syslog disable_plaintext_auth = no info_log_path = syslog mail_location = maildir:/home/%n/Maildir mbox_write_locks = fcntl namespace TMpublic { location = maildir:/iscsi/public/Maildir:INDEX=~/Maildir/public/index prefix = Public/ separator = / subscriptions = no type = public } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Sent { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { driver = pam } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } Any suggestions on how to get the Public folders to show the emails in Outlook? Thanks for the help.

1 0

error: include: personal location for included script `foo is unavailable
by Rudolf Schmidt 28 Jan '20

28 Jan '20

Hi, I want to split my active script into smaller chunks but I have issues with the include directive. My doveconf -n is # 2.3.9.2 (cf2918cac): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.9 (db4e9a2f) # OS: Linux 5.1.15-arch1-1-ARCH x86_64 Arch Linux auth_mechanisms = plain login auth_verbose = yes mail_location = maildir:~/mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext passdb { args = scheme=plain /etc/dovecot/plain.pwd driver = passwd-file } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service stats { unix_listener stats-writer { mode = 0777 } } ssl = required ssl_cert = </etc/letsencrypt/live/domain.com/fullchain.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = uid=vmail gid=vmail home=/var/mail/virtual/%Ld/%Ln driver = static } protocol lmtp { mail_plugins = " sieve" postmaster_address = rs@domain } I have a script vendors.sieve that I put into ~/sieve (my mails I store in ~/mails) and it does not find the script. include :personal "vendors"; Any ideas? -- Best Regards, Mit freundlichen Grüßen, Rudolf Schmidt

1 1