January 2020 - dovecot

[Dovecot] Postfixadmin/MySQL/Dovecot 2 and quota
by Matt 01 Apr '20

01 Apr '20

Hi all, I'm setting up a new email server based on the softwares list below: * Postfix * Dovecot 2.0.9 * MySQL * Postfixadmin 2.3.6 My current problem is I'm unsure whether quota feature is well configured or not. In order to test and validate, I use Roundcubemail GUI which provides percentage of the space used by the mailbox (quota should be applied per mailbox). Here, my dovecot configuration: dovecot.conf: dict { quotadict = mysql:/etc/dovecot/dovecot-dict-quota.conf } dovecot-dict-quota.conf: connect = host=localhost dbname=postfix user=postfix password=postfix map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } dovecot-sql.conf.ext: driver = mysql connect = host=localhost dbname=postfix user=postfix password=postfix default_pass_scheme = PLAIN-MD5 password_query = SELECT username AS user, password, \ CONCAT('*:storage=', CAST(quota AS CHAR), 'b') AS quota_rule \ FROM mailbox WHERE username = '%u' AND active='1' user_query = SELECT maildir, 89 AS uid, 12 AS gid, \ concat('dict:storage=',floor(quota/1000),'::proxy::quotadict') AS quota, \ CONCAT('*:storage=', CAST(quota AS CHAR), 'b') AS quota_rule \ FROM mailbox WHERE username = '%u' AND active='1' conf.d/90-quota.conf: plugin { quota = dict:user::proxy::quotadict quota_rule2 = Trash:storage=+10%% } conf.d/20-imap.conf: protocol imap { mail_plugins = $mail_plugins imap_quota } When I check postfixadmin GUI, I can see that quota in realtime in quota2 table, no problem. But Roundcubemail cannot get this information and shows a unknown quota. If I add this line: quota_rule = *:storage=1G in conf.d/90-quota.conf file, it works, I mean roundcubemail gets and shows information (but not the one stored in db of course). I think I'm close to the right configuration but I need your help to complete this setup. Thank you in advance for your help. Best, Matt

7 20

Namespace problem? duplicated folders...
by Gregory Sloop 30 Mar '20

30 Mar '20

I'm sure this is related to name-spaces, but for some reason [brain damage perhaps? :) ] I can't seem to figure out how to fix it. In TBird, for example, I have a folder tree that looks like this Inbox -| Folder-A Folder-B Folder-C Folder-A Folder-B Folder-C And Folder-A, Folder-B, Folder-C are the same folders, just shown twice. [In two different hierarchies.] I'm using mbox files to store mail. [If that matters.] Here's what I have in my conf files for namespace defs namespace { inbox = yes #hidden = yes prefix = INBOX/ separator = / } namespace inbox { mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Trash { special_use = \Trash } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } } I've had this problem for literally years, and it's not been that big of a deal (mostly irritating) - and I've tried to "fix" it before without any success. But now it's causing some issues and I'd really like to get this nagging issue gone. Since it's been so long since I last tried to fix it, I really can't recall any of settings changes I made in an attempt to fix it. I'd guess this is a trivial issue, but if so, can someone point me in the right direction. Thanks in advance! -Greg

2 5

Strategy for fts and Replication
by Francis Augusto Medeiros-Logeay 22 Feb '20

22 Feb '20

Hi there, I got successfully to replicate my mail server to another dovecot install using dsync, mainly for redundancy, and it works great. I want to try to install fts, as some of the mailboxes have tens of thousands of messages, and it takes minutes to get some results when searching via IMAP on a Roundcube interface. I want to experiment with fts-solr first, and firstly on my redundant server, ie., not on my main dovecot install. Is it ok to do this? I ask because I am afraid of how this whole reindexing on the redundant install will affect the production server. Also, any tips on something else than fts-solr? I tried it once, but it was so hard to get it right, so many configurations, java, etc., that I'd rather try something else. I also could try fts-elastic or something like that, but, again, having to maintain an elasticsearch install might use more resources than I think is worth. Any thoughts on that? Best, -- Francis

6 10

BUG: dsync and folder names with "@"
by spamvoll＠googlemail.com 20 Feb '20

20 Feb '20

Hello, dovecot version: 2.3.9.2 When I use dsync: /usr/bin/dsync -v -D -o plugin/quota= -f -u my(a)email.dom backup maildir:/tmp/mailbackup/Maildir it always stops when it comes to a folder with an "@" in its name. Error Message: Warning: Mailbox changes caused a desync. You may want to run dsync again: Remote lost mailbox GUID (maybe it was just deleted?) I dont know if it makes a difference, but my dovecot uses mdbox and the backup export is maildir. Hans

2 2

Dovecot 2.3 repo for CentOS 8.
by Reio Remma 15 Feb '20

15 Feb '20

Hello! Are there any plans for an official Dovecot repo for CentOS 8? Thanks, Reio

7 10

auth-client via SSL?
by Heiko Schlittermann 05 Feb '20

05 Feb '20

Hello, does dovecot support tls-on-connect for AF INET based auth-client sockets? Rationale behind my question: Exim can use the Dovecot auth-client socket to delegate the SMTP-AUTH authentication to Dovecot. Currently Exim supports the AF UNIX only for this socket. Jeremy makes progress in extending this to use AF INET sockets too. While it works with clear text communication already, during testing I was to setup the auch-client socket as an TLS server (tls-on-connect). It doesn't seem to work as I'd expect. The socket still offers clear-text only. Here my configuration snippets regarding this socket ssl = yes ssl_cert = </etc/dovecot/private/server.pem ssl_key = </etc/dovecot/private/server.pem service auth { … unix_listener auth-client { group = _exim mode = 0660 } inet_listener auth-client { name = exim port = 4711 ssl = yes } } SSL connections to :993 work as expected. Best regards from Dresden/Germany Viele Grüße aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ -

2 3

Vacation use different SMTP server
by azurit＠pobox.sk 03 Feb '20

03 Feb '20

Hi, in our setup, we have two SMTP servers: - one for receiving messages (this one is using Dovecot's LMTP for delivery) - one for sending messages (this one is used by our users as outgoing SMTP server) Only the first one has list of local domains, local e-mail accoutns etc. so the second one will ALWAYS use MX records for delivery - with this setup, we can allow to our users to add any domain to their accounts (for example, if someone will add gmail.com and create e-mail accounts on it, it's complete safe because we will never deliver mail for gmail.com locally). The only problem seems to be vacation messages (maybe other messages send by sieve), as Dovecot is using the first SMTP server for sending. My questions: Can i set different SMTP server for sieve messages? How exactly is Dovecot/Sieve choosing which SMTP server will be used? Thanks. azur

3 6

mail_filter and mail_filter_out broken somewhere between 2.2.27 and 2.3.4, problem still exists in git
by Michael Fincham 03 Feb '20

03 Feb '20

Hi Dovecot folks, I recently reported this bug in Debian (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=939442) after a stretch -> buster upgrade, however since I can reproduce it using the latest 2.4 from git I thought perhaps I ought to take it straight upstream. Here's the text I provided Debian about the issue, and below that I've included a syslog snip showing the same issue in a fresh build of git master as well. mail_filter_out seems to fully panic (backtrace below), whereas mail_filter is just behaving oddly. Thanks :) ***************************** Package: dovecot-core Version: 2.3.4.1-5+deb10u1 Severity: important Hi Dovecot maintainers, After upgrading from the latest stretch version of dovecot-core (2.2.27-3+deb9u5) to buster (2.3.4.1-5+deb10u1) the "mail filter" interface seems to have broken. This problem is also present in the sid version (2.3.7.2-1). Mail filter is used to run the content of messages being stored or retrieved through an arbitrary process. The interface used for the external process is described here: https://wiki.dovecot.org/Plugins/MailFilter It is expected that the process will be called with the arguments specified in the configuration then supplied a message body on stdin, however since upgrading it seems that the arguments are not being correctly passed, instead being all jammed in to $1 and separated by tabs, rather than expanded in to argv. Even stranger the first few lines of the body of the message appear in argv rather than in stdin... These reproduction steps were carried out on a fresh install of Debian Buster with the "dovecot-core" package installed initially from stretch then upgraded to the buster version, however this behaviour has also been observed on stretch systems after upgrading to buster. To reproduce, drop this minimalised config in to /etc/dovecot/conf.d/repro.conf: # Minimal maildir setup for LDA first_valid_uid = 0 mail_location = maildir:~/Maildir userdb { driver = static args = allow_all_users=yes uid=mail gid=mail home=/tmp/%u } # Mail filter configuration mail_plugins = $mail_plugins mail_filter plugin { mail_filter = mail-filter read %u mail_filter_out = mail-filter write %u } service mail-filter { executable = script /usr/local/bin/mail-filter.sh user = dovecot unix_listener mail-filter { mode = 0666 } } And create the example script in /usr/local/bin/mail-filter.sh with executable permissions: #!/bin/bash echo "arguments were: $@" 1>&2 tempfile=$(mktemp) cat > "${tempfile}" cat "${tempfile}" rm "${tempfile}" This script simply logs the arguments it recieves, writes stdin to a file then retrieves it again from the file. Here's a sample session of me using dovecot-lda to pass a message through this script, then retrieve it again with doveadm: root@scratch-b6fd8bdd:~# /usr/lib/dovecot/dovecot-lda -d fincham@scratch-b6fd8bdd -f 'Test <scratch-b6fd8bdd>' < /root/test.eml root@scratch-b6fd8bdd:~# doveadm fetch -u 'fincham@scratch-b6fd8bdd' hdr subject Test hdr: To: Test <fincham@scratch-b6fd8bdd> From: Test <fincham@scratch-b6fd8bdd> Subject: This is the test e-mail root@scratch-b6fd8bdd:~# tail /var/log/syslog | grep mail-filter Sep 5 00:07:48 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: write fincham@scratch-b6fd8bdd Sep 5 00:07:50 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: read fincham@scratch-b6fd8bdd However after upgrading dovecot to the buster version this process no longer works, and the logged argv of the example script is very strange: root@scratch-b6fd8bdd:~# apt dist-upgrade Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following NEW packages will be installed: liblua5.3-0 libsodium23 ssl-cert The following packages will be upgraded: dovecot-core [...] root@scratch-b6fd8bdd:~# /usr/lib/dovecot/dovecot-lda -d fincham@scratch-b6fd8bdd -f scratch-b6fd8bdd < /root/test.eml Aborted root@scratch-b6fd8bdd:~# doveadm fetch -u 'fincham@scratch-b6fd8bdd' hdr subject Test hdr: Hello! doveadm(fincham@scratch-b6fd8bdd): Error: Mailbox INBOX: UID=1: read() failed: Cached message size larger than expected (115 > 7, box=INBOX, UID=1) (read reason=mail stream) doveadm(fincham@scratch-b6fd8bdd): Error: Corrupted record in index cache file /tmp/fincham(a)scratch-b6fd8bdd/Maildir/dovecot.index.cache: UID 1: Broken physical size in mailbox INBOX: read() failed: Cached message size larger than expected (115 > 7, box=INBOX, UID=1) doveadm(fincham@scratch-b6fd8bdd): Error: read() failed: Cached message size larger than expected (115 > 7, box=INBOX, UID=1) doveadm(fincham@scratch-b6fd8bdd): Error: fetch(hdr) failed for box=INBOX uid=1: read() failed: Cached message size larger than expected (115 > 7, box=INBOX, UID=1) root@scratch-b6fd8bdd:~# tail /var/log/syslog | grep mail-filter Sep 5 00:07:48 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: write fincham@scratch-b6fd8bdd Sep 5 00:07:50 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: read fincham@scratch-b6fd8bdd Sep 5 00:12:11 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: write#011fincham@scratch-b6fd8bdd#011 To: Test <fincham@scratch-b6fd8bdd> From: Test <fincham@scratch-b6fd8bdd> Subject: This is the test e-mail Sep 5 00:12:14 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: read#011fincham@scratch-b6fd8bdd#011 To: Test <fincham@scratch-b6fd8bdd> From: Test <fincham@scratch-b6fd8bdd> Subject: This is the test e-mail -- Package-specific info: dovecot configuration --------------------- # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-5-cloud-amd64 x86_64 Debian 10.0 # Hostname: scratch-b6fd8bdd first_valid_uid = 0 mail_location = maildir:~/Maildir mail_plugins = " mail_filter" mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { mail_filter = mail-filter read %u mail_filter_out = mail-filter write %u } service mail-filter { executable = script /usr/local/bin/mail-filter.sh unix_listener mail-filter { mode = 0666 } user = dovecot } ssl = no userdb { driver = passwd } userdb { args = allow_all_users=yes uid=mail gid=mail home=/tmp/%u driver = static } -- System Information: Debian Release: 10.0 APT prefers stable-updates APT policy: (500, 'stable-updates'), (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 4.19.0-5-cloud-amd64 (SMP w/2 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages dovecot-core depends on: ii adduser 3.118 ii libapparmor1 2.13.2-10 ii libbz2-1.0 1.0.6-9.1 ii libc6 2.28-10 ii libexttextcat-2.0-0 3.4.5-1 ii libicu63 63.1-6 ii liblua5.3-0 5.3.3-1.1 ii liblz4-1 1.8.3-1 ii liblzma5 5.2.4-1 ii libpam-runtime 1.3.1-5 ii libpam0g 1.3.1-5 ii libsodium23 1.0.17-1 ii libssl1.1 1.1.1c-1 ii libstemmer0d 0+svn585-1+b2 ii libwrap0 7.6.q-28 ii lsb-base 10.2019051400 ii openssl 1.1.1c-1 ii ssl-cert 1.0.39 ii ucf 3.0038+nmu1 ii zlib1g 1:1.2.11.dfsg-1 dovecot-core recommends no packages. Versions of packages dovecot-core suggests: pn dovecot-gssapi <none> pn dovecot-imapd <none> pn dovecot-ldap <none> pn dovecot-lmtpd <none> pn dovecot-lucene <none> pn dovecot-managesieved <none> pn dovecot-mysql <none> pn dovecot-pgsql <none> pn dovecot-pop3d <none> pn dovecot-sieve <none> pn dovecot-solr <none> pn dovecot-sqlite <none> pn dovecot-submissiond <none> ii ntp 1:4.2.8p12+dfsg-4 Versions of packages dovecot-core is related to: ii dovecot-core [dovecot-common] 1:2.3.4.1-5+deb10u1 pn dovecot-dev <none> pn dovecot-gssapi <none> pn dovecot-imapd <none> pn dovecot-ldap <none> pn dovecot-lmtpd <none> pn dovecot-managesieved <none> pn dovecot-mysql <none> pn dovecot-pgsql <none> pn dovecot-pop3d <none> pn dovecot-sieve <none> pn dovecot-sqlite <none> -- no debconf information ***************************** And here it is on 2.4 from git: Sep 5 02:18:10 scratch-b6fd8bdd dovecot: master: Dovecot v2.4.devel () starting up for imap, pop3, lmtp Sep 5 02:18:10 scratch-b6fd8bdd systemd[1]: Started LSB: Dovecot init script. Sep 5 02:18:12 scratch-b6fd8bdd dovecot: mail-filter: Error: arguments were: write#011fincham@scratch-b6fd8bdd#011 To: Test <fincham@scratch-b6fd8bdd> From: Test <fincham@scratch-b6fd8bdd> Subject: This is the test e-mail Sep 5 02:18:12 scratch-b6fd8bdd dovecot: lda(fincham@scratch-b6fd8bdd)<4213><94f8BWRwcF11EAAA/w3Ckw>: Panic: file ostream.c: line 201 (o_stream_flush): assertion failed: (stream->stream_errno != 0) Sep 5 02:18:12 scratch-b6fd8bdd dovecot: lda(fincham@scratch-b6fd8bdd)<4213><94f8BWRwcF11EAAA/w3Ckw>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xef5fb) [0x7f85ed0715fb] -> /usr/lib/dovecot/libdovecot.so.0(+0xef661) [0x7f85ed071661] -> /usr/lib/dovecot/libdovecot.so.0(+0x4e808) [0x7f85ecfd0808] -> /usr/lib/dovecot/libdovecot.so.0(+0x52ec1) [0x7f85ecfd4ec1] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x96) [0x7f85ed1b24d6] -> /usr/lib/dovecot/libdovecot-lda.so.0(+0x4395) [0x7f85ed294395] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_finish+0x82) [0x7f85ed189e52] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0xd9) [0x7f85ed17c409] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_copy+0x52) [0x7f85ed1aea02] -> /usr/lib/dovecot/libdovecot-lda.so.0(+0x4221) [0x7f85ed294221] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x5c1bc) [0x7f85ed18a1bc] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver_save+0x1b0) [0x7f85ed295390] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x2f6) [0x7f85ed295ad6] -> /usr/libexec/dovecot/dovecot-lda(main+0x81b) [0x560cba869f4b] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f85ecdde09b] -> /usr/libexec/dovecot/dovecot-lda(_start+0x2a) [0x560cba86a65a]

2 3

Sieve prevents tigger of quota-warning
by Tom Sommer 31 Jan '20

31 Jan '20

Hi All I have a few reports from Sieve users, that they are not receiving quota warnings. It appears that when a mailbox is reaching quota, and sieve is responsible for moving mail the mail due to a rule, the quota-warning directive is not triggered? It works fine if Sieve is not part of the flow, and lmtp is alone in handling mail saving. Does anyone have a fix or insights into this? 2.3.9.2 Thanks -- Tom

2 1

Split storage & solr
by Daniel Miller 30 Jan '20

30 Jan '20

Is it possible to setup the following: 1. Primary mailserver with SMTP solutions and Dovecot. 2. Primary server will store recent mails. 3. Secondary server for archival storage and Solr. So I'm not looking for a distributed cluster - simply a splitting of designated functions. Clients would only be connecting to the primary server. I know Dovecot supports an alternate storage location - that's what made me think this was possible. What I'm more uncertain of is Solr. My (probably flawed) thinking is to take advantage of a low-cost high-availability cloud server to provide primary email services. Dovecot, Postfix, and even ASSP don't have significant memory or processor requirements by themselves - especially for my smaller user base. However - storage in the cloud can be at a premium. Therefore I'm thinking of continuing to self-host the archives. And my own server has the raw power & memory to handle Solr easily. What's triggering this is our ISP's quality has been deteriorating - and the alternates don't appear much better. Initial visualization would have a VPN/SSH connection between the servers, and NFS mounting my storage to the cloud server for archives. If our connection drops - in theory "current" mail handling is unaffected. My concerns/questions: 1. If Dovecot is unable to reach the remote Solr - upon re-connection will Solr be told about the new messages to index? Or do I need to setup a periodic re-scan? 2. Is there a "better" method of accessing the archive storage area than NFS? Either a different network file system or is there a way to do it with Dovecot directly? 3. What am I not taking into consideration in this setup that you think will be a problem? --- Daniel

2 1