August 2017 - dovecot

weakforced
by Mark Moseley 17 Aug '17

17 Aug '17

I've been playing with weakforced, so it fills in the 'fail2ban across a cluster' niche (not to mention RBLs). It seems to work well, once you've actually read the docs :) I was curious if anyone had played with it and was *very* curious if anyone was using it in high traffic production. Getting things to 'work' versus getting them to work *and* handle a couple hundred dovecot servers is a very wide margin. I realize this is not a weakforced mailing list (there doesn't appear to be one anyway), but the users here are some of the likeliest candidates for having tried it out. Mainly I'm curious if weakforced can handle serious concurrency and whether the cluster really works under load.

4 5

Ubuntu 16.04 dovecot-core requires deprecated ntpdate
by Michael Fox 17 Aug '17

17 Aug '17

I'm building a new Ubuntu 16.04 machine, including Dovecot. When I select the dovecot-core package in Synaptic, it also wants to install ntpdate. Problem: ntpdate has been replaced in Ubuntu with timedatectl. In fact, if ntpdate exists on the machine, ntpd will not work properly. See: https://help.ubuntu.com/lts/serverguide/NTP.html See: https://askubuntu.com/questions/769651/ntp-failed-to-synchronize-time See: https://ubuntu101.co.za/ubuntu/fix-ntp-on-ubuntu-16-starting-crashing/ http://www.ubuntugeek.com/install-and-configure-network-time-protocol-ntp-se rverclients-on-ubuntu-16-04-server.html . and more Indeed, I had to remove ntpdate in order to make ntpd work. I don't want to break ntpd. Now the dovecot-core package evidently requires ntpdate. I can't imagine why this dependency would exist. And I presume this dependency will prevent me from removing ntpdate after I install dovecot-core. Is the Debian package maintainer on the list? I don't know what to do. Any suggestions? Thanks in advance, Michael

3 4

Can't recursively delete mailboxes
by Murray T. | Dreamscape 17 Aug '17

17 Aug '17

Hi, This appears to be some sort of dovecot bug. When I use the -r option to recursively delete a mailbox and sub-folders, it removes the sub-folders but not the inbox. It gives me the error below: doveadm mailbox delete -u denise(a)lifelinetechnique.com.au -r -s inbox doveadm(denise(a)lifelinetechnique.com.au): Error: Can't delete mailbox inbox: INBOX can't be deleted. Our user database is in ldap and I am using dovecot-ee v2:2.2.31.2-1. Could you please acknowledge that this is a bug or let me know the correct way of recursively deleting a mailbox as there is nothing in the man page about the -r option. Thanks Murray Regards, Murray T. Level 3 Project Manager P: +61 8 9422 0894 F: +61 8 9422 0801 www.DreamscapeNetworks.com<http://www.dreamscapenetworks.com/> [dreamscape]<http://www.dreamscapenetworks.com/> DISCLAIMER: This e-mail and/or attachments are confidential and may also be privileged. If you are not the named recipient, please notify the sender immediately and do not disclose the contents to other persons or use it for any purpose or store or copy the information in any medium. Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Dreamscape Networks. Copyright (c) 2017 www.DreamscapeNetworks.com<http://www.dreamscapenetworks.com/>, All rights reserved.

3 3

Re: v2.2.32 release candidate released
by Joseph Tam 17 Aug '17

17 Aug '17

Timo Sirainen wrote: > There are various changes in this release that can be used to significantly reduce disk IO with: > 1) NFS storage especially, but I guess also other remote filesystems and even some with local disks > 2) When mail storage and INDEX storage are separated Thanks for these changes! Big win for my setup. My servers are not overly stressed, but how much performance gain (using any metric you choose) can one expect from these changes? > + mail_location can now include VOLATILEDIR=<path> parameter. This > is used for creating lock files and in future potentially other > files that don't need to exist permanently. The path could point to > tmpfs for example. This is especially useful to avoid creating lock > files to NFS or other remote filesystems. For example: > mail_location=sdbox:~/sdbox:VOLATILEDIR=/tmp/volatile/%2.256Nu/%u Is "/tmp/volatile" auto-created, or must be pre-created? > + mail_location's LISTINDEX=<path> can now contain a full path. > This allows storing mailbox list index to a different storage > than the rest of the indexes, for example to tmpfs. Is this in any way related to VOLATILEDIR? Can they be set to the same value without problems? > + mail_location can now include NO-NOSELECT parameter. This > automatically deletes any \NoSelect mailboxes that have no children. > These mailboxes are sometimes confusing to users. Sorry for my IMAP ignorance, but how can this situation come about? > + mail_location can now include ITERINDEX parameter. This tells Dovecot > to perform mailbox listing from the INDEX path instead of from the > mail root path. It's mainly useful when the INDEX storage is on a > faster storage. > + If mailbox_list_index_very_dirty_syncs=yes, the list index is no > longer refreshed against filesystem when listing mailboxes. This > allows the mailbox listing to be done entirely by only reading the > mailbox list index. > + Added mailbox_list_index_include_inbox setting to control whether > INBOX's STATUS information should be cached in the mailbox list > index. The default is "no", but it may be useful to change it to > "yes", especially if LISTINDEX points to tmpfs. So as I understand it, the optimzation comes about from segregating mail data information into 3 parts: raw mail, indices, and volatile components, putting them into increasingly better performing storage media. How do these I/O optimizations affect the client's view of a mailbox if their mailbox is subject to modification outside the dovecot system (e.g. procmail, mail readers directly modifies mailbox files)? Is there a trade-off between metadata consistency and performance, or it's a win-win all around? (I just saw your previous response to someone else, which I'll read more closely.) Joseph Tam <jtam.home(a)gmail.com>

2 1

v2.2.32 release candidate released
by Timo Sirainen 17 Aug '17

17 Aug '17

https://dovecot.org/releases/2.2/rc/dovecot-2.2.32.rc1.tar.gz https://dovecot.org/releases/2.2/rc/dovecot-2.2.32.rc1.tar.gz.sig There are various changes in this release that can be used to significantly reduce disk IO with: 1) NFS storage especially, but I guess also other remote filesystems and even some with local disks 2) When mail storage and INDEX storage are separated * imapc: Info-level line is logged every time when successfully connected to the remote server. This includes local/remote IP/port, which can be useful for matching against external logs. * config: Log a warning if plugin { key=no } is used explicitly. v2.3 will support "no" properly in plugin settings, but for now any value at all for a boolean plugin setting is treated as "yes", even if it's written as explicit "no". This change will now warn that it most likely won't work as intended. + Various optimizations to avoid accessing files/directories when it's not necessary. Especially avoid accessing mail root directories when INDEX directories point to a different filesystem. + mail_location can now include ITERINDEX parameter. This tells Dovecot to perform mailbox listing from the INDEX path instead of from the mail root path. It's mainly useful when the INDEX storage is on a faster storage. + mail_location can now include VOLATILEDIR=<path> parameter. This is used for creating lock files and in future potentially other files that don't need to exist permanently. The path could point to tmpfs for example. This is especially useful to avoid creating lock files to NFS or other remote filesystems. For example: mail_location=sdbox:~/sdbox:VOLATILEDIR=/tmp/volatile/%2.256Nu/%u + mail_location's LISTINDEX=<path> can now contain a full path. This allows storing mailbox list index to a different storage than the rest of the indexes, for example to tmpfs. + mail_location can now include NO-NOSELECT parameter. This automatically deletes any \NoSelect mailboxes that have no children. These mailboxes are sometimes confusing to users. + mail_location can now include BROKENCHAR=<char> parameter. This can be useful with imapc to access mailbox names that aren't valid mUTF-7 charset from remote servers. + If mailbox_list_index_very_dirty_syncs=yes, the list index is no longer refreshed against filesystem when listing mailboxes. This allows the mailbox listing to be done entirely by only reading the mailbox list index. + Added mailbox_list_index_include_inbox setting to control whether INBOX's STATUS information should be cached in the mailbox list index. The default is "no", but it may be useful to change it to "yes", especially if LISTINDEX points to tmpfs. + userdb can return chdir=<path>, which override mail_home for the chdir location. This can be useful to avoid accessing home directory on login. + userdb can return postlogin=<socket> to specify per-user imap/pop3 postlogin socket path. + cassandra: Add support for result paging by adding page_size=<n> parameter to the connect setting. + dsync/imapc, pop3-migration plugin: Strip also trailing tabs from headers when matching mails. This helps with migrations from Zimbra. + imap_logout_format supports now %{appended} and %{autoexpunged} + virtual plugin: Optimize IDLE to use mailbox list index for finding out when something has changed. - virtual plugin: A lot of fixes. In many cases it was also working very inefficiently or even incorrectly. - imap: NOTIFY parameter parsing was incorrectly "fixed" in v2.2.31. It was actually (mostly) working in previous versions, but broken in v2.2.31. - Modseq tracking didn't always work correctly. This could have caused imap unhibernation to fail or IMAP QRESYNC/CONDSTORE extensions to not work perfectly. - mdbox: "Inconsistency in map index" wasn't fixed automatically - dict-ldap: %variable values used in the LDAP filter weren't escaped. - quota=count: quota_warning = -storage=.. was never executed (try #2). v2.2.31 fixed it for -messages, but not for -storage. - imapc: >= 32 kB mail bodies were supposed to be cached for subsequent FETCHes, but weren't. - quota-status service didn't support recipient_delimiter - acl: Don't access dovecot-acl-list files with acl_globals_only=yes - mail_location: If INDEX dir is set, mailbox deletion deletes its childrens' indexes. For example if "box" is deleted, "box/child" index directory was deleted as well (but mails were preserved).

4 5

autocreate deprecated q
by voytek＠sbt.net.au 17 Aug '17

17 Aug '17

I'm getting "autocreate plugin is deprecated" I've removed 1) but still see error am I correct I don't need default IMAP folders in dovecot.conf as they come from 15-mailboxes.conf ? # grep autocreate dovecot.conf lda_mailbox_autocreate = yes mail_plugins = quota sieve autocreate mail_plugins = quota imap_quota autocreate do I just remove BOTH 'mail_plugin ... autocreate' ? 1) removed autocreate = INBOX autocreate2 = Sent autocreate3 = Trash autocreate4 = Drafts autocreate5 = Junk autosubscribe = INBOX autosubscribe2 = Sent autosubscribe3 = Trash autosubscribe4 = Drafts autosubscribe5 = Junk

2 3

Cannot login with method=GSSAPI
by Erik Haller 17 Aug '17

17 Aug '17

I am migrating an existing dovecot server to a new server. The existing server uses pam_krb5 and works with the plain and gssapi methods. The new server plain/pam_krb5 normal password authentication works. However, the gssapi (tickets) authentication is producing the following error: === Begin Error ==== imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=192.168.7.61, lip=192.168.7.97, TLS, session=<SPnD7NhWWtrAqAc9> === End Error === What is causing the "user=<>"? It should be "user=<erik>". I have been using Thunderbird SSL GSSAPI from a Debian Linux testing/buster XFCE desktop to connect to the existing server for years. When I point it to the new server, I receive the above error. ssh kerberos gssapi authentication is working fine on the new server. Most of the doveconf setting between the existing and new servers are the same. The existing server is 32 bit. The new server is 64 bit running in an LXC container. The existing server dovecot version is the same as the new server. Notes: dovecot version: 2.2.31 (65cde28) OS: Debian Linux testing/buster Arch: amd64 Client: Mozilla Thunderbird 52.2.1 (latest)

2 2

correct permissions /etc/dovecot ?
by voytek＠sbt.net.au 17 Aug '17

17 Aug '17

what permissions/ownership should /etc/dovecot/files have? keep seeing this error below, I can correct as per log, BUT, wanted first to check what it should be, rather than me 'fiddling' thanks, V SQL user is vmail first_valid_uid = 2000 last_valid_uid = 2000 # ls -al total 60 drwxr-xr-x 5 root root 4096 Aug 16 14:45 . drwxr-xr-x 82 root root 4096 Aug 16 08:34 .. drwxr-xr-x 2 root root 4096 Aug 12 21:22 conf.d -rw------- 1 vmail dovecot 0 Nov 5 2013 dovecot-master-users-password -rw------- 1 vmail dovecot 735 Aug 16 14:45 dovecot-mysql.conf -rw------- 1 vmail dovecot 491 Aug 15 23:09 dovecot-share-folder.conf -rw------- 1 vmail dovecot 320 Aug 15 23:09 dovecot-used-quota.conf -rw-r--r-- 1 root root 3414 Aug 16 14:43 dovecot.conf Aug 16 16:28:01 auth: Error: passwd-file: open(/etc/dovecot/dovecot-master-users-password) failed: Permission denied (euid=97(dovecot) egid=97(dovecot) missing +r perm: /etc/dovecot/dovecot-master-users-password, dir owned by 0:0 mode=0755) Aug 16 16:29:16 auth: Error: passwd-file: open(/etc/dovecot/dovecot-master-users-password) failed: Permission denied (euid=97(dovecot) egid=97(dovecot) missing +r perm: /etc/dovecot/dovecot-master-users-password, dir owned by 0:0 mode=0755)

3 2

Question (haven't tried yet)
by Ivan Warren 16 Aug '17

16 Aug '17

Hello, I have a question : if I have multiple servers sharing a single disk space (NFS... cluster), and a message is delivered via say LMTP or LDA to a mailbox, and an IMAP client is in IDLE mode on another server, Will it get notified of the arrival of the new message (and if yes, how ?) Thanks, --Ivan

1 0

FTS Lucene Search by Word Parts
by Webert de Souza Lima 16 Aug '17

16 Aug '17

Hello, as the dovecot documentation is only trivial for FTS Search options, I can't find information on advanced options. I have enabled it but it seems I can only search for entire words, i.e. the word Dovecot can be found by searching "dovecot", but not "dove". Is it possible to achieve this using FTS Lucene? I know that using Solr that achievable setting n-gram. Regards, Webert Lima DevOps Engineer at MAV Tecnologia *Belo Horizonte - Brasil*

1 0