May 2017 - dovecot - dovecot.org

Using Dovecot for a Geo-replicated World-wide Email Service
by Lennart Oldenburg 22 May '17

22 May '17

Hello everyone, my colleagues and I from Technische Universität Berlin are currently looking into the best way to setup a Dovecot-powered email service that is simultaneously accessible world-wide and still provides short response times by redirecting users to the geographically-closest cluster. We imagine the setup to make use of multiple data center regions in public clouds around the world, e.g. in Amazon Web Services (AWS) or Google Compute Platform (GCP). At the same time, state is considered to be shared, i.e. changes made on file system in any local cluster will have to be synchronized with all other clusters around the globe (mailbox replication) and mailboxes might be manipulated from any data center in the world, potentially at the same time. Naturally, problems arise. We have multiple questions and would be very interested in the best way to approach them based on your experiences and recommendations: 1) Which path should we take replication-wise? Please keep in mind that we are targeting replication between more than 2 clusters. Traditionally, in a rather small local setup with mostly reliable network, a distributed file system such as NFS or GlusterFS might be the best option. Unfortunately, though, when scaling to more than 2 data centers geographically-distributed around the globe, latency becomes severe and distributed file systems might not be the way to go anymore. What is your recommendation for replicating state? 2) We are mostly interested in providing short response times to client requests while maintaining global state consistency (that is, avoiding conflicts in state and not losing any user data). What are your recommendations in that direction? Any particular performance optimizations we could apply to Dovecot? 3) Is there a recommended way to measure and monitor Dovecot's service quality? For example, is there a way to export "live" metrics of a running Dovecot deployment to something such as Prometheus (https://prometheus.io)? 4) Maildir as the way to represent mailboxes and deliver emails on file system is kind of a given. I wonder, though, how well is dbox replicatable? What happens when dbox files are synchronized (e.g. via dsync) and a conflict arises? Does dbox offer a performance improvement over Maildir? 5) We perceive Dovecot to come with many optimizations to speed up "read" IMAP commands, i.e. non-state-changing ones. Is this perception correct? Or is Dovecot able to use its various index and log files for fast "write" IMAP commands (e.g. CREATE, DELETE, RENAME, APPEND, STORE, ...) as well? 6) Would DovecotPRO by Dovecot Oy offer any advantage to state replication? For example, could the Object Storage Plugin be used to replicate mailboxes through some object store service offered world-wide? 7) Is the setup we have in mind based on Dovecot a feasible solution for world-wide email comparable to services like Gmail? If not, what would you change or not do at all? Please excuse if some questions might be of rather basic nature, we are not at all Dovecot experts. I tried to gather as much information on that topic as possible from the wiki and sources found online but I might have missed an important page. In that case, please point me towards existing resources on the questions raised above. Thanks in advance and kind regards, Lennart Oldenburg

1 0

Sieve GLOBAL + Sieve LOCAL
by Luescher Claude 19 May '17

19 May '17

Hello List, I am using dovecot: ii dovecot-antispam 2.0+20130822-2 amd64 Dovecot plugins for training spam filters ii dovecot-core 1:2.2.10-1 amd64 secure POP3/IMAP server - core files ii dovecot-imapd 1:2.2.10-1 amd64 secure POP3/IMAP server - IMAP daemon ii dovecot-ldap 1:2.2.10-1 amd64 secure POP3/IMAP server - LDAP support ii dovecot-lmtpd 1:2.2.10-1 amd64 secure POP3/IMAP server - LMTP server ii dovecot-managesieved 1:2.2.10-1 amd64 secure POP3/IMAP server - ManageSieve server ii dovecot-mysql 1:2.2.10-1 amd64 secure POP3/IMAP server - MySQL support ii dovecot-pop3d 1:2.2.10-1 amd64 secure POP3/IMAP server - POP3 daemon ii dovecot-sieve 1:2.2.10-1 amd64 secure POP3/IMAP server - Sieve filters support doveconf -n | grep -i sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave editheader vnd.dovecot.duplicate sieve = ~/.dovecot.sieve sieve_after = ~/sieve_after sieve_dir = ~/sieve sieve_extensions = +editheader +vnd.dovecot.duplicate sieve_max_redirects = 20 protocols = " imap lmtp sieve pop3" service managesieve-login { inet_listener sieve { mail_plugins = " quota sieve" mail_plugins = " quota sieve" I would like to have a Global and per user based Local sieve_before file. Since doveconf does not indicate I believe my version does not support: sieve_global_path A path to a global sieve script file, which gets executed ONLY if user's private Sieve script doesn't exist, e.g. /var/lib/dovecot/default.sieve. Be sure to pre-compile this script manually using the sievec command line tool, as explained below. But what I would need is execute a global sieve_before for all users to apply some spam selection rules. Did anybody easily achieved it? An obvious workaround would be to create a script which goes through all current users mail directories and creates a sieve_before script with the settings I need and then modify the user creation to do the same when new mailboxes get created. Thank you!

3 3

lmtp segfault after upgrade
by Tom Sommer 19 May '17

19 May '17

I just upgraded our Director to 2.2.29.1 from 2.2.26, and now my dmesg and /var/log/messages are getting flooded by these errors: lmtp[45758]: segfault at 21 ip 00007fb412d3ad11 sp 00007ffe83ad2df0 error 4 in libdovecot.so.0.0.0[7fb412c95000+11c000] lmtp[46304]: segfault at 91 ip 00007f771ba64d11 sp 00007ffe830635a0 error 4 in libdovecot.so.0.0.0[7f771b9bf000+11c000] lmtp[45775]: segfault at 21 ip 00007f206ed81d11 sp 00007ffd0f0daad0 error 4 in libdovecot.so.0.0.0[7f206ecdc000+11c000] lmtp[46649]: segfault at 21 ip 00007ff4aa027d11 sp 00007ffc05841430 error 4 in libdovecot.so.0.0.0[7ff4a9f82000+11c000] lmtp[45774]: segfault at 21 ip 00007ff1f3e0bd11 sp 00007ffe0ba00340 error 4 in libdovecot.so.0.0.0[7ff1f3d66000+11c000] lmtp[45776]: segfault at 411 ip 00007f96ded95d11 sp 00007ffce6401870 error 4 in libdovecot.so.0.0.0[7f96decf0000+11c000] lmtp[46896]: segfault at 21 ip 00007fbf1aa70d11 sp 00007ffc79e22bf0 error 4 in libdovecot.so.0.0.0[7fbf1a9cb000+11c000] lmtp[46695]: segfault at 21 ip 00007fd5ccad6d11 sp 00007ffe19e759d0 error 4 in libdovecot.so.0.0.0[7fd5cca31000+11c000] lmtp[45773]: segfault at 21 ip 00007f0eb7b65d11 sp 00007fff9d02e570 error 4 in libdovecot.so.0.0.0[7f0eb7ac0000+11c000] lmtp[47101]: segfault at 21 ip 00007fa95b052d11 sp 00007fff2d5485a0 error 4 in libdovecot.so.0.0.0[7fa95afad000+11c000] lmtp[45772]: segfault at 21 ip 00007f2f52b49d11 sp 00007fff99a4a6e0 error 4 in libdovecot.so.0.0.0[7f2f52aa4000+11c000] lmtp[46595]: segfault at 21 ip 00007f0ccbdbad11 sp 00007ffd17bfe9e0 error 4 in libdovecot.so.0.0.0[7f0ccbd15000+11c000] lmtp[45770]: segfault at 21 ip 00007f94a576ed11 sp 00007ffc29c2ff00 error 4 in libdovecot.so.0.0.0[7f94a56c9000+11c000] lmtp[45769]: segfault at 21 ip 00007f34999b5d11 sp 00007ffe28818350 error 4 in libdovecot.so.0.0.0[7f3499910000+11c000] lmtp[47371]: segfault at 21 ip 00007f13c611fd11 sp 00007ffcdb000b10 error 4 in libdovecot.so.0.0.0[7f13c607a000+11c000] lmtp[47419]: segfault at 21 ip 00007f4010ff2d11 sp 00007ffd97519420 error 4 in libdovecot.so.0.0.0[7f4010f4d000+11c000] lmtp[47511]: segfault at 581 ip 00007fd5e3d21d11 sp 00007ffce61f29c0 error 4 in libdovecot.so.0.0.0[7fd5e3c7c000+11c000] lmtp[47277]: segfault at 21 ip 00007f04c3d6cd11 sp 00007ffc421da1b0 error 4 in libdovecot.so.0.0.0[7f04c3cc7000+11c000] lmtp[45771]: segfault at 21 ip 00007f55c690ed11 sp 00007fff35521fc0 error 4 in libdovecot.so.0.0.0[7f55c6869000+11c000] lmtp[47543]: segfault at 21 ip 00007f11ea946d11 sp 00007ffdd9a97690 error 4 in libdovecot.so.0.0.0[7f11ea8a1000+11c000] lmtp[47669]: segfault at 21 ip 00007f8f58abbd11 sp 00007ffdd5573220 error 4 in libdovecot.so.0.0.0[7f8f58a16000+11c000] lmtp[46959]: segfault at 41 ip 00007ff9d6f3ed11 sp 00007ffe28dc8ea0 error 4 in libdovecot.so.0.0.0[7ff9d6e99000+11c000] Any ideas? -- Tom

4 17

per user procmail filtering and dovecot-lda
by James Nord 18 May '17

18 May '17

Hi all, I recently migrated my system (postfix/Dovecot)from mbox to Maildir. Almost everything is working (phone and thunderbird show all my mail and folders with mail) However I am stuck on my .procmailrc rules # cat /etc/procmailrc SENDMAIL=/usr/sbin/sendmail SENDMAILFLAGS=-oi LOGFILE=/var/log/procmail.log ORGMAIL=${HOME}/Maildir/ DEFAULT=${ORGMAIL} MAILDIR=${HOME}/Maildir/ DELIVER="/usr/lib/dovecot/deliver -d $LOGNAME" # next line is to support user .procmailrc files INCLUDERC=$HOME/.procmailrc :0 w | $DELIVER # cat ~/.procmailrc :0 w * ^List-ID:.*jenkinsci-dev.googlegroups.com | $DELIVER -m mailing-lists/jenkinsci-dev :0 w * ^List-ID:.*jenkinsci-users.googlegroups.com | $DELIVER -m mailing-lists/jenkinsci-users However dovecot-lda complains that the mailboxes "mailing-lists/jenkinsci-users", "mailing-lists/jenkinsci-dev" etc do not exist so delivers to the INBOX. Everything I have read says that the mailbox specified by -m is the name as seen by the mail client, and this is what the client sees (see image). I have tried changing the mailbox to INBOX/mailing-lists/... inbox/mailing-lists/... but the same still happens. So I am missing something somewhere and my google foo is unfortunately letting me down this time. Is there someone that can help me? (I'm not subscribed to the list so a CC to me would be appreciated) Regards /James

5 7

Re: Sieve GLOBAL + Sieve LOCAL
by Joseph Tam 17 May '17

17 May '17

Luescher Claude <stargate(a)tango.lu> writes: > I am using dovecot: > ... > ii dovecot-sieve 1:2.2.10-1 amd64 > secure POP3/IMAP server - Sieve filters support > ... > doveconf -n | grep -i sieve > Since doveconf does not indicate I believe my version does not support: > > sieve_global_path I see mentions of this configuration variable in 7 year old dovecot posts, so it's been around a long time. You should do doveconf -a | grep sieve_global_path since "-n" won't list unchanged (defaulted) configuration values. Joseph Tam <jtam.home(a)gmail.com>

1 0

Conditionally Filter Mails During IMAP Fetch
by Ory Drilon 17 May '17

17 May '17

Hi, is there a way to arbitrarily filter out mails at the moment the IMAP request is made, based on properties of the IMAP session and the mail itself? All the filtering methods I find occur during delivery time.

2 1

Re: ACL and virtual user
by Davide Marchi 15 May '17

15 May '17

...I had under my eyes the solution I have not seen: * user=foo lrw Public user=bar lrwstipekxa Public/* user=bar lrwstipekxa I was enough (in my case) add the initial "*" wildcards, like this: * user=info(a)mydomain.com lrwsi where the "*" wildcards means all groups. I have verified and so the configuration is perfect, the user can not delete their own messages The fact remains that on the wiki dovecot ACL these steps, in the case of virtual users, are not clear, at least in my opinion. :-) I hope at least that my experience can be of assistance to other users. Many many thanks! Davide

1 0

ACL and virtual user
by Davide Marchi 15 May '17

15 May '17

Hi friends, I would like to remove write rights for a specific virtual user/email address. I've read the wiki "https://wiki2.dovecot.org/ACL" but there is nothing about it. So I started reading and rereading some old ML, like this: https://www.dovecot.org/list/dovecot/2006-June/013683.html where they talk about something like that but I would like to implement current guidelines for the "Global ACL file" and for virtual user. So I've try: * user=foo lrw Public user=bar lrwstipekxa Public/* user=bar lrwstipekxa But in my case for virtual user, so I tried (obviously I changed the email address with a generic one): -------------- #user=info(a)mydomain.com lrwsi -------------- Log: May 15 10:40:44 server dovecot: imap(info(a)mydomain.com): Error: Global ACL file /etc/dovecot/dovecot-acl line 1: Unknown ID 'lrwsi' -------------- #owner #user=info at mydomain.com lrwsi -------------- Log: May 15 10:47:58 server dovecot: imap(info(a)mydomain.com): Error: Global ACL file /etc/dovecot/dovecot-acl line 1: Missing ACL rights -------------- #user=info at mydomain.com lrwsi -------------- Log: May 15 10:51:57 server dovecot: imap(info(a)mydomain.com): Error: Global ACL file /etc/dovecot/dovecot-acl line 2: Unknown ID 'at' But none of these gave results, could you explain better or gime me an example? Many many thanks! Davide Italy

1 0

[BUG] doveadm kick doesn't play well with hibernate
by Christian Balzer 15 May '17

15 May '17

Hello, Dovecot 2.2.27 (Debian Jessie backports). When issuing a "doveadm kick" for a specific user I was greeted by: --- "warning: other connections would also be kicked from following users:" --- and a list of 22odd thousand users. As it turns out, kick wants to smack the hibernation proces(ses) that have hibernated sessions for this user, obviously the wrong approach here. If there's doveadm command to wake up and thus un-hibernate sessions for a specific user this could be used as a workaround but a quick glance suggest no such thing. This functionality is absolutely crucial for us as we need this for our custom mailbox migration functionality. If there's no workaround or fix within 2 months I'll be forced to disable hibernation again (with a heavy heart). Regards, Christian -- Christian Balzer Network/Systems Engineer chibi(a)gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/

2 2

Dovecot 2.2.29.1 + VPopMail 5.4.33
by Nicholas Chua 12 May '17

12 May '17

Hi all, I have a fresh installation of a Vpopmail 5.4.33 + Dovecot 2.2.29.1 + SQL As usual I followed the AuthDatabase/VPopMail wiki password_query = SELECT CONCAT(pw_name, '@', pw_domain) AS user, \ pw_clear_passwd AS password \ FROM vpopmail \ WHERE pw_name = '%n' AND pw_domain = '%d' user_query = SELECT pw_dir as home, \ 89 AS uid, 89 AS gid \ FROM vpopmail \ WHERE pw_name = '%n' AND pw_domain = '%d' For VPopMail, I prefer to have an individual table in the vpopmail database for each domain. So my structure will be something like this vpopmail.abc_com vpopmail.xyz_com To achieve this I add this configure when compiling VPopMail --disable-many-domains But dovecot will not read vpopmail.abc_com, instead it keep looking for vpopmail.vpopmail auth-worker(8643): Warning: mysql: Query failed, retrying: Table 'vpopmail.vpopmail' doesn't exist If I remove --disable-many-domains in the configuration, I have no issue. Is there a new password and user query to have this working? The last working version I had installed is Dovecot 2.2.15 Thank you nic

1 0