October 2017 - dovecot

Question regarding replication - duplicate emails
by Remko Lodder 21 Oct '17

21 Oct '17

Dear Dovecot and community, We run a small email service for our customers, based on two machines that are made “redundant or clustered” by using the replication feature of Dovecot. This works well, for most emails. Sometimes the following happends: Email to our support database arrives at the inbox. Every period a cronjob looks into that mailbox and parses the information and makes a support ticket from that message. The cronjob deletes the email afterwards and sees whether there are more new mails or not. In between the deletion and the (next) check, the email gets “resynced” again from the “other machine”, and is seen as a new email by the cronjob and gets handled again. So basically we get two tickets for one email. After that second run the email is no longer replicated. This behaviour can also be seen via webmail and email clients, this suggests that the replication might not be aware (enough) that the replication is occurring or that an item already had been synced from A to B (and does not need to get back from B to A after A deleted it). Is there a way to fiddle with the acknowledgement timing or give the processes some more space/time to get on par with eachother? Below is the configuration of machine B, they are synchronised through puppet, so only the hostname and IP addresses are different. (so for replication, A has: tcps:hostname_of_b:12346 and B has tcps:hostname_of_a:12346). Also note that HAproxy is prepared but not in use at all. # 2.2.33.1 (e9afa7f18): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.20 (7cd71ba) # OS: FreeBSD 11.1-RELEASE amd64 auth_mechanisms = plain login disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it haproxy_trusted_networks = IPv4_Haproxy IPv6_Haproxy lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_save_to_detail_mailbox = yes mail_fsync = always mail_location = sdbox:~/sdbox mail_plugins = " quota notify replication" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = . } passdb { driver = pam } plugin { imapsieve_mailbox1_before = file:/usr/local/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/usr/local/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_replica = tcps:hostname_of_machine_a:12346 sieve = ~/.dovecot.sieve sieve_after = /usr/local/etc/dovecot/sieve/after/ sieve_before = /usr/local/etc/dovecot/sieve/global/ sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_vacation_dont_check_recipient = yes } postmaster_address = postmaster(a)ourdomain.tld protocols = imap pop3 lmtp sieve replication_max_conns = 100 service aggregator { fifo_listener replication-notify-fifo { mode = 0666 } unix_listener replication-notify { mode = 0666 } } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service doveadm { inet_listener { port = 12346 ssl = yes } } service imap-login { inet_listener imap_haproxy { haproxy = yes port = 10143 } inet_listener imaps_haproxy { haproxy = yes port = 10144 ssl = yes } service_count = 1 } service imap { process_limit = 1024 } service lmtp { inet_listener lmtp { address = IPv4, IPv6, 127.0.0.1 ::1 port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3 { process_limit = 1024 } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } ssl_ca = </usr/local/etc/letsencrypt/live/host.ourdomain.tld/fullchain.pem ssl_cert = </usr/local/etc/letsencrypt/live/host.ourdomain.tld/cert.pem ssl_client_ca_file = /usr/local/certificates/letsencrypt-ca.pem ssl_key = # hidden, use -P to show it ssl_protocols = !SSLv2 !SSLv3 userdb { driver = passwd } verbose_proctitle = yes protocol lmtp { auth_username_format = %n mail_plugins = quota sieve replication notify postmaster_address = postmaster(a)ourdomain.tld } protocol lda { mail_plugins = " quota notify replication sieve replication notify" } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 50 mail_plugins = " quota notify replication imap_quota imap_sieve" }

2 3

Re: IMAP stops responding.
by Joseph Tam 20 Oct '17

20 Oct '17

S?ren Peter Skou <sps(a)DanskKabelTV.dk> writes: > I've experienced that IMAP/IMAPS stops responding. To restore service > there is only one way, restart Dovecot completely. This leads to > services being interrupted for some people, so it seems to only affect > some of the users on the server. But POP3/POP3s is still running > happily. Also, it happens more as we approach Rush Hour. Help us help you: the problem is most likely pointed out in the log files. Also, a dump of your configurations (doveconf -n) would help. Offhand, it seems you've hit an IMAP resource limit, such as mail_max_userip_connections, or perhaps process/memory limits (e.g. process_limit, vsz_limit, etc.). Joseph Tam <jtam.home(a)gmail.com>

1 0

fts_solr: Indexing failed: 500 Server Error
by Kirill Ponomarev 20 Oct '17

20 Oct '17

Hi, running dovecot 2.2.33.2 with solr 7.1.0 (with new XML scheme instead of JSON) on FreeBSD produces this error message: Oct 20 20:58:29 krion postfix/smtpd[13371]: disconnect from mail-yw0-f169.google.com[209.85.161.169] ehlo=2 +starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7 Oct 20 20:58:29 krion dovecot: indexer-worker(krion(a)krion.cc): Error: fts_solr: Indexing failed: 500 Server Error Oct 20 20:58:29 krion dovecot: indexer-worker(krion(a)krion.cc): Error: fts_solr: Indexing failed: 500 Server Error Oct 20 20:58:29 krion dovecot: indexer-worker(krion(a)krion.cc): Error: Mailbox INBOX: Transaction commit failed: FTS +transaction commit failed: backend deinit (attempted to index 1 messages (UIDs 166..166)) Oct 20 20:58:30 krion dovecot: imap(krion(a)krion.cc): Connection closed (IDLE finished 451.007 secs ago) in=43 +out=812 K.

1 0

IMAP stops responding.
by Søren Peter Skou 20 Oct '17

20 Oct '17

Hiya all, I've experienced that IMAP/IMAPS stops responding. To restore service there is only one way, restart Dovecot completely. This leads to services being interrupted for some people, so it seems to only affect some of the users on the server. But POP3/POP3s is still running happily. Also, it happens more as we approach Rush Hour. All of this leads me to believe that we're facing a problem with clients somehow. I've searched the net for the problem, and dating back I've found references to Apple units being the culprit. So we've taken an old Apple client, and tried to see how many sessions it actually starts, the result of that test was that regardless what we tried it only opened one extra session. A bit of information, Debian OS : dovecot-core 2:2.2.27-1~auto+44 amd64 secure POP3/IMAP server - core files dovecot-imapd 2:2.2.27-1~auto+44 amd64 secure POP3/IMAP server - IMAP daemon dovecot-mysql 2:2.2.27-1~auto+44 amd64 secure POP3/IMAP server - MySQL support dovecot-pop3d 2:2.2.27-1~auto+44 amd64 secure POP3/IMAP server - POP3 daemon dovecot-sieve 2:2.2.27-1~auto+44 amd64 secure POP3/IMAP server - Sieve filters support the Config itself is fairly simple, lookup through MySQL to find username/password etc - I don't really see any obvious option. Also, limiting users/sessions per IP isn't really a good idea as we have a great deal of customers behind NAT. Med venlig hilsen/Best regards Søren P. Skou Netværkstekniker [logo] Teglholmsgade 1 0900 København C Tlf: 69 12 12 12 E-mail: sps(a)danskkabeltv.dk<mailto:sps@danskkabeltv.dk> Web: www.danskkabeltv.dk<http://www.danskkabeltv.dk/>

2 1

v2.2.33.2 released
by Timo Sirainen 20 Oct '17

20 Oct '17

https://dovecot.org/releases/2.2/dovecot-2.2.33.2.tar.gz https://dovecot.org/releases/2.2/dovecot-2.2.33.2.tar.gz.sig One more patch release with some fixes: - doveadm: Fix crash in proxying (or dsync replication) if remote is running older than v2.2.33 - auth: Fix memory leak in %{ldap_dn} - dict-sql: Fix data types to work correctly with Cassandra

1 0

Re: HTTPS for http://xi.dovecot.fi/debian/
by Aki Tuomi 20 Oct '17

20 Oct '17

> On October 20, 2017 at 12:37 PM Florent B <florent(a)coppint.com> wrote: > > > Hi, > > We use Dovecot packages from http://xi.dovecot.fi/debian/. > > Could it be possible to serve it with HTTPS ? > > Thank you. > > Florent Hi! It has now https enabled with valid certificate. Aki

1 0

quota only for oungoing
by Aidar Kamalov 20 Oct '17

20 Oct '17

Hello! I setup quota and quota warnings for my mail system(dovecot+exim). By I want to reject only outgoing messages if quota exceeded. Is it possible to congi dovecot to not reject incoming messages? I use dovecot-lda for mail delivery. ldap_delivery: driver = pipe command = /usr/lib/dovecot/deliver -d "${lookup ldap{user=**********))(mai -f $sender_address -a $original_local_part@$original_domain message_prefix = message_suffix = delivery_date_add envelope_to_add return_path_add log_output return_output user = mail group = mail temp_errors = 64 : 69 : 70: 71 : 72 : 73 : 74 : 75 : 78 -- Aydar A. Kamalov

2 5

Iteration for ldap userdb does not work
by Holger A. Brinkhaus 20 Oct '17

20 Oct '17

Hi, I have a working ldap authentification for dovecot (with ldap based userdb query) for my Active Directory but iterations by doveadm -A runs in timeout. I tried now for three days and get it not to work. Best regards Holger /etc/dovecot/dovecot-ldap-userdb.conf is symlink to /etc/doveconf/doveconf-ldap.conf debug_level = 0 uris = ldaps://server1.office.hb-os.de:636 dn = cn=BINDUSER,cn=Users,dc=SUBDOM,dc=DOM,dc=de dnpass = MYPASSWORD auth_bind = yes ldap_version = 3 deref = never base = cn=Users,dc=SUBDOM,dc=DOM,dc=de scope = subtree user_attrs = =home=/usr/data/vmail/%d/%n, =mail=maildir:/usr/data/vmail/%d/%n/Maildir user_filter = (&(objectClass=person)(mail=%u)) pass_filter = (&(objectClass=person)(mail=%u)) iterate_attrs = mail=user iterate_filter = (objectClass=person) doveconf -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.9 auth_mechanisms = plain login base_dir = /var/run/dovecot/ default_vsz_limit = 512 M dict { expire = mysql:/etc/dovecot/dovecot-dict-expire.conf.ext } first_valid_uid = 100 imap_idle_notify_interval = 16 mins lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_gid = vmail mail_home = /usr/data/vmail/%d/ mail_location = maildir:~/Maildir:LAYOUT=fs mail_plugins = expire mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { antispam_backend = pipe antispam_debug_target = syslog antispam_pipe_program = /usr/data/vmail/spampipe.sh antispam_pipe_program_notspam_arg = --ham antispam_pipe_program_spam_arg = --spam antispam_pipe_tmp_dir = /tmp antispam_spam = Junk;SPAM;Spam antispam_trash = Trash expire = Trash expire2 = Trash/* expire3 = Junk expire_dict = proxy::expire sieve = /usr/data/vmail/%u/sieve/active-script.sieve sieve_before = /usr/data/vmail/sieve/spam-global.sieve sieve_dir = /usr/data/vmail/%d/%n/sieve/scripts/ sieve_max_redirects = 4 sieve_max_script_size = 1M sieve_quota_max_scripts = 128 } postmaster_address = ME(a)DOMAIN.de protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = postfix mode = 0600 user = postfix } user = vmail } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 10 service_count = 1 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl_cert = </etc/ssl/certs/MY_CERT.crt ssl_key = </etc/ssl/private/MY_CERT.key userdb { args = /etc/dovecot/dovecot-ldap-userdb.conf driver = ldap } protocol lmtp { mail_plugins = expire sieve } protocol lda { mail_plugins = expire sieve } protocol imap { mail_max_userip_connections = 10 mail_plugins = expire antispam }

2 2

Dovecot >=2.2.29 + Filesystem quota = incorrect storage information
by Macka 18 Oct '17

18 Oct '17

Hi, I have problem with quota storage information in dovecot-2.2.29 (and newer versions, too - 2.2.30.2). I use FS quota backend. ............................ # 2.2.29.1 (e0b76e3): /etc/dovecot/dovecot.conf # OS: Linux 4.4.68-1 x86_64 mail_plugins = quota plugin { quota = fs:User quota:mount=/poczta:user quota2 = fs:Group quota:mount=/poczta:group } protocol imap { mail_max_userip_connections = 10 mail_plugins = "imap_quota mail_log notify antispam" } ................................. After upgrade from 2.2.27 to 2.2.29 (newest 2.2.30.2 too), dovecot returns incorrect informations about quota storage: # doveadm quota get -u myusername Quota name Type Value Limit % User quota STORAGE 10960896 100000 10960 or # ( echo "A1 GETQUOTAROOT INBOX"; echo "A2 logout" ) | /usr/lib64/dovecot/imap -u myusername * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE NOTIFY SPECIAL-USE QUOTA] Logged in as myusername imap(myusername): Info: Logged out in=31 out=588 * QUOTAROOT INBOX "User quota" * QUOTA "User quota" (STORAGE 10960896 100000) A1 OK Getquotaroot completed (0.001 + 0.000 secs). * BYE Logging out A2 OK Logout completed (0.001 + 0.000 secs). Values are in BYTES and Limit in BLOCKS which gives insane information in % Other server with older version 2.2.27 returns correct numbers (both in BLOCKS) # doveadm quota get -u username2 Quota name Type Value Limit % user STORAGE 558336 614400 90 Is this a bug in new versions? How can I fix this? -- macka

3 6

Re: Last Login example
by Siarhei Chystsiakou 18 Oct '17

18 Oct '17

My config: config plugins: # cat /etc/dovecot/conf.d/90-last_login.conf > dict { > lastlogin = mysql:/etc/dovecot/dict.conf.d/90-lastlogin.conf > } > plugin { > last_login_dict = proxy::lastlogin > } config db connector: # cat /etc/dovecot/dict.conf.d/90-lastlogin.conf > connect=host=localhost dbname=mail_user user=userDB password=Pa$$w0rd > map { > pattern = shared/last-login/$user > table = mailbox > username_field = username > value_field = lastlogin > value_type = uint > fields { > username = $user > } > } config imap and pop3 # cat /etc/dovecot/dict.conf.d/20-imap.conf > protocol imap { > mail_plugins = $mail_plugins quota imap_quota imap_acl last_login > } > # cat /etc/dovecot/dict.conf.d/20-pop3.conf > protocol imap { > mail_plugins = $mail_plugins last_login > } In the database in the table with users to add a column. If you have questions, I'll try to answer 2017-10-14 20:20 GMT+02:00 Jorge Bastos <mysql.jorge(a)decimal.pt>: > Howdy, > > > > Where can i find a configuration example for the last login plugin, that > fits for latest dovecot version? > > Thanks in advanced! > > > > Jorge, >

1 0