- dovecot - dovecot.org

Mail received but not indexed?
by Marc Roos 28 Sep '19

28 Sep '19

I have recently that some users are complaining about that they are not getting emails. While I see that they are delivered and in the inbox. When I do a doveadm force-resync -u xxxx INBOX it resolves the problem. I think this is something recent. Where/how should I resolve this issue? Recent changes 1. did some os updates 2019-08-27 2. provider took away the ssd for index storage (maybe config issue having less iops storage?) CentOS Linux release 7.6.1810 (Core) dovecot-2.2.36-3.el7.x86_64 mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u:CONTROL=~/mail/control:INDEX=/var/d ovecot2/%u/index:LAYOUT=maildir++ PS during force-resync, I am getting these "index/.INBOX/dovecot.index.log was locked for 176 seconds" on some users

3 4

Debug one user possible?
by Marc Roos 27 Sep '19

27 Sep '19

Is it possible to mail debug just one user? Maybe via the userdb?

3 2

limits in high-security mode with ssl
by sergio 27 Sep '19

27 Sep '19

Hello. I'm using dovecot in high-security mode: service_count = 1 for imap-login service starttls is requires, so the maximum number of active connections is process_limit (which is set to 1024) for the same imap-login service. But with this setup I got: doveconf: Warning: service auth { client_limit=1000 } is lower than required under max. load (1324) doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1227) 1. Where did 1324 and 1227 come from? 2. How configuration should be updated so these warnings disappear? -- sergio.

1 0

doveadm / vsize
by Paul Macdonald 27 Sep '19

27 Sep '19

Hi, i want to keep track of user mailbox size, I'm using doveadm mailbox status -u <user> vsize INBOX for a given mailbox this is under reporting ( by a lot) is this recursive? what shoud i be using to get an accurate disk usage metric? thanks Paul. -- ------------------------- Paul Macdonald IFDNRG Ltd Web and video hosting ------------------------- t: 0131 5548070 m: 07970339546 e: paul(a)ifdnrg.com w: http://www.ifdnrg.com ------------------------- IFDNRG 40 Maritime Street Edinburgh EH6 6SA ---------------------------------------------------- Virtual Servers from £50.00pm High specification Dedicated Servers from £150.00pm ----------------------------------------------------

3 4

Re: Website Down? Not down.
by dundir＠gmail.com 27 Sep '19

27 Sep '19

Its not down, its most likely your providers DNS resolution. www.dovecot.org resolves to: ipv4 : 94.237.85.39 ipv6: 2a04:3542:1000:910:acc1:5bff:fe5e:3b83 Best Regards, D.

3 2

Website down?
by ＠lbutlr 27 Sep '19

27 Sep '19

I am not able to reach the dovecot site on any one of three connections (home, office, mobile). Trying to look up DNS shows no records. -- But just because you've seen me on your TV Doesn't mean I'm any more enlightened than you

1 0

File manager or browser for IMAP?
by Steve Litt 26 Sep '19

26 Sep '19

Hi all, I could really use a file manager or browser to browse my Dovecot IMAP. Ideally it would have hotkeys to move, copy, delete and send. The send part needn't be coded: Just a call to a shellscript which can handle the send the way it's locally the most convenient. Anyone know of such a file manager or browser for IMAP? SteveT Steve Litt Author: The Key to Everyday Excellence http://www.troubleshooters.com/key Twitter: http://www.twitter.com/stevelitt

11 13

Typo in wiki
by Mika Pflüger 26 Sep '19

26 Sep '19

Hi, on https://wiki2.dovecot.org/Plugins/LastLogin the link to the new documentation page is wrong and leads to a 404. The correct link is https://doc.dovecot.org/plugin-settings/last-login-plugin/ . I tried correcting it myself in the wiki, but the Captcha was too hard for me. Cheers, Mika

2 1

Dovecot UIDs and POP.
by Plutocrat 26 Sep '19

26 Sep '19

Hi, I recently performed a mail server migration for a client under fairly serious time constraints (ahem, yes one of those jobs). I would normally use imapsync to get all the mail copied to the new server, but under the circumstances, I had to write a script to rsync each of the mailboxes over ssh, from one server to another. This all worked fine, except for the fact that if a client was using a POP email client, when they connected to the new server, they re-downloaded all their mail, creating duplicates of everything in their Outlook client. Of course they weren't happy about this. IMAP email clients were OK. No duplicates, and everything was fine. So while I was migrating the mail, I did try for a while to understand the format of the UID files, but failed to do so in the available time, so the client just had to deal with duplicate emails. But now the smoke has cleared, I'd like to understand the problem a little better, and I was hoping someone on this forum could explain it to me, and the changes I'd need to make to the files so that the POP client DIDN'T download the duplicate emails. Here's what I understand. The list of dovecot files in an account looks like this: -rw-r----- 1 admin mail 0 Jul 11 2018 dovecot-acl-list -rw-r----- 1 admin mail 14K Sep 20 08:36 dovecot.index -rw-r----- 1 admin mail 216K Sep 20 17:24 dovecot.index.cache -rw-r----- 1 admin mail 29K Sep 20 17:24 dovecot.index.log -rw-r----- 1 admin mail 388 Dec 17 2017 dovecot.list.index -rw-r----- 1 admin mail 1.6K Jul 11 2018 dovecot.list.index.log -rw-r----- 1 admin mail 24 Jul 11 2018 dovecot.mailbox.log -rw-r----- 1 admin mail 53 Sep 12 09:42 dovecot-quota -rw-r----- 1 admin mail 52K Sep 20 17:24 dovecot-uidlist -rw-r----- 1 admin mail 8 Jul 11 2018 dovecot-uidvalidity -rw-r----- 1 admin mail 0 Jul 4 2016 dovecot-uidvalidity.5779bbeb It seems that the dovecot-uidlist is an index of all the mail files in the account. As far as I can gather, this should be the same on both servers after I've rsynced them over ssh: The files and the list of them in dovecot-uidlist should correspond. So what is it that the POP client looks at and decides to re-download all the emails? And how could I have stopped it from doing that, and get it to realize that the list of emails it holds internally is the same as the list on the server? What would I need to change in the dovecot files? Thanks for any insight into the process. As a secondary question -- and perhaps I should put this in a separate message -- I did notice that "doveadm sync" would apparently have helped me with this, but I wasn't able to get that to work either. I believe it was something to do with the fact that all the mailboxes were under the same linux user account on the target server, and I couldn't figure out all the paths and permissions in time. Would 'doveadm sync' have fixed all the UID and duplicate POP email issues? P.

6 11

NFS Locking and Submission Service Authentication
by Asai 26 Sep '19

26 Sep '19

Greetings, We're in the process of upgrading our Dovecot server to new hardware and new expanded storage. We planned on using an NFS share for the mail storage, as we're running Postfix / Dovecot on a VM and wanted to separate out the mail storage from the VM for backup reasons. I read as much as I could find on line regarding configuring Dovecot to use NFS, and set it up as best I could, but I'm still running into lock errors e.g.: Sep 25 10:30:35 triata4 dovecot: imap(user(a)triata.globalchange.media)<75580><enQ/AWSTkQkKCgDu>: Error: fcntl(/vmail/triata.globalchange.media/user/dovecot.index.log, write-lock, F_SETLKW) locking failed: No locks available Sep 25 10:30:35 triata4 dovecot: imap(user(a)triata.globalchange.media)<75580><enQ/AWSTkQkKCgDu>: Error: mail_index_wait_lock_fd() failed with file /vmail/triata.globalchange.media/user/dovecot.index.log: No locks available Also, I have been trying to configure the Dovecot submission service to authenticate with Postfix, but I keep running into errors which are related to how I'm configuring the authentication. How do we set the user@domain and password string for submission in the configs? Thank you. Please see doveconf -n below, if you see any other errors in this config, I'd be grateful if you pointed them out: # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: Linux 3.10.0-957.27.2.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) nfs auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot-debug.log deliver_log_format = msgid=%m: from=%f: %$ hostname = triata.globalchange.media mail_debug = yes mail_fsync = always mail_home = /vmail/%d/%n/home mail_location = maildir:/vmail/%d/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } plugin { mail_log_fields = uid box msgid from flags sieve = file:/vmail/%d/%n/sieve;active=/vmail/%d/%n/.dovecot.sieve } protocols = imap lmtp submission sieve service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener lmtp { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve { process_limit = 1024 } service submission-login { inet_listener submission { port = 587 } } ssl_cert = </etc/letsencrypt/live/triata.globalchange.media/fullchain.pem ssl_key = # hidden, use -P to show it submission_relay_host = triata.globalchange.media submission_relay_password = # hidden, use -P to show it submission_relay_port = 587 submission_relay_ssl = starttls submission_relay_trusted = yes submission_relay_user = %u userdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } protocol lda { mail_plugins = }

3 4

Spam Blocking by filtering on username / id
by Plutocrat 26 Sep '19

26 Sep '19

This is probably quite an easy question, but I haven't been able to find the answer. I'm running a server where all the email addresses are in the format "user(a)domain.com". I've noticed that a large number of fake login attempts use the format "user" eg. reception, service, root, admin. Is it possible to prevent any such logins to these email users without an @domain.com? Or maybe ignore them. Or drop them from the logging. P.

5 5

'director_tag' field returned from passdb lookup results in 'unknown passdb extra field'
by Dan 25 Sep '19

25 Sep '19

Hello all, I'm working with director in tandem with 'director_tags' to group some backend hosts. I've assigned the groups to the hosts by appending a '@' character after the IP address of a backend host under director_mail_servers: ... director_mail_servers = 192.168.0.1@foo 192.168.0.2@bar 192.168.0.3@foobar ... I can confirm that the backend hosts possess the correct tags by issuing: # doveadm director status mail server ip tag vhosts state state changed users 192.168.0.1 foo 100 up - 0 192.168.0.2 bar 100 up - 0 192.168.0.3 foobar 100 up - 0 And examining that the tags have been assigned correctly. My passdb lookup is using mysql, and it appears as such: passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } This file looks like: driver = mysql connect = host=localhost dbname=xxx user=xxx password=xxx password_query = select director_tag, username, domain, password, 'Y' as `proxy` from users where username = '%n' and domain = '%d' iterate_query = select director_tag, username, domain from users I have a database that contains the following columns: username, domain, password, home, UID, GID, active, and director_tag To assign users to a backend, I went off of the documentation that states: > " A passdb lookup can return "director_tag" field containing the wanted tag name. " To fulfill this, I return the 'director_tag' in the passdb lookup, through password_query. The mapping seems to work. When I change the row in the database, the user gets proxied to the new backend on the next connection. However, My question is, Why does this appear in the logs? "dovecot[1963]: imap-login: Debug: Ignoring unknown passdb extra field: director_tag" How could this be an unknown passdb extra field, when the documentation states that you may return a field containing this key? In the file core/src/login-common/client-common-auth.c - Starting at line 142, there is no if condition checking for 'director_tag'. This causes that line to be output to debug handle. Is this intentional? Thank you for your time. Full configuration: auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = yes director_mail_servers = 192.168.0.1@foo 192.168.0.2@bar 192.168.0.3@foobar director_servers = x.x.x.x y.y.y.y director_user_expire = 5 mins disable_plaintext_auth = no doveadm_port = 24245 haproxy_trusted_networks = x.x.x.x y.y.y.y 127.0.0.1 login_greeting = Dovecot At Your Service managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocols = imap pop3 service director { fifo_listener login/proxy-notify { mode = 0666 user = $default_login_user } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director inet_listener imap { haproxy = yes port = 143 } inet_listener imaps { haproxy = yes port = 993 ssl = yes } } service ipc { unix_listener ipc { user = dovecot } } service pop3-login { executable = pop3-login director inet_listener pop3 { haproxy = yes port = 110 } inet_listener pop3s { haproxy = yes port = 995 ssl = yes } } ssl_cert = </etc/dovecot/private/dovecot.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol doveadm { auth_socket_path = director-userdb }

2 1

Can't Get SASL to Work
by Christopher Wooley 25 Sep '19

25 Sep '19

I have been fighting with this for almost 2 days. I can't seem to get the SMTP server to authenticate through SASL. I have turned so many things off and on, I don't remember what I've done, anymore. Can someone look at these and tell me what I'm missing. I get this in the log, anytime postfix tries to authenticate. Sep 24 19:48:14 ubuntu postfix/smtpd[14464]: warning: SASL: Connect to smtpd failed: No such file or directory Sep 24 19:48:14 ubuntu postfix/smtpd[14464]: fatal: no SASL authentication mechanisms Sep 24 19:48:15 ubuntu postfix/master[14048]: warning: process /usr/lib/postfix/sbin/smtpd pid 14464 exit status 1 srw-rw-rw- 1 postfix postfix 0 Sep 24 19:10 auth 10-auth.conf disable_plaintext_auth = no auth_ssl_require_client_cert = no auth_mechanisms = plain !include auth-system.conf.ext -------------- 10-master.conf service imap-login { inet_listener imap { } inet_listener imaps { } } service pop3-login { inet_listener pop3 { } inet_listener pop3s { } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { mode = 0666 user = postfix group = postfix } } service imap { } service pop3 { } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 user = postfix group = postfix } unix_listener auth-userdb { mode = 0666 user = postfix group = postfix } } service auth-worker { } service dict { unix_listener dict { } } -------------- main.cf smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no append_dot_mydomain = no readme_directory = no compatibility_level = 2 smtpd_tls_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem smtpd_tls_key_file = /etc/ssl/private/ssl-cert-snakeoil.key smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination myhostname = ubuntu.local.acr-live.com alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mydestination = $myhostname, ubuntu, localhost.localdomain, xyz.com , localhost mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 mailbox_size_limit = 0 recipient_delimiter = + inet_interfaces = all inet_protocols = all smtpd_sasl_type = dovecot smtpd_sasl_auth_enable = yes smtp_sasl_auth_enable = yes broken_sasl_auth_clients = yes smtpd_tls_security_level = may ---------------------------------- master.cf smtp inet n - y - - smtpd submission inet n - y - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_tls_auth_only=yes -o smtpd_reject_unlisted_recipient=no -o smtpd_sasl_type=dovecot -o smtpd_sasl_path=private/auth -o smtpd_recipient_restrictions= -o smtpd_relay_restrictions=permit_sasl_authenticated,reject -o milter_macro_daemon_name=ORIGINATING pickup unix n - y 60 1 pickup cleanup unix n - y - 0 cleanup qmgr unix n - n 300 1 qmgr tlsmgr unix - - y 1000? 1 tlsmgr rewrite unix - - y - - trivial-rewrite bounce unix - - y - 0 bounce defer unix - - y - 0 bounce trace unix - - y - 0 bounce verify unix - - y - 1 verify flush unix n - y 1000? 0 flush proxymap unix - - n - - proxymap proxywrite unix - - n - 1 proxymap smtp unix - - y - - smtp relay unix - - y - - smtp -o syslog_name=postfix/$service_name showq unix n - y - - showq error unix - - y - - error retry unix - - y - - error discard unix - - y - - discard local unix - n n - - local virtual unix - n n - - virtual lmtp unix - - y - - lmtp anvil unix - - y - 1 anvil scache unix - - y - 1 scache maildrop unix - n n - - pipe flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient} uucp unix - n n - - pipe flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient) ifmail unix - n n - - pipe flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient) bsmtp unix - n n - - pipe flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient scalemail-backend unix - n n - 2 pipe flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension} mailman unix - n n - - pipe flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py ${nexthop} ${user}

2 2

Mail messages are occasionally getting "stuck" behind one message on the server with POP3
by David Salisbury 24 Sep '19

24 Sep '19

I'm currently using Dovecot 2.2.33.2, although I've had this particular issue happen periodically over the last few years with other Dovecot versions. It seems to only affect accounts I'm receiving with POP3, not IMAP ones. What happens is the "top-most" (or, oldest) message in the "cur" folder seems to get "stuck". So I'll, say, come back in after a break to check my email (Thunderbird in my case, latest version), and in one of my folders I'll have 30 copies of the same email (all with the same timestamp), and no newer messages delivered since the timestamp of that message. If I go look in the "cur" folder, it has lots of messages in it. The "new" folder is empty. Every time I click "Get Messages" in the mail client, I get another new copy of the same email and no other mail. If I go manually into the cur folder and move the top-most (again, meaning "oldest") message out (which of course corresponds to the message that I'm receiving copies of over-and-over repeatedly) then when I click "Get Messages" all of the other messages come pouring into my mail client just fine and everything becomes "un-stuck". I've done some experiments, and if I copy that same "bad" message back in to cur with its original filename, it gets stuck again. It is an encrypted message, so I tried decrypting it and putting it back in cur with the original name, and it gets stuck in the same fashion even when it is decrypted. I tried putting the bad message into an account that receives with IMAP, just to see if it would maybe keep messages from delivering or something, but new messages come into the IMAP account just fine, even with the bad message present, so the issue seems specific to POP3. Any ideas what might cause something like that? I've had it happen twice over the last two days (which is unusually frequent, hence my message here), and the message sources are varied. Today it was a normal email message from a person, and yesterday it was a Twitch notification. I don't see anything unusual in the logs (and I did turn on debugging to see if it gave some additional info), just normal POP3 logins and connections closings when I'm clicking "Get Messages". -David

1 0

Any need to be worried about occasional dsync errors ?
by Laura Smith 24 Sep '19

24 Sep '19

I am occasionally (maybe every 4 hours or less frequently) seeing the following two errors appear in my logs. Are they any cause for concern ? Error: Timeout during state=sync_mails (send=done recv=mails) I/O has stalled, no activity for 600 seconds (last sent=mail_request (EOL)

1 0

Quota plug-in, do the directors also need it enabled?
by Kristijan Savic - ratiokontakt GmbH 23 Sep '19

23 Sep '19

I am having an issue with the quota display not showing up in e.g. Roundcube. There are multiple Dovecot servers and three directors. A load balancer also does it's thing. The quota plug-in is enabled everywhere, except on the directors. Do those also need it enabled? -- Kristijan Savic

1 0

pigeonhole sieve "standalone"
by Greg Minshall 22 Sep '19

22 Sep '19

hi. i've been thinking (but, for years now!) of a replacement for procmail, and pigeonhold sieve seems like a good candidate. but, i run nmh, not dovecot. as i started install it (on arch linux), i started to worry that it may be foolish of me to try to run it outside of a dovecot system. any wisdom about that would be gratefully received. thanks in advance, and thanks for the software. Greg Minshall

1 0

mdbox + sis attachments. Error stream is larger than expected by 1 byte
by Oleg Krivonosov 20 Sep '19

20 Sep '19

Hello! I use Dovecot 2.3.4.1-5+deb10u1 (Debian 10 x64) with mdbox, sis and imap. I catch many errors with attachments "Stream is larger than expected" by 1 byte in emails from Xerox WorkCentre 3345. Examples: Sep 20 23:20:04 mailserver dovecot: imap(wasya)<606><AG1zywGTVodVrE84>: Error: Mailbox INBOX: UID=5002: read(attachments-connector(/var/mail/virtual/wasya/mdbox/storage/m.210)) failed: read(/var/mail/attachments/b3/56/b3563977baf186928d38f81480b6443cf6d6d7e4-b86d80062ab36f5d7f190000f7ccf497[base64:18 b/l]) failed: Stream is larger than expected (882801 > 882800, eof=0) (FETCH BODY[]) Sep 20 23:32:27 mailserver dovecot: imap(wasya)<4831><n0a39wGTZ8hVyu8l>: Error: Mailbox INBOX: UID=654996: read(attachments-connector(/var/mail/virtual/wasya/mdbox/storage/m.4372)) failed: read(/var/mail/virtual/wasya/mdbox/storage/m.4372 trailer) failed: Stream is larger than expected (38 > 37, eof=0) (FETCH BODY[]). This error breaks getting email in MS Outlook and breaks viewing attachment in the problem email. Please help me to debug and to fix this issue. -- Best regards Oleg

1 0

Re: Bug] Sieve vacation :addresses match only,>, case-sensitive?
by Klaus Steinberger 20 Sep '19

20 Sep '19

Hi, >> I never saw any mail system in which the local Part ist Case sensitive! > > Every Unix/Linux system (I?d say that represents the majority of mail systems) has a case sensitive local part. Most mail admins have set their systems up to normalize all users on lowercase names, but that is not and has not always been the case. at least in every large installation LDAP User Backend's (or AD or similar) are used, and they are definitly case insensitive for the CN or UID attributes. Really nobody who cares about his systems and users uses mailboxnames which just differ in case (too much danger of confusion!) At least sieve must reflect the same settings as the MDA, so if the MDA is set to downcase (which in my opionion is the only valuable setting) sieve must also downcase. Sincerly, Klaus

1 0

fts_solr: Error: fts_solr: received invalid uid '0'
by Fabian 20 Sep '19

20 Sep '19

Hi, we are trying to add full text search functionality with Solr to our Doveoct setup. Our Versions: OS: Debian 9 Tried versions: - Dovecot 2.2.7 with Solr 3.6 - Dovecot 2.3.4 with Solr 8.2 (2.2.7 from offical Debian repository, 2.3.4 from backports) Search is working mostly of the time perfrectly smooth. But sometimes following message appears in mail.err: dovecot: imap(username)<16189><UxYWLVuSYMasEQoK>: Error: fts_solr: received invalid uid '0' If this error occurs our webmail frontend delivers most of the time a timeout. Sometimes the search only takes really long. Are there any ideas why this error occurs? We are not able to reproduce the error in such a way that it would always be reproducible. However, we can reproduce the behavior in some form over and over again - but we do not know exactly what is decisive. If you need any further information please ask - thanks! :-) dovecont -n: 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.9.0-11-amd64 x86_64 Debian 9.11 ext4 # Hostname: localhost auth_master_user_separator = * auth_username_format = %Ln default_client_limit = 5000 default_process_limit = 1000 default_vsz_limit = 256M dict { sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no first_valid_uid = 480 imap_capability = +XDOVECOT lmtp_save_to_detail_mailbox = yes mail_attribute_dict = file:/var/spool/dovecot/user/%n/Maildir/dovecot-attributes mail_gid = vmail mail_location = maildir:/var/spool/dovecot/user/%d/%n/Maildir:LAYOUT=fs mail_plugins = " listescape mail_log notify quota acl fts fts_solr virtual" mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { hidden = yes inbox = yes list = no location = prefix = separator = / subscriptions = yes } namespace { list = yes location = maildir:%%h/Maildir:INDEXPVT=~/shared/%%u:LAYOUT=fs prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace Public { list = children location = maildir:/var/spool/dovecot/public:INDEXPVT=~/public:LAYOUT=fs prefix = Public/ separator = / subscriptions = no type = public } namespace Virtual { hidden = no list = yes location = virtual:/etc/dovecot/virtual:INDEX=/var/spool/dovecot/user/%u/virtual prefix = Virtual/ separator = / subscriptions = no } namespace inbox { inbox = no location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX/ separator = / subscriptions = no type = private } passdb { args = /etc/dovecot/masteruser.%s driver = passwd-file master = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { acl = vfile acl_anyone = allow acl_shared_dict = file:/var/spool/dovecot/public/shared-mailboxes.db fts = solr fts_autoindex = yes fts_solr = url=<a href="http://172.17.10.12:8983/solr/dovecot/" rel="noreferrer noopener" target="_blank">172.17.10.12:8983/solr/dovecot/</a> quota = dict:User quota::no-unset:proxy::sqlquota quota_rule = *:storage=10G sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /var/spool/dovecot/globalsieve/duplicate.sieve sieve_duplicate_default_period = 1h sieve_duplicate_max_period = 1d } protocols = " imap lmtp sieve pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service decode2text { executable = script /usr/lib/dovecot/<a href="http://decode2text.sh" rel="noreferrer noopener" target="_blank">decode2text.sh</a> unix_listener decode2text { mode = 0666 } user = dovecot } service dict { unix_listener dict { group = vmail user = vmail } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = no userdb { args = /etc/dovecot/mysql-userdb.conf.ext driver = sql } protocol lmtp { mail_plugins = " listescape mail_log notify quota acl fts fts_solr virtual acl quota sieve notify" quota_full_tempfail = yes } protocol imap { imap_capability = +XDOVECOT mail_max_userip_connections = 100 mail_plugins = " listescape mail_log notify quota acl fts fts_solr virtual imap_acl imap_quota" } remote <a href="http://172.17.10.10" rel="noreferrer noopener" target="_blank">172.17.10.10</a> { protocol imap { imap_metadata = yes } }

2 3

Second private namespace
by André Rodier 19 Sep '19

19 Sep '19

Hello, I have Dovecot working, with a private namespace only. I would like to know if the following is feasible. - Add a second private namespace, in another directory scheme, for instance /mnt/backup/homebox/<user>/Mails/... - This namespace would be read-only. Perhaps the index and flags should be stored in a different folder with r/w access (e.g. /var, or even /tmp, it's not important) Now, as a matter of fact, this folder would be mounted on demand, via systemd-automount and fuse and is likely to be remote. This might implies some flags or caching on Dovecot performance. Thanks for your insights, André

1 0

Recent Dovecot on old operating system
by geho781＠justmail.de 19 Sep '19

19 Sep '19

Hi, sorry for the dumb question and please ignore this post if you think it's far beyond; i know it's not the way to go, but for reasons ... Has anyone running a self compiled recent dovecot (2.2.36.4) on Debian-7 and does it work? Or thinks it should work. Surprisingly it actually compiles flawlessly on Debian-7, but i wonder wether it will become a complete mess replacingthe existing dovecot (2.1.7) with the new one, serving a few dozens users and any kind of mail clients. Libraries of the compiled Dovecot: $ ldd ./src/master/.libs/dovecot linux-vdso.so.1 => (0x00007ffeacfb1000) libdovecot.so.0 => /usr/local/00-DBAI/dovecot/lib/dovecot/libdovecot.so.0 (0x00007f557b2a9000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f557af1c000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f557ad18000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f557ab10000) /lib64/ld-linux-x86-64.so.2 (0x00007f557b5dc000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f557a8f4000) libc6 versions: Debian-7: 2.13-38 Debian-9: 2.24-11 thanks so far, gerald

2 1

Dovecot 2.3.7.2 lucene segfault and coredump
by Domenico Pastore 19 Sep '19

19 Sep '19

Hi, i have a problem with libclucene-core.so.2.3.3.4. I See segfault error to messages log, when dovecot store mail to INBOX. set 12 13:41:21 box02.par-tec.it kernel: doveadm-server[20285]: segfault at 38 ip 00007f575b68fc42 sp 00007ffc0272e750 error 4 in libclucene-core.so.2.3.3.4[7f575b569000+188000] set 12 13:41:22 box02.par-tec.it kernel: doveadm-server[18871]: segfault at 38 ip 00007f9bea6d2c42 sp 00007fff47bceae0 error 4 in libclucene-core.so.2.3.3.4[7f9bea5ac000+188000] In the dovecot logs: 2019-09-12 13:45:09 auth: Debug: ldap(test4444client(a)newsvilXXXX,10.0.8.128<mailto:test4444client@newsvilXXXX,10.0.8.128>): Finished userdb lookup 2019-09-12 13:45:09 auth: Debug: userdb out: USER 1 test4444client@newsvilXXXX home=/pec_store/pec_mailbox/pec_mailbox_devel01/4/5/6/7/1/45671_18231 quota_rule=*:bytes=1048576000 mail_plugins+= 2019-09-12 13:45:10 SKENO8Uvel0gbAAA5ZKy4w doveadm(test4444client@newsvilXXXX): Fatal: master: service(doveadm): child 27680 killed with signal 11 (core dumped) 2019-09-12 13:45:10 auth: Debug: master in: USER 1 test4444client@newsvilXXXX service=doveadm lip=192.160.10.3 lport=924 rip=10.0.8.128 rport=36760 2019-09-12 13:45:10 auth: Debug: ldap(test4444client@newsvilXXXX 10.0.8.128): Performing userdb lookup I have attach the coredump of error and dovecot.conf Thanks for support. Br, Domenico ------------------- info: ------------------- # 2.3.7.2 (3c910f64b): /pec/config/dovecot/dovecot-devel01/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: Linux 3.10.0-957.27.2.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: box02

2 5

fts-elastic plugin
by Filip Hanes 19 Sep '19

19 Sep '19

Hi all, I have recently worked on fts plugin for ElasticSearch. https://github.com/filiphanes/fts-elastic It is forked from fts-elasticsearch as you can see in PR https://github.com/atkinsj/fts-elasticsearch/pull/21 with following changes (maybe some more) - no duplicate json keys in indexed documents, so compatible with ES 7.x - 1 elastic index for all users with routing by username - _id field similar to fts-solr (uid/box_guid/user) - better json response parsing - no need to use _source - added rawlog_dir to config and some more - renamed all elasticsearch to elastic for shorter names - fixed http_client leaks - fixed some compile warnings I would appreciate if someone with better experience in c and dovecot code would review my code and suggest/push changes. In my opinion fts-elastic is now functionally on level as currently recommended FTS plugin: fts-solr. I would like to implement proper rescan inspired by fts-lucene so it would become the most complete fts plugin for dovecot. -- Filip Hanes

3 2

Imaptest stall
by Marc Roos 18 Sep '19

18 Sep '19

I have been testing with imaptest and getting 'stalls', I tried even building from source and static. Even running it on the same host. Anyone knows what I could doing wrong? [@~]# ./imaptest - append=100,0 logout=0 host=192.168.10.44 port=143 user=test2 pass=xxxxxxxx seed=100 secs=240 clients=1 mbox=64kb.mbox box=INBOX/test Logi Sele Appe 100% 100% 100% 1 1 5 1/ 1 0 0 9 1/ 1 0 0 13 1/ 1 0 0 5 1/ 1

3 7

Namespace overlap
by Daniel Miller 17 Sep '19

17 Sep '19

Given an existing default namespace: namespace inbox { type = private separator = / prefix = location = sdbox:/var/mail/%d/%n/sdbox inbox = yes hidden = no list = yes subscriptions = yes } And mailboxes like: INBOX INBOX/Archives INBOX/Archives/2018 if I then define a new namespace: namespace archives { type = private separator = / prefix = Archives/ location = mdbox:/var/mail/%d/%n/Archives/mdbox subscriptions = no list = children } What will happen to the previous existing mailboxes & mails? Will they simply be "masked" by the new namespace and remain pending other operations? Or would they be moved/deleted? If they remain - is it possible to refer to the old mailboxes via either IMAP or doveadm? -- Daniel

1 0

CVE-2019-11500 and LMTP error
by Gabriele Nencioni 16 Sep '19

16 Sep '19

Hi all, does the dovecot fixed version: 2.3.7.2, 2.2.36.4 (as the CVE-2019-11500 says) fix the LMTP error "Got unexpected reply" as well? The LMTP error "Got unexpected reply" is described here: https://dovecot.org/pipermail/dovecot/2018-August/112562.html https://dovecot.org/pipermail/dovecot/2018-August/112666.html Thanks in advance Regards, -- Gabriele Nencioni

2 1

Repo for Debian-Buster
by slater.ch 16 Sep '19

16 Sep '19

Hi, is there already a release date for Debian Buster packages? kind regards Christian

3 2

corrupted mdbox
by sch tel 16 Sep '19

16 Sep '19

Hello list, i run into another problem. It seems there are some null bytes in the middle of an compressed mdbox, if i look into it with hexdump after run doveadm dump. With mdbox-recover.pl i could recover ~2k messages. What should i do to recover this broken mdbox file and dsync can finish? Sep 06 18:47:28 doveadm: Error: zlib.read(/var/spool/mail/<user>/storage/m.770): unexpected EOF at 7947710 Sep 06 18:47:28 doveadm: Error: dsync(server1): read(zlib(/var/spool/mail/<user>/storage/m.770)) failed: read(/var/spool/mail/<user>/storage/m.770) failed: zlib.read(/var/spool/mail/<user>/storage/m.770): unexpected EOF at 7947710 (last sent=mail, last recv=mail_request (EOL)) Sep 06 18:47:29 dsync-local(<user>)<+mJrOaCNcl2eZwAA0H466A>: Error: zlib.read(/var/spool/mail/<user>/storage/m.770): unexpected EOF at 7947710 Sep 06 18:47:29 dsync-local(<user>)<+mJrOaCNcl2eZwAA0H466A>: Error: dsync(server1): read(zlib(/var/spool/mail/<user>/storage/m.770)) failed: read(/var/spool/mail/<user>/storage/m.770) failed: zlib.read(/var/spool/mail/<user>/storage/m.770): unexpected EOF at 7947710 (last sent=mail, last recv=mail_request (EOL))

2 1

question about Mailcrypt plugin
by Daniel Niewerth 14 Sep '19

14 Sep '19

Hi everybody, I have a question about the Dovecot Mailcrypt Plugin. I'm trying to understand what the security of this plugin is based on. The encryption with private and public key and elliptic curves is basically a good thing. But the keys are in the file system together with the encrypted mails. The passwords for the private keys are in the user database. I guess the passwords have to be in plaintext for it to work. Is that true, or did I misunderstand something? Then the encryption would make no sense at all, right? For what purpose was the plugin developed? Can anyone explain this to me? Best regards Daniel

2 2

New to dovecot admin, question about using LDAP for user-specific values
by Gerben Wierda 13 Sep '19

13 Sep '19

I am new to dovecot administration. I’ve read the Wiki but that hasn’t given me the understanding I need. When I query my LDAP (on macOS) on a value for user ‘gerben’, I can get that: dumbledore:~ gerben$ dscl /LDAPv3/127.0.0.1 -read /users/gerben GeneratedUID GeneratedUID: 780D870E-6B00-478E-AB70-3D3307215A82 I would like to use that value in dovecot settings, e.g. something like user_attrs = \ =mail=maildir://Library/Server/Mail/Data/mail/%{ldap:GeneratedUID} Is this possible and if so what do I exactly need to do to get this working? Gerben Wierda Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> Architecture for Real Enterprises <https://www.infoworld.com/blog/architecture-for-real-enterprises/> at InfoWorld On Slippery Ice <https://eapj.org/on-slippery-ice/> at EAPJ

2 4

Multiple certificate option
by remo＠Mattei.org 13 Sep '19

13 Sep '19

What is the best way to adopt multiple certs? Thanks.

8 9

keywords lost when moving a message in a folder
by wouldsmina 13 Sep '19

13 Sep '19

Hi, I migrate my dovecot server from Debian 9 to Debian 10 (Dovecot 2.3.4.1), since when I move a message in a folder, keywords systematically disappear. I'm using Thunderbird 60.8. Would an option have escaped me? or is there a bug on this version of dovecot? Dovecot 2.3.4.1 OS : Debian 10.0 x64 FS : Ext4 dovecot -n result : # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-5-amd64 x86_64 Debian 10.0 # Hostname: suzuki.domain.tld auth_cache_size = 1 k auth_failure_delay = 3 secs base_dir = /var/run/dovecot/ default_vsz_limit = 4 G first_valid_uid = 100 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = mail system ready. login_trusted_networks = 172.16.0.0/12 192.168.0.0/16 mail_fsync = always mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_prefetch_count = 20 mail_privileged_group = mail mailbox_idle_check_interval = 1 mins mmap_disable = yes namespace inbox { inbox = yes location = maildir:~/Maildir mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . subscriptions = yes type = private } passdb { driver = pam } plugin { acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes sieve = ~/sieve/.dovecot.sieve sieve_after = /etc/dovecot/sieve.after sieve_before = /etc/dovecot/sieve.before sieve_before2 = ~/sieve/.dovecot.vacation.sieve sieve_dir = ~/sieve sieve_editheader_forbid_add = X-Verified sieve_editheader_forbid_delete = X-Verified X-Seen sieve_editheader_max_header_size = 1k sieve_extensions = +editheader sieve_max_script_size = 2M } postmaster_address = postmaster(a)domain.tld protocols = " imap lmtp pop3" service auth-worker { process_min_avail = 6 service_count = 0 } service imap-login { process_min_avail = 10 service_count = 0 vsz_limit = 4 G } service imap { process_limit = 4096 vsz_limit = 4 G } service pop3 { process_limit = 1024 } ssl_cert = </etc/ssl/certs/suzuki.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 25 } thanks.

2 2

Outlook 2019 and IMAP
by John Regan 12 Sep '19

12 Sep '19

Hi, We've just upgraded a few of our desktops to use Outlook 2019, and the configuration is different from previous versions. There doesn't appear to be any ability to specify a username in addition to the user's email address. The username is different than the username specified in the email address. I had read about setting auth_username_format, but the user login name is "user" while the email address is user.name(a)company.com. I've included below my dovecot -n config. # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: Linux 5.1.16-300.fc30.x86_64 x86_64 Fedora release 30 (Thirty) # Hostname: orion.inside.example.com mail_gid = mail mail_home = mail/ mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mail_max_userip_connections = 500 mail_plugins = " listescape" mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational rege x imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { driver = pam } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 993 } process_min_avail = 6 service_count = 0 vsz_limit = 512 M } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = 192.168.1.100 port = 995 } } ssl_cert = </etc/letsencrypt/fullchain.pem ssl_cipher_list = ALL:!LOW:!EXP:!aNULL:!RC4::!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { driver = passwd } userdb { driver = passwd }

2 1

Re: Gnu sieve vs Dovecot sieve-filter - sieve-filter extremely slow at lda (writing emails to local mbox files)
by Zenaan Harkness 12 Sep '19

12 Sep '19

I am wondering why sieve-filter is so slow compared to gnu sieve. I run mpop (like getmail) to download from a pop3 server to a local mbox file: ~/mail/email-incoming-unsorted This step is very fast. The next step, I throw the email-incoming-unsorted mbox file at a sieve processor, to sort the emails from that mbox, into other mboxes, according to the sieve rules file. Up until a couple days ago I was using Gnu sieve. Gnu sieve balks on emails which have no x-message-id (?? something like this) header field, so after a few years, I finally decided to switch "up" to Dovecot/Pigeonhole's "sieve-filter" command. Using Gnu sieve, this mbox sorting step was even faster than mpop (/ getmail) - and mpop and getmail are really fast (compared with fetchmail), since they pipeline the email downloads. Even with 100s of emails, Gnu sieve would take only 10 to 20 seconds at most. Super fast. Using sieve-filter, all emails are being processed - including those without "message id header". This is good. But also, using sieve filter, is really slower - slower than the download step by an order of magnitude or two. See below for details, any ideas appreciated. To add to the below, I added: mbox_very_dirty_syncs = yes to the sieve-filter config, which slightly improves performance, but not by much (in comparison with Gnu sieve). TIA, ----- Forwarded message from Zenaan Harkness <zenaan(a)freedbms.net> ----- From: Zenaan Harkness <zenaan(a)freedbms.net> To: debian-user(a)lists.debian.org Date: Thu, 12 Sep 2019 08:06:12 +1000 Subject: Re: Gnu sieve vs Dovecot sieve-filter - sieve-filter extremely slow at lda (writing emails to local mbox files) On Thu, Sep 12, 2019 at 07:55:23AM +1000, Zenaan Harkness wrote: > Why is Gnu sieve so extremely fast to batch process an mbox file, but > while Dovecot's sieve-filter is an order of magnitude slower? > > Sequence: > > - mpop or getmail to pipeline download emails into temp mbox file > - filter that file > > Gnu sieve just flies through a local mbox file and saving emails to > other local mbox files. > > Gnu sieve rejects too many emails with "malformed" errors, so after a > few years I bit the bullet and upgraded to Dovecot's sieve-filter. > > Dovecot's sieve-filter, at present, is an order of magnitude slower. > > Here's my filter command (one line): > > /usr/bin/sieve-filter -veW -c $HOME/etc/email/sieve-dovecot-config.conf -o mail_location=mbox:~/mail:INBOX=~/mail/Inbox:INDEX=:UTF-8:VOLATILEDIR=/tmp/dovecot-volatile/%2.256Nu/%u:SUBSCRIPTIONS=dovecot_subscriptions ~/etc/email/sieve.rc email-incoming-unsorted > > The sieve script is fine now that I have the correct "require" > clauses (hint: "capability strings"). > > File ~/etc/email/sieve-dovecot-config.conf: > > protocols = pop > lda_mailbox_autocreate = yes > lda_mailbox_autosubscribe = yes > mail_fsync = never > > There's no re-sending of emails into my local Postfix SMTP server - I > checked the system logs and confirmed this (journalctl -f). > > I suspect that Gnu sieve was directly writing each email to the > appropriate sieve-determined mbox file (perhaps with only a sync at > the end of a single batch process - what I've attempted to achieve > above with sieve-filter), and that sieve-filter is instead passing > each email through some (dovecot) lda? > > Here's the output for a sieve-filter batch processing of 11 emails: > > $ /usr/bin/sieve-filter -veW -c /home/zen/etc/email/sieve-dovecot-config.conf -o mail_location=mbox:/home/zen/mail:INBOX=/home/zen/mail/Inbox:INDEX=:UTF-8:VOLATILEDIR=/tmp/dovecot-volatile/%2.256Nu/%u:SUBSCRIPTIONS=dovecot_subscriptions /home/zen/etc/email/sieve.rc email-incoming-unsorted > # PS0 Timestamp: 20190912@07:02:23 > info: filtering: [Tue, 3 Sep 2019 05:17:16 -0500; 10240 bytes] `Re: VentureBeat: The death of disk? H...'. > info: msgid=<CAMjeLr91T9R7APsuxQVuM3WbqDsxAfwn4=OYDeDX4FMcoRdGdQ(a)mail.gmail.com>: stored mail into mailbox 'l/cp/cp'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 07:29:53 -0400; 12968 bytes] `[zfs-devel] xattr naming format in Zo...'. > info: msgid=<15675101930.d5ba2E.12322(a)composer.zfsonlinux.topicbox.com>: stored mail into mailbox 'l/z/zdev'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 03 Sep 2019 15:29:09 +0300; 20461 bytes] `Re: [zfs-devel] xattr naming format i...'. > info: msgid=<23955051567513749(a)sas1-02732547ccc0.qloud-c.yandex.net>: stored mail into mailbox 'l/z/zdev'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 18:20:42 +0530; 18065 bytes] `Re: [Gluster-users] Issues with Geo-r...'. > info: msgid=<CADmkyZMxrfOANrAP+_URAHJcMqCqh=iGdajTSzkfQ5PCZsUfyg(a)mail.gmail.com>: stored mail into mailbox 'l/gl/user'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 09:34:20 -0400; 13342 bytes] `Re: tasksel'. > info: msgid=<20190903133420.GS6166(a)eeg.ccf.org>: stored mail into mailbox 'l/deb/user'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 06:56:07 -0700 (PDT); 12390 bytes] `[awx-project] Re: AWX on Kubernetes m...'. > info: msgid=<0715adb7-540f-4cff-9282-e1252c53c2e8(a)googlegroups.com>: stored mail into mailbox 'l/ansible/awx'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 07:01:27 -0700 (PDT); 12220 bytes] `[awx-project] Re: AWX on Kubernetes m...'. > info: msgid=<949b2c17-4254-49f1-83b4-cd54d15aa17d(a)googlegroups.com>: stored mail into mailbox 'l/ansible/awx'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 10:14:58 -0400; 25313 bytes] `Re: [zfs-devel] xattr naming format i...'. > info: msgid=<CAB5c7xpHCdFx1w3yA9FyRL-KQ8BUiCr4JbiDQRuFJj9nOgKxTg(a)mail.gmail.com>: stored mail into mailbox 'l/z/zdev'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 17:10:22 +0200; 7567 bytes] `Re: [asterisk-users] Playing MP3's in...'. > info: msgid=<20190903151022.354xpe6ds2vglher(a)red.localdomain>: stored mail into mailbox 'l/as/users'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Wed, 4 Sep 2019 01:04:49 +0900; 14858 bytes] `Re: [Hyperledger Fabric] a primitive ...'. > info: msgid=<160901d8-b903-9e9a-91ac-267571b0e24d(a)gmx.com>: stored mail into mailbox 'l/hl/fabric'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 09:55:22 -0700 (PDT); 13337 bytes] `[awx-project] Re: AWX on Kubernetes m...'. > info: msgid=<f9bc4e6a-8445-4b34-927a-35f577ffcc07(a)googlegroups.com>: stored mail into mailbox 'l/ansible/awx'. > info: message expunged from source mailbox upon successful move. > 2 ▶︎️ zen@eye 20190912@07:02:30 ~ $ > > > So about 3/4 of a second is spent by dovecot's sieve-filter, on each > email that it processes - watching it is painful given how fast Gnu > sieve has been for the last few years - it's almost (but not quite) > as slow as my previous fetchmail email download per-email time. > > Attached is a -D debug run of sieve-filter on 20 emails - slightly > longer than the above, and took roughly 15 seconds to run. > > Any help appreciated... On another test run of ~600 emails, sieve-filter is consistently running ~100% of one CPU (for about 4 minutes) to process these emails, which leads to the conclusion that despite what looks like should be a batch process, sieve-filter is perhaps reloading the rules for every single email that it processes, even though I gave it a whole mbox, and not a single email, to process. Can sieve-filter work the way it should / the way I want it / batch process a whole mbox - without reloading the sieve rules for every email? ----- End forwarded message -----

3 2

Re: [Bug] Sieve vacation :addresses match only,> case-sensitive?
by Klaus Steinberger 12 Sep '19

12 Sep '19

> I want to have it case-insensitive again, like in Pigeonhole version 0.4.24 i would also plea for making it case-insensitive again! We were hit by this too, and I never saw any mail system in which the local Part ist Case sensitive! Sincerly, Klaus Steinberger -- Rechnerbetriebsgruppe / IT, Fakultät für Physik Klaus Steinberger FAX: +49 89 28914280 Tel: +49 89 28914287

6 5

WARNING: using attachment_dir with plugin zlib can corrupt mails
by Patrick Cernko 12 Sep '19

12 Sep '19

Hello list, hello Dovecot developers, this week, I discovered a serious bug in Dovecot, that lead to several broken mails on our servers. The bug corrupts the first few characters of the mail header during saving. On our setup, it was almost always only the very first line of text, that was corrupted. Depending on the IMAP client (they seem to request different header fields, ... during mail access), the bug causes the imap process to hang up the TCP connection and log errors like this: > imap(USERNAME)<4767><TeQP4ASOTK5/AAAB>: Error: Corrupted record in index cache file /IMAP/mail/mailboxes/USERNAME/mdbox/mailboxes/Trash/dbox-Mails/dovecot.index.cache: UID 489113: Broken fields in mailbox Trash: read(attachments-connector(zlib(/IMAP/mail/mailboxes/USERNAME/mdbox/storage/m.813))): FETCH BODY[HEADER.FIELDS (RETURN-PATH SUBJECT)] got too little data: 2 vs 122 In our case that finally grabbed my attention, the client was the users iphone that did not display any new messages but his Thunderbird did. The bug seems to be triggered by a bad "interaction" of attachment_dir option and zlib plugin. If you use both, you most likely are affected, too, except you only use zlib plugin for reading previously compressed stored mails. That's also the workaround we use now: zlib plugin only enabled in mail_plugins but no plugin/zlib_save set. The bug occurs on very specific mails. Due to privacy reasons I could not provide sample mails here. Storing such mails seems to trigger the bug reproducible. I attached a very minimal doveconf -n config, that can be used to trigger the bug. If one of the developers is interested, I can try to generate an "anonymized" version of such a specific mail that still causes the issue. I discovered the bug on our productive systems, running latest Dovecot 2.2 release, but the latest 2.3 I used during debugging is affected, too. During debugging, I also found one hint, that might help find the bug: If you store a problematic mail with zlib_save=gz (or zlib_save=bz2) and then disable the zlib plugin in mail_plugins, you can call doveadm fetch -u test hdr all | grep -v ^hdr: | gzip --decompress on test's mailbox with only that one broken mail. This will display the beginning of the rfc822 mail text until gzip terminates with "gzip: stdin: unexpected end of file", approximately after twice the length of the mail HEADER. This might indicate, that dovecot stores the uncompressed size of the header in it's data structures although the mail is stored compressed. I also found a very efficient way to find all affected mails in our setup: doveadm -f flow fetch -A 'user guid mailbox uid seq flags hdr' all | \ grep -a "^[^ ]+ user=" | \ grep -avF ' hdr=Return-path: ' | \ grep -av '.* hdr=[[:print:][:space:]]*$' (runtime for ~6M mails on our servers was 20-30min) This can be even more optimized if you have a powerful storage system with GNU parallel: > doveadm user '*' | parallel "doveadm -f flow fetch -u '{}' 'user guid mailbox uid seq flags hdr' all | grep -a '^user=' | grep -avF ' hdr=Return-path: ' | grep -av '.* hdr=[[:print:][:space:]]*$' || true" (runtime for ~6M mails on our servers was ~4min) The command will give you a list of mails that possibly are affected, check the full output of doveadm fetch -u USERNAME hdr guid GUID | less to verify that the header is really broken. On our systems I found 39 mails within ~12M mails. I was able to recover these mails "manually" by reconstructing the Return-Path header line, importing the fixed mails and expunging the corrupt ones. Before importing, I had to disable zlib_save option obviously. Best regards, -- Patrick Cernko <pcernko(a)mpi-klsb.mpg.de> +49 681 9325 5815 Joint Administration: Information Services and Technology Max-Planck-Institute fuer Informatik & Softwaresysteme

4 4

pigeonhole/lda accessing -m folder
by matthias lay 12 Sep '19

12 Sep '19

Hi all, anybody knows, if there is a way to acces the folder from a call like this deovecot-lda -m destfolder from within a sieve script? thx in advance matze

5 9

Re: Gnu sieve vs Dovecot sieve-filter - sieve-filter extremely slow at lda (writing emails to local mbox files)
by Zenaan Harkness 12 Sep '19

12 Sep '19

(I did subscribe to this mailing list, albeit with zen at freedbms.net, so either way I'm getting all your emails - thank you -so- much for replying...) MUA is mutt, reading email in a terminal (sorry, forgot to mention this before). For many years now my email folder (mbox files) collection has grown to many GiB, mostly mailing lists. If I am to change email storage format, it should be mutt compatible; looking at https://wiki2.dovecot.org/MailboxFormat I see that only DJB's Maildir is compatible with both Dovecot ("a reliable choice" says the wiki), and mutt. I can imagine that sdbox or mdbox could be made "mutt compatible" so to speak, by running some sort of local IMAP server, and accessing my email from mutt that way; this is undesirable to my mind because this would require: 1) a new learning curve wrt mutt and reading email on IMAP servers 2) a new learning curve to set up a local IMAP server (securely) 3) the inability to use mutt without a local IMAP server to read my local email but such a setup would also have some quite desirable benefits: 1) once set up, multiple MUAs could be used, and I'd have a beginning grasp on setting up an IMAP server and front ends (this is something on my bucket list, to assist my local church with) 2) simpler remote "online" access to my local "offline" email store (e.g. using my mobile phone when on the road) by setting up a webmail server (much simpler (read "possible" to use on a mobile phone) than using a vpn and mutt...), thus freeing me up from the behemoth web email providers... Next, I do not know how to "pipe the messages to the dovecot lda". After downloading from my POP3 provider into a local mbox file (this is my step 1), then I sort the emails (this is my step 2): the following should be on a single line: /usr/bin/sieve-filter -veW -c $HOME/etc/email/sieve-dovecot-config.conf -o mail_location=mbox:~/mail:INBOX=~/mail/Inbox:INDEX=:UTF-8:VOLATILEDIR=/tmp/dovecot-volatile/%2.256Nu/%u:SUBSCRIPTIONS=dovecot_subscriptions ~/etc/email/sieve.rc email-incoming-unsorted As you can see from the above command, sieve-filter is given the name of the mbox ("mail folder") to sort, as its very last argument on the command line - so in this instance, sieve-filter really has no excuse, and should be not be re-reading the sieve rules script for each email - now perhaps that's not happening, I only made an assumption because of a CPU hitting 100% for a minute or two just to process a few 100 emails... What could also be happening (again, an assumption), is that sieve-filter is written to assume dovecot index files to be in existence. I disabled those with the "INDEX=" clause you see in the command above, which obviously has been given no value. The reason I figured out how to disable the creation of the indexes in the .imap directories, is that for my setup, Gnu sieve has proven that I should not need such indexes - with mbox files, just append each email to the end of the target "mailbox folder" mbox file, and we're done! This literally should not cost 100% CPU, even for one millisecond! But more importantly, because my working email folder is ~30GiB, without disabling this index creation step, sieve-filter forced the creation of indexes, which "took so long I gave up and hit CTRL-C, which did not work, so I kill -9'ed the sieve-filter and whatever other process was not stopping". Last year someone on debian-user recommended I upgrade to using Dovecot/Pigeonhole's sieve-filter (rather than Gnu sieve) due to the issues with Gnu sieve. I am starting to think that I should perhaps try to figure out if it's possible to (re)process the emails Gnu sieve has a problem with, to massage them into a shape that Gnu sieve accepts - then my immediate problem would certainly be solved... Thank you all again.. Zenaan

1 1

Subscribe to a fileinto :create mailbox?
by Larry Rosenman 11 Sep '19

11 Sep '19

Is it possible in pigeonhole? I.E. I do a fileinto :create "some/mail/box" How can I make it autosubscribe? doveconf -n: # 2.3.7.2 (3c910f64b): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: FreeBSD 12.0-STABLE amd64 # Hostname: thebighonker.lerctr.org auth_default_realm = lerctr.org auth_mechanisms = plain login auth_realms = lerctr.org thebighonker.lerctr.org tbh.lerctr.org thejonesonair.com thejonesonair.net why.net default_vsz_limit = 1 G deliver_log_format = msgid=%m: %$ (subject=%s from=%f size=%w) doveadm_password = # hidden, use -P to show it first_valid_gid = 0 first_valid_uid = 0 lda_mailbox_autocreate = yes listen = 192.147.25.65, :: login_access_sockets = tcpwrap mail_attribute_dict = file:%h/mail/.imap/dovecot-mail-attributes mail_location = mbox:~/mail:INBOX=~/mail/INBOX mail_log_prefix = "%s(%u/%p): " mail_plugins = " fts fts_solr notify virtual" mail_privileged_group = mail mail_server_admin = mailto:ler@lerctr.org mail_server_comment = LERCTR Mail Server managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds editheader mboxmetadata servermetadata vnd.dovecot.debug imapsieve vnd.dovecot.imapsieve namespace archive { hidden = no list = no location = mbox:~/MAIL-ARCHIVE prefix = ARCHIVE/ separator = / } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox INBOX { auto = create } mailbox SENT { special_use = \Sent } mailbox SPAM { special_use = \Junk } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } mailbox virtual/Flagged { special_use = \Flagged } mailbox virtual/all { special_use = \All } prefix = separator = / } namespace virtual { hidden = no list = yes location = virtual:~/MAIL-VIRTUAL prefix = Virtual/ separator = / } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } passdb { args = user=%Ln noauthenticate driver = static skip = authenticated } passdb { args = failure_show_msg=yes session=yes max_requests=20 driver = pam override_fields = domain=lerctr.org skip = authenticated } plugin { fts = solr fts_autoindex = yes fts_solr = url=http://thebighonker.lerctr.org:8983/solr/dovecot/ fts_tika = http://localhost:9998/tika/ imapsieve_mailbox1_before = file:/usr/local/share/dovecot-pigeonhole/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = SPAM imapsieve_mailbox2_before = file:/usr/local/share/dovecot-pigeonhole/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = SPAM imapsieve_mailbox2_name = * imapsieve_url = sieve://thebighonker.lerctr.org mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid size from subject vsize flags recipient_delimiter = +-_ sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_execute_bin_dir = /usr/local/share/dovecot-pigeonhole/sieve sieve_extensions = +editheader +vacation-seconds +mboxmetadata +servermetadata +vnd.dovecot.debug sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_pipe_bin_dir = /usr/local/share/dovecot-pigeonhole/sieve sieve_plugins = sieve_imapsieve sieve_extprograms } postmaster_address = postmaster(a)lerctr.org protocols = imap pop3 lmtp sieve recipient_delimiter = +-_ service anvil { unix_listener anvil { group = mail mode = 0666 } } service auth { unix_listener auth-client { mode = 0666 } unix_listener auth-master { mode = 0666 } } service doveadm { inet_listener http { port = 8080 ssl = yes } } service indexer-worker { drop_priv_before_exec = yes } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service stats { unix_listener stats-reader { group = mail mode = 0666 user = } unix_listener stats-writer { group = mail mode = 0666 user = } } service tcpwrap { unix_listener login/tcpwrap { group = $default_login_user mode = 0600 user = $default_login_user } } ssl_cert = </home/ler/letsencrypt-home/*.lerctr.org/fullchain.cer ssl_cipher_list = EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+AESGCM:EECDH:EDH+AESGCM:EDH+aRSA:HIGH:!MEDIUM:!LOW:!aNULL:!eNULL:!LOW:!RC4:!MD5:!EXP:!PSK:!SRP:!DSS ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } userdb { args = username_format=%Ln /etc/passwd driver = passwd-file override_fields = domain=lerctr.org } verbose_proctitle = yes protocol lmtp { mail_plugins = " fts fts_solr notify virtual sieve mail_log" } protocol lda { mail_plugins = " fts fts_solr notify virtual sieve mail_log" } protocol pop3 { mail_plugins = " fts fts_solr notify virtual mail_log" } protocol !doveadm { mail_plugins = " fts fts_solr notify virtual mail_log" } protocol imap { imap_capability = +SPECIAL-USE imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags imap_logout_format = in=%i out=%o fhc=%{fetch_hdr_count} fhb=%{fetch_hdr_bytes} fbc=%{fetch_body_count} fbb=%{fetch_body_bytes} del=%{deleted} exp=%{expunged} trash=%{trashed} imap_metadata = yes mail_max_userip_connections = 50 mail_plugins = " fts fts_solr notify virtual mail_log imap_sieve imap_zlib" } -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx(a)gmail.com US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

2 1

Case sensitive problem for vacation with dovecot lmtp pigeonhole
by kadafax＠gmail.com 11 Sep '19

11 Sep '19

Hi list, This issue was raised here by John Fawcett on 12/02/19 but got no response. So, as I'm facing the same issue, I would like to try again: When using the vacation sieve extension, if I send an email to user.name(a)mydomain.com it will work, but not to User.Name(a)mydomain.com. Is there a way to make the user part case insensitive so that the vacation auto-reply is fired whatever the way user part is written ? doveconf -n output: # 2.3.5.1 (7ec6d0ade): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.5 (2483b085) # OS: Linux 3.10.0-957.5.1.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) ext4 # Hostname: myserver.mydomain.com auth_master_user_separator = * auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* auth_verbose = yes debug_log_path = /var/log/dovecot-debug.log default_client_limit = 10640 default_process_limit = 2128 default_vsz_limit = 0 hostname = mydomain.com info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot-error.log login_greeting = Welcome mail_gid = vmail mail_location = maildir:/var/vmail/%Ln mail_plugins = quota fts fts_lucene mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = maildir:/var/vmail/%Ln mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /usr/local/etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = /usr/local/etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. quota = maildir:User quota quota_rule = *:storage=30GB quota_rule2 = Trash:storage=+200M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/sieve-scripts/%Ln.sieve sieve_dir = /var/vmail/%Ln/home/sieve } postmaster_address = postmaster(a)mydomain.com protocols = imap pop3 lmtp sieve sieve quota_full_tempfail = yes service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0777 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_min_avail = 10 } service imap { process_limit = 2128 } service lmtp { executable = lmtp -L process_min_avail = 15 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = nobody } ssl_cert = </etc/letsencrypt/live/myserver.mydomain.com/fullchain.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /usr/local/etc/dovecot/dovecot-ldap-userdb.conf.ext driver = ldap override_fields = uid=vmail gid=vmail home=/var/vmail/%Ln/home } protocol imap { mail_max_userip_connections = 30 mail_plugins = quota fts fts_lucene imap_quota } protocol lmtp { info_log_path = /var/log/dovecot-lmtp.log log_path = /var/log/dovecot-lmtp-errors.log mail_plugins = quota fts fts_lucene sieve } protocol sieve { mail_max_userip_connections = 20 }

2 2

Quota and maildir does not work with subfolders of INBOX
by Niels Kobschätzki 11 Sep '19

11 Sep '19

Hi, we just upgraded on FreeBSD (11.2-p14) from Dovecot 2.2 to Dovecot 2.3.7.2 and now we have kind of a big problem: the quota-plugin does not work correctly anymore. If a user has a subfolder of the INBOX (like .INBOX.foobar) the quota-calculation does not work anymore. It seems to be arbitrary but users get counted like double and more than they actually have in mails (the number of mails is correct, but the size gets calculated wrongly). The moment I remove those folders, the size gets calculated correctly. Unfortunately those folders are generated by some clients automatically afaik (like .INBOX.Trash) What can we do about that? There were already messages to the list about this in July 2018 but I didn't find any solution https://dovecot.org/pipermail/dovecot/2018-July/112439.html https://dovecot.org/pipermail/dovecot/2018-July/112440.html Niels

3 4

Off-site cloud backup (eg Amazon S3, Wasabi)
by James Brown 11 Sep '19

11 Sep '19

I’d like to synchronise our mail store to an Amazon S3 bucket or similar, for protection from ransomware. Server is running on macOS with maildir. Should I use a Dovecot process, rsync, a Mac app like Arq or Jungle Disk, or something else? Anyone have experience or opinions on this? Thanks, James.

3 2

Random duplicated emails
by Francis 11 Sep '19

11 Sep '19

Hello, I migrated our mail infrastructure to Dovecot on Ubuntu 18.04 some months ago. It works fine, but recently some users told me that they sometime receive duplicated emails. Same email content, same headers including message-id. I'm using two dovecot servers on two sites. Both server are in cluster. We don't use shared folders. All users that reported this issue so far are using the same server instance. The problematic emails are coming from local users on that instance too. The examples they given to me was emails with many recipients (To/CC). A specific message can be received twice (or more) by recipient A but only once by recipient B. I didn't see anything in the logs about sieve rules that redirect emails to others recipients. Where should I look to diagnostic this issue? Thanks. Server config: # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) doveconf: Warning: service auth { client_limit=1000 } is lower than required under max. load (2500) doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1503) # OS: Linux 4.15.0-60-generic x86_64 Ubuntu 18.04.3 LTS auth_default_realm = **edited** auth_master_user_separator = * auth_mechanisms = plain login default_process_limit = 500 first_valid_uid = 200 last_valid_uid = 200 mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " notify replication zlib quota" mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.debug mdbox_rotate_size = 10 M namespace inbox { inbox = yes location = mailbox "Éléments envoyés" { special_use = \Sent } mailbox "Éléments supprimés" { special_use = \Trash } mailbox Brouillon { special_use = \Drafts } mailbox "Courrier indésirable" { special_use = \Junk } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX/ separator = / } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes } plugin { mail_replica = tcps:**edited**:53683 quota = count:User quota quota_grace = 10%% quota_rule = *:storage=10G quota_rule2 = INBOX/Trash:storage=+100M quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 La boite courriel du destinataire est pleine - Mailbox is full quota_status_success = DUNNO quota_vsizes = yes quota_warning = storage=100%% quota-warning 100 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=85%% quota-warning 85 %u quota_warning5 = storage=75%% quota-warning 75 %u quota_warning6 = -storage=80%% quota-warning '-80' %u quota_warning7 = -storage=100%% quota-warning '-100' %u sieve = /var/vmail/domains/%Ld/%Ln/.dovecot.sieve sieve_before = /var/vmail/sieve/before.sieve sieve_default = /var/vmail/sieve/default.sieve sieve_extensions = +vnd.dovecot.debug sieve_global = /var/vmail/sieve/global zlib_save = lz4 } protocols = " imap lmtp sieve" service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service imap-login { inet_listener imap { port = 0 } process_min_avail = 10 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { address = **edited** port = 53684 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } } ssl = required ssl_cert = </etc/ssl/**edited**/**edited**.crtbundle ssl_client_ca_dir = /etc/ssl/certs ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = " notify replication zlib quota sieve" postmaster_address = postmaster@**edited** } protocol imap { mail_plugins = " notify replication zlib quota imap_zlib imap_quota" } -- Francis

4 4

Dovecot Director upgrade from 2.2 to 2.3
by Alessio Cecchi 10 Sep '19

10 Sep '19

Hi, I have a setup with 3 Dovecot Director v2.2.36 and director_consistent_hashing = yes ;-) Now I would like to upgrade to 2.3.7, first only Director and after also Backend. Can works fine a ring of director with mixed 2.2 and 2.3 version? Mi idea is to setup a new Director server with 2.3, stop one server with 2.2 and insert the new 2.3 in the current ring to check if works fine. If all works fine I will replace all Director 2.2 with 2.3 version. Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 6

dovecot duplicate emails
by Битус Павел 10 Sep '19

10 Sep '19

Hello. I have problems with duplicate emails when using replication and configured message filters on the client. Looks like when letter comes, it processed by replication and immediately after this processed by filter on client and moved to another folder. When it arrives to slave server into INBOX, it replicated back to master into INBOX and only after that - slave receives MOVE command. The only thing I see in the log is that the letter comes 2 times with same msgid: Sep 10 15:01:31 lmtp(USERNAME(a)DOMAIN.com.ua)<9306><UD3fJJuQd11aJAAA8J4ONg>: Info: sieve: msgid=<f3701949-360e-e15d-3d76-7465f23fb0ba(a)DOMAIN.com.ua>: stored mail into mailbox 'INBOX' Sep 10 15:01:31 indexer-worker(USERNAME(a)DOMAIN.com.ua)<12248><UD3fJJuQd11aJAAA8J4ONg:GEsdNJuQd13YLwAA8J4ONg>: Info: Indexed 1 messages in INBOX (UIDs 55963..55963) Sep 10 15:01:32 imap(USERNAME(a)DOMAIN.com.ua)<9674><zqysnTGSU8tYUeJ+>: Info: copy from INBOX: box=INBOX.Jira, uid=5034, msgid=<f3701949-360e-e15d-3d76-7465f23fb0ba(a)DOMAIN.com.ua>, size=23393 Sep 10 15:01:32 imap(USERNAME(a)DOMAIN.com.ua)<9674><zqysnTGSU8tYUeJ+>: Info: expunge: box=INBOX, uid=55963, msgid=<f3701949-360e-e15d-3d76-7465f23fb0ba(a)DOMAIN.com.ua>, size=23393 Sep 10 15:01:32 indexer-worker(USERNAME(a)DOMAIN.com.ua)<12248><zqysnTGSU8tYUeJ+:oC6vHZyQd13YLwAA8J4ONg>: Info: Indexed 1 messages in INBOX.Jira (UIDs 5034..5034) Sep 10 15:01:33 indexer-worker(USERNAME(a)DOMAIN.com.ua)<12248><pQBKNpuQd13/OgAA8J4ONg:WIrkC52Qd13YLwAA8J4ONg>: Info: Indexed 1 messages in INBOX (UIDs 55964..55964) Sep 10 15:01:33 imap(USERNAME(a)DOMAIN.com.ua)<9674><zqysnTGSU8tYUeJ+>: Info: copy from INBOX: box=INBOX.Jira, uid=5035, msgid=<f3701949-360e-e15d-3d76-7465f23fb0ba(a)DOMAIN.com.ua>, size=23393 Sep 10 15:01:33 imap(USERNAME(a)DOMAIN.com.ua)<9674><zqysnTGSU8tYUeJ+>: Info: expunge: box=INBOX, uid=55964, msgid=<f3701949-360e-e15d-3d76-7465f23fb0ba(a)DOMAIN.com.ua>, size=23393 Sep 10 15:01:34 indexer-worker(USERNAME(a)DOMAIN.com.ua)<9417><zqysnTGSU8tYUeJ+:gKqSOJ2Qd13JJAAA8J4ONg>: Info: Indexed 1 messages in INBOX.Jira (UIDs 5035..5035) Can you help me with this problem? dovecot --version 2.3.7.2 (3c910f64b) dovecot -n # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: Linux 4.19.66-gentoo x86_64 Gentoo Base System release 2.6 # Hostname: leon.DOMAIN.com.ua auth_cache_negative_ttl = 30 secs auth_cache_size = 256 M auth_cache_verify_password_with_worker = yes auth_gssapi_hostname = $ALL auth_krb5_keytab = /etc/krb5.keytab auth_mechanisms = gssapi ntlm plain login cram-md5 digest-md5 auth_worker_max_count = 2048 debug_log_path = /var/log/dovecot/dovecot_debug.log default_client_limit = 8492 default_vsz_limit = 2 G dict { lastlogin = mysql:/etc/dovecot/dovecot-dict-sql-lastlogin.conf } doveadm_password = # hidden, use -P to show it first_valid_uid = 8 imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags listen = * log_path = /var/log/dovecot/dovecot.log mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_fsync = never mail_gid = mail mail_location = maildir:%h mail_max_keyword_length = 4096 mail_plugins = " notify replication zlib fts fts_solr mail_log notify" mail_uid = mail maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_lazy_writes = no metric imap_commands { event_name = imap_command_finished } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } passdb { args = /etc/dovecot/dovecot-ldap-people.conf.ext driver = ldap } passdb { args = /etc/dovecot/dovecot-ldap-calendar.conf.ext driver = ldap } plugin { fts = solr fts_autoindex = yes fts_autoindex_max_recent_msgs = 100 fts_solr = url=http://SOLR_IP_ADDRESS:8983/solr/dovecot/ last_login_dict = proxy::lastlogin mail_log_cached_only = yes mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_replica = tcp:REPLICATE.DOMAIN.com.ua:4092 replication_dsync_parameters = -d -N -l 30 -U replication_full_sync_interval = 7 days sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /etc/dovecot/sieve/default.sieve sieve_before2 = /etc/dovecot/sieve/default_spam.sieve zlib_save = gz zlib_save_level = 9 } postmaster_address = root(a)DOMAIN.com.ua protocols = imap pop3 lmtp sieve replication_max_conns = 50 service aggregator { fifo_listener replication-notify-fifo { user = mail } unix_listener replication-notify { user = mail } } service auth { unix_listener auth-client { mode = 0660 user = mail } unix_listener auth-master { mode = 0600 user = mail } unix_listener auth-userdb { mode = 0600 user = mail } } service config { unix_listener config { mode = 0600 user = mail } } service dict { unix_listener dict { group = mail mode = 0600 user = mail } } service doveadm { inet_listener { port = 4092 } } service imap-login { process_min_avail = 8 service_count = 0 vsz_limit = 256 M } service imap { client_limit = 10 process_limit = 4096 process_min_avail = 50 service_count = 100 } service lmtp { process_limit = 70 process_min_avail = 20 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = mail mode = 0660 } vsz_limit = 20 G } ssl_cert = </etc/ssl/my_certs/fullchain1.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } userdb { args = /etc/dovecot/dovecot-ldap-people.conf.ext driver = ldap } userdb { args = uid=mail gid=mail home=/home/DOMAIN.com.ua/%Ln username_format=%Ln(a)DOMAIN.com.ua mail=maildir:/home/DOMAIN.com.ua/%Ln:UTF-8 driver = static } protocol lmtp { auth_username_format = %Lu mail_fsync = optimized mail_plugins = " notify replication zlib fts fts_solr mail_log notify sieve" } protocol lda { mail_fsync = optimized } protocol imap { imap_idle_notify_interval = 5 mins imap_max_line_length = 4 M imap_metadata = yes mail_max_userip_connections = 2048 mail_plugins = " notify replication zlib fts fts_solr mail_log notify fts fts_solr last_login" } protocol sieve { managesieve_max_line_length = 65536 }

1 0

New to dovecot admin, question about testing a dovecot configuration
by Gerben Wierda 10 Sep '19

10 Sep '19

I am new to dovecot administration, not new to postfix administration. When setting up a new postfix/dovecot combo from scratch, can I start with dovecot alone and still be able to test it? E.g. can I address the local delivery agent with some sort of test email? If so, how? I haven’t been able to find it in the wiki. Gerben Wierda Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> Architecture for Real Enterprises <https://www.infoworld.com/blog/architecture-for-real-enterprises/> at InfoWorld On Slippery Ice <https://eapj.org/on-slippery-ice/> at EAPJ

1 0

Force dovecot-uidlist reset
by Tom Sommer 09 Sep '19

09 Sep '19

Is there a way to force Dovecot to rebuild dovecot-uidlist from zero? It seems deleting all indexes and dovecot-* files followed by "doveadm force-resync" is not enough? It just gets the same UIDs? Perhaps from Maildir filenames? But I would like to reset the uid of all mails. Thanks -- Tom

1 1

doveadm mailbox list
by Daniel Miller 08 Sep '19

08 Sep '19

It's quite likely I'm doing it wrong, but... Given a valid mailbox... doveadm mailbox list -u <username> realmb returns "realmb" doveadm mailbox list -u <username> real* returns "realmb" Seems reasonable. Now, with a non-existent mailbox... doveadm mailbox list -u <username> bogus returns "bogus" doveadm mailbox list -u <username> bogus* returns "" Is this a bug or correct behavior? -- Daniel

2 1

TLS not working with iOS beta?
by Henrik Johansson 08 Sep '19

08 Sep '19

Hi, Have anyone else experienced problems using Dovecot with the mail app in beta releases of iOS/iPadOS 13? TLS is failing for my, it have worked fine for years and I am on the latest Dovecot version now, it works fine with older clients but not with the ones upgraded: Sep 04 19:49:16 imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization Sep 04 19:49:16 imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization Sep 04 19:49:16 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv2/v3 read client hello A Sep 04 19:49:16 imap-login: Debug: SSL alert: where=0x4008, ret=552: fatal handshake failure Sep 04 19:49:16 imap-login: Debug: SSL: where=0x2002, ret=-1: error Sep 04 19:49:16 imap-login: Debug: SSL: where=0x2002, ret=-1: error Sep 04 19:49:16 imap-login: Debug: SSL error: SSL_accept() failed: error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher Sep 04 19:49:16 imap-login: Debug: SSL: where=0x2002, ret=-1: error Sep 04 19:49:16 imap-login: Debug: SSL error: SSL_accept() failed: error:140800FF:SSL routines:ssl3_accept:unknown state Sep 04 19:49:16 imap-login: Info: Disconnected (no auth attempts in 0 secs): user=<>, rip=11.22.33.44, lip=11.22.33.44, TLS handshaking: SSL_accept() failed: error:140800FF:SSL routines:ssl3_accept:unknown state, session=<lz7m2r2rjPouOvrk> Working client: Sep 04 19:57:58 imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization Sep 04 19:57:58 imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization Sep 04 19:57:58 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv2/v3 read client hello A Sep 04 19:57:58 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client hello A Sep 04 19:57:58 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server hello A Sep 04 19:57:58 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write certificate A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write key exchange A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server done A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client certificate A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client key exchange A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read certificate verify A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read finished A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read finished A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write change cipher spec A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write finished A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data Sep 04 19:58:03 imap-login: Debug: SSL: where=0x20, ret=1: SSL negotiation finished successfully Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2002, ret=1: SSL negotiation finished successfully Sep 04 19:58:03 imap-login: Info: Login: user=<username>, method=LOGIN, rip=11.22.33.44, lip=11.22.33.44, mpid=28781, TLS, TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits), session=<LklR9r2Vtb4uOzra> Config: # egrep -v "^#|^$" 10-ssl.conf 10-auth.conf 10-ssl.conf:ssl = required 10-ssl.conf:ssl_cert = </path/to/fullchain.pem 10-ssl.conf:ssl_key = </path/to/privkey.pem 10-ssl.conf:ssl_dh = </path/to/dh.pem 10-ssl.conf:ssl_min_protocol = TLSv1.1 10-ssl.conf:ssl_cipher_list = ALL:!kRSA:!SRP:!kDHd:!DSS:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!RC4:!ADH:!LOW@STRENGTH 10-auth.conf:disable_plaintext_auth = yes 10-auth.conf:auth_mechanisms = login 10-auth.conf:!include auth-system.conf.ext # dovecot --version 2.3.7.2 (3c910f64b) Anyone seeing something similar and/or have suggestions on how to proceed? Regards Henrik

4 7