December 2020 - dovecot

Initial Replication
by David Morsberger 01 Jan '21

01 Jan '21

I searched the archives and do not see a question/answer for this. Should I perform a set of ordered steps after setting up replication? I have setup dovecot on both ends with identical config except for the remote hostname. Main concern is the existing server gets replicated to the new server. I want nothing from new server replicated to the old server. Risking "Trial and error" and "crossing fingers” doesn’t work with production emails Specifically: Do I need to setup the mail directories for the various virtual users on the remote? If so, do the directories need to empty? Should I initially do a manually one-way synch using doveadm from the existing server? If so, is there a way of turning off automatic replication while the manual one runs? Anything else? T David

2 2

Imap client with ~7500 imap cmds ~250/~500 read() syscall
by Marc Roos 31 Dec '20

31 Dec '20

I am playing a bit with the monitoring, and I have noticed there is one client of the chart with ~7500, while the others are between ~17 and ~60 (dovecot_user_num_cmds). I assume this is related to the client, or is it possible something is wrong on the server side? Is someone having also such numbers or is this really strange. Is it possible to limit these cmd's?

3 8

mail-crypt when using passwd-file driver
by Adam Gould 31 Dec '20

31 Dec '20

Hi, I am just finishing my first personal email server build so I would really appreciate it if the response to my question can be spelled out in enough detail since I am struggling to fill in the gaps to get this working. I have scoured Google and the archives but I am not fully understanding the implementation for my setup. I am looking for a solution to encrypt email using a mechanism that increases the difficulty of decrypting my email. For example, having a plain-text decryption key on disk would not be suitable. From what I understand mail-crypt supports using the user password supplied via IMAP to decrypt email. Can someone explain how this technically works? i.e. is the private key encrypted in disk and then unlocked using the supplied password? Can someone also please step me through 1. how I would ammen the configuration below in order to set this up. 2. the command required to generate keys 3. Any help in understanding the limitations such as password changes or caveats around generating keys securely I am happy to take my notes and update the wiki to help others. Thanks, Adam # 2.3.8 (9df20d2db): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.8 (b7b03ba2) # OS: Linux 4.18.0-240.1.1.el8_3.x86_64 x86_64 Red Hat Enterprise Linux release 8.3 (Ootpa) # Hostname: mailserver.example.com debug_log_path = /var/log/dovecot-debug.log info_log_path = /var/log/dovecot-info.log lda_mailbox_autocreate = yes log_path = /var/log/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_home = /var/mail/%d/%n/ mail_location = maildir:~/mail/ mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Archive { auto = create special_use = \Archive } mailbox Drafts { auto = create special_use = \Drafts } mailbox Junk { auto = create special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = create special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%n@%d /etc/dovecot/passdb driver = passwd-file } plugin { recipient_delimiter = - sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_dir = /etc/sieve/global/ } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } } service imap-login { inet_listener imap { address = * port = 143 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl_cert = </etc/letsencrypt/live/mail.example.com/fullchain.pem ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384: DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA: ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256: DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384: AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH: !EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { args = uid=vmail gid=vmail home=/var/mail/%d/%n driver = static } protocol lmtp { mail_fsync = optimized mail_plugins = " sieve" recipient_delimiter = - }

2 1

Solr and FTS - assertion failure
by Antonino Esposito 31 Dec '20

31 Dec '20

Hi, in the latest weeks i'm working on the Solr integration and immediately i've faced the assertion failure errors, on 2.0.19, 2.2.9 and 2.3.11.3 servers in our network. Reading the thread on debian ML, I realize this issue is related to nested MIME and it affects large mailboxes In my case, the error in dovecot.log pairs with the following on solr.log and it seems the rows value has the same value of the last UID recorded in the mailbox. For your reference, here is the Solr logs, where *2276996170* is the value passed by Dovecot as rows number and it clearly don't fit with the rows data type. Have you had experienced the same behaviour? Is there a workaround? Thanks Antonino 2020-12-30 12:07:33.897 ERROR (qtp1162918744-918) [ x:qadisha] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: For input string: "2276996170" at org.apache.solr.common.params.SolrParams.getInt(SolrParams.java:236) at org.apache.solr.search.QParser.getSortSpec(QParser.java:274) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:189) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:337) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:214) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1596) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221) at org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:500) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NumberFormatException: For input string: "2276996170" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:583) at java.lang.Integer.valueOf(Integer.java:766) at org.apache.solr.common.params.SolrParams.getInt(SolrParams.java:233) ... 46 more 2020-12-30 12:07:33.897 INFO (qtp1162918744-918) [ x:qadisha] o.a.s.c.S.Request [qadisha] webapp=/solr path=/select params={q={!lucene+q.op%3DAND}from:mykeyword+OR+to:mykeyword+OR+subject:mykeyword&fl=uid,score&sort=uid+asc&fq=% 2Bbox:6d2c381de797e95fbb160000c46e3ae7+%2Buser:info@qadisha.it&rows=2276996170&wt=xml} status=400 QTime=1

2 2

Re: import_environment = $import_environment DEBUG=1 where should this be logged?
by Odhiambo Washington 30 Dec '20

30 Dec '20

Hi Marc, The same document says this: rawlog binary It works by checking if dovecot.rawlog/ directory exists in the logged in user's home directory, and writing the traffic to yyyymmdd-HHMMSS-pid.in and .out files. Each connection gets their own in/out files. *Rawlog will simply skip users who don't have the dovecot.rawlog/ directory and the performance impact for those users is minimal.* On Wed, 30 Dec 2020 at 20:10, Marc Roos <M.Roos(a)f1-outsourcing.eu> wrote: > > You know by any chance how to limit this only for one user? > > > > -----Original Message----- > From: Odhiambo Washington [mailto:odhiambo@gmail.com] > Sent: 30 December 2020 18:00 > To: dovecot > Subject: Re: import_environment = $import_environment DEBUG=1 where > should this be logged? > > > Hi Marc, > > If you read the link keenly - Debugging/Rawlog - Dovecot Wiki > <https://wiki.dovecot.org/Debugging/Rawlog> - you will realize that you > need to do some stuff. > I have done them and they produced the results. > > Edit /path/to/dovecot/conf.d/10-master.conf and add the following: > > Under this block: > > add executable = imap postlogin to the block that has service imap { > .... > ... > } > > Then create another block in the same file as follows: > > service postlogin { > executable = script-login -d rawlog > unix_listener postlogin { > } > } > > Identify the user for whom you want the rawlogs: > > doveadm user -u technical(a)mydomain.name > userdb: technical(a)mydomain.name > user : technical(a)mydomain.name > home : /var/spool/virtual/mydomain.name/technical > uid : 26 > gid : 26 > > Now do the following: > cd /var/spool/virtual/mydomain.name/technical > mkdir dovecot.rawlog > chown -R 26:26 dovecot.rawlog > > systemctl restart dovecot (or however you do it in CentOS) > > Now login to imap as the user .. perform some operations. > > Now look inside > /var/spool/virtual/mydomain.name/technical/dovecot.rawlog/ > > PS: Adapt as necessary. I tested this on FreeBSD with dovecot- 2.3.11.3 > > Ignore the /tmp/rawlog/%u portion of the HOWTO. I did it too, but there > was nothing in /tmp/rawlog/ even though I created the directory and did > chmod 1777 /tmp/rawlog. > > Adios. > > > > > On Wed, 30 Dec 2020 at 19:30, Marc Roos <M.Roos(a)f1-outsourcing.eu> > wrote: > > > > > You can also set DEBUG environment to have rawlog log an info > message > why it's not doing anything: > import_environment = $import_environment DEBUG=1 > > Where should this message appear, I also don't have this one. > > [1] > https://wiki2.dovecot.org/Debugging/Rawlog > > > > > -- > > Best regards, > Odhiambo WASHINGTON, > Nairobi,KE > +254 7 3200 0004/+254 7 2274 3223 > "Oh, the cruft.", grep ^[^#] :-) > > > -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

1 0

Re: import_environment = $import_environment DEBUG=1 where should this be logged?
by Odhiambo Washington 30 Dec '20

30 Dec '20

Hi Marc, If you read the link keenly - Debugging/Rawlog - Dovecot Wiki <https://wiki.dovecot.org/Debugging/Rawlog> - you will realize that you need to do some stuff. I have done them and they produced the results. Edit /path/to/dovecot/conf.d/10-master.conf and add the following: Under this block: add executable = imap postlogin to the block that has service imap { .... ... } Then create another block in the same file as follows: service postlogin { executable = script-login -d rawlog unix_listener postlogin { } } Identify the user for whom you want the rawlogs: doveadm user -u technical(a)mydomain.name userdb: technical(a)mydomain.name user : technical(a)mydomain.name home : /var/spool/virtual/mydomain.name/technical uid : 26 gid : 26 Now do the following: cd /var/spool/virtual/mydomain.name/technical mkdir dovecot.rawlog chown -R 26:26 dovecot.rawlog systemctl restart dovecot (or however you do it in CentOS) Now login to imap as the user .. perform some operations. Now look inside /var/spool/virtual/mydomain.name/technical/dovecot.rawlog/ PS: Adapt as necessary. I tested this on FreeBSD with dovecot- 2.3.11.3 Ignore the /tmp/rawlog/%u portion of the HOWTO. I did it too, but there was nothing in /tmp/rawlog/ even though I created the directory and did chmod 1777 /tmp/rawlog. Adios. On Wed, 30 Dec 2020 at 19:30, Marc Roos <M.Roos(a)f1-outsourcing.eu> wrote: > > > You can also set DEBUG environment to have rawlog log an info message > why it's not doing anything: > import_environment = $import_environment DEBUG=1 > > Where should this message appear, I also don't have this one. > > [1] > https://wiki2.dovecot.org/Debugging/Rawlog > -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft.", grep ^[^#] :-)

2 2

import_environment = $import_environment DEBUG=1 where should this be logged?
by Marc Roos 30 Dec '20

30 Dec '20

You can also set DEBUG environment to have rawlog log an info message why it's not doing anything: import_environment = $import_environment DEBUG=1 Where should this message appear, I also don't have this one. [1] https://wiki2.dovecot.org/Debugging/Rawlog

1 0

Getting rawlogging working with dovecot 2.2
by Marc Roos 30 Dec '20

30 Dec '20

According to this manual[1] I just have to add these lines to my config, not? protocol imap { rawlog_dir = /tmp/rawlog/%u } I should be seeing something in this /tmp/rawlog if I delete an message via imap not? I have already made this folder 777 and /tmp/rawlog/test 777, but still nothing is there. The normal syslog of dovecot does not report any errors. [1] https://wiki2.dovecot.org/Debugging/Rawlog

1 2

Virtual folder with 'younger' not updating
by Jonathan Casiot 30 Dec '20

30 Dec '20

I have a virtual folder set up with the following in dovecot-virtual: INBOX INBOX.temp ALL YOUNGER 86400 The problem is that messages older than 86,400 seconds (24 hours) never disappear from the folder. Is there any fix for this apart from adding INDEX=MEMORY to the mailbox location in the config file? I'm running Dovecot 2.2.36 on CentOS. Regards, -- Jonathan

2 6

read state not propagated
by Marc Roos 30 Dec '20

30 Dec '20

I have migrated the inbox namespace from mbox to mdbox by doing a doveadm backup to the new server. Both servers are having CentOS7 + dovecot-2.2.36-6.el7_8.1.x86_64, both servers have identical configurations (new server has some different vsz_limits and stats enabled) Now a user complains that an imap mailbox he opens at the same time with a colleague does not propagate the 'read' state. Previously if his colleague read the message, it would show on his mailbox also 'read'. After the transfer it keeps being unread for him. Am I correct to assume that he should indeed get an updated to 'read'? Where should I look to fix this? What can be the cause of this?

3 5