- dovecot - dovecot.org

Definitive guide to running ObjectiveFS on top of a Clustered File System?
by Roel van Duijnhoven 06 Jun '21

06 Jun '21

Hey there, I am working on migrating a Dovecot system with + 50.000 users / 5TB on mail data to a new set-up that is better scalable. But I found it difficult to find a good solution. In this email I lay out what I learned, in the hope that others can profit. But also others can help me on this design before I actually start migrating. *My aim is to get a clustered file system setup for Dovecot. But with the additional constraint of not having to pay on a per-mailbox basis to some vendor. * Why a clustered file system? I believe this yields the best solution. Your machines are now stateless: consuming services. And scaling or recovery is now relatively easy. Why the constraint of not paying per-mailbox? If you did pay a fee per mailbox; you probably would be better of just using a managed mail service altogether. High-level ======== We are going to use ObjectiveFS as our clustered file system (not involved with them in any way). They use Fuse to make it behave like a POSIX filesystem. Due to the way they compact files that belong together it handles a large number of small files relatively fine. The great thing is: we can mount this filesystem on multiple machines. This piece of software is proprietary, but it's price is very reasonable, and it is a fixed monthly cost. We can use S3, Google Cloud Storage, or others as the actual storage system. We store metadata about the mailboxes in a database that is available to all machines. We can use any managed database service for this. We explicitly set the proxy_host for each user in this database. Thus every user gets routed to the same machine. We make sure to distributed the users among the available machines. All traffic is received by a TCP load balancer. This will forward the traffic to the underlying machines. Ideally this is a managed cloud service. I think there are two valid choices. But I prefer the first one: - Use a TCP pass-through load balancer. This way the source IP is retained without requiring any change to the protocol (an example of this is Google Cloud external TCP/UDP Network Load Balancing <https://cloud.google.com/load-balancing/docs/network>). - Use an TCP SSL terminating load balancer. And make it forward in HaProxy's PROXY format so we retain the source IP. That traffic is now routed to N-machines. Each of those machines runs Dovecot. And Dovecot is configured with Dovecot Proxy (not to be confused with HaProxy's PROXY protocol). Such that Dovecot will proxy the connection to another machine if required (or continue on this machine if you are already on the correct proxy_host). I believe this can be done using proxy_maybe <https://doc.dovecot.org/configuration_manual/authentication/proxies/>. Depending on the choice of load balancer you may need to configure Dovecot to understand HaProxy's PROXY format. We are going to use the maildir format. And store those files on the ObjectiveFS filesystem. The index to Dovecot is stored on a local SSD. Choice motivations =============== ObjectiveFS: - Can probably be another clustered file system as well. - But ObjectiveFS works quite well because it has support for handling many small files. And can be configured with a local SSD disk. - Using this software can get a clustered file system to run quite easy. Regarding the mailbox format: - dbox is said to be more performant. And having less files is an advantage in general. But one big disadvantage: because the index contains crucial information, you will need to place this index on the Clustered File System as well. Maildir does not have this problem, and that is why we prefer it. - obox is only available as Dovecot Pro, and solves a lot of problems. But you really can only get Dovecot Pro if you pay a fee per mailbox. We don't want that! Why don't use Dovecot Director as well? - You can not run Dovecot Director on the same machine as your Dovecot Backend / proxy. Thus we would need to introduce 2 additional hosts to just accept and route the traffic. This would increase the complexity of our solution. - But it could be build on top of this set-up fairly easy. - I don't see too many advantages for Director however currently. - One big advantage *could* be if it would automatically remove failed nodes. However this only works if you use Dovemon <https://doc.dovecot.org/configuration_manual/dovemon/>. And that is packaged with OX/Dovecot and thus requires paying per user. - So in case you want to put a node down you get a slightly easier API to do so. Namely *doveadm director add <backend server ip> 0*. However we can simply imitate this by updating our database and changing the proxy_host to another destination as well. Load balancer - You could also set-up your MX records to directly point to your Dovecot nodes. This should work as well I guess. - But I think the load balancer makes for better control, over changing and managing DNS. Questions ======== In general I'd like to know whether this is a good idea! But some other questions I have at this point that could make or break this setup: - Can a Dovecot backend instance be configured to *also* run Dovecot proxy? - Can a Dovecot Proxy node receive HaProxy PROXY traffic? Details ===== Once I am sure I am on the right track I'd like to post the settings I use for the software as well. But let's first discuss the stuff above! Best, Roel van Duijnhoven

2 1

possible minor bug: lmtp excessice newlines
by Laura Steynes 06 Jun '21

06 Jun '21

Aki, unnecessary newlines in lmtp received header? Received: from mail6.xxxxxxxxxx.com by mail6.xxxxxxx.com with LMTP id xxxxxxxxxxxxxxxxxx (envelope-from <xxxxxx(a)xxxxxxxx.net>) for <xxx(a)xxxxxxxx.net>; Sat, 05 Jun 2021 14:19:07 +0100 is this intentional Aki ? Evident is multiple clients, in widescreen, so is inot client squashing the header # Dovecot version 2.3.14 # Pigeonhole version 0.5.14 # OS: Linux 5.10.41 x86_64

1 0

Panic on indexer-worker and Dovecot stop to search on Solr
by Alessio Cecchi 03 Jun '21

03 Jun '21

Hi, I have setup on a (little busy) Dovecot server FTS with Solr and Virtual folder to enable "search in all folders" for users. All works fine until for some users the indexer-worker process crash. After this crash Dovecot stop to query Solr for new search in BODY, returning SERVERBUG, for all users on the server and only with a dovecot restart users can search in BODY again. Before the dovecot restart I can see that on Solr server no query from dovecot arriving anymore (for SEARCH BODY because SEARCH in FROM/TO/SUBJECT still works fine) Dovecot version is 2.3.14 (from RPM) and Apache Solr si 7.7.3. Here is the error found in the log: May 31 14:58:13 Panic: indexer-worker(user(a)email.com) session=<VA0oxZ/DXtG5Ydlx:0KzOIkLdtGDSawAAxbYlJw>: file http-client-request.c: line 1240 (http_client_request_send_more): assertion failed: (req->payload_input != NULL) May 31 14:58:13 Error: indexer-worker(user(a)email.com) session=<VA0oxZ/DXtG5Ydlx:0KzOIkLdtGDSawAAxbYlJw>: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7f3d91b2cac2] -> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f3d91b2cbce] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf3cde) [0x7f3d91b39cde] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf3d81) [0x7f3d91b39d81] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f3d91a8b25a] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_request_send_more+0x3dd) [0x7f3d91ad39ad] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_connection_output+0xf1) [0x7f3d91ad8101] -> /usr/lib64/dovecot/libdovecot.so.0(+0x11d2b0) [0x7f3d91b632b0] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x65) [0x7f3d91b51f45] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12b) [0x7f3d91b538fb] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x59) [0x7f3d91b52049] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f3d91b52288] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_wait+0xcd) [0x7f3d91ae120d] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connection_select+0xe4) [0x7f3d8fca1174] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x45d4) [0x7f3d8fc9d5d4] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_backend_get_last_uid+0x6e) [0x7f3d90e103fe] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xf952) [0x7f3d90e16952] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x10ef6) [0x7f3d90e17ef6] -> /usr/lib64/dovecot/lib20_virtual_plugin.so(+0x966a) [0x7f3d90bfc66a] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x10ba6) [0x7f3d90e17ba6] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_precache+0x2e) [0x7f3d91e3cb0e] -> dovecot/indexer-worker [user(a)email.com Virtual/All - 5300/20832](+0x2924) [0x5648464be924] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x65) [0x7f3d91b51f45] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12b) [0x7f3d91b538fb] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x59) [0x7f3d91b52049] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f3d91b52288] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f3d91abcbb3] -> dovecot/indexer-worker [user(a)email.com Virtual/All - 5300/20832](main+0xd7) [0x5648464be1f7] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f3d9169a555] -> dovecot/indexer-worker [user(a)email.com Virtual/All - 5300/20832](+0x22ba) [0x5648464be2ba] May 31 14:58:13 Fatal: indexer-worker(user(a)email.com) session=<VA0oxZ/DXtG5Ydlx:0KzOIkLdtGDSawAAxbYlJw>: master: service(indexer-worker): child 27602 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) on Solr side I found this error log: org.apache.solr.common.SolrException: Early EOF at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:190) at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2551) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:733) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:531) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:395) [...] and [com.ctc.wstx.exc.WstxLazyException] com.ctc.wstx.exc.WstxIOException: Early EOF at com.ctc.wstx.exc.WstxLazyException.throwLazily(WstxLazyException.java:45) at com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:728) at com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3678) at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:860) at org.apache.solr.handler.loader.XMLLoader.readDoc(XMLLoader.java:419) at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:260) at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:188) at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97) This is the relevant part of dovecot configuration: plugin { fts = solr fts_autoindex = yes fts_decoder = decode2text fts_enforced = body fts_index_timeout = 5s fts_solr = url=http://10.0.1.3:8983/solr/dovecot/ [...] } Is this an already know bug? Any workaround? Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 4

Dovecot Stats for Logwatch
by Plutocrat 03 Jun '21

03 Jun '21

Hi all, I run logwatch on a few servers and the script on there which parses the dovecot logs provides waaaaaay too much detail, and contains a bunch of stuff I'm not interested in. There's not much point in having a log summary if its 100Kb long! The one for exim is also terrible, but I've replaced that with eximstats, and a bit of bash scripting and have something I can work with. I might at some point look at replacing that too. Now to my question. I haven't found anything in the dovecot family which compares to eximstats. I was wondering if there possibly is a report tool, or a way of extracting stats from doveadm just for key metrics such as 'number of POP logins in the last 24 hours', number of failed passwords, etc. Things that might be interesting on a logwatch report. Or maybe someone has written a script already that can be used in logwatch in perl, python, bash, whatever. Any pointers, suggestions, or ideas for useful summary metrics to pull out of the logfiles are appreciated. I suspect I might end up writing one myself anyway, but I'm just keen to see what already exists, get the benefit of anyone else's experiences in this matter before I start. P.

3 8

fts_autoindex_exclude for everything in a namespace?
by Daniel J. Luke 02 Jun '21

02 Jun '21

Hi, I've recently moved a system from using procmail to delivering via dovecot-lmtp w/ sieve - and it's great! I've got a copy of incoming mail going into mbox archives (that I compress after months end), and my mail log is full of "Error: fts: Failed to sync mailbox #Archive: Mailbox isn't selectable", though. I /think/ fts_autoindex_exclude = #Archive/* is what I want, but it doesn't seem to silence the error. I don't need the #Archive mailboxes visible to clients, but I don't mind if they are. I tried some setting the archive prefix = AutoArchive and setting list = yes, but that didn't silence the error. I feel like I'm probably missing something obvious :) dovecot 2.3.14 / pigenohole 0.5.14 relevant config from dovecot -n: mail_location = mbox:~/Mail/:INBOX=~/.mbox:INDEX=~/.dovecot-indexes mail_plugins = fts fts_lucene zlib namespace { inbox = yes location = mailbox Drafts { auto = no special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = no special_use = \Trash } mailbox spam { auto = create special_use = \Junk } prefix = separator = / } namespace archive { inbox = no list = no location = mbox:~/MailArchive/ prefix = "#Archive/" separator = / } plugin { fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. recipient_delimiter = + sieve = file:~/.sieve;active=~/.dovecot.sieve sieve_default = /usr/local/etc/dovecot/sieve/default.sieve sieve_default_name = file_spam } and the archive sieve script (which works) looks like this: if currentdate :matches "month" "*" { set "month" "${1}"; } if currentdate :matches "year" "*" { set "year" "${1}"; } if true { fileinto :copy :create "#Archive/${year}-${month}"; } -- Daniel J. Luke

1 0

userdb volatiledir setting not working
by Александер Скобельцын 02 Jun '21

02 Jun '21

Hi all! I just can't deal with VOLATILEDIR userdb setting on my mailservers. For example dovecot-uidlist.lock files are still being created over our network mail store. So, several days ago I changed my user and passwd query setting to include this option and restarted all my dovecot servers. Now it looks like (we use mysql driver): user_query = \ SELECT \ `u`.`pop` AS `home`, \ `u`.`uid`, \ `u`.`gid`, \ CONCAT('maildir:', `u`.`smtp`, ':INDEX=/var/dovecot_indexes/%d/%u:VOLATILEDIR=/var/dovecot_indexes/%d/%u') AS `mail`, \ CONCAT('*:bytes=', `u`.`quota`, 'M') AS `quota_rule` ,\ CONCAT('url=http://', fts_solr, ':8983/solr/dovecot/') AS fts_solr \ FROM `users` AS `u` \ JOIN `domains` AS `d` ON `d`.`domain_id` = `u`.`domain_id` \ WHERE \ `u`.`username` = '%u' \ AND `u`.`enabled` = 1 \ AND `d`.`enabled` = 1 \ AND `u`.`type` IN ('local', 'piped') password_query = \ SELECT \ `u`.`username` AS `user`, \ `u`.`crypt` AS `password`, \ `u`.`pop` AS `userdb_home`, \ `u`.`uid` AS `userdb_uid`, \ `u`.`gid` AS `userdb_gid`, \ CONCAT('maildir:', `u`.`smtp`, ':INDEX=/var/dovecot_indexes/%d/%u:VOLATILEDIR=/var/dovecot_indexes/%d/%u') AS `userdb_mail`, \ CONCAT('*:bytes=', `u`.`quota`, 'M') AS `userdb_quota_rule` \ FROM `users` AS `u` \ JOIN `domains` AS `d` ON `d`.`domain_id` = `u`.`domain_id` \ WHERE \ `u`.`username` = '%u' \ AND `u`.`enabled` = 1 \ AND `d`.`enabled` = 1 \ AND `u`.`type` IN ('local', 'piped') The aim was to move .lock and another temp files out of network storage our mails are stored. We use CephFS for storing user emails (maildir format). In addition we use 3 imap servers, each mount exact same path in cephfs and 2 dovecot directors to handle user=imapserver mapping, so user shouldn't exist on more than one imapserver at a time. Am I must use CONTROL directive in addition to force move control files from network store for VOLATILEDIR setting to work properly or I just misconfigured something? Extra info: dovecot --version 2.3.14 (cee3cbc0d)

1 0

debugging file permissions wrong
by Laura Steynes 02 Jun '21

02 Jun '21

Hi, In trying to debug a strange error where client can't login, I enabled all the usual debug settings, this is all good, it works for imap and pop3 fine, but the problem is when used with dovecot's LDA there is a nasty issue. the file created by debug_log_path in this case /var/log/dovecot/debug.log , this file created as root, again this is fine for nice logging of imap and pop3, but this causes postfix not to deliver mail, for write permission denied, LDA runs as vmail, all my sub sections like *_listener also = vmail and all my uid/gid settings are also to user/group vmail obviously you wont main log files to not be owned by vmail for case of security, so is there a way to set the ownership of the debug file - apart from the obvious of remembering 40 minutes later when you get alert of high mailq level to chown the file :) If there is no way, may the developers take this as a feature request please. Thanks Loz

2 3

New Configuration
by Eric Shields | Mass Transit Honchkrow 29 May '21

29 May '21

Hi, I'm setting up an email server behind a NAT, and I have the necessary DNS records, iptables/ufw rules, and ACLs in place. However, I cannot sign in with my chosen username, which does exist on the same computer I'm hosting the server on. I am using PAM, not a database, to hold credential information, and SSL certificates are active. However, I am getting signal 15 for the following processes: anvil, config, and auth. I can confirm through telnet that the ports I am using are open, and these ports are also open on the WAN side, which I believe is needed. Do you have any pointers for me, or any additional clarifying details? - 🚏

2 1

Different Maildir Locations
by Gabx 28 May '21

28 May '21

Hi, on my vps (debian stable) I have an email system with postfix, dovecot, virtual users and domains. I have 3 users with different mail_locations. the first mail_location = maildir: ~ / Maildir the second mail_location = maildir: ~ / user1 / Maildir the third mail_location = maildir: ~ / user2 / mail In postfix the above schema is represented in virtual_mailbox_maps. In dovecot mail_location is set maildir:~/Maildir. The first user receives regular mail, the other two none. I tried to see if in dovecot there was the possibility to configure the mail_location specifically for each single user, but with namespaces and userdb I am a bit confused and I have not found specific examples for my case. regards Gab -- GPG fingerprint: 5A6A F742 ACEA C9A7 93A8 8D5E 7AEF 5A02 D080 89F7 Contact: xmpp://gabx@jabber.tcpreset.net

3 14

replicator: Panic: data stack: Out of memory when allocating ...
by Jörg Faudin Schulz 28 May '21

28 May '21

frequently, the replicator fails on allocating memory: replicator: Panic: data stack: Out of memory when allocating ... bytes. I already have increased default_vsz_limit=1024M but this only increased the ... value. What can be done here? dovecot_branch=release-2.3.14

2 3

Mailbox problem on centos O/S after some "fiddling"
by John Richards 28 May '21

28 May '21

Good day from Somerset, England Version 2.2.10 Problem after a huge centos update Data:- Server has 8 mailboxes under /var/mail/"user name" history:- After update, mail to ONE user was going into mqueue as undeliverable Had to delete and recreate the user. somewhere along the line (ME!) /var/mail/;imap var/mail/new were deleted After new user and reboot server no one could access mailboxes with password error except ONE user randomly....This created the /var/mail/.mbox folder and sub folders but with the owner as the user name. Then any mail for other users was put in the /var/new folder and just sat there. only the "owner" of this folder could get mail, everyone else was prohibited with password error. chown the folder (owner:group) to root did nothing BUT When .mbox was give PUBLIC execute permissions (771) suddenly everything works fine! I do not understand why dovecot creates this folder as we ONLY ever use pop3 (currently with plain text auth but going to secure when this sorted) for mail and also why when it writes the blank folder it "owns" it with the user accessing it (also the /var/mail/new folder takes the same owner) I have trolled everywhere and spent a whole day on this so any clues would be very gratefully accepted as do not like to leave this folder as 771 (note the folder INBOX within it is only 770 and the log file in that is 660) Is this INBOX needed at all for pop3 ? rgds John Richards UK ================= dovecot -n # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-514.2.2.el7.x86_64 x86_64 CentOS Linux release 7.3.1611 (Core ) ext4 auth_mechanisms = plain login disable_plaintext_auth = no mail_location = mbox:/var/mail/:INBOX=/var/mail/%u mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = shadow } protocols = pop3 service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } ssl_cert = </etc/pki/tls/certs/watchetwebdesign.co.uk.crt ssl_key = </etc/pki/tls/private/watchetwebdesign.co.uk.key userdb { driver = passwd }

2 1

NFS Locking Question
by Zane Zak 28 May '21

28 May '21

Hi! I'm hoping to get some clarification on NFS locking before deploying a new setup. We have a single exim server that delivers mail to mbox files in /var/mail. We have a separate server running dovecot that reads mail from these mbox files. So only one exim server and one dovecot server, but they're separate VMs and handle locking with dotlocks. I was noticing frequent mbox file corruption. I then set "mail_nfs_storage = yes", and have seen no mbox corruption since. However, I'm now seeing messages like this in the logs for every read... Error: nfs_flush_fcntl: fcntl(/var/mail/username, F_RDLCK) failed: Bad file descriptor Is there a configuration change that needs to be made on the dovecot or the NFS servers, or are these messages harmless. Thanks! ZZ

2 1

mail-crypt-plugin: encrypted user keys
by Daniel Schuermann 28 May '21

28 May '21

Hi, I tried to enable encrypted folder keys using mail-crypt-plugin. It works as expected when using unencrypted folder keys. When I add mail_crypt_require_encrypted_user_key = yes as shown below, I somehow manage to crash dovecot: dovecot: lmtp(82060): Fatal: master: service(lmtp): child 82060 killed with signal 6 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set service lmtp { drop_priv_before_exec=yes }) dovecot: lmtp(67814): Panic: file mail-user.c: line 229 (mail_user_deinit): assertion failed: ((*user)->refcount== 1) lmtp(root): Info: msgid=<07e3a23b2aaea60b(a)mx.2718282.net>: save failed to INBOX: generate_keypair(INBOX) failed: mail_crypt_require_encrypted_user_key set, cannot generate user keypair without password or key My config files: # 2.3.14 (cee3cbc0d): /etc/mail/imap.conf # OS: OpenBSD 6.9 amd64 auth_verbose = yes debug_log_path = /var/log/dovecot info_log_path = /var/log/dovecot mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_debug = yes namespace inbox { ... } passdb { args = /etc/mail/imap-sqlite.conf driver = sql } plugin { mail_crypt_curve = secp521r1 mail_crypt_require_encrypted_user_key = yes mail_crypt_save_version = 2 } protocols = imap lmtp service imap-login { ... } ssl = required ssl_cert = </etc/ssl/rsa.crt ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { args = /etc/mail/imap-sqlite.conf driver = sql override_fields = uid=vmail gid=vmail } # file: /etc/mail/imap-sqlite.conf driver = sqlite connect = /etc/mail/sqlite.db default_pass_scheme = BLF-CRYPT user_query = SELECT '/home/vmail/'||destination AS home FROM virtuals WHERE email = '%u' password_query = SELECT email as user, password, '%w' AS \ userdb_mail_crypt_private_password FROM credentials WHERE email = '%u'

2 1

missing +w perm
by Jeff Abrahamson 28 May '21

28 May '21

I've started seeing the following error in mail.err for precisely one user on precisely one domain: Error: mkdir(/var/mail/vmail//jeff/mail) failed: Permission denied (euid=1000(jeff) egid=1001(jeff) missing +w perm: /var/mail/vmail/, dir owned by 4000:4000 mode=0755) Error: Mailbox INBOX: Failed to autocreate mailbox: Internal error occurred. In mail.log I see a bit more: Warning: Failed to parse address from `From_'-line: Missing domain Error: mkdir(/var/mail/vmail//jeff/mail) failed: Permission denied (euid=1000(jeff) egid=1001(jeff) missing +w perm: /var/mail/vmail/, dir owned by 4000:4000 mode=0755) Error: mkdir(/var/mail/vmail//jeff/mail) failed: Permission denied (euid=1000(jeff) egid=1001(jeff) missing +w perm: /var/mail/vmail/, dir owned by 4000:4000 mode=0755) Error: Mailbox INBOX: Failed to autocreate mailbox: Internal error occurred. Refer to server log for more information. save failed to open mailbox INBOX: Mailbox INBOX: Failed to autocreate mailbox: Internal error occurred. And yet there's no apparent problem sending and receiving mail for user jeff, for all domains handled by the server. Other users have no issues. The /var/mail/vmail/ hierarchy is all owned by vmail:vmail (4000:4000) and directories are all 0755. I suspect a red herring except it's been continuing for a week now. I was reading this page (following some searching), but it doesn't seem pertinent in the end. https://wiki2.dovecot.org/MailLocation/mbox#A.2BAC8-var.2BAC8-mail.2BAC8_do… Any suggestions what this might be? -- Jeff Abrahamson +33 6 24 40 01 57 +44 7920 594 255 http://p27.eu/jeff/ http://mobilitains.fr/

2 2

backup v rsync
by Jeff Abrahamson 27 May '21

27 May '21

I've been migrating from a dovecot 2.2.22 instance to a dovecot 2.3.7.2 instance. (Said differently, from an ubuntu 16.04 host to a 20.04 host.) It's a tiny site: only a handful of users actually had to migrate, most either died on the old one or were born on the new one. The main user in need of migration was me. After fiddling with dsync and not getting it working, I threw in the towel. This is a migration I had to do once for precisely one user. I had been using the old host exclusively, so it was easy to shut down both dovecot instances the time it took me to rsync my data from one to the other. Now that seems to work, almost. Mail is being delivered to my inbox, anyway. Most other folders are looking inaccessible to my mail client (thunderbird, for this test). The main difference between the two servers, aside from the above, is a switch from mail_location = maildir:~/Maildir to mail_home = /var/mail/vmail/%d/%n mail_location = maildir:/var/mail/vmail/%d/%n/mail:LAYOUT=fs mail_privileged_group = mail I corrected permissions and ownership when I copied the data. Now I think the problem is that LAYOUT=fs means that I should have a slightly different folder hierarchy now. The question is how best to fix this. I have things like this .Archives.2020 .Archives.2020/new .Archives.2020/tmp .Archives.2020/cur when I think I should have things like this Archives Archives/2021 Archives/2021/new Archives/2021/tmp Archives/2021/cur And, again, this is one-off, so a manual fix seems most appropriate. I'm quite tempted simply to shut down the dovecot instance for a minute at a quiet moment (mail retries...) and rename directories as indicated above. I only have to rename five directories. Having gotten myself in a bit of a muddle on this, I thought perhaps a reality check that I'm not about to make it worse would be appropriate. Many thanks. -- Jeff Abrahamson +33 6 24 40 01 57 +44 7920 594 255 https://www.p27.eu/jeff/ https://www.mobilitains.fr/

2 2

API command for ACL Delete
by Rhodri Jones 27 May '21

27 May '21

I asked previously for info about the API options for doveadm acl delete but I think my question was misunderstood. I want to know the command I would use (with curl for instance) that corresponds with: doveadm acl delete -u <targetmailbox> INBOX user=<targetuser> Using the python client for doveadm http API I can see the valid parameters but I don't seem to be able to get the correct syntax. (doveadm) aclDelete ? invalid parameter: ? valid parameters for aclDelete: - userFile (string) - allUsers (boolean) - mailbox (string) - user (string) - socketPath (string) - id (string) --

2 2

API command for ACL Delete
by Rhodri Jones 26 May '21

26 May '21

Can anyone tell me how to do the following using the API. doveadm acl get -u <targetmailbox> INBOX doveadm acl delete -u <targetmailbox> INBOX user=$user Are the ACL API options documented somewhere? Thanks --

2 1

Re: [EXT] Re: Dovecot Stats for Logwatch
by Plutocrat 26 May '21

26 May '21

On 26/05/2021 14.33, Aki Tuomi wrote: > I am not sure what is not working for you... > does `doveadm stats dump` work? a) Before any stats config it works without error, but gives no output. doveadm stats dump metric_name field count sum min max avg median stddev %95 b) If I then enable conf.d/95-stats.conf containing service stats { inet_listener http { port = 9900 } } metric client_connections { filter = event=client_connection_finished } metric auth_success { filter = (event=auth_request_finished AND success=yes) } metric mail_delivery { filter = event=mail_delivery_finished } ... it works but doesn't seem to collect any data, even if left for 24 hours doveadm stats dump metric_name field count sum min max avg median stddev %95 client_connections duration 0 0 0 0 0.00 0 0.00 0 auth_success duration 0 0 0 0 0.00 0 0.00 0 mail_delivery duration 0 0 0 0 0.00 0 0.00 0 c) If I then enable the bits of config that I'm informed will populate the data repository: # In 10-master.conf mail_plugins = stats # In 20-imap.conf protocol imap { mail_plugins = $mail_plugins quota imap_quota } # In 20-pop.conf protocol pop3 { mail_plugins = $mail_plugins quota } ... it then complains doveadm stats dump Fatal: Plugin 'stats' not found from directory /usr/lib/dovecot/modules d) I just changed the in in 10-master-conf to reference old-stats instead of stats, based on my new knowledge and it doesn't complain about the missing module any more. But I'm still not collecting data after 30 mins. doveadm stats dump metric_name field count sum min max avg median stddev %95 client_connections duration 0 0 0 0 0.00 0 0.00 0 auth_success duration 0 0 0 0 0.00 0 0.00 0 mail_delivery duration 0 0 0 0 0.00 0 0.00 0 I can do wget http://localhost:9900/metrics and I just get a file that says VERSION stats-reader-server 2 0 At this point I'm thinking I'll just write a bash script to parse the logs, but thought I'd see it through as it might help other intrepid dovecot users. P.

2 3

Problem with per-user \Seen flag on public folder
by Bartek Szady 26 May '21

26 May '21

Hello I have noticed that when per-user \Seen flag is configured dovecot do not respond properly to 'store' command. eg.: =============== 42 uid store 6 +Flags (\Seen) 42 OK Store completed (0.001 + 0.000 secs). 43 IDLE + idling DONE 43 OK Idle completed (3.204 + 3.203 + 3.203 secs). =============== Thunderbird is confused by this and read emails turns into unread after a while. When global flag is stored dovecot responds as it should, I think: =============== 32 uid store 3 -Flags (\Flagged) * 2 FETCH (UID 3 FLAGS ()) 32 OK Store completed (0.002 + 0.000 + 0.001 secs). 33 IDLE + idling DONE 33 OK Idle completed (5.362 + 5.362 + 5.361 secs). =============== When per-user \Seen flag is not configured: =============== 25 uid store 6 +Flags (\Seen) * 4 FETCH (UID 6 FLAGS (\Seen)) 25 OK Store completed (0.002 + 0.000 + 0.001 secs). 26 IDLE + idling * OK Still here DONE 26 OK Idle completed (65.076 + 65.076 + 65.076 secs). =============== # doveconf -n # 2.3.14 (cee3cbc0d): dovecot-int.conf # OS: Linux 5.1.16 x86_64 # Hostname: #hidden auth_mechanisms = plain login base_dir = /var/run/dovecot-int default_internal_group = dovecot1 default_internal_user = dovecot1 default_login_user = dovenull1 disable_plaintext_auth = no login_log_format = int %$: %s mail_debug = yes mail_location = maildir:~/Mail/Maildir mail_log_prefix = "int-%s(%u)<%{pid}><%{session}>: " namespace { inbox = yes location = prefix = separator = / type = private } passdb { args = scheme=MD5-CRYPT /etc/dovecot/passwd-int driver = passwd-file } passdb { args = scheme=MD5-CRYPT /etc/dovecot/virtual-int driver = passwd-file } passdb { driver = pam } plugin { acl = vfile } protocols = pop3 imap service auth { user = root } service imap-login { group = dovenull1 inet_listener imap { address = <ips> } user = dovecot1 } service pop3-login { group = dovenull1 inet_listener pop3 { address = <ips> } } service stats { unix_listener stats-writer { mode = 0660 user = pubmail } } ssl_cert = </var/lib/openssl/certs/imap-int.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/virtual-int driver = passwd-file } userdb { driver = passwd } protocol imap { mail_plugins = acl imap_acl namespace { location = maildir:/srv/mail/public:LAYOUT=fs:INDEXPVT=~/Mail/public prefix = publiczne/ separator = / subscriptions = no type = public } ssl = yes } Bartek

2 1

What am I doing wrong?
by Ron Garret 26 May '21

26 May '21

I’m trying to run an imapsieve script when a user sends a message (this is part of a spam filter and I want to use outgoing messages to train the filter on good content). This is my config: plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve = file:/var/mail/sieve/incoming.sieve # From elsewhere to Junk imapsieve_mailbox1_name = Junk imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_before = file:/var/mail/sieve/spam.sieve # From Junk to elsewhere imapsieve_mailbox2_name = * imapsieve_mailbox2_from = Junk imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_before = file:/var/mail/sieve/ham.sieve # Sent imapsieve_mailbox3_name = 'Sent Messages' imapsieve_mailbox3_causes = APPEND imapsieve_mailbox3_before = file:/var/mail/sieve/sent.sieve sieve_pipe_bin_dir = /var/mail/sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment } The first two scripts (for training spam and ham respectively) both work. Once upon a time I had the third one working as well, but apparently I changed something that made it break and I have no idea what that could be. (Yes, I need to get my code into revision control. I’ve already beaten myself up over that more than once.) The problem appears to be in the config, not the sieve script, because the script is never compiled into a .svbin while the other scripts in the same directory are compiled. But just for the record, this is the sieve script: > ➔ cat /var/mail/sieve/sent.sieve > require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"]; > > if environment :matches "imap.user" "*" { > set "username" "${1}"; > } > > pipe :copy "train.py" [ "sent", "${username}" ]; Does anyone here see the problem that I’m missing? Thanks, rg

1 1

sqlite + IMAP Authentication issues - user unknown
by Scott Edwards 25 May '21

25 May '21

Hello, I'm not on the maillist, please reply all. It's been a few years since I've setup a mail server (I feel like a noob!). The following syslog message is isolating the auth issues. I have sqlite, and want to keep it this way. I don't see more than a few mailboxes to setup until sometime later, but if it does grow, I'll stand up PostgreSQL. FQDN is 'localcar.repair'. May 24 21:01:45 localcar dovecot: auth: Debug: client in: AUTH#0116#011PLAIN#011service=imap#011secured=tls#011session=Rb8dshnD/Gwv2Ixo#011lip=45.33.2.221#011rip=47.216.140.104#011lport=993#011rport=27900#011local_name=localcar.repair#011ssl_cipher=TLS_AES_128_GCM_SHA256#011ssl_cipher_bits=128#011ssl_pfs=KxANY#011ssl_protocol=TLSv1.3#011resp=<hidden> May 24 21:01:49 localcar dovecot: auth-worker(20614): Debug: sql(webmaster-foo(a)localcar.repair,47.216.140.104,<Rb8dshnD/Gwv2Ixo>): query: SELECT password, userid as user FROM users WHERE userid = 'webmaster-foo' AND domain = 'localcar.repair' May 24 21:01:49 localcar dovecot: auth-worker(20614): sql(webmaster-foo(a)localcar.repair,47.216.140.104,<Rb8dshnD/Gwv2Ixo>): unknown user (given password: topsecret) May 24 21:01:51 localcar dovecot: auth: Debug: client passdb out: FAIL#0116#011user=webmaster-l5buyjuxao8@localcar.repair#011original_user=webmaster-foo@localcar.repair May 24 21:01:51 localcar dovecot: imap-login: Debug: Ignoring unknown passdb extra field: original_user root@localcar:/etc/dovecot# sqlite3 users SQLite version 3.27.2 2019-02-25 16:06:06 Enter ".help" for usage hints. sqlite> .schema CREATE TABLE users ( userid VARCHAR(128) NOT NULL, domain VARCHAR(128) NOT NULL, password VARCHAR(64) NOT NULL, home VARCHAR(255) NOT NULL, uid INTEGER NOT NULL, gid INTEGER NOT NULL ); sqlite> select * from users; webmaster-foo|localcar.repair|topsecret|/tmp|1000|1000 Thunderbird naturally says "Login to server localcar.repair with username webmaster-foo(a)localcar.repair failed. [ ok ] Restarting dovecot (via systemctl): dovecot.service. root@localcar:/etc/dovecot# dovecot -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-16-amd64 x86_64 Debian 10.9 # Hostname: localcar.repair auth_debug = yes auth_verbose = yes auth_verbose_passwords = yes mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocols = " imap lmtp" ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } Thank you, Scott

2 2

Help with "doveadm" - Socket ?
by White, Daniel E. (GSFC-770.0)[NICS] 25 May '21

25 May '21

doveadm [-f formatter] mailbox status [-A|-u user] [-S socket_path] [-t] fields mailbox ... Where do I find this "socket_path", please ? I am trying to untangle a very old CentOS 6 instance. Thanks in advance.

4 7

Panic: file ostream-zlib.c
by Bouke J. Henstra | ICT Diensten 24 May '21

24 May '21

Hello, At the moment I am investigating an issue (probably) related to Dovecot /zlib. I am hoping someone is willing to help. Thanks. #Background information Every time a certain user checked her e-mail the following errors would appear in "/var/log/mail.err" (please find the lines below). I am using zlib (please find the details in the configuration below). I did include the Dovecot version, OS details, CPU details and file system details in this message. I did migrate the whole mail domain to Microsoft 365 (as the user has an active subscription and had to move on - for me this was the fastest method to help the user in this case). The mailbox still exists on the server. I suspect there's file corruption somewhere? I am routing mail from the domain to xxxxx-xx.mail.protection.outlook.com (just to be sure that the user doesn't miss anything - mx records have been updated). I did notice that the user had issues with receiving mail. Outlook would keep trying to fetch mail and was hard to close (user had to kill Outlook through the task manager - after migrating to Microsoft 365 this problem is resolved too). I would like to find out what causes/caused this problem. I would like to understand the problem and would like to try to avoid future problems. I did run "dovecot-sysreport" but I did notice the data is not anonymized. I can anonymize the details and submit those if required. #Dovcecot version root@s1:/# dovecot --version 2.3.4.1 (f79e8e7e4) #Operating system root@s1:/# uname -a Linux s1 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 GNU/Linux root@s1:/# cat /etc/debian_version 10.9 #CPU Architecture root@s1:/# lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian #File System ext4 errors=continue,usrjquota=quota.user,grpjquota=quota.group,jqfmt=vfsv0 0 1 #Errors May 17 22:37:11 s1 dovecot: imap(xxxx(a)xxxxx.xx)<18570><MxyEjIzCs8pSoYVo>: Panic: file ostream-zlib.c: line 37 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0 || zstream->ostream.error_handling_disabled) May 17 22:37:11 s1 dovecot: imap(xxxx(a)xxxxx.xx)<18570><MxyEjIzCs8pSoYVo>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xdb73b) [0x7f45c109973b] -> /usr/lib/dovecot/libdovecot.so.0(+0xdb7d1) [0x7f45c10997d1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a199) [0x7f45c1008199] -> /usr/lib/dovecot/modules/lib20_zlib_plugin.so(+0x3c6f) [0x7f45c0dc3c6f] -> /usr/lib/dovecot/libdovecot.so.0(+0xffd5e) [0x7f45c10bdd5e] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_destroy+0x16) [0x7f45c10bdd86] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x18d) [0x7f45c11d046d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x4d) [0x7f45c11a867d] -> dovecot/imap(+0x10f36) [0x5606b3187f36] -> dovecot/imap(+0x10ff3) [0x5606b3187ff3] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6f) [0x7f45c10afbef] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x136) [0x7f45c10b11e6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f45c10afc8c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f45c10afdf0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f45c1030123] -> dovecot/imap(main+0x325) [0x5606b3187bf5] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f45c0e1309b] -> dovecot/imap(_start+0x2a) [0x5606b3187d8a] May 17 22:37:11 s1 dovecot: imap(xxxx(a)xxxxx.xx)<18570><MxyEjIzCs8pSoYVo>: Fatal: master: service(imap): child 18570 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) May 17 22:54:52 s1 dovecot: imap(xxxx(a)xxxxx.xx)<20321><HrbpyozCn8tSoYVo>: Panic: file ostream-zlib.c: line 37 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0 || zstream->ostream.error_handling_disabled) May 17 22:54:52 s1 dovecot: imap(xxxx(a)xxxxx.xx)<20321><HrbpyozCn8tSoYVo>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xdb73b) [0x7fda4a55473b] -> /usr/lib/dovecot/libdovecot.so.0(+0xdb7d1) [0x7fda4a5547d1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a199) [0x7fda4a4c3199] -> /usr/lib/dovecot/modules/lib20_zlib_plugin.so(+0x3c6f) [0x7fda4a27ec6f] -> /usr/lib/dovecot/libdovecot.so.0(+0xffd5e) [0x7fda4a578d5e] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_destroy+0x16) [0x7fda4a578d86] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x18d) [0x7fda4a68b46d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x4d) [0x7fda4a66367d] -> dovecot/imap(+0x10f36) [0x5559c9e82f36] -> dovecot/imap(+0x10ff3) [0x5559c9e82ff3] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6f) [0x7fda4a56abef] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x136) [0x7fda4a56c1e6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7fda4a56ac8c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7fda4a56adf0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fda4a4eb123] -> dovecot/imap(main+0x325) [0x5559c9e82bf5] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7fda4a2ce09b] -> dovecot/imap(_start+0x2a) [0x5559c9e82d8a] May 17 22:54:52 s1 dovecot: imap(xxxx(a)xxxxx.xx)<20321><HrbpyozCn8tSoYVo>: Fatal: master: service(imap): child 20321 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) #Config root@s1:/opt/scripts# dovecot -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-16-amd64 x86_64 Debian 10.9 # Hostname: s1.xxxxx.xx auth_mechanisms = plain login disable_plaintext_auth = no imap_capability = +SPECIAL-USE XLIST listen = *,[::] lmtp_rcpt_check_quota = yes log_timestamp = "%Y-%m-%d %H:%M:%S " mail_max_userip_connections = 100 mail_plugins = quota zlib mail_privileged_group = vmail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO sieve = /var/vmail/%d/%n/.sieve sieve_after = /var/vmail/%d/%n/.ispconfig.sieve sieve_before = /var/vmail/%d/%n/.ispconfig-before.sieve sieve_max_actions = 100 sieve_max_redirects = 25 sieve_max_script_size = 2M zlib_save = gz zlib_save_level = 9 } protocols = imap pop3 lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service imap-login { client_limit = 1000 process_limit = 512 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service quota-status { client_limit = 1 executable = quota-status -p postfix unix_listener /var/spool/postfix/private/quota-status { group = postfix mode = 0660 user = postfix } } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl_cert = </etc/postfix/smtpd.cert ssl_cipher_list = ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384 ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { auth_verbose = yes mail_plugins = quota zlib quota imap_quota } protocol pop3 { auth_verbose = yes mail_plugins = quota zlib quota pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = quota zlib sieve quota postmaster_address = postmaster(a)xxxxx.xx } protocol lmtp { mail_plugins = quota zlib quota sieve postmaster_address = postmaster(a)xxxxx.xx -- Kind regards, Bouke J. Henstra

2 1

BUG report
by Oleg Pyzin 23 May '21

23 May '21

Hello! I've compiled Dovecot without any error, but couldn't run it. ------------- The error is: ------------- Feb 09 19:22:50 master: Panic: kevent(EV_ADD, READ, 21) failed: Invalid argument Feb 09 19:22:50 master: Error: Raw backtrace: 0 libdovecot.0.dylib 0x00216d58 askpass + 136 -> 1 libdovecot.0.dylib 0x00216e78 backtrace_get + 36 -> 2 libdovecot.0.dylib 0x00221fe8 i_set_debug_file + 248 -> 3 libdovecot.0.dylib 0x002220fc default_fatal_handler + 56 -> 4 dovecot 0x000039a0 dup2_append + 1884 -> 5 libdovecot.0.dylib 0x00221a8c i_panic + 120 -> 6 libdovecot.0.dylib 0x0022c13c io_loop_handle_add + 244 -> 7 libdovecot.0.dylib 0x0022f2c8 timeout_add_short + 416 -> 8 libdovecot.0.dylib 0x0022f574 io_add_to + 76 -> 9 dovecot 0x00007dd4 service_monitor_stop_close + 288 -> 10 dovecot 0x00008cdc services_monitor_start + 840 -> 11 dovecot 0x00004aa4 main + 3336 -> 12 dovecot 0x00003048 start + 68 ------------- dovecot -n: admin.MacPPC[/opt/local/var/log/dovecot] <467-7> $ sudo dovecot -n # 2.3.11.3 (): /opt/local/etc/dovecot/dovecot.conf # OS: Darwin 9.8.0 Power Macintosh hfs # Hostname: MacPPC.Dlink default_client_limit = 50 default_internal_group = _dovecot default_process_limit = 10 default_vsz_limit = 128 M info_log_path = /opt/local/var/log/dovecot/info listen = * log_path = /opt/local/var/log/dovecot/error login_plugin_dir = /opt/local/lib/dovecot mail_location = maildir:/private/srv/mail/%u namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } protocols = imap pop3 service imap { process_limit = 32 } service pop3 { process_limit = 32 } service submission { process_limit = 32 } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } ------------------------------ Will be a very gratefull for the help, Oleg

1 0

Issue with dovecot v2.3.4.1 and mail_crypt plugin. Email received, but not encrypted on disk until moved out of root Maildir folder. IMAP child crashes when mail is read from folder message was moved to.
by Ben Burk 22 May '21

22 May '21

Recently I attempted enabling the mail_crypt plugin on a per user basis as summarized below: uname -a Linux smtp 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 GNU/Linux dovecot --version 2.3.4.1 (f79e8e7e4) $ cat /etc/dovecot/conf.d/10-mailcrypt.conf mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_plugins = $mail_plugins mail_crypt plugin { mail_crypt_curve = secp521r1 mail_crypt_save_version = 2 mail_crypt_require_encrypted_user_key = yes } cat dovecot-sql.conf.ext driver = mysql connect = host=127.0.0.1 dbname=VVVVV user=WWWWWW password=XXXXXX user_query = SELECT user, \ concat('*:bytes=', quota) AS quota_rule, \ '/mail/%d/%n' AS home, \ uid, gid \ FROM virtual_users WHERE user='%u'; password_query = SELECT user, password, crypt as userdb_mail_crypt_save_version, '%w' AS userdb_mail_crypt_private_password FROM virtual_users WHERE user='%u'; dovecot -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-16-amd64 x86_64 Debian 10.9 # Hostname: smtp.burk.tech auth_verbose = yes mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_location = maildir:~/Maildir:LAYOUT=fs mail_plugins = quota mail_crypt mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { driver = pam } plugin { mail_crypt_curve = secp521r1 mail_crypt_require_encrypted_user_key = # hidden, use -P to show it mail_crypt_save_version = 2 } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0777 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = </etc/dovecot/fullchain.cer.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { driver = passwd } verbose_ssl = yes protocol lmtp { hostname = smtp.burk.tech postmaster_address = postmaster(a)burk.tech } protocol imap { mail_max_userip_connections = 50 } The issue I've noticed occurs when the following settings above are enabled and mail is received into the root of the Maildir fs, then mail is moved to another folder (including a file being deleted). Upon a client's retrieval of messages from the folder where mail was moved to, the log spits out the following: May 22 19:18:15 smtp dovecot: imap-login: Login: user=<ben>, method=PLAIN, rip=XXX.XXX.XXX.XXX, lip=YYY.YYY.YYY.YYY, mpid=31682, TLS, session=<7oASZPPCkN9EdIVp> May 22 19:18:15 smtp dovecot: imap(ben)<31682><7oASZPPCkN9EdIVp>: Error: Mailbox Trash: Saving mail: save: read() failed: read(/mail/burk.tech/ben/Maildir/Receipts/cur/1621724316.M515979P30417.smtp,S=2903,W=2964:2,S) failed: Private key not available: Cannot decrypt key f9e7a6b9dd24535a29915fb46336047b07ae5b9fea5eda8e03cf9a8fa009f88b: Cannot decrypt key 876c6bfbc355a86c3c924a5841bb2e5c6e1b0c5496d5116c4fcb9c511f803c87: Password not available May 22 19:18:15 smtp dovecot: imap(ben)<31682><7oASZPPCkN9EdIVp>: Error: Mailbox Trash: copy: i_stream_read() failed: read(/mail/burk.tech/ben/Maildir/Receipts/cur/1621724316.M515979P30417.smtp,S=2903,W=2964:2,S) failed: Private key not available: Cannot decrypt key f9e7a6b9dd24535a29915fb46336047b07ae5b9fea5eda8e03cf9a8fa009f88b: Cannot decrypt key 876c6bfbc355a86c3c924a5841bb2e5c6e1b0c5496d5116c4fcb9c511f803c87: Password not available May 22 19:18:15 smtp dovecot: imap(ben)<31682><7oASZPPCkN9EdIVp>: Error: Mailbox Trash: Saving mail: read() failed: read(/mail/burk.tech/ben/Maildir/Receipts/cur/1621724316.M515979P30417.smtp,S=2903,W=2964:2,S) failed: Private key not available: Cannot decrypt key f9e7a6b9dd24535a29915fb46336047b07ae5b9fea5eda8e03cf9a8fa009f88b: Cannot decrypt key 876c6bfbc355a86c3c924a5841bb2e5c6e1b0c5496d5116c4fcb9c511f803c87: Password not available (read reason=) May 22 19:18:15 smtp dovecot: imap(ben)<31682><7oASZPPCkN9EdIVp>: Panic: file ostream-encrypt.c: line 631 (o_stream_encrypt_close): assertion failed: (estream->finalized || estream->ctx_sym == NULL || estream->ostream.ostream.stream_errno != 0) May 22 19:18:15 smtp dovecot: imap(ben)<31682><7oASZPPCkN9EdIVp>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xdb73b) [0x7fdb3ce7273b] -> /usr/lib/dovecot/libdovecot.so.0(+0xdb7d1) [0x7fdb3ce727d1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a199) [0x7fdb3cde1199] -> /usr/lib/dovecot/libdovecot.so.0(+0x49276) [0x7fdb3cde0276] -> /usr/lib/dovecot/libdovecot.so.0(+0xffd5e) [0x7fdb3ce96d5e] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_destroy+0x16) [0x7fdb3ce96d86] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x18d) [0x7fdb3cfa946d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x4d) [0x7fdb3cf8167d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0x122) [0x7fdb3cf73f22] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x5badc) [0x7fdb3cf81adc] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_move+0x31) [0x7fdb3cf81c11] -> dovecot/imap(+0x128f5) [0x563bd267a8f5] -> dovecot/imap(command_exec+0x70) [0x563bd2687dc0] -> dovecot/imap(+0x1e3f2) [0x563bd26863f2] -> dovecot/imap(+0x1e494) [0x563bd2686494] -> dovecot/imap(client_handle_input+0x1b5) [0x563bd2686845] -> dovecot/imap(client_input+0x7e) [0x563bd2686d6e] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6f) [0x7fdb3ce88bef] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x136) [0x7fdb3ce8a1e6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7fdb3ce88c8c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7fdb3ce88df0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fdb3ce09123] -> dovecot/imap(main+0x325) [0x563bd2678bf5] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7fdb3cbf209b] -> dovecot/imap(_start+0x2a) [0x563bd2678d8a] May 22 19:18:15 smtp dovecot: imap-login: Debug: SSL alert: close notify May 22 19:18:15 smtp dovecot: imap(ben)<31682><7oASZPPCkN9EdIVp>: Fatal: master: service(imap): child 31682 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) My keys for the user are as follows: ben(a)smtp.burk.tech :~$ doveadm mailbox cryptokey list -U doveadm(ben): Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied Folder Active Public ID yes 876c6bfbc355a86c3c924a5841bb2e5c6e1b0c5496d5116c4fcb9c511f803c87 I've checked and it appears that the mail as its received into the root of Maildir is not initially encrypted. It is stored plaintext. However, when the mail is moved, it is then properly encrypted. ben(a)smtp.burk.tech :~/Maildir/cur$ file 1621726534.31820_1.smtp\:2\, 1621726534.31820_1.smtp:2,: SMTP mail, ASCII text ben(a)smtp.burk.tech :~/Maildir/Receipts/cur$ file 1621726595.M343917P30832.smtp\,S\=2927\,W\=2988\:2\,S 1621726595.M343917P30832.smtp,S=2927,W=2988:2,S: MailCrypt -- Ben Burk BURK.TECH System Administrator

1 0

Dovecot Subfolder Always Uses dot separator with mdbox
by Steven Smith 22 May '21

22 May '21

I’m trying to configure dovecot to use mailbox subfolders that look like both foo/bar on the IMAP client and the filesystem. But the subfolders created all look like foo.bar, not foo/bar with a proper nesting structure. This also happens when the folder ‘bar’ is moved into the folder ‘foo’—A new folder `foo.bar’ is created. I believe that i’m following the configuration rules at https://wiki.dovecot.org/MailboxFormat/Maildir#Maildir_and_filesystems and https://wiki.dovecot.org/MailLocation/Maildir. This issue happens whether I set separator to its default, ‘.’, or ‘/’: > mail_location = mdbox:/private/var/mail/tld.hostname.mail/%Ln/mdbox:LAYOUT=fs > namespace inbox { > … > separator = / > … > } The IMAP client is macOS Mail.app version 14.0. I’d greatly appreciate any help or pointers to fix this configuration issue. dovecot -n > # 2.3.13 (): /opt/local/etc/dovecot/dovecot.conf > # Pigeonhole version 0.5.13 () > # OS: Darwin 20.4.0 x86_64 apfs > # Hostname: mail.hostname.tld > aps_topic = com.apple.mail.XServer.ffffffff-ffff-fff-ffffffffff > auth_cache_size = 10 M > auth_gssapi_hostname = $ALL > auth_krb5_keytab = /opt/local/etc/dovecot/imap.keytab > auth_mechanisms = plain gssapi > auth_realms =mail.hostname.tld > auth_socket_path = /opt/local/var/run/dovecot/auth-userdb > auth_username_format = %Ln > debug_log_path = /opt/local/var/log/mail/mail-debug.log > disable_plaintext_auth = no > first_valid_gid = 6 > first_valid_uid = 6 > imap_id_log = * > imap_id_send = "name" * "version" * > imap_idle_notify_interval = 29 mins > imap_urlauth_submit_user = submit > info_log_path = /opt/local/var/log/mail/mail-info.log > last_valid_gid = 100 > lda_mailbox_autocreate = yes > log_path = /opt/local/var/log/mail/mail-err.log > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c > mail_access_groups = mail > mail_attachment_dir = /private/var/mail/tld.hostname.mail/attachments > mail_attachment_fs = sis posix:mode=0666 > mail_gid = mail > mail_home = /private/var/mail/tld.hostname.mail > mail_location = mdbox:/private/var/mail/tld.hostname.mail/%Ln/mdbox:LAYOUT=fs > mail_log_prefix = "%s(pid %p user %u): " > mail_plugins = quota zlib acl fts fts_solr fts_lucene > mail_privileged_group = mail > mail_uid = _dovecot > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve > mdbox_rotate_size = 200 M > namespace inbox { > inbox = yes > location = > mailbox Archive { > auto = subscribe > special_use = \Archive > } > mailbox Drafts { > auto = subscribe > special_use = \Drafts > } > mailbox Junk { > auto = create > special_use = \Junk > } > mailbox Notspam_train { > auto = create > special_use = \Junk > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Spam_train { > auto = create > special_use = \Junk > } > mailbox Trash { > auto = create > special_use = \Trash > } > prefix = > separator = / > } > passdb { > driver = pam > name = pam > } > plugin { > fts = solr > fts_autoindex = yes > fts_autoindex_exclude = \Junk > fts_solr = url=http://127.0.0.1:8983/solr/dovecot/ > imapsieve_mailbox1_before = file:/opt/local/etc/dovecot/sieve/report-spam.sieve > imapsieve_mailbox1_causes = COPY APPEND > imapsieve_mailbox1_name = Spam_train > imapsieve_mailbox2_before = file:/opt/local/etc/dovecot/sieve/report-ham.sieve > imapsieve_mailbox2_causes = COPY APPEND > imapsieve_mailbox2_name = Notspam_train > mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename > mail_log_fields = uid box msgid from subject size flags > quota_grace = 10%% > quota_rule = *:storage=16G > quota_rule2 = Trash:storage=+256M > quota_warning = storage=100%% quota-exceeded %u > quota_warning2 = storage=80%% quota-warning %u > recipient_delimiter = + > sieve = /private/var/mail/tld.hostname.mail/rules/%Ln/dovecot.sieve > sieve_after = /opt/local/etc/dovecot/sieve-after.d > sieve_before = /opt/local/etc/dovecot/sieve-before.d > sieve_dir = /private/var/mail/tld.hostname.mail/rules/%Ln/%u > sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment > sieve_pipe_bin_dir = /opt/local/etc/dovecot/sieve > sieve_plugins = sieve_imapsieve sieve_extprograms > sieve_quota_max_storage = 50M > } > postmaster_address = postmaster(a)hostname.tld > protocols = imap lmtp sieve > quota_full_tempfail = yes > sendmail_path = /opt/local/sbin/sendmail > service auth-worker { > group = mail > user = root > } > service auth { > extra_groups = _keytabusers > idle_kill = 15 mins > unix_listener /opt/local/var/spool/postfix/private/auth { > group = mail > mode = 0660 > user = _postfix > } > } > service dns-client { > unix_listener dns-client { > mode = 0600 > } > } > service imap-login { > inet_listener imap { > address = 127.0.0.1, ::1 > port = 143 > } > inet_listener imaps { > port = 993 > ssl = yes > } > process_min_avail = 6 > service_count = 0 > vsz_limit = 2 G > } > service imap { > process_limit = 200 > process_min_avail = 6 > } > service lmtp { > unix_listener /opt/local/var/spool/postfix/private/dovecot-lmtp { > group = mail > mode = 0660 > user = _postfix > } > } > ssl = required > ssl_ca = </opt/local/etc/certificates/cert.chain.pem > ssl_cert = </opt/local/etc/certificates/cert.cert.pem > ssl_cipher_list = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256 > ssl_dh = # hidden, use -P to show it > ssl_key = # hidden, use -P to show it > ssl_min_protocol = TLSv1.2 > ssl_prefer_server_ciphers = yes > userdb { > driver = passwd > name = passwd > } > verbose_proctitle = yes > protocol lda { > mail_fsync = optimized > mail_plugins = quota zlib acl fts fts_solr fts_lucene sieve push_notify > } > protocol imap { > mail_max_userip_connections = 50 > mail_plugins = quota zlib acl fts fts_solr fts_lucene imap_acl imap_quota imap_zlib imap_sieve > } > protocol lmtp { > mail_fsync = optimized > mail_plugins = quota zlib acl fts fts_solr fts_lucene sieve > }

1 0

Dovecot v2.3.13 reporting (very) incorrect vsize for some maildir folders
by Eirik Rye 22 May '21

22 May '21

Hello! Dovecot v2.3.13. Full, but anonymized, `doveconf -n` attached. We are in the process of migrating maildir-backed users from filesystem quotes, to using dovecot's "count" quota backend. This is not reflected in the doveconf output because we're overriding `quota`, `quota_rule` and `quota_vsizes` in userdb: {"quota":"count:User quota","quota_rule":"*:storage=15G","quota_vsizes":"yes"} Afterwards, for a couple users, we received reports that mail delivery had failed because they were over quota. When looking into it for a specific user, we noticed that the vsize reported for a particular folder (with 47k messages) was reported as being nearly 50 times larger than it's on-disk size: root@mail02:~# doveadm mailbox status -u anonymized_user 'messages recent unseen vsize' 'anonymized/folder/name' [...] anonymized/folder/name messages=47338 recent=0 unseen=0 vsize=14335366070 However, the filesystem itself reports a much smaller size (but correct message count): root@mail02:~# du -hd1 /home/anonymized_user/Maildir/.anonymized.folder.name/ 313M /home/anonymized_user/Maildir/.anonymized.folder.name/cur 36K /home/anonymized_user/Maildir/.anonymized.folder.name/tmp 4.0K /home/anonymized_user/Maildir/.anonymized.folder.name/new 320M /home/anonymized_user/Maildir/.anonymized.folder.name/ root@mail02:~# ls /home/anonymized_user/Maildir/.anonymized.folder.name/cur | wc -l 47338 I have tried: - `doveadm force-resync -u anonymized_user` - deleting the index files in the specific folder, and running `doveadm index -u anonymized_user '*'` as well as `doveadm mailbox status -u anonymized_user vsize '*'` - deleting all `*index*` files in maildir, and running `doveadm index -u anonymized_user '*'` as well as `doveadm mailbox status -u anonymized_user vsize '*'` - comparing all maildir-sizes (S=) with their actual sizes to see if there are discrepancies. There are none. - regardless of the above, set `maildir_broken_filename_sizes = yes`, deleted indexes and reindexed. Even after deleting the list index (dovecot.list.index) as well as the mailbox indexes, recalculating the vsize seems very quick so I feel like the incorrect vsize is being fetched from a cache somewhere instead of being recalculated..? Any idea what is causing dovecot to report this wildly incorrect mailbox size? Are there any other files (not matching `*index*`) that are responsible? I'd rather not touch the user's control files, for obvious reasons. On a side note: We're also experiencing some issues with `mail_vsize_bg_after_count`. It seems to work correctly by returning a temporary error and deferring to a background job, but the indexer-worker job often times never appears to start (or do anything) at all: dovecot 29622 0.0 0.0 4120 1124 ? S 15:22 0:00 dovecot/indexer [0 clients, 0 requests] 266248 29623 0.0 0.0 6008 4604 ? S 15:22 0:00 dovecot/indexer-worker [idling] Waiting a while, and attempting to refetch the quota returns the same temperror with no indexer-worker being started. However, this is a separate issue from the above and can be ignored for now. Best regards, Eirik Rye

4 11

director & backend on same server with director_proxy_maybe
by lucas.raynaud＠ircf.fr 21 May '21

21 May '21

Hello, I've 2 mail servers, as test server, with dovecot as backend and setup pop3 and imap connections and NFS. For prevent issues with simultaneous connections I want to setup up Director on the same VM and on the same instance as the backend Dovecot. I found in the documentation that this configuration is broken, but can work with development, and after research on the internet and on the questions encountered previously as on : https://dovecot.org/pipermail/dovecot/2015-July/101483.html https://dovecot.org/pipermail/dovecot/2011-September/130939.html https://dovecot.org/pipermail/dovecot/2012-June/136535.html I find it quite unclear if this is really possible with director_proxy_maybe, which according to the changelog, allows this setup to work. Today I've reached this level with those errors: IPs : mail4 : XX.XX.111.8 mail3 : XX.XX.51.247 doveadm director ring status director ip port type last failed status // on mail3 XX.XX.51.247 9090 self never synced XX.XX.111.8 9090 l+r never synced doveadm director status lucas.raynaud(a)ircf.fr Current: 5.196.111.8 (expires 2021-05-21 10:48:51) Hashed: 5.196.111.8 Initial config: login with lucas.raynaud(a)ircf.fr on mail4 mail log on mail4 May 21 10:51:52 mail4 dovecot: auth-worker(12917): Debug: sql(lucas.raynaud(a)ircf.fr,XX.XX.111.8,<4h3SK9PCYsIFxG8I>): query: SELECT password, 'y' as proxy_maybe, 'y' as director_proxy_maybe FROM mail_user WHERE (login = 'lucas.raynaud(a)ircf.fr' OR email = 'lucas.raynaud(a)ircf.fr') AND disableimap = 'n' AND server_id = '8' May 21 10:51:52 mail4 dovecot: auth: Debug: client passdb out: OK#0111#011user=lucas.raynaud@ircf.fr#011director_proxy_maybe=y#011lip=XX.XX.111.8#011lport=143#011proxy#011pass=x9Im.bprP!CWzbgQ May 21 10:51:52 mail4 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: director_proxy_maybe May 21 10:51:52 mail4 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: lip May 21 10:51:52 mail4 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: lport May 21 10:51:52 mail4 dovecot: imap-login: Error: proxy: host not given: user=<lucas.raynaud(a)ircf.fr>, method=PLAIN, rip=XX.XX.111.8, lip=XX.XX.111.8, secured, session=<4h3SK9PCYsIFxG8I> May 21 10:51:52 mail4 dovecot: imap-login: Disconnected (internal failure, 1 successful auths): user=<lucas.raynaud(a)ircf.fr>, method=PLAIN, rip=XX.XX.111.8, lip=XX.XX.111.8, secured, session=<4h3SK9PCYsIFxG8I> no log on mail3 login with lucas.raynaud(a)ircf.fr on mail3 mail log on mail3 May 21 10:55:07 mail3 dovecot: auth-worker(19907): Debug: sql(lucas.raynaud(a)ircf.fr,XX.XX.51.247,<Hh5yN9PCRtAFxDP3>): query: SELECT password, 'y' as proxy_maybe, 'y' as director_proxy_maybe FROM mail_user WHERE (login = 'lucas.raynaud(a)ircf.fr' OR email = 'lucas.raynaud(a)ircf.fr') AND disableimap = 'n' AND server_id = '8' May 21 10:55:07 mail3 dovecot: auth: Debug: client passdb out: OK#0111#011user=lucas.raynaud@ircf.fr#011director_proxy_maybe=y#011lip=XX.XX.51.247#011lport=143#011proxy#011pass=x9Im.bprP!CWzbgQ May 21 10:55:07 mail3 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: director_proxy_maybe May 21 10:55:07 mail3 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: lip May 21 10:55:07 mail3 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: lport May 21 10:55:07 mail3 dovecot: imap-login: proxy(lucas.raynaud(a)ircf.fr): Login failed to XX.XX.111.8:143: [UNAVAILABLE] Account is temporarily unavailable.: user=<lucas.raynaud(a)ircf.fr>, method=PLAIN, rip=XX.XX.51.247, lip=XX.XX.51.247, secured, session=<Hh5yN9PCRtAFxDP3> May 21 10:55:07 mail3 dovecot: imap-login: Disconnected (proxy dest auth failed): user=<lucas.raynaud(a)ircf.fr>, method=PLAIN, rip=XX.XX.51.247, lip=XX.XX.51.247, secured, session=<Hh5yN9PCRtAFxDP3> mail log on mail4 May 21 10:55:07 mail4 dovecot: auth-worker(13096): Debug: sql(lucas.raynaud(a)ircf.fr,XX.XX.51.247,<rI10N9PCyKsFxDP3>): query: SELECT password, 'y' as proxy_maybe, 'y' as director_proxy_maybe FROM mail_user WHERE (login = 'lucas.raynaud(a)ircf.fr' OR email = 'lucas.raynaud(a)ircf.fr') AND disableimap = 'n' AND server_id = '8' May 21 10:55:07 mail4 dovecot: auth: Debug: client passdb out: OK#0111#011user=lucas.raynaud@ircf.fr#011director_proxy_maybe=y#011lip=XX.XX.111.8#011lport=143#011proxy#011pass=x9Im.bprP!CWzbgQ May 21 10:55:07 mail4 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: director_proxy_maybe May 21 10:55:07 mail4 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: lip May 21 10:55:07 mail4 dovecot: imap-login: Debug: Ignoring unknown passdb extra field: lport May 21 10:55:07 mail4 dovecot: imap-login: Error: proxy: host not given: user=<lucas.raynaud(a)ircf.fr>, method=PLAIN, rip=XX.XX.51.247, lip=XX.XX.111.8, session=<rI10N9PCyKsFxDP3> May 21 10:55:07 mail4 dovecot: imap-login: Disconnected (internal failure, 1 successful auths): user=<lucas.raynaud(a)ircf.fr>, method=PLAIN, rip=XX.XX.51.247, lip=XX.XX.111.8, session=<rI10N9PCyKsFxDP3> sql query : password_query = SELECT password, 'y' as proxy_maybe, 'y' as director_proxy_maybe FROM mail_user WHERE (login = '%u' OR email = '%u') AND disable%Ls = 'n' AND server_id = '8' It seems that Director is working but not adding the "host" extra field when it should, also it's strange that the director_proxy_maybe is unknown. Is it really possible to configure director on same instance as backend? Is director_proxy_maybe working? Also do you have answers about my problems? Thanks for advance. Lucas ############ mail3's and mail4's configs are the same. dovecot -n # 2.2.27 (): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 () # OS: Linux 4.9.0-14-amd64 x86_64 Debian 9.13 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_proxy_self = XX.XX.111.8 auth_verbose = yes dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } director_mail_servers = XX.XX.111.8 XX.XX.51.247 director_servers = XX.XX.111.8 XX.XX.51.247 director_user_expire = 5 mins disable_plaintext_auth = no doveadm_port = 24245 mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " quota" mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage quota_warning = storage=90%% quota-warning 90 %u sieve = /var/vmail/%d/%n/.sieve sieve_after = /etc/dovecot/sieve/after.d sieve_before = /var/vmail/%d/%n/sieve sieve_dir = /var/vmail/%d/%n/sieve/ sieve_global_dir = /etc/dovecot/sieve } pop3_reuse_xuidl = yes pop3_save_uidl = yes pop3_uidl_format = UID%u-%v protocols = " imap sieve pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service director { fifo_listener login/proxy-notify { mode = 0666 user = $default_login_user } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap-login { executable = imap-login director inet_listener imap { port = 143 } } service ipc { unix_listener ipc { user = dovecot } } service pop3-login { executable = pop3-login director inet_listener pop3 { port = 110 } } service quota-warning { executable = script /usr/local/bin/mail-scripts/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } user = vmail } ssl_cert = </opt/ssl/ircf.crt ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { auth_socket_path = director-userdb } protocol doveadm { auth_socket_path = director-userdb } protocol lda { mail_plugins = " quota sieve quota" } protocol imap { auth_socket_path = director-userdb mail_max_userip_connections = 16 mail_plugins = " quota quota imap_quota" } protocol pop3 { auth_socket_path = director-userdb mail_max_userip_connections = 16 mail_plugins = " quota quota" } local 10.10.10.0/24 { doveadm_password = # hidden, use -P to show it }

1 0

Re: WG: blocking - sql(acastus@example.ch,IP): unknown user
by mauric＠gmx.ch 20 May '21

20 May '21

>Hi! >You can try auth_policy_server such as weakforced. See <https://github.com/PowerDNS/weakforced> https://github.com/PowerDNS/weakforced and <https://doc.dovecot.org/configuration_manual/authentication/auth_policy/#au thentication-auth-policy> https://doc.dovecot.org/configuration_manual/authentication/auth_policy/#aut hentication-auth-policy >Aki And this Weakforced, i dont found for Debian Booster, so I have try to install but this need other meny requirements pakage, so please this are also possible as package for Debian? here any running with this this setup? I don't found any example on the net... or you using fail2ban? Mauri

1 0

Sendmail and Doecot
by Stephane Magnier 20 May '21

20 May '21

Hi, I am using Dovecot v 2.2.32 and Sendmail 8.15.2 My "folder" config on Dovecot is : mbox:/var/spool2/%u/mail:LAYOUT=fs:INBOX=/var/spool/mail/%u But in fact, any email arriving into the mailbox, stays in Sendmail into the file location (by default) : /var/spool/mail/user1 and any folder created, can be reached via Dovecot and is located to /var/spool2/user1 If I want to backup the user1's emails, I need to backup the "dovecot" location ( /var/spool2/user1) but also the Sendmail file location ( /var/spool/mail/user1) If I don't do that, I loose fresh emails which are still into the Sendmail directory, which are not copied into the Dovecot location.. Sounds weird I would expect to backup only my Dovecot directory and my email client should see only the Dovecot directory I would expect to have small sendmail email file user ( /var/spool/mail/user1 ) and i should't car about these files. Everything should be copied automatically into my Dovecot location what do you think ? Best regards, Stéve

3 2

blocking - sql(acastus@example.ch,IP): unknown user
by mauric＠gmx.ch 19 May '21

19 May '21

hello running with following Setup booster, postfix, Dovecot, mysql and meny other nice tools searching for solutions I see meny other running with same questions like this, so please what i need to read or setup that i don't receive so many failed logon try? "unknown user" i see, meny "fix" this fail2ban, but iam shure that Dovecot has also any solution for this? so friendly asking how to close this request from trying for failed logon. May 14 time Dovecot/auth-worker(8075): Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth May 14 time Dovecot/auth-worker(8075): Debug: Module loaded: /usr/lib/dovecot/modules/auth/lib20_auth_var_expand_crypt.so May 14 time Dovecot/auth-worker(8075): Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so May 14 time Dovecot/auth-worker(8043): Debug: sql(acastus(a)example.ch,IP): query: SELECT username AS user, password, '/folder/example.ch/acastus' AS userdb_home, '10000' AS userdb_uid, '10000' AS userdb_gid, CONCAT('maildir:storage=', (quota*1024)) as userdb_quota FROM mailbox WHERE username='acastus(a)example.ch' AND active = 1 May 14 time Dovecot/auth-worker(8026): Info: sql(acastus(a)example.ch,IP): unknown user thanks Mauri

3 2

Separating Dovecot and Postfix
by White, Daniel E. (GSFC-770.0)[NICS] 14 May '21

14 May '21

I am struggling to update a very old set of mail servers. Some are supposed to be relays (MTAs by my understanding) while others are where the mailboxes live (MDA) In rebuilding the MDA servers, is postfix required on the same server or can dovecot connect to postfix on a separate MTA server ? The old MDAs have both installed, but the config files are a mess.

3 4

lmtp(5672): Fatal: master: service(lmtp): child 5672 killed with signal 11 (core dumped)
by Alex 14 May '21

14 May '21

Hi Sometimes I see such lines in the logs May 10 23:15:31 mail dovecot[374]: lmtp(452): Fatal: master: service(lmtp): child 452 killed with signal 11 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set /proc/sys/fs/suid_dumpable to 2) May 12 18:50:19 mail dovecot[374]: lmtp(3533): Fatal: master: service(lmtp): child 3533 killed with signal 11 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set /proc/sys/fs/suid_dumpable to 2) May 13 17:21:34 mail dovecot[4910]: lmtp(5672): Fatal: master: service(lmtp): child 5672 killed with signal 11 (core dumped) and postfix logs: postfix/lmtp[28025]: 502DA17E1BC3: to=<email(a)domain.com>, relay=1.0.1.1[1.0.1.1]:11111, delay=0.78, delays=0.69/0/0.09/0, dsn=4.4.2, status=deferred (lost connection with 1.0.1.1[1.0.1.1] while receiving the initial server greeting) It was the spam message This error was in some previous versions too Information for debugging: dovecot is working in lxc container with Centos 8 in Proxmox 5.4.101-1-pve #1 SMP PVE 5.4.101-1 (Fri, 26 Feb 2021 13:13:09 +0100) x86_64 x86_64 x86_64 GNU/Linux # gdb /usr/libexec/dovecot/lmtp 'core-lmtp-5672-!usr!libexec!dovecot!lmtp' GNU gdb (GDB) Red Hat Enterprise Linux 8.2-12.el8 Copyright (C) 2018 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/libexec/dovecot/lmtp...Reading symbols from /usr/lib/debug/usr/libexec/dovecot/lmtp-2.3.14-1.0.rhel8.x86_64.debug...done. done. [New LWP 5672] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `dovecot/lmtp'. Program terminated with signal SIGSEGV, Segmentation fault. #0 __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:120 120 movdqu (%rax), %xmm4 (gdb) bt #0 __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:120 #1 0x00007fbc3f65725b in p_strdup (pool=0x5620ae47d3e0, str=0x8 <error: Cannot access memory at address 0x8>) at strfuncs.c:51 #2 0x00007fbc3f5b292a in setting_copy () at settings-parser.c:1538 #3 0x00007fbc3f5b2d8a in settings_dup_full () at settings-parser.c:1600 #4 0x00007fbc3f9547ba in mail_user_alloc (parent_event=0x0, username=0x7fbc3fa0b24a "raw mail user", set_info=0x5620ae46e458, set=0x5620ae430220) at mail-user.c:113 #5 0x00007fbc3f9a1dc5 in raw_storage_create_from_set (set_info=<optimized out>, set=<optimized out>) at raw-storage.c:25 #6 0x00005620ad66769a in client_raw_user_create (client=0x5620ae4aa8b8, client=0x5620ae4aa8b8) at lmtp-client.c:105 #7 client_create (fd_in=17, fd_out=17, conn=0x7fff7071af50) at lmtp-client.c:177 #8 0x00007fbc3f5acc99 in master_service_client_connection_callback (service=0x5620ae427130, conn=0x7fff7071af50) at master-service.c:980 #9 0x00007fbc3f5ad015 in master_service_listen (l=0x5620ae427610) at master-service.c:1212 #10 0x00007fbc3f63904d in io_loop_call_io (io=0x5620ae5032b0) at ioloop.c:714 #11 0x00007fbc3f63a679 in io_loop_handler_run_internal (ioloop=ioloop@entry=0x5620ae4272d0) at ioloop-epoll.c:222 #12 0x00007fbc3f6390f0 in io_loop_handler_run (ioloop=0x5620ae4272d0) at ioloop.c:766 #13 0x00007fbc3f639268 in io_loop_run (ioloop=0x5620ae4272d0) at ioloop.c:739 #14 0x00007fbc3f5ac557 in master_service_run (service=0x5620ae427130, callback=callback@entry=0x5620ad666fa0 <client_connected>) at master-service.c:853 #15 0x00005620ad666dcd in main (argc=<optimized out>, argv=<optimized out>) at main.c:168 (gdb) bt full #0 __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:120 No locals. #1 0x00007fbc3f65725b in p_strdup (pool=0x5620ae47d3e0, str=0x8 <error: Cannot access memory at address 0x8>) at strfuncs.c:51 mem = <optimized out> len = <optimized out> #2 0x00007fbc3f5b292a in setting_copy () at settings-parser.c:1538 strlist_info = {module_name = 0x0, defines = 0x0, defaults = 0x0, type_offset = 18446744073709551615, struct_size = 0, parent_offset = 18446744073709551615, parent = 0x0, check_func = 0x0, expand_check_func = 0x0, dependencies = 0x0, dynamic_parsers = 0x0} #3 0x00007fbc3f5b2d8a in settings_dup_full () at settings-parser.c:1600 strlist_info = {module_name = 0x0, defines = 0x0, defaults = 0x0, type_offset = 18446744073709551615, struct_size = 0, parent_offset = 18446744073709551615, parent = 0x0, check_func = 0x0, expand_check_func = 0x0, dependencies = 0x0, dynamic_parsers = 0x0} #4 0x00007fbc3f9547ba in mail_user_alloc (parent_event=0x0, username=0x7fbc3fa0b24a "raw mail user", set_info=0x5620ae46e458, set=0x5620ae430220) at mail-user.c:113 pool = 0x5620ae47d3e0 #5 0x00007fbc3f9a1dc5 in raw_storage_create_from_set (set_info=<optimized out>, set=<optimized out>) at raw-storage.c:25 user = <optimized out> ns = <optimized out> ns_set = <optimized out> mail_set = <optimized out> error = 0x7fff7071af50 "\021" #6 0x00005620ad66769a in client_raw_user_create (client=0x5620ae4aa8b8, client=0x5620ae4aa8b8) at lmtp-client.c:105 sets = <optimized out> sets = <optimized out> #7 client_create (fd_in=17, fd_out=17, conn=0x7fff7071af50) at lmtp-client.c:177 rcpt_param_extensions = {0x5620ad66be6b "XRCPTFORWARD", 0x0} cap_rcpt_forward = {name = 0x5620ad66be6b "XRCPTFORWARD", params = 0x0} workarounds = <optimized out> lmtp_set = {protocol = SMTP_PROTOCOL_SMTP, capabilities = SMTP_CAPABILITY_NONE, workarounds = (unknown: 1886498356), hostname = 0x5620ae426760 "", login_greeting = 0x1 <error: Cannot access memory at address 0x1>, rawlog_dir = 0x0, ssl = 0x0, max_client_idle_time_msecs = 1886498432, max_pipelined_commands = 32767, max_bad_commands = 1062726285, max_recipients = 32700, command_limits = {max_parameters_size = 94698362529624, max_auth_size = 140446494082554, max_data_size = 140735079886496}, max_message_size = 94698362822992, mail_param_extensions = 0x2ae426758, rcpt_param_extensions = 0x7fff7071ae34, xclient_extensions = 0x7071aec0, socket_send_buffer_size = 15936827190723348992, socket_recv_buffer_size = 11513522008, event_parent = 0x5620ae46e150, debug = false, auth_optional = false, tls_required = false, mail_path_allow_broken = false, rcpt_domain_optional = false} client = 0x5620ae4aa8b8 pool = <optimized out> #8 0x00007fbc3f5acc99 in master_service_client_connection_callback (service=0x5620ae427130, conn=0x7fff7071af50) at master-service.c:980 No locals. #9 0x00007fbc3f5ad015 in master_service_listen (l=0x5620ae427610) at master-service.c:1212 service = 0x5620ae427130 conn = {fd = 17, listen_fd = 10, name = 0x5620ae4276c0 "lmtp", remote_ip = {family = 2, u = {ip6 = {__in6_u = {__u6_addr8 = "\n\000\003e", '\000' <repeats 11 times>, __u6_addr16 = {10, 25859, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {1694695434, 0, 0, 0}}}, ip4 = {s_addr = 1694695434}}}, local_ip = {family = 2, u = {ip6 = {__in6_u = {__u6_addr8 = "\n\000\n\a", '\000' <repeats 11 times>, __u6_addr16 = {10, 1802, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {118095882, 0, 0, 0}}}, ip4 = {s_addr = 118095882}}}, remote_port = 50660, local_port = 24, real_remote_ip = {family = 2, u = { ip6 = {__in6_u = {__u6_addr8 = "\n\000\003e", '\000' <repeats 11 times>, __u6_addr16 = {10, 25859, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {1694695434, 0, 0, 0}}}, ip4 = {s_addr = 1694695434}}}, real_local_ip = {family = 2, u = {ip6 = {__in6_u = {__u6_addr8 = "\n\000\n\a", '\000' <repeats 11 times>, __u6_addr16 = {10, 1802, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {118095882, 0, 0, 0}}}, ip4 = { s_addr = 118095882}}}, real_remote_port = 50660, real_local_port = 24, proxy = {hostname = 0x0, cert_common_name = 0x0, alpn = 0x0, alpn_size = 0, ssl = false, ssl_client_cert = false}, proxied = false, fifo = false, ssl = false, accepted = true} #10 0x00007fbc3f63904d in io_loop_call_io (io=0x5620ae5032b0) at ioloop.c:714 ioloop = 0x5620ae4272d0 t_id = 2 __func__ = "io_loop_call_io" #11 0x00007fbc3f63a679 in io_loop_handler_run_internal (ioloop=ioloop@entry=0x5620ae4272d0) at ioloop-epoll.c:222 ctx = 0x5620ae42c5f0 events = <optimized out> event = 0x5620ae42c670 --Type <RET> for more, q to quit, c to continue without paging--c list = 0x5620ae43adc0 io = <optimized out> tv = {tv_sec = 2147483, tv_usec = 0} events_count = <optimized out> msecs = <optimized out> ret = <optimized out> i = 0 j = <optimized out> call = <optimized out> __func__ = "io_loop_handler_run_internal" #12 0x00007fbc3f6390f0 in io_loop_handler_run (ioloop=0x5620ae4272d0) at ioloop.c:766 __func__ = "io_loop_handler_run" #13 0x00007fbc3f639268 in io_loop_run (ioloop=0x5620ae4272d0) at ioloop.c:739 __func__ = "io_loop_run" #14 0x00007fbc3f5ac557 in master_service_run (service=0x5620ae427130, callback=callback@entry=0x5620ad666fa0 <client_connected>) at master-service.c:853 No locals. #15 0x00005620ad666dcd in main (argc=<optimized out>, argv=<optimized out>) at main.c:168 set_roots = {0x5620ad86e640 <smtp_submit_setting_parser_info>, 0x5620ad86e580 <lda_setting_parser_info>, 0x5620ad86e2a0 <lmtp_setting_parser_info>, 0x0} service_flags = <optimized out> storage_service_flags = (MAIL_STORAGE_SERVICE_FLAG_USERDB_LOOKUP | MAIL_STORAGE_SERVICE_FLAG_TEMP_PRIV_DROP | MAIL_STORAGE_SERVICE_FLAG_NO_LOG_INIT | MAIL_STORAGE_SERVICE_FLAG_NO_IDLE_TIMEOUT) tmp_base_dir = 0x5620ae41e758 "" c = <optimized out> error = 0x7fff707492d8 ""

2 2

Outlook creating IMAP folders even though forbidden by ACLs
by Matt Grant 13 May '21

13 May '21

Hi! Is it well known that Outlook cannot process RFC 2086 or RFC 4134 ACLs? We are using Outlook 2013, 2016 and 2019, and with 2013 can definitely create shared folders with content that are not saved to the server, even though Outlook 'pretends' quite erroneously that they are - black hole for critical email that we must keep according to engineering contracts for design documentation purposes. Thank you! Matt Grant

2 1

Problem with replication between two servers (ispconfig)
by Claudio Junior 12 May '21

12 May '21

Hi people, I am setting up replication between two servers. The installation of the servers was done using ispconfig and the operating system is debian 10. The problem is that after following the guide for this configuration, https://wiki.dovecot.org/Replication, when I start the server, replication does not start. I have also checked, via tcpdump, on the port associated with doveadm. If I force it with the command "doveadm -D sync -u user @ domain -d -N -l 30 -U" the replication happens, but it is not updating when the user receives emails on server01. At this time server02 will not receive emails (via postfix), and will also not have users accessing via pop3 or imap, but I have the idea of configuring so that both servers have these services working. The configuration of the servers is below: root@server01:~# doveconf -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-16-amd64 x86_64 Debian 10.9 # Hostname: server01.example.net auth_mechanisms = plain login debug_log_path = /var/log/dovecot.debug disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 2727 imap_capability = +SEPCIAL-USE XLIST listen = *,[::] lmtp_rcpt_check_quota = yes log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_max_userip_connections = 100 mail_plugins = quota notify replication mail_privileged_group = vmail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { auto = subscribe autoexpunge = 60 days special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe autoexpunge = 60 days special_use = \Trash } prefix = separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { mail_replica = tcp:server02.example.net:2727 quota = dict:user::file:/var/vmailquota/%d/%n/.quotausage quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO sieve = /var/vmail/%d/%n/.sieve sieve_after = /var/vmail/%d/%n/.ispconfig.sieve sieve_before = /var/vmail/%d/%n/.ispconfig-before.sieve sieve_max_actions = 100 sieve_max_redirects = 25 sieve_max_script_size = 2M } protocols = imap pop3 lmtp service aggregator { fifo_listener replication-notify-fifo { mode = 0660 user = vmail } unix_listener replication-notify { mode = 0660 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service doveadm { inet_listener { port = 2727 } unix_listener doveadm-server { user = vmail } } service imap-login { client_limit = 1000 process_limit = 512 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service quota-status { client_limit = 1 executable = quota-status -p postfix unix_listener /var/spool/postfix/private/quota-status { group = postfix mode = 0660 user = postfix } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0660 user = vmail } } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl_cert = </etc/postfix/smtpd.cert ssl_cipher_list = ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384 ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { auth_verbose = yes mail_plugins = quota imap_quota } protocol pop3 { auth_verbose = yes mail_plugins = quota pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = sieve quota postmaster_address = postmaster(a)server01.example.net } protocol lmtp { mail_plugins = quota sieve postmaster_address = postmaster(a)server01.example.net } local_name server01.example.net { ssl_cert = </etc/letsencrypt/live/server01.example.net/fullchain.pem ssl_key = # hidden, use -P to show it } local_name mail.example.net { ssl_cert = </etc/letsencrypt/live/mail.example.net/fullchain.pem ssl_key = # hidden, use -P to show it } local_name mail.otherdomain1.com.br { ssl_cert = </etc/letsencrypt/live/mail.otherdomain1.com.br/fullchain.pem ssl_key = # hidden, use -P to show it } root@server02:/var/vmail# doveconf -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-16-amd64 x86_64 Debian 10.9 # Hostname: server02.example.net auth_mechanisms = plain login debug_log_path = /var/log/dovecot.debug disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 2727 listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_max_userip_connections = 100 mail_plugins = " notify replication" mail_privileged_group = vmail namespace inbox { inbox = yes location = mailbox Drafts { auto = no special_use = \Drafts } mailbox Junk { auto = subscribe autoexpunge = 60 days special_use = \Junk } mailbox Sent { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Trash { auto = subscribe autoexpunge = 60 days special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { mail_replica = tcp:server01.example.net:2727 quota = dict:user::file:/var/vmailquota/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve sieve_max_redirects = 25 } protocols = imap pop3 service aggregator { fifo_listener replication-notify-fifo { group = root mode = 0660 user = vmail } unix_listener replication-notify { group = root mode = 0660 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service doveadm { inet_listener { port = 2727 } unix_listener doveadm-server { user = vmail } } service imap-login { client_limit = 1000 process_limit = 512 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service replicator { unix_listener replicator-doveadm { group = root mode = 0660 user = vmail } } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl_cert = </etc/postfix/smtpd.cert ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = sieve quota postmaster_address = postmaster(a)example.net } protocol lmtp { mail_plugins = quota sieve postmaster_address = postmaster(a)example.net } root@server02:/var/vmail# Att Claudio Jr. -- Claudio da Silva Junior csjunior(a)gmail.com

2 2

Variable expansion in password field (again?!)
by Mathias Rabe 11 May '21

11 May '21

Hi list, I’m experimenting with the mail-crypt-plugin in Dovecot v2.3.4.1 on Debian 10 Buster. I’m also running a OpenLDAP-Server for user lookup. My LDAP-settings looks like this: hosts = ### dn = cn=###,dc=###,dc=## dnpass = ### tls = yes auth_bind = no ldap_version = 3 base = ou=people,dc=###,dc=## scope = subtree user_attrs = \ =quota_rule=*:bytes=%{ldap:mailQuota}, \ =home=/var/vmail/mailboxes/%d/%{ldap:uid}, \ =mail=maildir:/var/vmail/mailboxes/%d/%{ldap:uid}/mail user_filter = (&(mail=%u)(objectClass=person)(mailEnabled=TRUE)) pass_attrs = mail=user,userPassword=password,=userdb_mail_crypt_private_password=%w,mailCrypt=userdb_mail_crypt_save_version pass_filter = (&(mail=%u)(objectClass=person)(mailEnabled=TRUE)) iterate_attrs = mail=user iterate_filter = (objectClass=person) default_pass_scheme = SSHA-512 The users can log-in and everything works fine including the mail-crypt-plugin until an user password contains a % I get the following error message (example): Error: Failed to expand plugin setting mail_crypt_private_password = '"1234%Passwort"': Unknown variable '%P‘ I’ve found some mails but I didn’t found a solution. It seems that Dovecot tries to expand some variable in the password: https://dovecot.org/pipermail/dovecot/2019-July/116338.html https://dovecot.org/pipermail/dovecot/2020-April/118634.html So this seems to be a bug or something. Is there any new information on this topic or even any solution or workaround? Thank you very much! Mathias

3 2

sudden performance drop - i/o related
by Marcin Gryszkalis 11 May '21

11 May '21

Hi I have exim/dovecot server that worked great for last few years and two weeks ago it got ill ;) First were users reporting errors on saving mails to Sent (timeouts). Now the logs are infested with warnings about long waits: May 10 10:18: Maildir /mail/xxx Synchronization took 193 seconds (1 new msgs, 0 flag change attempts, 0 expunge attempts) May 10 10:18: Maildir /mail/xxx Synchronization took 125 seconds (1 new msgs, 0 flag change attempts, 0 expunge attempts) May 10 10:18: Maildir /mail/xxx Synchronization took 211 seconds (1 new msgs, 0 flag change attempts, 0 expunge attempts) May 10 10:18: Maildir /mail/xxx Synchronization took 107 seconds (8 new msgs, 0 flag change attempts, 0 expunge attempts) May 10 10:18: Transaction log file /mail/xxx was locked for 36 seconds (Mailbox was synchronized) May 10 10:18: Transaction log file /mail/xxx was locked for 160 seconds (Mailbox was synchronized) May 10 10:18: Transaction log file /mail/xxx was locked for 72 seconds (Mailbox was synchronized) May 10 10:18: Locking transaction log file /mail/xxx took 60 seconds (syncing) May 10 10:18: Locking transaction log file /mail/xxx took 38 seconds (syncing) May 10 10:18: Locking transaction log file /mail/xxx took 35 seconds (syncing) It looks like i/o risen from 150writes/s to 500writes/s (in top hours) - but there's no real change in number of emails or the volume. Number of users is steady (~100 active users, ~250 imap sessions), number of emails (by count or by volume) is rising and falling within 15% margin. The box is FreeBSD 11.4, dovecot is 2.3.13. Filesystem is ZFS, disks are fine, free space is around 20% (~200GB) Layout is Maildir. CPU is not overloaded (2x6core), same with memory (48GB). I didn't change anything in configuration. Tonight I did some finetuning like maildir_copy_with_hardlinks=yes or mail_fsync=never/optimized (I'm not happy with that but I'm afraid it won't really help and I'll be able to revert that). I'm also thinking about switching from Maildir to sdbox (I know it won't hurt). I don't know where to look to find where the i/o goes. I don't have any metrics/stats enabled (I looked at the docs but it looks it's not really simple and needs some digging to get valuable config). Maybe somebody has suggestions what to look for? For detailed per-process stats I need to rebuild kernel with dtrace (other night I guess)... Simple top (in i/o mode - similar to linux's iotop) doesn't catch short living processes (like LDA deliveries). best regards -- Marcin Gryszkalis, PGP 0xA5DBEEC7 http://fork.pl/gpg.txt

2 3

mdbox / selecting folder accesses hundrets of others
by Florian Lohoff 10 May '21

10 May '21

Dovecot: 2.3.4 Hi, i am debugging a problem where after a force-resync removed all cache files (All files are 44 byte). Trying to open a folder takes literally minutes (Huge mailbox - 220K folder, 45GByte compressed storage) After i straced the imap process i can see the imap process accesses hundrets of other completely unrelated folders: Thats from the logfile - You can see mutt selecting the mailbox "zzzzzzt": May 07 18:05:19 imap(inbox@wasteland)<20381><hlkrmL/B5qwgAwDvnxMdAGcc/5MSpYxQ>: Debug: fts: No fts setting - plugin disabled May 07 18:05:19 imap(inbox@wasteland)<20381><hlkrmL/B5qwgAwDvnxMdAGcc/5MSpYxQ>: Debug: quota: quota_over_flag check: quota_over_script unset - skipping May 07 18:05:19 imap(inbox@wasteland)<20381><hlkrmL/B5qwgAwDvnxMdAGcc/5MSpYxQ>: Debug: Mailbox zzzzzzt: Mailbox opened because: SELECT From strace -p 20381 - This goes on for minutes opening thousands of unrelated folders. It is noteworthy that it mostly opens nested folders which are rare here: [...] open("/var/vmail/wasteland/inbox/mdbox/mailboxes/gerries/are/hawt", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 15 fstat(15, {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0 getdents(15, /* 3 entries */, 32768) = 72 stat("/var/vmail/wasteland/inbox/mdbox/mailboxes/gerries/are/hawt/yeah", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0 stat("/var/vmail/wasteland/inbox/mdbox/mailboxes/gerries/are/hawt/yeah/dbox-Mails", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0 getdents(15, /* 0 entries */, 32768) = 0 close(15) = 0 open("/var/vmail/wasteland/inbox/mdbox/mailboxes/karl588594", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 15 fstat(15, {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0 getdents(15, /* 3 entries */, 32768) = 80 stat("/var/vmail/wasteland/inbox/mdbox/mailboxes/karl588594/b3914", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0 stat("/var/vmail/wasteland/inbox/mdbox/mailboxes/karl588594/b3914/dbox-Mails", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0 getdents(15, /* 0 entries */, 32768) = 0 close(15) = 0 open("/var/vmail/wasteland/inbox/mdbox/mailboxes/ben", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 15 fstat(15, {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0 getdents(15, /* 4 entries */, 32768) = 104 stat("/var/vmail/wasteland/inbox/mdbox/mailboxes/ben/gang", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0 stat("/var/vmail/wasteland/inbox/mdbox/mailboxes/ben/gang/dbox-Mails", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0 Reopening the same folder again is quick and seamless. So it seems this is some cache recreation code looking into other folders. I am bit puzzled on why it does this. mail_debug=yes did now show anything. Any more debug options for mdbox i might set to find out about its decisions? Flo -- Florian Lohoff f(a)zz.de Any sufficiently advanced technology is indistinguishable from magic.

3 5

Writing an custom imap command
by Ryan Beethe 10 May '21

10 May '21

I'm interested in writing a custom imap command that behaves a bit like IDLE but synchronizes some state that is specific to my mail client / mail server. I found that stateless commands were trivial to understand, and I really like the plugin pattern for registering custom commands. But I have a few questions on how to write a long-running command that I was not able to answer by reading through the code. This mailing list seems like the best place to ask them. For reference, the source code for my custom command is here: https://github.com/Splintermail/splintermail-client/blob/dev/server/xkeysyn… Thanks, Ryan ------ 1. Why does cmd-idle.c sometimes call client_command_free()? But sometimes it doesn't? For example, cmd_idle_continue() frees it in some branches but not others. That makes no sense to me; it seems like it should be based on your entrypoint (mailbox notify callback vs input callback vs timeout callback), not based on which branch of logic within that entrypoint. 2. Why does cmd-idle.c ever call client_destroy()? That seems like something that should be invoked only by the imap process, not by any command. It calls it in cmd-idle.c:idle_callback (which is a mailbox notify callback). It invokes it after idle_sync_now() when it detects that client->disconnected is set. Maybe that happens in imap_sync_init() or something? 3. Why does cmd-idle.c ever call client_disconnect()? That also seems like the responsibility of the imap process, and not any command. idle_client_input_more() detects when i_stream_read returns -1, meaning that the client has *already disconnected*. Then it calls client_disconnect(). I think this is the crazy part... the istream is effectively unique to the imap process, so it seems unreasonable that any command is responsible for cleaning it up; it should just always happen at the imap process level before exiting, right? 4. What does client_continue_pending_input() actually do, and under what conditions does it need to be called? There is one place that you *can't* call it; there is a section in imap-client.c:client_handle_input() that calls imap-client.c:client_handle_next_command(), which calls the cmd->func(). That makes sense; that's the input trigger for the command plugin, so maybe you only have to trigger it when you are receiving input that doesn't fit into the normal command args behavior. It has a comment that says "this function is called at the end of I/O callbacks (and only there)". It _is_ called by client_input() and by client_output(), but also by: - cmd-idle.c:idle_client_input (io_add_istream callback) - cmd-append.c:client_input_append (io_add_istream callback) - imap-search.c:cmd_search_more_callback (timeout_add callback) The first two cases seem to be the only io_add_istream() commands that even exist, so that explains them. I can't explain the imap-search.c case at all. Reading through it I have really no idea what client_continue_pending_input is really doing. My command has a DONE mechanic just like IDLE so I'm pretty sure I need to invoke this function, I'm just concerned I'm going to do it wrong if I don't understand the mechanics of it.

3 6

huge mdbox broken / Log synchronization error
by Florian Lohoff 10 May '21

10 May '21

Hi, its the second time in 6 Month i have issues with a huge mdbox. It broken in November and i restored from backup because force-resync twice didnt fix it. (And restore from borg-backup was MUCH faster than force-resync) This time i have heavy delivery into that mailbox (postfix concurrency 1) and i tried merging folders with doveadm move. The move froze and now i seem to have a broken mdbox: Errors are: May 05 10:21:09 lda(inbox(a)wasteland.rfc822.org): Error: Log synchronization error at seq=2736,offset=135540 for /var/vmail/wasteland.rfc822.org/inbox/mdbox/storage/dovecot.map.index: Extension record inc drops number below zero (uid=7757856, diff=-1, orig=0) trying root@pax:~# doveadm -D -v force-resync -u inbox(a)wasteland.rfc822.org \* now. Although this will mostly likely take 8-10 hours. Mailbox is 44GByte (Compressed mdbox) and 220K folders. This is dovecot on Debian/Stretch dovecot-core 1:2.2.27-3+deb9u7 dovecot-imapd 1:2.2.27-3+deb9u7 dovecot-ldap 1:2.2.27-3+deb9u7 dovecot-managesieved 1:2.2.27-3+deb9u7 dovecot-pop3d 1:2.2.27-3+deb9u7 dovecot-sieve 1:2.2.27-3+deb9u7 dovecot-solr 1:2.2.27-3+deb9u7 Flo -- Florian Lohoff f(a)zz.de Any sufficiently advanced technology is indistinguishable from magic.

3 10

How to set lmtp delivery file/directory permissions?
by Kurt Fitzner 08 May '21

08 May '21

I have my postfix/dovecot virtual mail system essentially (otherwise) configured where Postfix is calling Dovecot for its virtual transport: virtual_transport = lmtp:unix:private/dovecot-lmtp But I can't for the life of me see how to change the file permissions dovecot uses to create maildir folders and files. I looked at the maildir [1], lmtp [2], and other documents and they are either silent on permissions or seem to reference a shared mail permissions document [3] that claims file and folder permissions are inherited. I want all files dovecot creates to be made accessible to the mail group. Please tell me this wasn't hard-coded a la postfix. Thanks, Kurt Fitzner Links: ------ [1] https://wiki.dovecot.org/MailboxFormat/Maildir#Maildir_and_filesystems [2] https://doc.dovecot.org/configuration_manual/protocols/lmtp_server/ [3] https://doc.dovecot.org/admin_manual/filesystem_permission/

1 0

Issue with dovecot SASL
by Piyush Joshi 08 May '21

08 May '21

Hi, I am getting the following error while authenticating any user, using SASL with postfix, please help me to fix this issue. May 08 17:34:50 master: Info: Dovecot v2.2.36.4 (baf9232c1) starting up for imap, pop3, lmtp, sieve (core dumps disabled) May 08 17:35:10 master: Error: service(auth): command startup failed, throttling for 2 secs May 08 17:35:10 auth: Fatal: master: service(auth): child 6969 killed with signal 11 (core dumps disabled) *Dovecot Configuration Output* root@mail:/usr/local/dovecot/etc/dovecot/conf.d# /usr/local/dovecot/sbin/dovecot -n # 2.2.36.4 (baf9232c1): /usr/local/dovecot/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24.2 (aaba65b7) # OS: Linux 4.19.0-16-amd64 x86_64 Debian 10.9 # Hostname: mail.example.com auth_default_realm = example.com auth_mechanisms = plain login base_dir = /usr/local/dovecot/var/run/dovecot/ default_client_limit = 5000 default_process_limit = 500 default_vsz_limit = 512 M first_valid_uid = 1001 last_valid_uid = 1001 listen = 127.0.0.1 lmtp_save_to_detail_mailbox = yes log_path = /var/log/imap login_greeting = Messaging Server mail_gid = 1001 mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ mail_plugins = quota acl fts fts_lucene mail_uid = 1001 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds imapflags notify namespace inbox { inbox = yes location = mailbox Archive { auto = subscribe special_use = \Archive } mailbox "Deleted Items" { special_use = \Trash } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox "Junk E-mail" { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Items" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = . type = private } passdb { args = /usr/local/dovecot/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/usr/local/dovecot/dovecot-acl/acl acl_shared_dict = file:/maildir/domains/dovecot/shared-mailboxes/shared-mailboxes.db fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. fts_tika = http://localhost:9997/tika/ quota = maildir:User quota quota_rule = *:storage=1G quota_warning = storage=85%% quota-warning 85 %u quota_warning2 = storage=90%% quota-warning 90 %u quota_warning3 = storage=95%% quota-warning 95 %u sieve = ~/.dovecot.sieve sieve_after = /usr/local/dovecot/sieve/default.sieve sieve_before = ~/sieve/01_vacation.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags +vacation +vacation-seconds sieve_global_dir = /usr/local/dovecot/sieve/ sieve_quota_max_scripts = 5 sieve_vacation_send_from_recipient = yes sieve_vacation_use_original_recipient = yes } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { address = * port = 993 ssl = yes } } service imap-postlogin { executable = script-login /usr/local/dovecot/bin/postlogin.sh user = root } service imap { executable = /usr/local/dovecot/libexec/dovecot/imap imap-postlogin } service lmtp { executable = lmtp -L inet_listener lmtp { address = 127.0.0.1 ::1 port = 7025 } process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = vmail } service managesieve-login { inet_listener sieve-obsolete { port = 2000 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { address = * port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/dovecot/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl = required ssl_ca = </etc/ssl/localcerts/intermediate.crt ssl_cert = </etc/ssl/localcerts/mail.crt ssl_key = # hidden, use -P to show it submission_host = 127.0.0.1 userdb { args = /usr/local/dovecot/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = quota acl fts fts_lucene sieve } protocol sieve { managesieve_logout_format = bytes ( in=%i : out=%o ) } protocol lda { info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log mail_plugins = quota acl fts fts_lucene sieve virtual } protocol imap { mail_max_userip_connections = 50 mail_plugins = quota acl fts fts_lucene imap_quota imap_acl } Thanks, Regards

1 0

Doveadm sync verbose not working?
by Plutocrat 07 May '21

07 May '21

Hi Was trying a sync with eg this. old is a host set up in my ssh config file. the user exists on both sides. The sync completes correctly copying mails back and forth, but offers no feedback. doveadm sync -u user(a)domain.com -f -R remote:old If I add -v I don't get any more information. Am I missing something? The manual says its meant to give more info and progress. doveadm -v sync -u user(a)domain.com -f -R remote:old Putting -v elsewhere on the commandline results in an error, so I think its in the right place. However -D works, but its a bit too verbose ... doveadm -D sync -u user(a)domain.com -f -R remote:old dovecot --version 2.3.7.2 (3c910f64b) P.

1 0

Adding virtual folders to an existing dovecot installation
by Christian Wolf 06 May '21

06 May '21

Hello dear dovecot mailinglist, I am having a dovecot installation that is working so far (together with sieve and managesieve). Now, I found the possibility to add a virtual folder to the server promising. I sort various mails using a per-user sieve script into subfolders. This works well on the latop but the mobile device does not like too many folders to check. So, I wanted to create a virtual inbox that holds all unread messages. I found this documentation: https://doc.dovecot.org/configuration_manual/virtual_plugin/ Now, I have first one question: When I use the virtual folder and read a message/mark it as read, will this be reflected on the underlaying folder or will it cause trouble on dovecot? As far as I understand, usage of sieve filtering and the virtual plugin is considered critical as mentioned in the documentation. This is due to the fact, that the virtual mailboxes are read-only by default (unless a folder is prefixed with a !-symbol). Sieve will try to filter the virtual folder as well be default and moving a message to another folder will fail the read-only assumption. Honestly, I do not get the content of https://doc.dovecot.org/ configuration_manual/virtual_plugin/#sieve-filters-with-virtual-mailboxes by 100%. The first paragraph indicates that sieve plus virtual mailboxes are a complete no-go, while the second indicates that a safe configuration is needed. Could someone please elaborate this a bit more? I have not yet changed anything as I did not want blindly break my production mail system. The current (unaltered) configuration I will attach below. Bonus question: Is it possible to restrict the effect of the virtual plugin to certain (virtual) user accounts? Thank you very much Christian # dovecot --version 2.3.13 (89f716dc2) #dovecot -n # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) # OS: Linux 5.11.2-arch1-1 x86_64 # Hostname: server-hh.hh.lupus auth_mechanisms = plain login mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Templates { auto = subscribe } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = scheme=PLAIN username_format=%n /home/.vmail/%d/etc/shadow driver = passwd-file } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve stats_refresh = 30 secs stats_track_cmds = yes } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl = required ssl_cert = </etc/letsencrypt/live/dovecot/fullchain.pem ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM- SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA- AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128- SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128- SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256- SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS- AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256- SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128- SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:! MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { args = uid=vmail gid=vmail home=/home/.vmail/%d/mails/%n driver = static } protocol lmtp { mail_plugins = " sieve" postmaster_address = postmaster(a)example.com } protocol lda { mail_plugins = " sieve" } protocol imap { mail_plugins = " old_stats" }

2 1

disable pop3 ports?
by Dan Egli 05 May '21

05 May '21

I admit I don't quite understand dovecot's config yet, but this is driving me batty. I was looking at my server and noticed that dovecot was listening on the pop3 ports (110/TCP). Since I do not use pop3 at all, nor does anyone who has ever or ever will connect to the server, that seems like a needless waste. So I went through the config files and commented out every reference to pop3 in them. But when I restart dovecot, it STILL opens a listener on 110. How do I fix this? The ONLY external ports I want dovecot listening to are imap4 and imap4s. Thanks! -- Dan Egli From my Test Server

10 20

POP3 mailbox errors and changing permissions on folders
by John Richards 04 May '21

04 May '21

1 0

How to Host Multiple Mail Domains (Email Hosting) in iRedMail Full Featured Linux Mail Server
by Turritopsis Dohrnii Teo En Ming 04 May '21

04 May '21

Subject: How to Host Multiple Mail Domains (Email Hosting) in iRedMail Full Featured Linux Mail Server Author: Mr. Turritopsis Dohrnii Teo En Ming (TARGETED INDIVIDUAL) Country: Singapore Date: 3rd May 2021 Monday Type of Publication: PDF Manual Document Version: 20210503.01 ***IMPORTANT NOTICE*** Please note that Turritopsis Dohrnii Teo En Ming’s guide is based on Xiao Guoan’s guide at linuxbabe.com. Reference Guide Used by Teo En Ming: How to Host Multiple Mail Domains in iRedMail with Nginx Link: https://www.linuxbabe.com/mail-server/set-up-iredmail-multiple-domains-nginx Original Author: Xiao Guoan The following is a list of open-source software that will be automatically installed and configured by iRedMail. • Postfix SMTP server • Dovecot IMAP server • Nginx web server to serve the admin panel and webmail • OpenLDAP, MySQL/MariaDB, or PostgreSQL for storing user information • Amavised-new for DKIM signing and verification • SpamAssassin for anti-spam • ClamAV for anti-virus • Roundcube webmail • SOGo groupware, providing webmail, calendar (CalDAV), contacts (CardDAV), tasks and ActiveSync services. • Fail2ban for protecting SSH • mlmmj mailing list manager • Netdata server monitoring • iRedAPD Postfix policy server for greylisting In addition, you need to add MX, A and TXT records to your ISC BIND DNS domain name server. Redundant download links for Turritopsis Dohrnii Teo En Ming's PDF manual: [1] https://www.mediafire.com/file/q6txmx0rc1cwfzw/Additional+Mail+Domains+1st+… [2] https://www.docdroid.net/Q9TRL5D/additional-mail-domains-1st-release-pdf [3] https://www.scribd.com/document/506193434/Additional-Mail-Domains-1st-Relea… [4] https://drive.google.com/file/d/10AYGt4-omBeC3vXJfzswQq0M7E09gXpG/view?usp=… [5] https://drive.google.com/file/d/113dy0AKmBYGugBueK1vS_dJwI35J6XmJ/view?usp=… [6] https://drive.google.com/file/d/1dhpYIZp31ug5hoiVNbjh6uh99s_PEx8q/view?usp=… Mr. Turritopsis Dohrnii Teo En Ming, 43 years old as of 4th May 2021, is a TARGETED INDIVIDUAL living in Singapore. He is an IT Consultant with a System Integrator (SI)/computer firm in Singapore. He is an IT enthusiast. -----BEGIN EMAIL SIGNATURE----- The Gospel for all Targeted Individuals (TIs): [The New York Times] Microwave Weapons Are Prime Suspect in Ills of U.S. Embassy Workers Link: https://www.nytimes.com/2018/09/01/science/sonic-attack-cuba-microwave.html ******************************************************************************************** Singaporean Targeted Individual Mr. Turritopsis Dohrnii Teo En Ming's Academic Qualifications as at 14 Feb 2019 and refugee seeking attempts at the United Nations Refugee Agency Bangkok (21 Mar 2017), in Taiwan (5 Aug 2019) and Australia (25 Dec 2019 to 9 Jan 2020): [1] https://tdtemcerts.wordpress.com/ [2] https://tdtemcerts.blogspot.sg/ [3] https://www.scribd.com/user/270125049/Teo-En-Ming -----END EMAIL SIGNATURE-----

1 0

mail_crypt module and error with tmp directory
by Fiorenza Meini 04 May '21

04 May '21

Hi, I successfully enable mail_crypt module but I'm experiencing a strange behaviour with tmp directory while accessing with POP3 protocol: I see in log file: Error: istream-seekable: safe_mkstemp(/tmp/dovecot.pop3.) failed: Permission denied I changed tmp directory configuration (mail_temp_dir variable) and setting it with 777 permission, but the error is the same. On client side it's working everything, but I'd like to understand the error and if I have to be worried about it. Thank you and regards Fiorenza -- Fiorenza Meini/Spazio Web Via Dante Alighieri, 10 - 13900 - BIELLA tel. +39 015 2431982-1 - fax 015 2522600 https://esseweb.eu -- Questo messaggio e i suoi allegati ai sensi del Regolamento (UE) 2016/679 sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete pregati di darne immediata comunicazione al mittente e provvedere alla sua distruzione.

2 8

[Dovecot] Doveadm sometimes failed "to iterate through some users" after upgrade to 2.2
by Alessio Cecchi 03 May '21

03 May '21

Hi, I'm running every week in cron "doveadm expunge -A mailbox Spam before 30d" and monthly "doveadm quota recalc -A". After the upgrade to dovecot 2.2.12 (from 2.1.17), "doveadm -A" sometimes exit with this error: doveadm(user(a)domain.com): Error: User listing returned failure doveadm: Error: Failed to iterate through some users # echo $? 75 If I re-run the same command the user where doveadm stops changeing (always in alphabetical order): info@ mario@ pippo@ zazza@ and finally, after 3 or 4 run, ends with success. I have enable debug (via doveadm -D and auth/mail debub in 10-logging.con) but there are no errors or difference in the log when doveadm exit with the error (error code is 75). If I run # doveadm expunge -A mailbox Spam savedbefore 30d doveadm works fine Can somebody help me? Here is my configuration: # dovecot -n # 2.2.12: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.9 auth_debug = yes auth_master_user_separator = * auth_verbose = yes deliver_log_format = msgid=%m, from=%f, subject="%s": %$ dict { expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no dotlock_use_excl = no first_valid_gid = 89 first_valid_uid = 89 imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags imap_idle_notify_interval = 29 mins imap_logout_format = in=%i out=%o session=<%{session}> last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * mail_debug = yes mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota expire maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave vnd.dovecot.duplicate mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = username_format=%Ld /etc/dovecot/extra/alias-domains-denylogin.txt deny = yes driver = passwd-file } passdb { args = /etc/dovecot/extra/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash expire2 = Spam expire_dict = proxy::expire quota = maildir:UserQuota quota2 = dict:Quota Usage::noenforcing:proxy::sqlquota quota_grace = 10M quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /etc/dovecot/sieve/before.sieve sieve_dir = ~/sieve sieve_extensions = +vnd.dovecot.duplicate -vacation } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s, bytes=%i/%o, session=<%{session}> protocols = imap pop3 sieve service auth-worker { process_limit = 1000 } service auth { client_limit = 5500 unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { process_limit = 300 unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { process_min_avail = 4 service_count = 0 } service imap { process_limit = 4000 service_count = 100 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 0 } service pop3 { process_limit = 1023 service_count = 100 } service quota-warning { executable = script /etc/dovecot/scripts/quota-warning.sh unix_listener quota-warning { user = vpopmail } user = vpopmail } ssl_cert = </etc/dovecot/ssl/wildcard-myemail.net.pem ssl_key = </etc/dovecot/ssl/wildcard-myemail.net.key submission_host = localhost userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_location = maildir:~/Maildir:INDEX=MEMORY mail_plugins = quota expire sieve } protocol imap { mail_max_userip_connections = 15 mail_plugins = quota expire imap_quota } protocol sieve { mail_max_userip_connections = 2 } protocol pop3 { mail_max_userip_connections = 5 } -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz Cloud Email Hosting -> http://www.qboxmail.com @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it

3 3

Latest git FATAL error
by Joan Moreau 03 May '21

03 May '21

Hello On latest git of dovecot, I get Apr 24 04:07:36 gjserver dovecot[857958]: imap-login: Panic: file client-common.c: line 293 (client_disconnect): assertion failed: (client->prev == NULL && client->next == NULL) and login process crash On 2.3.14, there is no problems Hope it helps JM

3 17