June 2022 - dovecot - dovecot.org

Warning sqlpool(mysql): Query failed
by Alessio Cecchi 27 Jun '23

27 Jun '23

Hi, with the last dovecot version (2.3.13) I see many of this warnings via "doveadm log errors": Feb 17 03:26:28 Warning: dict(31513): conn unix:dict (pid=31330,uid=95): dict(mysql)<email1(a)company1.com>: sqlpool(mysql): Query failed, retrying: MySQL server has gone away (idled for 679 secs) Feb 17 03:48:53 Warning: dict(31013): conn unix:dict (pid=31805,uid=95): dict(mysql)<email2(a)company2.com>: sqlpool(mysql): Query failed, retrying: MySQL server has gone away (idled for 685 secs) Feb 17 03:54:12 Warning: dict(32529): conn unix:dict (pid=3147,uid=95): dict(mysql)<email3(a)company3.com>: sqlpool(mysql): Query failed, retrying: MySQL server has gone away (idled for 670 secs) What do they mean? I thinks are related to inactivity timeout on mysql server (where I have a wait_timeout=600), but these warnings could be problems or can be fixed with some setting in dovecot? My dovecot configuration about dict: dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext sieve = mysql:/etc/dovecot/dovecot-dict-sieve-sql.conf.ext sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } plugin { [...] acl_shared_dict = proxy::acl expire_dict = proxy::expire quota2 = dict:Quota Usage::noenforcing:proxy::sqlquota sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = dict:proxy::sieve;name=activesql [...] } service dict { process_limit = 500 unix_listener dict { group = vmail mode = 0660 user = vmail } } Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 1

dovecot-fts-solr Solr9 support
by PGNet Dev 24 Apr '23

24 Apr '23

I run dovecot-2.3.18 , & am deploying Solr search with 'dovecot-fts-solr' I run Solr standalone. When creating a dedicated 'dovecot' core, cp'ing from Dovecot's included solr schemas ls -al /usr/share/doc/dovecot/solr-* -rw-r--r-- 1 root root 12K Dec 3 06:48 /usr/share/doc/dovecot/solr-config-7.7.0.xml -rw-r--r-- 1 root root 2.7K Dec 3 06:48 /usr/share/doc/dovecot/solr-schema-7.7.0.xml -rw-r--r-- 1 root root 2.9K Dec 3 06:48 /usr/share/doc/dovecot/solr-schema.xml is tested only for Solr 7.7.0 https://doc.dovecot.org/configuration_manual/fts/solr/ "The steps described in this page are tested for Solr 7.7.0. For other versions, this these steps may need to be adjusted." which, per solr docs https://solr.apache.org/downloads.html About versions and support Version Description 9.x Current major version for feature releases (STABLE) 8.11.x Previous major version may sometimes receive critical bugfix releases 10 Next major version, yet to be released (UNSTABLE) <8.11 All older versions are End Of Life (EOL) is EOL'd and unsupported. In the dovecot 7.7.0 schema, FastLRUCache/LRUCache are used as the caching class(es), grep class=.*LRU /usr/share/doc/dovecot/solr-config-7.7.0.xml <filterCache class="solr.FastLRUCache" <queryResultCache class="solr.LRUCache" <documentCache class="solr.LRUCache" class="solr.search.LRUCache" For Solr 9, cat CHANGES.txt Upgrade Notes --------------------- ... * SOLR-13817: Deprecate legacy SolrCache implementations. Users are encouraged to transition their configurations to use org.apache.solr.search.CaffeineCache instead. (ab) running solr9 + dovecot-fts-solr search, based on the 7.7.0 schemas, results in errors, failed scan & search. Modifying the config to use CaffeineCache cp solr-config-7.7.0.xml solr-config-9.0.0.xml perl -pi -e '\ s|solr.FastLRUCache|solr.CaffeineCache|g; \ s|solr.LRUCache|solr.CaffeineCache|g; \ s|solr.search.LRUCache|solr.search.CaffeineCache|g;' \ solr-config-9.0.0.xml appears to do the trick. Atm, indexing & search -- both manually triggered, and on delivery receipt to Dovecot store -- is working with no errors (yet). I've not tested exhaustively yet, & don't know if this Cache class change in the schema is both necessary & sufficient for dovecot-fts-solr search use with solr 9.0.0 Given that for Solr "<8.11 All older versions are End Of Life (EOL)" it'd be useful for Dovecot to 'retest' for current, not-EOL'd Solr, and publish/package compatible schemas accordingly.

5 8

decrypt.rb
by Dave 11 Apr '23

11 Apr '23

Per the Dovecot site here: https://wiki.dovecot.org/Plugins/MailCrypt ... the "decrypt.rb" ruby script can be used to decrypt a Dovecot-encrypted message file from the command line. The script sort of runs successfully for me, in the sense that it doesn't error out, but it doesn't show the decrypted message. I've called it like so: decrypt.rb -k <PATH_TO_DOVECOT_KEYFILE> -f ./encrypted_message But when I do it just displays the info below and exits (I've subbed <DATA> tags for the actual data): ------------------[OUTPUT]----------------------- Key(s) (total: 1) - Key type : EC - Key digest: <DATA1> - Peer key : <DATA2> - Encrypted : <DATA3> - Kd hash : <DATA4> ------------------------------------------------------- I've tried specifying an output file as well, per the script's command line options, but the output file is 0 bytes. Does anyone have any suggestions? I *think* I'm using it the way it's intended to be used, but maybe I'm not?! -Dave

5 14

2.3.14: stale imap-login processes due to bogus user setup tying up process limit
by Volker Stolz 15 Dec '22

15 Dec '22

I just dealt with an interesting Dovecot issue on our small server, and thought I might share it. I also feel that although in the end it was a configuration error, Dovecot should somehow avoid leaving this processes around and hence filling up the process limit quickly. - set up Dovecot IMAP on Debian buster (either with the older 2.3.x Debian package, or 2.3.14 from the dovecot-repo) - config: ... namespace inbox { inbox = yes location = ... default settings ... ... passdb { args = scheme=HMAC-MD5 username_format=%u /etc/dovecot/users driver = passwd-file } protocols = " imap" ... userdb { args = blocking=no driver = passwd } ... - secret sauce: have a user in the passdb that doesn't have a physical account on the system yet (we were migrating to a new machine). The log-file shows successful authentication for that user (after we flipped DNS to point to this new machine), but over time (2-3 times/day) we were left with an increasing number of `imap-login` processes in state `pre-login`, eventually requiring a dovecot restart. Dialling up process_limits didn't seem to help, we'd just get more of these stale processes. After creating the user on the system, the behaviour reverted to normal, no more stale processes. I do not know what error message that user received on their end, but since we announced we were migrating, they probably just ignored it and thought it as part of the transition. Best, Volker (a happy & long Dovecot user)

3 4

Catch all for Metadata storage in SQL database
by Steffen Kaiser 27 Oct '22

27 Oct '22

Hi, the https://doc.dovecot.org/configuration_manual/imap_metadata/ sample uses mail_attribute_dict = file:%h/Maildir/dovecot-attributes which stores all keys=value pairs in the file. http://dovecot.2317879.n4.nabble.com/Dovecot-v2-3-9-3-HTTP-API-Endpoint-for… uses a SQL dict, however very specific ones. How can I use a SQL dict to store _all_ keys, as with a file based storage? I cannot find a documentation for "pattern" specification, that works as "catch all", in order to store anything not catched by patterns into the database. https://wiki.dovecot.org/Dictionary does not give any hint (in my eyes). -- Steffen Kaiser

3 4

dovecot mailing list (this mailing list), DKIM, SPF and DMARC
by Lev Serebryakov 21 Oct '22

21 Oct '22

My domain (serebrtyajov.spb.ru) has all these "new" e-mail technologies configured. It works fine till I write to this mailing list. After that I've got several DMARC reports about "spam" from my domain. All these reports are about my mailing list post. I don't have such problems with other mailing lists (FreeBSD ones, OpenJDK ones, and others). Looks like mailing list software for this mailing list is misconfigured. I'm sure, I'll get new after this message. -- // Black Lion AKA Lev Serebryakov

15 38

Multidomain ssl config ?
by Maurizio Caloro 15 Oct '22

15 Oct '22

on postfix now this seems to run, and with dovecot i need also handle this two domains, but appairing this error messages. like: Jun 29 20:49:28 Dovecot/imap-login: Info: Disconnected (no auth attempts in 0 secs): user=<>, rip=a.b.c.d, lip=37.120.190.188, TLS handshaking: SSL_accept() failed: error:14094416:SSL routines: ssl3_read_bytes:sslv3 alert certificate unknown: SSL alert number 46, session=<FdklDjkdfrkfi> Running with Debian Buster # dovecot --version 2.3.4.1 (f79e8e7e4) # nmail.caloro.ch local_name nmail.caloro.ch { ssl_cert = </etc/letsencrypt/live/nmail.caloro.ch/privkey.pem ssl_key = </etc/letsencrypt/live/nmail.caloro.ch/fullchain.pem } # nmail.calm-ness.ch local_name nmail.calm-ness.ch { ssl_cert = </etc/letsencrypt/live/nmail.calm-ness.ch/privkey.pem ssl_key = </etc/letsencrypt/live/nmail.calm-ness.ch/fullchain.pem } thanks for possible help

6 5

Panic: file mail-index-transaction-finish.c: line 185
by tobisworld＠gmail.com 19 Sep '22

19 Sep '22

Hi list I have weird issue with my Dovecot 2.3.17.1 (476cd46418) When deleting a certain amount of messages from my INBOX via my MUA (Evolution) all over sudden dovecot starts to panic Panic: file mail-index-transaction-finish.c: line 185 (mail_index_transaction_get_uid): assertion failed: (seq <= t->view- >map->hdr.messages_count) imap(REDACTED)<24075><P0/vLkXUNt4KQmMB>: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7f09274d4142] -> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7f09274d424e] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf72fe) [0x7f09274e22fe] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf73a1) [0x7f09274e23a1] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0927430e38] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0xf438d) [0x7f092789d38d] - > /usr/lib64/dovecot/libdovecot-storage.so.0(+0xf43da) [0x7f092789d3da] -> /usr/lib64/dovecot/libdovecot- storage.so.0(mail_index_transaction_finish+0x177) [0x7f092789da87] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0xf9ad6) [0x7f09278a2ad6] - > /usr/lib64/dovecot/libdovecot- storage.so.0(mail_index_transaction_commit_full+0x9d) [0x7f09278a2fad] -> /usr/lib64/dovecot/libdovecot- storage.so.0(mail_index_transaction_commit+0x1c) [0x7f09278a306c] -> /usr/lib64/dovecot/libdovecot- storage.so.0(mail_cache_set_seq_corrupted_reason+0x61) [0x7f092788c921] -> /usr/lib64/dovecot/libdovecot- storage.so.0(mail_set_mail_cache_corrupted+0xed) [0x7f09277fbbbd] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x7a3e9) [0x7f09278233e9] - > /usr/lib64/dovecot/libdovecot-storage.so.0(+0x7a6b0) [0x7f09278236b0] -> /usr/lib64/dovecot/libdovecot- storage.so.0(mail_get_physical_size+0x38) [0x7f09277e8ba8] -> /usr/libexec/dovecot//lib20_mail_log_plugin.so(+0x2465) [0x7f09263dd465] -> /usr/libexec/dovecot//lib15_notify_plugin.so(notify_contexts_mail_copy+ 0x5c) [0x7f09265e13ec] -> /usr/libexec/dovecot//lib15_notify_plugin.so(+0x25d4) [0x7f09265e25d4] -> /usr/libexec/dovecot//lib10_quota_plugin.so(+0x102ae) [0x7f09269f92ae] -> /usr/libexec/dovecot//lib01_acl_plugin.so(+0xef3e) [0x7f0926c10f3e] -> /usr/lib64/dovecot/libdovecot- storage.so.0(+0x50c4c) [0x7f09277f9c4c] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_move+0x58) [0x7f09277f9ec8] -> dovecot/imap(+0x1175b) [0x55a2c0ea575b] -> dovecot/imap(command_exec+0x64) [0x55a2c0eb3f04] -> dovecot/imap(+0x1dde2) [0x55a2c0eb1de2] -> dovecot/imap(+0x1de71) [0x55a2c0eb1e71] -> dovecot/imap(+0x1e039) [0x55a2c0eb2039] -> dovecot/imap(client_handle_input+0x205) [0x55a2c0eb22f5] -> dovecot/imap(client_input+0x79) [0x55a2c0eb28d9] imap(REDACTED)<24075><P0/vLkXUNt4KQmMB>: Fatal: master: service(imap): child 24075 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) >From that point on ANY access to INBOX from my MUA causes the error. The only way that I found to get around is to unsubscribe the INBOX folder and subscribe again. Then the error is gone until I delete a bunch of messages again. The error only occurs if I delete the messages within short time or select multiple (about 20msg at once). If I delete them one-by-one always waiting for expunge message in dovecot logs to appear then I can delete as many as I want. I have not verified with other MUA but in Evolution I can reproduce the issue. Cheers and have a good one tobi

4 14

Unclear how to get both user and domain quotas to work
by Hans van Zijst 10 Sep '22

10 Sep '22

Hi, First of all: my apologies for this long message. I'm trying to post all the information anybody could need to help me on my way. What I'm trying to make is an installation where both users and domains can have quotas. Ideally I would configure a quota for a domain and set a quota of 0 for all users within that domain, so that mail will be accepted up to the point where the domain quota is reached. Setting up user quota was fairly easy, there's lots of documentation around and examples to take inspiration from. Not so much for domain quota however, or I'm really bad at searching the Net. I'm running Debian Bullseye here: MariaDB 10.5.12 Dovecot 2.3.13 Postfixadmin 3.3.10 I want to use dictionaries in MySQL, because eventually this will be a platform with several Dovecot and Postfix servers. At this time I'm only running one Dovecot installation and a few Postfix servers (two for communication with the outside world and one to do the virus and spam scanning). My current setup for user quota is how Postfixadmin wants it: MariaDB [postfixadmin]> desc quota2; +----------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +----------+--------------+------+-----+---------+-------+ | username | varchar(100) | NO | PRI | NULL | | | bytes | bigint(20) | NO | | 0 | | | messages | int(11) | NO | | 0 | | +----------+--------------+------+-----+---------+-------+ 3 rows in set (0.001 sec) In /etc/dovecot/conf.d/90-quota.conf I have this: plugin { quota = dict:User quota::proxy::sqlquota quota_rule = *:bytes=1024M quota_rule2 = Trash:bytes=+100M quota_vsizes = yes quota_status_success = DUNNO quota_status_nouser = DUNNO quota_status_overquota = "552 5.2.2 Mailbox is full" } dict { sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } The SQL-configuration /etc/dovecot/dovecot-dict-sql.conf.ext is this: connect = host=... dbname=... user=... password=... map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } map { pattern = shared/expire/$user/$mailbox table = expires value_field = expire_stamp fields { username = $user mailbox = $mailbox } } I use LMTP to accept mail from Postfix; the quota-plugin is activated for LMTP, so every incoming mail should be counted. This works like a charm: if I don't configure a quota for a user, he gets the default 1GB, if I override it in Postfixadmin, then that's what he gets. I'm running quota-status on a TCP-port, so the gateway SMTP-servers can connect to Dovecot for a quotacheck to decide whether to accept or reject an incoming message. Again, this works flawlessly. I can't find any serious documentation about this mapping thing. In particular, what's with that "pattern"? All I can find in the documentation is: "The dict key must match exactly priv/quota/storage. The dict keys are hardcoded in the Dovecot code, so depending on what functionality you’re configuring you need to know the available dict keys used it." -- https://doc.dovecot.org/configuration_manual/dict/#dict-sql If anyone can point me at better documentation than the actual source code: please! I browsed through the quota-plugin source code, but I'm not a programmer and couldn't find what I thought I was looking for. That said, I'm not sure where I got the third "map" from, with the shared/expire pattern. I seem to have forgotten to document that bit of my journey. Now, domain quotas... I've tried all kinds of stuff, but at the moment I'm stuck with something that seems to almost work. I added a second quota root: plugin { quota2 = dict:Domain quota:%d:proxy::sqldomainquota quota2_rule = *:bytes=1024M quota2_status_success = DUNNO quota2_status_nouser = DUNNO quota2_status_overquota = "552 5.2.2 Domain quota exceeded" quota2_vsizes = yes } dict { sqldomainquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } This dict refers to the same mapping as the one for user quotas, and I think that's the way it should be. Please correct me if I'm wrong. This creates a new user in the quota2 table, by the name of just the domain. Unfortunately, it simply fills it with the numbers from the first user it can find in that domain, not all users in that domain combined. What I think I should do, is change something in the dict so that it sums up all users in that domain, something like this: SELECT SUM(bytes) FROM quota2 WHERE username LIKE '%@domain'; Problem is that I can't find anything useful online with regards to this, I'm not even sure if I'm thinking in the right direction here. Any hints, suggestions or corrections? Or pointers to articles or documentation I might have missed? Kind regards, Hans

2 1

Dovecot ACLs and XOAUTH2
by Felix Auringer 23 Aug '22

23 Aug '22

Hi, we're trying to set up an IMAP server using Dovecot as part of a new email system in a middle-sized company (around 150 employees). We have a keycloak identity server that is used for all logins and we would also like to use it for the new email system so that the non-technical employees (the majority) will have no issues using it. It's very nice that XOAUTH2 is already included in Dovecot, it works very well with our Roundcube web client. However, it seems that other clients like Thunderbird or Android Apps like FairEmail are not easy to set up as they have their XOAUTH providers hardcoded somewhere (https://bugzilla.mozilla.org/show_bug.cgi?id=1602166) Are you aware of any solution to integrate a dovecot server using XOAUTH2 into local clients (like Thunderbird)? We are currently trying to do it with passwords that can be set in Keycloak only for the purpose of email that can then be used to receive a valid token via a direct access grant but it's not a really nice solution as it introduces an additional password. Additionally, we would like to have all permission-related information saved in our identity server. For email, this includes shared mailboxes that a user is allowed to access. The ACL plugin that is used for shared mailboxes currently reads the permissions from disk which is not really feasible if the user base is large and setting these permissions should happen automatically. We are thinking the best way to do this is to encode the user's permissions in the token that dovecot receives in the login process. But as the token is only available in the authentication process and it does not seem to be intended to return such information from the password database (https://doc.dovecot.org/configuration_manual/authentication/password_databa…) we are unsure on how to process these information from the token. We did not find any plugin hooks in the authentication processes and the normal plugin hooks do not have access to information from the token anymore. What do you think is the best way to do this? We would be okay modifying some dovecot source code and contributing it back if desired. Best regards, Felix Auringer --- Gesellschaft für interkulturelles Zusammenleben gGmbH (GIZ) Felix Auringer IT Reformationsplatz 2 13597 Berlin Tel: 030/513 0100 00; Fax: 030/513 0100 09 www.giz.berlin; felix.auringer(a)giz.berlin Amtsgericht Charlottenburg HRB 200872 B Geschäftsführerin: Dr. Britta Marschke

2 6