March 2023 - dovecot - dovecot.org

Pigeonhole redirect is adding a message-id header when it already exists
by Sébastien Riccio 04 Aug '23

04 Aug '23

Hello, We are using sieve filter to redirect incoming mails to an external mailbox (gmail) an we are seeing some forwarded e-mails being rejected by gmail because of duplicate headers. I think was able to identify the probable cause of the duplicate headers: It seems that when the original message's "Message-ID" is not compliant, pigeonhole acts like if the header is not existing and adds it's own Message-ID. This result in having two "Message-ID" headers being submitted to the destination mail server. Original Message-ID: Message-ID: 1883biz_pay_after_purchase:0:0_572392900$ae7ed6e4d53b424c84aaf83b30c507e7 Pigeon hole adds his own Message-ID: Message-ID: <dovecot-sieve-1664386049-21512-0@510faa1ff2d9> Gmail answer: 550-5.7.1 [94.103.100.17] Our system has detected that this message is not RFC 550-5.7.1 5322 compliant: duplicate headers. To reduce the amount of spam sent 550-5.7.1 to Gmail, this message has been blocked. Please review 550 5.7.1 RFC 5322 specifications for more information. While I understand the original "Message-ID" seems to be all but compliant with RFCs, shouldn't a "Message-ID" be added only when "Message-ID" is really missing ? When redirecting a message, If the original "Message-ID" exists but is bogus shouldn't it still be forwarded untouched ? Also, maybe if pigeonhole is adding it's own "Message-ID", shouldn't it remove any previous "Message-ID" to avoid generating duplicate headers ? (but maybe it could break some DKIM stuff?) Any ideas how to workaround this issue ? Thanks a lot for your feedbacks! Sébastien RICCIO SYSTEM ADMINISTRATOR P +41 840 888 888 F +41 840 888 000 M sriccio(a)swisscenter.com<mailto:sriccio@swisscenter.com>

9 17

subscription, dots and subscribe to shared folder.
by Julien Nadal 24 Jul '23

24 Jul '23

Hello; We were able to set up shared folders in a cluster (using dovecot as proxy on dedicated front-end servers, without using director) by following https://doc.dovecot.org/configuration_manual/shared_mailboxes/cluster_setup/. Here is our related conf on imap farms (dovecot v2.3.16 on Almalinux): namespaces: mail_location = maildir:~/Maildir namespace default { inbox = yes location = prefix = separator = / } namespace shared { list = children location = imapc:~/shared/%%u:INDEXPVT=~/shared-pvt/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } imapc settings: imapc_features = fetch-bodystructure fetch-headers rfc822.size search modseq acl delay-login imapc_host = XXXXXX imapc_master_user = %{user} imapc_password = XXXXXX imapc_port = XXXXXX passdb { args = password=#hidden_use-P_to_show# userdb_namespace/shared/disabled=yes userdb_acl_user=%{auth_user} allow_nets=XXXXX/XX driver = static master = yes } dict { acl-mysql = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } This way, a folder is created under ~/ : shared & shared-pvt : When user A shares a folder with B, B will find ~/shared/userA/sharedfolder I should mention that the user authentication (backend: mysql) uses the full email address as we are in a multi-tenant context. So the folder looks like ~/shared/usera(a)domain.tld/folder Since then, everything works perfectly, we can share and see the shared folders in the webmail (SOGo). Unfortunately, accessing these folders through an imap client (tested on Thunderbird and MSOutlook) is impossible because the subscription fails (the folder is listed, but the subscription doesn't seem to do anything). I could not find any relevant logs but I was able to confirm on the command line: I am able to view folder with LIST, subscribe with an "OK" output, but the folder is not listed with LSUB. So I could see that the subscribtion file in maildir shows incorrect entries: V 2 Sent Trash Drafts Junk shared/testuser1@sub domain tld/INBOX/folder The dots in the folder names are replaced by tabs. I was able to recover things by editing the file as follows: Sent Trash Drafts Junk shared/testuser1(a)sub.domain.tld/INBOX/folder This means that Maildir++ and V2 subscription can't handle dots in folders. I found a previous discussion: https://dovecot.org/pipermail/dovecot/2021-January/121233.html But the final advice can't be applied on our side: - changing LAYOUT to FS is not possible for us - we have already migrated the dot separator to slash - changing the layout only for shared namespaces does not solve the problem and totally breaks the access to shared folders. Thanks a lot for your advice Best regards Julien

3 4

Warning sqlpool(mysql): Query failed
by Alessio Cecchi 27 Jun '23

27 Jun '23

Hi, with the last dovecot version (2.3.13) I see many of this warnings via "doveadm log errors": Feb 17 03:26:28 Warning: dict(31513): conn unix:dict (pid=31330,uid=95): dict(mysql)<email1(a)company1.com>: sqlpool(mysql): Query failed, retrying: MySQL server has gone away (idled for 679 secs) Feb 17 03:48:53 Warning: dict(31013): conn unix:dict (pid=31805,uid=95): dict(mysql)<email2(a)company2.com>: sqlpool(mysql): Query failed, retrying: MySQL server has gone away (idled for 685 secs) Feb 17 03:54:12 Warning: dict(32529): conn unix:dict (pid=3147,uid=95): dict(mysql)<email3(a)company3.com>: sqlpool(mysql): Query failed, retrying: MySQL server has gone away (idled for 670 secs) What do they mean? I thinks are related to inactivity timeout on mysql server (where I have a wait_timeout=600), but these warnings could be problems or can be fixed with some setting in dovecot? My dovecot configuration about dict: dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext sieve = mysql:/etc/dovecot/dovecot-dict-sieve-sql.conf.ext sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } plugin { [...] acl_shared_dict = proxy::acl expire_dict = proxy::expire quota2 = dict:Quota Usage::noenforcing:proxy::sqlquota sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = dict:proxy::sieve;name=activesql [...] } service dict { process_limit = 500 unix_listener dict { group = vmail mode = 0660 user = vmail } } Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 1

dovecot replication crashing
by Heiko Schlittermann 25 May '23

25 May '23

Hi, I'm a but clueless, having issues with replication. `doveadm dsync -u hans` works. But using the following replication setup, I see coredumps. Where to go next? Interestingly not for all users. (For testing purposes I've only 2 users. One having about 20 messages: here even the replication works, but the 2nd user (having about 14k messages) fails. The failure seems to happen immediatly after starting the replication attempt). I saw the replication crashing using Dovecot packages from the current Debian distro. In order to debug this, I'm now using 2.3.20, built from Git. The replicator config boils down to this (complete config is attached) replication_full_sync_interval = 90d replication_max_conns = 16 mail_plugins = $mail_plugins notify replication plugin { mail_replica = tcps:smtp-mz.example.com:9090 } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } service aggregator { fifo_listener replication-notify-fifo { user = dovecot mode = 0666 } unix_listener replication-notify { user = dovecot mode = 0666 } } The stacktrace I get looks like this: Stack trace of thread 729589: #0 0x00007f9e4f2c8ce1 __GI_raise (libc.so.6 + 0x38ce1) #1 0x00007f9e4f2b2537 __GI_abort (libc.so.6 + 0x22537) #2 0x00007f9e4f5fe8c6 default_fatal_finish (libdovecot.so.0 + 0x558c6) #3 0x00007f9e4f6ab601 i_internal_fatal_handler (libdovecot.so.0 + 0x102601) #4 0x00007f9e4f5fe589 i_panic (libdovecot.so.0 + 0x55589) #5 0x00007f9e4f5fe99e fd_set_nonblock (libdovecot.so.0 + 0x5599e) #6 0x00005564cbe6a08d cmd_dsync_ibc_stream_init (doveadm-server + 0x3008d) #7 0x00005564cbe6b772 cmd_dsync_run (doveadm-server + 0x31772) #8 0x00005564cbe6d284 doveadm_mail_next_user (doveadm-server + 0x33284) #9 0x00005564cbe6e4ba doveadm_mail_cmd_exec (doveadm-server + 0x344ba) #10 0x00005564cbe7eb71 doveadm_cmd_run_ver2 (doveadm-server + 0x44b71) #11 0x00005564cbe8300a doveadm_cmd_server_run_ver2 (doveadm-server + 0x4900a) #12 0x00007f9e4f6c1799 io_loop_call_io (libdovecot.so.0 + 0x118799) #13 0x00007f9e4f6c2e82 io_loop_handler_run_internal (libdovecot.so.0 + 0x119e82) #14 0x00007f9e4f6c1840 io_loop_handler_run (libdovecot.so.0 + 0x118840) #15 0x00007f9e4f6c1a00 io_loop_run (libdovecot.so.0 + 0x118a00) #16 0x00007f9e4f6343a3 master_service_run (libdovecot.so.0 + 0x8b3a3) #17 0x00005564cbe5d9a2 main (doveadm-server + 0x239a2) #18 0x00007f9e4f2b3d0a __libc_start_main (libc.so.6 + 0x23d0a) #19 0x00005564cbe5da2a _start (doveadm-server + 0x23a2a) 2023 Mar 28 00:09:28 clone doveadm(hans)<729589><TdSzNhcUImT1IQsAKQCViw>: Fatal: master: service(doveadm): child 729589 killed with signal 6 (core dumped) I'm a bit clueless now, even using the coredump and gdb. -- Heiko

3 7

dovecot-fts-solr Solr9 support
by PGNet Dev 24 Apr '23

24 Apr '23

I run dovecot-2.3.18 , & am deploying Solr search with 'dovecot-fts-solr' I run Solr standalone. When creating a dedicated 'dovecot' core, cp'ing from Dovecot's included solr schemas ls -al /usr/share/doc/dovecot/solr-* -rw-r--r-- 1 root root 12K Dec 3 06:48 /usr/share/doc/dovecot/solr-config-7.7.0.xml -rw-r--r-- 1 root root 2.7K Dec 3 06:48 /usr/share/doc/dovecot/solr-schema-7.7.0.xml -rw-r--r-- 1 root root 2.9K Dec 3 06:48 /usr/share/doc/dovecot/solr-schema.xml is tested only for Solr 7.7.0 https://doc.dovecot.org/configuration_manual/fts/solr/ "The steps described in this page are tested for Solr 7.7.0. For other versions, this these steps may need to be adjusted." which, per solr docs https://solr.apache.org/downloads.html About versions and support Version Description 9.x Current major version for feature releases (STABLE) 8.11.x Previous major version may sometimes receive critical bugfix releases 10 Next major version, yet to be released (UNSTABLE) <8.11 All older versions are End Of Life (EOL) is EOL'd and unsupported. In the dovecot 7.7.0 schema, FastLRUCache/LRUCache are used as the caching class(es), grep class=.*LRU /usr/share/doc/dovecot/solr-config-7.7.0.xml <filterCache class="solr.FastLRUCache" <queryResultCache class="solr.LRUCache" <documentCache class="solr.LRUCache" class="solr.search.LRUCache" For Solr 9, cat CHANGES.txt Upgrade Notes --------------------- ... * SOLR-13817: Deprecate legacy SolrCache implementations. Users are encouraged to transition their configurations to use org.apache.solr.search.CaffeineCache instead. (ab) running solr9 + dovecot-fts-solr search, based on the 7.7.0 schemas, results in errors, failed scan & search. Modifying the config to use CaffeineCache cp solr-config-7.7.0.xml solr-config-9.0.0.xml perl -pi -e '\ s|solr.FastLRUCache|solr.CaffeineCache|g; \ s|solr.LRUCache|solr.CaffeineCache|g; \ s|solr.search.LRUCache|solr.search.CaffeineCache|g;' \ solr-config-9.0.0.xml appears to do the trick. Atm, indexing & search -- both manually triggered, and on delivery receipt to Dovecot store -- is working with no errors (yet). I've not tested exhaustively yet, & don't know if this Cache class change in the schema is both necessary & sufficient for dovecot-fts-solr search use with solr 9.0.0 Given that for Solr "<8.11 All older versions are End Of Life (EOL)" it'd be useful for Dovecot to 'retest' for current, not-EOL'd Solr, and publish/package compatible schemas accordingly.

5 8

Custom SASL authentication
by David Koski 19 Apr '23

19 Apr '23

Hello, I'm looking for a good way to apply a custom hash to passwords. My hope is to add passwords to a (MySQL) database: INSERT INTO users (user='joblo', pass=MYHASH('plain-password').. For SASL authentication, my thought first was to apply the same hash to the issued password and compare it with the hashed password in the database. I soon discovered the sql driver supplied by Dovecot doesn't provide that ability, unless I'm missing something. I'm looking for documentation on how to implement a custom authentication script if needed. Regards, David Koski dkoski(a)sutinen.com

4 6

decrypt.rb
by Dave 11 Apr '23

11 Apr '23

Per the Dovecot site here: https://wiki.dovecot.org/Plugins/MailCrypt ... the "decrypt.rb" ruby script can be used to decrypt a Dovecot-encrypted message file from the command line. The script sort of runs successfully for me, in the sense that it doesn't error out, but it doesn't show the decrypted message. I've called it like so: decrypt.rb -k <PATH_TO_DOVECOT_KEYFILE> -f ./encrypted_message But when I do it just displays the info below and exits (I've subbed <DATA> tags for the actual data): ------------------[OUTPUT]----------------------- Key(s) (total: 1) - Key type : EC - Key digest: <DATA1> - Peer key : <DATA2> - Encrypted : <DATA3> - Kd hash : <DATA4> ------------------------------------------------------- I've tried specifying an output file as well, per the script's command line options, but the output file is 0 bytes. Does anyone have any suggestions? I *think* I'm using it the way it's intended to be used, but maybe I'm not?! -Dave

5 14

Error: /proc/self/status is larger than expected
by Elisamuel Resto Donate 06 Apr '23

06 Apr '23

Hello, Writing to get some insight as to how I might rid myself of a the pesky error message on the subject. Every binary throws out that error, every LMTP delivery... basically every time any dovecot binary is executed. I am unsure if this is configuration related or not, but intuition says it is an issue with a kernel feature or compiler issue. I saw the error message in the source code (file ), but I don't understand enough of the dovecot architecture or what the code itself is doing in this case. Below and excerpt of a few lines of logs. Since I don't know what specifically this would relate to (and I don't quite remember the command to send the "safe" version of the configuration), I am sending this message in hopes that whatever is needed can be asked and I will provide it. Regards, -Sam [... postfix and rspamd output...] Mar 20 22:17:02 gaia postfix/qmgr[21403]: 4PgcGQ33LfzdT4h: from=<list(a)someml.tld>, size=12064, nrcpt=1 (queue active) Mar 20 22:17:02 gaia postfix/smtpd[18941]: disconnect from some.list.server[1.2.3.4] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7 Mar 20 22:17:02 gaia dovecot: lmtp(18954): Error: /proc/self/status is larger than expected Mar 20 22:17:02 gaia dovecot: lmtp(18954): Connect from local Mar 20 22:17:02 gaia dovecot: lmtp(sam(a)samresto.dev)<18954><vbRKLa4hGWQKSgAAw1YR2w>: Error: /proc/self/status is larger than expected Mar 20 22:17:02 gaia dovecot: message repeated 2 times: [ lmtp(sam(a)samresto.dev)<18954><vbRKLa4hGWQKSgAAw1YR2w>: Error: /proc/self/status is larger than expected] Mar 20 22:17:02 gaia dovecot: lmtp(sam(a)samresto.dev)<18954><vbRKLa4hGWQKSgAAw1YR2w>: sieve: msgid=<84cf4a75-db54-f33c-cea2-f4772cd182d8(a)samresto.dev>: fileinto action: stored mail into mailbox 'Lists/Folder' Mar 20 22:17:02 gaia dovecot: lmtp(sam(a)samresto.dev)<18954><vbRKLa4hGWQKSgAAw1YR2w>: Error: /proc/self/status is larger than expected Mar 20 22:17:02 gaia dovecot: indexer-worker(sam(a)samresto.dev)<18956><vbRKLa4hGWQKSgAAw1YR2w:RtkiMa4hGWQMSgAAw1YR2w>: Error: /proc/self/status is larger than expected Mar 20 22:17:02 gaia postfix/lmtp[18943]: 4PgcGQ33LfzdT4h: to=<sam(a)samresto.dev>, relay=wyvern.samresto.dev[private/dovecot-lmtp], delay=0.4, delays=0.31/0/0.01/0.07, dsn=2.0.0, status=sent (250 2.0.0 <sam(a)samresto.dev> vbRKLa4hGWQKSgAAw1YR2w Saved) Mar 20 22:17:02 gaia dovecot: lmtp(18954): Disconnect from local: Logged out (state=READY) Mar 20 22:17:02 gaia dovecot: lmtp(18954): Error: /proc/self/status is larger than expected Mar 20 22:17:02 gaia postfix/qmgr[21403]: 4PgcGQ33LfzdT4h: removed

6 11

Dovecot error invalid service_fd
by Michel Verdier 30 Mar '23

30 Mar '23

Hi all On debian stable, dovecot 2.3.13+dfsg1-2+deb11u1 and postfix 3.5.17-0+deb11u1 I set postfix sending mails to dovecot for mailbox storage: mailbox_transport = lmtp:unix:private/dovecot-lmtp dovecot listen with lmtp: service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { mode = 0666 user = postfix group = postfix } } passdb { driver = pam args = session=yes cache_key=%u%r%l dovecot } userdb { driver = passwd } # ls -l /var/spool/postfix/private/dovecot-lmtp srw-rw-rw- 1 postfix postfix 0 27 mars 11:24 /var/spool/postfix/private/dovecot-lmtp I get logs like this: postfix/lmtp[271076]: 6DB551008B3C1: to=..., relay=...[private/dovecot-lmtp], delay=0.7, delays=0.29/0.04/0.11/0.26, dsn=2.0.0, status=sent (250 2.0.0 ... postfix/qmgr[197419]: 6DB551008B3C1: removed dovecot: lmtp(271077): Disconnect from local: Client has quit the connection (state=READY) dovecot: log(253615): Error: Received master input for invalid service_fd 27: 27 271079 BYE I identify 271079 as the pid for dovecot auth worker process. If I add a timeout in dovecot: service auth-worker { idle_kill = 10s } Then I see error log coming 10 seconds after the end of every lmtp connexion from postfix. There is no other log messages around even with debug enable in dovecot. And mails are correctly delivered. Am I missing something or is this to be ignored? Cheers

1 0

Dovecot submission proxy constraints
by Peter Wienemann 29 Mar '23

29 Mar '23

Dear Dovecot experts, in 2017 Stephan Bosch mentioned in an email about the Dovecot submission proxy [0]: "The current implementation does require that the proxy has direct access to the user's mailbox for BURL (e.g. by running it on the same host as imap), but that restriction should be resolved soon, allowing for more complex setups." Is the mentioned restriction (access to mailbox) still valid in 2023 or has this constraint been lifted in the meantime? Best regards, Peter [0] https://dovecot.org/pipermail/dovecot/2017-December/110282.html

1 0