January 2025 - dovecot

Different Replicator Crash??
by Kent Borg 03 Feb '25

03 Feb '25

I added the suggested "count--" and recompiled, and it now appears to work, the replication goes both ways, eyeballing the e-mail on the two servers they look the same. Good. But part of the time I am still getting what looks very much like the same crash, is there another "count--" I need to add to that file? > Jan 28 10:53:08 la dovecot[96175]: replicator: Panic: data stack: Out > of memory when allocating 268435496 bytes > Jan 28 10:53:08 la dovecot[96175]: replicator: Error: Raw backtrace: > #0 t_askpass[0x7fb404b0c0] -> #1 backtrace_append[0x7fb404b374] -> #2 > backtr\ > ace_get[0x7fb404b510] -> #3 execvp_const[0x7fb4057ba4] -> #4 > i_syslog_fatal_handler[0x7fb4058510] -> #5 i_panic[0x7fb3fa6808] -> #6 > t_pop_pass_st\ > r[0x7fb4050eb4] -> #7 connection_deinit[0x7fb4056430] -> #8 > pool_datastack_create[0x7fb407e7b0] -> #9 > array_bsearch_insert_pos_i[0x7fb40446d0] ->\ > #10 t_base64_scheme_decode[0x7fb404cb24] -> #11 > buffer_append[0x7fb404d3d0] -> #12 replicator_queue_push[0x558c4b4f40] > -> #13 replicator_brain_i\ > nit[0x558c4b53e4] -> #14 _start[0x558c4b3d70] -> #15 > _start[0x558c4b3f04] -> #16 io_loop_call_io[0x7fb4072f20] -> #17 > io_loop_handler_run_interna\ > l[0x7fb4075450] -> #18 io_loop_handler_run[0x7fb4075680] -> #19 > io_loop_run[0x7fb4075944] -> #20 master_service_run[0x7fb3fd84b0] -> > #21 main[0x5\ > 58c4b3930] -> #22 __libc_init_first[0x7fb3dd76d0] -> #23 > __libc_start_main[0x7fb3dd7780] -> #24 _start[0x558c4b3b80] > Jan 28 10:53:08 la dovecot[96175]: replicator: Fatal: master: > service(replicator): child 96714 killed with signal 6 (core dumps > disabled - https:\ > //dovecot.org/bugreport.html#coredumps) Thanks, -kb

3 2

Problem email client iPhone ios18.2
by andrzej.milewski＠gmail.com 03 Feb '25

03 Feb '25

Hello, I have a fairly new installation of the Dovecot server version 2.3.19.1 on Debian 12.8. In general, the server works correctly. It sends and receives emails without issues. The clients primarily use Thunderbird on desktops and Gmail on Android. However, some clients are using iPhones with various hardware versions. Those with iOS below version 18 work correctly without any issues. Those who have updated their phones to version 18 and now even 18.2 are experiencing problems with mailbox synchronization. On version 18.2, the issue appears as follows: When you open the app, you can read several messages. They load correctly, but, for instance, attachments do not. If I wait for some time, the attachment eventually loads. During this time, I observe the server logs, and it shows that a certain session on the Dovecot server has ended. Dec 17 13:13:01 iredmail0 dovecot[408]: imap(xxx(a)xxx.com)<44785><1QGZCHcp/+3AqM2O>: Disconnected: Connection closed (UID SEARCH finished 92.295 secs ago) in=2972 out=6076956 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=36 body_bytes=6059064 Dec 18 08:40:41 iredmail0 dovecot[408]: imap(xxx(a)xxx.com)<177412>: Disconnected: Connection closed (UID FETCH finished 90.079 secs ago) in=5126 out=1304237 deleted=0 expunged=0 trashed=0 hdr_count=94 hdr_bytes=289688 body_count=89 body_bytes=969615 Dec 18 08:42:19 iredmail0 dovecot[408]: imap(xxx(a)xxx.com)<177410><D5mqU4cpZ+nAqM2O>: Disconnected: Connection closed (UID FETCH finished 95.051 secs ago) in=4475 out=1162785 deleted=0 expunged=0 trashed=0 hdr_count=95 hdr_bytes=281674 body_count=61 body_bytes=839545 Dec 18 08:44:17 iredmail0 dovecot[408]: imap(edyta.kowalewska(a)laktopol.com.pl)<177918>: Disconnected: Connection closed (UID FETCH finished 92.132 secs ago) in=29399 out=30266644 deleted=0 expunged=0 trashed=0 hdr_count=565 hdr_bytes=1726228 body_count=628 body_bytes=28109455 After the session is closed, the attachment loads immediately. From the logs and router analysis, it appears that the download happens in another previously opened session. After this incident, I go to another message with an attachment, and the situation repeats. There are also situations where the email body does not load, and the app waits for the server until the session closes and the content is fetched in another session. It does not matter whether I connect to the server via the internet or through a local network. On an iPhone 7 with iOS 15.8.3, everything works smoothly. Attachments open without issues. On iOS 18.1.1, attachments do not load, and the session does not close on its own. It takes forever unless the app is closed. In the Gmail app, everything works correctly without any issues. This is clearly a problem with the mail client on iPhone. However, is there anything I can adjust in my server settings to mitigate this issue for users? My dovecot configuration https://pastebin.com/UPNQ5dZz

20 39

Dovecot v2.4.0 released
by Aki Tuomi 03 Feb '25

03 Feb '25

Hi all, after a very long wait we are finally happy to release Dovecot v2.4.0! Some IMPORTANT things to notice: - We have changed the signing key for 2.4 going forward, releases are signed with EF0882079FD4ED32BF8B23B2A1B09EF84EDC5219, which can be found at https://repo.dovecot.org/DOVECOT-REPO-GPG-2.4 and is signed with the previous key. The old key has been renamed to https://repo.dovecot.org/DOVECOT-REPO-GPG-2.3. - New 2.4 packages **are not** compatible with old 2.3 configuration, please carefully review https://doc.dovecot.org/2.4.0/installation/upgrade/2.3-to-2.4.html before installing the new packages. - We are happy to provide experimental arm64 support in the form of a Docker image. - Docker images are now ran rootless, with UID 1000 as vmail using built sources. Please take this into consideration when upgrading. Latest 2.3 image can be used with tag 2.3-latest, if you are not ready for this change. We won't be sending separate mail about Pigeonhole anymore as we will release one anyways when we release Dovecot, and Pigeonhole versioning has been changed to match Dovecot versioning. Source codes available at - https://www.dovecot.org/releases/2.4/dovecot-2.4.0.tar.gz - https://www.dovecot.org/releases/2.4/dovecot-2.4.0.tar.gz.sig - https://pigeonhole.dovecot.org/releases/2.4/dovecot-pigeonhole-2.4.0.tar.gz - https://pigeonhole.dovecot.org/releases/2.4/dovecot-pigeonhole-2.4.0.tar.gz… Binary packages in https://repo.dovecot.org/ Docker images in https://hub.docker.com/r/dovecot/dovecot Kind regards, Aki Tuomi Open-Xchange oy --- Dovecot Core NEWS ----------------- * config: dovecot_config_version must be the first non-comment line in configuration file. * config: dovecot_storage_version must be in the configuration file. * config: Many configuration options have changed so old configuration files do not work without rewrite. See https://doc.dovecot.org/main/installation/upgrade/2.3-to-2.4.html * config: New variable expansion syntax has been introduced, see https://doc.dovecot.org/main/core/settings/variables.html * config: Some default settings have changed. * config: plugin {} section has been removed. * *-login: With ssl=required, connections from login_trusted_networks are now also required to be SSL/TLS encrypted. * acl: Use ACL settings instead of Global ACL Directories. * auth-worker: auth_worker_max_count is replaced with service auth-worker { process_limit }. * auth: Weak password schemes are disabled by default, use auth_allow_weak_schemes to enable them. * auth_debug, mail_debug: Use log_debug filter instead. * config: All sections require a name, for example passdb/userdb: passdb static { password=secret } * db2: Remove Berkeley DB support. * dict-memcached: This is removed, use Redis instead. * director: Feature has been removed. See potential replacement at https://github.com/dovecot/tools/blob/main/director.lua * doveadm: USER environment variable is only supported with --no-userdb-lookup. One of -u, -F or -A must be used otherwise. * doveconf: Option -n is now default when running doveconf. * dsync: Use doveadm sync instead, legacy symlink has been removed. * fs-sis: Feature is now deprecated and has been made read-only. It will be removed in future release. * fts-lucene, fts-squat: These have been removed, use fts-flatcurve or fts-solr instead. * imap-login: IMAP compression is now handled in proxies. * imap_quota: SETQUOTA / quota_set has been removed. * imap_zlib: This plugin is no longer needed, it's always enabled. * imapc: All features are enabled by default, imapc_features can be used to explicitly disable features that are not wanted. * lib-storage: mbox driver is now frozen. * mail_compress: XZ and LZMA algorithm support has been removed. * mailbox-alias: Plugin has been removed. * old_stats, auth_stats: These have been removed. * openssl: Minimum supported version of OpenSSL is now 1.1.1. * openssl: Add support for OpenSSL 3.x * quota-dict, quota-dirsize: These have been removed, use quota-count instead. You can use quota_clone to copy quota usage to some database. * replicator: Feature has been removed. Use NFS or some other shared filesystem instead, or run doveadm sync in crontab. * stats: The bytes_in and bytes_out field in several events have been renamed as net_in_bytes and net_out_bytes. * zlib: Renamed to mail_compress plugin. + Experimental SMTPUTF8 and IMAP UTF8=ACCEPT support has been added. Needs --enable-experimental-mail-utf8 configure option and mail_utf8_extensions=yes setting. + Long running mail commands can be aborted with Ctrl-C / doveadm kick. + auth: LDAP driver now supports multi-value attributes. + auth: Add support for SCRAM-SHA-1-PLUS and SCRAM-SHA-256-PLUS. + auth: Add support for TLS channel binding. + auth: Support sending JA3 hash to policy server. + configure: Detect latest Lua version. + *-login: Support for TLS Server Name has been improved to allow pre-login settings. For example capabilities to be changed based on TLS Server Name. + *-login: Support for TLS ALPN has been added, connections with mismatching application are now refused. Missing ALPN is accepted. + fts-flatcurve: New Xapian based FTS plugin has been added. + imap: Support for INPROGRESS untagged messages as per RFC 9585. + lib-lua: Expose Dovecot DNS client. + lib-lua: Expose Dovecot HTTP client. + lib-sasl: Support SCRAM-SHA mechanisms. + lmtp: SNI support has been added which allows settings to be applied based on TLS Server Name. + sqlite: Support WAL mode. + stats: Submetric name size has been increased. + submission: Add submission_add_received_header setting to protect sender identity by suppressing the Received: header. - Many bugs have been fixed. Pigeonhole NEWS --------------- * Change configuration syntax to match new Dovecot configuration syntax. * vacation: Reduce default days to 60 from infinity * vacation: vacation_max_period=0 is now an error. * Version has been changed to match Dovecot version. + Added i;unicode-casecmp comparator. - Lots of bugs have been fixed.

17 41

Clarification Question about migrating_mailboxes and broken doc link
by Aleks 02 Feb '25

02 Feb '25

Hi. I have read the https://doc.dovecot.org/2.3/admin_manual/migrating_mailboxes/#id7 and there is one part which is not 100% clear to me. We have 2 Dovecot servers mail1 (2.2.36.4) and mail2 ( 2.3.21 ). Now I have understood that I need to add this to mail1 ``` service doveadm { inet_listener { port = 12354 } } doveadm_password = supersecret ``` and this on mail2 ``` doveadm_password = supersecret ``` The question is for me now: I think that this command `doveadm backup -Ru username tcp:host:port` should be executed on the mail2. Is this assumption right? What I also found is that on that Page https://doc.dovecot.org/2.3/admin_manual/doveadm_mailbox_commands/ the wiki is linked for the manpages but the wiki link shows this. ``` The old wiki has been closed Our primary documentation is now at https://doc.dovecot.org, please update your links. ``` What's the right link for the manpages in the web? Best regards Aleks

2 2

[Off-topic] ANN: Nauthilus
by Christian Rößner 01 Feb '25

01 Feb '25

Today I am pleased to announce the Nauthilus software. Nauthilus (N-Auth-ilus) is a centralized authentication server with a comprehensive policy engine. The idea behind this project is to connect services centrally in one place and perform extensive tests during authentication. # What can Nauthilus do and what problems does it solve? Many services on the Internet require authentication. Each application must then be connected to databases and must implement its own protective measures to ward off attackers. Every installation carries the risk of compromisation and therefore also access to the databases. With Nauthilus, the task is centralized in one place. Nauthilus is essentially an HTTP REST server that can be accessed by any application. It takes on the role of a guardian. Nauthilus integrates very well with Dovecot and Postfix. # Authentication process Nauthilus uses several authentication steps: ## 1. features Features include tests such as TLS verification, relay domains (is the system even responsible for the requested domain?), blocklists, RBLs and freely definable Lua features. Nauthilus has a powerful brute force concept with buckets to detect even slow attacks over days and weeks. It offers a bucket system for this purpose. ## 2. backend authentication Nauthilus includes LDAP support and Lua to perform the authentication itself. A large library of predefined functions is available in Lua, including SQL support. ## 3 Policies Policies are run through after authentication. Despite a successful login, the system can reject the login (or, conversely, allow it!). There is also space here for GeoIP lookups, etc. ## 4. Post-processing After the 3rd point, authentication has been completed, but at this point further tests can run in the background such as: * Check password policy and take action * Consult the Haveibeenpwnd network * GeoIP tracking across national borders and much more. ## Miscellaneous Nauthilus allows the free definition of so-called hooks. Each hook listens for a specific URI (callback) in the HTTP request. These callbacks are written by the administrator in Lua. In an initial proof of concept, Nauthilus can take on the role of a Dovecot director. This has already been tested with version 2.4.0. Currently, Nauthilus can dynamically delegate incoming connections to backends. The hooks concept is used here as an example. # Final words By integrating a Lua VM into the server, Nauthilus can be integrated and customized in almost any setup. See also the other Nauthilus-related projects listed in the appendix. To enable single sign-on (SSO), it can be operated with an Ory-Hydra server or the sister project nauthilus-keycloak can be used as a custom authenticator in Keycloak. 100% Open-Source 100% Community # Project https://github.com/croessner/nauthilus # Sub projects https://github.com/croessner/nauthilus-demo https://github.com/croessner/nauthilus-keycloak https://github.com/croessner/pfxhttp https://github.com/croessner/geoip-policyd # Mailing lists: https://lists.nauthilus.org N.B.: In the future, announcements are sent over the nauthilus-announce ML. This is just a hello world! Christian Rößner -- Rößner-Network-Solutions Zertifizierter ITSiBe / CISO Marburger Str. 70a, 36304 Alsfeld Fax: +49 6631 78823409, Mobil: +49 171 9905345 USt-IdNr.: DE225643613, https://roessner.website PGP fingerprint: 658D 1342 B762 F484 2DDF 1E88 38A5 4346 D727 94E5

2 2

Replication ade
by Hanns Mattes 01 Feb '25

01 Feb '25

Hei, mit 2.4 verschwindet ja, wenn ich das richtig verstanden habe, die Replication aus dem Werkzeugkasten von Dovecot. Obwohl ich nur ein recht kleines System mit etwa 500 Nutzern betreibe fand ich die Replication als ziemlich beruhigend :-) Meine Frage in die Runde: Wie wollt Ihr die Replikation ersetzen? Irgendwas mit NFS oder gibt es da eine empfohlene Herangehensweise? Fragende Grüße Hanns -- Sig for hire.

4 4

2.4.0 missing fields in documentation for event smtp_server_transaction_finished
by Christian Rößner 01 Feb '25

01 Feb '25

Hi, Event: smtp_server_transaction_finished remote_ip and remote_port are also available Feature-request: Please also add user to the fields, as lmtp_proxy=yes does a userdb lookup and is aware of the user. Would make things much easier... Regards Christian Rößner -- Rößner-Network-Solutions Zertifizierter ITSiBe / CISO Marburger Str. 70a, 36304 Alsfeld Fax: +49 6631 78823409, Mobil: +49 171 9905345 USt-IdNr.: DE225643613, https://roessner.website PGP fingerprint: 658D 1342 B762 F484 2DDF 1E88 38A5 4346 D727 94E5

2 1

2.4.0 on MacOS
by John Muccigrosso 01 Feb '25

01 Feb '25

I got bitten by the upgrade from 2.3 to 2.4 on my MacOS laptop. I know enough to be dangerous here and had a set-up that was working fine, but now I can't quite figure out how to modify my conf file to work correctly. The current error I'm getting is that there is no group "dovecot" even though I've created one and made the default user a member: > Fatal: service(dict) Group doesn't exist: dovecot More broadly I find the documentation hard to follow, as others seem to have noted in the past. Any help appreciated. Jackie

2 3

Help understanding doveadm-sync stateful
by Bruno MATEU 01 Feb '25

01 Feb '25

Hello, With the removal of the replication plugin in next releases, i'm trying to setup automatic synchronization using doveadm sync between my master and my replica. I managed to make the full sync work from my replica using the following comamnd `sync -A -N -R -f tcp:master-fqdn:12345` However i'd like to do stateful synchronization to avoid doing the full replication each time, and i haven't managed to do that. I tried the following: `doveadm sync -A -N -R -s "$(</var/lib/dovecot/sync-state)" tcp:master-fqdn:12345 > /var/lib/dovecot/sync-state`. However this output a `doveadm(username): Error: Saved sync state is invalid, falling back to full sync: Invalid base64 data` for every user. After looking at the state in /var/lib/dovecot/sync-state, i noticed that there is one line for every user with an associated state, plus one line with only a state. For some reason, the state is the same for every user, except one where the state is much longer. Do i need to run the command for each user, and give as an argument the corresponding state ? Is it enough to run the doveadm sync command with -A argument if i give the state in the last line outputted by doveadm sync ? Is this the right way to do it ? Best regards, Bruno MATEU

1 1

Different Replicator Crash??
by Kent Borg 31 Jan '25

31 Jan '25

I added the suggested "count--" and recompiled, and it now appears to work, the replication goes both ways, eyeballing the e-mail on the two servers they look the same. Good. But part of the time I am still getting what looks very much like the same crash, is there another "count--" I need to add to that file? > Jan 28 10:53:08 la dovecot[96175]: replicator: Panic: data stack: Out > of memory when allocating 268435496 bytes > Jan 28 10:53:08 la dovecot[96175]: replicator: Error: Raw backtrace: > #0 t_askpass[0x7fb404b0c0] -> #1 backtrace_append[0x7fb404b374] -> #2 > backtr\ > ace_get[0x7fb404b510] -> #3 execvp_const[0x7fb4057ba4] -> #4 > i_syslog_fatal_handler[0x7fb4058510] -> #5 i_panic[0x7fb3fa6808] -> #6 > t_pop_pass_st\ > r[0x7fb4050eb4] -> #7 connection_deinit[0x7fb4056430] -> #8 > pool_datastack_create[0x7fb407e7b0] -> #9 > array_bsearch_insert_pos_i[0x7fb40446d0] ->\ > #10 t_base64_scheme_decode[0x7fb404cb24] -> #11 > buffer_append[0x7fb404d3d0] -> #12 replicator_queue_push[0x558c4b4f40] > -> #13 replicator_brain_i\ > nit[0x558c4b53e4] -> #14 _start[0x558c4b3d70] -> #15 > _start[0x558c4b3f04] -> #16 io_loop_call_io[0x7fb4072f20] -> #17 > io_loop_handler_run_interna\ > l[0x7fb4075450] -> #18 io_loop_handler_run[0x7fb4075680] -> #19 > io_loop_run[0x7fb4075944] -> #20 master_service_run[0x7fb3fd84b0] -> > #21 main[0x5\ > 58c4b3930] -> #22 __libc_init_first[0x7fb3dd76d0] -> #23 > __libc_start_main[0x7fb3dd7780] -> #24 _start[0x558c4b3b80] > Jan 28 10:53:08 la dovecot[96175]: replicator: Fatal: master: > service(replicator): child 96714 killed with signal 6 (core dumps > disabled - https:\ > //dovecot.org/bugreport.html#coredumps) Thanks, -kb

1 0