January 2025 - dovecot

Replicator Crash
by Kent Borg 05 Feb '25

05 Feb '25

I have replicator working well enough…to crash: > Jan 28 11:26:12 la dovecot[1187]: replicator: Panic: data stack: Out > of memory when allocating 268435496 bytes > Jan 28 11:26:12 la dovecot[1187]: replicator: Error: Raw backtrace: #0 > t_askpass[0x7fa59db0c0] -> #1 backtrace_append[0x7fa59db374] -> #2 > backtra\ > ce_get[0x7fa59db510] -> #3 execvp_const[0x7fa59e7ba4] -> #4 > i_syslog_fatal_handler[0x7fa59e8510] -> #5 i_panic[0x7fa5936808] -> #6 > t_pop_pass_str\ > [0x7fa59e0eb4] -> #7 connection_deinit[0x7fa59e6430] -> #8 > pool_datastack_create[0x7fa5a0e7b0] -> #9 > array_bsearch_insert_pos_i[0x7fa59d46d0] -> \ > #10 t_base64_scheme_decode[0x7fa59dcb24] -> #11 > buffer_append[0x7fa59dd3d0] -> #12 replicator_queue_push[0x556e8f4f40] > -> #13 replicator_brain_in\ > it[0x556e8f53e4] -> #14 _start[0x556e8f3d70] -> #15 > _start[0x556e8f3f04] -> #16 io_loop_call_io[0x7fa5a02f20] -> #17 > io_loop_handler_run_internal\ > [0x7fa5a05450] -> #18 io_loop_handler_run[0x7fa5a05680] -> #19 > io_loop_run[0x7fa5a05944] -> #20 master_service_run[0x7fa59684b0] -> > #21 main[0x55\ > 6e8f3930] -> #22 __libc_init_first[0x7fa57676d0] -> #23 > __libc_start_main[0x7fa5767780] -> #24 _start[0x556e8f3b80] > Jan 28 11:26:12 la dovecot[1187]: replicator: Fatal: master: > service(replicator): child 1185 killed with signal 6 (core dumps > disabled - https://\ > dovecot.org/bugreport.html#coredumps) This is on Debian 12. > # dovecot --version > 2.3.19.1 (9b53102964) There are gigabytes of free RAM, though it says it is trying to allocate 268,435,496 bytes--on the stack--and that does seem wrong in any case. The Maildir being synced is over 430MB, is that too much for dsync?? Suggestions? Thanks, -kb

3 4

Clarification Question about migrating_mailboxes and broken doc link
by Aleks 02 Feb '25

02 Feb '25

Hi. I have read the https://doc.dovecot.org/2.3/admin_manual/migrating_mailboxes/#id7 and there is one part which is not 100% clear to me. We have 2 Dovecot servers mail1 (2.2.36.4) and mail2 ( 2.3.21 ). Now I have understood that I need to add this to mail1 ``` service doveadm { inet_listener { port = 12354 } } doveadm_password = supersecret ``` and this on mail2 ``` doveadm_password = supersecret ``` The question is for me now: I think that this command `doveadm backup -Ru username tcp:host:port` should be executed on the mail2. Is this assumption right? What I also found is that on that Page https://doc.dovecot.org/2.3/admin_manual/doveadm_mailbox_commands/ the wiki is linked for the manpages but the wiki link shows this. ``` The old wiki has been closed Our primary documentation is now at https://doc.dovecot.org, please update your links. ``` What's the right link for the manpages in the web? Best regards Aleks

2 2

[Off-topic] ANN: Nauthilus
by Christian Rößner 01 Feb '25

01 Feb '25

Today I am pleased to announce the Nauthilus software. Nauthilus (N-Auth-ilus) is a centralized authentication server with a comprehensive policy engine. The idea behind this project is to connect services centrally in one place and perform extensive tests during authentication. # What can Nauthilus do and what problems does it solve? Many services on the Internet require authentication. Each application must then be connected to databases and must implement its own protective measures to ward off attackers. Every installation carries the risk of compromisation and therefore also access to the databases. With Nauthilus, the task is centralized in one place. Nauthilus is essentially an HTTP REST server that can be accessed by any application. It takes on the role of a guardian. Nauthilus integrates very well with Dovecot and Postfix. # Authentication process Nauthilus uses several authentication steps: ## 1. features Features include tests such as TLS verification, relay domains (is the system even responsible for the requested domain?), blocklists, RBLs and freely definable Lua features. Nauthilus has a powerful brute force concept with buckets to detect even slow attacks over days and weeks. It offers a bucket system for this purpose. ## 2. backend authentication Nauthilus includes LDAP support and Lua to perform the authentication itself. A large library of predefined functions is available in Lua, including SQL support. ## 3 Policies Policies are run through after authentication. Despite a successful login, the system can reject the login (or, conversely, allow it!). There is also space here for GeoIP lookups, etc. ## 4. Post-processing After the 3rd point, authentication has been completed, but at this point further tests can run in the background such as: * Check password policy and take action * Consult the Haveibeenpwnd network * GeoIP tracking across national borders and much more. ## Miscellaneous Nauthilus allows the free definition of so-called hooks. Each hook listens for a specific URI (callback) in the HTTP request. These callbacks are written by the administrator in Lua. In an initial proof of concept, Nauthilus can take on the role of a Dovecot director. This has already been tested with version 2.4.0. Currently, Nauthilus can dynamically delegate incoming connections to backends. The hooks concept is used here as an example. # Final words By integrating a Lua VM into the server, Nauthilus can be integrated and customized in almost any setup. See also the other Nauthilus-related projects listed in the appendix. To enable single sign-on (SSO), it can be operated with an Ory-Hydra server or the sister project nauthilus-keycloak can be used as a custom authenticator in Keycloak. 100% Open-Source 100% Community # Project https://github.com/croessner/nauthilus # Sub projects https://github.com/croessner/nauthilus-demo https://github.com/croessner/nauthilus-keycloak https://github.com/croessner/pfxhttp https://github.com/croessner/geoip-policyd # Mailing lists: https://lists.nauthilus.org N.B.: In the future, announcements are sent over the nauthilus-announce ML. This is just a hello world! Christian Rößner -- Rößner-Network-Solutions Zertifizierter ITSiBe / CISO Marburger Str. 70a, 36304 Alsfeld Fax: +49 6631 78823409, Mobil: +49 171 9905345 USt-IdNr.: DE225643613, https://roessner.website PGP fingerprint: 658D 1342 B762 F484 2DDF 1E88 38A5 4346 D727 94E5

2 2

Replication ade
by Hanns Mattes 01 Feb '25

01 Feb '25

Hei, mit 2.4 verschwindet ja, wenn ich das richtig verstanden habe, die Replication aus dem Werkzeugkasten von Dovecot. Obwohl ich nur ein recht kleines System mit etwa 500 Nutzern betreibe fand ich die Replication als ziemlich beruhigend :-) Meine Frage in die Runde: Wie wollt Ihr die Replikation ersetzen? Irgendwas mit NFS oder gibt es da eine empfohlene Herangehensweise? Fragende Grüße Hanns -- Sig for hire.

4 4

2.4.0 missing fields in documentation for event smtp_server_transaction_finished
by Christian Rößner 01 Feb '25

01 Feb '25

Hi, Event: smtp_server_transaction_finished remote_ip and remote_port are also available Feature-request: Please also add user to the fields, as lmtp_proxy=yes does a userdb lookup and is aware of the user. Would make things much easier... Regards Christian Rößner -- Rößner-Network-Solutions Zertifizierter ITSiBe / CISO Marburger Str. 70a, 36304 Alsfeld Fax: +49 6631 78823409, Mobil: +49 171 9905345 USt-IdNr.: DE225643613, https://roessner.website PGP fingerprint: 658D 1342 B762 F484 2DDF 1E88 38A5 4346 D727 94E5

2 1

2.4.0 on MacOS
by John Muccigrosso 01 Feb '25

01 Feb '25

I got bitten by the upgrade from 2.3 to 2.4 on my MacOS laptop. I know enough to be dangerous here and had a set-up that was working fine, but now I can't quite figure out how to modify my conf file to work correctly. The current error I'm getting is that there is no group "dovecot" even though I've created one and made the default user a member: > Fatal: service(dict) Group doesn't exist: dovecot More broadly I find the documentation hard to follow, as others seem to have noted in the past. Any help appreciated. Jackie

2 3

Help understanding doveadm-sync stateful
by Bruno MATEU 01 Feb '25

01 Feb '25

Hello, With the removal of the replication plugin in next releases, i'm trying to setup automatic synchronization using doveadm sync between my master and my replica. I managed to make the full sync work from my replica using the following comamnd `sync -A -N -R -f tcp:master-fqdn:12345` However i'd like to do stateful synchronization to avoid doing the full replication each time, and i haven't managed to do that. I tried the following: `doveadm sync -A -N -R -s "$(</var/lib/dovecot/sync-state)" tcp:master-fqdn:12345 > /var/lib/dovecot/sync-state`. However this output a `doveadm(username): Error: Saved sync state is invalid, falling back to full sync: Invalid base64 data` for every user. After looking at the state in /var/lib/dovecot/sync-state, i noticed that there is one line for every user with an associated state, plus one line with only a state. For some reason, the state is the same for every user, except one where the state is much longer. Do i need to run the command for each user, and give as an argument the corresponding state ? Is it enough to run the doveadm sync command with -A argument if i give the state in the last line outputted by doveadm sync ? Is this the right way to do it ? Best regards, Bruno MATEU

1 1

Different Replicator Crash??
by Kent Borg 31 Jan '25

31 Jan '25

I added the suggested "count--" and recompiled, and it now appears to work, the replication goes both ways, eyeballing the e-mail on the two servers they look the same. Good. But part of the time I am still getting what looks very much like the same crash, is there another "count--" I need to add to that file? > Jan 28 10:53:08 la dovecot[96175]: replicator: Panic: data stack: Out > of memory when allocating 268435496 bytes > Jan 28 10:53:08 la dovecot[96175]: replicator: Error: Raw backtrace: > #0 t_askpass[0x7fb404b0c0] -> #1 backtrace_append[0x7fb404b374] -> #2 > backtr\ > ace_get[0x7fb404b510] -> #3 execvp_const[0x7fb4057ba4] -> #4 > i_syslog_fatal_handler[0x7fb4058510] -> #5 i_panic[0x7fb3fa6808] -> #6 > t_pop_pass_st\ > r[0x7fb4050eb4] -> #7 connection_deinit[0x7fb4056430] -> #8 > pool_datastack_create[0x7fb407e7b0] -> #9 > array_bsearch_insert_pos_i[0x7fb40446d0] ->\ > #10 t_base64_scheme_decode[0x7fb404cb24] -> #11 > buffer_append[0x7fb404d3d0] -> #12 replicator_queue_push[0x558c4b4f40] > -> #13 replicator_brain_i\ > nit[0x558c4b53e4] -> #14 _start[0x558c4b3d70] -> #15 > _start[0x558c4b3f04] -> #16 io_loop_call_io[0x7fb4072f20] -> #17 > io_loop_handler_run_interna\ > l[0x7fb4075450] -> #18 io_loop_handler_run[0x7fb4075680] -> #19 > io_loop_run[0x7fb4075944] -> #20 master_service_run[0x7fb3fd84b0] -> > #21 main[0x5\ > 58c4b3930] -> #22 __libc_init_first[0x7fb3dd76d0] -> #23 > __libc_start_main[0x7fb3dd7780] -> #24 _start[0x558c4b3b80] > Jan 28 10:53:08 la dovecot[96175]: replicator: Fatal: master: > service(replicator): child 96714 killed with signal 6 (core dumps > disabled - https:\ > //dovecot.org/bugreport.html#coredumps) Thanks, -kb

1 0

Replication Question: Who Initiates?
by Kent Borg 31 Jan '25

31 Jan '25

As I understand it, if I am doing replication between a pair of servers, I run the replicator on just one and it makes all the two way changes necessary to make the two machines match. But this only happens when some change happens on that machine that runs the replicator. First, is this correct? If so, is there a way to have that other machine give the first machine poke, to get it to run a replication? Thanks, -kb

2 1

Writing to a Dictionary Using Lua
by Andrea Gabellini 31 Jan '25

31 Jan '25

Hello, I am using the push_notification_lua plugin and would like to modify my script to write data into a dictionary configured as follows: dict_server { dict proxy { idle_timeout = 300s name = sql slow_warn = 1s } dict sql { sql_driver = mysql mysql localhost { dbname = xxx password = xxx user = xxx } dict_map "shared/dsync/$user" { sql_table = dsync dict_map_value_field last_event { type = uint } dict_map_key_field userid { value = $user } } } } I am not very familiar with Lua, and I couldn't find any relevant examples in the documentation or online. Would it be possible to provide a sample code snippet demonstrating how to write to this dictionary using Lua? Thank you! -- TIM San Marino S.p.A. Andrea Gabellini Engineering R&D TIM San Marino S.p.A. - https://www.telecomitalia.sm Via Ventotto Luglio, 212 - Piano -2 47893 - Borgo Maggiore - Republic of San Marino Tel: (+378) 0549 886237 Fax: (+378) 0549 886188 -- Informativa Privacy Questa email ha per destinatari dei contatti presenti negli archivi di TIM San Marino S.p.A.. Tutte le informazioni vengono trattate e tutelate nel rispetto della normativa vigente sulla protezione dei dati personali (Reg. EU 2016/679). Per richiedere informazioni e/o variazioni e/o la cancellazione dei vostri dati presenti nei nostri archivi potete inviare una email a privacy(a)telecomitalia.sm. Avviso di Riservatezza Il contenuto di questa e-mail e degli eventuali allegati e' strettamente confidenziale e destinato alla/e persona/e a cui e' indirizzato. Se avete ricevuto per errore questa e-mail, vi preghiamo di segnalarcelo immediatamente e di cancellarla dal vostro computer. E' fatto divieto di copiare e divulgare il contenuto di questa e-mail. Ogni utilizzo abusivo delle informazioni qui contenute da parte di persone terze o comunque non indicate nella presente e-mail potra' essere perseguito ai sensi di legge.

2 3