- dovecot - dovecot.org

Dovecot autoexpunge lock errors - Couldn't create dovecot.autoexpunge.lock
by mihai.aurel88＠gmail.com 03 Feb '25

03 Feb '25

Hello, We are seeing lots of errors like this in the logs: "dovecot: imap(user@domain)<31219><Dly3bTyt4tesENjH>: Error: autoexpunge: Couldn't create dovecot.autoexpunge.lock lock: file_create_locked(/home/virtual/domain/user/dovecot.autoexpunge.lock) failed: fstat(/home/virtual/domain/user/dovecot.autoexpunge.lock) failed: Stale file handle" We don't understand why Dovecot is trying to create the dovecot.autoexpunge.lock file in the mail home directory instead of mail location directory. For some accounts, the dovecot.autoexpunge.lock file is correctly created inside the mail home directory. Any ideas on what could be causing this behavior? # doveadm user user@domain field value uid 150 gid 8 home /home/virtual/domain/user mail maildir:~/Maildir quota_rule *:bytes=8192000000 # doveconf -n # 2.3.21.1 # Pigeonhole version 0.5.21.1 # OS: Linux 6.1.0-29-amd64 x86_64 Debian 12.9 auth_cache_verify_password_with_worker = yes auth_mechanisms = plain login auth_verbose = yes auth_worker_max_count = 256 default_client_limit = 10000 default_process_limit = 10000 default_vsz_limit = 1 G dict { lastlogin = mysql:/etc/dovecot/mysql/dovecot-dict-lastlogin.conf quotadict = mysql:/etc/dovecot/mysql/dovecot-dict-quota.conf } disable_plaintext_auth = no first_valid_uid = 100 imap_hibernate_timeout = 30 secs log_timestamp = "%Y-%m-%d %H:%M:%S " login_log_format_elements = user=<%u> method=%m rip=%r lip=%l pid=%p %c login_trusted_networks = mail_fsync = always mail_gid = 8 mail_home = /home/virtual/%d/%n mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = zlib quota mail_log notify mail_privileged_group = mail mail_uid = 150 maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave metric imap_command { filter = event=imap_command_finished AND tagged_reply_state=OK group_by = cmd_name } metric imap_select_no { filter = event=imap_command_finished AND cmd_name=SELECT AND tagged_reply_state=NO } metric imap_select_no_notfound { filter = event=imap_command_finished AND cmd_name=SELECT AND tagged_reply="NO*Mailbox doesn't exist:*" } metric storage_http_gets { filter = event=http_request_finished AND category=storage AND method=get } mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe autoexpunge = 30 days special_use = \Junk } mailbox Trash { auto = subscribe autoexpunge = 60 days special_use = \Trash } mailbox virtual/All { special_use = \All } prefix = } passdb { args = /etc/dovecot/mysql/dovecot-sql.conf driver = sql } plugin { last_login_dict = proxy::lastlogin last_login_key = # hidden, use -P to show it mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = dict:user::proxy::quotadict quota_rule2 = Trash:ignore quota_rule3 = Spam:ignore quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 The email account that you tried to reach is over quota quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve sieve_vacation_send_from_recipient = yes trash = /etc/dovecot/dovecot-trash.conf zlib_save = gz zlib_save_level = 6 } protocols = imap sieve pop3 lmtp service anvil { client_limit = 50000 unix_listener anvil-auth-penalty { mode = 00 } } service auth { client_limit = 50000 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = mail mode = 0660 user = vmail } user = root vsz_limit = 1 G } service dict { unix_listener dict { mode = 0660 user = vmail } } service imap-hibernate { unix_listener imap-hibernate { group = $default_internal_group mode = 0660 user = vmail } } service imap-login { inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_min_avail = 1 service_count = 0 } service imap { extra_groups = $default_internal_group process_min_avail = 16 unix_listener imap-master { user = $default_internal_user } vsz_limit = 2 G } service lmtp { process_min_avail = 16 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } vsz_limit = 1 G } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 16 service_count = 1 } service managesieve { vsz_limit = 1 G } service pop3-login { inet_listener pop3 { address = * port = 110 } inet_listener pop3s { address = * port = 995 } process_min_avail = 16 service_count = 1 } service pop3 { process_min_avail = 16 vsz_limit = 1 G } service quota-status { executable = /usr/lib/dovecot/quota-status -p postfix process_min_avail = 16 unix_listener /var/spool/postfix/private/quota-status { group = postfix mode = 0666 user = postfix } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0660 user = vmail } user = vmail } service stats { unix_listener stats-reader { mode = 0660 user = vmail } unix_listener stats-writer { mode = 0666 user = vmail } } ssl_cert = </etc/dovecot/ssl/cert.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1 userdb { args = /etc/dovecot/mysql/dovecot-sql.conf driver = sql } verbose_proctitle = yes protocol imap { imap_max_line_length = 64 k mail_max_userip_connections = 16 mail_plugins = zlib quota mail_log notify trash imap_quota } protocol pop3 { mail_max_userip_connections = 10 mail_plugins = zlib quota mail_log notify pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_fast_size_lookups = yes pop3_no_flag_updates = yes pop3_uidl_format = %08Xu%08Xv } protocol sieve { mail_max_userip_connections = 10 managesieve_max_compile_errors = 5 } protocol lda { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = zlib quota mail_log notify sieve sendmail_path = /usr/lib/sendmail syslog_facility = mail } protocol lmtp { mail_plugins = zlib quota mail_log notify sieve syslog_facility = mail }

1 0

Clarification Question about migrating_mailboxes and broken doc link
by Aleks 02 Feb '25

02 Feb '25

Hi. I have read the https://doc.dovecot.org/2.3/admin_manual/migrating_mailboxes/#id7 and there is one part which is not 100% clear to me. We have 2 Dovecot servers mail1 (2.2.36.4) and mail2 ( 2.3.21 ). Now I have understood that I need to add this to mail1 ``` service doveadm { inet_listener { port = 12354 } } doveadm_password = supersecret ``` and this on mail2 ``` doveadm_password = supersecret ``` The question is for me now: I think that this command `doveadm backup -Ru username tcp:host:port` should be executed on the mail2. Is this assumption right? What I also found is that on that Page https://doc.dovecot.org/2.3/admin_manual/doveadm_mailbox_commands/ the wiki is linked for the manpages but the wiki link shows this. ``` The old wiki has been closed Our primary documentation is now at https://doc.dovecot.org, please update your links. ``` What's the right link for the manpages in the web? Best regards Aleks

2 2

[Off-topic] ANN: Nauthilus
by Christian Rößner 01 Feb '25

01 Feb '25

Today I am pleased to announce the Nauthilus software. Nauthilus (N-Auth-ilus) is a centralized authentication server with a comprehensive policy engine. The idea behind this project is to connect services centrally in one place and perform extensive tests during authentication. # What can Nauthilus do and what problems does it solve? Many services on the Internet require authentication. Each application must then be connected to databases and must implement its own protective measures to ward off attackers. Every installation carries the risk of compromisation and therefore also access to the databases. With Nauthilus, the task is centralized in one place. Nauthilus is essentially an HTTP REST server that can be accessed by any application. It takes on the role of a guardian. Nauthilus integrates very well with Dovecot and Postfix. # Authentication process Nauthilus uses several authentication steps: ## 1. features Features include tests such as TLS verification, relay domains (is the system even responsible for the requested domain?), blocklists, RBLs and freely definable Lua features. Nauthilus has a powerful brute force concept with buckets to detect even slow attacks over days and weeks. It offers a bucket system for this purpose. ## 2. backend authentication Nauthilus includes LDAP support and Lua to perform the authentication itself. A large library of predefined functions is available in Lua, including SQL support. ## 3 Policies Policies are run through after authentication. Despite a successful login, the system can reject the login (or, conversely, allow it!). There is also space here for GeoIP lookups, etc. ## 4. Post-processing After the 3rd point, authentication has been completed, but at this point further tests can run in the background such as: * Check password policy and take action * Consult the Haveibeenpwnd network * GeoIP tracking across national borders and much more. ## Miscellaneous Nauthilus allows the free definition of so-called hooks. Each hook listens for a specific URI (callback) in the HTTP request. These callbacks are written by the administrator in Lua. In an initial proof of concept, Nauthilus can take on the role of a Dovecot director. This has already been tested with version 2.4.0. Currently, Nauthilus can dynamically delegate incoming connections to backends. The hooks concept is used here as an example. # Final words By integrating a Lua VM into the server, Nauthilus can be integrated and customized in almost any setup. See also the other Nauthilus-related projects listed in the appendix. To enable single sign-on (SSO), it can be operated with an Ory-Hydra server or the sister project nauthilus-keycloak can be used as a custom authenticator in Keycloak. 100% Open-Source 100% Community # Project https://github.com/croessner/nauthilus # Sub projects https://github.com/croessner/nauthilus-demo https://github.com/croessner/nauthilus-keycloak https://github.com/croessner/pfxhttp https://github.com/croessner/geoip-policyd # Mailing lists: https://lists.nauthilus.org N.B.: In the future, announcements are sent over the nauthilus-announce ML. This is just a hello world! Christian Rößner -- Rößner-Network-Solutions Zertifizierter ITSiBe / CISO Marburger Str. 70a, 36304 Alsfeld Fax: +49 6631 78823409, Mobil: +49 171 9905345 USt-IdNr.: DE225643613, https://roessner.website PGP fingerprint: 658D 1342 B762 F484 2DDF 1E88 38A5 4346 D727 94E5

2 2

Replication ade
by Hanns Mattes 01 Feb '25

01 Feb '25

Hei, mit 2.4 verschwindet ja, wenn ich das richtig verstanden habe, die Replication aus dem Werkzeugkasten von Dovecot. Obwohl ich nur ein recht kleines System mit etwa 500 Nutzern betreibe fand ich die Replication als ziemlich beruhigend :-) Meine Frage in die Runde: Wie wollt Ihr die Replikation ersetzen? Irgendwas mit NFS oder gibt es da eine empfohlene Herangehensweise? Fragende Grüße Hanns -- Sig for hire.

4 4

Re: 2.4.0 on MacOS
by John Muccigrosso 01 Feb '25

01 Feb '25

(Looks like I forgot to reply to the list last time.) With `sudo dovecot -F`, I'm now getting: Fatal: setrlimit(RLIMIT_DATA, 2147483648): Invalid argument The old(?) solution doesn't work for me: default_vsz_limit = 0 It reports: doveconf: Fatal: Error in configuration file /opt/homebrew/etc/dovecot/dovecot.conf: service(log): vsz_limit is too low > On Feb 1, 2025, at 08:55, John Muccigrosso <muccigrosso(a)icloud.com> wrote: > > On Feb 1, 2025, at 07:31, Aki Tuomi via dovecot <dovecot(a)dovecot.org> wrote: > >> You sure the default internal group isnt _dovecot as well? > > I'm not sure what you're asking. Do I not have the internal group set via this line?: > > default_internal_group = mail > > I think I may have to dig into group memberships, if this is the issue. I'm getting conflicting result when checking. That is, when I check the group "mail" for members, _dovecot isn't in it, even though checking which groups _dovecot belongs to includes "mail". The opposite happens for the "dovecot" group. And dovecot itself can't find the _dovecot or dovecot group. Seems to be a macOS thing with group membership. > >> You are missing >> >> mail_home = /Users/%{user} > > Thanks. As I said, I was trying to follow the documentation where they give the two lines under Maildir, but not this third one (even though it's given in the first part of this section).

3 4

2.4.0 missing fields in documentation for event smtp_server_transaction_finished
by Christian Rößner 01 Feb '25

01 Feb '25

Hi, Event: smtp_server_transaction_finished remote_ip and remote_port are also available Feature-request: Please also add user to the fields, as lmtp_proxy=yes does a userdb lookup and is aware of the user. Would make things much easier... Regards Christian Rößner -- Rößner-Network-Solutions Zertifizierter ITSiBe / CISO Marburger Str. 70a, 36304 Alsfeld Fax: +49 6631 78823409, Mobil: +49 171 9905345 USt-IdNr.: DE225643613, https://roessner.website PGP fingerprint: 658D 1342 B762 F484 2DDF 1E88 38A5 4346 D727 94E5

2 1

2.4.0 on MacOS
by John Muccigrosso 01 Feb '25

01 Feb '25

I got bitten by the upgrade from 2.3 to 2.4 on my MacOS laptop. I know enough to be dangerous here and had a set-up that was working fine, but now I can't quite figure out how to modify my conf file to work correctly. The current error I'm getting is that there is no group "dovecot" even though I've created one and made the default user a member: > Fatal: service(dict) Group doesn't exist: dovecot More broadly I find the documentation hard to follow, as others seem to have noted in the past. Any help appreciated. Jackie

2 3

Help understanding doveadm-sync stateful
by Bruno MATEU 01 Feb '25

01 Feb '25

Hello, With the removal of the replication plugin in next releases, i'm trying to setup automatic synchronization using doveadm sync between my master and my replica. I managed to make the full sync work from my replica using the following comamnd `sync -A -N -R -f tcp:master-fqdn:12345` However i'd like to do stateful synchronization to avoid doing the full replication each time, and i haven't managed to do that. I tried the following: `doveadm sync -A -N -R -s "$(</var/lib/dovecot/sync-state)" tcp:master-fqdn:12345 > /var/lib/dovecot/sync-state`. However this output a `doveadm(username): Error: Saved sync state is invalid, falling back to full sync: Invalid base64 data` for every user. After looking at the state in /var/lib/dovecot/sync-state, i noticed that there is one line for every user with an associated state, plus one line with only a state. For some reason, the state is the same for every user, except one where the state is much longer. Do i need to run the command for each user, and give as an argument the corresponding state ? Is it enough to run the doveadm sync command with -A argument if i give the state in the last line outputted by doveadm sync ? Is this the right way to do it ? Best regards, Bruno MATEU

1 1

Different Replicator Crash??
by Kent Borg 31 Jan '25

31 Jan '25

I added the suggested "count--" and recompiled, and it now appears to work, the replication goes both ways, eyeballing the e-mail on the two servers they look the same. Good. But part of the time I am still getting what looks very much like the same crash, is there another "count--" I need to add to that file? > Jan 28 10:53:08 la dovecot[96175]: replicator: Panic: data stack: Out > of memory when allocating 268435496 bytes > Jan 28 10:53:08 la dovecot[96175]: replicator: Error: Raw backtrace: > #0 t_askpass[0x7fb404b0c0] -> #1 backtrace_append[0x7fb404b374] -> #2 > backtr\ > ace_get[0x7fb404b510] -> #3 execvp_const[0x7fb4057ba4] -> #4 > i_syslog_fatal_handler[0x7fb4058510] -> #5 i_panic[0x7fb3fa6808] -> #6 > t_pop_pass_st\ > r[0x7fb4050eb4] -> #7 connection_deinit[0x7fb4056430] -> #8 > pool_datastack_create[0x7fb407e7b0] -> #9 > array_bsearch_insert_pos_i[0x7fb40446d0] ->\ > #10 t_base64_scheme_decode[0x7fb404cb24] -> #11 > buffer_append[0x7fb404d3d0] -> #12 replicator_queue_push[0x558c4b4f40] > -> #13 replicator_brain_i\ > nit[0x558c4b53e4] -> #14 _start[0x558c4b3d70] -> #15 > _start[0x558c4b3f04] -> #16 io_loop_call_io[0x7fb4072f20] -> #17 > io_loop_handler_run_interna\ > l[0x7fb4075450] -> #18 io_loop_handler_run[0x7fb4075680] -> #19 > io_loop_run[0x7fb4075944] -> #20 master_service_run[0x7fb3fd84b0] -> > #21 main[0x5\ > 58c4b3930] -> #22 __libc_init_first[0x7fb3dd76d0] -> #23 > __libc_start_main[0x7fb3dd7780] -> #24 _start[0x558c4b3b80] > Jan 28 10:53:08 la dovecot[96175]: replicator: Fatal: master: > service(replicator): child 96714 killed with signal 6 (core dumps > disabled - https:\ > //dovecot.org/bugreport.html#coredumps) Thanks, -kb

1 0

Replication Question: Who Initiates?
by Kent Borg 31 Jan '25

31 Jan '25

As I understand it, if I am doing replication between a pair of servers, I run the replicator on just one and it makes all the two way changes necessary to make the two machines match. But this only happens when some change happens on that machine that runs the replicator. First, is this correct? If so, is there a way to have that other machine give the first machine poke, to get it to run a replication? Thanks, -kb

2 1