- dovecot - dovecot.org

Missing IMAP folders after upgrading from 2.3.17 to 2.3.21.1
by Frank Kirschner 23 Oct '24

23 Oct '24

Hi, I have upgraded from 2.3.17 to 2.3.21.1. mail_location = maildir Now in some mailboxes are IMAP folder in subdir of INBOX missing. When I do a 'ls -la' in filesystem, I see the subfolder like other folder with identical owner an permissions. But when I do doveadm mailbox list -u [user] the IMAP folder is not displayed. So I have deleted all files in /opt/dovecot/[hash]/[user] because I use INDEX=/opt/dovecot/%2.256Nu/%u:ITERINDEX The restart dovecot and re-login the Imap client. All IMAP folders are rebuilt but the same folders like before are still missing. Any ideas please? thanks, Frank doveconf -n # 2.3.21.1 (d492236fa0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.21.1 (49005e73) # OS: Linux 5.15.149-1-pve x86_64 AlmaLinux release 8.10 (Cerulean Leopard) # Hostname: *** auth_verbose = yes disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it first_valid_uid = 1000 imap_id_send = lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_save_to_detail_mailbox = yes log_path = /var/log/dovecot.log login_greeting = IMAP ready. mail_fsync = always mail_full_filesystem_access = yes mail_gid = 1000 mail_home = /srv/mail/mail_storage2/%d/%n mail_location = maildir:~/mail:VOLATILEDIR=/dev/shm/dovecot/%2.256Nu/%u:INDEX=/opt/dovecot/%2.256Nu/%u:ITERINDEX mail_plugins = " notify replication" mail_privileged_group = vpostfix mail_uid = 1000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { recipient_delimiter = + replication_full_sync_interval = 1 hour sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /etc/dovecot/spam-global.sieve sieve_extensions = +vacation-seconds sieve_vacation_default_period = 1d sieve_vacation_max_period = 30d sieve_vacation_min_period = 0 sieve_vacation_use_original_recipient = yes } postmaster_address = ***** protocols = imap pop3 lmtp sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vpostfix } unix_listener replication-notify { mode = 0666 user = vpostfix } } service auth { inet_listener { address = 192.168.130.192 port = 3659 } unix_listener /var/spool/postfix/private/auth { group = vpostfix mode = 0666 user = vpostfix } unix_listener auth-userdb { group = vpostfix mode = 0600 user = vpostfix } } service config { unix_listener config { user = vpostfix } } service doveadm { inet_listener { port = 4711 } user = vpostfix } service imap-login { process_min_avail = 1 service_count = 1 } service lmtp { inet_listener lmtp { address = 192.168.130.192 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 user = vpostfix } } ssl_alt_cert = </etc/alt/fullchain.pem ssl_alt_key = # hidden, use -P to show it ssl_cert = </etc/fullchain.pem ssl_cipher_list = PROFILE=SYSTEM ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lmtp { mail_plugins = " notify replication sieve" } protocol lda { mail_plugins = " notify replication sieve" } protocol imap { mail_max_userip_connections = 10 mail_plugins = " notify replication quota imap_quota" }

3 5

disable sieve header
by Marc 22 Oct '24

22 Oct '24

Is it possible to disable emitting the sieve header? X-Sieve: Pigeonhole Sieve

1 0

Uppercase username emails are rejected
by lua8ds＠proton.me 20 Oct '24

20 Oct '24

When a sender writes my email address with my username uppercase, e.g. USERNAME(a)name.com, in the to: field of their MUA, my mail server rejects that email. /var/log/mail.log prints: <USERNAME(a)redacted.com>: host mail.redacted.com[private/dovecot-lmtp] said: 550 5.1.1 > <USERNAME(a)redacted.com> User doesn't exist: USERNAME(a)redacted.com (in reply to RCPT TO > command) In the manual, I found that: auth_username_format Default: %Lu Formatting applied to username before querying the auth database. You can use the standard variables here. Examples: %Lu Lowercases the username [...] My current setting was not the default; I had: auth_username_format = %n So I updated that setting to: auth_username_format = %Lu expecting that %Lu would lowercase the username - and that my problem would be fixed. It does not though. Updating this setting not only prompts my email server to still reject USERNAME(a)redacted.com, but now also rejects (lowercase username) username(a)redacted.com (which my server had no issue receiving beforehand). So I reverted auth_username_format to: auth_username_format = %n Emails sent to username(a)redacted.com land in my mailbox again - but emails sent to USERNAME(a)redacted.com still don't. What can I try to troubleshoot? Which config file shall I share with you?

10 22

vsz_limit status
by Joan Moreau 18 Oct '24

18 Oct '24

Hi How to get (programatically, c++) the current usage of memory, under the limit of vsz_limit ? https://doc.dovecot.org/2.3/configuration_manual/service_configuration/#ser… Thank you

2 15

replication Error with wrong file lock
by Frank Kirschner 18 Oct '24

18 Oct '24

Hi there, I have setup a replication between two dovecot servers. It seems working: # doveadm replicator status '*' && doveadm replicator dsync-status && doveadm replicator status username priority fast sync full sync success sync failed ******************** none 00:59:22 18:50:17 18:21:25 - ********************low 01:54:26 18:57:58 - y ********************none 21:42:55 21:47:55 21:36:24 - ******************** none 21:14:57 21:51:07 21:33:13 - ********************none 21:13:52 21:51:04 21:17:57 - ********************none 00:04:23 01:29:24 - y ********************none 02:34:15 21:56:23 02:34:15 - ******************** none 00:03:56 18:55:17 - y ********************none 01:18:42 01:18:42 19:18:46 - ********************none 21:20:04 21:24:46 21:43:49 - ********************none 00:05:13 00:09:36 21:47:55 - ********************none 00:16:50 18:55:17 18:45:57 - ********************none 00:04:23 18:55:17 38:07:24 y username type status - Not connected - Not connected - Not connected ********************normal Waiting for dsync to finish - Not connected Queued 'sync' requests 0 Queued 'high' requests 0 Queued 'low' requests 1 Queued 'failed' requests 0 Queued 'full resync' requests 0 Waiting 'failed' requests 3 Total number of known users 13 But on the one server, I found this errors repeating and this users are at the list above marked as failed: Oct 18 09:41:08 dsync-local(1(a)tld.de): Error: Couldn't lock /srv/mail/mail_storage/tld.de/1/.dovecot-sync.lock: fcntl(/srv/mail/mail_storage/tld.de/1/.dovecot-sync.lock, write-lock, F_SETLKW) locking failed: Timed out after 30 seconds (WRITE lock held by pid 71613) The directory shows: # ls -la /srv/mail/mail_storage/celebrate.de/1/ | grep '\.dove' drwxrwxrwx 5 vpostfix vpostfix 4096 Oct 18 08:55 .dovecot -rw------- 2 vpostfix vpostfix 0 Oct 18 09:37 .dovecot-sync.lock -rwxrwxrwx 1 vpostfix vpostfix 912 Mar 20 2023 .dovecot.sieve.log -rwxrwxrwx 1 vpostfix vpostfix 275 Oct 11 00:00 .dovecot.svbin So it seems, that there are more processes want accessing same locking file. But normally this and other files should not exist there: # ls -la /srv/mail/mail_storage/celebrate.de/fk/ | grep '\.index' -rw-rw-rw- 1 vpostfix vpostfix 194008 Oct 17 12:26 dovecot.index -rwxrwxrwx 1 vpostfix vpostfix 16890796 Oct 17 14:20 dovecot.index.cache -rw-rw-rw- 1 vpostfix vpostfix 7420 Oct 17 14:04 dovecot.index.log -rwxrwxrwx 1 vpostfix vpostfix 32824 Oct 17 11:46 dovecot.index.log.2 -rwxrwxrwx 1 vpostfix vpostfix 5752 Oct 9 05:14 dovecot.list.index -rwxrwxrwx 1 vpostfix vpostfix 7556 Oct 9 09:32 dovecot.list.index.log In my config, I have set: mail_location = maildir:~:VOLATILEDIR=/dev/shm/dovecot/%2.256Nu/%u:INDEX=/opt/dovecot/%2.256Nu/%u:ITERINDEX But below /dev/shm/dovecot and /opt/dovecot, directories and files are generated too Following my config and hoping for some useful hints: # dovecot -n # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24 (124e06aa) # OS: Linux 5.15.149-1-pve x86_64 CentOS Linux release 7.9.2009 (Core) # Hostname: host.tld.de disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it first_valid_uid = 1000 imap_id_send = lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_save_to_detail_mailbox = yes log_path = /var/log/dovecot.log mail_fsync = always mail_full_filesystem_access = yes mail_gid = 1000 mail_home = /srv/mail/mail_storage/%d/%n mail_location = maildir:~:VOLATILEDIR=/dev/shm/dovecot/%2.256Nu/%u:INDEX=/opt/dovecot/%2.256Nu/%u:ITERINDEX mail_plugins = " notify replication" mail_privileged_group = vpostfix mail_uid = 1000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { mail_replica = tcp:192.168.130.192:4711 recipient_delimiter = + replication_full_sync_interval = 1 hour replication_sync_timeout = 10 sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /etc/dovecot/spam-global.sieve sieve_extensions = +vacation-seconds sieve_vacation_default_period = 1d sieve_vacation_max_period = 30d sieve_vacation_min_period = 0 sieve_vacation_use_original_recipient = yes } protocols = imap pop3 lmtp sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vpostfix } unix_listener replication-notify { mode = 0666 user = vpostfix } } service auth { inet_listener { address = 192.168.130.191 port = 3659 } unix_listener /var/spool/postfix/private/auth { group = vpostfix mode = 0666 user = vpostfix } unix_listener auth-userdb { group = vpostfix mode = 0600 user = vpostfix } } service config { unix_listener config { user = vpostfix } } service doveadm { inet_listener { port = 4711 } user = vpostfix } service imap-login { process_min_avail = 1 service_count = 1 } service lmtp { inet_listener lmtp { address = 192.168.130.191 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 user = vpostfix } } ssl_cert = </opt/fullchain.pem ssl_cipher_list = ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!AES256-SHA:AES:CAMELLIA:!CAMELLIA128-SHA:!DHE-RSA-CAMELLIA128-SHA:!CAMELLIA256-SHA:!DHE-RSA-CAMELLIA256-SHA ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lmtp { mail_plugins = " notify replication sieve" } protocol lda { mail_plugins = " notify replication sieve" } protocol imap { mail_max_userip_connections = 10 mail_plugins = " notify replication quota imap_quota" }

2 2

Sieve script error when quota is full
by Kosti 18 Oct '24

18 Oct '24

Hello. My sieve script gives an error; "failed with unsuccessful implicit keep". Here is maillog file output; (Yes my user does has quota exceeded, but is this the issue for the sieve script and is it normal for the script give such output?) postfix dovecot: lda(me(a)mydomain.com)<50709><iEd4O9YLDGcVxgAA/dJgOw>: sieve: msgid=<x(a)jol.mysite.com>: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) postfix dovecot: lda(me(a)mydomain.com)<50709><iEd4O9YLDGcVxgAA/dJgOw>: sieve: Execution of script /etc/dovecot/sieve/default.sieve failed with unsuccessful implicit keep postfix dovecot: lda(me(a)mydomain.com)<50709><iEd4O9YLDGcVxgAA/dJgOw>: msgid=<x(a)jol.mysite.com>: rejected: Quota exceeded (mailbox for user is full) And /etc/dovecot/sieve/default.sieve file contains: require ["fileinto"]; if header :contains "X-Spam" "Yes" { fileinto "Junk"; stop; } Any idea would be much appreciated. Best regards.

1 0

Piegeonhole and PATH variable
by Julien Salort 18 Oct '24

18 Oct '24

Hello, I have a dovecot-sieve rule that has been working fine for years which calls sa-learn whenever a message is moved from or to the Junk folder. Recently, but I don't know when it started exactly, this no longer works. I get the following error message in mail.log: dovecot: imap: Error: Use of uninitialized value $ENV{"PATH"} in split at /usr/lib/x86_64-linux-gnu/perl-base/File/Spec/Unix.pm line 136. Error: program exec:/usr/lib/dovecot/sieve-pipe/sa-learn-spam.sh (2526990): Forcibly terminated with signal 15 Error: sieve: failed to pipe message to program `sa-learn-spam.sh': refer to server log for more information. [2024-10-16 10:16:51] Error: sieve: Execution of script /usr/lib/dovecot/sieve-pipe/report-spam.sieve failed The content of report-spam.sieve is require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"]; if environment :matches "imap.user" "*" { set "username" "${1}"; } pipe :copy "sa-learn-spam.sh" [ "${username}" ]; So my guess is that one of the require on the first line triggers this Unix.pm, and it fails because somehow the PATH variable is not set in this context. However, I am at a complete loss to determine how to change the environment variable in the context of dovecot-sieve. And also, I am not sure why this would be required. Does somebody know how to fix this problem, or alternatively point me to the relevant part in the documentation? Thanks, Julien

2 2

log: msgid= followed by space
by A. Schulze 18 Oct '24

18 Oct '24

Hello, I noted today unexpected spaces after 'msgid=' in the logs, dovecot-2.3.21.1 produce. First time a message from outlook was under inspection but I found also other sources. there was one (unanswered) question in the list archive, that may be related: https://dovecot.org/mailman3/archives/list/dovecot@dovecot.org/thread/JU3S2… here are some samples: lmtp(user@example)<22995><RtRIK/YvEWfTWQAAf9UaAw>: Info: save: box=INBOX/ietf/folder1/, uid=14143, msgid= <172917948321.1404061.2617721779338535905@dt-datatracker-78dc5ccf94-w8wgc>, size=7904 lmtp(user2(a)example.org)<23156><0cjVE809EWd0WgAAf9UaAw>: Info: save: box=INBOX/foo, uid=2632, msgid= <AM0PR06MB4100D06FAD74B7B33BFFDAA395472(a)AM0PR06MB4100.eurprd06.prod.outlook...., size=16755 But there are also "good" logs without the space for messages to other folders but same source (IETF in this case) lmtp(user@example)<23143><bhcjK288EWdnWgAAf9UaAw>: Info: save: box=INBOX/ietf/folder2/, uid=567, msgid=<62424081-d68e-4334-b9a2-a62d0759cea3(a)gulbrandsen.priv.no>, size=11394 I've no older logs to check if that happened also before updating to 2.3.21.1 Andreas

2 2

replicator: Panic: data stack: Out of memory when allocating 275GB
by Sergej Pupykin 15 Oct '24

15 Oct '24

Hello, I found similar reports in this ML with values less than mine but was not able to find the solution. I checked size of all dovecot.* files in /var/mail and it is less than 10Gb. Whole /var/mail dir is 1.8Tb. With vsz_limit it crashes much earlier, this log with 275GB produced by setting vsz_limit = 0 Server has 192GB of RAM, can it be some bug? Oct 14 15:51:59 mailserver dovecot[826177]: replicator: Panic: data stack: Out of memory when allocating 274877906984 bytes Oct 14 15:51:59 mailserver dovecot[826177]: replicator: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x48) [0x76ac1505e188] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x25) [0x76ac1505e305] -> /usr/lib/dovecot/libdovecot.so.0(+0x106312) [0x76ac1506c312] -> /usr/lib/dovecot/libdovecot.so.0(+0x1063b5) [0x76ac1506c3b5] -> /usr/lib/dovecot/libdovecot.so.0(+0x4c571) [0x76ac14fb2571] -> /usr/lib/dovecot/libdovecot.so.0(+0x4bbc4) [0x76ac14fb1bc4] -> /usr/lib/dovecot/libdovecot.so.0(+0xfee30) [0x76ac15064e30] -> /usr/lib/dovecot/libdovecot.so.0(+0x1288c1) [0x76ac1508e8c1] -> /usr/lib/dovecot/libdovecot.so.0(+0xfa870) [0x76ac15060870] -> /usr/lib/dovecot/libdovecot.so.0(+0xfa8f1) [0x76ac150608f1] -> /usr/lib/dovecot/libdovecot.so.0(buffer_append+0x60) [0x76ac15060c00] -> dovecot/replicator(replicator_queue_push+0x163) [0x58a11fac0a53] -> dovecot/replicator(+0x4e9d) [0x58a11fabfe9d] -> dovecot/replicator(+0x4ed8) [0x58a11fabfed8] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0x15f) [0x76ac1508381f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd6) [0x76ac15085296] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x58) [0x76ac15083ae8] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x49) [0x76ac15083ce9] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x20) [0x76ac14fed6a0] -> dovecot/replicator(main+0x1a5) [0x58a11fabe865] -> /usr/lib/libc.so.6(+0x25e08) [0x76ac14d90e08] -> /usr/lib/libc.so.6(__libc_start_main+0x8c) [0x76ac14d90ecc] -> dovecot/replicator(_start+0x25) [0x58a11fabe925]

3 3

How to deactivate "!include conf.d/auth-system.conf.ext" without touching "/etc/dovecot/conf.d/10-auth.conf"?
by postfix_dovecot＠gmx.de 14 Oct '24

14 Oct '24

Hello, I want to deactivate the auth-system.conf.ext because it slows down the system and makes no sense because I utilize passwdfile.conf.ext. I have put all my configurations into the local.conf file, and all the original configuration files remain untouched. I want to stay on this path because that way I know where I made changes, and updates won't cause me any hassle. But how can I deactivate an option that was activated in the original configuration files?" Jens

3 8