February 2025 - dovecot

[Dovecot] auth: Error: net_connect_unix(anvil-auth-penalty) failed: Permission denied
by mailing＠securitylabs.it 14 Sep '25

14 Sep '25

Hello, just upgraded one of my testing machine from dovecot 1.2.15 to 2.0.7. After some trouble with permissions now all seems working, but when I stop dovecot I have these errors in log: Nov 9 17:08:07 in dovecot: master: Warning: Killed with signal 15 (by pid=20362 uid=0 code=kill) Nov 9 17:08:07 in dovecot: pop3-login: Error: read(anvil) failed: EOF Nov 9 17:08:07 in dovecot: imap-login: Error: read(anvil) failed: EOF Nov 9 17:08:07 in dovecot: imap(mailing(a)securitylabs.it): Server shutting down. bytes=410/59361 Nov 9 17:08:07 in dovecot: imap(mailing(a)securitylabs.it): Server shutting down. bytes=79/26605 Nov 9 17:08:07 in dovecot: imap(mailing(a)securitylabs.it): Server shutting down. bytes=802/55708 Nov 9 17:08:07 in dovecot: imap(mailing(a)securitylabs.it): Server shutting down. bytes=258/1242 Nov 9 17:08:07 in dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF Nov 9 17:08:07 in dovecot: auth: Error: net_connect_unix(anvil-auth-penalty) failed: Permission denied dovecot -n: # 2.0.7: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-xen-686 i686 Debian squeeze/sid auth_cache_negative_ttl = 2 mins auth_cache_size = 1000 M auth_cache_ttl = 2 mins auth_master_user_separator = * auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@& disable_plaintext_auth = no dotlock_use_excl = yes first_valid_uid = 100 last_valid_uid = 105 lock_method = dotlock log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = Server ready. mail_fsync = never mail_gid = Debian-exim mail_location = maildir:~/Maildir:INDEX=/var/indexes/%d/%n mail_privileged_group = Debian-exim mail_uid = Debian-exim managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date namespace { inbox = yes location = prefix = separator = / type = private } namespace { location = maildir:~/Maildir/expunged prefix = EXPUNGED/ separator = / type = private } namespace { location = maildir:~/Maildir/deleted prefix = DELETED/ separator = / type = private } namespace { location = maildir:~/Maildir/deleted/expunged prefix = DELETED/EXPUNGED/ separator = / type = private } passdb { args = /usr/local/etc/dovecot-sql.conf driver = sql } passdb { args = /usr/local/etc/passwd.masterusers driver = passwd-file master = yes pass = yes } plugin { lazy_expunge = EXPUNGED/ DELETED/ DELETED/EXPUNGED/ mail_log_events = delete expunge mail_log_group_events = quota = maildir quota_warning = storage=80%% /usr/local/bin/dovecot-quota-warning.sh 80 quota_warning2 = storage=90%% /usr/local/bin/dovecot-quota-warning.sh 90 sieve = ~/.dovecot.sieve } pop3_no_flag_updates = yes protocols = imap pop3 service auth { unix_listener auth-userdb { group = Debian-exim mode = 0600 user = Debian-exim } } service imap-login { client_limit = 256 process_limit = 128 process_min_avail = 3 service_count = 1 } service imap { drop_priv_before_exec = yes process_limit = 256 vsz_limit = 256 B } service pop3-login { client_limit = 256 process_limit = 128 process_min_avail = 3 service_count = 1 } service pop3 { drop_priv_before_exec = yes process_limit = 256 vsz_limit = 256 B } ssl_cert = </usr/local/etc/dovecot.crt ssl_key = </usr/local/etc/dovecot.key userdb { args = /usr/local/etc/dovecot-sql.conf driver = sql } protocol imap { mail_max_userip_connections = 10 mail_plugins = " notify quota imap_quota mail_log lazy_expunge" } protocol pop3 { mail_max_userip_connections = 3 mail_plugins = " notify quota mail_log" pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = quota sieve postmaster_address = postmaster(a)securitylabs.it } My /usr/local/var/run/dovecot directory: 4 drwxr-sr-x 4 root staff 4096 Nov 9 17:08 . 4 drwxrwsrwx 3 root staff 4096 Nov 9 17:05 .. 0 srw------- 1 root staff 0 Nov 9 17:05 anvil 0 srw------- 1 root staff 0 Nov 9 17:05 anvil-auth-penalty 0 srw------- 1 root staff 0 Nov 9 17:05 auth-client 0 srw------- 1 dovecot staff 0 Nov 9 17:05 auth-login 0 srw------- 1 root staff 0 Nov 9 17:05 auth-master 0 srw------- 1 Debian-exim Debian-exim 0 Nov 9 17:05 auth-userdb 0 srw------- 1 dovecot staff 0 Nov 9 17:05 auth-worker 0 srw------- 1 root staff 0 Nov 9 17:05 config 0 srw------- 1 root staff 0 Nov 9 17:05 dict 0 srw------- 1 root staff 0 Nov 9 17:05 director-admin 0 srw-rw-rw- 1 root staff 0 Nov 9 17:05 dns-client 0 srw------- 1 root staff 0 Nov 9 17:05 doveadm-server 0 lrwxrwxrwx 1 root staff 35 Nov 9 17:05 dovecot.conf -> /usr/local/etc/dovecot/dovecot.conf 4 drwxr-xr-x 2 root root 4096 Nov 9 17:05 empty 4 drwxr-x--- 2 root dovenull 4096 Nov 9 17:05 login My emails are delivered as user Debian-exim and users are virtual in a MySQL DB. This system is behind a proxy (so the master user)

4 3

Crash in dovecot snippet when using imapc
by John van der Kamp 03 Sep '25

03 Sep '25

Hello, I've found a crash in a very specific setup. A dovecot server with imapc connection needs to receive an email with no body contents for the intent of generating a preview/snippet. It crashes somewhere deep in the jungle of istream and snapshots. I've included a script which sets up the systems to reproduce the crash. I've tested this with several versions. 2.3.16 doesn't seem to be affected, but 2.3.20 and 2.3.21 are affect. For me it produces a traceback like this, using the ubuntu version from here: https://packages.ubuntu.com/noble/dovecot-core (gdb) bt #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=140530132887360) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=140530132887360) at ./nptl/pthread_kill.c:78 #2 __GI___pthread_kill (threadid=140530132887360, signo=signo@entry=6) at ./nptl/pthread_kill.c:89 #3 0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #4 0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79 #5 0x00007fcfb8b37fe5 in default_fatal_finish (status=0, type=LOG_TYPE_PANIC) at ../lib/failures.c:465 #6 fatal_handler_real (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:477 #7 0x00007fcfb8be50d7 in i_internal_fatal_handler (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:879 #8 0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file %s: line %d (%s): assertion failed: (%s)") at ../lib/failures.c:530 #9 0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free (_snapshot=<optimized out>) at ../lib-mail/istream-header-filter.c:663 #10 i_stream_header_filter_snapshot_free (_snapshot=0x55dabe297a60) at ../lib-mail/istream-header-filter.c:655 #11 0x00007fcfb8bf25ac in i_stream_snapshot_free (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:253 #12 0x00007fcfb8bf2654 in i_stream_unref (stream=0x7ffc16cc7fa0) at ../lib/istream.c:66 #13 0x00007fcfb8d96baa in index_mail_write_body_snippet (mail=0x55dabe292058) at index/index-mail.c:1151 #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure (mail=0x55dabe292058, field=MAIL_CACHE_BODY_SNIPPET) at index/index-mail.c:1551 #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-mail.c:1602 #16 index_mail_get_special (_mail=0x55dabe292058, field=<optimized out>, value_r=0x7ffc16cc8050) at index/index-mail.c:1730 #17 0x00007fcfb8d16ffe in mail_get_special (mail=mail@entry=0x55dabe292058, field=field@entry=MAIL_FETCH_BODY_SNIPPET, value_r=value_r@entry=0x7ffc16cc8050) at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-storage/mail.c:418 #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050, mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/imap-fetch-body.c:615 #19 0x000055dabc52b5cc in imap_fetch_more_int (ctx=0x55dabe26e050, cancel=false) at ./src/imap/imap-fetch.c:562 #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050, cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617 #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./src/imap/cmd-fetch.c:382 #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at ./src/imap/imap-commands.c:201 #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized out>) at ./src/imap/imap-client.c:1237 #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized out>) at ./src/imap/imap-client.c:1307 #25 0x000055dabc52eeed in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1349 #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1363 #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1407 #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660) at ../lib/ioloop.c:737 #29 0x00007fcfb8bff81a in io_loop_handler_run_internal (ioloop=0x55dabe243fd0) at ../lib/ioloop-epoll.c:222 #30 0x00007fcfb8bff8d4 in io_loop_handler_run (ioloop=0x55dabe243fd0) at ../lib/ioloop.c:789 #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0) at ../lib/ioloop.c:762 #32 0x00007fcfb8b6ce57 in master_service_run (service=0x55dabe243e20, callback=callback@entry=0x55dabc533210 <client_connected>) at ../lib-master/master-service.c:878 #33 0x000055dabc51ad37 in main (argc=<optimized out>, argv=<optimized out>) at ./src/imap/main.c:575 John Hello, I've found a crash in a very specific setup. A dovecot server with imapc connection needs to receive an email with no body contents for the intent of generating a preview/snippet. It crashes somewhere deep in the jungle of istream and snapshots. I've included a script which sets up the systems to reproduce the crash. I've tested this with several versions. 2.3.16 doesn't seem to be affected, but 2.3.20 and 2.3.21 are affect. For me it produces a traceback like this, using the ubuntu version from here: https://packages.ubuntu.com/noble/dovecot-core (gdb) bt #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=140530132887360) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=140530132887360) at ./nptl/ pthread_kill.c:78 #2 __GI___pthread_kill (threadid=140530132887360, signo=signo@entry=6) at ./ nptl/pthread_kill.c:89 #3 0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/ raise.c:26 #4 0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79 #5 0x00007fcfb8b37fe5 in default_fatal_finish (status=0, type=LOG_TYPE_PANIC) at ../lib/failures.c:465 #6 fatal_handler_real (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:477 #7 0x00007fcfb8be50d7 in i_internal_fatal_handler (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:879 #8 0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file %s: line %d (%s): assertion failed: (%s)") at ../lib/failures.c:530 #9 0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free (_snapshot=<optimized out>) at ../lib-mail/istream-header-filter.c:663 #10 i_stream_header_filter_snapshot_free (_snapshot=0x55dabe297a60) at ../lib- mail/istream-header-filter.c:655 #11 0x00007fcfb8bf25ac in i_stream_snapshot_free (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:253 #12 0x00007fcfb8bf2654 in i_stream_unref (stream=0x7ffc16cc7fa0) at ../lib/ istream.c:66 #13 0x00007fcfb8d96baa in index_mail_write_body_snippet (mail=0x55dabe292058) at index/index-mail.c:1151 #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure (mail=0x55dabe292058, field=MAIL_CACHE_BODY_SNIPPET) at index/index-mail.c:1551 #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-mail.c:1602 #16 index_mail_get_special (_mail=0x55dabe292058, field=<optimized out>, value_r=0x7ffc16cc8050) at index/index-mail.c:1730 #17 0x00007fcfb8d16ffe in mail_get_special (mail=mail@entry=0x55dabe292058, field=field@entry=MAIL_FETCH_BODY_SNIPPET, value_r=value_r@entry=0x7ffc16cc8050) at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-storage/mail.c:418 #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050, mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/imap-fetch-body.c: 615 #19 0x000055dabc52b5cc in imap_fetch_more_int (ctx=0x55dabe26e050, cancel=false) at ./src/imap/imap-fetch.c:562 #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050, cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617 #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./src/imap/cmd- fetch.c:382 #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at ./src/imap/imap- commands.c:201 #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized out>) at ./src/ imap/imap-client.c:1237 #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized out>) at ./src/ imap/imap-client.c:1307 #25 0x000055dabc52eeed in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1349 #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/imap-client.c: 1363 #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8) at ./src/imap/ imap-client.c:1407 #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660) at ../lib/ ioloop.c:737 #29 0x00007fcfb8bff81a in io_loop_handler_run_internal (ioloop=0x55dabe243fd0) at ../lib/ioloop-epoll.c:222 #30 0x00007fcfb8bff8d4 in io_loop_handler_run (ioloop=0x55dabe243fd0) at ../ lib/ioloop.c:789 #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0) at ../lib/ ioloop.c:762 #32 0x00007fcfb8b6ce57 in master_service_run (service=0x55dabe243e20, callback=callback@entry=0x55dabc533210 <client_connected>) at ../lib-master/ master-service.c:878 #33 0x000055dabc51ad37 in main (argc=<optimized out>, argv=<optimized out>) at ./src/imap/main.c:575 John

2 7

Missing IMAP METADATA Plugin
by Adam Miller 23 Aug '25

23 Aug '25

OS: Ubuntu 22.04.1 Dovecot: 2.3.16 (7e2e900c1a) When trying to enable the “imap_metadata” plugin, I get the following error: Plugin 'imap_metadata' not found from directory /usr/lib/dovecot/modules I have searched high and low via Google, StackOverflow and even ChatGPT (haha) but I cannot seem to find the package that provides the plugin. Does Dovecot have to be built from source for this plugin to be included? Any help is appreciated. Thank you!

8 8

[Patch] Allow sieve redirects to specified domains only
by Kadlecsik József 10 Aug '25

10 Aug '25

Hello, The use case is simple: we have to forbid sieve redirects to outside, but should be able to redirect inside. The patch below adds a new sieve configuration option sieve_allowed_redirects to list the domains to which redirects are permitted and otherwise are not allowed. Please consider it. diff --git a/pigeonhole/doc/example-config/conf.d/90-sieve.conf b/pigeonhole/doc/example-config/conf.d/90-sieve.conf index 238bcf4..3b4ac65 100644 --- a/pigeonhole/doc/example-config/conf.d/90-sieve.conf +++ b/pigeonhole/doc/example-config/conf.d/90-sieve.conf @@ -126,6 +126,10 @@ plugin { # script execution. If set to 0, no redirect actions are allowed. #sieve_max_redirects = 4 + # The comma/space separated list of email domains where redirection is allowed. + # If the list is empty, there is no restriction. + sieve_allowed_redirects = + # The maximum number of personal Sieve scripts a single user can have. If set # to 0, no limit on the number of scripts is enforced. # (Currently only relevant for ManageSieve) diff --git a/pigeonhole/src/lib-sieve/cmd-redirect.c b/pigeonhole/src/lib-sieve/cmd-redirect.c index 6a3b0a4..17e9031 100644 --- a/pigeonhole/src/lib-sieve/cmd-redirect.c +++ b/pigeonhole/src/lib-sieve/cmd-redirect.c @@ -107,6 +107,24 @@ const struct sieve_action_def act_redirect = { .commit = act_redirect_commit, }; +static bool +cmd_redirect_allowed(struct sieve_instance *svinst, + const struct smtp_address *to_address) +{ + const char **domain; + bool result = TRUE; + + if (!svinst->allowed_redirects) + return TRUE; + for (domain = svinst->allowed_redirects; *domain != NULL; ++domain) { + if (strcasecmp(*domain, to_address->domain) == 0) { + return TRUE; + } + result = FALSE; + } + return result; +} + /* * Validation */ @@ -127,6 +145,12 @@ cmd_redirect_validate(struct sieve_validator *validator, if (!sieve_validator_argument_activate(validator, cmd, arg, FALSE)) return FALSE; + if (svinst->max_redirects == 0) { + sieve_command_validate_error(validator, cmd, + "local policy prohibits the use of a redirect action"); + return FALSE; + } + /* We can only assess the validity of the outgoing address when it is * a string literal. For runtime-generated strings this needs to be * done at runtime. @@ -148,14 +172,17 @@ cmd_redirect_validate(struct sieve_validator *validator, } } T_END; + if (!result && + !cmd_redirect_allowed(svinst, + sieve_address_parse_str(raw_address, &error))) { + sieve_command_validate_error(validator, cmd, + "local policy prohibits the use of a redirect action"); + return FALSE; + } + return result; } - if (svinst->max_redirects == 0) { - sieve_command_validate_error(validator, cmd, - "local policy prohibits the use of a redirect action"); - return FALSE; - } return TRUE; } @@ -238,6 +265,11 @@ cmd_redirect_operation_execute(const struct sieve_runtime_env *renv, "local policy prohibits the use of a redirect action"); return SIEVE_EXEC_FAILURE; } + if (!cmd_redirect_allowed(svinst, to_address)) { + sieve_runtime_error(renv, NULL, + "local policy prohibits the use of a redirect action"); + return SIEVE_EXEC_FAILURE; + } if (sieve_runtime_trace_active(renv, SIEVE_TRLVL_ACTIONS)) { sieve_runtime_trace(renv, 0, "redirect action"); diff --git a/pigeonhole/src/lib-sieve/sieve-common.h b/pigeonhole/src/lib-sieve/sieve-common.h index e79fb4d..d980c9d 100644 --- a/pigeonhole/src/lib-sieve/sieve-common.h +++ b/pigeonhole/src/lib-sieve/sieve-common.h @@ -209,6 +209,7 @@ struct sieve_instance { const struct smtp_address *user_email, *user_email_implicit; struct sieve_address_source redirect_from; unsigned int redirect_duplicate_period; + const char **allowed_redirects; }; /* diff --git a/pigeonhole/src/lib-sieve/sieve-settings.c b/pigeonhole/src/lib-sieve/sieve-settings.c index 47f70da..70addc7 100644 --- a/pigeonhole/src/lib-sieve/sieve-settings.c +++ b/pigeonhole/src/lib-sieve/sieve-settings.c @@ -267,4 +267,10 @@ void sieve_settings_load(struct sieve_instance *svinst) svinst->user_email = address; } } + svinst->allowed_redirects = NULL; + str_setting = sieve_setting_get(svinst, "sieve_allowed_redirects"); + if (str_setting != NULL && *str_setting != '\0') { + svinst->allowed_redirects = + (const char **)p_strsplit_spaces(svinst->pool, str_setting, ", "); + } } Best regards, Jozsef -- E-mail : kadlecsik.jozsef(a)wigner.hun-ren.hu PGP key: https://wigner.hu/~kadlec/pgp_public_key.txt Address: Wigner Research Centre for Physics H-1525 Budapest 114, POB. 49, Hungary

2 1

Re: Problem email client iPhone ios18.2
by Michael Slusarz 17 Jun '25

17 Jun '25

Timo confirmed this type of message (see below) breaks iOS mail clients regardless of the IMAP server used. So it is almost certainly a client issue. We are in contact with the Apple Mail developers and have provided this debug information to them. michael > On 02/10/2025 7:19 AM MST Timo Sirainen via dovecot <dovecot(a)dovecot.org> wrote: > > > On 10. Feb 2025, at 15.24, Florian Effenberger via dovecot <dovecot(a)dovecot.org> wrote: > > > > Hello, > > > > frido--- via dovecot wrote on 10.02.25 at 14:07: > >> The problem as described at the start of this thread is not solved in iOS 18.3. A bit more testing shows that it only appears to happen with attachments around 1MB and bigger. > > > > there seem to be two kind of problems, at least on my devices. > > I'm also wondering if there are multiple issues. I have one way of easily reproducing, but it has nothing to do with IMAP IDLE. Here's a test email which causes the hang - snipped it a bit in the middle which you can fill out yourself with: > > dd if=/dev/urandom bs=1024 count=3432 | base64 -w76 > > If you want to test multiple times, change Message-Id for every delivery. This mail can be simply saved to INBOX, e.g. with doveadm save: > > To: <user(a)example.com> > Subject: testing > Content-Type: multipart/mixed; boundary="foo" > Date: Tue, 17 Dec 2024 06:46:17 +0000 > From: <user(a)example.com> > Message-Id: <fooab(a)example.com> > > > > --foo > Content-Type: multipart/alternative; boundary="bar" > > --bar > Content-Transfer-Encoding: quoted-printable > Content-Type: text/html; charset=UTF-8 > > asdf > --bar-- > > --foo > Content-Description: test.pdf > Content-Disposition: attachment; filename="test.pdf" > Content-Transfer-Encoding: base64 > Content-Type: application/octet-stream; name="test.pdf" > > > lBjZO0luFuyUiOqQ8KPTTpMyPoNhlSTPPJauDuCVSwbZgaNyhnObul5qrdyPKo0NGiQKA3Kha34A > <61642 lines of random> > 4cNyk39T2wMsnQavfl8LXwG1vNQylpgmSN4p6fOpmGqQ > > --foo-- > > _______________________________________________ > dovecot mailing list -- dovecot(a)dovecot.org > To unsubscribe send an email to dovecot-leave(a)dovecot.org

3 3

Panic with FTS_BACKEND_FLAG_TOKENIZED_INPUT and virtual mailboxes: file index-search-result.c: line 132
by markus-dovecot＠doits.de 30 May '25

30 May '25

Originally posted and debugged here: https://github.com/slusarz/dovecot-fts-flatcurve/issues/66 I have to following panic with fts flatcurve enabled: ``` Panic: file index-search-result.c: line 132 (index_search_result_update_flags): assertion failed: (result->search_args->args == &search_arg) Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x46) [0x74e9890a2846] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x22) [0x74e9890a2962] -> /usr/lib/dovecot/libdovecot.so.0(+0x10d81b) [0x74e9890af81b] -> /usr/lib/dovecot/libdovecot.so.0(+0x10d8b7) [0x74e9890af8b7] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e2e2) [0x74e9890002e2] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x518a4) [0x74e9891d38a4] -> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x274e) [0x74e988bf8a9e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x5c) [0x74e9891eff9c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x39) [0x74e9891f0039] -> dovecot/imap(cmd_select_full+0x1df) [0x62354bd8326f] -> dovecot/imap(command_exec+0xa4) [0x62354bd8b444] -> dovecot/imap(+0x22392) [0x62354bd89392] -> dovecot/imap(+0x22444) [0x62354bd89444] -> dovecot/imap(client_handle_input+0x1bd) [0x62354bd8989d] -> dovecot/imap(client_input+0x74) [0x62354bd89e64] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6d) [0x74e9890c603d] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x13a) [0x74e9890c77aa] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x54) [0x74e9890c60e4] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x74e9890c62a0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x17) [0x74e989036ad7] -> dovecot/imap(main+0x570) [0x62354bd7ad20] -> /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x74e988c29d90] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x74e988c29e40] -> dovecot/imap(_start+0x25) [0x62354bd7ade5] ``` It happens when accessing virtual folder with filters like this: ``` something-*/excep-dev* header x-abc-exception "1" ``` The error happens on subsequent access of the folder: - Removing dovecot.index{,.log} files allows to access the virtual folder again - dovecot.index gets recreated on access and the folder can be accessed multiple times - Marking an e-mail read triggers the error, though! Now each next access raises the error - Removing dovecot.index{,.log} restores access again and the e-mails are still marked as read (so this information is not lost), until the next e-mail is marked as read: then the folder cannot be accessed again Slusarz debugged the backtrace and wrote a detailed description at https://github.com/slusarz/dovecot-fts-flatcurve/issues/66#issuecomment-229… which I'll quote here: > This is not a flatcurve bug - it is a bug within Dovecot's indexing/fts handling. > > (Detailed debug below, so that core team can understand the problem.) > > As discussed above, this bug is triggered with virtual mailboxes on subsequent accesses. The initial access will work fine; the assert crash only happens when the Dovecot lib-storage index code reads the indexes created for the virtual mailboxes. > > `index_search_result_update_appends()` in lib-storage/index/index-search-result.c > > * Inside this function, a "temporary search parameter to limit the search only to the new messages" is added to the mail search args. Later, it is checked that the mail_search args points to this same search parameter after the search is executed. It is this assert check that is failing, because the mail args is changing during the course of the search. > > * `i_assert(result->search_args->args == &search_arg);` > > > The search args are being altered in plugins/fts/fts-search.args.c, `fts_search_args_expand()`. Specifically, in that function, either `fts_search_args_expand_language_top_level()` or `fts_search_args_expand_tree()` is being called, which can replace the current args parameter with a new pointer. > > `fts_search_args_expand()` is being called in plugins//fts/fts-search.c by `fts_search_try_lookup()` when the FTS backend has the FTS_BACKEND_FLAG_TOKENIZED_INPUT flag set. Flatcurve has this flag set. > > This flag is defined in plugins/fts/fts-api-private.h. Comments say: > > ``` > /* Tokenize all the input. update_build_more() will be called a single > directly indexable token at a time. Searching will modify the search > args so that lookup() sees only tokens that can be directly > searched. */ > ``` > > Thus, the comments explicitly say the search args will be modified if this flag is set. This modification is incompatible with the assert check in `index_search_result_update_appends()`, which requires that the search args not be modified. > > The core team will have to determine whether this behavior is correct, or if the assert check is invalid. You can find some more discussion at https://github.com/slusarz/dovecot-fts-flatcurve/issues/66, but I think the most important part (the very detailed analysis from Slusarz, thanks again!) might be enough to understand and maybe fix the problem. Relevant information about the mail system: ``` # 2.3.21 (47349e2482): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.21 (f6cd4b8e) # OS: Linux 6.5.0-45-generic x86_64 Ubuntu 22.04.4 LTS [...] namespace { location = virtual:~/virtual:LAYOUT=maildir++ prefix = _/ separator = / } [...] plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/db/shared fts = flatcurve fts_autoindex = yes fts_filters = normalizer-icu snowball stopwords fts_filters_en = lowercase snowball stopwords fts_languages = de en fts_tokenizer_email_address = maxlen=100 fts_tokenizer_generic = algorithm=simple maxlen=30 fts_tokenizers = generic email-address sieve = file:~/sieve;active=~/.dovecot.sieve } [...] ``` I can provide more information if you need. Thanks for looking into it!

2 2

Thunderbird and Dovecot 2.4.0 Incompatibility
by Amit Chaudhari 27 May '25

27 May '25

I'm trying to setup a dovecot 2.4.0 imap server and have it work with Thunderbird. I'm able to connect/auth fine, however trying to save a message to a mailbox causes the connection to close. ``` Info: Disconnected: Too many invalid IMAP commands. in=744 out=2559 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0 ``` It appears thunderbird's imap APPEND command is using syntax which dovecot doesn't like? I enabled raw logs and pasted them below: Downgrading to 2.3 resolves the issue, so it’s either a regression, or some config issue I’m not able to pinpoint. ``` 1738942430.025029 5 ID ("name" "Thunderbird" "version" "128.7.0") 1738942430.029320 6 ENABLE UTF8=ACCEPT 1738942430.033907 7 select "Drafts" 1738942430.039796 8 append "Drafts" (\Draft) UTF8 (~{726+} 1738942430.039796 Message-ID: <ea714f62-7186-4bcb-b709-007e39c6a9d6(a)example.com <mailto:ea714f62-7186-4bcb-b709-007e39c6a9d6@example.com>> 1738942430.039796 Date: Fri, 7 Feb 2025 10:33:49 -0500 1738942430.039796 MIME-Version: 1.0 1738942430.039796 User-Agent: Mozilla Thunderbird 1738942430.039796 Content-Language: en-US 1738942430.039796 To: sender(a)example.com <mailto:sender@example.com> 1738942430.039796 From: User Name <user(a)example.com <mailto:user@example.com>> 1738942430.039796 Subject: test 1738942430.039796 X-Mozilla-Draft-Info: internal/draft; vcard=0; receipt=0; DSN=0; uuencode=0; 1738942430.039796 attachmentreminder=0; deliveryformat=0 1738942430.039796 X-Identity-Key: id2 1738942430.039796 Fcc: imap://user%40example.com@192.168.115.203/Sent 1738942430.039796 Content-Type: text/html; charset=UTF-8 1738942430.039796 Content-Transfer-Encoding: 7bit 1738942430.039796 1738942430.039796 <!DOCTYPE html> 1738942430.039796 <html> 1738942430.039796 <head> 1738942430.039796 1738942430.039796 <meta http-equiv="content-type" content="text/html; charset=UTF-8"> 1738942430.039796 </head> 1738942430.039796 <body> 1738942430.039796 <p><br> 1738942430.039796 </p> 1738942430.039796 </body> 1738942430.039796 </html> 1738942430.039796 ) ``` ``` 1738942430.025188 * ID ("name" "Dovecot") 1738942430.025234 5 OK ID completed (0.001 + 0.000 secs). 1738942430.029390 * ENABLED UTF8=ACCEPT 1738942430.029418 6 OK Enabled (0.001 + 0.000 secs). 1738942430.035478 * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) 1738942430.035531 * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. 1738942430.035559 * 0 EXISTS 1738942430.035585 * 0 RECENT 1738942430.035611 * OK [UIDVALIDITY 1738898190] UIDs valid 1738942430.035638 * OK [UIDNEXT 1] Predicted next UID 1738942430.035693 7 OK [READ-WRITE] Select completed (0.002 + 0.000 + 0.001 secs). 1738942430.040007 8 BAD Error in IMAP command APPEND: Missing ')' (0.001 + 0.000 secs). 1738942430.040140 * BAD Error in IMAP tag: Invalid tag (0.001 + 0.000 secs). 1738942430.040172 Date: BAD Error in IMAP command FRI,: Unknown command (0.001 + 0.000 secs). 1738942430.040233 MIME-Version: BAD Error in IMAP command 1.0: Unknown command (0.001 + 0.000 secs). 1738942430.040291 User-Agent: BAD Error in IMAP command MOZILLA: Unknown command (0.001 + 0.000 secs). 1738942430.040356 Content-Language: BAD Error in IMAP command EN-US: Unknown command (0.001 + 0.000 secs). 1738942430.040409 To: BAD Error in IMAP command sender(a)example.com <mailto:sender@example.com>: Unknown command (0.001 + 0.000 secs). 1738942430.040467 From: BAD Error in IMAP command USER: Unknown command (0.001 + 0.000 secs). 1738942430.040521 Subject: BAD Error in IMAP command TEST: Unknown command (0.001 + 0.000 secs). 1738942430.040574 X-Mozilla-Draft-Info: BAD Error in IMAP command INTERNAL/DRAFT;: Unknown command (0.001 + 0.000 secs). 1738942430.040638 * BAD Error in IMAP tag: Invalid tag (0.001 + 0.000 secs). 1738942430.040666 X-Identity-Key: BAD Error in IMAP command ID2: Unknown command (0.001 + 0.000 secs). 1738942430.040720 Fcc: BAD Error in IMAP command: Invalid command name (0.001 + 0.000 secs). 1738942430.040746 Content-Type: BAD Error in IMAP command TEXT/HTML;: Unknown command (0.001 + 0.000 secs). 1738942430.040802 Content-Transfer-Encoding: BAD Error in IMAP command 7BIT: Unknown command (0.001 + 0.000 secs). 1738942430.040862 * BAD Error in IMAP tag: Invalid tag (0.001 + 0.000 secs). 1738942430.040889 <!DOCTYPE BAD Error in IMAP command HTML>: Unknown command (0.001 + 0.000 secs). 1738942430.040966 <html> BAD Error in IMAP command: Invalid command name (0.001 + 0.000 secs). 1738942430.040991 * BAD Error in IMAP tag: Invalid tag (0.001 + 0.000 secs). 1738942430.041015 * BAD Error in IMAP tag: Invalid tag (0.001 + 0.000 secs). 1738942430.041032 * BYE Too many invalid IMAP commands. ```

5 7

Quota wrongfully reported as exceeded on shared mailbox
by adminkram＠tripelspark.de 07 May '25

07 May '25

Hi, I've been scratching my head without luck with the following problem: I've shared a mailbox from one user "COMPANY" to another user "JOE". Both users have quotas and both quotas are less than 50% used. When the user JOE tries to post a message into the shared folder, dovecot complains that the user would be over quota: bash# /usr/lib/dovecot/imap -u JOE 1 append shared/COMPANY/Sent/JOE {10} a NO [OVERQUOTA] Quota exceeded (mailbox for user is full) (0.057 + 0.000 + 0.057 secs). However, nobody is over quota: bash# doveadm quota get -u COMPANY Quota name Type Value Limit % User quota STORAGE 4754996 10485760 45 User quota MESSAGE 33695 - 0 bash# doveadm quota get -u JOE Quota name Type Value Limit % User quota STORAGE 14 1048576 0 User quota MESSAGE 3 - 0 Some more infos: 1. Dovecot-2.3.21 (+Debian updates) 2.3.21 (47349e2482) (NOTE: dovecot.org/mailing-lists says: Use dovecot –version to get it. The command is dovecot --version, though!) 2. I've performed "doveadm quota recalc" on both users, no change. 3. I've checked sharing ACL permissions, if I give just "l" permission, the response will be "NO [NOPERM] Permission denied". It does not seem to be an ACL permission reported wrongly. 4. I've tried increasing the quota size beyond reason (100 Gbytes for JOE), no change. 5. I've restarted dovecot after each change. 6. When trying to move a message into the shared Sent/joe folder in Thunderbird, same problem happens, Thunderbird displays the over quota message. Thunderbird cannot display the quota info for the shared folder, though. 7. I tried to enable rawlog (rawlog_dir in protocol imap). It does nothing helpful. It logs the manual commands from my pre-auth sessions (imap -u JOE) but it will stubbornly not log any commands issued by Thunderbird. ?! Any idea how to further debug this or what could be wrong? Thank you, Wolfgang

2 1

Problem email client iPhone ios18.2
by andrzej.milewski＠gmail.com 17 Apr '25

17 Apr '25

Hello, I have a fairly new installation of the Dovecot server version 2.3.19.1 on Debian 12.8. In general, the server works correctly. It sends and receives emails without issues. The clients primarily use Thunderbird on desktops and Gmail on Android. However, some clients are using iPhones with various hardware versions. Those with iOS below version 18 work correctly without any issues. Those who have updated their phones to version 18 and now even 18.2 are experiencing problems with mailbox synchronization. On version 18.2, the issue appears as follows: When you open the app, you can read several messages. They load correctly, but, for instance, attachments do not. If I wait for some time, the attachment eventually loads. During this time, I observe the server logs, and it shows that a certain session on the Dovecot server has ended. Dec 17 13:13:01 iredmail0 dovecot[408]: imap(xxx(a)xxx.com)<44785><1QGZCHcp/+3AqM2O>: Disconnected: Connection closed (UID SEARCH finished 92.295 secs ago) in=2972 out=6076956 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=36 body_bytes=6059064 Dec 18 08:40:41 iredmail0 dovecot[408]: imap(xxx(a)xxx.com)<177412>: Disconnected: Connection closed (UID FETCH finished 90.079 secs ago) in=5126 out=1304237 deleted=0 expunged=0 trashed=0 hdr_count=94 hdr_bytes=289688 body_count=89 body_bytes=969615 Dec 18 08:42:19 iredmail0 dovecot[408]: imap(xxx(a)xxx.com)<177410><D5mqU4cpZ+nAqM2O>: Disconnected: Connection closed (UID FETCH finished 95.051 secs ago) in=4475 out=1162785 deleted=0 expunged=0 trashed=0 hdr_count=95 hdr_bytes=281674 body_count=61 body_bytes=839545 Dec 18 08:44:17 iredmail0 dovecot[408]: imap(edyta.kowalewska(a)laktopol.com.pl)<177918>: Disconnected: Connection closed (UID FETCH finished 92.132 secs ago) in=29399 out=30266644 deleted=0 expunged=0 trashed=0 hdr_count=565 hdr_bytes=1726228 body_count=628 body_bytes=28109455 After the session is closed, the attachment loads immediately. From the logs and router analysis, it appears that the download happens in another previously opened session. After this incident, I go to another message with an attachment, and the situation repeats. There are also situations where the email body does not load, and the app waits for the server until the session closes and the content is fetched in another session. It does not matter whether I connect to the server via the internet or through a local network. On an iPhone 7 with iOS 15.8.3, everything works smoothly. Attachments open without issues. On iOS 18.1.1, attachments do not load, and the session does not close on its own. It takes forever unless the app is closed. In the Gmail app, everything works correctly without any issues. This is clearly a problem with the mail client on iPhone. However, is there anything I can adjust in my server settings to mitigate this issue for users? My dovecot configuration https://pastebin.com/UPNQ5dZz

26 58

Stats-writer permission denied
by Ken Wright 15 Apr '25

15 Apr '25

I've got a new Dovecot error (well, new to me, at least)! I ran tail /var/log/mail.err and got the following: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied I checked the file, and it's owned by www-data and the www-data group, with read and write permissions for owner and group... just like I have specified in /etc/dovecot/conf.d/10-master.conf. I can't understand why permission is denied when the file permissions appear to be correct. Can anyone shed any light on this for me? Ubuntu 20.04 Dovecot 2.3.7.2 Ken

3 2