August 2024 - dovecot

Quota wrongfully reported as exceeded on shared mailbox
by adminkram＠tripelspark.de 07 May '25

07 May '25

Hi, I've been scratching my head without luck with the following problem: I've shared a mailbox from one user "COMPANY" to another user "JOE". Both users have quotas and both quotas are less than 50% used. When the user JOE tries to post a message into the shared folder, dovecot complains that the user would be over quota: bash# /usr/lib/dovecot/imap -u JOE 1 append shared/COMPANY/Sent/JOE {10} a NO [OVERQUOTA] Quota exceeded (mailbox for user is full) (0.057 + 0.000 + 0.057 secs). However, nobody is over quota: bash# doveadm quota get -u COMPANY Quota name Type Value Limit % User quota STORAGE 4754996 10485760 45 User quota MESSAGE 33695 - 0 bash# doveadm quota get -u JOE Quota name Type Value Limit % User quota STORAGE 14 1048576 0 User quota MESSAGE 3 - 0 Some more infos: 1. Dovecot-2.3.21 (+Debian updates) 2.3.21 (47349e2482) (NOTE: dovecot.org/mailing-lists says: Use dovecot –version to get it. The command is dovecot --version, though!) 2. I've performed "doveadm quota recalc" on both users, no change. 3. I've checked sharing ACL permissions, if I give just "l" permission, the response will be "NO [NOPERM] Permission denied". It does not seem to be an ACL permission reported wrongly. 4. I've tried increasing the quota size beyond reason (100 Gbytes for JOE), no change. 5. I've restarted dovecot after each change. 6. When trying to move a message into the shared Sent/joe folder in Thunderbird, same problem happens, Thunderbird displays the over quota message. Thunderbird cannot display the quota info for the shared folder, though. 7. I tried to enable rawlog (rawlog_dir in protocol imap). It does nothing helpful. It logs the manual commands from my pre-auth sessions (imap -u JOE) but it will stubbornly not log any commands issued by Thunderbird. ?! Any idea how to further debug this or what could be wrong? Thank you, Wolfgang

2 1

Stats-writer permission denied
by Ken Wright 15 Apr '25

15 Apr '25

I've got a new Dovecot error (well, new to me, at least)! I ran tail /var/log/mail.err and got the following: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied I checked the file, and it's owned by www-data and the www-data group, with read and write permissions for owner and group... just like I have specified in /etc/dovecot/conf.d/10-master.conf. I can't understand why permission is denied when the file permissions appear to be correct. Can anyone shed any light on this for me? Ubuntu 20.04 Dovecot 2.3.7.2 Ken

3 2

Bug: Logout counters reset after recreating hibernated process
by Eirik Rye 12 Mar '25

12 Mar '25

Hello, # dovecot --version 2.3.20 (80a5ac675d) When using hibernation (`imap_hibernate_timeout = 10s`), counters such as deleted/expunged messages, appear to be reset after the user process is recreated. Steps to reproduce: 1. Configure IMAP hibernation 2. Login with IMAP client 3. Delete+expunge a message 4. IDLE, and wait 10s for process to be hibernated 5. Unhibernate process (`DONE`), and log out 6. Observe the logged line: 2023-09-05T17:36:26+0200 imap(<redacted>)<776933><jA2nYJ4EPdcqAVtAAADwAAutur4AAAAB>: Info: Disconnected: Logged out in=9 out=88 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0 The expected output is "deleted=1 expunged=1", not "deleted=0 expunged=0". If you follow the steps above, but skip the IDLE/hibernate step, the output is as expected. Interestingly, if the client is disconnected during hibernation (without recreating the process), the output from `imap-hibernate` is still as expected: 2023-09-05T17:45:15+0200 imap-hibernate(<redacted>)<770540><0qE0gJ4Emt8qAVtAAADwAAutur4AAAAB>: Info: Disconnected: Connection closed in=97 out=1020 deleted=1 expunged=1 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0 This leads me to believe the counters are lost during process recreation, not during hibernation. This is an issue as customers will often come to us asking why messages have disappeared in their inbox, or asking for help to identify which client is expunging messages. Right now, this information is lost if hibernation is used and the client is hibernated after expunging. Best regards, Eirik Rye

2 2

2.3.21.1 and oauth
by v＠minakov.pro 14 Nov '24

14 Nov '24

Hello! I'm using dovecot with keycloak for oauth authentication. My config is: client_id = dovecot client_secret = MY_SECRET introspection_url = https://MY_KEYCLOAK/realms/master/protocol/openid-connect/token/introspect introspection_mode = post pass_attrs = pass=%{oauth2:access_token} Everything worked great on version 2.3.20. After upgrading to version 2.3.21.1 oauth stopped working with errors: auth: Debug: http-client: conn [::1]:443 [1]: Got 401 response for request [Req1: POST https://MY_KEYCLOAK/realms/master/protocol/openid-connect/token/introspect] Unauthorized (took 5 ms + >Aug 16 00:23:58 auth: Error: oauth2(MY_EMAIL,127.0.0.1,<HADwdb8f+PN/AAAB>): oauth2 failed: Introspection failed: No username returned I tried all combination of configurations, debugging and versions of keycloak. Seems something was broken in this version of dovecot.

3 3

sieve crashes when two optional include scripts are missing
by Michal Hlavinka 26 Oct '24

26 Oct '24

Hi, we've found an issue with sieve plugin. When script includes two missing optional scripts, it causes crash. For reproducing the issue, script names must belong to same hash table node, so either two scripts with same name or scripts where 'hash(name) % table->size' results in same value, like missing_a and missing_aa in reproducer below. Reduced reproducer: cat >dovecot.min.conf <<EOF mail_location = maildir:~/Maildir plugin { sieve = file:/tmp;active=/tmp/dovecot.sieve } EOF cat >reproducer.sieve <<EOF require "include"; include :personal :optional "missing_a"; include :personal :optional "missing_aa"; EOF echo -n > empty sieve-test -c dovecot.min.conf reproducer.sieve empty any test mail would do, but empty file is enough for reproducer # return sblock->id causes the crash as sblock is NULL backtrace: (gdb) bt #0 sieve_binary_block_get_id (sblock=0x0) at src/lib-sieve/sieve-binary.c:421 #1 ext_include_execute_include (renv=0x5555555bbed8, include_id=1, flags=EXT_INCLUDE_FLAG_OPTIONAL) at plugins/include/ext-include-common.c:696 #2 opc_include_execute (renv=0x5555555bbed8, address=0x5555555bbf10) at plugins/include/cmd-include.c:399 #3 sieve_interpreter_operation_execute (interp=0x5555555bbe88) at src/lib-sieve/sieve-interpreter.c:901 #4 sieve_interpreter_continue (interp=0x5555555bbe88, interrupted=0x0) at src/lib-sieve/sieve-interpreter.c:959 #5 sieve_interpreter_start (interp=0x5555555bbe88, result=0x5555555b9230, interrupted=0x0) at src/lib-sieve/sieve-interpreter.c:1049 #6 sieve_interpreter_run (interp=0x5555555bbe88, result=0x5555555b9230) at src/lib-sieve/sieve-interpreter.c:1057 #7 sieve_run (sbin=0x55555558ee18, result=0x5555555b9230, eenv=0x7fffffffe890, ehandler=0x55555558c338) at src/lib-sieve/sieve.c:357 #8 sieve_test (sbin=0x55555558ee18, msgdata=0x7fffffffe9f0, senv=0x7fffffffea30, ehandler=0x55555558c338, stream=0x5555555b2cd0, flags=SIEVE_EXECUTE_FLAG_LOG_RESULT) at src/lib-sieve/sieve.c:598 #9 main (argc=5, argv=0x555555567a10) at src/sieve-tools/sieve-test.c:391 issue seems to be caused by lib-sieve/plugins/include/ext-include-common.c: ext_include_generate_include on first pass (script) the check for script already compiled into binary at 529: included = ext_include_binary_script_get_include_info(binctx, script); returns NULL, so code goes through 'else' block where it hits sieve_script_is_open(script) check. It includes script with NULL block, returns 0 so caller cmd_include_generate does not emit anything. On second pass the above 'included' check returns first missing script, it goes through true block where it just checks flags and returns non-zero, so caller this time goes through the emit code block and the previously included NULL sblock gets referenced causing crash later. Note: despite scripts have different name, they pass the "same script" check because sieve_file_script_equals compares 0==0 dev & inode numbers as scripts do not exist. This is why scripts do not have to have same names. Side effect of this is that as missing_a and missing_aa are treated like the same script, only first one is linked in compiled svbin. Which means that if missing_aa reappears later, svbin won't get recompiled. Another manifestation of the issue is when included missing_a script is present when main script is compiled and later missina_a is removed again. During execution, script is not recompiled and it aborts for hash table dupe check: sieve_binary_open->ext_include_binary_script_include->hash_table_insert There are several ways how to solve this. I've tried a few, but there were other corner cases that appeared. One tried way was for example ext_include_generate_include in the 'if (included)' replicate the check sieve_script_is_open(...) from else block, just check flags, no include and set result to 0. Another way to solve this is to return from ext_include_execute_include early if sblock is NULL just before it tries to derefernce it, similarly as it does if it fails 'once' check. Same for dumping in opc_include_dump. Whilst the first approach seems a bit better, it won't stop crashing in existing environments as it won't trigger svbin recompilation and svbin would fail. This means that the second soultion is sort of needed anyway. This still won't solve the problem with different missing scripts as only first one would be able to trigger recompilation when file becomes available. Just including both in the mentioned function won't work either as they would still be evaluated as same and hash table anti-dupe check would trigger abort. So some changes in script comparison seem needed anyway. This is what I've used in my approach. Return early from ext_include_execute_include and opc_include_dump if sblock is NULL and change sieve_file_script_equals to compare also script->location for missing scripts (when inode is 0). Let me know if you need more information. Cheers, Michal Hlavinka

2 1

Uppercase username emails are rejected
by lua8ds＠proton.me 20 Oct '24

20 Oct '24

When a sender writes my email address with my username uppercase, e.g. USERNAME(a)name.com, in the to: field of their MUA, my mail server rejects that email. /var/log/mail.log prints: <USERNAME(a)redacted.com>: host mail.redacted.com[private/dovecot-lmtp] said: 550 5.1.1 > <USERNAME(a)redacted.com> User doesn't exist: USERNAME(a)redacted.com (in reply to RCPT TO > command) In the manual, I found that: auth_username_format Default: %Lu Formatting applied to username before querying the auth database. You can use the standard variables here. Examples: %Lu Lowercases the username [...] My current setting was not the default; I had: auth_username_format = %n So I updated that setting to: auth_username_format = %Lu expecting that %Lu would lowercase the username - and that my problem would be fixed. It does not though. Updating this setting not only prompts my email server to still reject USERNAME(a)redacted.com, but now also rejects (lowercase username) username(a)redacted.com (which my server had no issue receiving beforehand). So I reverted auth_username_format to: auth_username_format = %n Emails sent to username(a)redacted.com land in my mailbox again - but emails sent to USERNAME(a)redacted.com still don't. What can I try to troubleshoot? Which config file shall I share with you?

10 22

dsync with vurtial folder leads to mess of msg ID
by Kirill A. Korinsky 25 Sep '24

25 Sep '24

Greetings, I have a setup with two dovecot servers and dsync replication between them. On both of them I have a virtual folders, forexample All with config: mx1# cat /etc/dovecot/virtual/All/dovecot-virtual * -Trash -Trash/* -Junk -Junk/* all All virtual folders are excluded from replication because it leads to broken replication with errors like: virtual.All can't be created. So, I have. replication_dsync_parameters = -d -l 30 -U -x virtual. Unfortently, after a couple of weeks I've noticed that msg id for the same message on the different servers aren't match. For example: mx1# doveadm fetch -u kirill(a)korins.ky 'uid' mailbox virtual.All header Message-ID 'macports/macports-ports/pull/22284/c1901369454(a)github.com' uid: 157786 mx1# doveadm fetch -u kirill(a)korins.ky 'uid' mailbox INBOX header Message-ID 'macports/macports-ports/pull/22284/c1901369454(a)github.com' uid: 5561 mx1# vs mx2# doveadm fetch -u kirill(a)korins.ky 'uid' mailbox virtual.All header Message-ID 'macports/macports-ports/pull/22284/c1901369454(a)github.com' uid: 157766 mx2# doveadm fetch -u kirill(a)korins.ky 'uid' mailbox INBOX header Message-ID 'macports/macports-ports/pull/22284/c1901369454(a)github.com' uid: 5561 mx2# as you may see msgid at INBOX are matched, but it isn't true for virtual.All, here 157786 vs 157766 So, here the question: how can I sync msg id inside virtual folder on this setup? -- wbr, Kirill

2 18

Dovecot v2.3.21.1 released
by Aki Tuomi 07 Sep '24

07 Sep '24

Hi all, we are releasing a CVE patch release 2.3.21.1. https://dovecot.org/releases/2.3/dovecot-2.3.21.1.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.21.1.tar.gz.sig Binary packages in https://repo.dovecot.org/ Docker images in https://hub.docker.com/r/dovecot/dovecot Kind regards, Aki Tuomi Open-Xchange oy --- - CVE-2024-23184: A large number of address headers in email resulted in excessive CPU usage. - CVE-2024-23185: Abnormally large email headers are now truncated or discarded, with a limit of 10MB on a single header and 50MB for all the headers of all the parts of an email. - oauth2: Dovecot would send client_id and client_secret as POST parameters to introspection server. These need to be optionally in Basic auth instead as required by OIDC specification. - oauth2: JWT key type check was too strict. - oauth2: JWT token audience was not validated against client_id as required by OIDC specification. - oauth2: XOAUTH2 and OAUTHBEARER mechanisms were not giving out protocol specific error message on all errors. This broke OIDC discovery. - oauth2: JWT aud validation was not performed if aud was missing from token, but was configured on Dovecot.

5 9

Re: Dovecot authentication hangs when ldap_start_tls_s() fails for invalid certificate
by Aki Tuomi 04 Sep '24

04 Sep '24

> On 19/11/2020 12:55 sebastiano degan <sebastiano.degan(a)frigoveneta.it> wrote: > > > On a Debian10 I've installed postfix 3.4.14 and dovecot 2.3.4.1. > I've configured multiple passdb sources and I expect that if one fails > the other ones are tested. > This is usually the case, except when the failure is due to an invalid > certificate from the ldap server. > In that case all authentication attempts from that moment on will fail. Hi! Can you try with latest CE release from https://repo.dovecot.org/ Regards, Aki

2 1

What are all the /var/spool/smtpd/offline/1689525601.XXXXlJ85yQ
by Steve Litt 03 Sep '24

03 Sep '24

Hi all, I just filled up my whole /var with zillions of files /var/spool/smtpd/offline , dating back to 2018. What are these files and is it safe for me to delete the ones more than a week old? Thanks, SteveT Steve Litt http://444domains.com

5 7