October 2024 - dovecot

Aggregating auth/proxy events
by matus.vlcek＠gmail.com 28 Oct '24

28 Oct '24

Hi, I'm working on aggregating event logs and trying to correlate all related events using session IDs. However, I’m struggling with some events as they're missing the session ID. 1. proxy_session_* events lack a session ID field, making it difficult to pair them with auth_request_* events. 2. when using dovecot as sasl for postfix with a unix listener, both auth_request_* and auth_passdb_request_* events are missing the session ID too. Example of the fields object for auth_request_finished when using smtp. It's missing multiple common fields, but session ID is, imho, the most important one. https://doc.dovecot.org/2.3/admin_manual/list_of_events/#id1 ``` categories: [ "auth", "service:auth" ] "fields": { "translated_user": "alex", "mechanism": "LOGIN", "real_remote_ip": "X.X.X.X", "user": "alex", "transport": "insecure", "local_ip": "X.X.X.X", "remote_ip": "X.X.X.X", "real_local_ip": "X.X.X.X", "policy_result": "ok", "service": "smtp", "duration": 96764, "original_user": "alex" }, ``` Would it be possible to add session ID fields to these logs? Alternatively, is there another method to correlate all the session related logs? Thanks, Matus Dovecot v. 2.3.21

1 0

LMTP rejecting aliased emails
by Nick Howitt 27 Oct '24

27 Oct '24

Hi, I am pretty new to Dovecot and totally new to LMTP. I have normal delivery working but I am struggling with aliases. In my mail queue I am seeing things like: -Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient------- E0F617055DB 720 Sat Oct 26 14:50:24 clamav(a)mydomain.com (host mail.mydomain.com[private/dovecot-lmtp] said: 451 4.3.0 <clamav(a)mydomain.com> Temporary internal error (in reply to end of DATA command)) clamav(a)mydomain.com But in my /etc/aliases, I am aliasing clamav to nick and nick(a)mydomain.com is a valid mail box that can receive normal mail. The corresponding line in the mail log is: 2024-10-26T14:55:56.385500+01:00 mail-www dovecot: lmtp(1371485): Error: lmtp-server: conn unix:pid=1371484,uid=102 [1]: rcpt clamav(a)mydomain.com: Failed to initialize user: Mail access for users with UID 107 not permitted (see first_valid_uid in config file, uid from userdb lookup). How do I get round this issue? I can drop the minimum UID to 0, but then all system users are deemed valid for mail which is not true. None of them are. Thanks, Nick

5 30

sieve crashes when two optional include scripts are missing
by Michal Hlavinka 27 Oct '24

27 Oct '24

Hi, we've found an issue with sieve plugin. When script includes two missing optional scripts, it causes crash. For reproducing the issue, script names must belong to same hash table node, so either two scripts with same name or scripts where 'hash(name) % table->size' results in same value, like missing_a and missing_aa in reproducer below. Reduced reproducer: cat >dovecot.min.conf <<EOF mail_location = maildir:~/Maildir plugin { sieve = file:/tmp;active=/tmp/dovecot.sieve } EOF cat >reproducer.sieve <<EOF require "include"; include :personal :optional "missing_a"; include :personal :optional "missing_aa"; EOF echo -n > empty sieve-test -c dovecot.min.conf reproducer.sieve empty any test mail would do, but empty file is enough for reproducer # return sblock->id causes the crash as sblock is NULL backtrace: (gdb) bt #0 sieve_binary_block_get_id (sblock=0x0) at src/lib-sieve/sieve-binary.c:421 #1 ext_include_execute_include (renv=0x5555555bbed8, include_id=1, flags=EXT_INCLUDE_FLAG_OPTIONAL) at plugins/include/ext-include-common.c:696 #2 opc_include_execute (renv=0x5555555bbed8, address=0x5555555bbf10) at plugins/include/cmd-include.c:399 #3 sieve_interpreter_operation_execute (interp=0x5555555bbe88) at src/lib-sieve/sieve-interpreter.c:901 #4 sieve_interpreter_continue (interp=0x5555555bbe88, interrupted=0x0) at src/lib-sieve/sieve-interpreter.c:959 #5 sieve_interpreter_start (interp=0x5555555bbe88, result=0x5555555b9230, interrupted=0x0) at src/lib-sieve/sieve-interpreter.c:1049 #6 sieve_interpreter_run (interp=0x5555555bbe88, result=0x5555555b9230) at src/lib-sieve/sieve-interpreter.c:1057 #7 sieve_run (sbin=0x55555558ee18, result=0x5555555b9230, eenv=0x7fffffffe890, ehandler=0x55555558c338) at src/lib-sieve/sieve.c:357 #8 sieve_test (sbin=0x55555558ee18, msgdata=0x7fffffffe9f0, senv=0x7fffffffea30, ehandler=0x55555558c338, stream=0x5555555b2cd0, flags=SIEVE_EXECUTE_FLAG_LOG_RESULT) at src/lib-sieve/sieve.c:598 #9 main (argc=5, argv=0x555555567a10) at src/sieve-tools/sieve-test.c:391 issue seems to be caused by lib-sieve/plugins/include/ext-include-common.c: ext_include_generate_include on first pass (script) the check for script already compiled into binary at 529: included = ext_include_binary_script_get_include_info(binctx, script); returns NULL, so code goes through 'else' block where it hits sieve_script_is_open(script) check. It includes script with NULL block, returns 0 so caller cmd_include_generate does not emit anything. On second pass the above 'included' check returns first missing script, it goes through true block where it just checks flags and returns non-zero, so caller this time goes through the emit code block and the previously included NULL sblock gets referenced causing crash later. Note: despite scripts have different name, they pass the "same script" check because sieve_file_script_equals compares 0==0 dev & inode numbers as scripts do not exist. This is why scripts do not have to have same names. Side effect of this is that as missing_a and missing_aa are treated like the same script, only first one is linked in compiled svbin. Which means that if missing_aa reappears later, svbin won't get recompiled. Another manifestation of the issue is when included missing_a script is present when main script is compiled and later missina_a is removed again. During execution, script is not recompiled and it aborts for hash table dupe check: sieve_binary_open->ext_include_binary_script_include->hash_table_insert There are several ways how to solve this. I've tried a few, but there were other corner cases that appeared. One tried way was for example ext_include_generate_include in the 'if (included)' replicate the check sieve_script_is_open(...) from else block, just check flags, no include and set result to 0. Another way to solve this is to return from ext_include_execute_include early if sblock is NULL just before it tries to derefernce it, similarly as it does if it fails 'once' check. Same for dumping in opc_include_dump. Whilst the first approach seems a bit better, it won't stop crashing in existing environments as it won't trigger svbin recompilation and svbin would fail. This means that the second soultion is sort of needed anyway. This still won't solve the problem with different missing scripts as only first one would be able to trigger recompilation when file becomes available. Just including both in the mentioned function won't work either as they would still be evaluated as same and hash table anti-dupe check would trigger abort. So some changes in script comparison seem needed anyway. This is what I've used in my approach. Return early from ext_include_execute_include and opc_include_dump if sblock is NULL and change sieve_file_script_equals to compare also script->location for missing scripts (when inode is 0). Let me know if you need more information. Cheers, Michal Hlavinka

2 1

Ignore specific subfolders within Maildir?
by Ralph Seichter 25 Oct '24

25 Oct '24

Consider Dovecot with "mail_location = maildir:~/.maildir". Is it possible to configure Dovecot to completely ignore directories like ~/.maildir/.foo ? The subdirectory .foo is created by third party software and the name cannot be changed. Because of the leading dot, Dovecot interprets ~/.maildir/.foo as IMAP folder /foo by default, creates the typical subdirectory structure with cur/new/tmp, and exposes ~/.maildir/.foo to IMAP clients. Not being able to change the name or location of .foo, I would like to tell Dovecot to ignore .foo's presence altogether. So far I have not found documentation about how that can be done (or if it can). Helpful pointers are appreciated. -Ralph

3 7

Re: "client_id" in lua script
by Michael Slusarz 25 Oct '24

25 Oct '24

> On 10/25/2024 9:47 AM MDT ecdlguy--- via dovecot <dovecot(a)dovecot.org> wrote: > > Hi, > > I'm trying to block specific MUAs, like e.g. Outlook-App for iOS and Android or Xiaomi Mail / MiMail using a lua script. > While implementing this, I noticed that "client_id" is always nil in the lua script or I am doing something wrong. > This is part of the postfix config: A note that this is explicitly not a supported IMAP ID use-case. Per https://www.rfc-editor.org/rfc/rfc2971.html#section-3: Servers MUST NOT deny access to or refuse service for a client based on information from the ID command. Clients MUST NOT refuse to operate or limit their operation with a server based on the ID response. Client blocking is much better handled via other mechanisms, that may incorporate ID information as a part of the decision making process but does not make a determinative decision based on that string. michael

1 0

"client_id" in lua script
by ecdlguy＠gmail.com 25 Oct '24

25 Oct '24

Hi, I'm trying to block specific MUAs, like e.g. Outlook-App for iOS and Android or Xiaomi Mail / MiMail using a lua script. While implementing this, I noticed that "client_id" is always nil in the lua script or I am doing something wrong. This is part of the postfix config: imap_id_retain = yes imap_id_log = * passdb { driver = lua args = file=/etc/dovecot/lua/badimapid.lua blocking=yes } Logging (imap_id_log = *) works, the dovecot log has entries like: imap(user@domain)<226><pWFZx9siRt+CUwMp>: ID sent: name=Thunderbird, version=115.15.0 Here is the content of the lua-script file: function auth_passdb_lookup(req) req.log_info('badimapid info: user="'..req.user..'", client_id="'..req.client_id..'"') return dovecot.auth.PASSDB_RESULT_NEXT, "" end function script_init() return 0 end function script_deinit() end In principle, "req.client_id" should contain the client id string according to https://doc.dovecot.org/2.3/configuration_manual/authentication/lua_based_a… However, dovecot log reads: auth-worker(163): Error: conn unix:auth-worker (pid=162,uid=401): auth-worker<27>: lua(user@domain): db-lua: lua_pcall(auth_passdb_lookup, 1, 2) failed: /etc/dovecot/lua/badimapid.lua:2: attempt to concatenate a nil value (field 'client_id') Why is client_id nil? Thanks, Thorsten

1 0

BUG: logging bug creeped back in pop3-login
by Laura Steynes 25 Oct '24

25 Oct '24

Oct 20 14:23:29 pop3-login: Info: Disconnected: Disconnected: Too many bad commands (no auth attempts in 0 secs): use....... Double logging of Disconnected

2 2

fixing sieve (?)
by Stefan G. Weichinger 24 Oct '24

24 Oct '24

Yesterday I was told that at a customer the vacation scripts don't work anymore. Looking closer it seems the whole sieve-funtionality is missing (eg. filtering mails into folders). I am in the process of analysing, and don't have a clue yet. Editing scripts via managesieved works (for example from roundcube). The scripts are in the right place, afai understand. I compare this with another setup where things work and don't see the difference. All this worked perfectly until maybe summer. This is Debian 12.7 bookworm, the dovecot repo still points to bullseye. # cat /etc/apt/sources.list.d/dovecot.list deb [arch=amd64 signed-by=/usr/share/keyrings/dovecot.gpg] https://repo.dovecot.org/ce-2.3-latest/debian/bullseye bullseye main There is rspamd in place with postfix if that is somehow relevant. The installation is on premise and has maybe 60-100 mailboxes in now 2 domains. That was the largest change this year: add a so-called "alias domain" in postfixadmin: mailbox(a)new.domain points to mailbox(a)old.domain I assume the sieve scripts should simply be executed as before. The basics of mailing etc work fine! I'd appreciate any pointers. trying to enable debug logs etc, for example I don't get anything into "sieve_user_log" --- # 2.3.21.1 (d492236fa0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.21.1 (49005e73) # OS: Linux 6.1.0-26-amd64 x86_64 Debian 12.7 # Hostname: mail2.my.tld auth_cache_size = 4 M default_process_limit = 300 listen = * mail_home = /mnt/mailboxes/%d/%u mail_location = maildir:~ mail_plugins = " notify quota acl" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags imapsieve vnd.dovecot.imapsieve namespace { inbox = yes location = prefix = separator = / type = private } namespace { location = maildir:/home/vmail/oeffentlich prefix = Oeffentlich/ separator = / subscriptions = yes type = public } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl = vfile:/etc/dovecot/dovecot-acl:cache_secs=300 acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes imapsieve_mailbox1_before = file:/etc/dovecot/sieve/global/learn-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/etc/dovecot/sieve/global/learn-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * sieve = /home/vmail/%d/%u/sieve/.dovecot.sieve sieve_before = /etc/dovecot/sieve/global/spam-global.sieve sieve_dir = /home/vmail/%d/%u/sieve/scripts/ sieve_extensions = +imapflags sieve_global_extensions = +vnd.dovecot.pipe sieve_pipe_bin_dir = /usr/bin sieve_plugins = sieve_imapsieve sieve_extprograms sieve_trace_dir = /tmp/sieve_trace/ sieve_trace_level = commands sieve_user_log = /home/vmail/%d/%u/sieve/sieve_user.log } protocols = imap sieve lmtp service auth { client_limit = 1200 unix_listener auth-client { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0777 } user = root } service imap-login { process_min_avail = 10 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } user = vmail } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = required ssl_ca = </etc/letsencrypt/pfsense/cloud_imap_mail_smtp_LE_cert.ca ssl_cert = </etc/letsencrypt/pfsense/cloud_imap_mail_smtp_LE_cert.fullchain ssl_cipher_list = ALL:HIGH:!SSLv2:!MEDIUM:!LOW:!EXP:!RC4:!MD5:!aNULL:@STRENGTH ssl_key = # hidden, use -P to show it ssl_key_password = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol lmtp { mail_debug = yes mail_plugins = sieve postmaster_address = logs(a)my.tld } protocol imap { mail_max_userip_connections = 20 mail_plugins = " notify quota acl imap_sieve imap_acl" } protocol sieve { managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags }

4 14

Issue with logrotate and dovecot
by jim＠themailshack.com 24 Oct '24

24 Oct '24

Something new here has cropped up, no changes to configs for some time. I've had my dovecot using the default /var/log/dovecot.log location for logging, logrotate used to back this up weekly and logging would continue using this same file. Now when logrotate runs, it correctly backs up the log but then creates a new dovecot.log-yyyymmdd file and dovecot starts logging to that. I'm at a loss as to why this is happening. My postfix and other processes are still using their correct log files, so it seems related only to dovecot. From dovecot/conf.d/10-logging.conf: # Log file to use for error messages. "syslog" logs to syslog, # /dev/stderr logs to stderr. #log_path = syslog log_path = /var/log/dovecot.log From /etc/logrotate.d/dovecot /var/log/dovecot.log { weekly rotate 4 missingok notifempty compress delaycompress sharedscripts postrotate doveadm log reopen endscript } Looks like something happened mid last month but other than periodic software updates nothing has changed for a while here. If I reboot the system, dovecot resumes logging to /var/log/dovecot.log, but as soon as logrotate runs (I did a --force to test) it logs to the new file again. Hoping this may sound familiar to someone on the list.

1 1

Struggling to get sieve running
by nick＠howitts.co.uk 23 Oct '24

23 Oct '24

Hi, I am new to Dovecot and Debian, but I seem to have got Dovecot up and running with local delivery and then switched to LMTP. I am running Debian 12.7 and Webmin 2.202 to try and get things running, but I've had to tweak files. Also Webmin did not install dovecot-lmtpd, dovecot-sieve or dovecot-managesieved, so I did that manually. I have tried following the configuration guide from Dovecot (but there seems to be little there) and other sources I've found on the internet, but there is nothing I have found to get sieve listening on port 4190 when checked with ss. My Dovecot config from `dovecot -n` is: ``` # 2.3.19.1 (9b53102964): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.19 (4eae2f79) # OS: Linux 6.1.0-26-amd64 x86_64 Debian 12.7 # Hostname: my.fqdn auth_username_format = %Ln disable_plaintext_auth = no first_valid_uid = 1000 mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap lmtp service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = </etc/letsencrypt/live/mydomain.com/fullchain.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } protocol lmtp { mail_plugins = " sieve" postmaster_address = nick(a)mydomain.com } protocol lda { mail_plugins = " sieve" } ``` Dovecot packages installed from the Debian repos are: ``` dovecot-core/stable,stable-security,now 1:2.3.19.1+dfsg1-2.1+deb12u1 amd64 [installed] dovecot-imapd/stable,stable-security,now 1:2.3.19.1+dfsg1-2.1+deb12u1 amd64 [installed] dovecot-lmtpd/stable,stable-security,now 1:2.3.19.1+dfsg1-2.1+deb12u1 amd64 [installed] dovecot-managesieved/stable,stable-security,now 1:2.3.19.1+dfsg1-2.1+deb12u1 amd64 [installed] dovecot-pop3d/stable,stable-security,now 1:2.3.19.1+dfsg1-2.1+deb12u1 amd64 [installed] dovecot-sieve/stable,stable-security,now 1:2.3.19.1+dfsg1-2.1+deb12u1 amd64 [installed] ``` Ultimately I intend to run manage sieve from Roundcube which I have already got running, or I may try a Thunderbird addon. Can anyone give me some pointers, please?

3 2