April 2023 - dovecot - dovecot.org

Indexer opening emails in write mode ?
by Andre Rodier 23 Apr '23

23 Apr '23

Hi, I have dovecot indexer working inside AppArmor, and I am confused by some error messages: > name="/home/users/andre/mails/maildir/.Sent/cur/1682251668.M110083P109814.pigment,S=1743,W=1776:2,S" pid=111784 > comm="indexer-worker" requested_mask="wd" denied_mask="wd" fsuid=1001 ouid=1001 Why the indexer would open email files in write mode, please ? Thanks for your advice

1 0

sieve traces not logged
by Christian Weiske 23 Apr '23

23 Apr '23

Hello, Some sieve filters do not work as expected and I want to debug why. For that I configured in /etc/dovecot/conf.d/90-sieve.conf sieve_trace_dir = /var/log/dovecot/ sieve_trace_level = matching and created /var/log/dovecot/ and gave it the user+group "Debian-exim". Unfortunately, no logs are written into that direcory. The normal log file in /var/log/dovecot.log contains no information - except that lmtp stored the mail in mailbox "test". The docs[1] only say that those two options need to be enabled. Sieve itself works, which I confirmed by creating a rule that moves mails with subject "test" into a separate folder. What am I doing wrong? ---- $ dovecot --version 2.3.20 (80a5ac675d) Running on Debian 11, packages from https://repo.dovecot.org/ce-2.3-latest/debian/bullseye Full sieve config from "dovecot -n": plugin { imapsieve_mailbox1_before = file:/var/lib/dovecot/sieve/learn-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/var/lib/dovecot/sieve/learn-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * sieve = file:~/sieve;active=~/.dovecot.sieve sieve_default = /var/lib/dovecot/sieve/default.sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_pipe_bin_dir = /etc/dovecot/sieve-bin/ sieve_plugins = sieve_imapsieve sieve_extprograms sieve_trace_debug = no sieve_trace_dir = /var/log/dovecot-sieve sieve_trace_level = matching } [1] https://doc.dovecot.org/configuration_manual/sieve/configuration/#trace-deb… -- Regards/Mit freundlichen Grüßen Christian Weiske -=≡ Geeking around in the name of science since 1982 ≡=-

1 1

Blacklistd
by Doug Hardie 22 Apr '23

22 Apr '23

Are there any plans to interface to blacklistd? -- Doug

6 9

quotas and Trash
by David Koski 22 Apr '23

22 Apr '23

Hello, I am testing quotas with the following configuration: plugin { quota_rule = *:storage=1M quota_rule2 = INBOX.Trash:storage=+100M quota_grace = 10%% } The 1M limit works perfectly but I had to prefix Trash with "INBOX." to get it to stop complaining about an unknown namespace. But now the quota_rule2 is ignored. Once the mailbox reaches 1M, even if it is all in INBOX.Trash, it will fail to deliver due to quota limit. Dovecot: 2.3.13+dfsg1-2+deb11u1:amd64 Debian 11 auth_mechanisms = plain login disable_plaintext_auth = no mail_plugins = quota mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX. separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_plugins = quota quota quota = maildir:User quota quota_grace = 10%% quota_rule = *:storage=1M quota_rule2 = INBOX.Trash:storage=+100M } protocols = " imap pop3" service anvil { unix_listener anvil-auth-penalty { group = vmail mode = 0660 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-client { mode = 0660 } } service imap-login { inet_listener imap { address = 172.16.0.1 } } service pop3-login { inet_listener pop3 { address = 172.16.0.1 } } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl = no ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota quota postmaster_address = dkoski(a)sutinen.com } protocol lda { mail_plugins = quota quota } protocol imap { mail_max_userip_connections = 14 mail_plugins = quota quota imap_quota } protocol pop3 { mail_plugins = quota quota } Regards, David Koski dkoski(a)sutinen.com

3 7

Message searching in Dovecot
by John Gateley 21 Apr '23

21 Apr '23

Hello, For mobile clients (gmail on Android), I need a server side searching solution. A few months ago, I tried getting Solr to work with Dovecot, but failed. The instructions here https://doc.dovecot.org/configuration_manual/fts/solr/ are quite out of date, they reference Debian 8 and 9 (current version 11), and Solr 7.7 (current version 9.2) My Solr experience is pretty much zero. Questions/Requests: * Is there a different tool than Solr I should be using for this? * Does anyone have a recent version of Solr and Debian working? * If you do, would you be willing to help me? * I cannot afford much, this is a home mailserver for my wife and I, but I could pay a small consulting fee. I use ansible to deploy the server, I'd be willing to contribute the playbooks somewhere for documentation, or help with the docs. Thanks! John

5 6

Re: Mailing list is being Spam Filtered by O-365
by White, Daniel E. (GSFC-770.0)[AEGIS] 20 Apr '23

20 Apr '23

From headers: Received: from BL0GCC02FT014.eop-gcc02.prod.protection.outlook.com (2a01:111:f400:7d05::209) by CYXPR09CA0010.outlook.office365.com (2603:10b6:930:d4::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.25 via Frontend Transport; Thu, 20 Apr 2023 11:49:06 +0000 Authentication-Results: spf=softfail (sender IP is 63.88.93.251) smtp.mailfrom=open-xchange.com; dkim=fail (signature did not verify) header.d=open-xchange.com;dmarc=fail action=oreject header.from=open-xchange.com;compauth=none reason=452 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning open-xchange.com discourages use of 63.88.93.251 as permitted sender) From: Aki Tuomi <aki.tuomi(a)open-xchange.com> Date: Thursday, April 20, 2023 at 07:49 Subject: [EXTERNAL] Re: Mailing list is being Spam Filtered by O-365 On 20/04/2023 14:18 EEST White, Daniel E. (GSFC-770.0)[AEGIS] via dovecot <dovecot(a)dovecot.org<mailto:dovecot@dovecot.org>> wrote: Is there any chance that SPF and DKIM records could be added to appear in the headers ? The gubba-mint folks are getting extremely medieval about email security. Um? Those already are added? Or what do you mean? Aki

3 2

Mailing list is being Spam Filtered by O-365
by White, Daniel E. (GSFC-770.0)[AEGIS] 20 Apr '23

20 Apr '23

Is there any chance that SPF and DKIM records could be added to appear in the headers ? The gubba-mint folks are getting extremely medieval about email security.

2 1

Request - extra variable in passdb for logging
by Bogusław Juza 20 Apr '23

20 Apr '23

Hi Staff, I'm creating the dovecot configuration for multiple user passwords e-mail site. It's working well, but I've one problem - I can't log, which password was used. I'm using auth-sql, the query looks like: password_query = \ SELECT `users`.`email` AS `user`, \ `shadow`.`passwd` AS `password`, \ FROM `shadow` WHERE `shadow`.`email` = '%u' AND \ ( ('%r'='127.0.0.1' AND `shadow`.`webmail`<>0) OR \ ('%r'<>'127.0.0.1' AND `shadow`.`imap`<>0 AND \ `shadow`.`hash`='%{sha512;rounds=5000:password}') \ ) LIMIT 1 The hash from random generated application password works as the selector, which password should be checked. It works fine, but in the log I have got only the e-mail and both IP addresses. I need to log one more information - which password was used (`shadow`.`id` column). It would be a great feature to have one more extra variable, which I could set in this query and which goes directly to log and nowhere else. I have tried something like: password_query = \ SELECT CONCAT(`users`.`email`,'#',`shadow`.`id`) AS `user`, ... and then "repair it" in user_query: SELECT `email` AS `user` WHERE `email` = REGEXP_SUBSTR('%u','[^#]+') and it even works well, except postfix-auth, which received email with #number as the username and it was problematic. So I'm kindly asking for this extra variable in next versions of Dovecot ;) Bogusław Juza

2 1

mbox to maildir migration, same server?
by kjohnson＠eclypse.org 20 Apr '23

20 Apr '23

What is the recommended procedure to migrate from mbox to maildir on a server already running dovecot? Ideally, this procedure would convert messages in an existing mbox INBOX and in ~/mail mboxes (as defined in my dovecot configuration) to storage in a ~/Maildir. Some of the INBOXs are rather large -- 100's of MB (one of the reasons for migrating). What I have tried has not worked for me. Below, I describe what I tried, how I tested before I tried, what went wrong, and the current setup (pre-migration). What I tried: ============= stop exim4 service dovecot stop Make dovecot listen only locally, not to external connections. /etc/dovecot/local.conf protocol imap { listen = 127.0.0.1:143 ssl_listen = 127.0.0.1:993 } protocol pop3 { listen = 127.0.0.1:110 ssl_listen = 127.0.0.1:995 } Modify dovecot configuration for maildir: Watch capitalization! #in 10-mail.conf: mail_location = maildir:/home/%u/Maildir namespace inbox { separator = / } dovecot -n, to confirm results and perform limited syntax checking. service dovecot start for each user, in a bash script: dsync -v -u "$U" mirror "mbox:/home/$U/mail:INBOX=/var/mail/$U" (if the conversion worked for all users, then reconfigure exim to use maildir and restart it; reconfigure dovecot to listen to external interfaces and restart.) How I tested: ============= Before trying this on the real server, I set up a matching (same versions of Debian, Exim, and Dovecot) VM using mbox, with a couple of accounts. One using POP3, one using IMAP. The IMAP account was accessed from Claws and from Outlook. The test accounts had perhaps a dozen messages on the server. I tried the conversion procedure, and everything I tried worked flawlessly. (Yes, next time I will move some real inbox files from the live server). What went wrong: ================ I encountered two types of problems here. The first was several dsync warnings (username changed): dsync(example): Warning: Mailbox changes caused a desync. You may want to run dsync again: Remote lost mailbox GUID 21bfa11a81f11763ac150000d09efc50 (maybe it was just deleted?) As I understand it, this can result in some or all messages being downloaded again by clients. My understanding may be wrong. I also don't understand how the mailbox could have changed -- exim4 was stopped, dovecot is only listening on 127.0.0.1, and I am the only user who uses a local mail client on that system, and this was not my account. The second problem I encountered seems to match Debian Bug report logs - #1011238. In part of the discussion of that report, the reporter mentions adding -1 to the command, as shown: dsync -v -u "$U" mirror -1 "mbox:/home/$U/mail:INBOX=/var/mail/$U" While this seemed to eliminate the dsync panic of #1011238, as far as I can tell, no old messages were actually moved from the old mbox files to the new maildirs. At that point I restored the backup mbox files and undid the changes to dovecot and exim4. Current setup: ============== Debian 11.4 Exim 4.94.2-7 dovecot 1:2.3.13+dfsg1-2 The current setup with Dovecot was a successful migration from IMAP-UW in December 2020. Output from dovecot -n (the list of trusted networks has been removed) # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) # OS: Linux 5.10.0-16-686-pae i686 Debian 11.4 # Hostname: localhost auth_verbose = yes mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } pop3_uidl_format = %08Xv%08Xu protocols = " imap pop3" ssl_cert = </etc/dovecot/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } Thank you to anyone who read this far. Ken

4 4

last_login plugin and $remote_ip
by Fabrizio Cuseo 19 Apr '23

19 Apr '23

Good morning. I am planning a dovecot system with: - 3 x glusterfs servers (with 2 volumes, 1 ssd for short term mail, and 1 with bigger hdd for long term archive mail) - 1 x mysql server (another server with active replica will be added) - 3 x mbox servers (with dovecot pop/imap/lmpt/sieve/postfix) - 3 x dovecot proxy/directors for pop3/imap/smtp - 4 x proxmox mail gateway for antispam/antivirus in front of smtp servers - 1 x centralized syslog server All have private ip addresses, and in front there is a firewall with HA_proxy to make high availability and load balancing. My only problem now is using last_login plugin; i have configured on the mailbox servers on pop3/imap, but the ip address that is written on mysql is the proxy/director address, not the real client ip address. No results using real_remote_ip. Apr 19 09:14:31 mailbox-01 dovecot: pop3-login: Login: user=<username(a)domain.it>, method=PLAIN, rip=172.16.27.31, lip=172.16.27.21, mpid=19723, session=<42nHLKv5JsqsEBsf> Apr 19 09:14:31 mailproxy-01 dovecot: pop3-login: proxy(username@domain.it,172.16.27.21:110): Started proxying to 172.16.27.21 (1.978 secs): user=<username(a)domain.it>, method=PLAIN, rip=212.66.96.188, lip=172.16.27.31, session=<u4+wLKv5ZUjUQmC8> Apr 19 09:14:34 mailbox-01 dovecot: pop3(username(a)domain.it)<19723><42nHLKv5JsqsEBsf>: Disconnected: Logged out top=0/0, retr=0/0, del=0/37, size=115779706 Apr 19 09:14:34 mailproxy-01 dovecot: pop3-login: proxy(username@domain.it,172.16.27.21:110): Disconnected by server (0s idle, in=45, out=82): user=<username(a)domain.it>, method=PLAIN, rip=212.66.96.188, lip=172.16.27.31, session=<u4+wLKv5ZUjUQmC8> in db I have last_ip: 172.16.27.31, not 212.66.96.188 ------------- dovecot -n # 2.3.16 (7e2e900c1a): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.16 (09c29328) # OS: Linux 5.15.0-69-generic x86_64 Ubuntu 22.04.2 LTS # Hostname: mailbox-01 auth_default_realm = XXXXXXXX.it default_client_limit = 2500 dict { mysql = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext sieve = mysql:/etc/dovecot/dict-sieve-sql.conf sql = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_api_key = # hidden, use -P to show it first_valid_gid = 89 first_valid_uid = 89 imap_client_workarounds = tb-extra-mailbox-sep delay-newmail login_greeting = Welcome to mail server mail_fsync = always mail_gid = 89 mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = quota mail_privileged_group = mail mail_uid = 89 mailbox_list_index_very_dirty_syncs = yes mdbox_rotate_size = 128 M mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { last_login_dict = proxy::sql last_login_key = # hidden, use -P to show it last_login_precision = ms quota = count:User quota quota_clone_dict = proxy::mysql quota_grace = 50M quota_rule2 = Trash:storage=+100M quota_vsizes = yes quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = dict:proxy::sieve;name=active sieve_extensions = +vacation-seconds sieve_vacation_default_period = 7d sieve_vacation_max_period = 30d sieve_vacation_min_period = 1h } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh protocols = " imap lmtp pop3" service dict { unix_listener dict { group = mail2023 mode = 0660 user = mail2023 } } service doveadm { inet_listener { port = 2425 } inet_listener http { port = 8080 } unix_listener doveadm-server { user = mail2023 } } service imap { process_limit = 1024 } service lmtp { inet_listener lmtp { port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = mail2023 mode = 0666 user = mail2023 } } service pop3 { process_limit = 250 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = mail2023 } user = mail2023 } service stats { unix_listener stats-reader { group = mail2023 mode = 0660 user = mail2023 } unix_listener stats-writer { group = mail2023 mode = 0660 user = mail2023 } } ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota sieve quota quota_clone } protocol !indexer-worker { mail_vsize_bg_after_count = 100 } protocol lda { mail_plugins = quota sieve quota quota_clone } protocol imap { mail_max_userip_connections = 10 mail_plugins = quota quota imap_quota quota_clone last_login } protocol pop3 { mail_max_userip_connections = 2 mail_plugins = quota quota quota_clone last_login } in dovecot-dict-sql.conf.ext there is: map { pattern = shared/last-login/$service/$user/$remote_ip table = mail_last_login value_field = last_access value_type = uint fields { userid = $user service = $service last_ip = $remote_ip } } -- --- Fabrizio Cuseo - mailto:f.cuseo@panservice.it

2 2