- dovecot - dovecot.org

dovecot-fts-solr Solr9 support
by PGNet Dev 24 Apr '23

24 Apr '23

I run dovecot-2.3.18 , & am deploying Solr search with 'dovecot-fts-solr' I run Solr standalone. When creating a dedicated 'dovecot' core, cp'ing from Dovecot's included solr schemas ls -al /usr/share/doc/dovecot/solr-* -rw-r--r-- 1 root root 12K Dec 3 06:48 /usr/share/doc/dovecot/solr-config-7.7.0.xml -rw-r--r-- 1 root root 2.7K Dec 3 06:48 /usr/share/doc/dovecot/solr-schema-7.7.0.xml -rw-r--r-- 1 root root 2.9K Dec 3 06:48 /usr/share/doc/dovecot/solr-schema.xml is tested only for Solr 7.7.0 https://doc.dovecot.org/configuration_manual/fts/solr/ "The steps described in this page are tested for Solr 7.7.0. For other versions, this these steps may need to be adjusted." which, per solr docs https://solr.apache.org/downloads.html About versions and support Version Description 9.x Current major version for feature releases (STABLE) 8.11.x Previous major version may sometimes receive critical bugfix releases 10 Next major version, yet to be released (UNSTABLE) <8.11 All older versions are End Of Life (EOL) is EOL'd and unsupported. In the dovecot 7.7.0 schema, FastLRUCache/LRUCache are used as the caching class(es), grep class=.*LRU /usr/share/doc/dovecot/solr-config-7.7.0.xml <filterCache class="solr.FastLRUCache" <queryResultCache class="solr.LRUCache" <documentCache class="solr.LRUCache" class="solr.search.LRUCache" For Solr 9, cat CHANGES.txt Upgrade Notes --------------------- ... * SOLR-13817: Deprecate legacy SolrCache implementations. Users are encouraged to transition their configurations to use org.apache.solr.search.CaffeineCache instead. (ab) running solr9 + dovecot-fts-solr search, based on the 7.7.0 schemas, results in errors, failed scan & search. Modifying the config to use CaffeineCache cp solr-config-7.7.0.xml solr-config-9.0.0.xml perl -pi -e '\ s|solr.FastLRUCache|solr.CaffeineCache|g; \ s|solr.LRUCache|solr.CaffeineCache|g; \ s|solr.search.LRUCache|solr.search.CaffeineCache|g;' \ solr-config-9.0.0.xml appears to do the trick. Atm, indexing & search -- both manually triggered, and on delivery receipt to Dovecot store -- is working with no errors (yet). I've not tested exhaustively yet, & don't know if this Cache class change in the schema is both necessary & sufficient for dovecot-fts-solr search use with solr 9.0.0 Given that for Solr "<8.11 All older versions are End Of Life (EOL)" it'd be useful for Dovecot to 'retest' for current, not-EOL'd Solr, and publish/package compatible schemas accordingly.

5 8

Indexer opening emails in write mode ?
by Andre Rodier 23 Apr '23

23 Apr '23

Hi, I have dovecot indexer working inside AppArmor, and I am confused by some error messages: > name="/home/users/andre/mails/maildir/.Sent/cur/1682251668.M110083P109814.pigment,S=1743,W=1776:2,S" pid=111784 > comm="indexer-worker" requested_mask="wd" denied_mask="wd" fsuid=1001 ouid=1001 Why the indexer would open email files in write mode, please ? Thanks for your advice

1 0

sieve traces not logged
by Christian Weiske 23 Apr '23

23 Apr '23

Hello, Some sieve filters do not work as expected and I want to debug why. For that I configured in /etc/dovecot/conf.d/90-sieve.conf sieve_trace_dir = /var/log/dovecot/ sieve_trace_level = matching and created /var/log/dovecot/ and gave it the user+group "Debian-exim". Unfortunately, no logs are written into that direcory. The normal log file in /var/log/dovecot.log contains no information - except that lmtp stored the mail in mailbox "test". The docs[1] only say that those two options need to be enabled. Sieve itself works, which I confirmed by creating a rule that moves mails with subject "test" into a separate folder. What am I doing wrong? ---- $ dovecot --version 2.3.20 (80a5ac675d) Running on Debian 11, packages from https://repo.dovecot.org/ce-2.3-latest/debian/bullseye Full sieve config from "dovecot -n": plugin { imapsieve_mailbox1_before = file:/var/lib/dovecot/sieve/learn-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/var/lib/dovecot/sieve/learn-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * sieve = file:~/sieve;active=~/.dovecot.sieve sieve_default = /var/lib/dovecot/sieve/default.sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_pipe_bin_dir = /etc/dovecot/sieve-bin/ sieve_plugins = sieve_imapsieve sieve_extprograms sieve_trace_debug = no sieve_trace_dir = /var/log/dovecot-sieve sieve_trace_level = matching } [1] https://doc.dovecot.org/configuration_manual/sieve/configuration/#trace-deb… -- Regards/Mit freundlichen Grüßen Christian Weiske -=≡ Geeking around in the name of science since 1982 ≡=-

1 1

Blacklistd
by Doug Hardie 22 Apr '23

22 Apr '23

Are there any plans to interface to blacklistd? -- Doug

6 9

quotas and Trash
by David Koski 21 Apr '23

21 Apr '23

Hello, I am testing quotas with the following configuration: plugin { quota_rule = *:storage=1M quota_rule2 = INBOX.Trash:storage=+100M quota_grace = 10%% } The 1M limit works perfectly but I had to prefix Trash with "INBOX." to get it to stop complaining about an unknown namespace. But now the quota_rule2 is ignored. Once the mailbox reaches 1M, even if it is all in INBOX.Trash, it will fail to deliver due to quota limit. Dovecot: 2.3.13+dfsg1-2+deb11u1:amd64 Debian 11 auth_mechanisms = plain login disable_plaintext_auth = no mail_plugins = quota mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX. separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_plugins = quota quota quota = maildir:User quota quota_grace = 10%% quota_rule = *:storage=1M quota_rule2 = INBOX.Trash:storage=+100M } protocols = " imap pop3" service anvil { unix_listener anvil-auth-penalty { group = vmail mode = 0660 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-client { mode = 0660 } } service imap-login { inet_listener imap { address = 172.16.0.1 } } service pop3-login { inet_listener pop3 { address = 172.16.0.1 } } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl = no ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota quota postmaster_address = dkoski(a)sutinen.com } protocol lda { mail_plugins = quota quota } protocol imap { mail_max_userip_connections = 14 mail_plugins = quota quota imap_quota } protocol pop3 { mail_plugins = quota quota } Regards, David Koski dkoski(a)sutinen.com

3 7

Message searching in Dovecot
by John Gateley 21 Apr '23

21 Apr '23

Hello, For mobile clients (gmail on Android), I need a server side searching solution. A few months ago, I tried getting Solr to work with Dovecot, but failed. The instructions here https://doc.dovecot.org/configuration_manual/fts/solr/ are quite out of date, they reference Debian 8 and 9 (current version 11), and Solr 7.7 (current version 9.2) My Solr experience is pretty much zero. Questions/Requests: * Is there a different tool than Solr I should be using for this? * Does anyone have a recent version of Solr and Debian working? * If you do, would you be willing to help me? * I cannot afford much, this is a home mailserver for my wife and I, but I could pay a small consulting fee. I use ansible to deploy the server, I'd be willing to contribute the playbooks somewhere for documentation, or help with the docs. Thanks! John

5 6

Re: Mailing list is being Spam Filtered by O-365
by White, Daniel E. (GSFC-770.0)[AEGIS] 20 Apr '23

20 Apr '23

From headers: Received: from BL0GCC02FT014.eop-gcc02.prod.protection.outlook.com (2a01:111:f400:7d05::209) by CYXPR09CA0010.outlook.office365.com (2603:10b6:930:d4::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.25 via Frontend Transport; Thu, 20 Apr 2023 11:49:06 +0000 Authentication-Results: spf=softfail (sender IP is 63.88.93.251) smtp.mailfrom=open-xchange.com; dkim=fail (signature did not verify) header.d=open-xchange.com;dmarc=fail action=oreject header.from=open-xchange.com;compauth=none reason=452 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning open-xchange.com discourages use of 63.88.93.251 as permitted sender) From: Aki Tuomi <aki.tuomi(a)open-xchange.com> Date: Thursday, April 20, 2023 at 07:49 Subject: [EXTERNAL] Re: Mailing list is being Spam Filtered by O-365 On 20/04/2023 14:18 EEST White, Daniel E. (GSFC-770.0)[AEGIS] via dovecot <dovecot(a)dovecot.org<mailto:dovecot@dovecot.org>> wrote: Is there any chance that SPF and DKIM records could be added to appear in the headers ? The gubba-mint folks are getting extremely medieval about email security. Um? Those already are added? Or what do you mean? Aki

3 2

Mailing list is being Spam Filtered by O-365
by White, Daniel E. (GSFC-770.0)[AEGIS] 20 Apr '23

20 Apr '23

Is there any chance that SPF and DKIM records could be added to appear in the headers ? The gubba-mint folks are getting extremely medieval about email security.

2 1

Request - extra variable in passdb for logging
by Bogusław Juza 20 Apr '23

20 Apr '23

Hi Staff, I'm creating the dovecot configuration for multiple user passwords e-mail site. It's working well, but I've one problem - I can't log, which password was used. I'm using auth-sql, the query looks like: password_query = \ SELECT `users`.`email` AS `user`, \ `shadow`.`passwd` AS `password`, \ FROM `shadow` WHERE `shadow`.`email` = '%u' AND \ ( ('%r'='127.0.0.1' AND `shadow`.`webmail`<>0) OR \ ('%r'<>'127.0.0.1' AND `shadow`.`imap`<>0 AND \ `shadow`.`hash`='%{sha512;rounds=5000:password}') \ ) LIMIT 1 The hash from random generated application password works as the selector, which password should be checked. It works fine, but in the log I have got only the e-mail and both IP addresses. I need to log one more information - which password was used (`shadow`.`id` column). It would be a great feature to have one more extra variable, which I could set in this query and which goes directly to log and nowhere else. I have tried something like: password_query = \ SELECT CONCAT(`users`.`email`,'#',`shadow`.`id`) AS `user`, ... and then "repair it" in user_query: SELECT `email` AS `user` WHERE `email` = REGEXP_SUBSTR('%u','[^#]+') and it even works well, except postfix-auth, which received email with #number as the username and it was problematic. So I'm kindly asking for this extra variable in next versions of Dovecot ;) Bogusław Juza

2 1

last_login plugin and $remote_ip
by Fabrizio Cuseo 19 Apr '23

19 Apr '23

Good morning. I am planning a dovecot system with: - 3 x glusterfs servers (with 2 volumes, 1 ssd for short term mail, and 1 with bigger hdd for long term archive mail) - 1 x mysql server (another server with active replica will be added) - 3 x mbox servers (with dovecot pop/imap/lmpt/sieve/postfix) - 3 x dovecot proxy/directors for pop3/imap/smtp - 4 x proxmox mail gateway for antispam/antivirus in front of smtp servers - 1 x centralized syslog server All have private ip addresses, and in front there is a firewall with HA_proxy to make high availability and load balancing. My only problem now is using last_login plugin; i have configured on the mailbox servers on pop3/imap, but the ip address that is written on mysql is the proxy/director address, not the real client ip address. No results using real_remote_ip. Apr 19 09:14:31 mailbox-01 dovecot: pop3-login: Login: user=<username(a)domain.it>, method=PLAIN, rip=172.16.27.31, lip=172.16.27.21, mpid=19723, session=<42nHLKv5JsqsEBsf> Apr 19 09:14:31 mailproxy-01 dovecot: pop3-login: proxy(username@domain.it,172.16.27.21:110): Started proxying to 172.16.27.21 (1.978 secs): user=<username(a)domain.it>, method=PLAIN, rip=212.66.96.188, lip=172.16.27.31, session=<u4+wLKv5ZUjUQmC8> Apr 19 09:14:34 mailbox-01 dovecot: pop3(username(a)domain.it)<19723><42nHLKv5JsqsEBsf>: Disconnected: Logged out top=0/0, retr=0/0, del=0/37, size=115779706 Apr 19 09:14:34 mailproxy-01 dovecot: pop3-login: proxy(username@domain.it,172.16.27.21:110): Disconnected by server (0s idle, in=45, out=82): user=<username(a)domain.it>, method=PLAIN, rip=212.66.96.188, lip=172.16.27.31, session=<u4+wLKv5ZUjUQmC8> in db I have last_ip: 172.16.27.31, not 212.66.96.188 ------------- dovecot -n # 2.3.16 (7e2e900c1a): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.16 (09c29328) # OS: Linux 5.15.0-69-generic x86_64 Ubuntu 22.04.2 LTS # Hostname: mailbox-01 auth_default_realm = XXXXXXXX.it default_client_limit = 2500 dict { mysql = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext sieve = mysql:/etc/dovecot/dict-sieve-sql.conf sql = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_api_key = # hidden, use -P to show it first_valid_gid = 89 first_valid_uid = 89 imap_client_workarounds = tb-extra-mailbox-sep delay-newmail login_greeting = Welcome to mail server mail_fsync = always mail_gid = 89 mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = quota mail_privileged_group = mail mail_uid = 89 mailbox_list_index_very_dirty_syncs = yes mdbox_rotate_size = 128 M mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { last_login_dict = proxy::sql last_login_key = # hidden, use -P to show it last_login_precision = ms quota = count:User quota quota_clone_dict = proxy::mysql quota_grace = 50M quota_rule2 = Trash:storage=+100M quota_vsizes = yes quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = dict:proxy::sieve;name=active sieve_extensions = +vacation-seconds sieve_vacation_default_period = 7d sieve_vacation_max_period = 30d sieve_vacation_min_period = 1h } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh protocols = " imap lmtp pop3" service dict { unix_listener dict { group = mail2023 mode = 0660 user = mail2023 } } service doveadm { inet_listener { port = 2425 } inet_listener http { port = 8080 } unix_listener doveadm-server { user = mail2023 } } service imap { process_limit = 1024 } service lmtp { inet_listener lmtp { port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = mail2023 mode = 0666 user = mail2023 } } service pop3 { process_limit = 250 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = mail2023 } user = mail2023 } service stats { unix_listener stats-reader { group = mail2023 mode = 0660 user = mail2023 } unix_listener stats-writer { group = mail2023 mode = 0660 user = mail2023 } } ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota sieve quota quota_clone } protocol !indexer-worker { mail_vsize_bg_after_count = 100 } protocol lda { mail_plugins = quota sieve quota quota_clone } protocol imap { mail_max_userip_connections = 10 mail_plugins = quota quota imap_quota quota_clone last_login } protocol pop3 { mail_max_userip_connections = 2 mail_plugins = quota quota quota_clone last_login } in dovecot-dict-sql.conf.ext there is: map { pattern = shared/last-login/$service/$user/$remote_ip table = mail_last_login value_field = last_access value_type = uint fields { userid = $user service = $service last_ip = $remote_ip } } -- --- Fabrizio Cuseo - mailto:f.cuseo@panservice.it

2 2