- dovecot - dovecot.org

Message searching in Dovecot
by John Gateley 21 Apr '23

21 Apr '23

Hello, For mobile clients (gmail on Android), I need a server side searching solution. A few months ago, I tried getting Solr to work with Dovecot, but failed. The instructions here https://doc.dovecot.org/configuration_manual/fts/solr/ are quite out of date, they reference Debian 8 and 9 (current version 11), and Solr 7.7 (current version 9.2) My Solr experience is pretty much zero. Questions/Requests: * Is there a different tool than Solr I should be using for this? * Does anyone have a recent version of Solr and Debian working? * If you do, would you be willing to help me? * I cannot afford much, this is a home mailserver for my wife and I, but I could pay a small consulting fee. I use ansible to deploy the server, I'd be willing to contribute the playbooks somewhere for documentation, or help with the docs. Thanks! John

5 6

Re: Mailing list is being Spam Filtered by O-365
by White, Daniel E. (GSFC-770.0)[AEGIS] 20 Apr '23

20 Apr '23

From headers: Received: from BL0GCC02FT014.eop-gcc02.prod.protection.outlook.com (2a01:111:f400:7d05::209) by CYXPR09CA0010.outlook.office365.com (2603:10b6:930:d4::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.25 via Frontend Transport; Thu, 20 Apr 2023 11:49:06 +0000 Authentication-Results: spf=softfail (sender IP is 63.88.93.251) smtp.mailfrom=open-xchange.com; dkim=fail (signature did not verify) header.d=open-xchange.com;dmarc=fail action=oreject header.from=open-xchange.com;compauth=none reason=452 Received-SPF: SoftFail (protection.outlook.com: domain of transitioning open-xchange.com discourages use of 63.88.93.251 as permitted sender) From: Aki Tuomi <aki.tuomi(a)open-xchange.com> Date: Thursday, April 20, 2023 at 07:49 Subject: [EXTERNAL] Re: Mailing list is being Spam Filtered by O-365 On 20/04/2023 14:18 EEST White, Daniel E. (GSFC-770.0)[AEGIS] via dovecot <dovecot(a)dovecot.org<mailto:dovecot@dovecot.org>> wrote: Is there any chance that SPF and DKIM records could be added to appear in the headers ? The gubba-mint folks are getting extremely medieval about email security. Um? Those already are added? Or what do you mean? Aki

3 2

Mailing list is being Spam Filtered by O-365
by White, Daniel E. (GSFC-770.0)[AEGIS] 20 Apr '23

20 Apr '23

Is there any chance that SPF and DKIM records could be added to appear in the headers ? The gubba-mint folks are getting extremely medieval about email security.

2 1

Request - extra variable in passdb for logging
by Bogusław Juza 20 Apr '23

20 Apr '23

Hi Staff, I'm creating the dovecot configuration for multiple user passwords e-mail site. It's working well, but I've one problem - I can't log, which password was used. I'm using auth-sql, the query looks like: password_query = \ SELECT `users`.`email` AS `user`, \ `shadow`.`passwd` AS `password`, \ FROM `shadow` WHERE `shadow`.`email` = '%u' AND \ ( ('%r'='127.0.0.1' AND `shadow`.`webmail`<>0) OR \ ('%r'<>'127.0.0.1' AND `shadow`.`imap`<>0 AND \ `shadow`.`hash`='%{sha512;rounds=5000:password}') \ ) LIMIT 1 The hash from random generated application password works as the selector, which password should be checked. It works fine, but in the log I have got only the e-mail and both IP addresses. I need to log one more information - which password was used (`shadow`.`id` column). It would be a great feature to have one more extra variable, which I could set in this query and which goes directly to log and nowhere else. I have tried something like: password_query = \ SELECT CONCAT(`users`.`email`,'#',`shadow`.`id`) AS `user`, ... and then "repair it" in user_query: SELECT `email` AS `user` WHERE `email` = REGEXP_SUBSTR('%u','[^#]+') and it even works well, except postfix-auth, which received email with #number as the username and it was problematic. So I'm kindly asking for this extra variable in next versions of Dovecot ;) Bogusław Juza

2 1

mbox to maildir migration, same server?
by kjohnson＠eclypse.org 20 Apr '23

20 Apr '23

What is the recommended procedure to migrate from mbox to maildir on a server already running dovecot? Ideally, this procedure would convert messages in an existing mbox INBOX and in ~/mail mboxes (as defined in my dovecot configuration) to storage in a ~/Maildir. Some of the INBOXs are rather large -- 100's of MB (one of the reasons for migrating). What I have tried has not worked for me. Below, I describe what I tried, how I tested before I tried, what went wrong, and the current setup (pre-migration). What I tried: ============= stop exim4 service dovecot stop Make dovecot listen only locally, not to external connections. /etc/dovecot/local.conf protocol imap { listen = 127.0.0.1:143 ssl_listen = 127.0.0.1:993 } protocol pop3 { listen = 127.0.0.1:110 ssl_listen = 127.0.0.1:995 } Modify dovecot configuration for maildir: Watch capitalization! #in 10-mail.conf: mail_location = maildir:/home/%u/Maildir namespace inbox { separator = / } dovecot -n, to confirm results and perform limited syntax checking. service dovecot start for each user, in a bash script: dsync -v -u "$U" mirror "mbox:/home/$U/mail:INBOX=/var/mail/$U" (if the conversion worked for all users, then reconfigure exim to use maildir and restart it; reconfigure dovecot to listen to external interfaces and restart.) How I tested: ============= Before trying this on the real server, I set up a matching (same versions of Debian, Exim, and Dovecot) VM using mbox, with a couple of accounts. One using POP3, one using IMAP. The IMAP account was accessed from Claws and from Outlook. The test accounts had perhaps a dozen messages on the server. I tried the conversion procedure, and everything I tried worked flawlessly. (Yes, next time I will move some real inbox files from the live server). What went wrong: ================ I encountered two types of problems here. The first was several dsync warnings (username changed): dsync(example): Warning: Mailbox changes caused a desync. You may want to run dsync again: Remote lost mailbox GUID 21bfa11a81f11763ac150000d09efc50 (maybe it was just deleted?) As I understand it, this can result in some or all messages being downloaded again by clients. My understanding may be wrong. I also don't understand how the mailbox could have changed -- exim4 was stopped, dovecot is only listening on 127.0.0.1, and I am the only user who uses a local mail client on that system, and this was not my account. The second problem I encountered seems to match Debian Bug report logs - #1011238. In part of the discussion of that report, the reporter mentions adding -1 to the command, as shown: dsync -v -u "$U" mirror -1 "mbox:/home/$U/mail:INBOX=/var/mail/$U" While this seemed to eliminate the dsync panic of #1011238, as far as I can tell, no old messages were actually moved from the old mbox files to the new maildirs. At that point I restored the backup mbox files and undid the changes to dovecot and exim4. Current setup: ============== Debian 11.4 Exim 4.94.2-7 dovecot 1:2.3.13+dfsg1-2 The current setup with Dovecot was a successful migration from IMAP-UW in December 2020. Output from dovecot -n (the list of trusted networks has been removed) # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) # OS: Linux 5.10.0-16-686-pae i686 Debian 11.4 # Hostname: localhost auth_verbose = yes mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } pop3_uidl_format = %08Xv%08Xu protocols = " imap pop3" ssl_cert = </etc/dovecot/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } Thank you to anyone who read this far. Ken

4 4

last_login plugin and $remote_ip
by Fabrizio Cuseo 19 Apr '23

19 Apr '23

Good morning. I am planning a dovecot system with: - 3 x glusterfs servers (with 2 volumes, 1 ssd for short term mail, and 1 with bigger hdd for long term archive mail) - 1 x mysql server (another server with active replica will be added) - 3 x mbox servers (with dovecot pop/imap/lmpt/sieve/postfix) - 3 x dovecot proxy/directors for pop3/imap/smtp - 4 x proxmox mail gateway for antispam/antivirus in front of smtp servers - 1 x centralized syslog server All have private ip addresses, and in front there is a firewall with HA_proxy to make high availability and load balancing. My only problem now is using last_login plugin; i have configured on the mailbox servers on pop3/imap, but the ip address that is written on mysql is the proxy/director address, not the real client ip address. No results using real_remote_ip. Apr 19 09:14:31 mailbox-01 dovecot: pop3-login: Login: user=<username(a)domain.it>, method=PLAIN, rip=172.16.27.31, lip=172.16.27.21, mpid=19723, session=<42nHLKv5JsqsEBsf> Apr 19 09:14:31 mailproxy-01 dovecot: pop3-login: proxy(username@domain.it,172.16.27.21:110): Started proxying to 172.16.27.21 (1.978 secs): user=<username(a)domain.it>, method=PLAIN, rip=212.66.96.188, lip=172.16.27.31, session=<u4+wLKv5ZUjUQmC8> Apr 19 09:14:34 mailbox-01 dovecot: pop3(username(a)domain.it)<19723><42nHLKv5JsqsEBsf>: Disconnected: Logged out top=0/0, retr=0/0, del=0/37, size=115779706 Apr 19 09:14:34 mailproxy-01 dovecot: pop3-login: proxy(username@domain.it,172.16.27.21:110): Disconnected by server (0s idle, in=45, out=82): user=<username(a)domain.it>, method=PLAIN, rip=212.66.96.188, lip=172.16.27.31, session=<u4+wLKv5ZUjUQmC8> in db I have last_ip: 172.16.27.31, not 212.66.96.188 ------------- dovecot -n # 2.3.16 (7e2e900c1a): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.16 (09c29328) # OS: Linux 5.15.0-69-generic x86_64 Ubuntu 22.04.2 LTS # Hostname: mailbox-01 auth_default_realm = XXXXXXXX.it default_client_limit = 2500 dict { mysql = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext sieve = mysql:/etc/dovecot/dict-sieve-sql.conf sql = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_api_key = # hidden, use -P to show it first_valid_gid = 89 first_valid_uid = 89 imap_client_workarounds = tb-extra-mailbox-sep delay-newmail login_greeting = Welcome to mail server mail_fsync = always mail_gid = 89 mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = quota mail_privileged_group = mail mail_uid = 89 mailbox_list_index_very_dirty_syncs = yes mdbox_rotate_size = 128 M mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { last_login_dict = proxy::sql last_login_key = # hidden, use -P to show it last_login_precision = ms quota = count:User quota quota_clone_dict = proxy::mysql quota_grace = 50M quota_rule2 = Trash:storage=+100M quota_vsizes = yes quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = dict:proxy::sieve;name=active sieve_extensions = +vacation-seconds sieve_vacation_default_period = 7d sieve_vacation_max_period = 30d sieve_vacation_min_period = 1h } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh protocols = " imap lmtp pop3" service dict { unix_listener dict { group = mail2023 mode = 0660 user = mail2023 } } service doveadm { inet_listener { port = 2425 } inet_listener http { port = 8080 } unix_listener doveadm-server { user = mail2023 } } service imap { process_limit = 1024 } service lmtp { inet_listener lmtp { port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = mail2023 mode = 0666 user = mail2023 } } service pop3 { process_limit = 250 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = mail2023 } user = mail2023 } service stats { unix_listener stats-reader { group = mail2023 mode = 0660 user = mail2023 } unix_listener stats-writer { group = mail2023 mode = 0660 user = mail2023 } } ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota sieve quota quota_clone } protocol !indexer-worker { mail_vsize_bg_after_count = 100 } protocol lda { mail_plugins = quota sieve quota quota_clone } protocol imap { mail_max_userip_connections = 10 mail_plugins = quota quota imap_quota quota_clone last_login } protocol pop3 { mail_max_userip_connections = 2 mail_plugins = quota quota quota_clone last_login } in dovecot-dict-sql.conf.ext there is: map { pattern = shared/last-login/$service/$user/$remote_ip table = mail_last_login value_field = last_access value_type = uint fields { userid = $user service = $service last_ip = $remote_ip } } -- --- Fabrizio Cuseo - mailto:f.cuseo@panservice.it

2 2

Re: Mailing list update
by Peter Wienemann 19 Apr '23

19 Apr '23

Dear Aki, On 12.04.23 13:31, Aki Tuomi via dovecot wrote: > We finally managed to move from mailman2 to mailman3, and mail archives are now at https://dovecot.org/mailman3/archives and they can be now searched, too. thanks for this. > Please let us know if you have face any issues with the mailing list and we'll look into it. Is there a way to keep old links to archived messages (such as https://dovecot.org/pipermail/dovecot/<year>-<month>/<message-id>.html) intact? Best regards, Peter

3 3

Custom SASL authentication
by David Koski 19 Apr '23

19 Apr '23

Hello, I'm looking for a good way to apply a custom hash to passwords. My hope is to add passwords to a (MySQL) database: INSERT INTO users (user='joblo', pass=MYHASH('plain-password').. For SASL authentication, my thought first was to apply the same hash to the issued password and compare it with the hashed password in the database. I soon discovered the sql driver supplied by Dovecot doesn't provide that ability, unless I'm missing something. I'm looking for documentation on how to implement a custom authentication script if needed. Regards, David Koski dkoski(a)sutinen.com

4 6

imap Error: rename failed: Too many links
by Andrey Repin 18 Apr '23

18 Apr '23

Greetings, All! I'm facing a problem with one particular directory in a mailbox. The exact error line is: Apr 7 12:00:21 mxs dovecot: imap(vera): Error: rename(/home/vera/.mail/INBOX/Майя Николаевна/new/1678353168.M710968P25673.mxs.ads.ccenter.lan,S=5442,W=5513:2,ST, /home/vera/.mail/INBOX/Майя Николаевна/cur/1678353168.M710968P25673.mxs.ads.ccenter.lan,S=5442,W=5513:2,ST) failed: Too many links The rest of the mailbox behave correctly, as I can tell, but this specific subdir fails to report unread mail correctly. What are the options to fix it? # dovecot --version 2.2.22 (fe789d2) Config is in attachment, but I doubt it is of any interest. -- With best regards, Andrey Repin Friday, April 7, 2023 16:37:09 Sorry for my terrible english...

4 6

Maildir compression
by Jose David Bravo A 18 Apr '23

18 Apr '23

Hello, I would like to compress the maildir of some of our users, I'm thinking to use this script: [ https://github.com/George-NG/dovecot-maildir-compress/blob/master/dovecot-m… | https://github.com/George-NG/dovecot-maildir-compress/blob/master/dovecot-m… ] My question is, how dovecot will calculate the quota for that user after the compression? will dovecot calculate the quota of that user using the real size of the message file after compression, or the plain size before compression? Is it necessary to recreate the index of that maildir after the compression is made? If so, what command should I use? Thank you! Jose Bravo

2 1