December 2022 - dovecot

dovecot-fts-solr Solr9 support
by PGNet Dev 24 Apr '23

24 Apr '23

I run dovecot-2.3.18 , & am deploying Solr search with 'dovecot-fts-solr' I run Solr standalone. When creating a dedicated 'dovecot' core, cp'ing from Dovecot's included solr schemas ls -al /usr/share/doc/dovecot/solr-* -rw-r--r-- 1 root root 12K Dec 3 06:48 /usr/share/doc/dovecot/solr-config-7.7.0.xml -rw-r--r-- 1 root root 2.7K Dec 3 06:48 /usr/share/doc/dovecot/solr-schema-7.7.0.xml -rw-r--r-- 1 root root 2.9K Dec 3 06:48 /usr/share/doc/dovecot/solr-schema.xml is tested only for Solr 7.7.0 https://doc.dovecot.org/configuration_manual/fts/solr/ "The steps described in this page are tested for Solr 7.7.0. For other versions, this these steps may need to be adjusted." which, per solr docs https://solr.apache.org/downloads.html About versions and support Version Description 9.x Current major version for feature releases (STABLE) 8.11.x Previous major version may sometimes receive critical bugfix releases 10 Next major version, yet to be released (UNSTABLE) <8.11 All older versions are End Of Life (EOL) is EOL'd and unsupported. In the dovecot 7.7.0 schema, FastLRUCache/LRUCache are used as the caching class(es), grep class=.*LRU /usr/share/doc/dovecot/solr-config-7.7.0.xml <filterCache class="solr.FastLRUCache" <queryResultCache class="solr.LRUCache" <documentCache class="solr.LRUCache" class="solr.search.LRUCache" For Solr 9, cat CHANGES.txt Upgrade Notes --------------------- ... * SOLR-13817: Deprecate legacy SolrCache implementations. Users are encouraged to transition their configurations to use org.apache.solr.search.CaffeineCache instead. (ab) running solr9 + dovecot-fts-solr search, based on the 7.7.0 schemas, results in errors, failed scan & search. Modifying the config to use CaffeineCache cp solr-config-7.7.0.xml solr-config-9.0.0.xml perl -pi -e '\ s|solr.FastLRUCache|solr.CaffeineCache|g; \ s|solr.LRUCache|solr.CaffeineCache|g; \ s|solr.search.LRUCache|solr.search.CaffeineCache|g;' \ solr-config-9.0.0.xml appears to do the trick. Atm, indexing & search -- both manually triggered, and on delivery receipt to Dovecot store -- is working with no errors (yet). I've not tested exhaustively yet, & don't know if this Cache class change in the schema is both necessary & sufficient for dovecot-fts-solr search use with solr 9.0.0 Given that for Solr "<8.11 All older versions are End Of Life (EOL)" it'd be useful for Dovecot to 'retest' for current, not-EOL'd Solr, and publish/package compatible schemas accordingly.

5 8

decrypt.rb
by Dave 11 Apr '23

11 Apr '23

Per the Dovecot site here: https://wiki.dovecot.org/Plugins/MailCrypt ... the "decrypt.rb" ruby script can be used to decrypt a Dovecot-encrypted message file from the command line. The script sort of runs successfully for me, in the sense that it doesn't error out, but it doesn't show the decrypted message. I've called it like so: decrypt.rb -k <PATH_TO_DOVECOT_KEYFILE> -f ./encrypted_message But when I do it just displays the info below and exits (I've subbed <DATA> tags for the actual data): ------------------[OUTPUT]----------------------- Key(s) (total: 1) - Key type : EC - Key digest: <DATA1> - Peer key : <DATA2> - Encrypted : <DATA3> - Kd hash : <DATA4> ------------------------------------------------------- I've tried specifying an output file as well, per the script's command line options, but the output file is 0 bytes. Does anyone have any suggestions? I *think* I'm using it the way it's intended to be used, but maybe I'm not?! -Dave

5 14

ot: how to t/s TBird problems ?
by Voytek Eymont 22 Mar '23

22 Mar '23

I have a Dovecot/Postfix/MariaDB on a Centos, just have a user ask me: ------------------ I recently upgraded my Thunderbird email client and have experienced problems since. It appears that when Tbird polls for new messages it gets held up waiting for a response from the server I'm using POP port 995. Any ideas as to why I'm having a problem ? ------------------- how to investigate such issue ? CentOS release 6.10 (Final) # dovecot --version 2.3.11.3 (502c39af9) load average on server 0.05/0.01 user log shows like: Oct 12 10:08:50 pop3-login: Info: Login: user=<user(a)tld.com.au>, method=PLAIN, rip=49.111.222.333, lip=103.xxx.yyy.zzz, mpid=17371, TLS, session=<E83xW8rqDPkxw3EW> Oct 12 10:09:34 pop3(user(a)tld.com.au)<17371><E83xW8rqDPkxw3EW>: Info: Disconnected: Logged out top=0/0, retr=0/0, del=0/114194, size=10616009472 Oct 12 10:14:36 imap-login: Info: Login: user=<user(a)tld.com.au>, method=PLAIN, rip=49.111.222.333, lip=103.xxx.yyy.zzz, mpid=17460, TLS, session=<ftiLcMrqR/gxw3EW> Oct 12 10:14:37 imap(user(a)tld.com.au)<17460><ftiLcMrqR/gxw3EW>: Info: Logged out in=152 out=69724 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=1 body_bytes=68628 Oct 12 10:28:27 pop3-login: Info: Login: user=<user(a)tld.com.au>, method=PLAIN, rip=49.111.222.333, lip=103.xxx.yyy.zzz, mpid=17714, TLS, session=<+WsXosrqK/gxw3EW> Oct 12 10:29:16 pop3(user(a)tld.com.au)<17714><+WsXosrqK/gxw3EW>: Info: Disconnected: Logged out top=0/0, retr=1/76174, del=0/114195, size=10616085600 Oct 12 10:32:19 imap-login: Info: Login: user=<user(a)tld.com.au>, method=PLAIN, rip=49.111.222.333, lip=103.xxx.yyy.zzz, mpid=17817, TLS, session=<bZftr8rqq/gxw3EW> Oct 12 10:32:20 imap(user(a)tld.com.au)<17817><bZftr8rqq/gxw3EW>: Info: Logged out in=153 out=135592 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=1 body_bytes=134503 Oct 12 10:45:58 imap-login: Info: Login: user=<user(a)tld.com.au>, method=PLAIN, rip=49.111.222.333, lip=103.xxx.yyy.zzz, mpid=18081, TLS, session=<U3674Mrqzfgxw3EW> Oct 12 10:45:59 imap(user(a)tld.com.au)<18081><U3674Mrqzfgxw3EW>: Info: Logged out in=152 out=24602 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=1 body_bytes=23514 Oct 12 10:48:49 pop3-login: Info: Login: user=<user(a)tld.com.au>, method=PLAIN, rip=49.111.222.333, lip=103.xxx.yyy.zzz, mpid=18135, TLS, session=<y1/r6srqtPsxw3EW> Oct 12 10:49:53 pop3(user(a)tld.com.au)<18135><y1/r6srqtPsxw3EW>: Info: Disconnected: Logged out top=0/0, retr=2/166190, del=0/114197, size=10616251743 Oct 12 10:50:42 imap(user(a)tld.com.au)<3274><cr10yqTqifgxw3EW>: Info: Logged out in=192516 out=5957294 deleted=0 expunged=0 trashed=0 hdr_count=263 hdr_bytes=358656 body_count=126 body_bytes=5110403 Oct 12 10:50:42 imap(user(a)tld.com.au)<3271><hLl0yqTqh/gxw3EW>: Info: Logged out in=78324 out=58677 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0 Voytek

4 3

doveadm proxy list has strange entries
by Tobias Florek 02 Jan '23

02 Jan '23

Hi! I am using dovecot from the latest docker image: Version 2:2.3.20-3+debian11 When getting the proxy list I get the following entries. # doveadm -f flow proxy list username=user(a)example.com proto=imap src ip=10.254.0.13 dest ip=172.23.6.194 port=143 username=username proto=service src ip=src-ip dest ip=dest-ip port=dest-port username=user2(a)example.com proto=imap src ip=10.254.0.13 dest ip=172.23.6.194 port=143 username=username proto=service src ip=src-ip dest ip=dest-ip port=dest-port username=user3(a)example.com proto=imap src ip=10.254.0.13 dest ip=172.23.6.194 port=143 username=username proto=service src ip=src-ip dest ip=dest-ip port=dest-port ... These correspond to the http API's response: # curl -H "Authorization: X-Dovecot-API $(echo -n MY_API_KEY|base64 -w0)" http://localhost:8080/doveadm/v1 -H "Content-Type: application/json" -d '[["proxyList",{},"tag1"]]' | jq [ [ "doveadmResponse", [ { "username": "user(a)example.com", "service": "imap", "src-ip": "10.254.0.13", "dest-ip": "172.23.6.194", "dest-port": "143" }, { "username": "username", "service": "service", "src-ip": "src-ip", "dest-ip": "dest-ip", "dest-port": "dest-port" }, ... ], "tag1" ] ] This does not look right. Cheers, Tobias Florek

3 2

Error: Failed to get quota resource STORAGE: quota-fs: quotactl(Q_GETQUOTA, NFS:/home) failed: No such file or directory
by Jorge Concha C. 01 Jan '23

01 Jan '23

Hello everyone, I'm migrating my dovecot (imap) server from CentOS-6 to RockyLinux-9, but I can't get the QUOTA-FS plugin to work. When trying the "doveadm quota get -u user" command, it returns an error "doveadm(user): Error: Failed to get quota resource STORAGE: quota-fs: quotactl(Q_GETQUOTA, nfs:/home) failed: No such file or directory" If the user has his "$home" on a local drive, everything works fine, but if the user has his $home on a nfs-filesystem, it throws that error. On CentOS-6, everything works fine, with the same users on the same nfs-filesystem. I don't know if it is the product of the OS change or the dovecot version change. CentOS6 -> Dovecot 2.2.10 RockyLinux-9 -> Dovecot 2.3.16 Does anyone have any ideas? Thanks -- Jorge Concha C. Área de Sistemas - CEC U. de Chile

3 12

Iteration failure on passwd-file when listing all users?
by EML 31 Dec '22

31 Dec '22

Dovecot 2.3.16, Ubuntu 22.04. I'm trying to get a list of all dovecot users, on a simple test setup on my local network (which "works", in that I can get IMAP mails from dovecot): passdb { driver = passwd-file args = /var/mail/vhosts/%d/shadow } userdb { driver = passwd-file args = /var/mail/vhosts/%d/passwd } The command 'doveadm user -u foo(a)example.com' correctly shows that user. However, any attempt to wildcard the user fails: # doveadm user -u *(a)example.com Error: auth-master: userdb list: User listing returned failure Fatal: user listing failed The log file shows Error: conn unix:auth-worker (pid=13471,uid=129): auth-worker<1>: passwd-file(*(a)example.com): passwd-file: User iteration isn't currently supported with %variable paths Any ideas? And, if I can get this to work, is this method any better than just reading /var/mail/vhosts/example.com/passwd and extracting the first field to get the user name? Or just finding all the maildir directories in /var/mail/vhosts/example.com? Thanks. If you're interested, I'm doing this so that I can automate postfix setup (by mapping new addresses to known dovecot users in the postfix virtual mailbox file). The vmailbox file might look like: # 'user1' is a known dovecot user foo(a)example.com example.com/user1

3 6

Permissions issue with doveadm mailbox command
by James Moe 31 Dec '22

31 Dec '22

I have been exploring dovecot and came across this: $ sudo doveadm mailbox list doveadm(root): Error: Mail access for users with UID 303 not permitted (see first_valid_uid in config file, uid from mail_uid setting). Hmm. After setting "first_valid_uid = 300": $ sudo doveadm mailbox list doveadm(root)<4380><4VP8EENer2McEQAAvbJltg>: Error: chdir(/root/) failed: Permission denied (euid=303(vmail) egid=303(vmail) missing +x perm: /root, dir owned by 0:0 mode=0700) INBOX I do not understand the error message. The current configuration. (Note I have changed first_valid_uid to its original value.) $ postconf -n # 2.3.15 (0503334ab1): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.15 (e6a84e31) # OS: Linux 5.14.21-150400.24.33-default x86_64 btrfs # Hostname: sma-server3.sma.com auth_mechanisms = plain login base_dir = /var/run/dovecot/ debug_log_path = /data01/var/log/dovecot/dovecot-debug.log disable_plaintext_auth = no first_valid_uid = 1001 info_log_path = /data01/var/log/dovecot/dovecot-info.log log_path = /data01/var/log/dovecot/dovecot.log log_timestamp = %Y-%m-%dT%H:%M:%S mail_gid = vmail mail_location = maildir:/var/mail/vmail/%u/ mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service log { group = users user = dovecot } ssl = no ssl_cipher_list = ALL:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!RC4:!ADH:!LOW@STRENGTH ssl_options = no_compression ssl_prefer_server_ciphers = yes userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } -- James Moe moe dot james at sohnen-moe dot com 520.743.3936 Think.

2 1

Permissions for dovecot logging
by James Moe 31 Dec '22

31 Dec '22

dovecot 2.3.15 opensuse LEAP 15.4 I changed logging to use a path rather than syslog. Doing so makes it easier to work with fail2ban. Dovecot fails to start with the error: Can't open log file /data01/var/log/dovecot.log: Permission denied Permissions: drwxrwxr-x 1 root users 104 Feb 25 2018 /data01/ drwxrwxr-x 1 sma-user3x users 102 Dec 17 14:50 /data01/var/ drwxrwxr-x 1 sma-user3x users 146 Dec 27 15:37 /data01/var/log/ drwxrwxr-x 1 dovecot users 22 Dec 27 15:47 /data01/var/log/dovecot/ "dovecot" is a member of "users". What "permission" am I missing? Note: A long time ago I had a problem with programs consuming all available space on the system disk with log or backup files. I have since gotten in the habit of putting log files on a non-system disk. -- James Moe moe dot james at sohnen-moe dot com 520.743.3936 Think.

4 6

Self hosting solution for Christmas
by Andre Rodier 29 Dec '22

29 Dec '22

Hello everyone, Here my present for Christmas: a new version of HomeBox, the self hosted email solution. Feel free to drop comments, create issues, update the docs, etc. I released this quickly before going on vacation, so you may find some issues. However, this is mostly stable, and the code is easy to modify. Also, you can now add and remove components individually, with an Ansible playbook. https://github.com/progmaticltd/homebox System installation and features - Custom Debian installer generation with full disk encryption and fully automatic installation. - Unlock the system upon boot by entering the passphrase through SSH. - Install packages only from Debian stable (Bullseye). - Automatic letsencrypt certificates generation using Lego and DNS challenge. - Automatic security updates (optional). - Centralised authentication with an LDAP users database, SSL certificate, password policies, PAM integration. - AppArmor activated with a profile for all daemons. - XKCD Random passwords generated and saved into pass (or on file system). - Can be used at home, on a dedicated or virtual server hosted online. - Flexible IP address support: IPv4 only, IPv6 only, and IPv4+IPv4 or IPv4+IPv6. - Embedded DNS server, with CAA, DNSSEC and SSHFP (SSH fingerprint) support. - Grade A https sites, HSTS implemented by default. - Automatic configuration of OpenPGP Web Key Directory. - Automatic firewall rules for inbound, outbound and forwarding traffic, using nftables. - Restricted outbound web sites access to the minimum. - Automatic update of DNS keys, DNS servers and glue records on Gandi. Emails - Postfix configuration and installation, with LDAP lookups, internationalised email aliases, fully SSL compliant. - Generate DKIM keys, SPF and DMARC DNS records. The DKIM keys are generated every year. - Automatic copy of sent emails into the sent folder. - Automatic creation of the postmaster account and special email addresses using RFC 2142 specifications. - Dovecot configuration, IMAPS, POP3S, Quotas, ManageSieve. - Simple spam and ham learning by moving emails in and out the Junk folder - Sieve and vacation scripts. - Virtual folders for server search: unread messages, conversations view, all messages, flagged and messages labelled as "important". - Email addresses with recipient delimiter included, e.g. john.doe+lists(a)dbcooper.com. - Optional master user creation, e.g. for families with children or moderated communities. - Server side full text search inside emails, attached documents and files and compressed archives, with better results than GMail. - SOGo webmail with sieve filters management, password change form, Calendar and Address book management, GUI to import other account emails. - Powerful and light antispam system with rspamd and optional access to the web interface. - Antivirus for inbound and outbound emails with clamav (optional). - Automatic configuration for Thunderbird and Outlook using published XML and other clients with special DNS records (RFC 6186). Calendar and Address book - Install and configure a CalDAV / CardDAV server, with automatic discovery (RFC 6186). - Groupware functionality in a web interface, with SOGo. - Recurring events, email alerts, shared address books and calendars. - Mobile devices compatibility: Android, Apple iOS, BlackBerry 10 and Windows mobile through Microsoft ActiveSync. Other optional features - Incremental backups, encrypted, on multiple destination (SFTP, S3, Samba share or USB drive), with email and Jabber reporting. - Jabber server, using ejabberd, with LDAP authentication, direct or offline file transfer and optional server to server communication. - Static web site skeleton configuration, with https certificates and A+ security grade by default. Development - YAML files validation on each commit, using travis-ci. - End to end integration tests for the majority of components. - Playbooks to facilitate the installation or removal of development packages. - Global debug flag to activate the debug mode of all components. - Fully open source Ansible scripts licensed under GPLv3. Merry Christmas, André

4 7

Sieve duplicates
by Ferenc 28 Dec '22

28 Dec '22

Hi list, I’m trying this again from a different account because of lots of SPF warnings. I have Sieve scripts that duplicate e-mails. I don’t want that. My dovecot version is 2.3.4.1 (f79e8e7e4), Pigeonhole version 0.5.4 on Linux 5.4.189-2-pve x86_64 Debian 10.13. My scripts get called on an imap COPY action (imapsieve extension). The mail is fed to an external script (spam/ham training or retraining) via a pipe :copy action (vnd.dovecot.pipe extension). An header is added to keep track of what happened (editheader extension). I use "if header :contains” for my (simple) logic. If contains header X, do this, add header Y. If contains header Y, do that, add header X. I end each "if” part with “keep;" and “stop;” commands. Simple versions of the scripts, without the if-then construct, seem to work ok with imapsieve. As soon as I try to use if-then-else I find a duplicate of the processed e-mail; the original mail seems to be untouched, the duplicate appears to have been processed. With macos mail not all duplicates show up but with Roundcube webmail I can see them all. The scripts are like this: # without if-then require … pipe :copy … addheader … keep; # with if-then require … if header :contains pipe :copy … addheader … keep; stop; if not exists pipe :copy … addheader … keep; stop; keep; The scripts get called from “imapsieve_mailbox1_before” for example. Cheers, Ferenc

2 2