December 2022 - dovecot

Warning sqlpool(mysql): Query failed
by Alessio Cecchi 27 Jun '23

27 Jun '23

Hi, with the last dovecot version (2.3.13) I see many of this warnings via "doveadm log errors": Feb 17 03:26:28 Warning: dict(31513): conn unix:dict (pid=31330,uid=95): dict(mysql)<email1(a)company1.com>: sqlpool(mysql): Query failed, retrying: MySQL server has gone away (idled for 679 secs) Feb 17 03:48:53 Warning: dict(31013): conn unix:dict (pid=31805,uid=95): dict(mysql)<email2(a)company2.com>: sqlpool(mysql): Query failed, retrying: MySQL server has gone away (idled for 685 secs) Feb 17 03:54:12 Warning: dict(32529): conn unix:dict (pid=3147,uid=95): dict(mysql)<email3(a)company3.com>: sqlpool(mysql): Query failed, retrying: MySQL server has gone away (idled for 670 secs) What do they mean? I thinks are related to inactivity timeout on mysql server (where I have a wait_timeout=600), but these warnings could be problems or can be fixed with some setting in dovecot? My dovecot configuration about dict: dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext sieve = mysql:/etc/dovecot/dovecot-dict-sieve-sql.conf.ext sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } plugin { [...] acl_shared_dict = proxy::acl expire_dict = proxy::expire quota2 = dict:Quota Usage::noenforcing:proxy::sqlquota sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = dict:proxy::sieve;name=activesql [...] } service dict { process_limit = 500 unix_listener dict { group = vmail mode = 0660 user = vmail } } Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 1

dovecot-fts-solr Solr9 support
by PGNet Dev 24 Apr '23

24 Apr '23

I run dovecot-2.3.18 , & am deploying Solr search with 'dovecot-fts-solr' I run Solr standalone. When creating a dedicated 'dovecot' core, cp'ing from Dovecot's included solr schemas ls -al /usr/share/doc/dovecot/solr-* -rw-r--r-- 1 root root 12K Dec 3 06:48 /usr/share/doc/dovecot/solr-config-7.7.0.xml -rw-r--r-- 1 root root 2.7K Dec 3 06:48 /usr/share/doc/dovecot/solr-schema-7.7.0.xml -rw-r--r-- 1 root root 2.9K Dec 3 06:48 /usr/share/doc/dovecot/solr-schema.xml is tested only for Solr 7.7.0 https://doc.dovecot.org/configuration_manual/fts/solr/ "The steps described in this page are tested for Solr 7.7.0. For other versions, this these steps may need to be adjusted." which, per solr docs https://solr.apache.org/downloads.html About versions and support Version Description 9.x Current major version for feature releases (STABLE) 8.11.x Previous major version may sometimes receive critical bugfix releases 10 Next major version, yet to be released (UNSTABLE) <8.11 All older versions are End Of Life (EOL) is EOL'd and unsupported. In the dovecot 7.7.0 schema, FastLRUCache/LRUCache are used as the caching class(es), grep class=.*LRU /usr/share/doc/dovecot/solr-config-7.7.0.xml <filterCache class="solr.FastLRUCache" <queryResultCache class="solr.LRUCache" <documentCache class="solr.LRUCache" class="solr.search.LRUCache" For Solr 9, cat CHANGES.txt Upgrade Notes --------------------- ... * SOLR-13817: Deprecate legacy SolrCache implementations. Users are encouraged to transition their configurations to use org.apache.solr.search.CaffeineCache instead. (ab) running solr9 + dovecot-fts-solr search, based on the 7.7.0 schemas, results in errors, failed scan & search. Modifying the config to use CaffeineCache cp solr-config-7.7.0.xml solr-config-9.0.0.xml perl -pi -e '\ s|solr.FastLRUCache|solr.CaffeineCache|g; \ s|solr.LRUCache|solr.CaffeineCache|g; \ s|solr.search.LRUCache|solr.search.CaffeineCache|g;' \ solr-config-9.0.0.xml appears to do the trick. Atm, indexing & search -- both manually triggered, and on delivery receipt to Dovecot store -- is working with no errors (yet). I've not tested exhaustively yet, & don't know if this Cache class change in the schema is both necessary & sufficient for dovecot-fts-solr search use with solr 9.0.0 Given that for Solr "<8.11 All older versions are End Of Life (EOL)" it'd be useful for Dovecot to 'retest' for current, not-EOL'd Solr, and publish/package compatible schemas accordingly.

5 8

mbox to maildir migration, same server?
by kjohnson＠eclypse.org 20 Apr '23

20 Apr '23

What is the recommended procedure to migrate from mbox to maildir on a server already running dovecot? Ideally, this procedure would convert messages in an existing mbox INBOX and in ~/mail mboxes (as defined in my dovecot configuration) to storage in a ~/Maildir. Some of the INBOXs are rather large -- 100's of MB (one of the reasons for migrating). What I have tried has not worked for me. Below, I describe what I tried, how I tested before I tried, what went wrong, and the current setup (pre-migration). What I tried: ============= stop exim4 service dovecot stop Make dovecot listen only locally, not to external connections. /etc/dovecot/local.conf protocol imap { listen = 127.0.0.1:143 ssl_listen = 127.0.0.1:993 } protocol pop3 { listen = 127.0.0.1:110 ssl_listen = 127.0.0.1:995 } Modify dovecot configuration for maildir: Watch capitalization! #in 10-mail.conf: mail_location = maildir:/home/%u/Maildir namespace inbox { separator = / } dovecot -n, to confirm results and perform limited syntax checking. service dovecot start for each user, in a bash script: dsync -v -u "$U" mirror "mbox:/home/$U/mail:INBOX=/var/mail/$U" (if the conversion worked for all users, then reconfigure exim to use maildir and restart it; reconfigure dovecot to listen to external interfaces and restart.) How I tested: ============= Before trying this on the real server, I set up a matching (same versions of Debian, Exim, and Dovecot) VM using mbox, with a couple of accounts. One using POP3, one using IMAP. The IMAP account was accessed from Claws and from Outlook. The test accounts had perhaps a dozen messages on the server. I tried the conversion procedure, and everything I tried worked flawlessly. (Yes, next time I will move some real inbox files from the live server). What went wrong: ================ I encountered two types of problems here. The first was several dsync warnings (username changed): dsync(example): Warning: Mailbox changes caused a desync. You may want to run dsync again: Remote lost mailbox GUID 21bfa11a81f11763ac150000d09efc50 (maybe it was just deleted?) As I understand it, this can result in some or all messages being downloaded again by clients. My understanding may be wrong. I also don't understand how the mailbox could have changed -- exim4 was stopped, dovecot is only listening on 127.0.0.1, and I am the only user who uses a local mail client on that system, and this was not my account. The second problem I encountered seems to match Debian Bug report logs - #1011238. In part of the discussion of that report, the reporter mentions adding -1 to the command, as shown: dsync -v -u "$U" mirror -1 "mbox:/home/$U/mail:INBOX=/var/mail/$U" While this seemed to eliminate the dsync panic of #1011238, as far as I can tell, no old messages were actually moved from the old mbox files to the new maildirs. At that point I restored the backup mbox files and undid the changes to dovecot and exim4. Current setup: ============== Debian 11.4 Exim 4.94.2-7 dovecot 1:2.3.13+dfsg1-2 The current setup with Dovecot was a successful migration from IMAP-UW in December 2020. Output from dovecot -n (the list of trusted networks has been removed) # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) # OS: Linux 5.10.0-16-686-pae i686 Debian 11.4 # Hostname: localhost auth_verbose = yes mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } pop3_uidl_format = %08Xv%08Xu protocols = " imap pop3" ssl_cert = </etc/dovecot/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } Thank you to anyone who read this far. Ken

4 4

decrypt.rb
by Dave 11 Apr '23

11 Apr '23

Per the Dovecot site here: https://wiki.dovecot.org/Plugins/MailCrypt ... the "decrypt.rb" ruby script can be used to decrypt a Dovecot-encrypted message file from the command line. The script sort of runs successfully for me, in the sense that it doesn't error out, but it doesn't show the decrypted message. I've called it like so: decrypt.rb -k <PATH_TO_DOVECOT_KEYFILE> -f ./encrypted_message But when I do it just displays the info below and exits (I've subbed <DATA> tags for the actual data): ------------------[OUTPUT]----------------------- Key(s) (total: 1) - Key type : EC - Key digest: <DATA1> - Peer key : <DATA2> - Encrypted : <DATA3> - Kd hash : <DATA4> ------------------------------------------------------- I've tried specifying an output file as well, per the script's command line options, but the output file is 0 bytes. Does anyone have any suggestions? I *think* I'm using it the way it's intended to be used, but maybe I'm not?! -Dave

5 14

ot: how to t/s TBird problems ?
by Voytek Eymont 22 Mar '23

22 Mar '23

I have a Dovecot/Postfix/MariaDB on a Centos, just have a user ask me: ------------------ I recently upgraded my Thunderbird email client and have experienced problems since. It appears that when Tbird polls for new messages it gets held up waiting for a response from the server I'm using POP port 995. Any ideas as to why I'm having a problem ? ------------------- how to investigate such issue ? CentOS release 6.10 (Final) # dovecot --version 2.3.11.3 (502c39af9) load average on server 0.05/0.01 user log shows like: Oct 12 10:08:50 pop3-login: Info: Login: user=<user(a)tld.com.au>, method=PLAIN, rip=49.111.222.333, lip=103.xxx.yyy.zzz, mpid=17371, TLS, session=<E83xW8rqDPkxw3EW> Oct 12 10:09:34 pop3(user(a)tld.com.au)<17371><E83xW8rqDPkxw3EW>: Info: Disconnected: Logged out top=0/0, retr=0/0, del=0/114194, size=10616009472 Oct 12 10:14:36 imap-login: Info: Login: user=<user(a)tld.com.au>, method=PLAIN, rip=49.111.222.333, lip=103.xxx.yyy.zzz, mpid=17460, TLS, session=<ftiLcMrqR/gxw3EW> Oct 12 10:14:37 imap(user(a)tld.com.au)<17460><ftiLcMrqR/gxw3EW>: Info: Logged out in=152 out=69724 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=1 body_bytes=68628 Oct 12 10:28:27 pop3-login: Info: Login: user=<user(a)tld.com.au>, method=PLAIN, rip=49.111.222.333, lip=103.xxx.yyy.zzz, mpid=17714, TLS, session=<+WsXosrqK/gxw3EW> Oct 12 10:29:16 pop3(user(a)tld.com.au)<17714><+WsXosrqK/gxw3EW>: Info: Disconnected: Logged out top=0/0, retr=1/76174, del=0/114195, size=10616085600 Oct 12 10:32:19 imap-login: Info: Login: user=<user(a)tld.com.au>, method=PLAIN, rip=49.111.222.333, lip=103.xxx.yyy.zzz, mpid=17817, TLS, session=<bZftr8rqq/gxw3EW> Oct 12 10:32:20 imap(user(a)tld.com.au)<17817><bZftr8rqq/gxw3EW>: Info: Logged out in=153 out=135592 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=1 body_bytes=134503 Oct 12 10:45:58 imap-login: Info: Login: user=<user(a)tld.com.au>, method=PLAIN, rip=49.111.222.333, lip=103.xxx.yyy.zzz, mpid=18081, TLS, session=<U3674Mrqzfgxw3EW> Oct 12 10:45:59 imap(user(a)tld.com.au)<18081><U3674Mrqzfgxw3EW>: Info: Logged out in=152 out=24602 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=1 body_bytes=23514 Oct 12 10:48:49 pop3-login: Info: Login: user=<user(a)tld.com.au>, method=PLAIN, rip=49.111.222.333, lip=103.xxx.yyy.zzz, mpid=18135, TLS, session=<y1/r6srqtPsxw3EW> Oct 12 10:49:53 pop3(user(a)tld.com.au)<18135><y1/r6srqtPsxw3EW>: Info: Disconnected: Logged out top=0/0, retr=2/166190, del=0/114197, size=10616251743 Oct 12 10:50:42 imap(user(a)tld.com.au)<3274><cr10yqTqifgxw3EW>: Info: Logged out in=192516 out=5957294 deleted=0 expunged=0 trashed=0 hdr_count=263 hdr_bytes=358656 body_count=126 body_bytes=5110403 Oct 12 10:50:42 imap(user(a)tld.com.au)<3271><hLl0yqTqh/gxw3EW>: Info: Logged out in=78324 out=58677 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0 Voytek

4 3

doveadm proxy list has strange entries
by Tobias Florek 02 Jan '23

02 Jan '23

Hi! I am using dovecot from the latest docker image: Version 2:2.3.20-3+debian11 When getting the proxy list I get the following entries. # doveadm -f flow proxy list username=user(a)example.com proto=imap src ip=10.254.0.13 dest ip=172.23.6.194 port=143 username=username proto=service src ip=src-ip dest ip=dest-ip port=dest-port username=user2(a)example.com proto=imap src ip=10.254.0.13 dest ip=172.23.6.194 port=143 username=username proto=service src ip=src-ip dest ip=dest-ip port=dest-port username=user3(a)example.com proto=imap src ip=10.254.0.13 dest ip=172.23.6.194 port=143 username=username proto=service src ip=src-ip dest ip=dest-ip port=dest-port ... These correspond to the http API's response: # curl -H "Authorization: X-Dovecot-API $(echo -n MY_API_KEY|base64 -w0)" http://localhost:8080/doveadm/v1 -H "Content-Type: application/json" -d '[["proxyList",{},"tag1"]]' | jq [ [ "doveadmResponse", [ { "username": "user(a)example.com", "service": "imap", "src-ip": "10.254.0.13", "dest-ip": "172.23.6.194", "dest-port": "143" }, { "username": "username", "service": "service", "src-ip": "src-ip", "dest-ip": "dest-ip", "dest-port": "dest-port" }, ... ], "tag1" ] ] This does not look right. Cheers, Tobias Florek

3 2

Error: Failed to get quota resource STORAGE: quota-fs: quotactl(Q_GETQUOTA, NFS:/home) failed: No such file or directory
by Jorge Concha C. 01 Jan '23

01 Jan '23

Hello everyone, I'm migrating my dovecot (imap) server from CentOS-6 to RockyLinux-9, but I can't get the QUOTA-FS plugin to work. When trying the "doveadm quota get -u user" command, it returns an error "doveadm(user): Error: Failed to get quota resource STORAGE: quota-fs: quotactl(Q_GETQUOTA, nfs:/home) failed: No such file or directory" If the user has his "$home" on a local drive, everything works fine, but if the user has his $home on a nfs-filesystem, it throws that error. On CentOS-6, everything works fine, with the same users on the same nfs-filesystem. I don't know if it is the product of the OS change or the dovecot version change. CentOS6 -> Dovecot 2.2.10 RockyLinux-9 -> Dovecot 2.3.16 Does anyone have any ideas? Thanks -- Jorge Concha C. Área de Sistemas - CEC U. de Chile

3 12

Iteration failure on passwd-file when listing all users?
by EML 31 Dec '22

31 Dec '22

Dovecot 2.3.16, Ubuntu 22.04. I'm trying to get a list of all dovecot users, on a simple test setup on my local network (which "works", in that I can get IMAP mails from dovecot): passdb { driver = passwd-file args = /var/mail/vhosts/%d/shadow } userdb { driver = passwd-file args = /var/mail/vhosts/%d/passwd } The command 'doveadm user -u foo(a)example.com' correctly shows that user. However, any attempt to wildcard the user fails: # doveadm user -u *(a)example.com Error: auth-master: userdb list: User listing returned failure Fatal: user listing failed The log file shows Error: conn unix:auth-worker (pid=13471,uid=129): auth-worker<1>: passwd-file(*(a)example.com): passwd-file: User iteration isn't currently supported with %variable paths Any ideas? And, if I can get this to work, is this method any better than just reading /var/mail/vhosts/example.com/passwd and extracting the first field to get the user name? Or just finding all the maildir directories in /var/mail/vhosts/example.com? Thanks. If you're interested, I'm doing this so that I can automate postfix setup (by mapping new addresses to known dovecot users in the postfix virtual mailbox file). The vmailbox file might look like: # 'user1' is a known dovecot user foo(a)example.com example.com/user1

3 6

Permissions issue with doveadm mailbox command
by James Moe 31 Dec '22

31 Dec '22

I have been exploring dovecot and came across this: $ sudo doveadm mailbox list doveadm(root): Error: Mail access for users with UID 303 not permitted (see first_valid_uid in config file, uid from mail_uid setting). Hmm. After setting "first_valid_uid = 300": $ sudo doveadm mailbox list doveadm(root)<4380><4VP8EENer2McEQAAvbJltg>: Error: chdir(/root/) failed: Permission denied (euid=303(vmail) egid=303(vmail) missing +x perm: /root, dir owned by 0:0 mode=0700) INBOX I do not understand the error message. The current configuration. (Note I have changed first_valid_uid to its original value.) $ postconf -n # 2.3.15 (0503334ab1): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.15 (e6a84e31) # OS: Linux 5.14.21-150400.24.33-default x86_64 btrfs # Hostname: sma-server3.sma.com auth_mechanisms = plain login base_dir = /var/run/dovecot/ debug_log_path = /data01/var/log/dovecot/dovecot-debug.log disable_plaintext_auth = no first_valid_uid = 1001 info_log_path = /data01/var/log/dovecot/dovecot-info.log log_path = /data01/var/log/dovecot/dovecot.log log_timestamp = %Y-%m-%dT%H:%M:%S mail_gid = vmail mail_location = maildir:/var/mail/vmail/%u/ mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service log { group = users user = dovecot } ssl = no ssl_cipher_list = ALL:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!RC4:!ADH:!LOW@STRENGTH ssl_options = no_compression ssl_prefer_server_ciphers = yes userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } -- James Moe moe dot james at sohnen-moe dot com 520.743.3936 Think.

2 1

Permissions for dovecot logging
by James Moe 31 Dec '22

31 Dec '22

dovecot 2.3.15 opensuse LEAP 15.4 I changed logging to use a path rather than syslog. Doing so makes it easier to work with fail2ban. Dovecot fails to start with the error: Can't open log file /data01/var/log/dovecot.log: Permission denied Permissions: drwxrwxr-x 1 root users 104 Feb 25 2018 /data01/ drwxrwxr-x 1 sma-user3x users 102 Dec 17 14:50 /data01/var/ drwxrwxr-x 1 sma-user3x users 146 Dec 27 15:37 /data01/var/log/ drwxrwxr-x 1 dovecot users 22 Dec 27 15:47 /data01/var/log/dovecot/ "dovecot" is a member of "users". What "permission" am I missing? Note: A long time ago I had a problem with programs consuming all available space on the system disk with log or backup files. I have since gotten in the habit of putting log files on a non-system disk. -- James Moe moe dot james at sohnen-moe dot com 520.743.3936 Think.

4 6