February 2023 - dovecot

Inconsistent filtering with debugging
by Christian Wolf 11 Mar '23

11 Mar '23

Hello, I have the problem, that I have a Postfix/Dovecot combination running with Sieve activated. The sieve script is running in general as some messages get filtered and moved to the appropriate folders. Now, I see that for the envelope filter, the behavior differs depending if I am debugging the rules or if the mails are received in a regular way. I have something like this: require "envelope"; # ... if envelope :is "to" "foo(a)subdaomin.example.com" { fileinto "INBOX.bar"; stop; } # ... The thing is if I call sieve-filter on the INBOX, I get the information that the mail is to be moved to the appropriate folder. However during delivery the mail is not moved there. Other rules in the script are working so it is installed in general. Of course, I could send mails to the mail address for testing but I have no clue on where to look for issues, especially as the "main debugging tool" for the rules (sieve-filter) is strangely behaving differently. The reason, I wanted to use the envelope was because the mails are delivered to foo(a)subdomain.example.com. From there, some virtual aliases are forwarded to a cental account bar(a)example.net using postfix virtual aliases. Thus, the Delivered-To header is always showing the value bar(a)example.net which is not suited for filtering. I could filter the Received headers but hoped for a more "high level" solution. Can you give me any advice? Christian

3 3

Flags/tags lost when moving e-mails to another folder
by Alexis Ben Miloud--Josselin 06 Mar '23

06 Mar '23

Hi, I have an issue with e-mail tags/flags. The IMAP user is using Thunderbird or Roundcube with Thunderbird tags. When the user (with an IMAP client) moves an e-mail from a folder to another, tags set on the e-mail are lost. When we check the filesystem, we can see the "a" tag: -rw-rw---- 2 test mailusers 2498 Dec 29 15:31 'cur/1676643827.M563793P15241.<HOSTNAME>,S=2498,W=2551:2,a' When we move the e-mail to another folder, the tag disappear: -rw-rw---- 2 test mailusers 2498 Dec 29 15:31 '.test/cur/1676644340.M214404P22014.<HOSTNAME>,S=2498,W=2551:2,' In both folders, the dovecot-keywords file is: 0 $LABEL1 The problem appeared after a major system upgrade (Debian 9 to 10). Dovecot version went from 2.2.27 to 2.3.4.1. Some system informations: # dovecot --version 2.3.4.1 (f79e8e7e4) # cat /etc/debian_version 10.13 # uname -a Linux <HOSTNAME> 4.19.0-23-amd64 #1 SMP Debian 4.19.269-1 (2022-12-20) x86_64 GNU/Linux # findmnt /var TARGET SOURCE FSTYPE OPTIONS /var /dev/vda3 ext4 rw,nosuid,relatime # findmnt /home TARGET SOURCE FSTYPE OPTIONS /home /dev/vdb ext4 rw,nosuid,nodev,noexec,relatime The output of `dovecot -n output` is here (link expires in one year): https://paste.evolix.org/?96d46acaea638568#56g7AQLuzJxxy4xintpexJCPgiAAG23c… Here are the operations I see on an e-mail file using strace: 15507 link("/home/test/Maildir/cur/1672326270.M619318P11927.<HOSTNAME>,S=2498,W=2551:2,Sa", "/home/test/Maildir/.test/tmp/1672326456.M737812P15507.<HOSTNAME>") = 0 15507 rename("/home/test/Maildir/.test/tmp/1672326456.M737812P15507.<HOSTNAME>", "/home/test/Maildir/.test/cur/1672326456.M737813P15507.<HOSTNAME>,S=2498,W=2551:2,S") = 0 15507 unlink("/home/test/Maildir/cur/1672326270.M619318P11927.<HOSTNAME>,S=2498,W=2551:2,Sa") = 0 Setting mbox_lazy_writes to yes and/or unsetting mail_cache_fields doesn't change anything. Any idea of what I am missing? Could this be a bug in Dovecot? Thanks in advance, -- Alexis Ben Miloud--Josselin <abenmiloud(a)evolix.fr> Evolix - Hébergement et Infogérance Open Source Marseille (37 rue Guibal, Pôle Média, 13003) / Paris / Montréal http://evolix.com | Twitter: @Evolix @EvolixNOC | http://blog.evolix.com

2 2

Setting up the mail-crypt plugin with virtual accounts that have no home directories
by Jeremy 05 Mar '23

05 Mar '23

Hi again, I am using dovecot 2.3.16, along with postfix and a PostgreSQL database for managing virtual accounts. After an initial topic from me about encrypting already existent mail, I could now use some pointers on how to set up the mail-crypt plugin for pure virtual accounts (i.e. that have no matching system users and/or home directories. I hope somebody can clarify a few things that are not entirely clear to me yet. After doing my own research, I believe the following should be possible: I'd like to use the password of virtual email accounts to let dovecot encrypt/decrypt the keys needed to encrypt/decrypt the mail in the relevant folders. As per the documentation @ https://doc.dovecot.org/configuration_manual/mail_crypt_plugin/ I believe this would be all the configuration I need: # Config mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_plugins = $mail_plugins mail_crypt plugin { mail_crypt_curve = secp521r1 # or some other preferred curve mail_crypt_save_version = 2 mail_crypt_require_encrypted_user_key = yes # necessary for encrypting keys with user password } # File: /etc/dovecot/dovecot-sql.conf.ext password_query = SELECT \ email as user, password, \ '%w' AS userdb_mail_crypt_private_password \ FROM virtual_users WHERE email='%u'; My first question is: Is it possible to configure mail_attribute_dict in such a way as to not use home directories. Since I only use virtual accounts, without those accounts having home directories, can I somehow tell dovecot to save the attributes into the PostsreSQL database, for instance? If not, can you suggest another approach, without having to create home directories for virtual users? My second question is: The documentation warns about not using password directly in the above SQL query: > Choosing encryption key > DO NOT use password directly. It can contain % which is interpreted as > variable expansion and can cause errors. Does this refer to not accidentally substituting '%w' with password? In other words, if I leave the above query as is, should I be good, even if plain text passwords of users potentially have % signs in them? Or would I need to take further measurements? (The passwords in my database are already hashed, by the way). I hope somebody can offer some guidance on this. Thanks.

2 7

"sh: doveadm: command not found" with doveadm -Dv backup -R
by James Brown 01 Mar '23

01 Mar '23

I’m moving from an old mail server to a new one. On the new one, trying to transfer mailboxes from the old machine using doveadm. It fails saying it can’f find doveadm: % sudo doveadm -Dv backup -R -u myuser(a)bordo.com.au remote:mail.bordo.com.au Password: Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so, 0x000A): symbol not found in flat namespace '_acl_backend_nonowner_lookups_iter_deinit' (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so, 0x000A): symbol not found in flat namespace '_quota_get_resource' (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so, 0x000A): symbol not found in flat namespace '_fts_backend_lookup' (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so, 0x000A): symbol not found in flat namespace '_mail_crypt_acl_secure_sharing_enabled' (this is usually intentional, so just ignore this message) Feb 28 12:42:02 Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm Feb 28 12:42:02 Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so, 0x000A): symbol not found in flat namespace '_acl_backend_nonowner_lookups_iter_deinit' (this is usually intentional, so just ignore this message) Feb 28 12:42:02 Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so, 0x000A): symbol not found in flat namespace '_quota_get_resource' (this is usually intentional, so just ignore this message) Feb 28 12:42:02 Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so, 0x000A): symbol not found in flat namespace '_fts_backend_lookup' (this is usually intentional, so just ignore this message) Feb 28 12:42:02 Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so, 0x000A): symbol not found in flat namespace '_mail_crypt_acl_secure_sharing_enabled' (this is usually intentional, so just ignore this message) Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: auth-master: userdb lookup(myuser(a)bordo.com.au): Started userdb lookup Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: auth-master: conn unix:/usr/local/var/run/dovecot/auth-userdb: Connecting Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: auth-master: conn unix:/usr/local/var/run/dovecot/auth-userdb (uid=0): Client connected (fd=9) Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: auth-master: userdb lookup(myuser(a)bordo.com.au): auth USER input: myuser(a)bordo.com.au home=/usr/local/virtual/bordo.com.au/myuser mail=maildir:/usr/local/virtual/bordo.com.au/myuser uid=102 gid=102 quota=*:storage=0 Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: auth-master: userdb lookup(myuser(a)bordo.com.au): Finished userdb lookup (username=myuser(a)bordo.com.au home=/usr/local/virtual/bordo.com.au/myuser mail=maildir:/usr/local/virtual/bordo.com.au/myuser uid=102 gid=102 quota=*:storage=0) Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: Added userdb setting: mail=maildir:/usr/local/virtual/bordo.com.au/myuser Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: Added userdb setting: plugin/quota=*:storage=0 Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: Effective uid=102, gid=102, home=/usr/local/virtual/bordo.com.au/myuser Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: open(/proc/self/stat) failed: No such file or directory Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: open(/proc/self/io) failed: No such file or directory Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/usr/local/virtual/bordo.com.au/myuser Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: maildir++: root=/usr/local/virtual/bordo.com.au/myuser, index=, indexpvt=, control=, inbox=/usr/local/virtual/bordo.com.au/myuser, alt= Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: brain M: Namespace has location maildir:/usr/local/virtual/bordo.com.au/myuser Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: Mailbox INBOX: Couldn't open mailbox in list index: Mailbox not found Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: Namespace : Using permissions from /usr/local/virtual/bordo.com.au/myuser: mode=0700 gid=default (root(a)mail.bordo.com.au) Password: sh: doveadm: command not found Feb 28 12:42:06 dsync-local(myuser(a)bordo.com.au)<SJq/Iepb/WMOLAEAiJz6GQ>: Error: read(mail.bordo.com.au) failed: EOF (version not received) Feb 28 12:42:06 dsync-local(myuser(a)bordo.com.au)<SJq/Iepb/WMOLAEAiJz6GQ>: Error: Remote command returned error 127: ssh mail.bordo.com.au doveadm dsync-server -umyuser(a)bordo.com.au -U Feb 28 12:42:06 dsync-local(myuser(a)bordo.com.au)<SJq/Iepb/WMOLAEAiJz6GQ>: Debug: User session is finished Feb 28 12:42:06 dsync-local(myuser(a)bordo.com.au)<SJq/Iepb/WMOLAEAiJz6GQ>: Debug: auth-master: conn unix:/usr/local/var/run/dovecot/auth-userdb (uid=0): Disconnected: Connection closed (fd=9) Does this mean the remote machine can’t fine doveadm? (Both machines are running macOS) I could probably transfer the mailboxes using rsync, but I thought that doveadm was a better way. Thanks, James.

2 2

Re: Auth variables unknown variable -- about to go crazy
by efeizbudak 28 Feb '23

28 Feb '23

1 0

Auth variables unknown variable -- about to go crazy
by efeizbudak 28 Feb '23

28 Feb '23

Hi everyone, I'm trying to configure my email server to encrypt mails on a per user basis. I have the following in my conf: mail_plugins = $mail_plugins mail_crypt mail_attribute_dict = file:%h/Mail/dovecot-attributes plugin { mail_crypt_curve = secp521r1 mail_crypt_save_version = 2 mail_crypt_require_encrypted_user_key = yes mail_crypt_private_password = %N{password} } And I'm getting %password unknown variable error. I use pam to store the passwords. All I want is to be able to hash the user password and use that to encrypt/decrypt the keys. Can someone _please_ help me? -- All the best, Efe The funny quote of this email is trivial and left as an exercise.

2 1

re: "Timeout (180s) while waiting for lock for transaction log file" since 2.3 upgrade?
by graeme vetterlein 26 Feb '23

26 Feb '23

This is an old thread (but still the best search match for the error message). Several other people pointed at NFS, so it got me to look. Indeed what appear to happen (for me at least) was .... The folders being imapped, some of them were on a NFSv3 mount. The NFS server had been shutdown. The imap process appeared to hang (presumably trying to read from the defunct NFS mount point) this held the lock , so other imap processes (possibly not using NFS) hung awaiting the lock. The "fix" was to bing the NFS server back online ...and as it turned out restart a number of things like autofs and dovecot (Normally NFS clients just carry on, so the timeout must have affected things) My long-term fix came from: https://docs.oracle.com/cd/E19620-01/805-5233/z400163399f/index.html Namely to add -soft in /etc/auto_master This should cause a "fail" instead of a hang. -- Graeme

1 0

Cannot reach documentation site
by Ken Bass 26 Feb '23

26 Feb '23

doc.dovecot.org does not have a DNS entry. It worked a few days ago, but not now.

3 3

quota groups
by Ken Young 26 Feb '23

26 Feb '23

Hello, How can I set up groups to give each group a different quota? for example, group A has mail storage 1GB, and users in this group get 1GB quota etc. Thanks. Ken Y

1 0

Small suggestion for improvement of pigeonhole/imapsieve documentation
by Matthias Petermann 25 Feb '23

25 Feb '23

Hello all, this is my first post on this list and I would like to introduce myself briefly. My name is Matthias and I am from Saxony in Germany. I'm working with Unix and mail systems for a long time - actually with focus for NetBSD and Cyrus IMAP. Occasionally I also use Dovecot, which has several reasons. Today, one of the reasons was that I would like to trigger Sieve scripts when moving mail within mailbox folders - i.e. at an unspecified time after the actual mail arrival. In doing so, I came across the imapsieve extension for Pigeonhole fairly quickly and decided that the administrative Sieve scripts in imapsieve were exactly what I needed. However, despite having debug logging fully up, I couldn't figure out until just now why my sieve script wasn't running despite having IMAP events logged. Even intentionally defining an invalid script did not result in an error, so it seemed to be ignored altogether. Until I just now came across that I seem to have misunderstood the documentation. The template shown for the imapsieve_mailboxXXX_ parameters suggested to me that a sequence number of three digits must / can be specified here. Thus, I started numbering my parameters with imapsieve_mailbox001. However, while researching in various forums, I came across only examples that use imapsieve_mailbox1_ as the first parameter. In desperation I tried this out - now it works for me, too. Long story short - I don't know if I'm the only one who has made this mistake so far. A little hint in the documentation[1] would have helped me, how the XXX is meant exactly. I am referring to this section: ``` This setting configures the name of a mailbox for which administrator scripts are configured. The `XXX’ in this setting is a sequence number, which allows configuring multiple associations between Sieve scripts and mailboxes. The settings defined hereafter with matching sequence numbers apply to the mailbox named by this setting. The sequence of configured mailboxes ends at the first missing imapsieve_mailboxXXX_name setting. This setting supports wildcards with a syntax compatible with the IMAP LIST command, meaning that this setting can apply to multiple or even all (“*”) mailboxes. ``` What do you think about adding a small explanatory sentence to the documentation? For example: ``` This setting configures the name of a mailbox for which administrator scripts are configured. The `XXX’ in this setting is a sequence number, which allows configuring multiple associations between Sieve scripts and mailboxes. The sequence number must begin with the digit 1 without preceding zeros and may have a maximum of three digits. The settings defined hereafter with matching sequence numbers apply to the mailbox named by this setting. The sequence of configured mailboxes ends at the first missing imapsieve_mailboxXXX_name setting. This setting supports wildcards with a syntax compatible with the IMAP LIST command, meaning that this setting can apply to multiple or even all (“*”) mailboxes. ``` Many greetings Matthias [1] https://doc.dovecot.org/configuration_manual/sieve/plugins/imapsieve/

1 0