February 2023 - dovecot

Setting up the mail-crypt plugin with virtual accounts that have no home directories
by Jeremy 05 Mar '23

05 Mar '23

Hi again, I am using dovecot 2.3.16, along with postfix and a PostgreSQL database for managing virtual accounts. After an initial topic from me about encrypting already existent mail, I could now use some pointers on how to set up the mail-crypt plugin for pure virtual accounts (i.e. that have no matching system users and/or home directories. I hope somebody can clarify a few things that are not entirely clear to me yet. After doing my own research, I believe the following should be possible: I'd like to use the password of virtual email accounts to let dovecot encrypt/decrypt the keys needed to encrypt/decrypt the mail in the relevant folders. As per the documentation @ https://doc.dovecot.org/configuration_manual/mail_crypt_plugin/ I believe this would be all the configuration I need: # Config mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_plugins = $mail_plugins mail_crypt plugin { mail_crypt_curve = secp521r1 # or some other preferred curve mail_crypt_save_version = 2 mail_crypt_require_encrypted_user_key = yes # necessary for encrypting keys with user password } # File: /etc/dovecot/dovecot-sql.conf.ext password_query = SELECT \ email as user, password, \ '%w' AS userdb_mail_crypt_private_password \ FROM virtual_users WHERE email='%u'; My first question is: Is it possible to configure mail_attribute_dict in such a way as to not use home directories. Since I only use virtual accounts, without those accounts having home directories, can I somehow tell dovecot to save the attributes into the PostsreSQL database, for instance? If not, can you suggest another approach, without having to create home directories for virtual users? My second question is: The documentation warns about not using password directly in the above SQL query: > Choosing encryption key > DO NOT use password directly. It can contain % which is interpreted as > variable expansion and can cause errors. Does this refer to not accidentally substituting '%w' with password? In other words, if I leave the above query as is, should I be good, even if plain text passwords of users potentially have % signs in them? Or would I need to take further measurements? (The passwords in my database are already hashed, by the way). I hope somebody can offer some guidance on this. Thanks.

2 7

"sh: doveadm: command not found" with doveadm -Dv backup -R
by James Brown 01 Mar '23

01 Mar '23

I’m moving from an old mail server to a new one. On the new one, trying to transfer mailboxes from the old machine using doveadm. It fails saying it can’f find doveadm: % sudo doveadm -Dv backup -R -u myuser(a)bordo.com.au remote:mail.bordo.com.au Password: Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so, 0x000A): symbol not found in flat namespace '_acl_backend_nonowner_lookups_iter_deinit' (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so, 0x000A): symbol not found in flat namespace '_quota_get_resource' (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so, 0x000A): symbol not found in flat namespace '_fts_backend_lookup' (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so, 0x000A): symbol not found in flat namespace '_mail_crypt_acl_secure_sharing_enabled' (this is usually intentional, so just ignore this message) Feb 28 12:42:02 Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm Feb 28 12:42:02 Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so, 0x000A): symbol not found in flat namespace '_acl_backend_nonowner_lookups_iter_deinit' (this is usually intentional, so just ignore this message) Feb 28 12:42:02 Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so, 0x000A): symbol not found in flat namespace '_quota_get_resource' (this is usually intentional, so just ignore this message) Feb 28 12:42:02 Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so, 0x000A): symbol not found in flat namespace '_fts_backend_lookup' (this is usually intentional, so just ignore this message) Feb 28 12:42:02 Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: dlopen(/usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so, 0x000A): symbol not found in flat namespace '_mail_crypt_acl_secure_sharing_enabled' (this is usually intentional, so just ignore this message) Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: auth-master: userdb lookup(myuser(a)bordo.com.au): Started userdb lookup Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: auth-master: conn unix:/usr/local/var/run/dovecot/auth-userdb: Connecting Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: auth-master: conn unix:/usr/local/var/run/dovecot/auth-userdb (uid=0): Client connected (fd=9) Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: auth-master: userdb lookup(myuser(a)bordo.com.au): auth USER input: myuser(a)bordo.com.au home=/usr/local/virtual/bordo.com.au/myuser mail=maildir:/usr/local/virtual/bordo.com.au/myuser uid=102 gid=102 quota=*:storage=0 Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: auth-master: userdb lookup(myuser(a)bordo.com.au): Finished userdb lookup (username=myuser(a)bordo.com.au home=/usr/local/virtual/bordo.com.au/myuser mail=maildir:/usr/local/virtual/bordo.com.au/myuser uid=102 gid=102 quota=*:storage=0) Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: Added userdb setting: mail=maildir:/usr/local/virtual/bordo.com.au/myuser Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au)<76814><>: Debug: Added userdb setting: plugin/quota=*:storage=0 Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: Effective uid=102, gid=102, home=/usr/local/virtual/bordo.com.au/myuser Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: open(/proc/self/stat) failed: No such file or directory Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: open(/proc/self/io) failed: No such file or directory Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/usr/local/virtual/bordo.com.au/myuser Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: maildir++: root=/usr/local/virtual/bordo.com.au/myuser, index=, indexpvt=, control=, inbox=/usr/local/virtual/bordo.com.au/myuser, alt= Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: brain M: Namespace has location maildir:/usr/local/virtual/bordo.com.au/myuser Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: Mailbox INBOX: Couldn't open mailbox in list index: Mailbox not found Feb 28 12:42:02 doveadm(myuser(a)bordo.com.au): Debug: Namespace : Using permissions from /usr/local/virtual/bordo.com.au/myuser: mode=0700 gid=default (root(a)mail.bordo.com.au) Password: sh: doveadm: command not found Feb 28 12:42:06 dsync-local(myuser(a)bordo.com.au)<SJq/Iepb/WMOLAEAiJz6GQ>: Error: read(mail.bordo.com.au) failed: EOF (version not received) Feb 28 12:42:06 dsync-local(myuser(a)bordo.com.au)<SJq/Iepb/WMOLAEAiJz6GQ>: Error: Remote command returned error 127: ssh mail.bordo.com.au doveadm dsync-server -umyuser(a)bordo.com.au -U Feb 28 12:42:06 dsync-local(myuser(a)bordo.com.au)<SJq/Iepb/WMOLAEAiJz6GQ>: Debug: User session is finished Feb 28 12:42:06 dsync-local(myuser(a)bordo.com.au)<SJq/Iepb/WMOLAEAiJz6GQ>: Debug: auth-master: conn unix:/usr/local/var/run/dovecot/auth-userdb (uid=0): Disconnected: Connection closed (fd=9) Does this mean the remote machine can’t fine doveadm? (Both machines are running macOS) I could probably transfer the mailboxes using rsync, but I thought that doveadm was a better way. Thanks, James.

2 2

Re: Auth variables unknown variable -- about to go crazy
by efeizbudak 28 Feb '23

28 Feb '23

1 0

Auth variables unknown variable -- about to go crazy
by efeizbudak 28 Feb '23

28 Feb '23

Hi everyone, I'm trying to configure my email server to encrypt mails on a per user basis. I have the following in my conf: mail_plugins = $mail_plugins mail_crypt mail_attribute_dict = file:%h/Mail/dovecot-attributes plugin { mail_crypt_curve = secp521r1 mail_crypt_save_version = 2 mail_crypt_require_encrypted_user_key = yes mail_crypt_private_password = %N{password} } And I'm getting %password unknown variable error. I use pam to store the passwords. All I want is to be able to hash the user password and use that to encrypt/decrypt the keys. Can someone _please_ help me? -- All the best, Efe The funny quote of this email is trivial and left as an exercise.

2 1

re: "Timeout (180s) while waiting for lock for transaction log file" since 2.3 upgrade?
by graeme vetterlein 26 Feb '23

26 Feb '23

This is an old thread (but still the best search match for the error message). Several other people pointed at NFS, so it got me to look. Indeed what appear to happen (for me at least) was .... The folders being imapped, some of them were on a NFSv3 mount. The NFS server had been shutdown. The imap process appeared to hang (presumably trying to read from the defunct NFS mount point) this held the lock , so other imap processes (possibly not using NFS) hung awaiting the lock. The "fix" was to bing the NFS server back online ...and as it turned out restart a number of things like autofs and dovecot (Normally NFS clients just carry on, so the timeout must have affected things) My long-term fix came from: https://docs.oracle.com/cd/E19620-01/805-5233/z400163399f/index.html Namely to add -soft in /etc/auto_master This should cause a "fail" instead of a hang. -- Graeme

1 0

Cannot reach documentation site
by Ken Bass 26 Feb '23

26 Feb '23

doc.dovecot.org does not have a DNS entry. It worked a few days ago, but not now.

3 3

quota groups
by Ken Young 26 Feb '23

26 Feb '23

Hello, How can I set up groups to give each group a different quota? for example, group A has mail storage 1GB, and users in this group get 1GB quota etc. Thanks. Ken Y

1 0

Small suggestion for improvement of pigeonhole/imapsieve documentation
by Matthias Petermann 25 Feb '23

25 Feb '23

Hello all, this is my first post on this list and I would like to introduce myself briefly. My name is Matthias and I am from Saxony in Germany. I'm working with Unix and mail systems for a long time - actually with focus for NetBSD and Cyrus IMAP. Occasionally I also use Dovecot, which has several reasons. Today, one of the reasons was that I would like to trigger Sieve scripts when moving mail within mailbox folders - i.e. at an unspecified time after the actual mail arrival. In doing so, I came across the imapsieve extension for Pigeonhole fairly quickly and decided that the administrative Sieve scripts in imapsieve were exactly what I needed. However, despite having debug logging fully up, I couldn't figure out until just now why my sieve script wasn't running despite having IMAP events logged. Even intentionally defining an invalid script did not result in an error, so it seemed to be ignored altogether. Until I just now came across that I seem to have misunderstood the documentation. The template shown for the imapsieve_mailboxXXX_ parameters suggested to me that a sequence number of three digits must / can be specified here. Thus, I started numbering my parameters with imapsieve_mailbox001. However, while researching in various forums, I came across only examples that use imapsieve_mailbox1_ as the first parameter. In desperation I tried this out - now it works for me, too. Long story short - I don't know if I'm the only one who has made this mistake so far. A little hint in the documentation[1] would have helped me, how the XXX is meant exactly. I am referring to this section: ``` This setting configures the name of a mailbox for which administrator scripts are configured. The `XXX’ in this setting is a sequence number, which allows configuring multiple associations between Sieve scripts and mailboxes. The settings defined hereafter with matching sequence numbers apply to the mailbox named by this setting. The sequence of configured mailboxes ends at the first missing imapsieve_mailboxXXX_name setting. This setting supports wildcards with a syntax compatible with the IMAP LIST command, meaning that this setting can apply to multiple or even all (“*”) mailboxes. ``` What do you think about adding a small explanatory sentence to the documentation? For example: ``` This setting configures the name of a mailbox for which administrator scripts are configured. The `XXX’ in this setting is a sequence number, which allows configuring multiple associations between Sieve scripts and mailboxes. The sequence number must begin with the digit 1 without preceding zeros and may have a maximum of three digits. The settings defined hereafter with matching sequence numbers apply to the mailbox named by this setting. The sequence of configured mailboxes ends at the first missing imapsieve_mailboxXXX_name setting. This setting supports wildcards with a syntax compatible with the IMAP LIST command, meaning that this setting can apply to multiple or even all (“*”) mailboxes. ``` Many greetings Matthias [1] https://doc.dovecot.org/configuration_manual/sieve/plugins/imapsieve/

1 0

Restored mail folders conflict with renamed original ones
by Nikolaos Milas 24 Feb '23

24 Feb '23

Hello, We are running Dovecot 2.3.18 (together with Postfix) on CentOS 7. It uses a Maildir structure. We have a user who has organized his mailbox as shows up at the end of this mail. The user had forgotten that he had renamed his "Grants Active \ European" (.Grants Active.European) folder to "Grants Active \ International" (.Grants Active.International), he thought that it was lost and requested a restore of "Grants Active \ European" (with all of its subfolders) from backup. Since we did that (using a simple "cp -rp" from a backup, observing timestamps, ownership and permissions), mail client software won't see "European" mail folder in "Grants Active" mail folder; "International" mail folder shows up but only with two folders, those which do not exist in "European" branch (obviously, these were created later). I have tried to run the command "doveadm index -u userx Grants\ Active" and likewise for each and every mail subfolder under "Grants\ Active"; I have also tried deleting the dovecot.index.cache and dovecot.index.log from the latter ("Grants\ Active") and then run: doveadm -vD force-resync -u userx Grants\ Active but it didn't do the trick: Mail clients are still stuck as I explained. In the end, I thought that the snag might be that mail folders that are not appearing have identical (duplicate) UIDs; As an example: [root@mailx .Grants Active.European]# cat dovecot-uidlist 3 V1548680759 N1 G64e70a1130343f61003c0000715faa8d [root@mailx .Grants Active.European]# cd .. [root@mailx Maildir]# cd .Grants\ Active.International [root@mailx .Grants Active.International]# cat dovecot-uidlist 3 V1548680759 N1 G64e70a1130343f61003c0000715faa8d If so, how should we overcome this issue? I guess I could simply delete the restored folders to return to an acceptable state? If nevertheless we need to restore the ".European" mail folder from a backup (as it was before getting renamed), what would be the right way to do so? Is there another way to resolve the issue without deleting the restored mail folders? Here is the list of mail folders in this mailbox (I have intentionally removed lots of them, to reduce irrelevant content; this mailbox includes a large number of mail folders): # doveadm mailbox list -u userx IGARSS 2024 IAC2022 Junk ... Junk E-mail INBOX.EMISSION ... Trash Trash.Equality Trash.sadf ... Grants Active Grants Active.European Grants Active.European.URBAN RELEAF Grants Active.European.SMURBS Grants Active.European.SMURBS.In Kind Grants Active.European.SMURBS.Final Reports Grants Active.European.RI-URBANS Grants Active.European.QA4EO Grants Active.European.ATMO-ACCESS Grants Active.European.IGOSP Grants Active.European.GAUSS Grants Active.European.E-SHAPE Grants Active.European.CIROCCO Grants Active.European.InCASE Grants Active.European.EIFFEL Grants Active.European.ARSINOE Grants Active.International Grants Active.International.2324 Grants Active.International.GAUSS Grants Active.International.CIROCCO Grants Active.International.IGOSP Grants Active.International.ATMO-ACCESS Grants Active.International.URBAN RELEAF Grants Active.International.InCASE Grants Active.International.ARSINOE Grants Active.International.RI-URBANS Grants Active.International.E-SHAPE Grants Active.International.EIFFEL Grants Active.International.QA4EO Grants Active.International.SMURBS Grants Active.International.SMURBS.Final Reports Grants Active.International.SMURBS.In Kind Grants Active.International.India Grants Active.Urban ReLeaf Grants Active.National Grants Active.National.ΑΓ ΠΑΡΑΣΚΕΥΗ Grants Active.National.ΠΕΤΡΟΥΠΟΛΗ Grants Active.National.CLIMATTICA Grants Active.National.ΝΑΥΠΛΙΟ Grants Active.National.ΕΔΒΜ Grants Active.National.PANGEA Grants Active.National.ΕΛΙΔΕΚ Grants Active.National.ΕΛΙΔΕΚ.OXONIAN - Δέσποινα Grants Active.National.ΕΛΙΔΕΚ.Νασια PhD Grants Active.National.CLIMPACT Sent ... INBOX Please, provide some guidance on what we should do to make these mail folders work correctly again. Sorry, I am not very knowledgeable on mail inner architecture and operations; please let me know if you need additional info. Thanks in advance, Nick

2 6

How to get a server listed in the IMAP Test wiki?
by Leander Beernaert 24 Feb '23

24 Feb '23

Hey, We recently announced Gluon (https://github.com/ProtonMail/gluon/) our IMAP server library we are using in Proton Bridge(https://github.com/ProtonMail/proton-bridge). We would love to have it have it listed in the IMAP Server Compliancy Status wiki page (https://imapwiki.org/ImapTest/ServerStatus). What do we need to do or whom do we need to contact to make this happen? Additionally, We have been using running imaptest (https://github.com/dovecot/imaptest) against our server library, but due to variety of configuration parameters, we would really appreciate it (if possible) if someone could point out to us the test setup used to validate each of those servers. Thank you in advance for your time. Kind Regards, Leander Beernaert Proton AG

4 6