- dovecot - dovecot.org

Missing mailboxes after migration from Maildir -> mdbox
by Ansgar 08 Mar '23

08 Mar '23

I migrated from existing dovecot Maildir to dbox. After adding separator to the namespace it migrated all Inbox mails, but the other mailboxes like user-created ones and "Junk", "Sent" and so on has not been migrated. Any advice would be helpful, here is my dovconf: https://pastebin.com/rDSeBcLe My migration I made using: 1. Edit dovecot.conf with new mdbox and adding separator 2. dsync mirror per user with old maildir:~/mail 3. missing mailboxes: tried with doveadm sync also with -f mode and specified mailboxes. No luck. dsync -v -u admin(a)sajdowski.de mirror maildir:~/mail mail_home = /var/vmail/mailboxes/%d/%n #mail_location = maildir:~/mail:LAYOUT=fs mail_location=mdbox:~/mdbox My directorys: root@mail:/var/vmail/mailboxes/example.de/admin# ls -l insgesamt 56 drwxrwx--- 2 vmail vmail 4096 8. Mär 01:14 cur -rw-rw---- 1 vmail vmail 160 8. Mär 01:14 dovecot.index.log -rw-rw---- 1 vmail vmail 3712 8. Mär 01:14 dovecot.list.index.log -rw-rw---- 1 vmail vmail 96 8. Mär 01:14 dovecot.mailbox.log -rw-rw---- 1 vmail vmail 51 8. Mär 01:14 dovecot-uidlist -rw-rw---- 1 vmail vmail 8 8. Mär 01:14 dovecot-uidvalidity -r--r--r-- 1 vmail vmail 0 8. Mär 00:57 dovecot-uidvalidity.6407cf6d drwxrwx--- 20 vmail vmail 4096 8. Mär 02:51 mail drwx------ 7 vmail vmail 4096 8. Mär 13:52 mailboxes drwx------ 13 vmail vmail 4096 8. Mär 02:36 Maildir drwx------ 5 vmail vmail 4096 8. Mär 04:20 mdbox drwxrwx--- 2 vmail vmail 4096 8. Mär 01:14 new -rw-rw---- 1 vmail vmail 28 8. Mär 01:14 subscriptions drwxrwx--- 2 vmail vmail 4096 8. Mär 01:14 tmp drwxrwx--- 2 vmail vmail 4096 8. Mär 00:57 xapian-indexes root@mail:/var/vmail/mailboxes/example.de/admin/mail# ls -l insgesamt 87576 drwxrwx--- 2 vmail vmail 9113600 8. Mär 02:20 cur -rw-rw---- 1 vmail vmail 1602296 8. Mär 02:20 dovecot.index -rw-rw---- 1 vmail vmail 73030876 8. Mär 02:41 dovecot.index.cache -rw-rw---- 1 vmail vmail 441656 8. Mär 02:39 dovecot.index.log -rw-rw---- 1 vmail vmail 83568 8. Mär 02:20 dovecot.index.log.2 -rwxrwx--- 1 vmail vmail 206490 17. Jun 2019 dovecot.index.thread -rwxrwx--- 1 vmail vmail 51 2. Aug 2019 dovecot-keywords -rw-rw---- 1 vmail vmail 1392 8. Mär 01:25 dovecot.list.index -rw-rw---- 1 vmail vmail 6712 8. Mär 02:41 dovecot.list.index.log -rwxrwx--- 1 vmail vmail 1464 8. Mär 02:20 dovecot.mailbox.log -rw-rw---- 1 vmail vmail 4875528 8. Mär 02:20 dovecot-uidlist -rwxrwx--- 1 vmail vmail 8 8. Mär 01:25 dovecot-uidvalidity -rwxrwx--- 1 vmail vmail 0 17. Jun 2019 dovecot-uidvalidity.5a6912f0 drwxrwx--- 5 vmail vmail 4096 7. Mär 20:14 Drafts -rw-rw---- 1 vmail vmail 49 6. Mär 11:10 maildirsize drwxrwx--- 5 vmail vmail 4096 7. Mär 20:44 'Mailling Listen' drwxrwx--- 2 vmail vmail 204800 8. Mär 00:04 new drwxrwx--- 5 vmail vmail 4096 7. Mär 20:14 Sent drwxrwx--- 5 vmail vmail 4096 7. Mär 20:44 Spam drwxrwx--- 5 vmail vmail 4096 26. Feb 19:44 'SPF Fail' -rw-rw---- 1 vmail vmail 23 8. Mär 02:20 subscriptions drwxrwx--- 2 vmail vmail 4096 8. Mär 02:20 tmp drwxrwx--- 5 vmail vmail 4096 26. Feb 19:12 Trash drwxrwx--- 5 vmail vmail 4096 7. Mär 22:30 ubb drwxrwx--- 10 vmail vmail 4096 27. Feb 18:44 xapian-indexes

1 0

index locally
by tomate aceite 08 Mar '23

08 Mar '23

Hello, Do you know if is possible to store indexes locally instead of NFS in dovecot 2.2.7 ? Thanks.

3 2

Dovecot "Mailbox sharing between users".
by Jobst Schmalenbach 07 Mar '23

07 Mar '23

Hi I have appended the important bits of my server configuration to the end of this email I am trying to get "Mailbox sharing between users" to work. For starters it is very hard to follow the doc only https://doc.dovecot.org/configuration_manual/shared_mailboxes/shared_mailbo… without an actual example. I tried to use this (sorry in German) https://dokuwiki.tachtler.net/doku.php?id=tachtler:dovecot_shares and this got me a little further down the track. I followed the instructions required for the dovecot config files. Restarted dovecot, not errors displayed in the logs. I have got access to two user accounts, say USER1 and USER2. I executed this lines doveadm acl set -u USER1 INBOX user=USER2 admin create delete expunge insert lookup post read write write-deleted write-seen doveadm acl set -u USER2 INBOX user=USER1 admin create delete expunge insert lookup post read write write-deleted write-seen and created (just to be sure) two folders for each user /home/USER1/Mail/shared/USER1 /home/USER1/Mail/shared/USER2 /home/USER2/Mail/shared/USER1 /home/USER2/Mail/shared/USER2 The tried to create a mailboxes like so /home/USER1/Mail/shared/USER1/testa /home/USER1/Mail/shared/USER2/testb /home/USER2/Mail/shared/USER1/testa /home/USER2/Mail/shared/USER2/testb They are not created and no error is displayed - even looking at the dovecot logs. What I see in the folders for BOTH users though (for every folder) dovecot.list.index.log On the command line I can see the .imap files and do a 'ls -alR' in the folders, but no created mbox. I can definitely see the folders, but I cannot create a mbox file. What am I doing wrong? Help please. --------------------------------------------------------------------------------------------------------------------------------------------------------------- Here are the important configuration items # 2.3.16 (7e2e900c1a): /etc/dovecot/dovecot.conf # OS: Linux 4.18.0-425.13.1.el8_7.x86_64 x86_64 AlmaLinux release 8.7 (Stone Smilodon) xfs namespace shared { list = children location = maildir:~/Mail/:INDEX=~/Mail/shared/%%u:CONTROL=~/Mail/shared/ prefix = shared/%%u/ separator = / subscriptions = yes type = shared } mail_location = mbox:/home/%u/Mail/:INBOX=/var/mail/%u mail_plugins = acl passdb { driver = pam } plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db } service auth { unix_listener auth-userdb { group = dovecot mode = 0777 user = dovecot } } protocol imap { mail_plugins = " acl imap_acl" } --

1 0

Flags/tags lost when moving e-mails to another folder
by Alexis Ben Miloud--Josselin 06 Mar '23

06 Mar '23

Hi, I have an issue with e-mail tags/flags. The IMAP user is using Thunderbird or Roundcube with Thunderbird tags. When the user (with an IMAP client) moves an e-mail from a folder to another, tags set on the e-mail are lost. When we check the filesystem, we can see the "a" tag: -rw-rw---- 2 test mailusers 2498 Dec 29 15:31 'cur/1676643827.M563793P15241.<HOSTNAME>,S=2498,W=2551:2,a' When we move the e-mail to another folder, the tag disappear: -rw-rw---- 2 test mailusers 2498 Dec 29 15:31 '.test/cur/1676644340.M214404P22014.<HOSTNAME>,S=2498,W=2551:2,' In both folders, the dovecot-keywords file is: 0 $LABEL1 The problem appeared after a major system upgrade (Debian 9 to 10). Dovecot version went from 2.2.27 to 2.3.4.1. Some system informations: # dovecot --version 2.3.4.1 (f79e8e7e4) # cat /etc/debian_version 10.13 # uname -a Linux <HOSTNAME> 4.19.0-23-amd64 #1 SMP Debian 4.19.269-1 (2022-12-20) x86_64 GNU/Linux # findmnt /var TARGET SOURCE FSTYPE OPTIONS /var /dev/vda3 ext4 rw,nosuid,relatime # findmnt /home TARGET SOURCE FSTYPE OPTIONS /home /dev/vdb ext4 rw,nosuid,nodev,noexec,relatime The output of `dovecot -n output` is here (link expires in one year): https://paste.evolix.org/?96d46acaea638568#56g7AQLuzJxxy4xintpexJCPgiAAG23c… Here are the operations I see on an e-mail file using strace: 15507 link("/home/test/Maildir/cur/1672326270.M619318P11927.<HOSTNAME>,S=2498,W=2551:2,Sa", "/home/test/Maildir/.test/tmp/1672326456.M737812P15507.<HOSTNAME>") = 0 15507 rename("/home/test/Maildir/.test/tmp/1672326456.M737812P15507.<HOSTNAME>", "/home/test/Maildir/.test/cur/1672326456.M737813P15507.<HOSTNAME>,S=2498,W=2551:2,S") = 0 15507 unlink("/home/test/Maildir/cur/1672326270.M619318P11927.<HOSTNAME>,S=2498,W=2551:2,Sa") = 0 Setting mbox_lazy_writes to yes and/or unsetting mail_cache_fields doesn't change anything. Any idea of what I am missing? Could this be a bug in Dovecot? Thanks in advance, -- Alexis Ben Miloud--Josselin <abenmiloud(a)evolix.fr> Evolix - Hébergement et Infogérance Open Source Marseille (37 rue Guibal, Pôle Média, 13003) / Paris / Montréal http://evolix.com | Twitter: @Evolix @EvolixNOC | http://blog.evolix.com

2 2

Re: Corrupted sizes using zlib plugin
by Tim Evers 06 Mar '23

06 Mar '23

Am 06.03.23 um 11:42 schrieb Aki Tuomi: >> On 06/03/2023 12:32 EET Tim Evers <te-ml-ext(a)artfiles.de> wrote: >> >> >> Hi, >> >> since I did not get any feedback on my bug report post regarding >> corrupted sizes while using zlib >> (https://dovecot.org/pipermail/dovecot/2023-February/126105.html) I >> would like to confirm that this bug report reached someone in charge. >> >> Or if not - I would kindly ask for directions on how to post it in a way >> that spawns some action. >> >> Thx >> >> Tim > Can you try erasing dovecot.index.cache and see if the problem fixes itself? It's possible your cache contains invalid values from old errors. > > Aki Did that already several times (other dovecot.index.* files too). The only thing that helps is actually deleting the dovecot-uidlist file but the problem returns. Tim

2 3

Corrupted sizes using zlib plugin
by Tim Evers 06 Mar '23

06 Mar '23

Hi, since I did not get any feedback on my bug report post regarding corrupted sizes while using zlib (https://dovecot.org/pipermail/dovecot/2023-February/126105.html) I would like to confirm that this bug report reached someone in charge. Or if not - I would kindly ask for directions on how to post it in a way that spawns some action. Thx Tim

2 1

Setting up the mail-crypt plugin with virtual accounts that have no home directories
by Jeremy 05 Mar '23

05 Mar '23

Hi again, I am using dovecot 2.3.16, along with postfix and a PostgreSQL database for managing virtual accounts. After an initial topic from me about encrypting already existent mail, I could now use some pointers on how to set up the mail-crypt plugin for pure virtual accounts (i.e. that have no matching system users and/or home directories. I hope somebody can clarify a few things that are not entirely clear to me yet. After doing my own research, I believe the following should be possible: I'd like to use the password of virtual email accounts to let dovecot encrypt/decrypt the keys needed to encrypt/decrypt the mail in the relevant folders. As per the documentation @ https://doc.dovecot.org/configuration_manual/mail_crypt_plugin/ I believe this would be all the configuration I need: # Config mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_plugins = $mail_plugins mail_crypt plugin { mail_crypt_curve = secp521r1 # or some other preferred curve mail_crypt_save_version = 2 mail_crypt_require_encrypted_user_key = yes # necessary for encrypting keys with user password } # File: /etc/dovecot/dovecot-sql.conf.ext password_query = SELECT \ email as user, password, \ '%w' AS userdb_mail_crypt_private_password \ FROM virtual_users WHERE email='%u'; My first question is: Is it possible to configure mail_attribute_dict in such a way as to not use home directories. Since I only use virtual accounts, without those accounts having home directories, can I somehow tell dovecot to save the attributes into the PostsreSQL database, for instance? If not, can you suggest another approach, without having to create home directories for virtual users? My second question is: The documentation warns about not using password directly in the above SQL query: > Choosing encryption key > DO NOT use password directly. It can contain % which is interpreted as > variable expansion and can cause errors. Does this refer to not accidentally substituting '%w' with password? In other words, if I leave the above query as is, should I be good, even if plain text passwords of users potentially have % signs in them? Or would I need to take further measurements? (The passwords in my database are already hashed, by the way). I hope somebody can offer some guidance on this. Thanks.

2 7

bug report: imap-login not killed
by Sacha 05 Mar '23

05 Mar '23

Hi, We have growing imap-login processes until we reach the max processes. This occurs when a particular user have a login error due to our LDAP misconfiguration: --- Mar 4 14:59:33 hera dovecot[2226963]: auth: Error: plain(john.doe,XX.XX.XX.XX,<13C0eBP2354lqXpO>): user not found from any userdbs Mar 4 14:59:33 hera dovecot[2226963]: imap: Error: auth-master: login: request [1001652225]: Login auth request failed: Authenticated user not found from userdb, auth lookup id=1001652225 (auth connected 2 msecs ago, request took 1 msecs, client-pid=2235348 client-id=1) Mar 4 14:59:33 hera dovecot[2226963]: imap-login: Internal login failure (pid=2235348 id=1): user=<ohn.doe>, method=PLAIN, rip=XX.XX.XX.XX, lip=18 5.233.100.1, mpid=2235359, TLS, session=<13C0eBP2354lqXpO> --- The origin of is issue is when passdb finds the user and userdb not. The result is imap-login processes with no timeout growing until we reach the max processes. * OS version Debian stable - Bullseye * Dovecot version dpkg -l |grep dovecot ii dovecot-antispam 2.0+20171229-1+b7 amd64 Dovecot plugins for training spam filters ii dovecot-common 1:2.1.7-7+deb7u1 all Transitional package for dovecot ii dovecot-core 1:2.3.13+dfsg1-2+deb11u1 amd64 secure POP3/IMAP server - core files ii dovecot-core-dbgsym 1:2.3.13+dfsg1-2+deb11u1 amd64 debug symbols for dovecot-core ii dovecot-imapd 1:2.3.13+dfsg1-2+deb11u1 amd64 secure POP3/IMAP server - IMAP daemon ii dovecot-imapd-dbgsym 1:2.3.13+dfsg1-2+deb11u1 amd64 debug symbols for dovecot-imapd ii dovecot-ldap 1:2.3.13+dfsg1-2+deb11u1 amd64 secure POP3/IMAP server - LDAP support ii dovecot-managesieved 1:2.3.13+dfsg1-2+deb11u1 amd64 secure POP3/IMAP server - ManageSieve server ii dovecot-mysql 1:2.3.13+dfsg1-2+deb11u1 amd64 secure POP3/IMAP server - MySQL support ii dovecot-pop3d 1:2.3.13+dfsg1-2+deb11u1 amd64 secure POP3/IMAP server - POP3 daemon ii dovecot-sieve 1:2.3.13+dfsg1-2+deb11u1 amd64 secure POP3/IMAP server - Sieve filters support * doveconf -n --- # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) doveconf: Warning: service auth { client_limit=1000 } is lower than required under max. load (4096) doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (4099) # OS: Linux 5.10.0-21-cloud-amd64 x86_64 Debian 11.6 ext4 # Hostname: XXX auth_mechanisms = plain login default_process_limit = 1024 first_valid_gid = 8 first_valid_uid = 109 last_valid_gid = 8 last_valid_uid = 109 login_greeting = XXX listening. mail_access_groups = mail mail_gid = 8 mail_location = maildir:/srv/vmail/%d/%n mail_privileged_group = mail mail_uid = 109 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy i nclude variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags notify imapsieve vnd.dovecot.imapsieve vnd.dovecot.filter namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = /etc/dovecot/dovecot-ldap-girondix.conf driver = ldap } passdb { args = /etc/dovecot/dovecot-oauth2.conf.ext driver = oauth2 mechanisms = xoauth2 oauthbearer } plugin { quota_rule = *:storage=100M quota_rule2 = Trash:storage=10%% recipient_delimiter = + sieve = /srv/vmail/%d/%n/dovecot.sieve sieve_default = /var/lib/dovecot/sieve/default.sieve sieve_dir = /srv/vmail/%d/%n/sieve sieve_extensions = +notify +imapflags +vnd.dovecot.filter sieve_filter_bin_dir = /usr/lib/dovecot/sieve-filter sieve_global_extensions = +vnd.dovecot.pipe sieve_max_script_size = 1M sieve_pipe_bin_dir = /etc/dovecot/sieve sieve_pipe_socket_dir = sieve-pipe sieve_plugins = sieve_imapsieve sieve_extprograms sieve_redirect_envelope_from = orig_recipient } protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = mail mode = 0600 user = postfix } unix_listener auth-master { group = mail mode = 0660 user = vmail } unix_listener auth-userdb { group = mail mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 0 } } service imap { vsz_limit = 1 G } service lmtp { executable = lmtp -L process_min_avail = 2 } service pop3-login { inet_listener pop3 { port = 0 } } ssl_cert = </srv/letsencrypt/pem/mail.aquilenet.fr.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } userdb { args = /etc/dovecot/dovecot-ldap-girondix.conf driver = ldap } protocol lmtp { auth_username_format = %n info_log_path = /var/log/dovecot-lmtp.log mail_plugins = " sieve" } protocol lda { mail_plugins = " mailbox_alias sieve" } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep imap_idle_notify_interval = 30 secs mail_max_userip_connections = 50 mail_plugins = quota mailbox_alias imap_quota } protocol pop3 { mail_plugins = quota mailbox_alias acl pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } ---

2 2

Multiple scripts with postlogin
by Dan Conway 03 Mar '23

03 Mar '23

Hello, I'm having trouble understanding how to execute separate scripts with postlogin. According to the documentation, it should be as simple as: You can run multiple post-login scripts by just giving multiple scripts as parameters to|script-login|, for example: executable = script-login rawlog /usr/local/bin/postlogin.sh /usr/local/bin/postlogin2.sh Given this information, I supplied two scripts to service imap-postlogin below: service imap { executable = imap imap-postlogin } service imap-postlogin { executable = script-login -d rawlog /usr/local/bin/postlogin.pl /usr/local/bin/postlogin2.pl user = $default_internal_user unix_listener imap-postlogin { } } Both scripts simply print "Script 1" and "Script 2" to STDERR. Here is what they look like: #!/usr/bin/perl use strict; use warnings; print STDERR "Script 1\n"; system("/usr/lib/dovecot/script-login"); After logging in, only the first script is executed. The second one is ignored. I tried not executing "/usr/lib/dovecot/script-login" in the first script, in hopes that it would execute the second script, but it only denies the connection after printing "Script 1" imap-postlogin: Error: Script 1 imap(<user>): Error: Post-login script denied access to user <user> Am I missing something? Thanks.

2 1

OAuth2: local validation with RFC9068 tokens
by Tomas Habarta 01 Mar '23

01 Mar '23

Hello, my IdP is kind of progressive and implemented RFC9068, where all access tokens now come with typ "at+JWT". Since the setup has used local validation, I had to switch and currently use introspection endpoint. Looked around at the src and there seems to be relatively simple check of the token typ checking the only fixed value of "JWT" -- do you think you could consider tuning it a little bit so that local validation works also with such tokens? I am not an expert on OAuth2 so have no idea whether this is a valid request, but think that such a token is still JWT but has the required structure per RFC, which should not anyhow be in collision with a simple "JWT" typ. Saying that, I would not wonder if the statement is not correct :) Many thanks, Tomas

1 0