December 2021 - dovecot

FTS solr or xapian and Public Folders
by David Williams 13 May '22

13 May '22

Hi all I hope someone can help with this. I'm looking at setting up FTS on a Dovecot server which has some quite big pubic folders (100,000+ emails each) and would appreciate some info about how fts both Xapian and Solr works with public folders. As background of the folders can receive emails and I use a dummy user and an alias file to deliver them for example testfolder: dummy+Public/MYTESTFOLDER(a)example.com delivers email addressed to testfolder(a)example.com to MYTESTFOLDER in the namespace Public. Has anyone successfully setup FTS with public folders using either Xapian or Solr and what happens when one user changes a folder, does that change show up for other users? E.G.: When an email is delivered by dummy and indexed will that email be indexed for other users as well? Similarly what happens that when an email is moved between, deleted from or copied to public folders by one user will that change then show up for the other users or not? I guess the basic question is: Are public folders indexed once and then the indexes are made available to each user with access or are they indexed for each user? Any info or experience that anyone has will be appreciated. Many thanks David

2 1

Private \seen flags lost during message move for other users
by Cube . 28 Mar '22

28 Mar '22

Hello, I have a shared mailbox set up with a private index: maildir:/var/mail/public:INDEXPVT=~/Maildir/public Per user \Seen flags are working fine until one user moves a message into another folder/sub-mailbox. For that user doing the move the message remains \Seen however for other users who have already seen the message the flag is lost and it appears unread in the new location. If the message is moved back it still appears unread to the other user even if had been read in that folder/sub-mailbox in the past. Is there anyway to retain per user \Seen that follows the message when its moved? Thank you.

2 1

imap_metadata plugin panic
by Elisamuel Resto 23 Mar '22

23 Mar '22

Hello, This may be covered somewhere but recently I enabled the metadata plugin to work with sieve as part of some updates I did a while back and I hadn't checked my logs for any issues with it and so far they're not completely taking my system down. Right now, upon trying to delete a folder I noticed the following in my logs... what am I missing? I see the broken SQL query, but I don't know enough about the dict system or the metadata plugin to know how to add the missing information or fix it otherwise. Regards, Elisamuel Resto Nov 15 12:19:19 wyvern dovecot[461]: dict(51438): Panic: lib-sql: Too many bind args (2) for statement: SELECT meta_key FROM metadata WHERE meta_key LIKE AND username = ? Nov 15 12:19:19 wyvern dovecot[461]: dict(51438): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x43) [0x7f449789d073] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x20) [0x7f449789d190] -> /usr/lib/dovecot/libdovecot.so.0(+0xfaf1f) [0x7f44978a9f1f] -> /usr/lib/dovecot/libdovecot.so.0(+0xfafb1) [0x7f44978a9fb1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4cd20) [0x7f44977fbd20] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](+0x875a) [0x555e60d7775a] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](sql_statement_query+0x42) [0x555e60d7f262] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](+0xd97f) [0x555e60d7c97f] -> /usr/lib/dovecot/libdovecot.so.0(dict_iterate_values+0x25) [0x7f4497868615] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](+0xa929) [0x555e60d79929] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](+0xb224) [0x555e60d7a224] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](+0xb381) [0x555e60d7a381] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](dict_command_input+0xd9) [0x555e60d7a579] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](+0x95b8) [0x555e60d785b8] -> /usr/lib/dovecot/libdovecot.so.0(connection_input_default+0x15e) [0x7f44978a16ce] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6b) [0x7f44978bfebb] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x13b) [0x7f44978c15cb] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x51) [0x7f44978bff61] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x41) [0x7f44978c0131] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x14) [0x7f4497831f74] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](main+0x189) [0x555e60d78139] -> /usr/lib/libc.so.6(__libc_start_main+0xd5) [0x7f44972cfb25] -> dovecot/dict [0 clients, 0 lookups:0/0/0/0, 0 iters:0/0/0/0, 0 commits:0/0/0/0](_start+0x2e) [0x555e60d7819e] Nov 15 12:19:19 wyvern dovecot[461]: imap(sam(a)samresto.dev)<51449><+BGq2NfQM/7Pisr9>: Error: Mailbox Trash/Processed: dict_iterate(priv/c841ad0291c27461ac670100a07d9965/) failed: Connection closed (reply took 0.204 secs (0.204 in dict wait, 0.000 in other ioloops, 0.000 in locks)) Nov 15 12:19:19 wyvern dovecot[461]: dict(51438): Fatal: master: service(dict): child 51438 killed with signal 6 (core dumped)

4 12

Dovecot v2.3.17.1 Released
by Aki Tuomi 07 Feb '22

07 Feb '22

We are happy to announce 2.3.17.1 patch release of Dovecot. This contains some fixes for issues found after 2.3.17 release. https://dovecot.org/releases/2.3/dovecot-2.3.17.1.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.17.1.tar.gz.sig Binary packages in https://repo.dovecot.org/ Docker images in https://hub.docker.com/r/dovecot/dovecot Aki Tuomi Open-Xchange oy -- - dsync: Add back accidentically removed parameters. - lib-ssl-iostream: Fix assert-crash when OpenSSL returned syscall error without errno. - master: Dovecot failed to start if ssl_ca was too large.

7 8

test-file-cache.c needs #ifdef HAVE_RLIMIT_AS
by Rupert Gallagher 07 Feb '22

07 Feb '22

test-file-cache.c:259:24: error: use of undeclared identifier 'RLIMIT_AS' test_assert(getrlimit(RLIMIT_AS, &rl_cur) == 0); ^ test-file-cache.c:267:24: error: use of undeclared identifier 'RLIMIT_AS' test_assert(setrlimit(RLIMIT_AS, &rl_new) == 0); ^ test-file-cache.c:270:24: error: use of undeclared identifier 'RLIMIT_AS' test_assert(setrlimit(RLIMIT_AS, &rl_cur) == 0); ^ test-file-cache.c:276:24: error: use of undeclared identifier 'RLIMIT_AS' test_assert(setrlimit(RLIMIT_AS, &rl_new) == 0); ^ test-file-cache.c:279:24: error: use of undeclared identifier 'RLIMIT_AS' test_assert(setrlimit(RLIMIT_AS, &rl_cur) == 0); ^

3 9

v2.3.17 quota-dict related sql errors
by Pascal Volk 06 Feb '22

06 Feb '22

Hi there, I've updated Dovecot from v2.3.14 (cee3cbc0d) to v2.3.17 (e2aa53df5b). Right after the update, dovecot begun flooding the mail.log and PostgreSQL's log. My dict sql configuration: ,--[ .../dovecot/dovecot-dict-sql.conf.ext ]-- | map { | pattern = priv/quota/storage | table = userquota | username_field = uid | value_field = bytes | } | map { | pattern = priv/quota/messages | table = userquota | username_field = uid | value_field = messages | } `-- This worked for decades. Since the update dovecot is passing the user's email address, instead the user's UID to the query. And therefor I have lots of errors in the log files: see attached `mail.log' and `postgresql-14-main.log' excerpts. The `userquota' table looks like this: Table "public.userquota" Column | Type | Collation | Nullable | Default ----------+---------+-----------+----------+--------- uid | bigint | | not null | bytes | bigint | | not null | 0 messages | integer | | not null | 0 ... Foreign-key constraints: "fkey_userquota_uid_users" FOREIGN KEY (uid) REFERENCES users(uid) ON DELETE CASCADE (full database structure: https://bitbucket.org/pvo/vmm/src/vmm-0.7.0/pgsql/create_tables.pgsql) Regards, Pascal -- Ubuntu is an ancient African word meaning “I can’t install Debian.” -- unknown

3 4

Dsync replication - delayed replication (Sync lock)
by Daniel Botting 31 Jan '22

31 Jan '22

Hi, *Our setup:* Two Debian 10 machines that are setup to replicate mail between them, we have round robin DNS setup so a user can connect to either server. *What should happen:* Mail is delivered to either server and replicated across straight away to their mailbox on the other server so it does not matter which one they are connected to they will receive it fairly soon after delivery. *What actually happens:* In some instances the user will experience a delayed receipt of messages if they are not connected to the server that the message is initially delivered to, sometimes the delay is 5/10 minutes, we had a recent support ticket submitted where it was over an hour. Error message seen in mail.err: Sep 1 10:16:15 <host name> dovecot: dsync-local(<username>)<WELSGEERTl8FEAAAZJ7A9A>: Error: Couldn't lock /path/to/mailbox/.dovecot-sync.lock: fcntl(/path/to/mailbox/.dovecot-sync.lock, write-lock, F_SETLKW) locking failed: Timed out after 30 seconds (WRITE lock held by pid 3697) Process 3697 is dovecot/doveadm-server. *Doveconf -n output:* # 2.3.10.1 (a3d0e1171): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.10 (67bf5bd7) # OS: Linux 4.19.0-10-amd64 x86_64 Debian 10.5 # Hostname: <server hostname> auth_verbose = yes default_vsz_limit = 0 doveadm_password = # hidden, use -P to show it first_valid_gid = 8 first_valid_uid = 8 last_valid_gid = 8 last_valid_uid = 8 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_gid = 8 mail_location = maildir:~/Maildir mail_plugins = " notify replication" mail_uid = 8 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex im ap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext editheader imapfla gs namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { mail_replica = tcps:<server being replicated to>:<port being used> sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +editheader +imapflags } postmaster_address = postmaster@<domain address> protocols = " imap sieve pop3" replication_max_conns = 12 service aggregator { fifo_listener replication-notify-fifo { user = mail } unix_listener replication-notify { user = mail } } service auth { unix_listener /var/run/dovecot-exim-bridge { mode = 0660 user = Debian-exim } } service doveadm { inet_listener { port = <port being used for replication> ssl = yes } } service imap-login { inet_listener imap { port = 143 } } inet_listener imaps { port = 993 ssl = yes } process_limit = 512 process_min_avail = 4 service_count = 1 } service imap { process_limit = 1024 } service managesieve-login { inet_listener sieve { port = <sieve port> } process_min_avail = 1 service_count = 8 vsz_limit = 256 M } service managesieve { process_limit = 1024 } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } ssl = required ssl_cert = </path/to/ssl/cert/fullchain.pem ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL ssl_client_ca_dir = /etc/ssl/certs ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap override_fields = home=/var/mail/%u uid=mail gid=mail } protocol lda { mail_plugins = " notify replication sieve" } protocol imap { mail_max_userip_connections = 32 mail_plugins = " notify replication" } protocol sieve { mail_max_userip_connections = 10 managesieve_logout_format = bytes=%i/%o managesieve_max_line_length = 65536 } Kind regards Daniel

3 3

Re: Expunged message reappeared
by Claudio Corvino 05 Jan '22

05 Jan '22

Hi Alex, thanks for your answer, I think this could be related to Dovecot version in use, in our case it is the same: 2.3.4.1 (f79e8e7e4). It could be useful to know if someone can confirm this assumption so we can try an upgrade to the latest version hoping this could solve our issue. Regards On 07/12/21 08:36, Joelly Alexander wrote: > > Don't have the issue with duplicate logs but same problem with > re-appearing of deleted messages very often in a two node setup with > dsync replication. > Messages sometimes have to be deleted multiple times until they stay > in the thrash folder. > > Running dovecot version 2.3.4.1 (f79e8e7e4) > > When i 'move' messages to the Trash folder they seem to stay there but > when i delete them they usually come back. > > Alex > > On 12/3/21 5:53 PM, Claudio Corvino wrote: >> >> I forgot to mention that sometimes users receive duplicate messages >> and Dovecot log: >> >> Dec 3 10:16:12 xxx dovecot: >> lmtp(xxx@xxx)<13468><OOFDAlXgqWGcNAAAIQnY6g:12>: Warning: >> /mnt/mail-storage-lv0007/xxx/Maildir/dovecot-uidlist: Duplicate file >> entry at line 2072: 1638522075.M489729P12338.xxx,S=5854,W=5943 (uid >> 71164 -> 71166) >> >> Thanks >> >> Regars >> >> On 03/12/21 17:50, Claudio Corvino wrote: >>> >>> Hi, >>> >>> I have two IMAP/LMTP Dovecot server in replica (version 2.3.4.1), I >>> use LDAP/AD for /userdb, /replica is working, but sometimes I get >>> these errors into log: >>> >>> Dec 3 15:47:15 xxx dovecot: imap(xxx)<28251><TOSxmj7SPRYloY5/>: >>> Warning: Maildir /mnt/mail-storage-lv0007/xxx/Maildir: Expunged >>> message reappeared, giving a new UID (old uid=233140, >>> file=1638540323.M588041P27708.xxx,S=84376,W=85507:2,) >>> >>> The user delete the message and then the message reappears into mail >>> client after one second. >>> >>> Does anybody knows how can I fix it? >>> >>> Thanks in advance >>> >>> Regards >>> >>

2 3

how to delete attachment and replace some words in subject
by lujie 05 Jan '22

05 Jan '22

hi, all, I need to filter email as follow rules: 1, delete the attachments; 2, replace some word in subject with code; 3, redirect to other email with the modified subject and original text in body. sieve and exrprograms used in my script. But error in run sieve script while try to set body text , which contains invalid character "cr lf". Thanks. -- Best Regards, Jessica

2 1

Permission denied
by Ken Wright 02 Jan '22

02 Jan '22

My server has been running just fine for over a year, until I upgraded PostfixAdmin and somehow messed things up. Now I can't log in, and my mail client (Evolution, if it matters) keeps popping up an authentication request. The server is Ubuntu 20.04, my Dovecot is version 2.3.7.2, and Evolution is 3.40.4-1. I've also tried logging in with Roundcube with no success. Any insight or suggestions will be gratefully received! Here's the most recent entry from /var/log/mail.err: Dec 28 22:05:02 grace dovecot: lmtp(me(a)mydomain.com)<910309><SOFiMl7Qy2Hl4w0AThsuZQ>: Error: lmtp- server: conn unix:pid=906625,uid=117 [6]: rcpt me(a)mydomain.com: Mailbox INBOX: open(/var/mail/vmail/me(a)mydomain.com/tmp/1640747102.M850878P910309.grac ) failed: Permission denied Any further information available upon request!

3 6