- dovecot - dovecot.org

doveadm search error
by Christoph Fürstaller 20 May '22

20 May '22

Hello, I've upgraded dovecote to 2.3.19 (b3ad6004dc) a week ago. Since then I get an error when searching for messages from all users in a particular folder: doveadm search -A mailbox 'INBOX.learn' Error: auth-master: userdb list: Disconnected unexpectedly doveadm: Error: Failed to iterate through some users This is in the logs: dovecot: auth: Panic: file userdb-blocking.c: line 124 (userdb_blocking_iter_next): assertion failed: (ctx->conn != NULL) dovecot: auth: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x41) [0x7f1c12675c91] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x22) [0x7f1c12675db2] -> /usr/lib/dovecot/libdovecot.so.0(+0x10b0bb) [0x7f1c126830bb] -> /usr/lib/dovecot/libdovecot.so.0(+0x10b157) [0x7f1c12683157] -> /usr/lib/dovecot/libdovecot.so.0(+0x5d375) [0x7f1c125d5375] -> dovecot/auth [0 wait, 0 passdb, 0 userdb](+0x157a7) [0x55e0278287a7] -> dovecot/auth [0 wait, 0 passdb, 0 userdb](+0x1954b) [0x55e02782c54b] -> dovecot/auth [0 wait, 0 passdb, 0 userdb](+0x36ca7) [0x55e027849ca7] -> dovecot/auth [0 wait, 0 passdb, 0 userdb](+0x2ab86) [0x55e02783db86] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0x15f) [0x7f1c1269976f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xcf) [0x7f1c1269b02f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x54) [0x7f1c12699a54] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f1c12699bc0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x17) [0x7f1c1260b207] -> dovecot/auth [0 wait, 0 passdb, 0 userdb](main+0x3c8) [0x55e027829588] -> /lib/x86_64-linux-gnu/libc.so.6(+0x29fd0) [0x7f1c122cbfd0] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x7d) [0x7f1c122cc07d] -> dovecot/auth [0 wait, 0 passdb, 0 userdb](_start+0x2e) [0x55e02782976e] dovecot: auth: Fatal: master: service(auth): child 19452 killed with signal 6 (core dumped) I've system users from passwd and users from a passwd-file: userdb { driver = passwd } userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } If I disable the passwd-file, a search works fine, but not with both user backends. This worked fine with version 2.3.18. Someone discovered this behavior too? This is my config: # 2.3.19 (b3ad6004dc): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.19 (4eae2f79) # OS: Linux 5.13.0-41-generic x86_64 Ubuntu 21.10 auth_mechanisms = plain login auth_username_format = %n default_vsz_limit = 2 G mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . } passdb { driver = pam } passdb { args = scheme=ARGON2I username_format=%u /etc/dovecot/users driver = passwd-file } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /var/mail/moveSpam.sieve } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 0 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = required ssl_cert = </etc/letsencrypt/live/mailserver/fullchain.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { driver = passwd } userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } verbose_proctitle = yes protocol lmtp { mail_plugins = " sieve" } protocol lda { mail_plugins = " sieve" } protocol imap { mail_max_userip_connections = 30 } Best regards, Christoph

2 1

%{listener} not working
by cesco 19 May '22

19 May '22

Hi I'm trying to use the variable %{listener} in the following password mysql query: password_query = SELECT accounts.mcf AS password FROM accounts, domains WHERE user = '%n' AND accounts.domain = '%d' AND accounts.domain = domains.domain AND domains.host = '%{hostname}' AND accounts.service IN ('master', '%{listener}') but it doesn't expand, giving: Failed to expand password_query=SELECT accounts.mcf AS password FROM accounts, domains WHERE user = '%n' AND accounts.domain = '%d' AND accounts.domain = domains.domain AND domains.host = '%{hostname}' AND accounts.service IN ('master', '%{listener} ') : Unknown variable '%listener' %{hostname} works correctly, as well as other variables I tried. many thanks Franz ------------------- below my doveconf -n: # 2.3.19 (b3ad6004dc): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.19 (4eae2f79) # OS: Linux 5.10.0-13-amd64 x86_64 Debian 11.3 # Hostname: example.org auth_debug = yes auth_mechanisms = plain login auth_verbose = yes disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 12345 hostname = example.org login_log_format_elements = method=%m lip=%l %c %k mail_fsync = never mail_location = maildir:/var/vmail/%d/%n@%d/:INDEX=/var/vmail_indexes/%d/%n@%d/ mail_max_userip_connections = 200 mail_plugins = " notify replication fts fts_lucene" maildir_stat_dirs = yes maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/sql.conf.ext driver = sql } plugin { fts = lucene fts_autoindex = yes fts_autoindex_max_recent_msgs = 80 fts_index_timeout = 90 fts_lucene = whitespace_chars=@. last_login_dict = proxy::lastlogin last_login_key = # hidden, use -P to show it sieve_extensions = +vnd.dovecot.filter sieve_global_extensions = +vnd.dovecot.filter sieve_plugins = sieve_extprograms } protocols = imap pop3 lmtp sieve submission service aggregator { fifo_listener replication-notify-fifo { user = dovemail } unix_listener replication-notify { user = dovemail } } service dict { unix_listener dict { group = dovemail mode = 0660 user = root } } service doveadm { inet_listener { port = 12345 } } service imap-login { service_count = 1 } service imap { executable = imap } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 5 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { service_count = 1 } service pop3 { executable = pop3 } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = dovemail mode = 0600 user = dovemail } } service submission-login { inet_listener submissions { port = 465 ssl = yes } } ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL:!RC4::!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes submission_max_mail_size = 100000000 B submission_max_recipients = 100 submission_relay_host = 127.0.0.1 submission_relay_port = 10025 userdb { args = uid=1000 gid=1000 home=/var/vmail/%d/%n@%d/ plain_pass=#hidden_use-P_to_show# user_part=%n domain_part=%d driver = static } userdb { driver = prefetch } userdb { args = /etc/dovecot/sql.conf.ext driver = sql } valid_chroot_dirs = /var/vmail:/var/vmail_indexes protocol imap { mail_plugins = " notify replication fts fts_lucene imap_zlib last_login" } protocol pop3 { mail_plugins = " notify replication fts fts_lucene last_login" } protocol lmtp { mail_fsync = optimized mail_plugins = " notify replication fts fts_lucene sieve" }

2 3

[Rocky Linux 8.6] expire plugin file disappeared
by Masakazu Hirose 19 May '22

19 May '22

Hi, Rocky Linux Dovecot no longer reads the expired plugin from the library directory. I use Rocky Linux packages. Rocky or Redhat may fail to package. "/usr/lib64/dovecot/lib20_expire_plugin.so" seems to have disappeared in the new package. This may not be relevant to the Dovecot project. # /var/log/messages May 18 08:21:22 xxxxxxx dovecot[3508407]: imap(yyyyyyy): Error: Plugin 'expire' not found from directory /usr/lib64/dovecot # cat /etc/redhat-release Rocky Linux release 8.6 (Green Obsidian) # rpm -qa | grep dovecot dovecot-2.3.16-2.el8.x86_64 dovecot-mysql-2.3.16-2.el8.x86_64 Thank you, Masakazu

2 2

Re: Duplicate messages if message is moved when using dsync
by Thom Pol 18 May '22

18 May '22

Never mind. It did not work, issue still exists. I guess my network was a little slower when I thought it worked, so the emails were moved by my local client only after the sync had already happened between the servers. So, back to the original message. Anyone any idea how this moving on local client while the remote servers are still running the dsync can work correctly without getting a duplicate? Kind regards, Thom Pol On 18/05/2022 16:24, Paul Kudla (SCOM.CA Internet Services Inc.) wrote: > Your Welcome > > > > Happy Wednesday !!! > Thanks - paul > > Paul Kudla > > > Scom.ca Internet Services<http://www.scom.ca> > 004-1009 Byron Street South > Whitby, Ontario - Canada > L1N 4S3 > > Toronto 416.642.7266 > Main?1.866.411.7266 > Fax?1.888.892.7266

2 1

FTS Solr -- no commit on message delete
by Shawn Heisey 18 May '22

18 May '22

I've been chasing down oddities in behavior related to FTS Solr for a while now. First the part that isn't a problem with dovecot: In cases where message deletes bypass the Trash folder (such as shift-delete and normal workflow with the Drafts and Sent folders), Thunderbird does not inform Dovecot about the change in a way that causes Dovecot to delete the entries from Solr. When you completely exit Thunderbird, it then sends something to dovecot that results in the messages being deleted from Solr. Today I discovered a related problem that I think IS in Dovecot. I did a shift-delete on some messages, and then exited Thunderbird. Dovecot did delete the messages from Solr, but it did not send a commit to make those changes affect subsequent searches. Is this something worth opening a bug? Where would I do that? Thanks, Shawn

1 1

Upgrading from 2.3.18 --> 2.3.19
by Paul Kudla (SCOM.CA Internet Services Inc.) 18 May '22

18 May '22

Ok currently running 2.3.18 Would like to upgrade to 2.3.19 (to keep current) Since these are production servers can i down grade back to 2.3.18 if nessesary? Issues at hand is there seems to be conflicting messages about the replication being fixed or not ? so would like to basically try 2.3.19 to see if it fixes some of my issues (large folder counts etc failing to replicate properly) If it does or stays the same then i am ok with that for now. if 2.3.19 causes issues can i go back to 2.3.18 as i already have scripts etc sorting out the sync issues? pls advise -- Happy Wednesday !!! Thanks - paul Paul Kudla Scom.ca Internet Services <http://www.scom.ca> 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3 Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266

1 0

Re: Can sync/migrate all mail from remote imap account except the main "INBOX"
by Darren Mobley 18 May '22

18 May '22

Your example pointed me to a solution, excellent ! In my case I only needed one pass, I had to supply both imapc_list_prefix AND imapc_prefix AND the "namespace/inbox/prefix=" options. In case anyone runs into this in the future, here's a decent place to start for debugging: /usr/bin/doveadm -Dv -o imapc_list_prefix=INBOX -o imapc_prefix=INBOX. -o "namespace/inbox/prefix=" -o mail_fsync=never -o dsync_commit_msgs_interval=100 -o imapc_features=rfc822.size,fetch-headers -o dsync_features=empty-header-workaround -o imapc_port=143 -o imapc_host=127.0.0.1 -o imapc_password=yayforpasswords -o imapc_user=mailto:wah@hellodemo.ppl -o imapc_rawlog_dir=/tmp/raww -o mail_prefetch_count=20 sync -1Ru mailto:saywhatnow@letest.tld imapc: Thanks again for digging through the old emails where the magic incantations were, I really appreciate it! Darren ---- On Wed, 18 May 2022 05:19:53 -0500 Sami Ketola <sami(a)ketola.io> wrote ---- So adding "-o imapc_list_prefix=INBOX" to the doveadm command line should fix that, I would think ? So is the remote INBOX really 'INBOX' or 'INBOX.INBOX'? Command line arguments depend on which one it is. I managed to find some old archived emails with some tips on how it was solved back then. It was done with something like this: normal doveadm command: 'LOG_STDERR_TIMESTAMP="%b %d %H:%M:%S.%{usecs} " ' + '/usr/bin/doveadm' + dsync_debug + \ " -o mail_prefetch_count=10 " + \ " -o imapc_features=fetch-headers " + \ " -o imapc_host=" + legacy_host + \ " -o imapc_port=" + legacy_port + \ " -o imapc_list_prefix=" + imapc_prefix + \ " -o imapc_prefix=" + imapc_prefix + \ " -o imapc_user=" + self.legacy_user + \ " -o imapc_master_user=" + self.legacy_user + \ " -o imapc_password=" + imapc_master_password + \ " -o mail_fsync=never " + \ " backup -u " + self.uid + \ " -R imapc: >> " + dsync_err_log_location + self.uid + ".dsync_err.log 2>&1" If I remember correctly imapc_prefix was 'INBOX/' since this customer had INBOX/INBOX setup with / as separator. and if user had INBOX/INBOX detected this command was run on top of that to migrate INBOX: 'LOG_STDERR_TIMESTAMP="%b %d %H:%M:%S.%{usecs} " ' + '/usr/bin/doveadm' + dsync_debug + \ " -o namespace/inbox/prefix= " + \ " -o mail_prefetch_count=10 " + \ " -o imapc_features=fetch-headers " + \ " -o imapc_host=" + legacy_host + \ " -o imapc_port=" + legacy_port + \ " -o imapc_user=" + self.legacy_user + \ " -o imapc_master_user=" + self.legacy_user + \ " -o imapc_password=" + imapc_master_password + \ " -o mail_fsync=never " + \ " backup -u " + self.uid + \ " -m INBOX/INBOX " + \ " -R imapc: >> " + dsync_err_log_location + self.uid + ".dsync_err.log 2>&1" HTH, Sami

1 0

Re: dovecot Digest, Vol 229, Issue 23
by Thom Pol 18 May '22

18 May '22

I must say I was a bit sceptical if this sieve script would resolve it, but it seems like it totally did! Thanks a lot for your reply, hope this helps someone else in the future as well. Kind regards, Thom Pol On 16/05/2022 08:12, Paul Kudla (SCOM.CA Internet Services Inc.) wrote: > ok duplicsate emails (even across dsync, replication etc) is typically > handled via a global sieve script > > > I use : > > # cat duplicates.sieve > require "duplicate"; # for dovecot >= 2.2.18 > > if duplicate { > discard; > stop; > } > > for the scripts > > and setup sieve to work via my dovecot.conf file > > relative parts below : > > -------------------------------------------------------------------- > > protocols = imap pop3 lmtp sieve > > protocol lmtp { > mail_plugins = $mail_plugins sieve > postmaster_address =monitor(a)scom.ca > } > > > protocol lda { > mail_plugins = $mail_plugins sieve > } > > > plugin { > ..... > > sieve =file:~/sieve;active=~/sieve/.dovecot.sieve > > sieve_duplicate_default_period = 1h > sieve_duplicate_max_period = 1d > sieve_extensions = +duplicate +notify +imapflags +vacation-seconds > sieve_global_dir = /usr/local/etc/dovecot/sieve > sieve_before = /usr/local/etc/dovecot/sieve/duplicates.sieve > > ..... > } > > service managesieve-login { > process_limit = 1000 > vsz_limit = 1g > inet_listener sieve { > port = 4190 > } > } > > protocol sieve { > managesieve_implementation_string = Dovecot Pigeonhole > managesieve_max_line_length = 65536 > } > > > > ------------------------------------------------------------------ > > > note the sieve_before which handles duplictes during delivery etc. > > > Happy Monday !!! > Thanks - paul > > Paul Kudla > > > Scom.ca Internet Services<http://www.scom.ca> > 004-1009 Byron Street South > Whitby, Ontario - Canada > L1N 4S3 > > Toronto 416.642.7266 > Main?1.866.411.7266 > Fax?1.888.892.7266

2 1

dovecot virtual users with sendmail, my solution
by Christopher McCrory 18 May '22

18 May '22

Hello... I wanted to use dovecot virtual users with sendmail. As you probably know, this doesn't work by default. The more scalable solution is probably to use LDAP, but I really don't like it. This is how I did it https://github.com/chrismcc-gmail/dovecot-virtual-sendmail Basically I abused glibc's db feature in /etc/nsswitch in /var/db/Makefile something like this: $(VAR_DB)/passwd.db: /etc/dovecot/usersfile @printf %s "$(patsubst %.db,%,$(@F))... " @$(AWK) 'BEGIN { FS=":"; OFS=":" } \ /^[ \t]*$$/ { next } \ /^[ \t]*#/ { next } \ /^root/ { next } \ /^[^#]/ { printf ".%s ", $$1; \ printf "%s:x:65534:65534:%s:/home/vmail/%s:/sbin/nologin\n", $$1, $$1, $$1; \ }' $^ | \ $(MAKEDB) -o $@ - @echo "done." make -C /var/db getent passwd ; shows ... virtualuser:x:65534:65534:virtualuser:/home/vmail/virtualuser:/sbin/nologin othervuser:x:65534:65534:othervuser:/home/vmail/othervuser:/sbin/nologin A longer explanation and more info is in the github repository It takes about 10 minutes to go from new clean server to working email server with this I've been running this in production for about a week now without any problems. Question: Has anyone else done this? I don't see any downsides, but I might be missing something. -- Christopher McCrory To the optimist, the glass is half full. To the pessimist, the glass is half empty. To the engineer, the glass is twice as big as it needs to be.

2 1

Preventing deletion of mails in some directories?
by Philip Rhoades 18 May '22

18 May '22

People, I can't get the system "chattr" to do what I want so I am wondering if there is some way to prevent deletion of mails in the INBOX and the Sent folders? Renaming and moving files needs to be possible for Maildir functions and also for me - so I only need deletion prevention . . Is this possible from Dovecot? Thanks, Phil. -- Philip Rhoades PO Box 896 Cowra NSW 2794 Australia E-mail: phil(a)pricom.com.au

3 2