May 2020 - dovecot - dovecot.org

FTS-lucene errors : language not available for stemming
by David Gessel 21 May '20

21 May '20

I'm getting some log errors with clucene that I am having no luck tracking down on the interwebs. Errors: May 19 05:05:16 indexer-worker(gessel(a)blackrosetech.com)<62971><aPAEI3zLw17A/QAA0J78UA:EF25M3zLw1779QAA0J78UA>: Error: lucene index /mail/blackrosetech.com/gessel//lucene-indexes: IndexWriter::addDocument() failed (#4): language not available for stemming May 19 05:05:16 indexer-worker: Error: May 19 05:05:16 indexer-worker(gessel(a)blackrosetech.com)<62971><aPAEI3zLw17A/QAA0J78UA:EF25M3zLw1779QAA0J78UA>: Error: Mailbox Security: Mail search failed: Internal error occurred. Refer to server log for more information. [2020-05-19 05:05:16] May 19 05:05:16 indexer-worker(gessel(a)blackrosetech.com)<62971><aPAEI3zLw17A/QAA0J78UA:EF25M3zLw1779QAA0J78UA>: Error: Mailbox Security: Transaction commit failed: FTS transaction commit failed: transaction context (attempted to index 1 messages (UIDs 152736..152736)) Config: FreeBSD 11.3-RELEASE-p8 #0 r360490 dovecot-2.3.10_3 clucene-2.3.3.4_19 py37-pystemmer-2.0.0.1 py37-snowballstemmer-1.2.1 icu-67.1,1 plugin { #setting_name = value expire = Trash mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size fts_autoindex=yes #zlib_save_level = 6 # 1..9 #zlib_save = gz # or bz2 } plugin { fts = lucene # Lucene-specific settings, good ones are: fts_lucene = whitespace_chars=@. mime_parts } I am considering switch to xapian (solr and java... pls noe) as the port is quite tempting from an ease of integration perspective, but the easiest solution would be to resolve these odd indexing errors. Anyone have a clue? -David

6 11

dsync multiple mailboxes per connection?
by Felipe Gasper 21 May '20

21 May '20

Hello, Is it possible to batch syncs of multiple mailboxes in a single dsync session? We’re transferring users between servers and want to use dsync to transfer the mail; ideally we’d like to avoid creating a separate TCP connection per user. Thank you! cheers, -Felipe Gasper

3 2

how to automatically delete folders from Trash after x days
by chayes＠afo.net 20 May '20

20 May '20

This works great for keeping the Trash from overflowing ... doveadm expunge -A mailbox Trash before 30d ... but it doesn't delete folders that the user put in the trash. What is the recommended method for erasing folders from the trash?

2 1

question about director_mail_servers setting
by Joe Wong 20 May '20

20 May '20

Hello, I have a question about director_mail_servers setting for director, I am aware that I can define a list of dovecot backend hostname there but I am not sure if the hashing apply to the hostname or underlying IP address of each hostname given? What if the IP of a specify hostname is changed? Will director pickup the new IP automatically? Will the same user hashed into the original IP still be mapped back to the new IP in this case? - Joe

1 0

Problem with tika
by Arjen Heidinga 19 May '20

19 May '20

Hello all, For some time now we've bin using solr as a search engine (working great). I have added tika for searching inside documents, however it keeps crashing when indexing. When just indexing mails that arrive works good, but a reindex on all mail constantly crashes with below stacktrace. When I observe the packets with wireshark I see a HTTP-flow going to tika and suddenly, midway a document dovecot (or the server) sends a RST. Sometimes after thousands of documents, sometimes after a few. Usually after a few hundred. It appears there are less errors using http than https. Relevant config: OS: CentOS6, fully updated plugin { fts = solr batch_size = 1 fts_solr = url=https://username:password@solr-01.vevida.net:443/solr/dovecot/ #fts_tika = https://solr-01.vevida.net:443/tika/ batch_size = 1000 fts_autoindex=yes soft_commit=no } # dovecot --version 2.3.9.3 (9f41b88fa) # Configure options: --docdir=%{_docdir}/dovecot \ --disable-static \ --with-nss \ --with-shadow \ --with-pam \ --with-gssapi=plugin \ --with-ldap=plugin \ --with-sql=plugin \ --with-pgsql \ --with-sqlite \ --with-zlib \ --with-bzlib \ --with-lzma \ --with-libcap \ --with-ssl=openssl \ --with-ssldir=%{ssldir} \ --with-solr \ --with-docs # It is compiled agains the latest openssl # Tika and Solr: Both latest versions. #Stacktrace: doveadm(info(a)samenmetrenske.nl): Info: Sent: Caching mails seq=1..161 doveadm(info(a)samenmetrenske.nl): Panic: file http-client-request.c: line 1173 (http_client_request_send_more): assertion failed: (req->payload_input != NULL) doveadm(info(a)xxxxxxxxxxxxxxxxxxx.x): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x2f) [0x7f95d805acbf] -> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x26) [0x7f95d805add6] -> /usr/lib64/dovecot/libdovecot.so.0(+0xe90ba) [0x7f95d80660ba] -> /usr/lib64/dovecot/libdovecot.so.0(+0xe9161) [0x7f95d8066161] -> /usr/lib64/dovecot/libdovecot.so.0(+0x41158) [0x7f95d7fbe158] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_request_send_more+0x424) [0x7f95d8005094] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_connection_output+0x11a) [0x7f95d800a24a] -> /usr/lib64/dovecot/libssl_iostream_openssl.so(+0x8f6a) [0x7f95d57a2f6a] -> /usr/lib64/dovecot/libdovecot.so.0(+0x114483) [0x7f95d8091483] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x61) [0x7f95d807e581] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xdc) [0x7f95d808076c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x5c) [0x7f95d807e67c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f95d807e8c8] -> /usr/lib64/dovecot/libdovecot.so.0(+0x89105) [0x7f95d8006105] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_request_send_payload+0x1f) [0x7f95d80063cf] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xd31d) [0x7f95d6ad931d] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_parser_more+0x1a) [0x7f95d6ad83ca] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_build_mail+0x761) [0x7f95d6ad6401] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x114ca) [0x7f95d6add4ca] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_precache+0x2a) [0x7f95d835ab4a] -> doveadm(+0x31e75) [0x55e7c1052e75] -> doveadm(+0x321fb) [0x55e7c10531fb] -> doveadm(+0x2c321) [0x55e7c104d321] -> doveadm(+0x2c577) [0x55e7c104d577] -> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x1e8) [0x55e7c104ec38] -> doveadm(doveadm_cmd_run_ver2+0x52e) [0x55e7c105fafe] -> doveadm(doveadm_cmd_try_run_ver2+0x37) [0x55e7c105fb97] -> doveadm(main+0x21a) [0x55e7c1062aca] -> /lib64/libc.so.6(__libc_start_main+0x100) [0x7f95d7983d20] -> doveadm(+0x1c479) [0x55e7c103d479]

1 1

doveadm backup, question
by dovecot-user 19 May '20

19 May '20

Hello everyone. I am an inexperienced user of dovecot and I faced probably a simple problem, but I still do not understand how to solve it. The issue is related to using doveadm backup - I want to backup mailboxes; make this copy on the same machine (hereinafter, files with backup copies will be taken from this machine into the storage by a separate mechanism, this is not related to the issue). I am using dovecot 2.2.27 (c0f36b0) on Debian GNU/Linux 9 (stretch). The options related to storing mail and attachments are as follows: mail_location = mdbox:/var/mail/%d/%n mail_attachment_dir = /var/mail/%d/%n/attach Thus, the mail of user(a)domain.tld is stored in /var/mail/domain.tld/user/, and mail attachments inside this directory, for example: /var/mail/domain.tld/user/attach. When I start copying (as root while in test mode): #doveadm backup -u "user(a)domain.tld" sdbox:/backup/domain.tld/user, copying passes, but, as I understand it, the attachments are not copied (I am judging so far by the volume of the initial storage and the received one, it should not differ VERY much, but it differs by 10 times, it seems that it’s just the total amount of attachments). Tell me, plz, how to run doveadm backup correctly so that everything is copied so that you can subsequently deploy it from the backup? Thanks.

1 0

Multiple vulnerabilities in Dovecot
by Aki Tuomi 18 May '20

18 May '20

Dear subscribers, we are sending notifications for three vulnerabilities, - CVE-2020-10957 - CVE-2020-10958 - CVE-2020-10967 Please find them below --- Aki Tuomi Open-Xchange Oy ------------------ Open-Xchange Security Advisory 2020-05-18 Product: Dovecot Vendor: OX Software GmbH Internal reference: DOV-3784 Vulnerability type: NULL pointer dereference (CWE-476) Vulnerable version: 2.3.0 - 2.3.10 Vulnerable component: submission, lmtp Report confidence: Confirmed Solution status: Fixed by Vendor Fixed version: 2.3.10.1 Researcher credits: Philippe Antoine (Catena Cyber) Vendor notification: 2020-03-24 Solution date: 2020-04-02 Public disclosure: 2020-05-18 CVE reference: CVE-2020-10957 CVSS: 7.5 (CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H) Vulnerability Details: Sending malformed NOOP command causes crash in submission, submission-login or lmtp service. Risk: Remote attacker can keep submission-login service down, causing denial of service attack. For lmtp the risk is neglible, as lmtp is usually behind a trusted MTA. Steps to reproduce: Send ``NOOP EE"FY`` to submission port, or similarly malformed command. Solution: Upgrade to fixed version. ------------------ Open-Xchange Security Advisory 2020-05-18 Product: Dovecot IMAP server Vendor: OX Software GmbH Internal reference: DOV-3875 Vulnerability type: Improper handling of input data (CWE-20) Vulnerable version: 2.3.0 - 2.3.10 Vulnerable component: submission, lmtp Report confidence: Confirmed Solution status: Fixed by Vendor Fixed version: 2.3.10.1 Researcher credits: Philippe Antoine (Catena Cyber) Vendor notification: 2020-03-23 Solution date: 2020-04-02 Public disclosure: 2020-05-18 CVE reference: CVE-2020-10958 CVSS: 5.3 (CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:L) Vulnerability Details: Sending command followed by sufficient number of newlines triggers a use-after-free bug that might crash submission-login, submission or lmtp service. Risk: Remote attacker can keep submission-login service down, causing denial of service attack. For lmtp the risk is neglible, as lmtp is usually behind a trusted MTA. Steps to reproduce: This can be currently reproduced with ASAN or Valgrind. Reliable way to crash has not yet been discovered. Solution: Upgrade to fixed version. ------------------ Open-Xchange Security Advisory 2020-05-18 Product: Dovecot Vendor: OX Software GmbH Internal reference: DOV-1745 Vulnerability type: Improper input validation (CWE-20) Vulnerable version: 2.3.0 - 2.3.10 Vulnerable component: submission, lmtp Report confidence: Confirmed Solution status: Fixed by Vendor Fixed version: 2.3.10.1 Researcher credits: mailbox.org Vendor notification: 2020-03-20 Solution date: 2020-04-02 Public disclosure: 2020-05-18 CVE reference: CVE-2020-10967 CVSS: 5.3 (CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:L) Vulnerability Details: Sending mail with empty quoted localpart causes submission or lmtp component to crash. Risk: Malicious actor can cause denial of service to mail delivery by repeatedly sending mails with bad sender or recipient address. Steps to reproduce: Send mail with envelope sender or recipient as ``<""@example.org>``. Workaround: For submission there is no workaround, but triggering the bug requires valid credentials. For lmtp, one can implement sufficient filtering on MTA level to prevent mails with such addresses from ending up in LMTP delivery. Solution: Upgrade to fixed version. ------------------

1 0

dovecot sync not pushing automatically ?
by Laura Smith 17 May '20

17 May '20

Hi, I'm aware its an async process, but despite sending test messages and then waiting a few minutes, the stats are still unchanged : $ sudo doveadm replicator status Queued 'sync' requests 0 Queued 'high' requests 0 Queued 'low' requests 0 Queued 'failed' requests 0 Queued 'full resync' requests 0 Waiting 'failed' requests 0 Total number of known users 18 However if I manually run it using : sudo doveadm -D sync -u test(a)example.com -1 -d -N -l 30 -U It syncs fine and the mails are visible on the standby server. The master server is setup as per the docs : replication_dsync_parameters = -1 -d -N -l 30 -U plugin { mail_replica = tcps:foobar.example.com:11867 } service replicator { process_min_avail = 1 } service aggregator { fifo_listener replication-notify-fifo { user = foobar } unix_listener replication-notify { user = foobar } } service replicator { unix_listener replicator-doveadm { mode = 0600 user = foobar } } replication_max_conns = 10

1 0

iterate_query with static userdb ?
by Laura Smith 17 May '20

17 May '20

Hi, Going by the "static userdb" example on this page (https://wiki.dovecot.org/VirtualUsers#homedirs), tried to achieve a similar setup in conjunction with pgsql for passdb. However I get an error "auth: Warning: sql: Ignoring changed iterate_query in /etc/dovecot/local_sql_users.conf, because userdb sql not used. (If this is intentional, set userdb_warning_disable=yes)" Does this mean the following is not a valid config ? Or at least I will not be able to achieve iteration ? mail_location = maildir:/foobar/mail/%d/%n/Maildir passdb { driver = sql args = /etc/dovecot/local_sql_users.conf } userdb { driver = static args = uid=foo gid=bar home=/foobar/mail/%d/%n }

3 4

"/etc/dovecot/dovecot.conf: passdb is missing driver"
by Laura Smith 17 May '20

17 May '20

Hi, I'm trying to get dovecot working with postgres, I'm on Debian 10 and have installed dovecot-pgsql from the Dovecot repo (https://repo.dovecot.org/ce-2.3-latest/debian/....). I have the following in my local.conf : passdb sql { args = /etc/dovecot/local_sql_users.conf } And the following in the referenced file: driver = pgsql connect = host=localhost dbname=foobar user=bar password=foo default_pass_scheme = ARGON2ID password_query = select dovecot_username as user,password from get_user('%u') iterate_query = select dovecot_username as user from get_users() So surely it should work ? doveconf -n just to complete the picture .... # 2.3.10 (0da0eff44): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.10 (bf8ef1c2) # OS: Linux 4.19.0-9-amd64 x86_64 Debian 10.4 xfs # Hostname: foobar.example.com auth_verbose = yes auth_verbose_passwords = sha1:7 doveadm_password = # hidden, use -P to show it mail_location = maildir:/foobar/mail/%d/%n/Maildir mail_plugins = " notify replication" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/local_sql_users.conf name = sql } plugin { mail_replica = tcps:standby.example.com:11867 sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap lmtp replication_dsync_parameters = -1 -d -N -l 30 -U service aggregator { fifo_listener replication-notify-fifo { user = foobar } unix_listener replication-notify { user = foobar } } service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } vsz_limit = 2 G } service doveadm { inet_listener { port = 11867 ssl = yes } } service imap-login { process_min_avail = 5 service_count = 1 } service lmtp { process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } user = foobar } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieves { address = port = 5190 ssl = yes } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = foobar } } ssl = required ssl_ca = </etc/foobar/ssl/example.com.chain ssl_cert = </etc/foobar/ssl/example.com.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { driver = passwd } userdb { args = uid=foobar gid=foobar home=/foobar/mail/%d/%n driver = static } protocol lmtp { mail_plugins = sieve postmaster_address = foobar(a)example.com } protocol lda { deliver_log_format = msgid=%m: %$ mail_plugins = sieve postmaster_address = postmaster quota_full_tempfail = yes rejection_reason = Your message to <%t> was automatically rejected:%n%r } protocol imap { mail_max_userip_connections = 20 }

2 1