- dovecot - dovecot.org

User found but password failure
by Mauricio Tavares 11 Nov '19

11 Nov '19

Trying to figure out which step is causing me not to be able to login. I am using a password file, passdb { driver = passwd-file args = scheme=SHA512-CRYPT username_format=%u /etc/dovecot/passwd } We will assume that the pw I created using 'doveadm pw -s SHA512-CRYPT' matches the password I will be using to login below. I crank the debugging mode, auth_debug_passwords = yes auth_debug = yes And then try to login [root@mail ~]# nc -t localhost 143 * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN AUTH=GSSAPI] Howdy a login raub(a)example.com password [blank] which from what I gathered from /var/log/dovecot means it found matching username but did not match password: secured session=lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB lip=::1 rip=::1 lport=143 rport=36340 resp=base64-reply (previous base64 data may contain sensitive data) Aug 26 03:25:52 auth: Debug: passwd-file(raub@example.com,::1,<lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB>): lookup: user=raub(a)example.com file=/etc/dovecot/passwd Aug 26 03:25:52 auth: Debug: client passdb out: OK 1 user=raub(a)example.com Aug 26 03:25:52 auth: Debug: master in: REQUEST 1520959489 26560 1 63ee7f45236f85fd39573a5c8a2eb46a session_pid=26563 request_auth_token Aug 26 03:25:52 auth-worker(26562): Debug: passwd(raub@example.com,::1,<lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB>): lookup Aug 26 03:25:52 auth-worker(26562): Info: passwd(raub@example.com,::1,<lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB>): unknown user Aug 26 03:25:52 auth: Debug: passwd-file(raub@example.com,::1,<lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB>): lookup: user=raub(a)example.com file=/etc/dovecot/passwd Aug 26 03:25:52 auth: Error: plain(raub@example.com,::1,<lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB>): user not found from any userdbs Aug 26 03:25:52 auth: Debug: master userdb out: NOTFOUND 1520959489 Aug 26 03:25:52 imap: Error: Authenticated user not found from userdb, auth lookup id=1520959489 (auth connected 1 msecs ago, handshake 0 msecs ago, request took 1 msecs, client-pid=26560 client-id=1) Aug 26 03:25:52 imap-login: Info: Internal login failure (pid=26560 id=1) (internal failure, 1 successful auths): user=<raub(a)example.com>, method=PLAIN, rip=::1, lip=::1, mpid=26563, secured, session=<lyJttvyQ9I0AAAAAAAAAAAAAAAAAAAAB> Per [1] I decided to see what the response (base64-reply) I am submitting to dovecot looks like: echo 'base64-reply' | base64 -d raub(a)example.compassword Which has the right username and password but follows the userpassword format, not useruserpassword as I was led to believe by [1]. Is that to be expected? If it is then the issue is further down the line. [1] https://wiki.dovecot.org/Debugging/Authentication

3 5

Dovecot on CentOS8 / RHEL8
by Aki Tuomi 11 Nov '19

11 Nov '19

If you are planning to run dovecot on CentOS8 or RedHat 8 using native packages, please be aware that pigeonhole is not currently available (but they are working on it), see https://bugzilla.redhat.com/show_bug.cgi?id=1766985 . We are working on getting our CentOS8 packages available on repo.dovecot.org. Aki

1 0

MariaDB database for users and passwords?
by Ken Wright 10 Nov '19

10 Nov '19

Greetings, All! I've got an email server set up with Postfix and Dovecot, but Dovecot can't find any users. I'd ilke to use the database I set up in MariaDB for users (referenced by Postfix), but I'm just not sure how to do it. There's a line in dovecot.conf specifying /etc/dovecot/passwd.db, but obviously that's not the one I set up in MariaDB. Is what I want even possible? Ken -- Registered Linux user #483005 If you ever think international relations make sense, remember this: because a Serb shot an Austrian in Bosnia, Germany invaded Belgium.

8 14

G Suite migration using dsync
by Paul Pace 09 Nov '19

09 Nov '19

Hello, I am trying to migrate a domain from G Suite using dsync and am not clear on how to configure so I am getting failures. This is the configuration in dovecot.conf: imapc_host = imap.gmail.com imapc_port = 993 imapc_password = 1234 imapc_ssl = imaps imapc_ssl_verify = yes imapc_features = gmail-migration I also tried with imapc_ssl_verify commented. For simplicity, both the username and password are the same for G Suite and Dovecot and are verified to access the G Suite account with other tools. Here is the command (from the wiki) and output: # doveadm backup -a 'virtual/All' -O '-$GmailHaveLabels' -R -u user(a)example.com imapc: dsync(user(a)example.com): Info: imapc(imap.gmail.com:993): Connected to 2607:f8b0:4003:c0f::6c:993 (local IPv6:port) dsync(user(a)example.com): Warning: imapc(imap.gmail.com:993): Server disconnected unexpectedly: SSL_connect() failed: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure: SSL alert number 40 - reconnecting (delay 0 ms) dsync(user(a)example.com): Info: imapc(imap.gmail.com:993): Connected to 74.125.30.109:993 (local IPv4:port) dsync(user(a)example.com): Error: imapc(imap.gmail.com:993): Server disconnected unexpectedly: SSL_connect() failed: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure: SSL alert number 40 - disconnecting dsync(user(a)example.com): Error: User initialization failed: imapc: Login to imap.gmail.com failed: Disconnected from server In reading through the wiki, it seems I am missing something, but I can't figure out what is missing and internet searches don't provide further useful information on the dovecot tools. Thank you, Paul

1 0

Reading config-value from file does not work.
by Jakobus Schürz 08 Nov '19

08 Nov '19

Hi guys! In https://doc.dovecot.org/configuration_manual/config_file/config_file_syntax/ i ca see this: Reading value from file It’s possible to read the value for a setting from a file: key = </path/to/file The value is read exactly as the entire contents of the file. This includes all the whitespace and newlines. The paths are relative to the currently parsed config file’s directory, similar to how !include works. The file is read immediately whenever parsing the configuration file, so if it changes afterwards it requires a configuration reload to see the changes. This functionality is especially useful for reading SSL certificates and keys. So i thought it's a good idea, to write the password for dnpass for my ldap in this notation in my config dnpass = <private/dnpass /etc/dovecot/private/dnpass contains only this myVerYSecretLdApPAssworD no space, no newline. But dovecot cannot authenticate on the ldap-host. I also tried dnpass = </private/dnpass dnpass = </etc/dovecot/private/dnpass Then i tried a file /etc/dovecot/conf.d/00-dnpass.conf with dnpass = myVerYSecretLdApPAssworD It should be included with dovecot.conf !include conf.d/*.conf The only notation that works is the line dnpass = myVerYSecretLdApPAssworD in /etc/dovecot/dovecot-ldap-userdb.conf.ext I thought it's a good idea to write it in an extra file, so i have to manipulate only one small file with ansible, when i renew the password. Is it a bug or does it work as designed, and the doc is wrong? thank you Jakob

1 1

dovecot ignores users sieve-script
by Jakobus Schürz 08 Nov '19

08 Nov '19

Hi there! I recognized that dovecot ignores users sieve-script. The global and the post-sieve-scripts are working, but users sievescript is ignored. You can see the sieve trace-files last lines 65: finishing match with result: not matched 65: jump if result is true 65: not jumping 65: jumping to line 67 67: jumping to line 77 ## Finished executing script 'global' ## Started executing script 'junk' 2: header test 2: starting `:contains' match with `i;ascii-casemap' comparator: 2: extracting `X-Spam' headers from message 2: finishing match with result: not matched 2: jump if result is false 2: jumping to line 6 ## Finished executing script 'junk' Between global and junk-Script should sieve run users sieve script. Before the Upgrade to 2.3.8 sieve was working correct. Now ALL messages go to INBOX, no one is filtered anymore. compiling the sievescript with sudo -u vmail sievec sievefilter.sieve gives no errormessage... My users sievefilter is: /var/mail/jakobus.schuerz/sieve/sievefilter.sieve This directory contains: lrwxrwxrwx 1 vmail vmail 17 Nov 7 12:44 jakobus.schuerz.sieve -> sievefilter.sieve -rw------- 1 vmail vmail 36834 Nov 7 13:25 jakobus.schuerz.svbin -rw------- 1 vmail vmail 45588 Nov 7 11:43 sievefilter.sieve The dovecot conf dump: # 2.3.8 (9df20d2db): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.8 (b7b03ba2) # OS: Linux 4.19.0-6-amd64 x86_64 Debian 10.1 # Hostname: mail.mydomain.example auth_mechanisms = plain login cram-md5 digest-md5 auth_socket_path = /var/run/dovecot/auth-userdb disable_plaintext_auth = no first_valid_uid = 1000 imap_capability = +XDOVECOT +SPECIAL-USE imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags delay-newmail lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = *, ::1 login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c tls=%k login_trusted_networks = 127.0.0.1/8 192.168.0.0/24 192.168.1.0/24 172.17.0.0/24 172.18.0.0/24 mail_gid = vmail mail_home = /var/mail/%u mail_location = maildir:~/Maildir:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/Maildir/%u:CONTROL=/var/lib/dovecot/db/control/Maildir/%u mail_plugins = zlib quota acl listescape mail_log notify virtual fts fts_lucene mail_privileged_group = vmail mail_server_admin = mailto:user@mydomain.example mail_shared_explicit_inbox = yes mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace Geteilt { hidden = no inbox = no list = children location = maildir:/var/mail/%%u/Maildir:LAYOUT=fs:INDEXPVT=/var/lib/dovecot/db/indexes/shared/%u/%%u prefix = Geteilt/%%n/ separator = / subscriptions = no type = shared } namespace History { hidden = no inbox = no list = children location = maildir:/var/mail/archiv/%u/:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/archiv/%u:CONTROL=/var/lib/dovecot/db/control/archiv/%u mailbox Archiv { auto = subscribe special_use = \Archive } prefix = Archiv/ separator = / subscriptions = no type = private } namespace Synoptic { hidden = no ignore_on_failure = no list = children location = virtual:/var/mail/virtual/synoptic:INDEX=/var/lib/dovecot/db/indexes/virtual/synoptic/%u mailbox AKTUELL { auto = subscribe special_use = \All } mailbox Markierte { auto = subscribe special_use = \Flagged } mailbox Neue { auto = subscribe } mailbox Rechnungen { auto = subscribe } mailbox WICHTIG { auto = subscribe } prefix = Synoptic/ separator = / subscriptions = no } namespace Tulln { list = children location = maildir:/var/mail/public/tulln:SUBSCRIPTIONS=subscriptions/%u:LAYOUT=fs:INDEXPVT=/var/lib/dovecot/db/indexes/public/tulln/%u mailbox Familie { auto = subscribe } mailbox Versicherungen { auto = subscribe } mailbox familyextracard { auto = subscribe } prefix = Tulln/ separator = / subscriptions = no type = public } namespace inbox { hidden = no inbox = yes location = mailbox "Deleted Messages" { auto = no autoexpunge = 30 days special_use = \Trash } mailbox Drafts { auto = no special_use = \Drafts } mailbox Entwürfe { auto = no special_use = \Drafts } mailbox "Gelöschte Elemente" { auto = no autoexpunge = 30 days special_use = \Trash } mailbox "Gelöschte Objekte" { auto = no autoexpunge = 30 days special_use = \Trash } mailbox Gesendet { auto = no special_use = \Sent } mailbox "Gesendete Elemente" { auto = no special_use = \Sent } mailbox "Gesendete Objekte" { auto = no special_use = \Sent } mailbox Junk { auto = subscribe autoexpunge = 30 days special_use = \Junk } mailbox Mistkübel { auto = no autoexpunge = 30 days special_use = \Trash } mailbox Papierkorb { auto = no autoexpunge = 30 days special_use = \Trash } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no autoexpunge = 30 days special_use = \Junk } mailbox Synoptic/AKTUELL { auto = no comment = All my messages special_use = \All } mailbox Trash { auto = no autoexpunge = 30 days special_use = \Trash } prefix = INBOX/ separator = / subscriptions = no type = private } namespace subscriptions { hidden = yes list = no location = prefix = subscriptions = yes } passdb { args = /etc/dovecot/dovecot-ldap-passdb-user.conf.ext driver = ldap name = user } passdb { args = /etc/dovecot/dovecot-ldap-passdb-admin.conf.ext driver = ldap name = admin } passdb { args = /etc/dovecot/dovecot-ldap-passdb-others.conf.ext driver = ldap name = others } plugin { acl = vfile:/etc/dovecot/dovecot-acl:cache_secs=300 acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db fts = lucene fts_autoindex = yes fts_autoindex_max_recent_msgs = 80 fts_index_timeout = 90 fts_lucene = whitespace_chars=@. normalize no_snowball imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = INBOX/Junk imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = INBOX/Junk imapsieve_mailbox2_name = * mail_home = /var/mail/%u mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid subject from flags setting_name = sieve, managedsieve sieve = file:/var/mail/%u/sieve/;active=/var/mail/%u/sieve/%u.sieve sieve_after = /etc/dovecot/sieve/after.d/ sieve_before = /etc/dovecot/sieve/global.sieve sieve_global_dir = /etc/dovecot/sieve/ sieve_global_extensions = +vnd.dovecot.pipe sieve_pipe_bin_dir = /etc/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_trace_dir = /var/log/dovecot/sieve/trace sieve_trace_level = matching sieve_user_log = /var/mail/%u/sieve/sieve.log } postmaster_address = postmaster@localhost protocols = " imap lmtp lmtp sieve pop3 sieve" service anvil { client_limit = 199999 unix_listener anvil-auth-penalty { mode = 0700 } } service auth { unix_listener auth-client { group = vmail mode = 0666 user = Debian-exim } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 1 service_count = 1 } service imap-postlogin { executable = script-login -d rawlog /etc/dovecot/acl_groups.py unix_listener imap-postlogin { group = vmail user = vmail } } service imap { executable = imap imap-postlogin process_limit = 1024 vsz_limit = 400 M } service lmtp { user = Debian-exim vsz_limit = 400 M } service stats { unix_listener stats-writer { group = vmail mode = 0600 user = vmail } } ssl = required ssl_cert = </etc/letsencrypt/live/imap.mydomain.example/fullchain.pem ssl_cipher_list = ALL:!ADH:!LOW:!SSLv2:!SSLv3:!EXP:!aNULL:+HIGH:+MEDIUM ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap-userdb.conf.ext default_fields = home=/var/mail/%u uid=vmail gid=vmail driver = ldap } verbose_proctitle = yes protocol lmtp { auth_username_format = %Ln mail_plugins = zlib quota acl listescape mail_log notify virtual fts fts_lucene quota sieve acl } protocol lda { info_log_path = log_path = mail_plugins = zlib quota acl listescape mail_log notify virtual fts fts_lucene quota sieve acl } protocol imap { mail_max_userip_connections = 60 mail_plugins = zlib quota acl listescape mail_log notify virtual fts fts_lucene imap_quota imap_acl imap_sieve rawlog_dir = %h/rawlog } Thanks for your help

2 2

[2.3.8] possible replication issue
by KOCIK Fabien (Acoss) 08 Nov '19

08 Nov '19

Hi, I'm dealing for few days with replication issues between two dovecot containerized instances in Kubernetes environment. After some search, I found this thread that reports exactly the same symptoms I encountered : https://dovecot.org/pipermail/dovecot/2019-October/117353.html I can confirm that downgrade of dovecot to 2.3.7.2 fixes the issue. Testing usage of -T parameter with a very low value (10s) on 2.3.8 version is reducing perceptible impact of the issue (mails are replicated within the following five minutes). I'm not expert in dovecot architecture but I tried to put an eye in the code and I'm wondering if this commit is not creating a potential phantom stalled I/O timeout ? https://github.com/dovecot/core/commit/ec817bb2185bb21b34ba6bdd83b32af16dd0… In fact, I was unable to see when the counter incremented by io_set_pending is decremented ? Hoping that this information can help... Regards. Fabien. +--------+ For technical details : Container images are based on Arch Linux distro image. Here is the configuration of Dovecot server : dovecot -n # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # OS: Linux 4.19.66-coreos x86_64 ext4 # Hostname: mailstore-0.mailstore.piafe.svc.cluster.local disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 12345 login_greeting = PIAFE Mail Server Ready mail_location = mdbox:/var/mail/%Ld/%Ln mail_plugins = " notify replication quota" namespace inbox { hidden = no inbox = yes list = yes location = mailbox Brouillons { auto = subscribe special_use = \Drafts } mailbox Corbeille { auto = subscribe special_use = \Trash } mailbox "Courrier indésirable" { auto = subscribe special_use = \Junk } mailbox Envoyés { auto = subscribe special_use = \Sent } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/ldap.conf.ext driver = ldap } plugin { mail_replica = tcp:mailstore-1.mailstore.piafe.svc.cluster.local quota = dirsize:User quota quota_grace = 10%% quota_max_mail_size = 20M quota_rule = *:storage=1G quota_rule2 = Trash:storage=+200M quota_status_access = DUNNO quota_status_nouser = DUNNO quota_status_overquota = 552 5.5.2 Mailbox is full } postmaster_address = postmaster(a)piafe.recouv.fr replication_dsync_parameters = -d -N -l 30 -U -T 10 service aggregator { fifo_listener replication-notify-fifo { user = mailer } unix_listener replication-notify { user = mailer } } service doveadm { inet_listener { port = 12345 } inet_listener http { port = 8080 } } service lmtp { inet_listener lmtp { address = * port = 24 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = mailer mode = 0660 } } ssl = no userdb { args = /etc/dovecot/ldap.conf.ext default_fields = uid=500 gid=500 home=/var/mail/%Ld/%Ln driver = ldap } protocol imap { mail_plugins = " notify replication quota quota" }

1 0

Sendmail lmtp delivery and director
by Marc Roos 07 Nov '19

07 Nov '19

If I have sendmail configured to deliver to dovecot lmpt and use the director are incoming messages directed to the correct/configured server? Sendmail.mc with: FEATURE(`local_lmtp',`[IPC]',`FILE /var/run/dovecot/lmtp')dnl

1 0

Using attachment_dir with plugin zlib corrupt mails
by MAREN ZUBIZARRETA 06 Nov '19

06 Nov '19

Hello: I have found the same problem reported above by Patrick Cernko affecting our system and corrupting our messages. Even worse, Outlook 2016 will no synchronize and the clients cannot see any message, even if there is only one corrupted mail per mailbox. I cannot figure out a feasible workaround for our system, and I can see that in new version 2.38 the bug is not fixed. Will this issue be treated soon? Thanks a lot Maren Zubizarreta WARNING: using attachment_dir with plugin zlib can corrupt mails Patrick Cernko pcernko at mpi-klsb.mpg.de<mailto:dovecot%40dovecot.org?Subject=Re:%20Re%3A%20WARNING%3A%20using%20attachment_dir%20with%20plugin%20zlib%20can%20corrupt%20mails&In-Reply-To=%3C1ab8edcc-bef4-3b41-d09a-c523674a5031%40mpi-klsb.mpg.de%3E> Fri Jul 19 17:52:37 EEST 2019 * Previous message: index worker 2.3.7 undefined symbol errors<https://dovecot.org/list/dovecot/2019-July/116617.html> * Next message: Address family not supported by protocol<https://dovecot.org/list/dovecot/2019-July/116571.html> * Messages sorted by: [ date ]<https://dovecot.org/list/dovecot/2019-July/date.html#116570> [ thread ]<https://dovecot.org/list/dovecot/2019-July/thread.html#116570> [ subject ]<https://dovecot.org/list/dovecot/2019-July/subject.html#116570> [ author ]<https://dovecot.org/list/dovecot/2019-July/author.html#116570> ________________________________ Hello list, hello Dovecot developers, this week, I discovered a serious bug in Dovecot, that lead to several broken mails on our servers. The bug corrupts the first few characters of the mail header during saving. On our setup, it was almost always only the very first line of text, that was corrupted. Depending on the IMAP client (they seem to request different header fields, ... during mail access), the bug causes the imap process to hang up the TCP connection and log errors like this: > imap(USERNAME)<4767><TeQP4ASOTK5/AAAB>: Error: Corrupted record in index cache file /IMAP/mail/mailboxes/USERNAME/mdbox/mailboxes/Trash/dbox-Mails/dovecot.index.cache: UID 489113: Broken fields in mailbox Trash: read(attachments-connector(zlib(/IMAP/mail/mailboxes/USERNAME/mdbox/storage/m.813))): FETCH BODY[HEADER.FIELDS (RETURN-PATH SUBJECT)] got too little data: 2 vs 122 In our case that finally grabbed my attention, the client was the users iphone that did not display any new messages but his Thunderbird did. The bug seems to be triggered by a bad "interaction" of attachment_dir option and zlib plugin. If you use both, you most likely are affected, too, except you only use zlib plugin for reading previously compressed stored mails. That's also the workaround we use now: zlib plugin only enabled in mail_plugins but no plugin/zlib_save set. The bug occurs on very specific mails. Due to privacy reasons I could not provide sample mails here. Storing such mails seems to trigger the bug reproducible. I attached a very minimal doveconf -n config, that can be used to trigger the bug. If one of the developers is interested, I can try to generate an "anonymized" version of such a specific mail that still causes the issue. I discovered the bug on our productive systems, running latest Dovecot 2.2 release, but the latest 2.3 I used during debugging is affected, too. During debugging, I also found one hint, that might help find the bug: If you store a problematic mail with zlib_save=gz (or zlib_save=bz2) and then disable the zlib plugin in mail_plugins, you can call doveadm fetch -u test hdr all | grep -v ^hdr: | gzip --decompress on test's mailbox with only that one broken mail. This will display the beginning of the rfc822 mail text until gzip terminates with "gzip: stdin: unexpected end of file", approximately after twice the length of the mail HEADER. This might indicate, that dovecot stores the uncompressed size of the header in it's data structures although the mail is stored compressed. I also found a very efficient way to find all affected mails in our setup: doveadm -f flow fetch -A 'user guid mailbox uid seq flags hdr' all | \ grep -a "^[^ ]+ user=" | \ grep -avF ' hdr=Return-path: ' | \ grep -av '.* hdr=[[:print:][:space:]]*$' (runtime for ~6M mails on our servers was 20-30min) This can be even more optimized if you have a powerful storage system with GNU parallel: > doveadm user '*' | parallel "doveadm -f flow fetch -u '{}' 'user guid mailbox uid seq flags hdr' all | grep -a '^user=' | grep -avF ' hdr=Return-path: ' | grep -av '.* hdr=[[:print:][:space:]]*$' || true" (runtime for ~6M mails on our servers was ~4min) The command will give you a list of mails that possibly are affected, check the full output of doveadm fetch -u USERNAME hdr guid GUID | less to verify that the header is really broken. On our systems I found 39 mails within ~12M mails. I was able to recover these mails "manually" by reconstructing the Return-Path header line, importing the fixed mails and expunging the corrupt ones. Before importing, I had to disable zlib_save option obviously. Best regards, -- Patrick Cernko <pcernko at mpi-klsb.mpg.de<https://dovecot.org/mailman/listinfo/dovecot>> +49 681 9325 5815 Joint Administration: Information Services and Technology Max-Planck-Institute fuer Informatik & Softwaresysteme

6 11

Trying to replace anti spam plugin with IMAPSieve; no joy
by Brad Koehn 06 Nov '19

06 Nov '19

I’m trying to follow the instructions on the wiki to move from antispam to IMAPSieve, but it’s not working at all. A bit of a background: dovecot-core 2:2.3.8-4~buster dovecot-imapd 2:2.3.8-4~buster dovecot-lmtpd 2:2.3.8-4~buster dovecot-managesieved 2:2.3.8-4~buster dovecot-pgsql 2:2.3.8-4~buster dovecot-pop3d 2:2.3.8-4~buster dovecot-sieve 2:2.3.8-4~buster dovecot-solr 2:2.3.8-4~buster My regular sieve scripts work fine for messages arriving through LMTP. I’ve updated my 20-imap.conf thusly: ## ## IMAP specific settings ## protocol imap { mail_max_userip_connections = 30 # Space separated list of plugins to load (default is global mail_plugins). mail_plugins = $mail_plugins imap_acl imap_sieve } plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_trace_dir = /tmp sieve_trace_level = matching # From elsewhere to Spam folder imapsieve_mailbox1_name = Spam imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_before = file:/usr/lib/dovecot/sieve/report-spam.sieve # From Spam folder to elsewhere imapsieve_mailbox2_name = * imapsieve_mailbox2_from = Spam imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve sieve_pipe_bin_dir = /usr/lib/dovecot/sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment } ======== I created the report-spam.sieve and report-ham.sieve scripts in /usr/lib/dovecot/sieve and compiled them with sievec: require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"]; if environment :matches "imap.mailbox" "*" { set "mailbox" "${1}"; } if string "${mailbox}" "Trash" { stop; } pipe :copy "sa-learn-pipe.sh" [ "ham" ]; ======== require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"]; pipe :copy "sa-learn-pipe.sh" [ "spam" ]; ======== Lastly, I have the sa-learn-pipe.sh script in /usr/lib/dovecot/sieve: #!/bin/bash echo "`date` learning $*" >> /tmp/sa-learn # We run amavisd-new on another Kubernetes pod and communicate with it via socat on both ends; this worked with the old antispam plugin. if [ "$*" = "ham" ] ; then socat - TCP:amavis:4443 else socat - TCP:amavis:4444 fi exit 0 ======== The /usr/lib/dovecot/sieve directory looks like this: drwxr-xr-x 1 root root 4096 Nov 4 20:49 . drwxr-xr-x 1 root root 4096 Nov 3 20:06 .. -rwxr-xr-x 1 root root 237 Nov 3 15:56 report-ham.sieve -rw-r--r-- 1 root root 384 Nov 4 20:49 report-ham.svbin -rwxr-xr-x 1 root root 122 Nov 3 15:56 report-spam.sieve -rw-r--r-- 1 root root 290 Nov 4 20:49 report-spam.svbin -rwxr-xr-x 1 root root 208 Nov 3 17:58 sa-learn-pipe.sh When the sieve scripts run when mail arrives during LMTP delivery they dutifully log their work in /tmp, so I know sieve works and sieve logging works too. The sieve scripts above never log anything, nor does /tmp/sa-learn ever get created as it would if sa-learn-pipe.sh were ever run. As far as I can tell I’ve duplicated the configuration from the wiki almost exactly, but dovecot is just not running the sieve scripts when messages are moved into/out of Junk. What am I missing?

2 1

newbie question on a dovecot buffer
by Richard Bown 06 Nov '19

06 Nov '19

Hi apologies for what will seem a daft question I'm getting problems with my mail provider who is using dovecot. I've loaded Dovecot on a 8 core ARMHF SBC version 2.2.33.2 on Ubuntu 18.04. The SBC is headless. and the load looks very light and only needs to store/cache 4GB of mail Can I use Dovecot as a buffer between my mail providers Dovecot servers and the several IMAP mail clients on my own network ? ie, so Dovecot would mirror the mail on my account with my mail provider, So my local dovecot server downloads mail and caches/stores on the local SBC and all the local users use IMAP from that, instead of from my mail provider. Can I import the same directory structure as help on my mail providers server ? I'm trying to avoid the timeout problem I get when using IMAP from my mail provider. Can it be done ? and can you withstand the questions on configuration, I haven't used Dovecot before so its a vertical learning curve Thanks -- Best wishes /73 Richard Bown Email : richard(a)g8jvm.com HTTP : http://www.g8jvm.com ###################################################################### Ham Call: G8JVM . QRV: 50-432 MHz + Microwave Maidenhead QRA: IO82SP38, LAT. 52 39.720' N LONG. 2 28.171 W QRV 6mtrs 200W, 4mtrs 150W, 2mtrs 300W, 70cms 200W, Microwave 1296MHz 110W, 2320MHz 100W, 5760MHz 10W & 10368MHz 5W OS: Linux Mint 19.2 x86_64 Tina, on a Dell Inspiron 3580 laptop ###################################################################### Come back Guy Fawkes, all is forgiven

3 3

sievec *.sieve problem.
by Reio Remma 05 Nov '19

05 Nov '19

Hello! For the second time I've tripped onto this banana peel. :) I had 2 sieve files in a directory that I wanted to compile: sievec *.sieve The result of this is that first.sieve is compiled into second.sieve instead of first.svbin, thus destroying the source of second.sieve. Please consider this a bug report. :) Good luck, Reio

2 2

Help with master user
by Marcio Merlone 04 Nov '19

04 Nov '19

Hi, Trying to implement a master user (1) for auditing purposes without luck. Ubuntu 18.04.3, canonical official repos only, no ppa nor self-compiled anything. From the log below I understand the master password succeeds but AD auth fails. I am pretty sure I missed something here. Also, notice the messages "Ignoring unknown passdb extra field: original_user". Log: Nov 1 14:02:32 netuno dovecot: auth: Debug: client in: AUTH#0112#011PLAIN#011service=imap#011secured#011session=H2WM7kuWFKYKCQgI#011lip=10.9.8.8#011rip=10.9.8.8#011lport=143#011rport=42516#011resp=<hidden> Nov 1 14:02:32 netuno dovecot: auth: Debug: passwd-file(master,10.9.8.8,master,<H2WM7kuWFKYKCQgI>): Master user lookup for login: test.account Nov 1 14:02:32 netuno dovecot: auth: Debug: passwd-file(master,10.9.8.8,master,<H2WM7kuWFKYKCQgI>): lookup: user=master file=/etc/dovecot/master-users Nov 1 14:02:32 netuno dovecot: auth: passwd-file(master,10.9.8.8,master,<H2WM7kuWFKYKCQgI>): Master user logging in as test.account Nov 1 14:02:32 netuno dovecot: auth: ldap(test.account,10.9.8.8,<H2WM7kuWFKYKCQgI>): invalid credentials Nov 1 14:02:34 netuno dovecot: auth: Debug: client passdb out: FAIL#0112#011user=test.account#011authz#011original_user=master#011auth_user=master Nov 1 14:02:34 netuno dovecot: imap-login: Debug: Ignoring unknown passdb extra field: original_user Nov 1 14:02:34 netuno dovecot: imap-login: Debug: Ignoring unknown passdb extra field: auth_user Nov 1 14:02:42 netuno dovecot: imap-login: Aborted login (auth failed, 1 attempts in 10 secs): user=<test.account>, method=PLAIN, rip=10.9.8.8, lip=10.9.8.8, secured, session=<H2WM7kuWFKYKCQgI> doveconf -n: https://pastebin.com/3cAvfNqB root@netuno:/etc/dovecot# grep -v "^\s*#\|^\s*$" /etc/dovecot/dovecot-ldap.conf.ext hosts = ad.example.net auth_bind = yes auth_bind_userdn = cn=%Lu,CN=Users,DC=ad,DC=example,DC=net base = DC=ad,DC=example,DC=net scope = base user_attrs = \ =home=/mnt/maildirs/%Lu, \ =uid=vmail,\ =gid=vmail user_filter = (&(objectClass=person)(uid=%Lu)(mail=*(a)example.net)) root@netuno:/etc/dovecot# root@netuno:/etc/dovecot# cat /etc/dovecot/global-acls * user=master lr root@netuno:/etc/dovecot# (1) https://doc.dovecot.org/configuration_manual/authentication/master_users/ Best regards -- *Marcio Merlone*

1 1

Bug report
by Peter Nabbefeld 02 Nov '19

02 Nov '19

Hello, I've got a problem fetching my emails using getmail. First question is, of course, what's causing the problems. I'm going to think that getmail get's an error code from dovecot when trying to connect and just signals the most common error, so it seems to be a problem with dovecot. Sorry but I cannot upload any files with the silly web interface of my provider, so I cannot attach a core dump here. :-( Kind regards Peter ~/.getmail/log ========= 2019-11-01 21:44:20 Delivery error (command deliver 42245 error (127, exec of command deliver failed (change UID/GID to vmail/vmail failed ([Errno 1] Operation not permitted)))) dovecot-info.log ========== Oct 31 20:27:00 lda(mein.name(a)gmx.de)<42409><2+iBJIQ1u12ppQAA7IIYzw>: Panic: file istream-crlf.c: line 24 (i_stream_crlf_read_common): assertion failed: (ret != -2) Oct 31 20:27:00 lda(mein.name(a)gmx.de)<42409><2+iBJIQ1u12ppQAA7IIYzw>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xff784) [0x7f4ec8e62784] -> /usr/lib/dovecot/libdovecot.so.0(+0xff7c1) [0x7f4ec8e627c1] -> /usr/lib/dovecot/libdovecot.so.0(+0x493aa) [0x7f4ec8dac3aa] -> /usr/lib/dovecot/libdovecot.so.0(+0x4bf62) [0x7f4ec8daef62] -> /usr/lib/dovecot/libdovecot.so.0(+0x111076) [0x7f4ec8e74076] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_memarea+0x81) [0x7f4ec8e70291] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x37) [0x7f4ec8e70477] -> /usr/lib/dovecot/libdovecot.so.0(+0x1164a8) [0x7f4ec8e794a8] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_memarea+0x81) [0x7f4ec8e70291] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x37) [0x7f4ec8e70477] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x4c) [0x7f4ec8e70f3c] -> /usr/lib/dovecot/libdovecot.so.0(io_stream_copy+0x8a) [0x7f4ec8e8bbea] -> /usr/lib/dovecot/libdovecot.so.0(+0x12aa86) [0x7f4ec8e8da86] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_send_istream+0x4f) [0x7f4ec8e8b86f] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_save_continue+0x2e) [0x7f4ec901114e] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_save_continue+0x27) [0x7f4ec8fafbe7] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_continue+0x47) [0x7f4ec8f815b7] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_copy+0xe8) [0x7f4ec8f72388] -> /usr/lib/dovecot/libdovecot-storage.so.0(maildir_copy+0x6d) [0x7f4ec8fab99d] -> /usr/lib/dovecot/libdovecot-lda.so.0(+0x3c46) [0x7f4ec90a2c46] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x11004) [0x7f4ec8af0004] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x59b0c) [0x7f4ec8f81b0c] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4edb8) [0x7f4ec67e0db8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_result_execute+0x54d) [0x7f4ec67d65cd] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x57a79) [0x7f4ec67e9a79] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_multiscript_run+0xa9) [0x7f4ec67eac49] -> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x376b) [0x7f4ec685c76b] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x199) [0x7f4ec90a46d9] dovecot -n ======= # 2.3.8 (9df20d2db): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.8 (b7b03ba2) # OS: Linux 5.3.7-arch1-2-ARCH x86_64 Arch Linux # Hostname: mein.host.name auth_verbose = yes disable_plaintext_auth = no info_log_path = /var/log/dovecot-info.log log_path = /var/log/dovecot.log mail_location = maildir:~/Maildir:LAYOUT=fs mail_plugins = " fts fts_solr" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext editheader namespace inbox { inbox = yes location = mailbox Entwürfe { special_use = \Drafts } mailbox Gesendet { special_use = \Sent } mailbox Spamverdacht { special_use = \Junk } mailbox Trash { special_use = \Trash } prefix = } passdb { args = username_format=%u /etc/dovecot/passwd driver = passwd-file } plugin { fts = solr fts_solr = url=http://localhost:8983/solr/ sieve = file:/home/vmail/sieve;active=/home/vmail/.dovecot.sieve sieve_extensions = +editheader } service stats { unix_listener stats-reader { group = vmail mode = 0666 user = vmail } unix_listener stats-writer { group = vmail mode = 0666 user = vmail } } ssl = no ssl_dh = # hidden, use -P to show it userdb { args = uid=vmail gid=vmail home=/home/vmail/%d/%n driver = static } protocol lda { mail_plugins = " fts fts_solr sieve" postmaster_address = mein.name(a)gmx.de }

3 2

warning: NFS hangs with dovecot 2.3.8 on Debian buster
by Jan-Pieter Cornet 01 Nov '19

01 Nov '19

A warning to those considering to upgrade to Debian 10 (buster): we have seen occasional NFS hangs with dovecot when using the stock debian buster kernel (4.19.67-2+deb10u1). When we downgrade to the debian stretch kernel (4.9.189-3+deb9u1), the issue does not occur. Note that we *only* downgraded the kernel, the rest of the OS is still debian buster. Dovecot 2.3.8. A little more info: we have a dovecot cluster, using mdbox for storage, on an NFS server (netapp Cmode version 9.6P2). We use a dovecot director layer, so a user is always connected to the same back-end dovecot server. The NFS hang occurs on the back-end server. Once the process hangs, other processes trying to write to the same mailbox, will get an error like this: Timeout (180s) while waiting for lock for transaction log file /var/mail/.../index/storage/dovecot.map.index.log (WRITE lock held by pid XXXX) The stuck process itself doesn't seem to do anything, is stuck in "D" disk state, "strace" doesn't show anything (and after attaching, strace itself needs a kill -KILL to stop). The only way to unwedge the process seems to be to do a kill -KILL of the stuck process. Reading from the mailbox is still possible. We are in the process of contacting the linux-nfs folks about this, but I'm trying to reproduce this on a test-cluster first, to be able to present a well-documented case. Since this hang doesn't happen immediately, but takes a few hours to a day to occur in the wild, or a few thousand writes to the same mailbox, it's a bit hard to debug. -- Jan-Pieter Cornet <johnpc(a)xs4all.net> Systeembeheer XS4ALL Internet bv www.xs4all.nl

1 1

Dovecot and MySQL aborted connections.
by Benjamin Connelly 01 Nov '19

01 Nov '19

We also spotted these sql connections getting aborted, upon upgrading MySQL from 5.6 to 5.7. (Going back to 5.6 we don't see them!) Turning on mysql general query logging we can see it is Dovecot's mysql connections that inquire about or update quota usage in particular: *** /logs//mysql.log *** 2019-10-30T10:52:22.624690-07:00 2 Connect dovecot@localhost on npomail using Socket 2019-10-30T10:52:40.019780-07:00 2 Query SELECT bytes FROM quota2 WHERE username = 'a@bla' 2019-10-30T10:52:40.020948-07:00 2 Query SELECT messages FROM quota2 WHERE username = 'a@bla' 2019-10-30T10:53:40.113374-07:00 2 [Note] Aborted connection 2 to db: 'npomail' user: 'dovecot' host: 'localhost' (Got an error reading communication packets) Increasing mysqld's net_read_timeout and net_write_timeout values to 3600 does not stop the aborted connections. It seems asymptomatic. If the connection to mysql is still there to do quota operations, Dovecot seems to use it. If it's been aborted by the server, Dovecot seems to just (re)connect. . . But I'd love to help find the root or file a bug report or learn what it is we've configured wrong. . . Benjamin

4 5

dovecot and ldap
by Maciej Milaszewski IQ PL 01 Nov '19

01 Nov '19

Hi Sorry for my question... I use dovecot+ldap How realy works (in dovecot-2.2.x ) lists of LDAP hosts to use ? ------ dovecot.conf ---- hosts = ldap.domain.pl:389 ldap-slave.domain.pl:389 #uris = -------- This is simpe HA ? I mean if ldap.domain.pl have problem another request go to ldap-slave.domain.pl ?

2 1

Broken uids: Director lmtp proxy able to change the destination address?
by Erik de Waard 31 Oct '19

31 Oct '19

Hi, Director -> Backends -> NFS setup. Problem: Broken dovecot-uidlists because of our postfix LMTP delivery using a different address than the IMAP client lookups Error: Mailbox INBOX: Broken file dovecot-uidlist line 169: Invalid data: example: LMTP delivery: lmtp(info#domain.nl(a)mail.local) IMAP lookup: info(a)domain.nl doveadm director status info#domain.nl(a)mail.local Current: 10.0.10.223 (expires 2019-11-01 08:52:42) Hashed: 10.0.10.223 Initial config: 10.0.10.223 doveadm director status info(a)domain.nl Current: 10.0.10.113 (expires 2019-11-01 13:47:27) Hashed: 10.0.10.113 Initial config: 10.0.10.113 On the backends these would write to the same NFS maildir: Now Director would assign these addresses to different backend servers and thus result in the expected corrupted control files. at this time i'm unable to reconfigure the Postfix LMTP delivery in a way that these use the same lookup format. Questions: 1. Disable LMTP updating dovecot-uidlist and indexes. Perhaps using Maildir:CONTROL=/dev/null :INDEX=MEMORY for lmtp-sql.conf? 2. Since i'm using director/proxy is it possible to rewrite the address using SQL in the passdb query? 3. What would be the best settings to avoid corruption in this broken setup.

1 0

encrypt incoming emails with public gpg key before they are stored to maildir
by Fourhundred Thecat 31 Oct '19

31 Oct '19

Hello, I have asked on the postfix mailing list for a solution, how to encrypt incoming emails with public gpg key My original idea was to use a smtpd-milter, which would encrypt all incoming plaintext messages of given user, using the users public gpg key. This way, it would look as if the original sender has sent the message encrypted. Somebody suggested this might be better done in Dovecot, as it is Dovecot who stores the emails in Maildir. Does anybody have any suggestions how to implement this scheme in Dovecot ? I am including the original email below: > when new email arrives, and it is not already encrypted, I would like to > run it through a filter, which would encrypt the message with my public > gpg key, as if the original sender has sent the email encrypted. > > Why do I want to do this ? Why not ask the sender to send encrypted > messages to start with ? > > Lets say my bank sends me emails. I cannot force my bank to use gpg > encryption. I am happy they use email at all, instead of paper mail. > > My email server is untrusted. It can be hacked into and emails stolen. > Full disk encryption will not help, because the disk must be decrypted > during runtime. > > With my scheme, all emails would be stored encrypted on my server, and > decryption key does not exist on the server (emails are decrypted on my > local client) > > What would be the best way to implement this ? > > Can such filter work, without ever storing plaintext email on disk ? thanks,

3 2

Dovecot HSM
by Júlio Covolato 31 Oct '19

31 Oct '19

Hi. I'm looking for a tutorial/how-to for a HSM (Hierarchical /Storage/ Management). keeping old messages for a user in a cheap storage and recent messages in a faster one. I see on dovecot2 wiki an alternative for hsm as "Alternate storage", but I don't now if it's a good solution for me. The expected result is a faster imap/pop access for new messages on a "heavy mail users" mailbox, and obviously low cost! Thanks in advance. -- ---------------------------------- _ Engº Julio Cesar Covolato 0v0 <julio(a)psi.com.br> /(_)\ F: +55 11 99175-9260 ^ ^ PSI INTERNET ----------------------------------

2 1

Re: Dovecot HSM
by Marc Roos 30 Oct '19

30 Oct '19

1 0

How do I search for "greetings of the day"
by Ralf Hildebrandt 30 Oct '19

30 Oct '19

My users say that "greetings of the day" occurs in academic spam a lot. Since I don't trust my users, I opted to verify this bold claim. But how do I search for a sequence of multiple words? doveadm import -u restore(a)backup.invalid mdbox:/home/copymail/mdbox "" mailbox INBOX BODY "Greetings of the day" doveadm import -u restore(a)backup.invalid mdbox:/home/copymail/mdbox "" mailbox INBOX TEXT "Greetings of the day" (what's the difference betweeen BODY and TEXT?) Is it even possible, given that "the" and "of" are probably stop words? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | https://www.charite.de

2 1

Recovering emails from "dead" server
by Mark | Small World Systems 29 Oct '19

29 Oct '19

I have a Linix server that won't boot. I've patched in a live CD and mounted the original HDDs so I can browse the files using SSH. I am trying to find where all my customer emails are stored so that I can (hopefully) copy them off this "dead" server onto a new server which is also running cPanel and Dovecot. Does anyone know where Dovecot stores emails for each email account when it runs under cPanel? I can see a directory called /home/<cpanel username>/mail/<domain name>/<mail username> (I've used <xxxxx> in place of the actual text to obviscate the actual user information). If I navigate to that directory I can see lots of interesting sub directories e.g. cur, new and tmp plus some files such as: dovecot.index.cache, dovecote.index, dovecote-uidlist But then I'm stumped. I was hoping to find a big fat directory that contained all the mail files. Is anyone able to tell me what I need to copy across to get all the old emails off the "dead" server onto the new one? Thank you.

3 2

Sieve problem - Timeout while waiting for lock
by Domenico Pastore 29 Oct '19

29 Oct '19

Hi, I have an issue with dovecot 2.3.7.2. When lmtp stores a message into INBOX, I get this error: 2019-10-29 16:32:53 lmtp(“test@XXXX”)<28874><4IPiEytbuF3KcAAAioI89A>: Error: sieve: Execution of script /config/dovecot/common/default.sieve was aborted due to temporary failure 2019-10-29 16:32:53 lmtp(“test@XXXX”)<23470>: Error: sieve: msgid=CE595339-2976-6DFD-FDAD-EEA15CC53218@XXXX: failed to store into mailbox ‘INBOX’: Timeout while waiting for lock This problem only occurs for one mailbox and when it receives a big amount of messages. With the previous version (dovecot 2.2.15) this did never occur. The file /config/dovecot/common/default.sieve is in local server, no on NFS filesystem BR, Domenico ----------------- # 2.3.7.2 (3c910f64b): /config/dovecot/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: Linux 3.10.0-1062.1.2.el7.x86_64 x86_64 CentOS Linux release 7.7.1908 (Core) auth_debug = yes auth_master_user_separator = * auth_username_format = base_dir = /data/dovecot/var/run/dovecot default_client_limit = 8192 default_vsz_limit = 768 M dict { sqlquota = mysql:/config/dovecot/common/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it first_valid_gid = 101 first_valid_uid = 102 imap_id_send = import_environment = TZ info_log_path = /LOGS/dovecot/dovecot.log instance_name = dovecot listen = dovecot log_path = /LOGS/dovecot/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_gid = 101 mail_location = maildir:%h/Maildir mail_max_userip_connections = 50 mail_plugins = quota fts fts_solr mail_log notify virtual mail_uid = 102 maildir_broken_filename_sizes = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.debug mbox_write_locks = fcntl namespace { hidden = yes list = no location = virtual:/config/dovecot/common/virtual:INDEX=~/virtual:SUBSCRIPTIONS=.subscriptions/%u prefix = Messaggi in arrivo. separator = . subscriptions = no } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . subscriptions = yes type = private } passdb { args = /config/dovecot/common/passwd.masterusers driver = passwd-file master = yes } passdb { args = /config/dovecot/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = /config/dovecot/common/dovecot-ldap-multiuser.conf driver = ldap master = yes } passdb { args = /config/dovecot/common/dovecot-ldap-webmail.conf.ext driver = ldap master = yes } plugin { fts = solr fts_autoindex = yes fts_autoindex_max_recent = 1000 fts_solr = url=https://192.160.10.81:8983/solr/dovecot/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change save mailbox_create mail_log_fields = uid box msgid size flags quota = dict:User quota::proxy::sqlquota quota_rule2 = INBOX.Trash:storage=+100M sieve_default = /dovecot/common/default.sieve sieve_extensions = +vnd.dovecot.debug } postmaster_address = posta(a)XXXX.it service anvil { client_limit = 8000 } service auth { client_limit = 12000 unix_listener auth-userdb { mode = 0600 } } service dict { process_limit = 7500 process_min_avail = 10 unix_listener dict { mode = 0600 user = dovecot } } service doveadm { chroot = client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = group = idle_kill = 0 inet_listener { port = 924 } privileged_group = process_limit = 0 process_min_avail = 10 protocol = service_count = 100 type = unix_listener doveadm-server { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 5000 process_min_avail = 10 } service imap { drop_priv_before_exec = yes process_limit = 5000 process_min_avail = 10 } service lmtp { inet_listener lmtp { port = 24 } process_limit = 2500 process_min_avail = 10 } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve { drop_priv_before_exec = yes process_limit = 1024 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_limit = 1000 process_min_avail = 5 } service pop3 { drop_priv_before_exec = yes process_limit = 1000 process_min_avail = 5 } ssl_cert = </config/dovecot/common/certs/dovecot.pem ssl_client_ca_file = /config/solr-certs/solr-ssl.pem ssl_key = # hidden, use -P to show it userdb { args = /config/dovecot/dovecot/dovecot-ldap.conf driver = ldap } protocol lmtp { mail_plugins = quota mail_log notify sieve virtual fts fts_solr } protocol lda { mail_plugins = quota mail_log notify sieve virtual } protocol imap { mail_plugins = quota imap_quota mail_log notify virtual fts fts_solr } protocol pop3 { mail_plugins = quota mail_log notify virtual pop3_uidl_format = UID%u-%v }

1 1

Error: Mailbox INBOX: Broken file dovecot-uidlist: Invalid data
by Dirk 29 Oct '19

29 Oct '19

Hi list, today a user reported messages are disappearing from his INBOX and I discovered a two digit number of mailboxes with broken dovecot-uidlist files. The log says: Oct 29 08:51:34 mail02 dovecot: lmtp(info(a)example.com)<5270><QOoVJIbvt12WFAAAJOVBUw>: Error: Mailbox INBOX: Broken file /var/vmail/example.com/info/dovecot-uidlist line 88: Invalid data: And the respective files do look broken: 4596 :1572344253.M529133P14861.mail03.example.com,S=53053,W=54233 ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@4597 :1572344797.M93833P15905.mail03.example.com,S=126866,W=129256 Setup : Behind a director I am running two dovecot backend servers sharing the vmail directory via NFS. Do you have an idea where this might come from? How can I debug this further? Cheers, Dirk

1 0