- dovecot - dovecot.org

Search in Mailbox via doveadm REST-API
by rudolf＠padaru.de 07 Sep '20

07 Sep '20

Hey, i need some help with the http-api from doveadm, i try to search a string in the mail-text from all mails in my inbox. My curl-post looks like these: curl -v -X POST -u <user>:<password> -H "Content-Type: application/json" -d '[["search",{"query":["mailbox","INBOX","TEXT","8887"],"user":"<mailUser>"}, "tag1"]]' http://<server>:8080/doveadm/v1 | python -mjson.tool > out.json I receive only empty results, do have anyone an idea? Mit freundlichen Grüßen Yours sincerely <https://padaru.de/> Pascal Rudolf | Geschäftsführer PADARU-IT | Bielebohstraße 10 | 02736 Beiersdorf @: rudolf(a)padaru.de | www: <http://padaru.de/> padaru.de mobil: +49 (0) 17 772 595 32

1 0

LMTP-Process stays in RCPT TO state
by Pascal Christen 07 Sep '20

07 Sep '20

Hi We tried to update our Dovecot Director-Ring to 2.3 (latest version) and had one problem/bug. When we try to deliver an email from Exim to a receiving Dovecot (2.3 latest) via the LMTP-Proxy the LMTP process stays open forever and never closes. So after a few hours, we hit the LMPT-Process limit. The problem only exists if the receiving Dovecot reports "Quota exceeded (mailbox for user is full)" back to the Exim. If this is the case the state on the LMPT process stays at "lmtp: [_IP_ RCPT TO] (lmtp) and never closes. The TCP-Connection to the Exim is then in CLOSE_WAIT state on the Proxy. On the receiving Dovecot we set the "lmtp_rcpt_check_quota=yes" option. For me, it sounds similar to this bug: https://dovecot.org/list/dovecot/2018-January/110697.html but without the SEGFAULT which makes it harder to debug it. May you guys know why it doesn't change the state to IDLE or READY after reporting the quota issue or see something fishy in the commit above. If not I try to get a trace of the hanging process. Greetings Pascal

2 6

Backup questions - errors
by Mark Patruck 07 Sep '20

07 Sep '20

Hi, although i've search the archives, there are questions left regarding backup strategies and errors that occur. I'm using dovecot 2.3.11.3 with mdbox (mailbox size 50MB - 30GB). Every night, i ran the following command: doveadm -o mail_fsync=never -o plugin/quota= -o plugin/zlib_save=gz \ backup -u account mdbox:/nfs/storage/account/mdbox ### incremental backup Thanks to zlib compression, i save a lot of storage space, nonetheless i'd go with an incremental backup, but it seems to only merge changes, comparing to rsyncs' --link-dest option where you can put all "changes of the day" in an extra folder. I'd like to prevent loosing mail that gets deleted by accident/on purpose. How can i achieve this with dsync and w/o doing a full backup every day? ### backup errors After a few days w/o any errors, doveadm prints the following lines (only for the biggest, 30GB mailbox) two days in a row. As dovecot has self- healing capabilities, i think these errors/warnings can be ignored and should go away...? Error: Transaction log ....mdbox/dovecot.list.index.log: duplicate transaction log sequence (38) Error: Index ....mdbox/dovecot.list.index: Lost log for seq=38 offset=63312: Missing middle file seq=38 (between 38..4294967295, we have seqs 37,39): .log.2 contains file_seq=37 (initial_mapped=1, reason=Index mapped) Warning: fscking index file ....mdbox/dovecot.list.index Error: Fixed index file ....mdbox/dovecot.list.index: log_file_seq 38 -> 39 Error: Mailbox list index was marked as fsck'd ....mdbox/dovecot.list.index Error: Mailbox list index was marked as fsck'd ....mdbox/dovecot.list.index Error: Mailbox list index was marked as fsck'd ....mdbox/dovecot.list.index Thanks, -Mark -- Mark Patruck ( mark at wrapped.cx ) GPG key 0xF2865E51 / 187F F6D3 EE04 1DCE 1C74 F644 0D3C F66F F286 5E51 https://www.wrapped.cx

1 1

sieve_max_script_size is ignored
by Zdeněk Zámečník 06 Sep '20

06 Sep '20

I am in troubles with compiling sieve scripts larger than 1MB. I see in logs following errors: Aug 19 13:10:26 mail dovecot: lmtp(z.z(a)xxx.xxx)<22117><uNBGHKIIPV9lVgAA5ldI4A>: Error: sieve: autoreply: line 16818: quoted string started at line 3 is too long (longer than 1048576 bytes) Aug 19 13:10:26 mail dovecot: lmtp(z.z(a)xxx.xxx)<22117><uNBGHKIIPV9lVgAA5ldI4A>: Error: sieve: autoreply: parse failed Aug 19 13:10:26 mail dovecot: lmtp(z.z(a)xxx.xxx)<22117><uNBGHKIIPV9lVgAA5ldI4A>: Error: sieve: Failed to compile script `/var/vmail/xxx.xxx/z.z/sieve/autoreply.sieve' I tried to set sieve_max_script_size = 2M or 0 (unlimited). I also tried to run "sievec -o sieve_max_script_size=2M /var/vmail/xxx.xxx/z.z/sieve/autoreply.sieve" but the result is still same. Below is my config. Can you please advice where can be the problem? # 2.3.10.1 (a3d0e1171): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.10 (67bf5bd7) # OS: Linux 5.4.44-2-pve x86_64 Debian 10.4 # Hostname: mail.xxx.xxx auth_cache_size = 2 M auth_cache_ttl = 5 mins auth_master_user_separator = * auth_mechanisms = plain login auth_policy_check_before_auth = no auth_policy_hash_nonce = # hidden, use -P to show it auth_policy_report_after_auth = no auth_policy_server_timeout_msecs = 1500 auth_policy_server_url = http://127.0.0.1:8090/ dict { acl = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no imap_client_workarounds = delay-newmail imap_id_log = * imap_id_retain = yes login_trusted_networks = 127.0.0.0/8 mail_gid = 2000 mail_home = /var/vmail/%d/%n mail_location = mdbox:~/mdbox:ALT=/var/vmail-archive/%d/%n/mdbox mail_max_userip_connections = 60 mail_plugins = acl zlib fts quota mail_uid = 2000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds editheader imapsieve vnd.dovecot.imapsieve mdbox_rotate_interval = 1 days mdbox_rotate_size = 16 M namespace { inbox = yes location = mailbox "Deleted Items" { auto = no special_use = \Trash } mailbox "Deleted Messages" { auto = no special_use = \Trash } mailbox Drafts { auto = no special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox "Junk E-mail" { auto = no special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Items" { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no special_use = \Junk } mailbox Trash { special_use = \Trash } mailbox virtual/All { auto = no special_use = \All } prefix = separator = / type = private } namespace { inbox = no list = children location = mdbox:/var/vmail/%%d/%%n/mdbox:ALT=/var/vmail-archive/%%d/%%n/mdbox prefix = shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = /etc/dovecot/masters.db driver = passwd-file master = yes pass = yes } plugin { acl = vfile acl_shared_dict = proxy::acl imapsieve_mailbox1_before = file:/var/vmail/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/var/vmail/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * imapsieve_mailbox3_before = file:/var/vmail/sieve/report-spam.sieve imapsieve_mailbox3_causes = COPY imapsieve_mailbox3_name = Junk imapsieve_mailbox4_before = file:/var/vmail/sieve/report-ham.sieve imapsieve_mailbox4_causes = COPY imapsieve_mailbox4_from = Junk imapsieve_mailbox4_name = * mailbox_alias_new = Sent Messages mailbox_alias_new2 = Sent Items mailbox_alias_new3 = Deleted Items mailbox_alias_old = Sent mailbox_alias_old2 = Sent mailbox_alias_old3 = Trash quota = dict:User quota::proxy::quota quota_grace = 10%% quota_rule2 = Trash:ignore quota_rule3 = Junk:ignore quota_warning = storage=80%% quota-warning 90 %u quota_warning2 = storage=85%% quota-warning 95 %u quota_warning3 = storage=95%% quota-warning 105 %u sieve = /var/vmail/%d/%n/sieve/.sieve sieve_after = /var/vmail/%d/%n/sieve/autoreply.sieve sieve_before = /var/vmail/sieve/global.sieve sieve_dir = /var/vmail/%d/%n/sieve sieve_extensions = +editheader +vacation-seconds sieve_global_dir = /var/vmail/sieve/ sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_max_redirects = 20 sieve_max_script_size = 2M sieve_pipe_bin_dir = /usr/lib/dovecot sieve_plugins = sieve_imapsieve sieve_extprograms sieve_vacation_send_from_recipient = yes } protocols = imap sieve lmtp pop3 service auth-worker { unix_listener auth-worker { user = vmail } user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = vmail } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { process_min_avail = 10 service_count = 0 vsz_limit = 512 M } service imap { executable = imap process_limit = 3500 unix_listener imap-master { user = $default_internal_user } vsz_limit = 2 G } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve { process_limit = 400 } service pop3-login { process_min_avail = 3 service_count = 0 vsz_limit = 320 M } service pop3 { process_limit = 200 vsz_limit = 320 M } service quota-warning { executable = script /etc/dovecot/quota_warning.sh unix_listener quota-warning { mode = 0666 user = vmail } user = vmail } ssl_cert = </etc/ssl/private/multi.z-technics.cz.dovecot.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it stats_writer_socket_path = userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } protocol sieve { managesieve_implementation_string = Dovecot Pigeonhole managesieve_max_compile_errors = 5 } protocol imap { mail_plugins = quota imap_quota fts mailbox_alias imap_acl acl imap_zlib imap_sieve } protocol lda { mail_fsync = optimized } protocol lmtp { mail_fsync = optimized mail_plugins = quota sieve acl }

2 4

imapc master user & shared mailboxes
by Toma 04 Sep '20

04 Sep '20

Hello, I am trying to configure dovecot like this: https://wiki.dovecot.org/SharedMailboxes/ClusterSetup imapc_host = 127.0.0.1 imapc_master_user = %u #imapc_user = imapc_password = XXXXXXXX namespace { type = shared separator = / prefix = shared/%%u/ location = imapc:~/shared/%%u } plugin { acl_shared_dict = proxy::acl } However, contrary to the article claims, imapc master user is set to the destination username. This makes it impossible to check the ACLs against the logged user. If user.b shares mailbox with user.a, accessing /shared/user.b as user.a a will log in as "user.b*user.b". Can we tell dovecot to use the currently authenticated user for imapc_master_user and the remote user for imapc_user, so it logs in as "user.b*user.a"? I'm using dovecot 2.3.11.3 (502c39af9). Thanks! Toma

1 0

syslog logging + 2.3.10 + centos 7
by Matt Bryant 04 Sep '20

04 Sep '20

All, Running dovecot 2.3.10 on a CentOS 7.x and logging via syslog seems to stop when i restart rsyslog process. Reload doesnt get it working again only a dovecot restart. Tested this with dovecot 2.2.36 and this did not exhibit the same behaviour. systemctl restart dovecot rsyslog ----- this gets a working state systemctl restart rsyslog ------------- this restarts syslog other processes like exim continue to log but dovecot stops systemctl reload dovecot -------------- this has no impact logging still does not work systemctl restart dovecot ------------- this fixes the issue. Are there any changes to the log process between 2.2 and 2.3 that might explain this ? rgds Matt

1 1

How to backup maildir
by Robert Moskowitz 04 Sep '20

04 Sep '20

I have been thinking, and reading, on how to back up my mailserver. I have not found any approach that seems ready to use. I have run years without any backup, but would really like to have something in place. I figure I can attach a USB drive and backup to that, then from there rsync to something elsewhere. Further if that USB drive is a full mailserver image, I actually have a 'hot backup' where I only have to put the backup drive into a system and boot up at the last backup. But this means properly copying all of /home/vmail and probably /home/sieve plus the /var/lib/mysql Are there good tools that nicely does this? Or do I choose a time late at night (only I am sometimes in non-US timezones) to shut down all services and just use rsync? And stopping services itself is thought provoking. What if Dovecot, amavis, mysql, or whatelse is in the middle of writing out a mail file what happens to that file and restart. Just scary stuff and, in part, why I have never tackled this in the past. thanks for all feedback

4 6

Deleted mail reappearing
by John S 04 Sep '20

04 Sep '20

Recently I upgraded dovecot quite a few major versions. Since then some mailboxes are experiencing deleted mail reappearing. It seems to be happening roughly once a day, but not for every mail client or mailbox, I have at least two clients with this issue: Thunderbird and Outlook. Storage is local under two partitions, one as alternative storage. # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 () # OS: Linux 5.4.0-29-generic x86_64 Ubuntu 20.04 LTS ext4 auth_mechanisms = plain login digest-md5 base_dir = /var/run/dovecot/ default_client_limit = 4000 default_vsz_limit = 512 M dict { expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no lda_mailbox_autocreate = yes log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = mdbox:/home/vmail/%d/%n mail_max_userip_connections = 240 mail_plugins = quota mail_privileged_group = mail mdbox_rotate_size = 64 M namespace { inbox = yes location = prefix = separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = dict:user_quota::proxy:/var/run/dovecot/dict:quota quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = file:~/sieve;active=~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_dir = /etc/dovecot/sieve/ } protocols = imap pop3 service anvil { client_limit = 2615 } service auth { client_limit = 5536 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = root mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { client_limit = 2115 executable = /usr/lib/dovecot/imap-login inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_limit = 2000 } service imap { executable = /usr/lib/dovecot/imap process_limit = 2000 } service pop3-login { client_limit = 1024 executable = /usr/lib/dovecot/pop3-login inet_listener pop3 { address = * port = 110 } inet_listener pop3s { address = * port = 995 } process_limit = 512 } service pop3 { executable = /usr/lib/dovecot/pop3 } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl_cert = < /etc/dovecot/ssl/crt.pem ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes syslog_facility = local0 userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { imap_max_line_length = 64 k mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota pop3_uidl_format = %08Xu%08Xv } protocol lda { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = quota sieve postmaster_address = postmaster@localhost sendmail_path = /usr/lib/sendmail } dovecot-sql.conf: driver = mysql connect = host=127.0.0.1 dbname=mail user=mail password=#### password_query = SELECT IF('%m' = 'DIGEST-MD5', users.password_digest_md5, NULL) as password, IF('%m' = 'DIGEST-MD5', NULL, 'Y') as nopassword, username FROM users LEFT JOIN domains ON users.domain = domains.domain WHERE users.username = '%u' AND users.domain = '%d' AND users.active = 'yes' AND domains.state = '1' AND ((users.password = encrypt('%w', LEFT(users.password, 11)) AND ('%m' = 'PLAIN' OR '%m' = 'LOGIN')) OR ('%m' = 'DIGEST-MD5')) user_query = SELECT maildir as home, uid, gid, CONCAT(CONCAT(type, ':', maildir), ':ALT=/home/altmail/%d/%n') AS mail FROM users LEFT JOIN domains ON users.domain = domains.domain WHERE users.username = '%u' AND domains.state = '1' iterate_query = SELECT SUBSTRING_INDEX(email, '@', 1) AS username, domain FROM users;

3 3

Sieve: deleteheader now working with duplicate filter for implicit keep
by Alec Moskvin 04 Sep '20

04 Sep '20

Hello, I have a rule to always delete a header. If the message gets fileinto'd, the header is gone, but if it's delivered into the INBOX through implicit keep, the header does not get deleted. Interestingly, if I remove the "if duplicate" filter, the header does get deleted as expected. I'm using the latest Dovecot 2.3.11.3 / Pigeonhole 0.5.11. Any idea why this is happening? Thanks, Alec === Sieve filter require ["fileinto", "editheader", "duplicate", "mailbox"]; deleteheader "X-Bad-Header"; if duplicate { fileinto :create "Trash"; stop; } if header :matches "subject" "*filter*" { fileinto :create "Filtered"; } === Example (note ${RANDOM} to bypass duplicate filter) $ /usr/libexec/dovecot/deliver << -end- To: me(a)example.com From: someone(a)example.com Subject: Test Message Message-Id: <${RANDOM}(a)example.net> Date: Thu, 20 Aug 2020 00:21:24 +0000 (UTC) X-Bad-Header: DELETE ME Hello world -end-

2 4

ltmp delivery with ldap and prefetch
by Marc Roos 04 Sep '20

04 Sep '20

I am able to authorize and connect via imap. But I am not able to deliver a message via lmtp I am getting this error message. passdb didn't return userdb entries, trying the next userdb [1] dovecot-ldap.conf.ext pass_attrs = uid=user,userPassword=password,host=host,homeDirectory=userdb_home,uidNu mber=userdb_uid,gidNumber=userdb_gid user_attrs = uid=%{ldap:uidNumber},gid=%{ldap:gidNumber},home=%{ldap:homeDirectory} [2] pass / user db's passdb { args = /etc/dovecot/dovecot-ldap.conf.ext auth_verbose = default default_fields = deny = no driver = ldap master = no mechanisms = name = override_fields = pass = no result_failure = continue result_internalfail = continue result_success = return-ok skip = never username_filter = } userdb { args = /etc/dovecot/special-userdb auth_verbose = default default_fields = driver = passwd-file name = override_fields = result_failure = continue result_internalfail = continue result_success = return-ok skip = never } userdb { args = auth_verbose = default default_fields = driver = prefetch name = override_fields = result_failure = continue result_internalfail = continue result_success = return-ok skip = never }

1 0