- dovecot - dovecot.org

doveadm sync issues (revised)
by Sven Strickroth 10 May '19

10 May '19

Hi, I'm experiencing serious issues using doveadm sync; mails are not replicating and/or mails are duplicating. I wanted to use this for migrating external mailboxes to my server (in order to overcome the time when some servers/clients still use the old server and some already my server, as described on <https://wiki2.dovecot.org/Tools/Doveadm/Sync>). Finally I was able to reproduce this with the 2.3 branch (cf. <https://dovecot.org/pipermail/dovecot/2019-April/115600.html>). I'm sorry that I can only test the version 2.3.4.1, but according to the changelog I think this is still an issue with the latest version. I tried to find a few steps in order to replicate this easily: For testing purposes I simulate both the remote and local server on my Dovecot instance. I created remote(a)DOMAIN.TLD and local(a)DOMAIN.TLD accounts and both use mdbox and are empty. First, send two mails to remote(a)DOMAIL.TLD (Subjects: Test1 and Test2). Start backup remote IMAP account (remote(a)DOMAIN.TLD) to local account (local(a)DOMAIN.TLD) $ doveadm -o imapc_user="remote(a)DOMAIN.TLD" -o imapc_password="PW" -o imapc_host="localhost" backup -R -u local(a)DOMAIN.TLD imapc: Both mailboxes are in sync now (containing Test1 und Test2). Now send two messages to remote(a)DOMAIN.TLD (Subjects Test3 and Test4) and two messages to local(a)DOMAIN.TLD (Subjects Test5 and Test6). Run sync: $ doveadm -o imapc_user="remote(a)DOMAIN.TLD" -o imapc_password="PW" -o imapc_host="localhost" sync -R -u local(a)DOMAIN.TLD imapc: See that syncing fails: remote(a)DOMAIN.TLD contains: Test1,2,3,4,5,6 (as expected) local(a)DOMAIN.TLD contains: Test1,2,5,6,4 (Test3 is missing here) When not using the "-R" parameter it seems to work. Running sync again: $ doveadm -o imapc_user="remote(a)DOMAIN.TLD" -o imapc_password="PW" -o imapc_host="localhost" sync -R -u local(a)DOMAIN.TLD imapc: causes messages to duplicate: remote(a)DOMAIN.TLD contains: Test1,2,6,5,4,3,5,6,4,6,5,4,3,5,6 local(a)DOMAIN.TLD contains: Test1,2,5,4,6,4,5,6 (3 is still missing) This is independent of the "-R" parameter. I could also experience this issue when using one-way sync, however, the first one-way sync is still ok, but then messages start duplicating as for the two-way sync without "-R": $ doveadm -o imapc_user="remote(a)DOMAIN.TLD" -o imapc_password="PW" -o imapc_host="localhost" sync -R -1 -u local(a)DOMAIN.TLD imapc: Hope you can replicate this, too, and finally fix the issue. -- Best regards, Sven Strickroth PGP key id F5A9D4C4 @ any key-server

2 1

Re: Dovecot does not recognize imported messages
by lty 10 May '19

10 May '19

I changed the user permissions, this is a must. Because an email is sent to this user, all emails are identified. > Did you do something like "chown -R vmail:vmail /var/mail" after copy messages? > > ср, 8 мая 2019 г. в 12:57, lty via dovecot <dovecot(a)dovecot.org>: > > Today I tried to import mail directly on the server's home directory on the server. I copied it directly. I didn't use the imap protocol. Dovcot didn't recognize these emails. I didn't display these imported emails on my page, recalculating disk quotas. It will not increase the used capacity. I try to send an email to this user, dovcot recognizes all the emails, and recalculates the disk cooperation, this time is correct. > Why can't I identify the mail I imported? > > [root@imap2 ~]# dovecot -n > # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.24 (124e06aa) > # OS: Linux 2.6.32-431.el6.x86_64 x86_64 CentOS release 6.5 (Final) > # Hostname: imap > auth_master_user_separator = * > auth_mechanisms = PLAIN LOGIN > default_internal_user = vmail > default_vsz_limit = 2 G > deliver_log_format = msgid=%m, subject=%s, from=%f, %$ > dict { > acl = mysql:/etc/dovecot/dovecot-share-folder.conf > quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf > } > disable_plaintext_auth = no > first_valid_uid = 2000 > last_valid_uid = 2000 > listen = * > mail_fsync = never > mail_gid = 2000 > mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ > mail_plugins = quota lazy_expunge zlib fts fts_solr > mail_uid = 2000 > maildir_stat_dirs = yes > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext > mmap_disable = yes > namespace { > list = children > location = maildir:/%%Lh/Maildir/:INDEX=/%%Lh/Maildir/Shared/%%u > prefix = Shared/%%u/ > separator = / > subscriptions = yes > type = shared > } > namespace { > hidden = yes > list = no > location = maildir:/var/vmail/backup/expunged/%d/%n > prefix = .DELETED/ > separator = / > } > namespace inbox { > inbox = yes > location = > mailbox Drafts { > auto = subscribe > special_use = Drafts > } > mailbox Junk { > auto = subscribe > special_use = Junk > } > mailbox Sent { > auto = subscribe > special_use = Sent > } > mailbox Trash { > auto = subscribe > special_use = Trash > } > prefix = > separator = / > } > passdb { > args = /etc/dovecot/dovecot-mysql.conf > driver = sql > } > passdb { > args = /etc/dovecot/dovecot-fallback-password-mysql.conf > driver = sql > } > passdb { > args = /etc/dovecot/dovecot-fallback-mysql.conf > driver = sql > } > passdb { > args = /etc/dovecot/dovecot-master-users-password > driver = passwd-file > master = yes > } > plugin { > acl = vfile:/etc/dovecot/dovecot-acl > acl_globals_only = yes > auth_socket_path = /var/run/dovecot/auth-master > fts_autoindex_exclude = Junk > fts_autoindex_exclude2 = Trash > fts_autoindex_exclude3 = Sent > fts_autoindex_exclude4 = Drafts > fts_autoindex_max_recent_msgs = 20 > lazy_expunge = .DELETED/ > mail_replica = tcp:10.0.11.152:1111 [1] > quota = dict:user::proxy::quotadict > quota_grace = 0%% > quota_max_mail_size = 70M > quota_rule = *:storage=1G > quota_rule2 = *:messages=0 > quota_warning = storage=85%% quota-warning 85 %u > quota_warning2 = storage=90%% quota-warning 90 %u > quota_warning3 = storage=95%% quota-warning 95 %u > sieve = /%Lh/sieve/dovecot.sieve > sieve_default = /var/vmail/sieve/dovecot.sieve > sieve_default_name = managesieve > sieve_dir = /%Lh/sieve > } > protocols = pop3 imap sieve > service anvil { > client_limit = 20500 > } > service auth { > client_limit = 20500 > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0666 > user = postfix > } > unix_listener auth-master { > group = vmail > mode = 0666 > user = vmail > } > unix_listener auth-userdb { > group = vmail > mode = 0660 > user = vmail > } > } > service dict { > unix_listener dict { > group = vmail > mode = 0660 > user = vmail > } > } > service imap-login { > process_min_avail = 24 > service_count = 0 > vsz_limit = 2 G > } > service imap { > process_limit = 1024000 > process_min_avail = 24 > vsz_limit = 2 G > } > service pop3-login { > service_count = 0 > } > service quota-warning { > executable = script /usr/local/bin/dovecot-quota-warning.sh > unix_listener quota-warning { > group = vmail > mode = 0660 > user = vmail > } > } > ssl_ca = </etc/pki/tls/certs/c.cn.ca-bundle > ssl_cert = </etc/pki/tls/certs/c.cn.crt > ssl_key = # hidden, use -P to show it > syslog_facility = local6 > userdb { > args = /etc/dovecot/dovecot-mysql.conf > driver = sql > } > userdb { > args = /etc/dovecot/dovecot-fallback-user-mysql.conf > driver = sql > } > protocol lda { > auth_socket_path = /var/run/dovecot/auth-master > lda_mailbox_autocreate = yes > mail_fsync = optimized > mail_plugins = quota lazy_expunge zlib fts fts_solr sieve > syslog_facility = local6 > } > protocol imap { > imap_client_workarounds = tb-extra-mailbox-sep > imap_id_send = NAME dovecot VERSION 2.1.17 > mail_max_userip_connections = 20 > mail_plugins = quota lazy_expunge zlib fts fts_solr imap_quota mail_log imap_zlib notify > } > protocol pop3 { > mail_max_userip_connections = 20 > mail_plugins = quota lazy_expunge zlib fts fts_solr mail_log notify > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > pop3_uidl_format = %08Xu%08Xv > } -- С уважением, Рустам Багов. Links: ------ [1] http://10.0.11.152:1111

1 0

Dovecot not surviving OpenLDAP restart
by Dag Nygren 09 May '19

09 May '19

Hi! Running Dovecot 2.2.36 and authenticating against an OpenLDAP 2.4.45 server. Now since some update of dovecot it will not be able to authenticate your logins after a restart of the LDAP service is restarted without a reboot of the dovecot server. Anything new here that I should be aware of? Best Regards Dag

2 2

Re: Re: dict lookup too long time
by hfh＠cndns.com 09 May '19

09 May '19

logging some error,dict process is not dead. hfh(a)cndns.com From: Aki Tuomi via dovecot Date: 2019-05-09 14:21 To: hfh(a)cndns.com; dovecot Subject: Re: dict lookup too long time On 9.5.2019 8.00, hfh--- via dovecot wrote: hi, The following error message sometimes appears on my server log,it causes the mailbox to exceed the disk capacity limit. May 9 09:54:11 imap dovecot: imap(hfh(a)cndns.com): Error: dict-client: Lookup 'priv/quota/storage' failed: Connection closed: read(size=8192) failed: Connection reset by peer (reply took 2.728 secs (2.729 in dict wait, 0.000 in other ioloops, 0.000 in locks)) Is the dict process by chance dying or logging some error(s)? Aki

1 0

dovecot dict protocol
by Peter Hudec 09 May '19

09 May '19

Hi there is there any written specification for dovecot dict protocol instead of the code? Small example is in docs on https://wiki.dovecot.org/AuthDatabase/Dict and theere is link for all commands/reply in the code https://wiki.dovecot.org/AuthDatabase/Dict I’m writing small dict server and need some explanation for CMD: I, what does the flags means and how should the result looks like CMD: D, I thing this is the same as C, but coud be done later /async/ The reply codes are not very clear. I understand O, N, F. The rest is strange to me. Maybe this is good question do devel mailing list. regards Peter

1 0

UID to file?
by Carles Pina i Estany 09 May '19

09 May '19

Hello, In the past I was in a situation that given an email UID I wanted to know which file in the Maildir directory this UID was saved. I did a small parser for $HOME/Maildir/dovecot-uidlist where, if the file had: 1000 W1838 S1796 :1557351041.10447_1.example I would ask for the UID 1000 and would return :1557351041.10447_1.example (and I knew that it was in $HOME/Maildir/new) Is there any other way using dovecot tools to do the same, to avoid parsing an internal dovecot file? Thank you! -- Carles Pina i Estany GPG Key 0x8CD5C157

2 2

Push Notification clarification (MessageNew w/o LMTP or LDA)
by Christian Balzer 09 May '19

09 May '19

Hello, Both the examples on the Push Notification wiki page and the XAPS plugin docs seem to suggest or state that LMTP/LDA is required. However IMAP IDLE notifications work without either of these (please no religious discussion of why Dovecot LMTP is the best thing since sliced bread and that everybody should use it). The LUA part of the push notification docs however states that events other than MessageNew are supported. So my question is, when not using LMTP/LDA and a LUA script and of course the correct mail_plugins definition (either global or for IMAP), will a message delivery trigger MessageNew? Regards, Christian -- Christian Balzer Network/Systems Engineer chibi(a)gol.com Rakuten Communications

2 3

dict lookup too long time
by hfh＠cndns.com 09 May '19

09 May '19

hi, The following error message sometimes appears on my server log,it causes the mailbox to exceed the disk capacity limit. May 9 09:54:11 imap dovecot: imap(hfh(a)cndns.com): Error: dict-client: Lookup 'priv/quota/storage' failed: Connection closed: read(size=8192) failed: Connection reset by peer (reply took 2.728 secs (2.729 in dict wait, 0.000 in other ioloops, 0.000 in locks)) dovecot -n # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24 (124e06aa) # OS: Linux 2.6.32-504.el6.x86_64 x86_64 CentOS release 6.6 (Final) # Hostname: imap3bak.sh.chengmail.me auth_master_user_separator = * auth_mechanisms = PLAIN LOGIN deliver_log_format = msgid=%m, subject=%s, from=%f, %$ dict { acl = mysql:/etc/dovecot/dovecot-share-folder.conf quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf } disable_plaintext_auth = no first_valid_uid = 2000 last_valid_uid = 2000 listen = * mail_fsync = never mail_gid = 2000 mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ mail_plugins = quota lazy_expunge zlib mail_uid = 2000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mmap_disable = yes namespace { list = children location = maildir:/%%Lh/Maildir/:INDEX=/%%Lh/Maildir/Shared/%%u prefix = Shared/%%u/ separator = / subscriptions = yes type = shared } namespace { hidden = yes list = no location = maildir:/var/vmail/backup/expunged/%d/%n prefix = .DELETED/ separator = / } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } passdb { args = /etc/dovecot/dovecot-fallback-password-mysql.conf driver = sql } passdb { args = /etc/dovecot/dovecot-fallback-mysql.conf driver = sql } passdb { args = /etc/dovecot/dovecot-master-users-password driver = passwd-file master = yes } plugin { acl = vfile:/etc/dovecot/dovecot-acl acl_globals_only = yes auth_socket_path = /var/run/dovecot/auth-master lazy_expunge = .DELETED/ quota = dict:user::proxy::quotadict quota_grace = 0%% quota_max_mail_size = 70M quota_rule = *:storage=1G quota_rule2 = *:messages=0 quota_warning = storage=85%% quota-warning 85 %u quota_warning2 = storage=90%% quota-warning 90 %u quota_warning3 = storage=95%% quota-warning 95 %u sieve = /%Lh/sieve/dovecot.sieve sieve_default = /var/vmail/sieve/dovecot.sieve sieve_default_name = managesieve sieve_dir = /%Lh/sieve } protocols = pop3 imap sieve service anvil { client_limit = 20500 } service auth { client_limit = 20500 unix_listener /var/spool/postfix/dovecot-auth { group = postfix mode = 0666 user = postfix } unix_listener auth-master { group = vmail mode = 0666 user = vmail } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service imap-login { process_min_avail = 24 service_count = 0 vsz_limit = 2 G } service imap { process_limit = 1024000 process_min_avail = 24 vsz_limit = 2 G } service pop3-login { service_count = 0 } service quota-warning { executable = script /usr/local/bin/dovecot-quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } } ssl_ca = </etc/pki/tls/certs/cndns.com.ca-bundle ssl_cert = </etc/pki/tls/certs/cndns.com.crt ssl_key = # hidden, use -P to show it syslog_facility = local6 userdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } userdb { args = /etc/dovecot/dovecot-fallback-user-mysql.conf driver = sql } protocol lda { auth_socket_path = /var/run/dovecot/auth-master lda_mailbox_autocreate = yes mail_fsync = optimized mail_plugins = quota lazy_expunge zlib sieve postmaster_address = root syslog_facility = local6 } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep imap_id_send = NAME dovecot VERSION 2.1.17 mail_max_userip_connections = 20 mail_plugins = quota lazy_expunge zlib imap_quota mail_log imap_zlib notify } protocol pop3 { mail_max_userip_connections = 20 mail_plugins = quota lazy_expunge zlib mail_log notify pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } hfh(a)cndns.com

2 1

How to auto-subscribe to virtual namespace/prefix?
by MRob 09 May '19

09 May '19

In virtual namespace I can have: namespace myvirt { location = blah prefix=Virtual/ separator=/ mailbox "My folder" { auto = subscribe } } And i see email clients successful display subscribed "My folder" but the top-level prefix "Virtual/" is not subscribed. I see the Virtual/ prefix available as a noselect folder in the list of unsubscribed folders and if I manual subscribe to it, folder tree display is perfect. So how can i also make auto-subscription to the "Virtual/" prefix noselect folder?

1 2

dsync to backup to directory?
by Felipe Gasper 08 May '19

08 May '19

Hello, We have a hosting operation and are interested in offloading mail for some of our machines to a separate machine. We want the ability, though, to migrate accounts between the “offloaded” and “unified” configurations. Toward that end, is it possible to “merge” two different mail directories (either maildir or mdbox) via doveadm? “doveadm backup” looks like the closest match, but it doesn’t appear to accept an arbitrary directory as its synchronization source. Thank you! -Felipe Gasper Mississauga, Ontario

1 0