On 20/02/2017 9:09 PM, Timo Sirainen wrote:

On 17 Feb 2017, at 21.37, Wolfgang Hennerbichler wogri@wogri.com wrote:

...

Hi Dovecot Users,

I’ve configured dovecot dsync replication and I see troubles in the logs and get user complaints which I can’t explain. I found similar threads on this mailinglist, but I couldn’t find a solution anywhere. Does anybody have dsync running without problems on a high volume mailserver?

I see the following logs, examples given:

Feb 17 18:16:49 dovecot dovecot: imap(zoechi): Warning: /var/mail/zoechi/dovecot-uidlist: Duplicate file entry at line 10395: 1487350019.M138380P28563.dovecot.wogri.at,S=18930,W=19377 (uid 41092 -> 41093) - retrying by re-reading from beginning

with this one I’m not sure - it might be that this is completely OK because due to replication UIDs clash. Maybe that’s OK, but I couldn’t find a confirmation.

Feb 17 18:16:49 dovecot dovecot: imap(zoechi): Warning: Maildir /var/mail/zoechi: Expunged message reappeared, giving a new UID (old uid=41092, file=1487350019.M138380P28563.dovecot.wogri.at,S=18930,W=19377)

There seems to be something weird with using Maildir and replication. Haven't had time to debug it and it's likely not an easy bug to fix, so for now the solution would be to use only sdbox/mdbox with replication.

I have experienced this quite a few times. It is tricky to reliably reproduce. I am also using replication with Maildir.

This appears to be a timing/locking issue of some sort. One way to trigger this seems to be to have a new mail come in and be delivered, and then as soon as it appears in Thunderbird, delete it straight away.

Thunderbird is set to move mails to Trash when they are deleted and expunge the Inbox on exit.

My replication partner sees the new mail has come in, and then it pushes it back to the original. Meanwhile the one on the original has already been deleted by the client so the mail re-appears back on the original again.

The latency to my replication partner is 150ms or so but this problem might become easier to reproduce if the delay is increased.

I can't say that this is definitely how to reproduce it all of the time but when I have experience this, that's roughly the sequence of events.

Reuben

On Feb 22, 2017, at 15:51, Wolfgang Hennerbichler wogri@wogri.com wrote: Thank you, I can confirm that after the migration to sdbox I don’t see those errors anymore.

Turns out that message still re-appear although all my mailboxes are converted to sdbox. The most likely case for a message to re-appear is that an unread message in the inbox is deleted. On the next imap sync it re-appears (only sometimes, not always). Although this time the server does not log any errors. This is my current dovecot config:

# dovecot -n # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.9.4 x86_64 Debian 8.7 ext4 auth_verbose = yes debug_log_path = /var/log/dovecot.debug doveadm_password = # hidden, use -P to show it first_valid_gid = 106 first_valid_uid = 104 hostname = localhost last_valid_gid = 106 last_valid_uid = 104 mail_gid = dovecot mail_location = sdbox:/var/mail/sdbox/%n mail_plugins = quota fts fts_lucene virtual notify replication mail_temp_dir = /var/lib/dovecot/tmp mail_uid = dovecot managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext editheader namespace { list = children location = virtual:/var/mail/virtual/%n prefix = virtual. separator = . } namespace inbox { inbox = yes list = yes location = mailbox "Deleted Messages" { auto = subscribe special_use = \Trash } mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { special_use = \Trash } prefix = separator = . subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { default_language = de fts = lucene fts_lucene = whitespace_chars=@. mail_replica = tcp:172.16.1.2:12345 quota = count:User quota quota_rule = *:storage=6G quota_rule2 = Trash:storage=+200M quota_rule3 = Spam:ignore quota_vsizes = yes quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /etc/sieve/%n.sieve sieve_default = /etc/sieve/default.sieve sieve_dir = ~/sieve sieve_extensions = +editheader } pop3_deleted_flag = $POP3Deleted postmaster_address = postmaster@wogri.at protocols = " imap lmtp sieve pop3" service aggregator { fifo_listener replication-notify-fifo { user = dovecot } unix_listener replication-notify { user = dovecot } } service doveadm { inet_listener { port = 12345 } } service imap-postlogin { executable = script-login /usr/local/bin/mail-location.sh } service imap { process_limit = 1024 } service lmtp { inet_listener lmtp { port = 2003 } unix_listener lmtp { user = dovecot } user = dovecot } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service pop3 { process_limit = 1024 } service quota-warning { executable = script /usr/local/sbin/quota-warning.sh unix_listener quota-warning { user = dovecot } user = dovecot } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 } } ssl = required ssl_cert =

wogri