Replication weirdness

[Daniel Miller]

First, I have to say this. After configuring everything correctly - and 
that means *everything* correctly - Dovecot replication Just Works. I'm 
not sure how (yes I do - Timo & Co. Magic) - but it does. Real-time new 
sync is near instantaneous.

Now the problem. Or the background for the problem. My primary server 
uses sdbox for primary storage, mdbox for archival storage, and 
fts-solr. I spun up a second server, using sdbox, mdbox, and 
fts-flatcurve. My namespaces are as defined below. As best I can tell 
(based on diff comparing two 'doveconf -n' outputs) my namespaces are 
the same on both servers.

namespace archives {
   list = children
   location = mdbox:/var/mail/%d/%n/Archives/mdbox
   mailbox Unsorted {
     auto = no
     special_use = \Archive
   }
   prefix = INBOX/Archives/
   separator = /
   subscriptions = no
   type = private
}
namespace inbox {
   alias_for =
   hidden = no
   inbox = yes
   list = yes
   location =
   mailbox "Deleted Messages" {
     auto = no
     autoexpunge = 30 days
     special_use = \Trash
   }
   mailbox Drafts {
     auto = subscribe
     special_use = \Drafts
  }
   mailbox Sent {
     auto = subscribe
     special_use = \Sent
   }
   mailbox Trash {
     auto = subscribe
     autoexpunge = 30 days
     special_use = \Trash
   }
   prefix = INBOX/
   separator = /
   subscriptions = no
   type = private
}
namespace lists {
   list = children
   location = mdbox:/var/mail/%d/%n/Lists/mdbox
   prefix = INBOX/Lists/
   separator = /
   subscriptions = no
   type = private
}
namespace subscriptions {
   hidden = yes
   list = no
   location =
   prefix =
   separator = /
   subscriptions = yes
   type = private
}
namespace usershares {
   list = yes
   location = sdbox:/var/mail/%%d/%%n/sdbox:NO-NOSELECT
   prefix = INBOX/shared/%%d/%%n/
   separator = /
   subscriptions = no
   type = shared
}
namespace virtual {
   list = children
   location = virtual:/var/mail/%d/%n/virtual
   mailbox Flagged {
     comment = All my flagged messages
     special_use = \Flagged
   }
   prefix = INBOX/virtual/
   separator = /
   subscriptions = no
}

I also have:
plugin {
   mailbox_alias_new = Sent Messages
   mailbox_alias_new2 = Sent Items
   mailbox_alias_new3 = Deleted Messages
   mailbox_alias_old = Sent
   mailbox_alias_old2 = Sent
   mailbox_alias_old3 = Trash
}

This setup worked fine with my single server. Then I enabled replication 
- just on the primary. Dsync went to work (it seemed to take forever for 
the initial sync but that's what happens with large mailboxes and slow 
internet connections).

The problem came up with certain subfolders. And I believe it only 
happens with subfolders that have spaces in their names. I had two 
user's mailboxes (under Sent), one of which had a "Sent Messages" 
symlink alias for "Sent", that started generating tens or hundreds of 
duplicates during sync. Fortunately those subfolders only had a few 
mails in them. But I had trees looking like:

[...] (below is under /var/mail/domain/user/sdbox/mailboxes/Sent/)
Proposal 
Requests-temp-c6e003375e64a961c93d00009db5accb-temp-1-temp-f80b1a00ce9aa961a86-temp-2
Proposal 
Requests-temp-c6e003375e64a961c93d00009db5accb-temp-1-temp-f80b1a00ce9aa961a86-temp-3
Proposal 
Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-023fa4271c9ca9611ade0400b88bfabe
Proposal 
Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-023fa4271c9ca9611ad-temp-1
Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-1
Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-2
Proposal 
Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-2-temp-1-temp-1
Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-3
Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-4
Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-5
Proposal 
Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-e2aa0f35c99ba961356500009db5accb
[...]

I kept stopping, cleaning up the folders, and re-starting - and they 
kept regenerating. I tried renaming the folders to eliminate the spaces 
and I think that helped in one case - for the others I just moved the 
folders outside of the mail area completely to let the sync finish.

Now that it's been stable for a day or two - I enabled sync in the other 
direction. And after setting *all* the required parameters instead of 
just most of them...it's working. But...I'm nervous about moving the 
problem folders back over. I will say, if it makes any difference, my 
primary server *was* running version 2.3.13 and I just updated it to 
2.3.17. The remote is also 2.3.17.

--
Daniel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dovecot.org/pipermail/dovecot/attachments/20211203/cae5f992/attachment.htm>