<html><head>
<style id="pgp_css" type="text/css"><!----></style><style id="css_styles" type="text/css"><!--blockquote.cite { margin-left: 5px; margin-right: 0px; padding-left: 10px; padding-right:0px; border-left: 1px solid #cccccc }
blockquote.cite2 {margin-left: 5px; margin-right: 0px; padding-left: 10px; padding-right:0px; border-left: 1px solid #cccccc; margin-top: 3px; padding-top: 0px; }
a img { border: 0px; }
li[style='text-align: center;'], li[style='text-align: center; '], li[style='text-align: right;'], li[style='text-align: right; '] { list-style-position: inside;}
body { font-family: Segoe UI; font-size: 12pt; }
.quote { margin-left: 1em; margin-right: 1em; border-left: 5px #ebebeb solid; padding-left: 0.3em; }--></style><style type="text/css"><!--#xe19ca2d1779e486 #x05dc9e08d2d7403 blockquote.cite2, #xe19ca2d1779e486 blockquote.cite2
{margin-left: 5px; margin-right: 0px; padding-left: 10px; padding-right: 0px; border-left: 1px solid rgb(204, 204, 204); margin-top: 3px; padding-top: 0px;}
#xe19ca2d1779e486 #x05dc9e08d2d7403, #xe19ca2d1779e486
{font-family: "Segoe UI"; font-size: 12pt;}
--></style>
</head>
<body><div>And...one more.</div><div><br /></div><div>I'm now seeing (again) messages like:</div><div><br /></div><div>Dec 3 14:29:14 cloud1 dovecot: doveadm(obfuscated)<334017><e3FHNjmaqmHBGAUAuIv6vg>: Error: Duplicate mailbox GUID bcb9ca36ae36aa617f0a00009db5accb for mailboxes INBOX/Sent Messages and INBOX/Sent - giving a new GUID fc30bd0a3a9aaa61c1180500b88bfabe to INBOX/Sent<br />Dec 3 14:38:59 cloud1 dovecot: doveadm(<span>obfuscated</span>)<334394><an5KIoOcqmE6GgUAuIv6vg>: Error: Duplicate mailbox GUID fc30bd0a3a9aaa61c1180500b88bfabe for mailboxes INBOX/Sent Messages and INBOX/Sent - giving a new GUID f4338038839caa613a1a0500b88bfabe to INBOX/Sent<br /></div>
<div><br /></div><div>Having one message for the initial sync I suppose is reasonable. A second...maybe? But I'm getting nervous I'm about to start seeing the endless temp folders again.</div><div id="signature_old">--<div>Daniel</div></div><div><br /></div>
<div>------ Original Message ------</div>
<div>From: "Daniel Miller" <<a href="mailto:dmiller@amfes.com">dmiller@amfes.com</a>></div>
<div>To: "Daniel Miller" <<a href="mailto:dmiller@amfes.com">dmiller@amfes.com</a>>; <a href="mailto:dovecot@dovecot.org">dovecot@dovecot.org</a></div>
<div>Sent: 12/3/2021 2:39:25 PM</div>
<div>Subject: Re: Replication weirdness</div><div><br /></div>
<div id="xe19ca2d1779e486"><blockquote cite="em2573799c-dba1-43b1-bb38-ad7aab004d2e@blackdragon" type="cite" class="cite2">
<div>Another item.</div><div><br /></div><div>Again, it may be a 2.3.13 issue and I'm now on 2.3.17. But...I had problem when using the "-N" parameter for dsync. So - I just have (had):</div><div><br /></div><div>replication_dsync_parameters = -d -l 30 -U -x INBOX/virtual -x INBOX/shared</div>
<div><br /></div><div>Now that things are working - I wanted to have my other namespaces sync as well. So I went to:</div><div><br /></div><div>replication_dsync_parameters = -d -l 30 -U -n INBOX -n INBOX/Archives -n INBOX/Lists -x INBOX/virtual -x INBOX/shared</div><div><br /></div><div>This appears to be working (the sync is just starting)...but I'm seeing lock errors in the logs such as:</div><div>Dec 3 14:34:24 bubba dovecot: doveadm(<a href="mailto:dmiller@amfes.com">dmiller@amfes.com</a>)<31785><TV+0LlGbqmEpfAAAnbWsyw>: Error: Couldn't lock /var/mail/amfes.com/dmiller/.dovecot-sync.lock: fcntl(/var/mail/amfes.com/dmiller/.dovecot-sync.lock, write-lock, F_SETLKW) locking failed: Timed out after 30 seconds (WRITE lock held by pid 31373)<br /></div><div><br /></div><div>Checking the pid in question I see it's actively syncing a folder in my mailbox. So I'm guessing, purely guessing, that by having multiple namespaces explicitly directed to sync Dovecot is trying to start a sync process for each of those namespaces - but all of them share a common lock and therefore only one operation is allowed at a time.</div><div><br /></div><div>Am I correct, and whether or not I am - how can I correct these errors? Do I dare try going back to just "-N"?</div><div><br /></div><div id="signature_old">--<div>Daniel</div></div><div><br /></div>
<div>------ Original Message ------</div>
<div>From: "Daniel Miller" <<a href="mailto:dmiller@amfes.com">dmiller@amfes.com</a>></div>
<div>To: <a href="mailto:dovecot@dovecot.org">dovecot@dovecot.org</a></div>
<div>Sent: 12/3/2021 2:16:28 PM</div>
<div>Subject: Replication weirdness</div><div><br /></div>
<div id="x05dc9e08d2d7403"><blockquote cite="emdfeed6a3-2f07-48db-a93b-ee88608e411c@blackdragon" type="cite" class="cite2">
<div>First, I have to say this. After configuring everything correctly - and that means *everything* correctly - Dovecot replication Just Works. I'm not sure how (yes I do - Timo & Co. Magic) - but it does. Real-time new sync is near instantaneous.</div><div><br /></div>
<div>Now the problem. Or the background for the problem. My primary server uses sdbox for primary storage, mdbox for archival storage, and fts-solr. I spun up a second server, using sdbox, mdbox, and fts-flatcurve. My namespaces are as defined below. As best I can tell (based on diff comparing two 'doveconf -n' outputs) my namespaces are the same on both servers.</div><div><br /></div><div>namespace archives {
</div><div> list = children
</div><div> location = mdbox:/var/mail/%d/%n/Archives/mdbox
</div><div> mailbox Unsorted {
</div><div> auto = no
</div><div> special_use = \Archive
</div><div> }
</div><div> prefix = INBOX/Archives/
</div><div> separator = /
</div><div> subscriptions = no
</div><div> type = private
</div><div>}
</div><div>namespace inbox {
</div><div> alias_for =
</div><div> hidden = no
</div><div> inbox = yes
</div><div> list = yes
</div><div> location =
</div><div> mailbox "Deleted Messages" {
</div><div> auto = no
</div><div> autoexpunge = 30 days
</div><div> special_use = \Trash
</div><div> }
</div><div> mailbox Drafts {
</div><div> auto = subscribe
</div><div> special_use = \Drafts
</div><div> }
</div><div> mailbox Sent {
</div><div> auto = subscribe
</div><div> special_use = \Sent
</div><div> }
</div><div> mailbox Trash {
</div><div> auto = subscribe
</div><div> autoexpunge = 30 days
</div><div> special_use = \Trash
</div><div> }
</div><div> prefix = INBOX/
</div><div> separator = /
</div><div> subscriptions = no
</div><div> type = private
</div><div>}
</div><div>namespace lists {
</div><div> list = children
</div><div> location = mdbox:/var/mail/%d/%n/Lists/mdbox
</div><div> prefix = INBOX/Lists/
</div><div> separator = /
</div><div> subscriptions = no
</div><div> type = private
</div><div>}
</div><div>namespace subscriptions {
</div><div> hidden = yes
</div><div> list = no
</div><div> location =
</div><div> prefix =
</div><div> separator = /
</div><div> subscriptions = yes
</div><div> type = private
</div><div>}
</div><div>namespace usershares {
</div><div> list = yes
</div><div> location = sdbox:/var/mail/%%d/%%n/sdbox:NO-NOSELECT
</div><div> prefix = INBOX/shared/%%d/%%n/
</div><div> separator = /
</div><div> subscriptions = no
</div><div> type = shared
</div><div>}
</div><div>namespace virtual {
</div><div> list = children
</div><div> location = virtual:/var/mail/%d/%n/virtual
</div><div> mailbox Flagged {
</div><div> comment = All my flagged messages
</div><div> special_use = \Flagged
</div><div> }
</div><div> prefix = INBOX/virtual/
</div><div> separator = /
</div><div> subscriptions = no
</div><div>}
</div><div><br /></div><div>I also have:</div><div>plugin {</div><div> mailbox_alias_new = Sent Messages
</div><div> mailbox_alias_new2 = Sent Items
</div><div> mailbox_alias_new3 = Deleted Messages
</div><div> mailbox_alias_old = Sent
</div><div> mailbox_alias_old2 = Sent
</div><div> mailbox_alias_old3 = Trash</div><div>}</div><div><br /></div><div>This setup worked fine with my single server. Then I enabled replication - just on the primary. Dsync went to work (it seemed to take forever for the initial sync but that's what happens with large mailboxes and slow internet connections).</div><div><br /></div><div>The problem came up with certain subfolders. And I believe it only happens with subfolders that have spaces in their names. I had two user's mailboxes (under Sent), one of which had a "Sent Messages" symlink alias for "Sent", that started generating tens or hundreds of duplicates during sync. Fortunately those subfolders only had a few mails in them. But I had trees looking like:</div><div><br /></div><div>[...] (below is under /var/mail/domain/user/sdbox/mailboxes/Sent/)</div><div>Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-1-temp-f80b1a00ce9aa961a86-temp-2
</div><div>Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-1-temp-f80b1a00ce9aa961a86-temp-3
</div><div>Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-023fa4271c9ca9611ade0400b88bfabe
</div><div>Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-023fa4271c9ca9611ad-temp-1
</div><div>Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-1
</div><div>Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-2
</div><div>Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-2-temp-1-temp-1
</div><div>Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-3
</div><div>Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-4
</div><div>Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-5
</div><div>Proposal Requests-temp-c6e003375e64a961c93d00009db5accb-temp-2-temp-e2aa0f35c99ba961356500009db5accb
</div><div>[...]</div><div><br /></div><div>I kept stopping, cleaning up the folders, and re-starting - and they kept regenerating. I tried renaming the folders to eliminate the spaces and I think that helped in one case - for the others I just moved the folders outside of the mail area completely to let the sync finish.</div><div><br /></div><div>Now that it's been stable for a day or two - I enabled sync in the other direction. And after setting *all* the required parameters instead of just most of them...it's working. But...I'm nervous about moving the problem folders back over. I will say, if it makes any difference, my primary server *was* running version 2.3.13 and I just updated it to 2.3.17. The remote is also 2.3.17.</div><div><br /></div><div id="signature_old">--<div>Daniel</div></div>
</blockquote></div>
</blockquote></div>
</body></html>