<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<font face="Helvetica, Arial, sans-serif">please appreciate that i
am trying to help ....<br>
<br>
after much testing i can now reporduce the replication issues at
hand<br>
<br>
I am running on freebsd 12 & 13 stable (both test and
production servers)<br>
<br>
sdram drives etc ...<br>
<br>
Basically replication works fine until reaching a folder quantity
of ~ 256 or more<br>
<br>
to reproduce using doveadm i created folders like <br>
<br>
INBOX/folder-0<br>
INBOX/folder-1<br>
</font><font face="Helvetica, Arial, sans-serif">INBOX/folder-2</font><br>
<font face="Helvetica, Arial, sans-serif"><font face="Helvetica,
Arial, sans-serif">INBOX/folder-3 <br>
and so forth </font>......<br>
<br>
I created 200 folders and they replicated ok on both servers<br>
<br>
I created another 200 (400 total) and the replicator got stuck and
would not update the mbox on the alternate server anymore and is
still updating 4 days later ?<br>
<br>
basically replicator goes so far and either hangs or more likely
bails on an error that is not reported to the debug reporting ?<br>
<br>
however dsync will sync the two servers but only when run manually
(ie all the folders will sync)<br>
<br>
I have two test servers avaliable if you need any kind of access -
again here to help.<br>
<br>
[07:28:42] mail18.scom.ca [root:0] ~<br>
# sync.status<br>
Queued 'sync' requests
0
<br>
Queued 'high' requests
6
<br>
Queued 'low' requests
0
<br>
Queued 'failed' requests
0
<br>
Queued 'full resync' requests
0
<br>
Waiting 'failed' requests
0
<br>
Total number of known users
255
<br>
<br>
username type
status <br>
<a class="moz-txt-link-abbreviated" href="mailto:paul@scom.ca">paul@scom.ca</a> normal Waiting for dsync to
finish<br>
<a class="moz-txt-link-abbreviated" href="mailto:keith@elirpa.com">keith@elirpa.com</a> incremental Waiting for dsync to
finish<br>
<a class="moz-txt-link-abbreviated" href="mailto:ed.hanna@dssmgmt.com">ed.hanna@dssmgmt.com</a> incremental Waiting for dsync to
finish<br>
<a class="moz-txt-link-abbreviated" href="mailto:ed@scom.ca">ed@scom.ca</a> incremental Waiting for dsync to
finish<br>
<a class="moz-txt-link-abbreviated" href="mailto:nick@elirpa.com">nick@elirpa.com</a> incremental Waiting for dsync to
finish<br>
<a class="moz-txt-link-abbreviated" href="mailto:paul@paulkudla.net">paul@paulkudla.net</a> incremental Waiting for dsync to
finish<br>
<br>
<br>
i have been going through the c code and it seems the replication
gets requested ok<br>
<br>
replicator.db does get updated ok with the replicated request for
the mbox in question.<br>
<br>
however i am still looking for the actual replicator function in
the lib's that do the actual replication requests<br>
<br>
the number of folders & subfolders is defanately the issue -
not the mbox pyhsical size as thought origionally.<br>
<br>
if someone can point me in the right direction, it seems either
the replicator is not picking up on the number of folders to
replicat properly or it has a hard set limit like 256 / 512 /
65535 etc and stops the replication request thereafter.<br>
<br>
I am mainly a machine code programmer from the 80's and have
concentrated on python as of late, 'c' i am starting to go through
just to give you a background on my talents.<br>
<br>
It took 2 months to finger this out.<br>
<br>
this issue also seems to be indirectly causing the duplicate
messages supression not to work as well.<br>
<br>
python programming to reproduce issue (loops are for last run
started @ 200 - fyi) :<br>
<br>
# cat mbox.gen<br>
#!/usr/local/bin/python2<br>
<br>
import os,sys<br>
<br>
from lib import *<br>
<br>
<br>
user = '<a class="moz-txt-link-abbreviated" href="mailto:paul@paulkudla.net">paul@paulkudla.net</a>'<br>
<br>
"""<br>
for count in range (0,600) :<br>
box = 'INBOX/folder-%s' %count<br>
print count<br>
command = '/usr/local/bin/doveadm mailbox create -s -u %s
%s' %(user,box)<br>
print command<br>
a = commands.getoutput(command)<br>
print a<br>
"""<br>
<br>
for count in range (0,600) :<br>
box = 'INBOX/folder-0/sub-%' %count<br>
print count<br>
command = '/usr/local/bin/doveadm mailbox create -s -u %s
%s' %(user,box)<br>
print command<br>
a = commands.getoutput(command)<br>
print a<br>
<br>
<br>
<br>
#sys.exit()<br>
<br>
<br>
</font>
<div class="moz-signature"><font color="blue">
<meta http-equiv="content-type" content="text/html;
charset=UTF-8">
<title></title>
</font></div>
</body>
</html>