[Dovecot] dsync on large mailbox "fails"

[Jeff Kletsky]

On 04/24/2012 10:08 AM, Timo Sirainen wrote:
>
>>>> 25934 jeff          1 103    0   135M   102M CPU2    2 128:47 100.00% doveadm
>>> Looks like it could be in an infinite loop. To find out where:
>>>
> That's a lot of getdirentries() calls in there, repeating all the time. So I don't think it's an infinite loop, just a lot of slowness repeating the same work over and over again. See if it helps if you add:
>
> maildir_very_dirty_syncs = yes


I hope I understood correctly when I tried

$ dsync -Dv -o maildir_very_dirty_syncs=yes backup maildir:~/Maildir

into fresh directories (moving the previous Maildir aside).

I didn't see any significant change (and it did not complete in several 
hours) on the usual UFS directory, and have also tried ZFS to see if it 
might be a filesystem issue. That has not completed either after over 
nine  hours.

 From what I can see, I am guessing that it is "hanging" in index 
creation. Below, "mail" is the source directory, and "Maildir" is the 
destination of the sync. This is about nine hours of run time.

[jeff at mail ~]$ ls -Al mail/.imap/cron_2010-07-10
total 9
-rw-------  1 jeff  jeff  4910320 Feb 22  2011 dovecot.index
-rw-------  1 jeff  jeff  3600384 Apr 25 01:18 dovecot.index.cache
-rw-------  1 jeff  jeff       56 Feb 22  2011 dovecot.index.log
[jeff at mail ~]$ ls -Al Maildir/.cron_2010-07-10
total 26
drwx------  2 jeff  jeff   102291 Apr 25 01:17 cur
-rw-------  1 jeff  jeff       36 Apr 25 01:18 dovecot-keywords
-rw-------  1 jeff  jeff  4122971 Apr 25 01:17 dovecot-uidlist
-rw-------  1 jeff  jeff       24 Apr 25 07:19 dovecot-uidlist.lock
-rw-------  1 jeff  jeff  1637008 Apr 25 07:03 dovecot.index
-rw-------  1 jeff  jeff  9442304 Apr 25 07:19 dovecot.index.cache
-rw-------  1 jeff  jeff    87984 Apr 25 07:19 dovecot.index.log
-rw-------  1 jeff  jeff  1048600 Apr 25 07:03 dovecot.index.log.2
-rw-------  1 jeff  jeff        0 Apr 24 22:12 maildirfolder
drwx------  2 jeff  jeff        2 Apr 24 22:12 new
drwx------  2 jeff  jeff        2 Apr 25 01:17 tmp