[Dovecot] repeating dsync - questions

[Bob Gustafson]

Something must be wrong with my setup. It took 14+ hours for the first
dsync pass and it hasn't finished yet on the rerun to pick up the latest
mail (around 12+ hours). I have about 9 GB of mail! The destination
system is not fast, but..

Maybe I will wait for the 2.2 release..

Thanks for your response.

Bob G

On Fri, 2013-04-05 at 22:45 -0500, Ken A wrote:
> It's about 300GB of mboxes (I don't know how many messages) and dsync
> took about 3 hours from scratch in the destination. But, when I sync
> more frequently, it's much quicker, and depends on the number of
> changes. With no changes it runs in about 15 min.
> Ken A.
> 
> 
> On 4/5/2013 1:54 PM, Bob Gustafson wrote:
> > I tried that a week or so ago, with a 'dsync -R backup', but got the
> > funny named directories, so I read more and am trying the 'dsync
> > mirror' which doesn't require the -R.
> > 
> > How long does the sync step take for you? (Normalize to # of
> > messages..)
> > 
> > Bob G
> > 
> > On Apr 5, 2013, at 13:48, Ken A <ka at pacific.net> wrote:
> > 
> >> I've found that when going in one direction, using "backup -R"
> >> rather than mirror, works better. I'm going from mbox to sdbox, but
> >> doing roughly the same thing you are, rsync and then dsync.
> >> 
> >> Ken A.
> >> 
> >> On 4/5/2013 10:40 AM, Bob Gustafson wrote:
> >>> 
> >>> I'm moving/converting email from one system to another
> >>> 
> >>> The source system is: Dual core x86_64 6GB memory 180 GB raid1
> >>> disks ext4 Fedora 9 Dovecot 1.0.15 Maildir format
> >>> 
> >>> The destination system is: Dual core x86_64 2GB memory 1TB raid1
> >>> disks ext4 Fedora 18 Dovecot 2.1.15 sdbox format
> >>> 
> >>> I am moving mail in a series of steps:
> >>> 
> >>> 0) cd ( to mail user home directory ) mkdir Maildir
> >>> 
> >>> 1) # rsync mail from source system (hoho4) to current (hoho0) 
> >>> time rsync -arv --times hoho4:/home/bobgus/Maildir/
> >>> /home/bobgus/Maildir/
> >>> 
> >>> This step takes about 37 minutes the first time
> >>> 
> >>> 2) # dsync mirror/convert to sdbox format on current system time
> >>> dsync mirror maildir:~/Maildir
> >>> 
> >>> This step takes about 858 minutes (!!)
> >>> 
> >>> Looking at -D messages indicates that dsync is deciding between 
> >>> duplicates much of the time.
> >>> 
> >>> Looking at the results in a mail browser (Evolution), it seems
> >>> fine, although the latest mails are not there.
> >>> 
> >>> 3) # pick up more current mail - use same script as in 1) time
> >>> rsync -arv --times hoho4:/home/bobgus/Maildir/
> >>> /home/bobgus/Maildir/
> >>> 
> >>> This step takes about 5 minutes although this varies depending on
> >>> the amount of new mail.
> >>> 
> >>> 4) # repeat the 2) dsync step time dsync mirror
> >>> maildir:~/Maildir
> >>> 
> >>> This is where things look peculiar. When I look at the directory
> >>> of sdbox/mailboxes, I see duplicated directories
> >>> 
> >>> [bobgus at hoho0 sdbox]$ cd mailboxes [bobgus at hoho0 mailboxes]$ ls 
> >>> Apple Mail To Do Apple Mail To
> >>> Do_2a47983780615e5179600000ba55d82c Deleted Messages Deleted
> >>> Messages_2847983780615e5179600000ba55d82c Drafts 
> >>> Drafts_2447983780615e5179600000ba55d82c Important 
> >>> Important_2947983780615e5179600000ba55d82c INBOX 
> >>> INBOX_1547983780615e5179600000ba55d82c Sent 
> >>> Sent_2547983780615e5179600000ba55d82c Trash 
> >>> Trash_2747983780615e5179600000ba55d82c
> >>> 
> >>> The 2nd dsync step has not completed yet. I'm wondering whether
> >>> the extra directory will be magically moved into the older
> >>> directory.
> >>> 
> >>> (This is unlikely because the file names are duplicated in the
> >>> new directory)
> >>> 
> >>> -------
> >>> 
> >>> Is this expected behavior? Are there command changes I can make
> >>> to speed up the process? Eliminate the duplicate directories?
> >>> 
> >>> 
> >>> 
> >>> 
> >> 
> >> -- Ken Anderson Pacific Internet - http://www.pacific.net
> > 
>