[Dovecot] repeating dsync - questions

[Ken A]

It's about 300GB of mboxes (I don't know how many messages) and dsync
took about 3 hours from scratch in the destination. But, when I sync
more frequently, it's much quicker, and depends on the number of
changes. With no changes it runs in about 15 min.
Ken A.


On 4/5/2013 1:54 PM, Bob Gustafson wrote:
> I tried that a week or so ago, with a 'dsync -R backup', but got the
> funny named directories, so I read more and am trying the 'dsync
> mirror' which doesn't require the -R.
> 
> How long does the sync step take for you? (Normalize to # of
> messages..)
> 
> Bob G
> 
> On Apr 5, 2013, at 13:48, Ken A <ka at pacific.net> wrote:
> 
>> I've found that when going in one direction, using "backup -R"
>> rather than mirror, works better. I'm going from mbox to sdbox, but
>> doing roughly the same thing you are, rsync and then dsync.
>> 
>> Ken A.
>> 
>> On 4/5/2013 10:40 AM, Bob Gustafson wrote:
>>> 
>>> I'm moving/converting email from one system to another
>>> 
>>> The source system is: Dual core x86_64 6GB memory 180 GB raid1
>>> disks ext4 Fedora 9 Dovecot 1.0.15 Maildir format
>>> 
>>> The destination system is: Dual core x86_64 2GB memory 1TB raid1
>>> disks ext4 Fedora 18 Dovecot 2.1.15 sdbox format
>>> 
>>> I am moving mail in a series of steps:
>>> 
>>> 0) cd ( to mail user home directory ) mkdir Maildir
>>> 
>>> 1) # rsync mail from source system (hoho4) to current (hoho0) 
>>> time rsync -arv --times hoho4:/home/bobgus/Maildir/
>>> /home/bobgus/Maildir/
>>> 
>>> This step takes about 37 minutes the first time
>>> 
>>> 2) # dsync mirror/convert to sdbox format on current system time
>>> dsync mirror maildir:~/Maildir
>>> 
>>> This step takes about 858 minutes (!!)
>>> 
>>> Looking at -D messages indicates that dsync is deciding between 
>>> duplicates much of the time.
>>> 
>>> Looking at the results in a mail browser (Evolution), it seems
>>> fine, although the latest mails are not there.
>>> 
>>> 3) # pick up more current mail - use same script as in 1) time
>>> rsync -arv --times hoho4:/home/bobgus/Maildir/
>>> /home/bobgus/Maildir/
>>> 
>>> This step takes about 5 minutes although this varies depending on
>>> the amount of new mail.
>>> 
>>> 4) # repeat the 2) dsync step time dsync mirror
>>> maildir:~/Maildir
>>> 
>>> This is where things look peculiar. When I look at the directory
>>> of sdbox/mailboxes, I see duplicated directories
>>> 
>>> [bobgus at hoho0 sdbox]$ cd mailboxes [bobgus at hoho0 mailboxes]$ ls 
>>> Apple Mail To Do Apple Mail To
>>> Do_2a47983780615e5179600000ba55d82c Deleted Messages Deleted
>>> Messages_2847983780615e5179600000ba55d82c Drafts 
>>> Drafts_2447983780615e5179600000ba55d82c Important 
>>> Important_2947983780615e5179600000ba55d82c INBOX 
>>> INBOX_1547983780615e5179600000ba55d82c Sent 
>>> Sent_2547983780615e5179600000ba55d82c Trash 
>>> Trash_2747983780615e5179600000ba55d82c
>>> 
>>> The 2nd dsync step has not completed yet. I'm wondering whether
>>> the extra directory will be magically moved into the older
>>> directory.
>>> 
>>> (This is unlikely because the file names are duplicated in the
>>> new directory)
>>> 
>>> -------
>>> 
>>> Is this expected behavior? Are there command changes I can make
>>> to speed up the process? Eliminate the duplicate directories?
>>> 
>>> 
>>> 
>>> 
>> 
>> -- Ken Anderson Pacific Internet - http://www.pacific.net
> 

-- 
Ken Anderson
Pacific Internet - http://www.pacific.net