[Dovecot] problems when using dsync between servers

[Yann Dupont]

Hello,
I'm trying to migrate from maildir++ to mdbox. The maildir++ is a 
leftover of our previous setup (was using courier)

We have 3 virtuals servers for our +70000 accounts.

Now, I'd like to use mdbox, so I'm trying using dsync for that. I setup 
a new virtual server, configured for mdbox.

1st, I tried a basic setup: I established a file with all my accounts 
(taken from LDAP) and then I use this script :

FILENAME=$1

while read LINE
do

dsync -u $LINE  backup $LINE at my_new_server

done < $FILENAME


But it's quite slow and dsync hangs.

So I tried another approach using multi threaded python.

Much faster (using all the cores), but dsync still hangs on some 
accounts. After a certain time dsync exit with an error like that :

  Panic: file dsync-worker-local.c: line 1688 
(local_worker_save_msg_continue): assertion failed: (ret == -1)
dsync-remote(E072184L): Error: Raw backtrace: 
/usr/lib/dovecot/libdovecot.so.0(+0x3fd8a) [0x7f34fc577d8a] -> 
/usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) 
[0x7f34fc577e72] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) 
[0x7f34fc55119f] -> dsync() [0x40ff0b] -> dsync() [0x410142] -> 
dsync(dsync_worker_msg_save+0x99) [0x40f639] -> dsync() [0x40e24a] -> 
dsync() [0x40d8ef] -> dsync() [0x40dbfc] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x48) [0x7f34fc5830d8] 
-> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) 
[0x7f34fc58403f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) 
[0x7f34fc583068] -> 
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f34fc5713a3] -> dsync(main+0x9e1) [0x4078b1] -> 
/lib/libc.so.6(__libc_start_main+0xfd) [0x7f34fbde9c4d] -> dsync() 
[0x406cf9]



This is on a RUNNNING server.

My aim was to make some pass on the running server, then stop it and 
make a last pass to finish the process.

Is dsync supposed to run only on a stopped server ?

Is my approach totally wrong ?



-- 
Yann Dupont - Service IRTS, DSI Université de Nantes
Tel : 02.53.48.49.20 - Mail/Jabber : Yann.Dupont at univ-nantes.fr