[Dovecot] problems when using dsync between servers
Hello, I'm trying to migrate from maildir++ to mdbox. The maildir++ is a leftover of our previous setup (was using courier)
We have 3 virtuals servers for our +70000 accounts.
Now, I'd like to use mdbox, so I'm trying using dsync for that. I setup a new virtual server, configured for mdbox.
1st, I tried a basic setup: I established a file with all my accounts (taken from LDAP) and then I use this script :
FILENAME=$1
while read LINE do
dsync -u $LINE backup $LINE@my_new_server
done < $FILENAME
But it's quite slow and dsync hangs.
So I tried another approach using multi threaded python.
Much faster (using all the cores), but dsync still hangs on some accounts. After a certain time dsync exit with an error like that :
Panic: file dsync-worker-local.c: line 1688 (local_worker_save_msg_continue): assertion failed: (ret == -1) dsync-remote(E072184L): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x3fd8a) [0x7f34fc577d8a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f34fc577e72] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f34fc55119f] -> dsync() [0x40ff0b] -> dsync() [0x410142] -> dsync(dsync_worker_msg_save+0x99) [0x40f639] -> dsync() [0x40e24a] -> dsync() [0x40d8ef] -> dsync() [0x40dbfc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x48) [0x7f34fc5830d8] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f34fc58403f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f34fc583068] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f34fc5713a3] -> dsync(main+0x9e1) [0x4078b1] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f34fbde9c4d] -> dsync() [0x406cf9]
This is on a RUNNNING server.
My aim was to make some pass on the running server, then stop it and make a last pass to finish the process.
Is dsync supposed to run only on a stopped server ?
Is my approach totally wrong ?
-- Yann Dupont - Service IRTS, DSI Université de Nantes Tel : 02.53.48.49.20 - Mail/Jabber : Yann.Dupont@univ-nantes.fr
On Wed, 2011-12-07 at 12:02 +0100, Yann Dupont wrote:
Much faster (using all the cores), but dsync still hangs on some accounts. After a certain time dsync exit with an error like that :
Panic: file dsync-worker-local.c: line 1688 (local_worker_save_msg_continue): assertion failed: (ret == -1)
Unfortunately there are still some bugs in dsync.. But you could work around this by checking if dsync returned failure, and if it did just retry the user.
This is on a RUNNNING server.
My aim was to make some pass on the running server, then stop it and make a last pass to finish the process.
That should work.
Is dsync supposed to run only on a stopped server ?
No.
Le 08/12/2011 06:15, Timo Sirainen a écrit :
On Wed, 2011-12-07 at 12:02 +0100, Yann Dupont wrote:
Much faster (using all the cores), but dsync still hangs on some accounts. After a certain time dsync exit with an error like that :
Panic: file dsync-worker-local.c: line 1688 (local_worker_save_msg_continue): assertion failed: (ret == -1)
Unfortunately there are still some bugs in dsync.. But you could work around this by checking if dsync returned failure, and if it did just retry the user.
ok will try this.
This is on a RUNNNING server.
My aim was to make some pass on the running server, then stop it and make a last pass to finish the process.
That should work.
Is dsync supposed to run only on a stopped server ?
No.
OK, thanks for your answer,
-- Yann Dupont - Service IRTS, DSI Université de Nantes Tel : 02.53.48.49.20 - Mail/Jabber : Yann.Dupont@univ-nantes.fr
participants (2)
-
Timo Sirainen
-
Yann Dupont