I've been running continous dsync backups of our Maildirs for a few weeks now, with the destination dsync server using mdbox and SIS. The idea was that the destination server would act as a warm copy of all our active users data.
The active servers are using Maildir, and has:
$ df -h /usr/local/atmail/users/
Filesystem Size Used Avail Use% Mounted on
/dev/atmailusers 14T 12T 2.2T 85% /usr/local/atmail/users
$ df -hi /usr/local/atmail/users/
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/atmailusers 145M 113M 33M 78% /usr/local/atmail/usersvery little of this is compressed (zlib plugin enabled during christmas).
I'm surprised that the destination server is so large, was expecting zlib and mdbox and SIS would compress it down to much less than what we're seeing (12TB -> 5TB):
$ df -h /srv/mailbackup
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/mailbackupvg-mailbackuplv
5.7T 4.8T 882G 85% /srv/mailbackupLots and lots of the attachement storage is duplicated into identical files, instead of hard linked.
When running "doveadm purge -u $user", we're seeing lots of
Error: unlink(/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab) failed: No such file or directory"/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is missing, but there are 205 other copies of this file named /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with identical sha1sum.
Also we see corrupted indexes during the purge. This makes me quite uncertain if dsync is a workable backup solution.. or if we can trust mdboxes.
Also on the source side, during dsync, we see too many problems. Some samples:
Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX
Error: command BOX-LIST failed
Error: Worker server's mailbox iteration failed
Error: read() from worker server failed: EOF
Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name
Error: read() from proxy client failed: EOF
Error: Unexpected finish reply: 1 596fec275888dbd89f6d1f5356c22db6 3720 0 \dsync-expunged 0
Error: Unexpected reply from server: 1 12200572a70726fca946da6f9378dc03 3721 0 \dsync-expunged 0
Error: Failed to sync mailbox INBOX.INBOX.Gerda: Mailbox doesn't exist: INBOX/Gerda
Error: command BOX-LIST failed
Error: read() failed: Broken pipe
Panic: file dsync-worker-local.c: line 1678 (local_worker_save_msg_continue): assertion failed: (ret == -1)
Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x367703c680] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x35) [0x367703c765] -> /usr/lib64/dovecot/libdovecot.so.0 [0x367703bb93] -> /usr/bin/dsync [0x40f48d] -> /usr/bin/dsync [0x40f589] -> /usr/bin/dsync(dsync_worker_msg_save+0x8e) [0x40eb3e] -> /usr/bin/dsync [0x40d71a] -> /usr/bin/dsync [0x40cdbf] -> /usr/bin/dsync [0x40d105] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x48) [0x3677047278] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0xd5) [0x36770485c5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x2d) [0x367704720d] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x3677035a83] -> /usr/bin/dsync(main+0x71e) [0x406c4e] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3e3941d994] -> /usr/bin/dsync [0x406369]Do you have any idea for what our problems might be? Should we:
avoid SIS ?
avoid doing Maildir on one side and mdbox on the other?
try other dovecot version for dsync?
anything else?-jf
------------- destination server, running dovecot v2.0.14 -------- mail_attachment_dir = /srv/mailbackup/attachments mail_location = mdbox:~/mdbox mail_plugins = zlib mdbox_rotate_size = 5 M namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { driver = static } plugin { zlib_save = gz zlib_save_level = 9 } protocols = service auth-worker { user = $default_internal_user } service auth { unix_listener auth-userdb { mode = 0600 user = mailbackup } } ssl = no userdb { args = home=/srv/mailbackup/%256Hu/%d/%n driver = static } -------------/destination server --------
-jf