I'm using dsync to synchronize emails on a laptop where wifi connectivity sometimes fails in the middle of a sync. I have a shell script that runs dsync, and here is one line of it including the output of dsync:
- dsync -f -m realmail mirror /home/paulproteus/projects/ssh-attach/run ssh rose.makesad.us dsync dsync-local(paulproteus): Error: dbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Unexpectedly lost uid=337460 dsync-local(paulproteus): Error: msg guid lookup failed: Internal error occurred. Refer to server log for more information. [2012-02-02 11:02:12] dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Rebuilding index dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 130608.broken dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 66159.broken dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 336269.broken dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 dsync-remote(paulproteus): Error: proxy server timed out dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 dsync-local(paulproteus): Warning: Transaction log file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/dovecot.index.log was locked for 1528 seconds
That seemed problematic, but not dangerous.
Then I ran a fresh sync, and found 3000 (of 60,000) messages had been deleted and expunged from the "realmail" box.
I'm guessing this is some bad interaction with sdbox and partial file downloads?
I haven't read the code for this, but I would guess the dsync process isn't being atomic about file transfers, so it is leaving half-completed transfers in place, which results in corrupt files when they're next examined.
dovecot -n output:
# 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-2-amd64 x86_64 Debian wheezy/sid mail_location = sdbox:~/Maildir/dbox passdb { driver = pam } protocols = " imap" ssl_cert =
-- Asheesh.