[Dovecot] Problems restoring from dump
Hello all,
First, the vital stats: FreeBSD 6.1-STABLE, dovecot 1.0.rc7 from ports.
I have run into a vexing little problem that has brought my deployment of a shiny new dovecot server to a screeching halt. Using 'periodic' I've configured regular backups to occur using 'dump'. I have a monthly level 0, a weekly level 4, and a daily level 8. These dumps take advantage of snapshots to ensure they are taking a backup of a consistent filesystem. As part of the beta testing, I decided to delete my maildir and attempt to recover from the dumps.
Recovery went well, as dump/restore is a wonderful piece of software. I did a quick check at the file level to be sure my mailbox looked like it had when I took the last level 8 dump, and everything looked good. The problem came when I started up Thunderbird to take a look at my mail. When I selected a message, say one that said "knockout daily run output", I'd be reading a different message entirely, like a recent email from my boss! Most of the messages were like this - hopelessly scrambled! I tried just deleting the account in Thunderbird and connecting again, but that didn't change a thing.
What happened!? Dump has faithfully restored the files, including the dovecot indexes... so why are my messages scrambled?
Thanks for your help, Paul
Paul Lathrop wrote:
Hello all,
First, the vital stats: FreeBSD 6.1-STABLE, dovecot 1.0.rc7 from ports.
I have run into a vexing little problem that has brought my deployment of a shiny new dovecot server to a screeching halt. Using 'periodic' I've configured regular backups to occur using 'dump'. I have a monthly level 0, a weekly level 4, and a daily level 8. These dumps take advantage of snapshots to ensure they are taking a backup of a consistent filesystem. As part of the beta testing, I decided to delete my maildir and attempt to recover from the dumps.
Recovery went well, as dump/restore is a wonderful piece of software. I did a quick check at the file level to be sure my mailbox looked like it had when I took the last level 8 dump, and everything looked good. The problem came when I started up Thunderbird to take a look at my mail. When I selected a message, say one that said "knockout daily run output", I'd be reading a different message entirely, like a recent email from my boss! Most of the messages were like this - hopelessly scrambled! I tried just deleting the account in Thunderbird and connecting again, but that didn't change a thing.
What happened!? Dump has faithfully restored the files, including the dovecot indexes... so why are my messages scrambled?
Thanks for your help, Paul
Try wiping out the indexes... It may be a situation where the in-memory index did not get refreshed from the file index for some reason...
Leeman
On Mon, Sep 18, 2006 at 01:12:50PM -0700, Paul Lathrop wrote:
Hello all,
First, the vital stats: FreeBSD 6.1-STABLE, dovecot 1.0.rc7 from ports.
I have run into a vexing little problem that has brought my deployment of a shiny new dovecot server to a screeching halt. Using 'periodic' I've configured regular backups to occur using 'dump'. I have a monthly level 0, a weekly level 4, and a daily level 8. These dumps take advantage of snapshots to ensure they are taking a backup of a consistent filesystem. As part of the beta testing, I decided to delete my maildir and attempt to recover from the dumps.
Recovery went well, as dump/restore is a wonderful piece of software. I did a quick check at the file level to be sure my mailbox looked like it had when I took the last level 8 dump, and everything looked good. The problem came when I started up Thunderbird to take a look at my mail. When I selected a message, say one that said "knockout daily run output", I'd be reading a different message entirely, like a recent email from my boss! Most of the messages were like this - hopelessly scrambled! I tried just deleting the account in Thunderbird and connecting again, but that didn't change a thing.
Are the subject lines/body scrambled in any other clients? try mutt for instance.
Are the headers mixed up aswell?
What happened!? Dump has faithfully restored the files, including the dovecot indexes... so why are my messages scrambled?
Thanks for your help, Paul
Mark
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wed, 20 Sep 2006, Paul Lathrop wrote:
Mark Adams wrote:
Are the subject lines/body scrambled in any other clients? try mutt for instance.
Are the headers mixed up aswell?
Mark,
Strangely enough, they aren't! What could be happening? Does this mean the problem is with Thunderbird?
Hmm, I had such a problem once with both Mozilla and Thunderbird, but you said you've purged the local copy. BTW: There are two files per mailbox, a *.msf and one without extension, you need to delete both, the msf is the index, but also stores the settings, like download status.
The problem was that the user had more than 2GB mail in INBOX, what let the local storage (mbox) of both applications crash silently. When restarted the complete file was mixed up, some mails OK, some headers only, some header and body mixed up. This can happen anytime, I've noted, when the sync process is interrupted very heavily, e.g. when you terminate mozilla during sync. I later tested this with a smaller mailbox as well.
The solution was:
a) move some mails to lower the INBOX below 2GB manually in the Maildir, and b) remove the INBOX cache on the client.
Bye.
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (GNU/Linux)
iQEVAwUBRRKSGC9SORjhbDpvAQJAaAf/TwGIjnPP0Ca4m+4n19T+BFlzimnLu5Il GYlc3a5tkWjXHLytnpX9vPwrIYEGRdwDvmsVhyJeC/C0V94050erfdgyZo1TAVQD hcn/+QhkHbtb6E0UVdFq5T7puQ+UoFTYlZeVnOT4b5zbIFobCBA1WTbv6PCG/OFY i6S2LjupyMjx5o7wuzFsb466PhSTT7ZBTJ6xl4KwQ6NuXht1yLiI7BpmO0bF1XB8 61Tv/9xakDrR5p0hWnmDbsGcsRpcfodqFXUhLLCF4Y+fbWag0lbchB0APjH+ndCz LkNJoLN4B5k0Nr4FQX/DnDtI3EGQg99XW0O+HfEm9ZSYwdXpteSsFA== =NCG+ -----END PGP SIGNATURE-----
participants (4)
-
Leeman Strout
-
Mark Adams
-
Paul Lathrop
-
Steffen Kaiser