<!doctype html>
<html>
<head>
<meta charset="UTF-8">
</head>
<body>
<div>
<br>
</div>
<blockquote type="cite">
<div>
On 15/07/2019 09:22 Aki Tuomi via dovecot <dovecot@dovecot.org> wrote:
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<blockquote type="cite">
<div>
On 15/07/2019 09:00 Mike via dovecot <
<a href="mailto:dovecot@dovecot.org">dovecot@dovecot.org</a>> wrote:
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
Hello,
</div>
<div>
<br>
</div>
<div>
I was forced to do an emergency server migration with about 3000
</div>
<div>
mailboxes and I botched it somewhat. Hoping to find some help here.
</div>
<div>
<br>
</div>
<div>
My symptoms are, users showing apparently deleted mail messages, some
</div>
<div>
replicated mail, and a time period of mail that is just 'missing'.
</div>
<div>
<br>
</div>
<div>
The source server is centos 6.10 with dovecot 2.2.5, and the destination
</div>
<div>
is ubuntu 16 / dovecot 2.2.22. User mailboxes are mdbox format and I did
</div>
<div>
an rsync from the old to the new. My screw up however was that my rsync
</div>
<div>
was not --delete files, and had to be run multiple times over the course
</div>
<div>
of a few weeks due to to bandwidth disparities and the sheer size of the
</div>
<div>
mail spool approaching 700GB (source server had 10mbps. Should have used
</div>
<div>
sneakernet instead). The bottom line however is that I wound up having
</div>
<div>
many extra m.* files in each user storage dir that were not actually
</div>
<div>
present on the source when the final, final sync was made and I went
</div>
<div>
live with the migration server. I know this is my fsckup but now I have
</div>
<div>
to deal with it.
</div>
<div>
<br>
</div>
<div>
I first tried to fix the symptoms with a force-resync on the boxes not
</div>
<div>
realizing the above, and that seemed to make things worse by only
</div>
<div>
showing new mail as of the server migration date. Ugh.
</div>
<div>
<br>
</div>
<div>
New mail is being delivered and this does appear in user mail boxes. But
</div>
<div>
as I said, there seems to be some users with a problem with old mail
</div>
<div>
presumably deleted on the prior server suddenly re-appearing (probbly
</div>
<div>
due to the extraneous m.* mdbox files present). A more pressing problem
</div>
<div>
is that there is simply a time period of mail that is in user mailboxes
</div>
<div>
(on the old server) from before the migration, which is not displaying
</div>
<div>
now for these users on the new server.
</div>
<div>
<br>
</div>
<div>
I have investigated this and hacked on "mdbox-recover.pl" somewhat so
</div>
<div>
that it iterates over all mdbox files for a user and I can extract ALL
</div>
<div>
messages, and see that, yes, the mail really is present in the mdbox
</div>
<div>
files, but pop/imap never shows it. This is puzzling to me. I have a
</div>
<div>
test user that when I check imap, it shows only 21 message in inbox and
</div>
<div>
1 in sent. But if I extract the messages from the mdbox files, then I
</div>
<div>
actually have 1117 messages.
</div>
<div>
<br>
</div>
<div>
I know I can't just rsync the source server again with the right
</div>
<div>
--delete flag, there has been lots of mail deliveries to the migrated
</div>
<div>
boxes and so I don't want to lose all of that and create yet more
</div>
<div>
problems. But I need to find some way to make these present mail
</div>
<div>
messages appear for my users again. Im thinking it's got to be a flag of
</div>
<div>
some kind since the data is there it's just ignored. Wondering what you
</div>
<div>
would reccomend?
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
Thanks.
</div>
<div>
<br>
</div>
<div>
Mike-
</div>
</blockquote>
<div>
<br>
</div>
<div>
Try with one mailbox first.
</div>
<div>
<br>
</div>
<div>
Delete dovecot.index.map files and run force-resync.
</div>
<div>
<br>
</div>
<div class="io-ox-signature">
<pre>---
Aki Tuomi</pre>
</div>
</blockquote>
<div>
<br>
</div>
<div>
Sorry I ment dovecot.map.index there
</div>
<div class="io-ox-signature">
<pre>---
Aki Tuomi</pre>
</div>
</body>
</html>