[Dovecot] Thunderbird message cache out of sync after repetitive rsyncs...

[Stan Hoeppner]

On 12/24/2013 11:02 AM, Charles Marcus wrote:
> Ok, hopefully there is a solution to this.
> 
> I've been experimenting with multiple rsyncs in preparation for pulling
> the trigger on the mail server switch, but have a problem that I really
> want to fox before doing so.
> 
> Apparently something causes Thunderbirds local message cache to get out
> of sync with dovecot after a sync.
> 
> Here is the series of commands I'm running:
> 
> 1. stop postfix, stop dovecot on new server
> 
> 2. rsync -rltgovDHP --delete --exclude-from 'excludes.txt'
> /path/to/vmail/example.com/ /var/vmail/example.com/
> 
> 3. chown vmail:vmail /var/vmail
> 
> 4. start dovecot, start postfix
> 
> ls -al /var/vmail/example.com/user/cur
> 
> shows all of the messages that should be there, and all perms are correct.
> 
> Go to my account that is pointed to this mail server/account, and none
> of the new messages show up. Also, some messages are still showing up
> that shouldn't.
> 
> I've tried compacting the folders, closing/relaunching Thunderbird, but
> nothing helps.
> 
> The only way to get them to show up is to go to the local Thunderbird
> cache for the account, and delete the files associated with the folder
> having the problem.
> 
> The problem is, ALL folders will have this problem, which means that
> everyone will need to delete ALL of their local cahced folders.
> 
> This will be a major support problem.
> 
> Anyone have any ideas?

The source of the problem is almost certainly out of sync Dovecot index
files between the old and new servers, and thus TBird.  After an rsync
copy of the mails the new server must create the indexes on-the-fly when
TBird connects, and the resulting new indexes are likely not identical
to the old server.  Thus TBird is seeing a different mailbox view.

TBird keeps its own indexes for all IMAP folders.  It has nothing little
or nothing to do with local cached copies of folders.  I don't use GLODA
and I don't cache locally, but I still have a .msf file for each Dovecot
IMAP folder, some of them multiple MBs in size.  These are strictly
indexes.  It's these local indexes not being in sync with your new
Dovecot server indexes that I'm pretty sure is the cause of your problem.

If the mailbox contents are identical before/after the copy, you might
try copying the indexes over from the old mail server, preserving
permissions, creation time, atime, etc.  If the server indexes are
identical before/after the rsync you should avoid this problem, assuming
everything else is identical, including server hostnames, IP addresses,
encryption key, etc, etc.  TBird tracks mailboxes by server name in
Account Settings after all.  If the server name changes that'll cause
TBird to create an alternate local folder hierarchy in the profile
directory.  And that'll wreak havoc on your indexes, mailbox view, etc.

-- 
Stan