[Dovecot] Problem with huge IMAP Archive after Courier migration

[Stan Hoeppner]

On 1/2/2012 10:17 AM, Preacher wrote:
...
> So I forced to install the Debisn 7.0 packages with 2.0.15 and finally
> got the server running, I also restarted the whole machine to empty caches.
> But the problem I got was that in the huge folder hierarchy the
> downloaded headers in the individual folders disappeared, some folders
> showed a few very old messages, some none. Also some subfolders
> disappeared.
> I checked this with Outlook and Thunderbird. The difference was, that
> Thunderbird shows more messages (but not all) than Outlook in some
> folders, but also none in some others. Outlook brought up a message in
> some cases, that the connection timed out, although I set the timeout to
> 60s.
...
> Anyone a clue what's wrong here?

Absolutely.  What's wrong is a lack of planning, self education, and
patience on the part of the admin.

Dovecot gets its speed from its indexes.  How long do you think it takes
Dovecot to index 37GB of maildir messages, many thousands per directory,
hundreds of directories, millions of files total?  Until those indexes
are built you will not see a complete folder tree and all kinds of stuff
will be missing.

For your education:  Dovecot indexes every message and these indexes are
the key to its speed.  Normally indexing occurs during delivery when
using deliver or lmtp, so the index updates are small and incremental,
keeping performance high.  You tried to do this and expected Dovecot to
instantly process it all:

http://www.youtube.com/watch?v=THVz5aweqYU

If you don't know, that's a coal train car being dumped.  100 tons of
coal in a few seconds.  Visuals are always good teaching tools.  I think
this drives the point home rather well.

-- 
Stan