[Dovecot] Performance with 200k messages in Maildir

[Thomas Harold]

On 7/19/2011 5:54 AM, Ricardo Branco wrote:
> If you have 200k all within one folder progs like TB will have issues
> loading it all up and may hang when you try to do moves/deletes etc, not
> sure if mutt stores a local cache of headers, thats the biggest worry.
> Biggest single folder ive seen at our office had 60k messages, it loads
> slowly on a cold cache in TB.
> Biggest mailbox has over 350k, my mailbox is over 250k, ofcourse thats
> across several folders.
> Mailserver is on VMware server (local drives), datastore (with maildirs)
> is separate NFS server on 11x2TB SATA R6 array (has other SAS disks for
> other things).
> If you have it spread out in different folders then it wont be so bad.
>

60k in a single folder is about the upper limit for TBird (TBird v2 was 
actually better suited for this).  But drag-n-drop breaks if you try to 
do more then 3-5k messages at a time.  When a mailbox gets over 30-50k 
messages, I archive some of them off to a sub-folder in Thunderbird. 
One of my TBird mailboxes is about 880,000 messages, almost 6GB of 
email, spread across dozens of directories.

Assuming MailDir storage, the bigger issue will be (a) how well the 
filesystem handles tens of thousands of files in a single folder (b) the 
physical disks / speed / number of spindles (c) how busy the CPU is on 
the server and maybe (d) the amount of server RAM that can be used as 
cache/buffer.  Ext3 is probably fine as long as directory indexing is 
turned on, but ext4 might be better (or something else that deals well 
with lots of small files).

The other side is how fast the disks are on the local client.  An SSD 
drive or 10k RPM drive on the local desktop helps a lot when you get up 
into the larger mailboxes.