[Dovecot] dovecot Digest, Vol 89, Issue 25

[Alan Brown]

Timo Sirainen <tss at iki.fi> Wrote:

 > On Mon, 2010-09-06 at 14:26 +0100, Alan Brown wrote:
 > > Is there any way of enforcing a Maildir per-folder message limit in
 > > Dovecot?

 > No.

Would you consider it as a feature request?

 > > We're finding major performance hits once a threshold of 
files/directory
 > > is exceeded. This applies across all common filesystems with the
 > > threshold differing depending on the FS. GFS2 pretty much _STOPS_ 
for 10
 > > minutes when users put 65k+ messages in one folder...

 > mdbox would probably work nicely.  :)

Yes, it probably would, however I have several layers of resistance to 
get past in order to implement that (ie, up to 2 years to get approval 
to switch to dbox)

 > > I'm currently running 1.1.20 but looking to update (RHEL5 cluster
 > > installation)

 > With v1.2+ you could also try maildir_very_dirty_syncs=yes

That won't help much for initial directory opens, which is where most of 
the pain is. Past that point, the index of ~/Maildir/.folder/cur is 
mostly cached until you hit about 16,000 messages(files) on GFS2 and 
some people are insisting on 32k+ files in one folder with the odd few 
trying 65k or more (I'm surprised their clients don't break).

For info sake: we have Dovecot running on a pair of dedicated dual E5530 
(quad-core X86 1300MHZ FSB cpu) servers (RH cluster+GFS2) with 24Gb RAM 
apiece and ~2TB of 8Gb/s san-attached storage available.

There are only ~250 accounts but there's around 600Gb in the maildir 
folder areas and another 300Gb in a separate dedicated inbox area - and 
I believe there's another 1TB of stuff in client local folders we're 
trying to pull back to the servers in order to simplify backups.

Even with this kind of horsepower available, over-large folders can 
render the setup completely unresponsive for minutes at a time. I know 
this is a case of "work smarter, not harder", but I'm facing major 
delays until we're able to change the on-disk storage format.

Alan