[Dovecot] Migration questions...

[Richard Hobbs]

Seth Mattinen wrote:
> Richard Hobbs wrote:
>> Seth Mattinen wrote:
>>> Phillip Macey wrote:
>>>> On 14/05/2009 5:11 PM, Steffen Kaiser wrote:
>>>>> On Wed, 13 May 2009, Richard Hobbs wrote:
>>>>>> The main complaint we have from users is that their IMAP Inbox, with
>>>>>> 5000 emails in it takes ages to appear, and no amount of coaxing will
>>>>>> convince them to split their inbox into multiple folders.
>>>>> Oh, we serve Maildir via Dovecot IMAP and 5000 messages per folder are
>>>>> a wimp. Problems start if the user:
>>>> We are having some performancec issues on our server at the moment - all
>>>> I can put it down to is the large size of some maildirs. Eg. `ls -ld
>>>> Maildir/cur` might show a directory >20Mb in size for some of our users
>>>> (20-30k emails).
>>>> (Performance issues == everything is running ok then all of a sudden
>>>> load avg goes through the roof, system cpu time goes crazy. Reading mail
>>>> grinds to a halt. Then everything recovers just as suddenly and the load
>>>> avg gradually returns to normal levels)
>>> Are you using ext3 by chance? Vanilla ext3 without directory indexing
>>> (or whatever it's called) *hates* directories with a lot of files - like
>>> maildir. Personally, I use XFS, which doesn't suffer from this problem
>>> since it uses b-trees instead of a table(!) like ext3 does.
>> This raises another question for me actually...
>>
>> We will have one volume for indexes and another volume for data (using
>> maildir). We will be using the latest stable Debian Linux distro.
>>
>> Any opinions on the best filesystem to use? We would normally go
>> ReiserFS for large volumes, and ext3 for small volumes because of the
>> unlimited inodes in reiserfs, but i understand that support for that is
>> beginning to disappear given that Hans Reiser got into a bit of trouble
>> a few years ago.
>>
>> Anyway... that would leave ext3, but is there a better choice i could
>> make in this instance? We do want performance, of course, but also
>> complete reliability and resilience when it comes to system crashes
>> etc... we do *not* want data corruption, and ext3 we know has a very
>> good journalling and data recovery methods. Well... they're very mature,
>> anyway.
>>
> 
> I used to use ext3, ran into its horrible performance even with
> directory indexing enabled, went to XFS and never looked back. All of my
> systems are Debian stable. Reiser3 is part of the kernel so I wouldn't
> worry about that; Namesys considered it complete and stopped work on it
> long before the whole murder thing. Both Reiser3 and XFS have worse
> reputations than ext3, however, I've seen ext3 filesystems hosed beyond
> repair, too. All my XFS filesystems have battery-backed cache
> controllers, so it hasn't happened to me yet, hopefully never. ;) One
> catch with XFS (such as with LVM) to keep in mind is you can't ever
> shrink it, only grow.

Trouble is... i've been googling this as well, just now, and loads of
people say XFS has the better performance, but loads of other people say
ReiserFS has the better performance.

We have battery backed up RAID controllers too, in this new system, and
the systems are UPSd, so on that basis i'm still none the wiser! lol

I appreciate your experience with XFS is a positive one, but even the
dovecot web site says XFS might now be a good choice...

http://wiki.dovecot.org/MailboxFormat/Maildir

What a tough decision! I know it probably won't make too much difference
in my situation, but i want this to be a very long-term solution, so
want to do it right first time!

Any other opinions on XFS vs Reiserfs with Dovecot maildir?

Thanks again!

Richard.


> ext3 is mature but IMHO completely unsuitable for a busy mail server or
> any situation where you'll have a bajillion of files in one directory.
> The exact point at which ext3 will screw you over obviously depends on
> many factors. But when it happens it'll probably be painful to reformat
> to something better.
> 
> ~Seth
> 
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email 
> ______________________________________________________________________
> 
> 

-- 
Richard Hobbs (IT Specialist)
Toshiba Research Europe Ltd. - Cambridge Research Laboratory
Email: richard.hobbs at crl.toshiba.co.uk
Web: http://www.toshiba-europe.com/research/
Tel: +44 1223 436999        Mobile: +44 7811 803377
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3306 bytes
Desc: S/MIME Cryptographic Signature
Url : http://dovecot.org/pipermail/dovecot/attachments/20090515/2a558e2d/attachment-0001.bin