[Dovecot] Directory Layout Performance
From a performance perspective: Which Directory layout is better:
- All mailboxes are stored in a single directory and prefixed with a dot or
- Maildirs using physical directories, such as: Maildir/folder/subfolder/
It looks like the second option -- File Sytem Layout (LAYOUT=fs) --
could spread the load in a better way ......
I do not expect a user reaching maximum number of subfolders in linux ,
but what about those heavy users with a lot of sub-folders:
Is there a significant performance gain by choosing a specific
Directory Layout ...
Any Comments?
Regards,
Mario Antonio
Mario Antonio Garcia wrote:
From a performance perspective: Which Directory layout is better:
- All mailboxes are stored in a single directory and prefixed with a dot or
- Maildirs using physical directories, such as: Maildir/folder/subfolder/
It looks like the second option -- File Sytem Layout (LAYOUT=fs) -- could spread the load in a better way ...... I do not expect a user reaching maximum number of subfolders in linux , but what about those heavy users with a lot of sub-folders: Is there a significant performance gain by choosing a specific Directory Layout ...
Any Comments?
I use a filesystem that handles this better than ext3 such as XFS or Reiser.
~Seth
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wed, Jul 01, 2009 at 11:48:04AM -0700, Seth Mattinen wrote:
Mario Antonio Garcia wrote:
From a performance perspective:
[...]
I use a filesystem that handles this better than ext3 such as XFS or Reiser.
Ext3 should be fine for huge directories these days (given mount option dir_index it uses hashes for directory lists, but that should be the default in newer installations). You can find out whether it's on with "tune2fs <device>".
Regards
- -- tomás -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iD4DBQFKTEzDBcgs9XrR2kYRAojgAJ9Al08lSp8A0nL3fYJWK2q3Nu0zaACY9s0O RaCFNfvJLFLRy08mSYdEOA== =dovI -----END PGP SIGNATURE-----
tomas@tuxteam.de wrote:
On Wed, Jul 01, 2009 at 11:48:04AM -0700, Seth Mattinen wrote:
Mario Antonio Garcia wrote:
From a performance perspective:
[...]
I use a filesystem that handles this better than ext3 such as XFS or Reiser.
Ext3 should be fine for huge directories these days (given mount option dir_index it uses hashes for directory lists, but that should be the default in newer installations). You can find out whether it's on with "tune2fs <device>".
Not sure about maildir, but I experienced horrible performance with ext3 on a very busy postfix queue a few years ago. The dir_index was on, it was on a battery-backed hardware RAID, but it just couldn't handle it until I reformatted /var to XFS in the middle of the day out of desparation. Yeah, I know Reiser should have better performance with lots of small files, but I had a scary reiser-ate-itself experience once. As far as huge mbox files, XFS should have an advantage there.
I don't have any numbers beyond ext3 "mail is backed up, people are complaining, and the load average is through the roof" vs. XFS "it's easily keeping up with the queue, everyone is happy, and the load average is sane". ;)
~Seth
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wed, Jul 01, 2009 at 11:06:24PM -0700, Seth Mattinen wrote:
tomas@tuxteam.de wrote:
On Wed, Jul 01, 2009 at 11:48:04AM -0700, Seth Mattinen wrote:
Mario Antonio Garcia wrote:
From a performance perspective:
[...]
I use a filesystem that handles this better than ext3 such as XFS or Reiser.
Ext3 should be fine for huge directories [...]
Not sure about maildir, but I experienced horrible performance with ext3 on a very busy postfix queue a few years ago [...]
Thanks, Seth for this info. Definitely worth keeping in mind.
[...] Yeah, I know Reiser should have better performance with [...]
Hm. I think Reiser is for those "make sure you know what aou're doing". I have myself some partitions on Reiser, but I'm not so sure I would base critical stuff on it.
I'm too maxed-out at the moment to go trolling through the kernel change logs to see whether anything might have changed in dir_index in the last three years, but it might be interesting.
Thanks, regards
- -- tomás -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iD8DBQFKTFi4Bcgs9XrR2kYRArk5AKCB6WdEL/+uR2S3fQYM8CGop17NuACffgbH s+IpuDtKbl1MqZWU6GeGhYE= =IJm2 -----END PGP SIGNATURE-----
On 7/2/2009, tomas@tuxteam.de (tomas@tuxteam.de) wrote:
[...] Yeah, I know Reiser should have better performance with [...]
Hm. I think Reiser is for those "make sure you know what aou're doing". I have myself some partitions on Reiser, but I'm not so sure I would base critical stuff on it.
Lets please stop with the FUD.
Yes, Reiserfs had a few bugs early on... but I have been running 3 production servers on it for 4 years, with zero problems (even went through 3 unclean shutdowns)... yes, there have been problems, but, the same can be said of every single filesystem out there.
If you based your decision of what filesystem to use on whether or not there have ever been any serious problems encountered with it, you wouldn't have a computer.
--
Best regards,
Charles
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Thu, Jul 02, 2009 at 06:01:01AM -0400, Charles Marcus wrote:
On 7/2/2009, tomas@tuxteam.de (tomas@tuxteam.de) wrote: [...]
Hm. I think Reiser is [...]
Lets please stop with the FUD.
Oops. Sorry. I didn't want to step on anyone's toes.
Yes, Reiserfs had a few bugs early on... but I have been running 3 production servers on it for 4 years [...]
YMMV. As I said, I have a couple of big (albeit not heavily used) FS on Reiser and am quite happy up to now.
If you based your decision of what filesystem to use on whether or not there have ever been any serious problems encountered with it, you wouldn't have a computer.
No. I base my assessment on other people's experiences. One of the real downsides or Reiser at the moment seems to be that it is more prone to silent corruption (triggered by hardware problems). I don't have the time now to provide references, but if interested, I'll dig it up (maybe it's time to take this off-list anyway :)
Regards
- -- tomás -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iD8DBQFKTJXiBcgs9XrR2kYRAilGAJ4uRlV5jjEw8mFCuvsVzacF28UWzwCfUX5B hTuJyygkx8BY2C0ks1gmS7Y= =X8xH -----END PGP SIGNATURE-----
On 7/2/2009, tomas@tuxteam.de (tomas@tuxteam.de) wrote:
If you based your decision of what filesystem to use on whether or not there have ever been any serious problems encountered with it, you wouldn't have a computer.
No. I base my assessment on other people's experiences.
I said, '... any serious problems encountered with it'... by *anyone*, not just you... so this includes other peoples experiences.
One of the real downsides or Reiser at the moment seems to be that it is more prone to silent corruption (triggered by hardware problems).
All filesystems are prone to this problem... hardware problems can cause all kinds of nasty things to happen, many of them 'silently'.
I am *not* saying reiserfs is perfect - no fs is... I'm just saying that there are *plenty* of people who have had just as serious problems with ext3, xfs, and every other filesystem out there.
The fact is, reiserfs performance is awesome for mail servers (especially IMAP servers), and has been rock solid for me for as long as I've been using it...
I don't have the time now to provide references, but if interested, I'll dig it up (maybe it's time to take this off-list anyway :)
Not necessary, I'm fully aware of them.
All that said, I'll probably be migrating my next systems to either ext4 or brtfs, depending on *when*... I wish zfs was gpl so it could make it into the linux kernel. Heck, I've been considering trying out FreeBSD for that one reason...
But you're right, this is way OT now, and I don't have time to carry it on privately... good luck (to you and everyone) with whatever fs you choose...
--
Best regards,
Charles
Seth,
XFS ussually performs better handling large files, and Reiser handling small files .... I also think that File system like XFS could handle Large Directoreis in a better way: http://oss.sgi.com/projects/xfs/papers/xfs_usenix/index.html Nevertheless, this thread http://dovecot.org/list/dovecot/2007-January/018994.html mentioned about poor performance for XFS ...... It would be interesting to hear about production environments using XFS .....
Mario Antonio
participants (4)
-
Charles Marcus
-
Mario Antonio Garcia
-
Seth Mattinen
-
tomas＠tuxteam.de