[Dovecot] Dovecot stable slow
A few days ago I installed dovecot stable to replace uw-imap. The install went well and all boxes were converted ok. When accessing the new imap server though, certain operations seem much slower, in particular, moving mail between boxes is very slow, and I have received several complaints from users that the mail server has slowed down. I can't figure out what the slow point is as maildir should be faster than mbox.
I sense much NT in you. NT leads to Bluescreen. Bluescreen leads to downtime. Downtime leads to suffering. NT is the path to the darkside. Powerful Unix is.
Public Key: ftp://ftp.tallye.com/pub/lorenl_pubkey.asc Fingerprint: CEE1 AAE2 F66C 59B5 34CA C415 6D35 E847 0118 A3D2
"Loren M. Lang" lorenl@alzatex.com writes:
A few days ago I installed dovecot stable to replace uw-imap. The install went well and all boxes were converted ok. When accessing the new imap server though, certain operations seem much slower, in particular, moving mail between boxes is very slow, and I have received several complaints from users that the mail server has slowed down. I can't figure out what the slow point is as maildir should be faster than mbox.
What UNIX are you running on, what filesystem and what kind of hardware are we talking about? Is maildir_copy_with_hardlinks set?
I find that there are circumstances on modern hardware when mbox is actually faster than maildir. An Intel box with a good disk system will easily load and traverse a multi-megabyte mbox in the time it takes to do a few seeks on the disk (ie finding the various mails in a maildir).
Of course, on dovecot, you can't have folders in folders with mbox, so it's not really an alternativefor me. But maybe you should try it to compare. If your users mostly do operations on massive amounts of mail, it may be an improvement on modern hardware.
Anders "Quest" Qvist
Ye olde jungle proverb: Even the Phantom must turn on the light to see what's in his goth wardrobe.
On Sun, Jun 12, 2005 at 09:09:55AM +0200, Quest wrote:
"Loren M. Lang" lorenl@alzatex.com writes:
A few days ago I installed dovecot stable to replace uw-imap. The install went well and all boxes were converted ok. When accessing the new imap server though, certain operations seem much slower, in particular, moving mail between boxes is very slow, and I have received several complaints from users that the mail server has slowed down. I can't figure out what the slow point is as maildir should be faster than mbox.
What UNIX are you running on, what filesystem and what kind of hardware are we talking about? Is maildir_copy_with_hardlinks set?
Linux 2.4 kernel on redhat 9 using reiserfs on all partitions with raid1 on 2 100 gig 7200 rpm wd drives. P4 2.4G 800Mhz FSB w/HT but HT has been recently turned off do to the vulnerability in it. Also 1G DDR 2700 ram. It's pretty close to the example config file so that option is turned off. I could try enabling it, but I am currently running 0.99.x, with it's own default config modified slightly, but it that option is still disabled and runs just fine so I don't think that is the slow part. Some difference between stable and 0.99.x seems to cause the slowdown. I could compare the two config files, but I think it's a difference in implementation rather than a different configuration.
I find that there are circumstances on modern hardware when mbox is actually faster than maildir. An Intel box with a good disk system will easily load and traverse a multi-megabyte mbox in the time it takes to do a few seeks on the disk (ie finding the various mails in a maildir).
I'd still expect maildir to beat mbox in modifying/deleting specific emails in a large (2000+) email box, but I can see how searching or downloading could be faster using mbox.
Of course, on dovecot, you can't have folders in folders with mbox, so it's not really an alternativefor me. But maybe you should try it to compare. If your users mostly do operations on massive amounts of mail, it may be an improvement on modern hardware.
I may try this, but as this is a live mail server, I can only do limited testing.
-- Anders "Quest" Qvist
Ye olde jungle proverb: Even the Phantom must turn on the light to see what's in his goth wardrobe.
-- I sense much NT in you. NT leads to Bluescreen. Bluescreen leads to downtime. Downtime leads to suffering. NT is the path to the darkside. Powerful Unix is.
Public Key: ftp://ftp.tallye.com/pub/lorenl_pubkey.asc Fingerprint: CEE1 AAE2 F66C 59B5 34CA C415 6D35 E847 0118 A3D2
On Tue, 2005-06-14 at 01:41 -0700, Loren M. Lang wrote:
Linux 2.4 kernel on redhat 9 using reiserfs on all partitions with raid1 on 2 100 gig 7200 rpm wd drives. P4 2.4G 800Mhz FSB w/HT but HT has been recently turned off do to the vulnerability in it. Also 1G DDR 2700 ram. It's pretty close to the example config file so that option is turned off. I could try enabling it, but I am currently running 0.99.x, with it's own default config modified slightly, but it that option is still disabled and runs just fine so I don't think that is the slow part. Some difference between stable and 0.99.x seems to cause the slowdown. I could compare the two config files, but I think it's a difference in implementation rather than a different configuration.
How much slower is 1.0-stable, when moving how many messages and how large the source and destination mailboxes are? I can't really think of any difference that would make 0.99.x faster, so there could be some bug but I'm not seeing it in my tests here.
On 14.6.2005, at 12:14, Timo Sirainen wrote:
On Tue, 2005-06-14 at 01:41 -0700, Loren M. Lang wrote:
Linux 2.4 kernel on redhat 9 using reiserfs on all partitions with raid1 on 2 100 gig 7200 rpm wd drives. P4 2.4G 800Mhz FSB w/HT but HT has been recently turned off do to the vulnerability in it. Also 1G DDR 2700 ram. It's pretty close to the example config file so that option is turned off. I could try enabling it, but I am currently running 0.99.x, with it's own default config modified slightly, but it that option is still disabled and runs just fine so I don't think that is the slow part. Some difference between stable and 0.99.x seems to cause the slowdown. I could compare the two config files, but I think it's a difference in implementation rather than a different configuration.
How much slower is 1.0-stable, when moving how many messages and how large the source and destination mailboxes are? I can't really think of any difference that would make 0.99.x faster, so there could be some bug but I'm not seeing it in my tests here.
For everyone: the problem seems to be fsync() and fdatasync() calls which could delay for over a second. I'm not sure if this is reiserfs-specific problem or just specific to this one system. Anyway, these calls aren't exactly required but they try to make it sure that in case the system crashes nothing is left in inconsistent state. I'll see if I could get them called less often, and also disabled optionally..
Loren M. Lang wrote:
A few days ago I installed dovecot stable to replace uw-imap. The install went well and all boxes were converted ok. When accessing the new imap server though, certain operations seem much slower, in particular, moving mail between boxes is very slow, and I have received several complaints from users that the mail server has slowed down. I can't figure out what the slow point is as maildir should be faster than mbox.
What email client is being used? I've seen some similar behaviour with Thunderbird which I believe (from reading Bugzilla) may be a Thunderbird IMAP bug rather than a Dovecot issue.
John
-- John Allen Bofferdange, Luxembourg allen@vo.lu http://www.homepages.lu/allen
On Sun, Jun 12, 2005 at 09:52:25PM +0200, John and Catherine Allen wrote:
Loren M. Lang wrote:
A few days ago I installed dovecot stable to replace uw-imap. The install went well and all boxes were converted ok. When accessing the new imap server though, certain operations seem much slower, in particular, moving mail between boxes is very slow, and I have received several complaints from users that the mail server has slowed down. I can't figure out what the slow point is as maildir should be faster than mbox.
What email client is being used? I've seen some similar behaviour with Thunderbird which I believe (from reading Bugzilla) may be a Thunderbird IMAP bug rather than a Dovecot issue.
Actually, the mail clients were mozilla, thunderbird, and kmail, and maybe even squirrel, can't remember if I tested that though. All were slow with dovecot stable, but when I downgraded to 0.99.x, the problems went away and performance was back to around uw-imap's performance or better.
John
-- John Allen Bofferdange, Luxembourg allen@vo.lu http://www.homepages.lu/allen
-- I sense much NT in you. NT leads to Bluescreen. Bluescreen leads to downtime. Downtime leads to suffering. NT is the path to the darkside. Powerful Unix is.
Public Key: ftp://ftp.tallye.com/pub/lorenl_pubkey.asc Fingerprint: CEE1 AAE2 F66C 59B5 34CA C415 6D35 E847 0118 A3D2
This might help:
- Set dir_index features on hard disk.
tune2fs -l /dev/hda3 | grep features
if you see dir_index then you are all set. If dir_index is missing then add it using: # umount /dev/hda3 # tune2fs -O dir_index /dev/hda3 # e2fsck -fD /dev/hda3 # mount /dev/hda3 Now you can handle ginormous maildir directories with out breaking a sweat.
When we switched from UW-IMAP mbox to dovecot maildir it was slightly slower (3% maybe) at downloading headers but much faster in every other aspect.
Jeff Graves, MCSA Customer Support Engineer Image Source, Inc. 10 Mill Street Bellingham, MA 02019
508.966.5200 - Phone 508.966.5170 - Fax jeff@image-src.com - Email www.image-src.com
-----Original Message----- From: dovecot-bounces@dovecot.org [mailto:dovecot-bounces@dovecot.org] On Behalf Of Loren M. Lang Sent: Saturday, June 11, 2005 12:27 PM To: dovecot@dovecot.org Subject: [Dovecot] Dovecot stable slow
A few days ago I installed dovecot stable to replace uw-imap. The install went well and all boxes were converted ok. When accessing the new imap server though, certain operations seem much slower, in particular, moving mail between boxes is very slow, and I have received several complaints from users that the mail server has slowed down. I can't figure out what the slow point is as maildir should be faster than mbox.
I sense much NT in you. NT leads to Bluescreen. Bluescreen leads to downtime. Downtime leads to suffering. NT is the path to the darkside. Powerful Unix is.
Public Key: ftp://ftp.tallye.com/pub/lorenl_pubkey.asc Fingerprint: CEE1 AAE2 F66C 59B5 34CA C415 6D35 E847 0118 A3D2
On Mon, Jun 13, 2005 at 09:42:28AM -0400, Jeff Graves wrote:
This might help:
- Set dir_index features on hard disk.
tune2fs -l /dev/hda3 | grep features
if you see dir_index then you are all set. If dir_index is missing then add it using: # umount /dev/hda3 # tune2fs -O dir_index /dev/hda3 # e2fsck -fD /dev/hda3 # mount /dev/hda3 Now you can handle ginormous maildir directories with out breaking a sweat.
Actually, we're using reiserfs for all our server filesystems, which everything I've read about it says it's generally a much better performer for maildir than ext3 in most respects.
When we switched from UW-IMAP mbox to dovecot maildir it was slightly slower (3% maybe) at downloading headers but much faster in every other aspect.
Jeff Graves, MCSA Customer Support Engineer Image Source, Inc. 10 Mill Street Bellingham, MA 02019
508.966.5200 - Phone 508.966.5170 - Fax jeff@image-src.com - Email www.image-src.com
-----Original Message----- From: dovecot-bounces@dovecot.org [mailto:dovecot-bounces@dovecot.org] On Behalf Of Loren M. Lang Sent: Saturday, June 11, 2005 12:27 PM To: dovecot@dovecot.org Subject: [Dovecot] Dovecot stable slow
A few days ago I installed dovecot stable to replace uw-imap. The install went well and all boxes were converted ok. When accessing the new imap server though, certain operations seem much slower, in particular, moving mail between boxes is very slow, and I have received several complaints from users that the mail server has slowed down. I can't figure out what the slow point is as maildir should be faster than mbox.
I sense much NT in you. NT leads to Bluescreen. Bluescreen leads to downtime. Downtime leads to suffering. NT is the path to the darkside. Powerful Unix is.
Public Key: ftp://ftp.tallye.com/pub/lorenl_pubkey.asc Fingerprint: CEE1 AAE2 F66C 59B5 34CA C415 6D35 E847 0118 A3D2
-- I sense much NT in you. NT leads to Bluescreen. Bluescreen leads to downtime. Downtime leads to suffering. NT is the path to the darkside. Powerful Unix is.
Public Key: ftp://ftp.tallye.com/pub/lorenl_pubkey.asc Fingerprint: CEE1 AAE2 F66C 59B5 34CA C415 6D35 E847 0118 A3D2
participants (5)
-
Jeff Graves
-
John and Catherine Allen
-
Loren M. Lang
-
Quest
-
Timo Sirainen