On Feb 25, 2010, at 3:29 PM, Bradley Giesbrecht wrote:

On Feb 25, 2010, at 2:17 PM, Terry Barnum wrote:

...

I have postfix/dovecot/mysql installed using MacPorts on a quad-core 2.8GHz MacPro running Snow Leopard (10.6.2). I moved the base mail directory to a pair of 10k RPM Raptors that are mirrored (/Volumes/email/) and everything seems to be working fine on an unused domain with very little traffic. I used imapsync to pull everything from the current mailserver to this test server to play with.

Our current mailserver's IMAP performance *really* suffers when an IMAP folder exceeds ~500MB or ~5k messages. I've become very tired of being the mailbox police trying to get my 20 users to delete email or divide into smaller mailboxes.

Are there folks on the list running postfix/dovecot on similar Mac hardware that can share their experiences? Specifically, are there any limitations (file descriptors, other?) that can impact performance I should be aware of? How does dovecot on the Mac deal with >500GB maildirs?

Is there a recommended tool for loading and testing the server? Postal?

I have 20 IMAP users, with a total of ~37GB of mail currently.

Thanks for any help and insight.

Mac OS X 10.5.8 dovecot 1.2.9 Two mirrored 7200rpm sata drives. dbox mailbox format

Thanks Brad. What hardware is this running on? Also, are you using the plain vanilla Apple RAID tool for mirroring or something else like SoftRAID?

My Inbox is 3.5G on disk and I'm happy with my performance. Mail.app took 8 sec. to search my Inbox for "postfix" but caches mail. Telnet imap login took 55 sec. for 'search text "postfix"'. New mail notifications in Mail.app are near instant. I know your old box fails here.

// Brad

Terry Barnum digital OutPost San Diego, CA

http://www.dop.com 800/464-6434

On Feb 25, 2010, at 5:17 PM, Bradley Giesbrecht wrote:

On Feb 25, 2010, at 4:31 PM, Terry Barnum wrote:

...

On Feb 25, 2010, at 3:29 PM, Bradley Giesbrecht wrote:

...

On Feb 25, 2010, at 2:17 PM, Terry Barnum wrote:

...

I have postfix/dovecot/mysql installed using MacPorts on a quad-core 2.8GHz MacPro running Snow Leopard (10.6.2). I moved the base mail directory to a pair of 10k RPM Raptors that are mirrored (/Volumes/email/) and everything seems to be working fine on an unused domain with very little traffic. I used imapsync to pull everything from the current mailserver to this test server to play with.

Our current mailserver's IMAP performance *really* suffers when an IMAP folder exceeds ~500MB or ~5k messages. I've become very tired of being the mailbox police trying to get my 20 users to delete email or divide into smaller mailboxes.

Are there folks on the list running postfix/dovecot on similar Mac hardware that can share their experiences? Specifically, are there any limitations (file descriptors, other?) that can impact performance I should be aware of? How does dovecot on the Mac deal with >500GB maildirs?

Is there a recommended tool for loading and testing the server? Postal?

I have 20 IMAP users, with a total of ~37GB of mail currently.

Thanks for any help and insight.

Mac OS X 10.5.8 dovecot 1.2.9 Two mirrored 7200rpm sata drives. dbox mailbox format

Thanks Brad. What hardware is this running on? Also, are you using the plain vanilla Apple RAID tool for mirroring or something else like SoftRAID?

DP G5 2Ghz. 5.5GB ram. Apple Disk Utility software RAID 1.

Users are virtual in mysql. I also have a fairly busy application hitting mysql. Mysql seems to be cacheing around 120 threads which is less the config max and I usually have a around 700MB of free memory.

User Inbox count around 200 on 20 domains but we are a Print company and have to except large attachments and store indefinitely.

I looked at the wiki on dbox but shied away from it because the compatibility matrix said postfix didn't like it. Did I read that wrong?

Would you mind sharing your postfix -n and dovecot -n? Edited and offlist would be fine.

Did you use macports to build dovecot?

Yes. 1.2.10.

If so I'm working of some new packages you may be interested in. Mostly dovecot-sieve and dovecot-managesieve.

I have started to look at sieve and can see how it could be useful for my iPhone users. Currently they have to leave Mail.app running on their desktop to sort mail into IMAP folders since the phone doesn't have any sort by rules filters. So yes, I'm interested.

Your welcome to hit me up off list if you have some non-dovecot questions. I'm small time compared to many here but I love dovecot and won't be looking elsewhere till I come home broke and bleeding.

// Brad

Terry Barnum digital OutPost San Diego, CA

http://www.dop.com 800/464-6434

On Feb 25, 2010, at 9:42 PM, Timo Sirainen wrote:

On 26.2.2010, at 3.50, Terry Barnum wrote:

...

I looked at the wiki on dbox but shied away from it because the
compatibility matrix said postfix didn't like it. Did I read that
wrong?

You need to be delivering mails with Dovecot LDA when using dbox.
And that might help improve performance even when you're using
maildir.

...

...

Did you use macports to build dovecot?

Yes. 1.2.10.

You could try if maildir_very_dirty_syncs=yes helps.

With v2.0+ mdbox will probably work very nicely. dbox (with v1.x or
v2.0) still uses one file/message and I guess if you had trouble
with 5k+ messages in a mailbox with Maildir, you'll probably hit the
same slowness with dbox.

Will there be a way to convert from dbox t mdbox in version v2.0+?

// Brad

On Feb 25, 2010, at 9:42 PM, Timo Sirainen wrote:

On 26.2.2010, at 3.50, Terry Barnum wrote:

...

I looked at the wiki on dbox but shied away from it because the compatibility matrix said postfix didn't like it. Did I read that wrong?

You need to be delivering mails with Dovecot LDA when using dbox. And that might help improve performance even when you're using maildir.

Thank you Timo. Sorry, my newbie misunderstanding of postfix and dovecot. I think I now understand the compatibility matrix to mean that postfix doesn't work with dbox or mdbox formats natively, it must use dovecot.

With v2.0+ mdbox will probably work very nicely. dbox (with v1.x or v2.0) still uses one file/message and I guess if you had trouble with 5k+ messages in a mailbox with Maildir, you'll probably hit the same slowness with dbox.

The 500MB+, 5k+ message problem is with another mailserver not running postfix/dovecot. It's the reason I've started investigating postfix with your software and so far I"m very happy.

Thanks, -Terry

On Feb 27, 2010, at 2:08 AM, Stan Hoeppner wrote:

Terry Barnum put forth on 2/26/2010 4:20 PM:

...

The 500MB+, 5k+ message problem is with another mailserver not running postfix/dovecot. It's the reason I've started investigating postfix with your software and so far I"m very happy.

Hi Terry,

If it is remotely a possibility, I'd recommend going with Linux instead of OSX. Going Linux gets you access to the XFS filesystem which gives the best overall performance for maildirs. It seems very likely that your current production server is suffering at 5K files and 500MB mailboxen due to a deficient filesystem, among other things. I'm partial to Debian Stable (Lenny) myself. I use mbox with XFS due to the fact I archive all my list mail and thus generate some pretty large mbox files, growing every day. If I decide to switch to or add maildirs later, I've already got the filesystem that performs best for both worlds.

If you must go with OSX, avoid HFS+ and go with ZFS. ZFS performance is likely inferior to XFS for maildirs, but it would still be much better than HFS+.

Thank you Stan. Unfortunately I read that Apple ran into licensing issues with Sun/Oracle over ZFS and removed it from Snow Leopard before it was released last year.

-Terry

On 27/02/2010 10:08, Stan Hoeppner wrote:

Terry Barnum put forth on 2/26/2010 4:20 PM:

...

The 500MB+, 5k+ message problem is with another mailserver not running postfix/dovecot. It's the reason I've started investigating postfix with your software and so far I"m very happy.

Hi Terry,

If it is remotely a possibility, I'd recommend going with Linux instead of OSX. Going Linux gets you access to the XFS filesystem which gives the best overall performance for maildirs. It seems very likely that your current production server is suffering at 5K files and 500MB mailboxen due to a deficient filesystem, among other things. I'm partial to Debian Stable (Lenny) myself. I use mbox with XFS due to the fact I archive all my list mail and thus generate some pretty large mbox files, growing every day. If I decide to switch to or add maildirs later, I've already got the filesystem that performs best for both worlds.

XFS offers some nice features such as resize of partitions and snapshots, however, it appears to sometimes need some tuning for small file write applications? (adjust log buffers I think?). (Some people have also suffered dataloss in the event of sudden powerloss - bad interactions with the writeback cache.. Obviously likely with any filesystem though)

You might also want to benchmark Ext4 since it's apparently now considered stable and easy to migrate from ext3. Google apparently are migrating internally from ext2 to ext4 having benchmarked it to be similar speed to XFS (for *their* workloads!)

Ed W

On 26/02/2010 01:17, Bradley Giesbrecht wrote:

User Inbox count around 200 on 20 domains but we are a Print company and have to except large attachments and store indefinitely.

You are a print company and you *only* have a 3.5GB inbox?

I helped my friend at the local print shop upgrade his storage last year and we put in 3TB of SAS attached to the Rampage server. He has burned through that in a year without a problem...

(None of this relates to Dovecot, it's just where he saves the files before deleting the emails...)

:-)

Good luck

Ed W