[Dovecot] Dovecot and SSD
I'm considering setting up my own mailserver with Dovecot 2.0 (as soon as it's out) and SSD. I'm debating whether it's worth it or not. I have been running a mailserver with the Dovecot 1.1 train for a couple of years and it's been flawless and amazing. I'm thinking about using either FreeBSD 8.1 w/ZFS or OpenBSD 4.8-beta on a 64 GB or 80 GB SSD (Corsair or Intel) using Dovecot 2.0 and mdbox. It seems like mdbox would benefit the most from an SSD because reads would be extremely fast while minimal writes (comparatively) take place due to the way mdbox is structured. I guess what I'm saying is that even if the SSD has average to slow write speeds the fast read speeds would make a big difference in many operations. Any thoughts? Alternately, run either option on a hard drive with indexes on fast flash memory. Anyone tried something like that? Thank you.
Bryan
On 29.7.2010, at 1.05, Bryan Vyhmeister wrote:
It seems like mdbox would benefit the most from an SSD because reads would be extremely fast while minimal writes (comparatively) take place due to the way mdbox is structured.
I'd think that with SSD mdbox would make less sense, since fragmentation isn't so much of a problem with SSDs. I've no benchmarks, but I wouldn't be surprised if sdbox performance was faster. In any case mdbox requires an extra index file that sdbox doesn't, which also increases disk writes.
On Wed, Jul 28, 2010 at 7:19 PM, Timo Sirainen tss@iki.fi wrote:
I'd think that with SSD mdbox would make less sense, since fragmentation isn't so much of a problem with SSDs. I've no benchmarks, but I wouldn't be surprised if sdbox performance was faster. In any case mdbox requires an extra index file that sdbox doesn't, which also increases disk writes.
I didn't realize that sdbox would be faster in that scenario. Maybe this has already been covered but is mdbox generally slower than sdbox or is that just in this scenario. Thanks for your response.
Bryan
On Wed, 2010-07-28 at 21:32 -0400, Bryan Vyhmeister wrote:
On Wed, Jul 28, 2010 at 7:19 PM, Timo Sirainen tss@iki.fi wrote:
I'd think that with SSD mdbox would make less sense, since fragmentation isn't so much of a problem with SSDs. I've no benchmarks, but I wouldn't be surprised if sdbox performance was faster. In any case mdbox requires an extra index file that sdbox doesn't, which also increases disk writes.
I didn't realize that sdbox would be faster in that scenario. Maybe this has already been covered but is mdbox generally slower than sdbox or is that just in this scenario. Thanks for your response.
I'm not saying that sdbox is definitely faster there. It might be. It might not be. I don't know. Would be nice if someone used both in production and showed some benchmarks. :)
On 2010-07-28 19:05:25 -0400, Bryan Vyhmeister wrote:
I'm considering setting up my own mailserver with Dovecot 2.0 (as soon as it's out) and SSD. I'm debating whether it's worth it or not. I have been running a mailserver with the Dovecot 1.1 train for a couple of years and it's been flawless and amazing. I'm thinking about using either FreeBSD 8.1 w/ZFS or OpenBSD 4.8-beta on a 64 GB or 80 GB SSD (Corsair or Intel) using Dovecot 2.0 and mdbox. It seems like mdbox would benefit the most from an SSD because reads would be extremely fast while minimal writes (comparatively) take place due to the way mdbox is structured. I guess what I'm saying is that even if the SSD has average to slow write speeds the fast read speeds would make a big difference in many operations. Any thoughts? Alternately, run either option on a hard drive with indexes on fast flash memory. Anyone tried something like that? Thank you.
maybe dont put the whole FS on the SSD.
we had great speed ups when we just moved the FS journal onto the SSD.
darix-- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org
On Wed, Jul 28, 2010 at 7:46 PM, Marcus Rueckert darix@opensu.se wrote:
maybe dont put the whole FS on the SSD.
we had great speed ups when we just moved the FS journal onto the SSD.
You're referring to the ZFS ZIL or other filesystem journal?
Bryan
On 2010-07-28 21:32:18 -0400, Bryan Vyhmeister wrote:
On Wed, Jul 28, 2010 at 7:46 PM, Marcus Rueckert darix@opensu.se wrote:
maybe dont put the whole FS on the SSD.
we had great speed ups when we just moved the FS journal onto the SSD.
You're referring to the ZFS ZIL or other filesystem journal?
In our case it was the XFS journal. XFS supports external journal devices.
darix-- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org
Bryan Vyhmeister put forth on 7/28/2010 6:05 PM:
I'm considering setting up my own mailserver with Dovecot 2.0 (as soon as it's out) and SSD. I'm debating whether it's worth it or not. I have been running a mailserver with the Dovecot 1.1 train for a couple of years and it's been flawless and amazing. I'm thinking about using either FreeBSD 8.1 w/ZFS or OpenBSD 4.8-beta on a 64 GB or 80 GB SSD (Corsair or Intel) using Dovecot 2.0 and mdbox. It seems like mdbox would benefit the most from an SSD because reads would be extremely fast while minimal writes (comparatively) take place due to the way mdbox is structured. I guess what I'm saying is that even if the SSD has average to slow write speeds the fast read speeds would make a big difference in many operations. Any thoughts? Alternately, run either option on a hard drive with indexes on fast flash memory. Anyone tried something like that? Thank you.
How many users?
-- Stan
On Thu, Jul 29, 2010 at 5:16 AM, Stan Hoeppner stan@hardwarefreak.com wrote:
How many users?
To start with, about 20 users. It will grow to around 100 users at the max.
Bryan
Bryan Vyhmeister put forth on 7/29/2010 6:40 AM:
On Thu, Jul 29, 2010 at 5:16 AM, Stan Hoeppner stan@hardwarefreak.com wrote:
How many users?
To start with, about 20 users. It will grow to around 100 users at the max.
At 100 users you should have ~500MB/user after OS and software overhead. That should be more than enough for most average users I would think.
-- Stan
On 07/29/10 21:40, Bryan Vyhmeister wrote:
On Thu, Jul 29, 2010 at 5:16 AM, Stan Hoeppnerstan@hardwarefreak.com wrote:
How many users?
To start with, about 20 users. It will grow to around 100 users at the max.
Umm... apart from "shiney toys", is there any legitemate reason for such expense on performance tuning?
I ask because we have about 30 users using our IMAP server (Dovecot 1.1), which also serves as our main Samba server, and until recently was also a CUPS server.
This on a Sun V100 ( 450MHz UltraSPARC-IIi ) with 1G RAM and two dull, standard IDE HDDs ( a 40GB for OS and mail, and a 320GB for Samba).
It spends a lot of its life idle, so handling 100 users would be no issue.
I seriously doubt any machine you put together today would be anywhere near as slow, so I fail to understand why you'd be spending extra money and time on performance tuning.
-- Curtis Maloney
Curtis Maloney wrote:
On 07/29/10 21:40, Bryan Vyhmeister wrote:
On Thu, Jul 29, 2010 at 5:16 AM, Stan Hoeppnerstan@hardwarefreak.com wrote:
How many users?
To start with, about 20 users. It will grow to around 100 users at the max.
Umm... apart from "shiney toys", is there any legitemate reason for such expense on performance tuning?
I ask because we have about 30 users using our IMAP server (Dovecot 1.1), which also serves as our main Samba server, and until recently was also a CUPS server.
This on a Sun V100 ( 450MHz UltraSPARC-IIi ) with 1G RAM and two dull, standard IDE HDDs ( a 40GB for OS and mail, and a 320GB for Samba).
I was going to bring up something like that, but I wasn't sure it was fair comparing my server.
I do not have SSD's. But, I have a larger user base and I have a Sun T5220, which is 8 core, 8 threads per core, etc. The drives are 15Krpm SAS. Using Sendmail, mimedefang, Dovecot, Squirrelmail, etc. Also running Apache2 with Drupal, MySQL, gene sequencing apps, PostgreSQL, Samba for file sharing and printer spooling, as well as some other stuff. Squirrelmail/Dovecot is virtually instantaneous even with very large mbox format inboxes. Thunderbird mail checks via Dovecot are also virtually instantaneous. I have over 1500 accounts on the system, although probably only a few hundred are very active. Granted, it is summer, and things will pick up in the Fall, but I doubt it will push this server too much.
--
Chris Hoogendyk
- O__ ---- Systems Administrator c/ /'_ --- Biology & Geology Departments (*) \(*) -- 140 Morrill Science Center
<hoogendyk@bio.umass.edu>
---------------
Erdös 4
On Thu, Jul 29, 2010 at 9:57 PM, Chris Hoogendyk hoogendyk@bio.umass.edu wrote:
Curtis Maloney wrote:
Umm... apart from "shiney toys", is there any legitemate reason for such expense on performance tuning?
I ask because we have about 30 users using our IMAP server (Dovecot 1.1), which also serves as our main Samba server, and until recently was also a CUPS server.
This on a Sun V100 ( 450MHz UltraSPARC-IIi ) with 1G RAM and two dull, standard IDE HDDs ( a 40GB for OS and mail, and a 320GB for Samba).
I was going to bring up something like that, but I wasn't sure it was fair comparing my server.
I do not have SSD's. But, I have a larger user base and I have a Sun T5220, which is 8 core, 8 threads per core, etc. The drives are 15Krpm SAS. Using Sendmail, mimedefang, Dovecot, Squirrelmail, etc. Also running Apache2 with Drupal, MySQL, gene sequencing apps, PostgreSQL, Samba for file sharing and printer spooling, as well as some other stuff. Squirrelmail/Dovecot is virtually instantaneous even with very large mbox format inboxes. Thunderbird mail checks via Dovecot are also virtually instantaneous. I have over 1500 accounts on the system, although probably only a few hundred are very active. Granted, it is summer, and things will pick up in the Fall, but I doubt it will push this server too much.
I do understand where both of you are coming from completely. I have also run a similar setup as the Sun Fire V100 with a Sun Fire V120 and it performed beautifully. There are two main scenarios where using an SSD is interesting to me. Both scenarios involve using a Mac mini. In one case it would be a 1.5 GHz Mac mini G4 with a 16 GB SLC Transcend SSD. Another would be what I was referring to earlier with an Intel Mac mini 2.0 GHz Core 2 Duo. In either scenario, I don't like being stuck with only a single hard drive. I feel more confident that an SSD is less likely to fail in this type of environment. As a benefit, its performance also makes up for other performance issues with the given hardware. Maybe these are bad scenarios and examples but they seem to be potentially good use cases for an SSD. Certainly using an SSD, particularly an IDE one with a Mac mini G4 is size limiting which makes this impractical for anything but a very small setup or even personal installation.
If I were building a more powerful server setup, I would absolutely use a ZFS mirror or raidz1/raidz2 with SSD L2ARC and/or ZIL or potentially hardware RAID with two or more hard drives. While this would be a better design in every way it is also substantially more expensive. Thanks for your comments and I do understand that all of this may be somewhat silly. I will be implementing something with SSD if only for the fun of it.
Bryan
Bryan Vyhmeister wrote:
On Thu, Jul 29, 2010 at 9:57 PM, Chris Hoogendyk hoogendyk@bio.umass.edu wrote:
Curtis Maloney wrote:
Umm... apart from "shiney toys", is there any legitemate reason for such expense on performance tuning?
I ask because we have about 30 users using our IMAP server (Dovecot 1.1), which also serves as our main Samba server, and until recently was also a CUPS server.
This on a Sun V100 ( 450MHz UltraSPARC-IIi ) with 1G RAM and two dull, standard IDE HDDs ( a 40GB for OS and mail, and a 320GB for Samba).
I was going to bring up something like that, but I wasn't sure it was fair comparing my server.
I do not have SSD's. But, I have a larger user base and I have a Sun T5220, which is 8 core, 8 threads per core, etc. The drives are 15Krpm SAS. Using Sendmail, mimedefang, Dovecot, Squirrelmail, etc. Also running Apache2 with Drupal, MySQL, gene sequencing apps, PostgreSQL, Samba for file sharing and printer spooling, as well as some other stuff. Squirrelmail/Dovecot is virtually instantaneous even with very large mbox format inboxes. Thunderbird mail checks via Dovecot are also virtually instantaneous. I have over 1500 accounts on the system, although probably only a few hundred are very active. Granted, it is summer, and things will pick up in the Fall, but I doubt it will push this server too much.
I do understand where both of you are coming from completely. I have also run a similar setup as the Sun Fire V100 with a Sun Fire V120 and it performed beautifully. There are two main scenarios where using an SSD is interesting to me. Both scenarios involve using a Mac mini.
Nothing against SSD's.
Where are we coming from?
Well I guess we were both thinking in terms of general ideas of system tuning -- looking at a system from the perspective of all the components and trying to see where the limiting factors are. The interesting thing about the T5220 is that the clock speed is less than half what most Intel based servers are these days, but it is highly tuned for a multi-threaded web2 type environment. As well as 8 core and 8 threads per core, it has 8 encryption accelerators for all those ssl connections. Throw a huge load of web and mail transactions at it, and it outperforms the competition. We have an example of an 8 core AMD based system that is virtually new, purchased to be a multi-department web server, and our T5220 is noticeably faster carrying a heavier load. That's tuning. However, if we had non-threaded, compute intensive applications, the scales would tip the other way.
From that perspective, I don't see anything wrong with a Mac mini for a small group of users. But, if cost is an issue, I think I would be trying to get a better tuned custom Intel or AMD box for the money, rather than adding an SSD to a Mac mini. Multi-core cpu's are becoming pretty standard. Adding memory is a big item. Faster disk drives in general can make a difference -- compare all the options. Look at the bus structure and the general i/o capabilities. Although I'm the primary Solaris admin, we've been using Ubuntu for more and more things. It performs quite well. All of those things get juggled together when choosing a system and deciding how to set it up and tune it.
There's certainly nothing wrong with experimenting and playing with these things, as long as you can do it without stressing out your budget.
--
Chris Hoogendyk
- O__ ---- Systems Administrator c/ /'_ --- Biology & Geology Departments (*) \(*) -- 140 Morrill Science Center
<hoogendyk@bio.umass.edu>
---------------
Erdös 4
On Thu, Jul 29, 2010 at 9:05 PM, Curtis Maloney cmaloney@umd.com.au wrote:
Umm... apart from "shiney toys", is there any legitemate reason for such expense on performance tuning?
SSDs are a perfectly legitimate technology for a mail server which deals with a lot of random IO. Hell, they're a perfectly legitimate application of technology for most server environments if you can deal with the storage density, and size them appropriately. They're more reliable, use less power, and in PCI-E form are shockingly fast.
In fact, I'd go so far as to say that SSDs should be seeing much wider adoption in IT. Most people are far too scared about the write cycles in particular...as long as you don't fill the drive completely full and your OS supports TRIM, most SSD's will outlast the machine they're put into. We've got a Solaris box with a fairly small (32GB, I think?) Intel SSD in it for its OS; it's mostly empty (the meat and potatoes are the 4x2TB drives used for data storage) and I expect the drive will never die.
I ask because we have about 30 users using our IMAP server (Dovecot 1.1), which also serves as our main Samba server, and until recently was also a CUPS server.
30 users? I've got almost ten times that- as I'm sure many other admins here do and I'm on the low end of the scale in terms of # of users. Our mail spool sits on 4x 10k RPM U320 drives in RAID10, and iowait values that climb up towards ~15% sometimes. As we look at replacing the system, we're strongly eying an SSD because it'll be a huge jump in performance. An SSD will also help with cutting power; I'm hoping we'll end up using half as much power. It'll also be more reliable, though we've dropped only one drive in the course of 5-6 years in this particular machine.
This on a Sun V100 ( 450MHz UltraSPARC-IIi ) with 1G RAM and two dull, standard IDE HDDs ( a 40GB for OS and mail, and a 320GB for Samba).
You're not using redundant storage (is there a particular reason why? Mailspools are far too important and dynamic to trust to just nightly tape), you've got a less-than-40GB mailspool, and "1 user" is not a standardized unit. I'm guessing your server is also sitting behind something between a T1 and a sDSL line? Or that such a line sits between your server and your users?
I have half a dozen users who have more email than your entire mailspool. Our total mailspool is creeping past half a terabyte. We run mailing lists with hundreds of local and remote users on them that generate big spikes in load, and we do spam processing (even with spamd, spamassassin is a hog.)
-B
On 7/29/2010 2:16 AM, Stan Hoeppner wrote:
Bryan Vyhmeister put forth on 7/28/2010 6:05 PM:
I'm considering setting up my own mailserver with Dovecot 2.0 (as soon as it's out) and SSD. I'm debating whether it's worth it or not. I have been running a mailserver with the Dovecot 1.1 train for a couple of years and it's been flawless and amazing. I'm thinking about using either FreeBSD 8.1 w/ZFS or OpenBSD 4.8-beta on a 64 GB or 80 GB SSD (Corsair or Intel) using Dovecot 2.0 and mdbox. It seems like mdbox would benefit the most from an SSD because reads would be extremely fast while minimal writes (comparatively) take place due to the way mdbox is structured. I guess what I'm saying is that even if the SSD has average to slow write speeds the fast read speeds would make a big difference in many operations. Any thoughts? Alternately, run either option on a hard drive with indexes on fast flash memory. Anyone tried something like that? Thank you.
I've been wondering about SSD as well. I have about 1000 users many which have really large inboxes and I'm using maildir format. SDD has no seek time and I wonder if those drive heads are all over the place as people do searhes and such. So if anyone does this I'm interested to see how it works.
Marc Perkel put forth on 7/29/2010 4:35 PM:
I've been wondering about SSD as well. I have about 1000 users many which have really large inboxes and I'm using maildir format. SDD has no seek time and I wonder if those drive heads are all over the place as people do searhes and such. So if anyone does this I'm interested to see how it works.
In this case, where mailbox file storage likely outstrips cost effective SSD capacity, simply moving the Dovecot index files to SSD should make quite a difference in MUA responsiveness. Moving the local MTA's spool to the SSD would help as well by taking seek pressure off the disks.
Not all SSDs are created equal. Read reviews before purchasing. Get one with good mutli threaded/multi user random read/write performance.
-- Stan
participants (8)
-
Brett Dikeman
-
Bryan Vyhmeister
-
Chris Hoogendyk
-
Curtis Maloney
-
Marc Perkel
-
Marcus Rueckert
-
Stan Hoeppner
-
Timo Sirainen