[Dovecot] SSD drives are really fast running Dovecot

[Ed W]

On 17/01/2011 02:20, Stan Hoeppner wrote:
> Ed W put forth on 1/16/2011 4:11 PM:
>>> Using XFS with delayed logging mount option (requires kernel 2.6.36 or later).
>>>
>>> XFS has natively used delayed allocation for quite some time, coalescing
>>> multiple pending writes before pushing them into the buffer cache.  This not
>>> only decreases physical IOPS, but it also decreases filesystem fragmentation by
>>> packing more files into each extent.  Decreased fragmentation means fewer disk
>>> seeks required per file read, which also decreases physical IOPS.  This also
>>> greatly reduces the wasted space typical of small file storage.  Works very well
>>> with maildir, but also with the other mail storage formats.
>> What happens if you pull out the wrong cable in the rack, kernel lockup/oops,
>> power failure, hot swap disk pulled, or something else which causes an
>> unexpected loss of a few seconds of written data?
> Read the XFS FAQ.  These questions have been answered hundreds of times since
> XFS was released in Irix in 1994.  I'm not your personal XFS tutor.

Why the hostile reply?

The question was deeper than your response?


>> Surely your IOPs are hard limited by the number of fsyncs (and size of any
>> battery backed ram)?
> Depends on how your applications are written and how often they call fsync.  Do
> you mean BBWC?  WRT delayed logging BBWC is mostly irrelevant.  Keep in mind
> that for delayed logging to have a lot of metadata writes in memory someone, or
> many someones, must be doing something like an 'rm -rf' or equivalent on a large
> dir with many thousands of files.  Even in this case, the processing is _very_ fast.

You have completely missed my point.

Your data isn't safe until it hits the disk.  There are plenty of ways 
to spool data to ram rather than committing it, but they are all 
vulnerable to data loss until the data is written to disk.

You wrote: "filesystem metadata write operations are pushed almost 
entirely into RAM", but if the application requests an fsync then you 
still have to write it to disk?  As such you are again limited by disk 
IO, which itself is limited by the performance of the device (and 
temporarily accelerated by any persistent write cache).  Hence my point 
that your IOPs are generally limited by the number of fsyncs and any 
persistent write cache?

As I write this email I'm struggling with getting a server running again 
that has just been rudely powered down due to a UPS failing (power was 
fine, UPS failed...).  This isn't such a rare event (IMHO) and hence I 
think we do need to assume that at some point every machine will suffer 
a rude and unexpected event which looses all in progress write cache.  I 
have no complaints at XFS in general, but I think it's important that 
filesystem designers in general have give some thought to this event and 
recovering from it?


Please try not to be so hostile in your email construction - we aren't 
all idiots here, and even if we were, your writing style is not 
conducive to us wanting to learn from your apparent wealth of experience?

Regards

Ed W