[Dovecot] minimize mbox mdbox fragmentation

[Stan Hoeppner]

Hi Timo,

Any chance the mbox/mdbox writer code could be modified to do physical
preallocation on files to help avoid file(system) fragmentation?
Constantly appending a file is the prime recipe for causing
fragmentation, and mbox is notorious for this--not a fault of Dovecot
but the nature of the mbox beast.  Obviously maildir doesn't have such a
problem, but some (many?) of us still prefer mbox for many reasons, fast
full body search being one, portability being another.  mdbox file
fragmentation could benefit from such a change as well.

I was having a discussion on the XFS list about this, trying to tweak
XFS mount options to mitigate the fragmentation effects.  Alas, there is
no way to do this purely at the filesystem level.  From Dave Chinner,
one of the lead XFS devs:

"What you want is _physical_ preallocation, not speculative
preallocation. i.e. look up XFS_IOC_RESVSP or FIEMAP so your
application does _permanent_ preallocate past EOF. Alternatively, the
filesystem will avoid the truncation on close() if the file has the
APPEND attribute set and the application is writing via O_APPEND...

The filesystem cannot do everything for you. Sometimes the
application has to help...."

How great would the effort need to be to implement something like this?
 Would the return on investment be sufficient to justify doing this, in
your eyes?

-- 
Stan