Timo Sirainen put forth on 10/21/2010 9:52 AM:
On Wed, 2010-10-20 at 23:50 -0500, Stan Hoeppner wrote:
Oh, interesting. I didn't know that was possible. And even better: Linux has fallocate() that can do it for other filesystems than just XFS. Or looks like it's only XFS and ext4 (ext3 doesn't support it). I don't know if other OSes support this. There exists posix_fallocate() which would widen the platforms that would support this Timo.
I know about posix_fallocate(), but that does a different thing. It would work only when this works:
Ahh, ok, didn't realize that it was different than the Linux specific call.
Maybe in future I could make mdbox
support writing to files whose size has been preallocated by actually writing NUL bytes, but that requires some extra code.
You may also want to look at posix_fadvise() as well (if you're not using it already) which might increase Dovecot's overall disk performance a bit.
I once wrote a patch for that and asked if anyone could confirm if it made things worse or better, but no one ever did. http://dovecot.org/list/dovecot/2007-November/026819.html
I wasn't aware of this. I don't really have the resources, skill set, or time to test patches, or I'd do so. :(
NOTE: I don't believe fallocate() in either posix or linux only form will actually accomplish decreased m[d]box file fragmentation. I don't believe it actually increases the file size on disk, i.e. physically allocating additional free extents tailing the end of the file. fallocate() is _speculative_ preallocation, which isn't what you want.
Why do you believe that? The man pages and everything tell me that it's about physical preallocation:
fallocate is used to preallocate blocks to a file. For filesystems which support the fallocate system call, this is done quickly by allo‐ cating blocks and marking them as uninitialized, requiring no IO to the data blocks. This is much faster than creating a file by filling it with zeros.Also "du" shows that the file's size is actually the preallocated size. That's how I checked which filesystems supported it.
And I think Linux fallocate() with XFS internally does exactly what XFS_IOC_RESVSP does, which your quoted mail mentioned being about physical preallocation.
Well that's cool then.
I don't know if it changes anything in the sequence above, but Dovecot uses mmap i/o.
mmap is used only for index files, not for mdbox files.
Ahh, that's really good to know. IIRC the comments for enabling mmap in dovecot.conf don't state this. I thus assumed enabling mmap was for all files.
It would be beneficial I think if you'd sub to the xfs list Timo and pick some brains.
Maybe, but there is still so much other stuff to do, like bug fixing. :)
Looks like you've got a handle on it. Like I said, I know just enough about this stuff to make me dangerous. ;)
-- Stan