[Dovecot] index IO patterns

[Timo Sirainen]

On 12.5.2012, at 10.32, Cor Bosman wrote:

>> Mail is always a random IO workload, unless your mailbox count is 1,
>> whether accessing indexes or mail files.  Regarding the other two
>> questions, you'll likely need to take your own measurements.
> 
> Wait, maybe there is a misunderstanding. I mean the IO inside one
> index file, not across the different mailboxes. So within 1 index
> file that covers a mailbox with say 10.000 emails, how does the IO
> occur. I would guess pretty random as well, but on the other hand
> i guess in some ways it could be pretty linear too. If dovecot keeps
> most changes in memory and writes it all back in 1 go. 


Usually the index files are small enough that I think OS reads the whole files into memory anyway. Anyway..:

 * dovecot.index: The header is always accessed first. After that it's accessed as necessary. Many IMAP clients fetch all message flags when selecting mailbox, so this causes a sequential read of the entire file. Also with mmap_disable=yes the whole file is always read into memory.

 * dovecot.index.log: Usually the last few kilobytes of the file are read into memory when mailbox is opened, and after that data is appended and read from it. In some situations the reader might seek to an older data (e.g. to beginning) and read the rest of the file sequentially.

 * dovecot.index.cache: Accessed randomly, depending on what data is needed to be looked up. Typically clients fetch only the last few messages, so the end of the file is accessed sequentially.

Writes are typically appends + rewrites, but currently there are also a few more complex things which I want to get rid of (perhaps for v2.2).