v2.2.32 release candidate released

[Timo Sirainen]

On 16 Aug 2017, at 23.59, Joseph Tam <jtam.home at gmail.com> wrote:
> 
> 
> Timo Sirainen wrote:
> 
>> There are various changes in this release that can be used to significantly reduce disk IO with:
>> 1) NFS storage especially, but I guess also other remote filesystems and even some with local disks
>> 2) When mail storage and INDEX storage are separated
> 
> Thanks for these changes!  Big win for my setup.  My servers are not
> overly stressed, but how much performance gain (using any metric you
> choose) can one expect from these changes?

Depends a lot of things, but for the one specific NFS installation we reduced NFS IOPS by something like 70% And actually they haven't finished setting up the local LISTINDEX changes yet, so maybe even more.

>> + mail_location can now include VOLATILEDIR=<path> parameter. This
>>   is used for creating lock files and in future potentially other
>>   files that don't need to exist permanently. The path could point to
>>   tmpfs for example. This is especially useful to avoid creating lock
>>   files to NFS or other remote filesystems. For example:
>>   mail_location=sdbox:~/sdbox:VOLATILEDIR=/tmp/volatile/%2.256Nu/%u
> 
> Is "/tmp/volatile" auto-created, or must be pre-created?

Autocreated.

>> + mail_location's LISTINDEX=<path> can now contain a full path.
>>   This allows storing mailbox list index to a different storage
>>   than the rest of the indexes, for example to tmpfs.
> 
> Is this in any way related to VOLATILEDIR?  Can they be set to the same
> value without problems?

Yeah, can be set to the same directory. Maybe even a good idea.

>> + mail_location can now include NO-NOSELECT parameter. This
>>   automatically deletes any \NoSelect mailboxes that have no children.
>>   These mailboxes are sometimes confusing to users.
> 
> Sorry for my IMAP ignorance, but how can this situation come about?

a CREATE noselectbox/
--> With this change it's the same as CREATE "noselectbox" = will become selectable.

or

a CREATE noselectbox/childbox
b DELETE noselectbox/childbox
--> noselectbox is left behind. With this change it will be auto-deleted.

>> + mail_location can now include ITERINDEX parameter. This tells Dovecot
>>   to perform mailbox listing from the INDEX path instead of from the
>>   mail root path. It's mainly useful when the INDEX storage is on a
>>   faster storage.
>> + If mailbox_list_index_very_dirty_syncs=yes, the list index is no
>>   longer refreshed against filesystem when listing mailboxes. This
>>   allows the mailbox listing to be done entirely by only reading the
>>   mailbox list index.
>> + Added mailbox_list_index_include_inbox setting to control whether
>>   INBOX's STATUS information should be cached in the mailbox list
>>   index. The default is "no", but it may be useful to change it to
>>   "yes", especially if LISTINDEX points to tmpfs.
> 
> So as I understand it, the optimzation comes about from segregating mail
> data information into 3 parts: raw mail, indices, and volatile components,
> putting them into increasingly better performing storage media.
> 
> How do these I/O optimizations affect the client's view of a mailbox
> if their mailbox is subject to modification outside the dovecot system
> (e.g.  procmail, mail readers directly modifies mailbox files)?  Is there
> a trade-off between metadata consistency and performance, or it's a
> win-win all around?

I think already even with mailbox_list_index=yes it won't notice external changes done to the mailboxes when doing STATUS calls, so better to avoid that. It does get fixed automatically once the folder is selected though.