[Dovecot] Design: Adding checksums to index files

[Timo Sirainen]

On 5.8.2013, at 17.52, Attila Nagy <bra at fsn.hu> wrote:

> On 08/05/13 14:47, Timo Sirainen wrote:
>> I've been planning on adding these for years. Maybe it's about time soon. I guess they could be added already to v2.2, but enabled only by a new setting because it requires file format changes that old Dovecots can't then read. I could probably patch v2.1 also so it is able to at least read the new format without failing. For v2.3 this new format could then be made the default.
>> 
> What would these solve? Pointing out errors in dovecot, operating system, or faulty hardware?
> Modern file/storage systems checksum the data all the way to the platters.

Currently most corruptions happen due to NFS or other shared filesystems. Also local filesystems can corrupt themselves in some bad conditions. So they're useful at least for:

 * figuring out if the "corruption" is actual filesystem corruption or a bug in Dovecot

 * detecting corruption early and not giving user corrupted data

 * fixing corruption in a way that doesn't cause more problems (e.g. in some bad cases IMAP UIDs could go over 2^31 causing trouble with some clients)

I hear about index corruption all the time. It's not always obvious where the problem is..