[Dovecot] Broken physical size caching in Dovecot 2.1.10

[Rob Redpath]

On 05/02/13 02:06, Timo Sirainen wrote:
> On Tue, 2013-01-29 at 19:58 +0000, Rob Redpath wrote:
>
>> I'm encountering two, probably related, errors in my Dovecot 2.1.10
>> install. The issue relates to compressed email stored in Maildir format
>> directories.
>>
>> In some cases, a mailbox will become inaccessible, and the following
>> will be logged from a doveadm fetch:-
>>
>> doveadm(info at example.com): Error: Corrupted index cache file
>> /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache: Broken
>> physical size for mail UID 2777
>> doveadm(info at example.com): Error: Cached message size smaller than
>> expected (18996 < 64624)
>> doveadm(info at example.com): Error: Maildir filename has wrong S value,
>> renamed the file from
>> /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z
>> to
>> /var/spool/virtual_mail/info_example.com_d/cur/1277451630.H877760P14612.mailserver.example.com,S=18996:2,Z
>>
>> Interestingly, the file mentioned in the error (
>> /var/spool/virtual_mail/info_example.com_d/dovecot.index.cache ) doesn't
>> exist.
>>
>> The uncompressed size of the file is 64624 bytes, the compressed size is
>> 18996.
> Dovecot's zlib plugin requires that the maildir filenames
> have ,S=uncompressed size. Otherwise you get those errors.
> http://dovecot.org/tools/maildir-size-check.sh can be used to fix the
> filenames.
>
>

Thanks for this. The problem I'm encountering is that Dovecot renames 
files, and does so to their size on disk rather than their uncompressed 
size, undoing any corrective action that's been taken. For example:-

[root at mailserver ~]# ls 
/var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ
/var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ 

[root at mailserver ~]# zcat 
/var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ 
| wc
     345    1530   24587
[root at mailserver ~]# mv 
/var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ 
/var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=24587:2,SZ 


[root at mailserver ~]# doveadm fetch -u "info at example.com" text all > 
/dev/null
<snip>
doveadm(info at example.com): Error: Maildir filename has wrong S value, 
renamed the file from 
/var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=24587:2,SZ 
to 
/var/spool/virtual_mail/info_example.com_d/cur/1287153356.H188882P6490.mailserver.example.net,S=4580:2,SZ
</snip>