[Dovecot] Cached message size larger than expected
Hi guys,
I recently turned on IMAP zlib compression for Maildir and noticed that I've started getting these errors in the maillog.
Error: Cached message size larger than expected (1627 > 1548) Error: Maildir filename has wrong S value, renamed the file from /home/crc.id.au/netwiz/.System Generated.Cron Jobs/cur/1392224150.M52270P1073.mail.crc.id.au,S=1627:2,S,Z to /home/crc. id.au/netwiz/.System Generated.Cron Jobs/cur/1392224150.M52270P1073.mail.crc.id.au,S=1627:2,S,Z Error: Corrupted index cache file /home/crc.id.au/netwiz/.System Generated.Cron Jobs/dovecot.index.cache: Broken physical size for mail UID 10916 Error: read(zlib(/home/crc.id.au/netwiz/.System Generated.Cron Jobs/cur/1392224150.M52270P1073.mail.crc.id.au,S=1627:2,S,Z)) failed: Invalid argument Disconnected: Internal error occurred. Refer to server log for more information. [2014-02-14 15:08:20]
For these emails, I can no longer retrieve the contents of those messages in my mail client.
I noticed a thread similar to this for Dovecot 2.1.3 (or so) - however I'm getting the same on 2.2.12.
Has there been any solutions for this?
-- Steven Haigh
Email: netwiz@crc.id.au Web: https://www.crc.id.au Phone: (03) 9001 6090 - 0412 935 897 Fax: (03) 8338 0299
On 14/02/14 15:20, Steven Haigh wrote:
Hi guys,
I recently turned on IMAP zlib compression for Maildir and noticed that I've started getting these errors in the maillog.
Error: Cached message size larger than expected (1627 > 1548) Error: Maildir filename has wrong S value, renamed the file from /home/crc.id.au/netwiz/.System Generated.Cron Jobs/cur/1392224150.M52270P1073.mail.crc.id.au,S=1627:2,S,Z to /home/crc. id.au/netwiz/.System Generated.Cron Jobs/cur/1392224150.M52270P1073.mail.crc.id.au,S=1627:2,S,Z Error: Corrupted index cache file /home/crc.id.au/netwiz/.System Generated.Cron Jobs/dovecot.index.cache: Broken physical size for mail UID 10916 Error: read(zlib(/home/crc.id.au/netwiz/.System Generated.Cron Jobs/cur/1392224150.M52270P1073.mail.crc.id.au,S=1627:2,S,Z)) failed: Invalid argument Disconnected: Internal error occurred. Refer to server log for more information. [2014-02-14 15:08:20]
For these emails, I can no longer retrieve the contents of those messages in my mail client.
I noticed a thread similar to this for Dovecot 2.1.3 (or so) - however I'm getting the same on 2.2.12.
Has there been any solutions for this?
Looking further into this, it seems like the file has been compressed twice:
# file 1392222687.M581212P24455.mail.crc.id.au\,S\=2165\:2\,S\,Z 1392222687.M581212P24455.mail.crc.id.au,S=2165:2,S,Z: gzip compressed data, was "1392222687.M581212P24455.mail.c", from Unix, last modified: Thu Feb 13 03:31:27 2014, max compression
# gzip -l 1392222687.M581212P24455.mail.crc.id.au\,S\=2165\:2\,S\,Z compressed uncompressed ratio uncompressed_name 2165 2084 0.1% 1392222687.M581212P24455.mail.crc.id.au,S=2165:2,S,Z
If I use the following, I get the actual body of the email content. # gunzip -c 1392222687.M581212P24455.mail.crc.id.au\,S\=2165\:2\,S\,Z | gunzip -c
This doesn't seem right for mail that has been sent, or moved from the inbox to the trash folder...
Is it possible that when the mail is moved between folders, it is somehow compressed a second time?
-- Steven Haigh
Email: netwiz@crc.id.au Web: https://www.crc.id.au Phone: (03) 9001 6090 - 0412 935 897 Fax: (03) 8338 0299
On 14.2.2014, at 13.32, Steven Haigh netwiz@crc.id.au wrote:
Looking further into this, it seems like the file has been compressed twice:
This doesn't seem right for mail that has been sent, or moved from the inbox to the trash folder...
Is it possible that when the mail is moved between folders, it is somehow compressed a second time?
Can you reproduce this by copying/moving a mail? I can't. What's your doveconf -n output?
On 14/02/14 15:39, Timo Sirainen wrote:
On 14.2.2014, at 13.32, Steven Haigh netwiz@crc.id.au wrote:
Looking further into this, it seems like the file has been compressed twice:
This doesn't seem right for mail that has been sent, or moved from the inbox to the trash folder...
Is it possible that when the mail is moved between folders, it is somehow compressed a second time?
Can you reproduce this by copying/moving a mail? I can't. What's your doveconf -n output?
I'm digging a bit futher... I can't get the trash folder to update at all now via thunderbird - I'm trying to get a common method - but can't quite figure it out.
Config attached.
I'm wondering... After I enabled compression on store, I ran a script that went through and compressed files with gzip and compressed existing content - also attached.
It seemed to make sense while reading through the script, so I'm not quite sure if its something I missed either...
-- Steven Haigh
Email: netwiz@crc.id.au Web: https://www.crc.id.au Phone: (03) 9001 6090 - 0412 935 897 Fax: (03) 8338 0299
On 14/02/14 15:49, Steven Haigh wrote:
On 14/02/14 15:39, Timo Sirainen wrote:
On 14.2.2014, at 13.32, Steven Haigh netwiz@crc.id.au wrote:
Looking further into this, it seems like the file has been compressed twice:
This doesn't seem right for mail that has been sent, or moved from the inbox to the trash folder...
Is it possible that when the mail is moved between folders, it is somehow compressed a second time?
Can you reproduce this by copying/moving a mail? I can't. What's your doveconf -n output?
I'm digging a bit futher... I can't get the trash folder to update at all now via thunderbird - I'm trying to get a common method - but can't quite figure it out.
Config attached.
I'm wondering... After I enabled compression on store, I ran a script that went through and compressed files with gzip and compressed existing content - also attached.
It seemed to make sense while reading through the script, so I'm not quite sure if its something I missed either...
I *think* I may have gotten to the bottom of this... I believe that the locking in the script didn't work - then while the compression script was running, dovecot delivered mail into that mailbox. This was then compressed a second time by the compression script.
Narrowing down to the specific emails and times that the messages causing problems were processed by Dovecot - these were all done when the script file was being run.
As such, this is certainly not a dovecot problem and I apologise for the noise on the list :)
-- Steven Haigh
Email: netwiz@crc.id.au Web: https://www.crc.id.au Phone: (03) 9001 6090 - 0412 935 897 Fax: (03) 8338 0299
participants (2)
-
Steven Haigh
-
Timo Sirainen