Hello, same problem here after upgrading from 2.0.18 to 2.1.0, apparently it happens only on servers with qmail, not on servers with exim or dovecot as lda:

I'm using the dovecot LDA, but then it's not clear if the messages affected are REALLY old and thus might predate the use of the dovecot LDA...

Mar 2 10:18:45 mercurio dovecot: pop3(user): Error: Cached message size smaller than expected (59998 < 60150) Mar 2 10:18:45 mercurio dovecot: pop3(user): Error: Maildir filename has wrong S value, renamed the file from /home/vpopmail/domains/2/root/Maildir/cur/1330679783.8428.mercurio,S=59998:2, to /home/vpopmail/domains/2/root/Maildir/cur/1330679783.8428.mercurio,S=60150:2, Mar 2 10:18:45 mercurio dovecot: pop3(user): Error: Corrupted index cache file /home/vpopmail/domains/2/root/Maildir/dovecot.index.cache: Broken physical size for mail UID 40669 Mar 2 10:18:45 mercurio dovecot: pop3(user): Error: read(/home/vpopmail/domains/2/root/Maildir/cur/1330679783.8428.mercurio,S=59998:2,) failed: Input/output error (uid=40669)

-- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt@charite.de | http://www.charite.de

• Timo Sirainen tss@iki.fi:

On 2.3.2012, at 12.25, Ralf Hildebrandt wrote:

...

Starting with 2.1.1 we suddely encounter quite a lot of these messages:

Mar 2 11:09:28 postamt dovecot: imap(username): Error: Maildir filename has wrong S value, renamed the file from /home/a/i/username/Maildir/.A*Teens.Eink&AOQ-ufe, Spenden etc/cur/1323207735.M64829P19819.postamt.charite.de,S=5137:2,S to /home/a/i/username/Maildir/.A*Teens.Eink&AOQ-ufe, Spenden etc/cur/1323207735.M64829P19819.postamt.charite.de,S=5137:2,S .. While this has (assumedly) been working with 2.0.18.

Dovecot v2.0 didn't detect these problems, and might have truncated some mails in some situations.

COuld be!

...

Another issue with this: This fixes ONE file, and throws an error. Repeatedly accessing this folder fixes more files, until at some point all files were fixed.

Right, because after it notices a problem it disconnects the client since it can't really do anything else. Running doveadm fetch for all the mails should fix all of them.

Ah yes, good idea

Mar 2 11:39:39 postamt dovecot: imap-login: Login: user=<user>, method=PLAIN, rip=141.42.206.38, lip=141.42.206.36, mpid=28959, secured Mar 2 11:39:41 postamt dovecot: imap(user): Error: Cached message size smaller than expected (168202 < 170440) Mar 2 11:39:41 postamt dovecot: imap(user): Error: Maildir filename has wrong S value, renamed the file from /home/g/z/user/Maildir/.Partys/cur/1289296464.M845813P3466.postamt.charite.de,S=168202:2,SZ to /home/g/z/user/Maildir/.Partys/cur/1289296464.M845813P3466.postamt.charite.de,S=168202:2,SZ Mar 2 11:39:41 postamt dovecot: imap(user): Error: Corrupted index cache file /home/g/z/user/Maildir/.Partys/dovecot.index.cache: Broken physical size for mail UID 81 Mar 2 11:39:41 postamt dovecot: imap(user): Error: read(/home/g/z/user/Maildir/.Partys/cur/1289296464.M845813P3466.postamt.charite.de,S=168202:2,SZ) failed: Input/output error (uid=81) Mar 2 11:39:41 postamt dovecot: imap(user): Disconnected: Internal error occurred. Refer to server log for more information. [2012-03-02 11:39:41] in=735 out=5258

Look at that renaming operation: It simply reused the same name:

from /home/g/z/user/Maildir/.Partys/cur/1289296464.M845813P3466.postamt.charite.de,S=168202:2,SZ to /home/g/z/user/Maildir/.Partys/cur/1289296464.M845813P3466.postamt.charite.de,S=168202:2,SZ

Alternatively you can just tell Dovecot not to care about it: maildir_broken_filename_sizes=yes. Although you probably can't do that if you have compressed mails.

In the case above that mail was gzipped twice :(

-- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt@charite.de | http://www.charite.de

I'm having this problem also, with a very very few users.

But in my case the email isn't double gzip, just single like normal.

Error: read(.../.Deleted
Messages/cur/1331840112.M186676P27974.5013:2,) failed: Input/output
error (uid=250)

All I have to do is rename the file to add back the lost S= part and
all is fine. This has happened in the inbox, deleted, and trash folders so far. and
always after a change, the S= exists for new emails. It's like it's
loosing it on adding the read flag, and mailbox moves

But out of millions of emails, only a very few are like this, that I
know of, around 6 emails. I manually fixed them, will be looking to
see if this issue comes back.

Quoting Timo Sirainen tss@iki.fi:

On 2.3.2012, at 12.43, Ralf Hildebrandt wrote:

...

...

Alternatively you can just tell Dovecot not to care about it:
maildir_broken_filename_sizes=yes. Although you probably can't do
that if you have compressed mails.

In the case above that mail was gzipped twice :(

Yes, looks like Dovecot can't correctly fix the wrong S size for
gzipped mails. I don't know if I should bother fixing it, especially
since in your case the doubly-gzipped mails will look corrupted to
user..

And found two more users with this issue, but while looking at it, I
see another related issue, but it's not throwing an error.

all email in the INBOX/new and /cur are correct

but in .Trash/cur since I upgraded from 2.0.19 to 2.1 they have double
S and W tags.

1331941500.M220929P17982.5013,S=24845,W=25526,S=24845,W=25526:2,Sa

This is happening for all folder moves.

the Sent folder isn't affected, but I assume cause an email wasn't
moved in that case.

Quoting Ralf Hildebrandt Ralf.Hildebrandt@charite.de:

• Patrick Domack patrickdk@patrickdk.com:

...

I'm having this problem also, with a very very few users.

But in my case the email isn't double gzip, just single like normal.

Error: read(.../.Deleted Messages/cur/1331840112.M186676P27974.5013:2,) failed: Input/output error (uid=250)

All I have to do is rename the file to add back the lost S= part and all is fine. This has happened in the inbox, deleted, and trash folders so far. and always after a change, the S= exists for new emails. It's like it's loosing it on adding the read flag, and mailbox moves

Yes, I'm also seeing it now with mailboxes where no mail is doubly gzipped.

-- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt@charite.de | http://www.charite.de

On 20.3.2012, at 16.55, Patrick Domack wrote:

but in .Trash/cur since I upgraded from 2.0.19 to 2.1 they have double S and W tags.

1331941500.M220929P17982.5013,S=24845,W=25526,S=24845,W=25526:2,Sa

This is happening for all folder moves.

Fixed: http://hg.dovecot.org/dovecot-2.1/rev/3599790da3d7

• Timo Sirainen tss@iki.fi:

On 20.3.2012, at 16.55, Patrick Domack wrote:

...

but in .Trash/cur since I upgraded from 2.0.19 to 2.1 they have double S and W tags.

1331941500.M220929P17982.5013,S=24845,W=25526,S=24845,W=25526:2,Sa

This is happening for all folder moves.

Fixed: http://hg.dovecot.org/dovecot-2.1/rev/3599790da3d7

That doesn't seem to work:

Mar 21 15:32:50 postamt dovecot: imap(jkamp): Error: Maildir filename has wrong S value, renamed the file from /home/j/k/jkamp/Maildir/cur/1330501473.M742455P30506.postamt.charite.de,S=36307:2,S to /home/j/k/jkamp/Maildir/cur/1330501473.M742455P30506.postamt.charite.de,S=36307:2,S Mar 21 15:32:50 postamt dovecot: imap(jkamp): Error: read(/home/j/k/jkamp/Maildir/cur/1330501473.M742455P30506.postamt.charite.de,S=36307:2,S) failed: Input/output error (uid=5270)

It's renaming itself to itself again?

-- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt@charite.de | http://www.charite.de

• Timo Sirainen tss@iki.fi:

...

It's renaming itself to itself again?

Hmm. Yeah, this is a bit problematic for compressed mails. If the S=size isn't correct, Dovecot fixes it by stat()ing the file and using it as the size. And that's of course wrong. Also Dovecot can't simply remove the S=size, because the current Maildir code assumes that it always exists for compressed mails. There's no easy and efficient way to fix this.. Maybe you could just manually rename the files to have correct S=size? :) zcat file | wc should give the right size.

Right now the whole system is down because nobody can acces his/her mails due to this.

-- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt@charite.de | http://www.charite.de

Quoting Timo Sirainen tss@iki.fi:

On 21.3.2012, at 17.52, Ralf Hildebrandt wrote:

...

• Timo Sirainen tss@iki.fi:

...

...

It's renaming itself to itself again?

Hmm. Yeah, this is a bit problematic for compressed mails. If the S=size isn't correct, Dovecot fixes it by stat()ing the file and using it as the size. And that's of course wrong. Also Dovecot can't simply remove the S=size, because the current Maildir code assumes that it always exists for compressed mails. There's no easy and efficient way to fix this.. Maybe you could just manually rename the files to have correct S=size? :) zcat file | wc should give the right size.

Right now the whole system is down because nobody can acces his/her mails due to this.

All of your mails are compressed and have wrong S=size in the
filename? You can disable the check with the attached patch, but I'm
not sure if there are other places where it fails. At least quota
calculations won't be correct.

The issue only started happening since I upgraded to 2.1.1, it didn't
exist before then, I have check my system, and files before the date
of upgrade are fine, only files/emails moved after upgrading to 2.1.1
have lost the S= value.

I have made something that can pretty easily fix the issue, but it
only stays fixed till another email gets moved and looses it's S= value.

Sorry, I haven't had time to test out 2.1.3 yet.

This will print out the commands needed to fix the files though.

find . -name '*hostname:*' -exec 'gzip' '-l' '{}' ';' | awk
'/hostname/ {for(x=4;x split(fn,a,":"); print "mv \"" fn "\" \"" a[1] ",S=" $2 ":" a[2] "\"";}'

Just change hostname to dovecot is using in your files.