<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"></head><body><div>There has definetly been fixes in this area since 2.2.13 and most likely your issue is fixed.</div><div><br></div><div><br></div><div id="composer_signature"><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">---<div>Aki Tuomi</div><div>Dovecot oy</div></div><div><br></div><div style="font-size:100%;color:#000000"><!-- originalMessage --><div>-------- Original message --------</div><div>From: fauno <fauno@partidopirata.com.ar> </div><div>Date: 04/08/2018 18:44 (GMT+02:00) </div><div>To: dovecot@dovecot.org </div><div>Subject: replication fails and corrupts index with zlib enabled </div><div><br></div></div>Hi, I have two Debian Jessie servers with Dovecot 2.2.13 TCP replication<br>on that have worked fine for years, but now one of them is running low<br>on disk space, so I wanted to try enabling zlib.<br><br>I crafted a script following the description given in<br>https://wiki.dovecot.org/Plugins/Zlib and xz'ed some inboxes on the<br>stand-by server, the one with low disk space. So every email in those<br>inboxes is xz'ed but the file name hasn't changed and contains the<br>original size.<br><br>This server is on stand-by so most of the email is replicated<br>unidirectionally to it. But administrative emails like cronjobs and<br>monitoring are delivered locally, so it replicates those to the hot server.<br><br>The issue appeared when this stand-by server receives such an email and<br>tries to replicate them to the other server.<br><br>I'm attaching the full snippet of the log from the hot server, because<br>it throws a longer backtrace. The short version is like this:<br><br>dovecot[8438]: imap(redacted@address.org): Error: Cached message size<br>larger than expected (478 > 289)<br>dovecot[8438]: imap(redacted@address.org): Error: Maildir filename has<br>wrong S value, renamed the file from<br>/srv/email/address.org/redacted/Maildir/cur/1533393328.M502775P20341.standby_server,S=478:2,<br>to<br>/srv/email/address.org/redacted/Maildir/cur/1533393328.M502775P20341.standby_server,S=289:2,<br>dovecot[8438]: imap(redacted@address.org): Error: Corrupted index cache<br>file /srv/email/address.org/redacted/Maildir/dovecot.index.cache: Broken<br>physical size for mail UID 45123<br><br>After this there's an error and the replication fails. The file is<br>there, it's gzipped and can be zcat'ed but it appears as a blank email<br>on clients.<br><br>I've recovered a backup but the issue persists. I also changed from xz<br>to gz because the Debian package docs only mention gz and bzip2, but the<br>issue is the same.<br><br>From what I understand and tested, the stand-by server is receiving the<br>email and compressing it but maintaining the original size on the file<br>name. So that's ok, but when the hot server receives the copy, it<br>believes the size is wrong and changes it to the compressed size. Then<br>for some reason the index gets corrupted.<br><br>I'm attaching the doveconf for both servers. They're mostly the same,<br>and the only changes introduced were the zlib plugin and its options.<br>Also the script(s) that I used to compress the inboxes.<br><br>Am I correct? Is it an issue of replicator not understanding the emails<br>are compressed? I couldn't find anything related to zlib with<br>replication. Maybe it's something fixed in newer versions and I should<br>go that rabbit hole?<br><br>Thanks! :)<br></body></html>