zlib errors after upgrading from 2.3.10.1 to 2.3.11.3

newer
how fetch zlibed mails with doveadm

Robert Nowotny

22 Aug 2020 22 Aug '20

1:25 p.m.

I get ZLIB Errors after dovecot upgrade from 2.3.10.1 to 2.3.11.3

Thunderbird reports some error when try to save/open a bigger attachment.

Aug 21 15:27:34 lxc-imap dovecot: imap(acsida)<63870>: Error: Mailbox Sent: UID=40826: read(zlib(/home/vmail/virtualmailboxes/acsida/storage/m.2409)) failed: read(/home/vmail/virtualmailboxes/acsida/storage/m.2409) failed: Broken pipe (FETCH BODY[]) Aug 21 15:27:34 lxc-imap dovecot: imap(acsida)<63871><9A0...>: Error: Mailbox Sent: UID=40826: read(zlib(/home/vmail/virtualmailboxes/acsida/storage/m.2409)) failed: read(/home/vmail/virtualmailboxes/acsida/storage/m.2409) failed: Broken pipe (FETCH BODY[]) Aug 21 15:30:11 lxc-imap dovecot: imap(cpotzinger)<63889>: Error: Mailbox Sent: UID=29495: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Aug 21 15:30:17 lxc-imap dovecot: imap(cpotzinger)<64169>: Error: Mailbox Sent: UID=29495: read(zlib(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703)) failed: read(/home/vmail/virtualmailboxes/cpotzinger/storage/m.4703) failed: Broken pipe (FETCH BODY[]) Aug

anything I can tweak or do ?

Robert

Attachments:

attachment.html (text/html — 1.7 KB)

Show replies by date

Robert Nowotny

25 Aug 25 Aug

2:35 p.m.

New subject: zlib errors after upgrading

I get ZLIB Errors after dovecot upgrade from 2.3.10.1 to 2.3.11.3

many of those entries....

after installing 2.3.10.1 those files are still broken - but new files are working again.

anything I can tweak or do ?

Aki Tuomi

3:51 p.m.

New subject: zlib errors after upgrading

...

On 25/08/2020 14:35 Robert Nowotny rnowotny@rotek.at wrote:

I get ZLIB Errors after dovecot upgrade from 2.3.10.1 to 2.3.11.3

Aug 21 15:27:34 lxc-imap dovecot: imap(acsida)<63870>: Error: Mailbox Sent: UID=40826: read(zlib(/home/vmail/virtualmailboxes/acsida/storage/m.2409)) failed: read(/home/vmail/virtualmailboxes/acsida/storage/m.2409) failed: Broken pipe (FETCH BODY[])....

many of those entries....

after installing2.3.10.1 those files are still broken - but new files are working again.

anything I can tweak or do ?

Hi!

Is it possible your mdbox file is corrupted? Can you share doveconf -n?

Aki

Robert Nowotny

26 Aug 26 Aug

10:42 a.m.

New subject: zlib errors after upgrading

I had 4 broken mdbox files for 4 different users. After deleting them, dovecot re-indexed, now everything is ok again.

I can try to update tonight again to 2.3.11.3 and see if some broken files will appear again tomorrow.

Am 25.08.2020 um 14:51 schrieb Aki Tuomi:

...

...
On 25/08/2020 14:35 Robert Nowotny rnowotny@rotek.at wrote:

I get ZLIB Errors after dovecot upgrade from 2.3.10.1 to 2.3.11.3

Aug 21 15:27:34 lxc-imap dovecot: imap(acsida)<63870>: Error: Mailbox Sent: UID=40826: read(zlib(/home/vmail/virtualmailboxes/acsida/storage/m.2409)) failed: read(/home/vmail/virtualmailboxes/acsida/storage/m.2409) failed: Broken pipe (FETCH BODY[])....

many of those entries....

after installing2.3.10.1 those files are still broken - but new files are working again.

anything I can tweak or do ?

Hi!

Is it possible your mdbox file is corrupted? Can you share doveconf -n?

Aki

Robert Nowotny

28 Aug 28 Aug

1:10 p.m.

New subject: zlib errors after upgrading

unfortunately after re-installing 2.3.11.3, those broken-pipe errors appear again. (on different, new files) so for sure there is something broken with zlib. I will go back to2.3.10.1 now. attached is the configuration file, though it did not change between 2.3.10.1 and 2.3.11.3

Robert

Am 26.08.2020 um 09:42 schrieb Robert Nowotny:

...

I had 4 broken mdbox files for 4 different users. After deleting them, dovecot re-indexed, now everything is ok again.

I can try to update tonight again to 2.3.11.3 and see if some broken files will appear again tomorrow.

Am 25.08.2020 um 14:51 schrieb Aki Tuomi:

...
...
On 25/08/2020 14:35 Robert Nowotnyrnowotny@rotek.at wrote:

I get ZLIB Errors after dovecot upgrade from 2.3.10.1 to 2.3.11.3

Aug 21 15:27:34 lxc-imap dovecot: imap(acsida)<63870>: Error: Mailbox Sent: UID=40826: read(zlib(/home/vmail/virtualmailboxes/acsida/storage/m.2409)) failed: read(/home/vmail/virtualmailboxes/acsida/storage/m.2409) failed: Broken pipe (FETCH BODY[])....

many of those entries....

after installing2.3.10.1 those files are still broken - but new files are working again.

anything I can tweak or do ?

Hi!

Is it possible your mdbox file is corrupted? Can you share doveconf -n?

Aki

Timo Sirainen

2:15 p.m.

New subject: zlib errors after upgrading

On 28. Aug 2020, at 12.10, Robert Nowotny rnowotny@rotek.at wrote:

...

unfortunately after re-installing 2.3.11.3, those broken-pipe errors appear again. (on different, new files) so for sure there is something broken with zlib. I will go back to 2.3.10.1 now. attached is the configuration file, though it did not change between 2.3.10.1 and 2.3.11.3

Oh, this is with xz, not with gz like we first thought. There are some known problems with xz, which is why we are planning on removing it entirely. I'm not sure why v2.3.11 breaks it, although there are some changes related to it but it might be coincidence. I can't reproduce this problem though.

In any case, I'd recommend switching to zlib or to zstd in v2.3.12. (In theory you could leave the existing mails xz-compressed, but best would be to re-compress everything via dsync so old mails can be read when we eventually remove xz support.)

Robert Nowotny

5:36 p.m.

New subject: zlib errors after upgrading

ok, I did install zstd (zstd command line interface 64-bits v1.4.4, by Yann Collet)

reconfigured & compiled dovecot 2.3.10.1 again with option : with-zstd

changed the configuration file to :

zlib_save = zstd

and restarted dovecot -

now I have following log file errors :

Aug 28 16:17:39 lxc-imap dovecot: imap(rnowotny)<237930>: Error: zlib_save: Unknown handler: zstd Aug 28 16:17:44 lxc-imap dovecot: imap(rprager)<237939>: Error: zlib_save: Unknown handler: zstd Aug 28 16:17:44 lxc-imap dovecot: imap(rprager)<237940>: Error: zlib_save: Unknown handler: zstd Aug 28 16:17:44 lxc-imap dovecot: imap(rprager)<237941>: Error: zlib_save: Unknown handler: zstd ...

what I do wrong here ?

Am 28.08.2020 um 13:15 schrieb Timo Sirainen:

...

On 28. Aug 2020, at 12.10, Robert Nowotny mailto:rnowotny@rotek.at> wrote:

...
unfortunately after re-installing 2.3.11.3, those broken-pipe errors appear again. (on different, new files) so for sure there is something broken with zlib. I will go back to2.3.10.1 now. attached is the configuration file, though it did not change between 2.3.10.1 and 2.3.11.3

Oh, this is with xz, not with gz like we first thought. There are some known problems with xz, which is why we are planning on removing it entirely. I'm not sure why v2.3.11 breaks it, although there are some changes related to it but it might be coincidence. I can't reproduce this problem though.

In any case, I'd recommend switching to zlib or to zstd in v2.3.12. (In theory you could leave the existing mails xz-compressed, but best would be to re-compress everything via dsync so old mails can be read when we eventually remove xz support.)

Timo Sirainen

5:49 p.m.

New subject: zlib errors after upgrading

zstd support was added in v2.3.11 (not v2.3.12 like I mistakenly tried to say in the earlier mail).

...

On 28. Aug 2020, at 16.36, Robert Nowotny rnowotny@rotek.at wrote:

ok, I did install zstd (zstd command line interface 64-bits v1.4.4, by Yann Collet)

reconfigured & compiled dovecot 2.3.10.1 again with option : with-zstd

changed the configuration file to :

zlib_save = zstd

and restarted dovecot -

now I have following log file errors :

Aug 28 16:17:39 lxc-imap dovecot: imap(rnowotny)<237930>: Error: zlib_save: Unknown handler: zstd Aug 28 16:17:44 lxc-imap dovecot: imap(rprager)<237939>: Error: zlib_save: Unknown handler: zstd Aug 28 16:17:44 lxc-imap dovecot: imap(rprager)<237940>: Error: zlib_save: Unknown handler: zstd Aug 28 16:17:44 lxc-imap dovecot: imap(rprager)<237941>: Error: zlib_save: Unknown handler: zstd ...

what I do wrong here ?

Am 28.08.2020 um 13:15 schrieb Timo Sirainen:

...
On 28. Aug 2020, at 12.10, Robert Nowotny mailto:rnowotny@rotek.at> wrote:

...
unfortunately after re-installing 2.3.11.3, those broken-pipe errors appear again. (on different, new files) so for sure there is something broken with zlib. I will go back to 2.3.10.1 now. attached is the configuration file, though it did not change between 2.3.10.1 and 2.3.11.3

Oh, this is with xz, not with gz like we first thought. There are some known problems with xz, which is why we are planning on removing it entirely. I'm not sure why v2.3.11 breaks it, although there are some changes related to it but it might be coincidence. I can't reproduce this problem though.

In any case, I'd recommend switching to zlib or to zstd in v2.3.12. (In theory you could leave the existing mails xz-compressed, but best would be to re-compress everything via dsync so old mails can be read when we eventually remove xz support.)

Robert Nowotny

5:52 p.m.

New subject: zlib errors after upgrading

got it - I see a lot of commits zstd related, after the release 2.3.11.3

is it considered as stable ?

Am 28.08.2020 um 16:49 schrieb Timo Sirainen:

...

zstd support was added in v2.3.11 (not v2.3.12 like I mistakenly tried to say in the earlier mail).

...
On 28. Aug 2020, at 16.36, Robert Nowotny mailto:rnowotny@rotek.at> wrote:

ok, I did install zstd (zstd command line interface 64-bits v1.4.4, by Yann Collet)

reconfigured & compiled dovecot 2.3.10.1 again with option : with-zstd

changed the configuration file to :

zlib_save = zstd

and restarted dovecot -

now I have following log file errors :

Aug 28 16:17:39 lxc-imap dovecot: imap(rnowotny)<237930>: Error: zlib_save: Unknown handler: zstd Aug 28 16:17:44 lxc-imap dovecot: imap(rprager)<237939>: Error: zlib_save: Unknown handler: zstd Aug 28 16:17:44 lxc-imap dovecot: imap(rprager)<237940>: Error: zlib_save: Unknown handler: zstd Aug 28 16:17:44 lxc-imap dovecot: imap(rprager)<237941>: Error: zlib_save: Unknown handler: zstd ...

what I do wrong here ?

Am 28.08.2020 um 13:15 schrieb Timo Sirainen:

...
On 28. Aug 2020, at 12.10, Robert Nowotny mailto:rnowotny@rotek.at> wrote:

...
unfortunately after re-installing 2.3.11.3, those broken-pipe errors appear again. (on different, new files) so for sure there is something broken with zlib. I will go back to2.3.10.1 now. attached is the configuration file, though it did not change between 2.3.10.1 and 2.3.11.3

Oh, this is with xz, not with gz like we first thought. There are some known problems with xz, which is why we are planning on removing it entirely. I'm not sure why v2.3.11 breaks it, although there are some changes related to it but it might be coincidence. I can't reproduce this problem though.

In any case, I'd recommend switching to zlib or to zstd in v2.3.12. (In theory you could leave the existing mails xz-compressed, but best would be to re-compress everything via dsync so old mails can be read when we eventually remove xz support.)

Robert Nowotny

31 Aug 31 Aug

1:22 p.m.

New subject: zlib errors after upgrading

ok, I installed zstd successfully, it works - but how i re-compress everything via dsync ? (on the same server?)

I did :

sudo doveadm backup -D -u "${mailbox_username}" "mdbox:/home/vmail/virtualmailboxes/${mailbox_username}_backup" sudo service dovecot stop sudo mv "/home/vmail/virtualmailboxes/${mailbox_username}" "/home/vmail/virtualmailboxes/${mailbox_username}_original" sudo mv "/home/vmail/virtualmailboxes/${mailbox_username}_backup" "/home/vmail/virtualmailboxes/${mailbox_username}" sudo service dovecot start

but somehow that does not work correctly, thunderbird claims that mailboxes are not existing, etc ...

so what is the correct way to recompress the mailboxes locally ?

Robert

Am 28.08.2020 um 13:15 schrieb Timo Sirainen:

...

On 28. Aug 2020, at 12.10, Robert Nowotny mailto:rnowotny@rotek.at> wrote:

...
unfortunately after re-installing 2.3.11.3, those broken-pipe errors appear again. (on different, new files) so for sure there is something broken with zlib. I will go back to2.3.10.1 now. attached is the configuration file, though it did not change between 2.3.10.1 and 2.3.11.3

Oh, this is with xz, not with gz like we first thought. There are some known problems with xz, which is why we are planning on removing it entirely. I'm not sure why v2.3.11 breaks it, although there are some changes related to it but it might be coincidence. I can't reproduce this problem though.

In any case, I'd recommend switching to zlib or to zstd in v2.3.12. (In theory you could leave the existing mails xz-compressed, but best would be to re-compress everything via dsync so old mails can be read when we eventually remove xz support.)

1537

Age (days ago)

1546

Last active (days ago)

List overview

9 comments

3 participants

participants (3)

Aki Tuomi
Robert Nowotny
Timo Sirainen

zlib errors after upgrading from 2.3.10.1 to 2.3.11.3

tags

participants (3)