[Dovecot] Antwort: Antwort: Re: WG: on High Load using IMAPSYNC : Panic: file ostream-lzma.c: line 147: unreached. Dovecot 2.2.12 with zlib/XZ compression --> solved !

when using xz compression, on high load (syncing thousands of emails) dovecot panics with : Panic: file ostream-lzma.c: line 147: unreached

after some investigation I found out that the call

ret = lzma_code(zs, LZMA_FINISH);

returns LZMA_OK (what is enumerated integer 0) sometimes.

again - this error will only show up under heavy load, for instance when syncing a lot of emails to a new mailbox with imapsync or dsync.

After patching the file ostream-lzma.c as follows, it seems to work now correctly.

Someone with more knowledge about the lzma library and that codepart should probably look over it, before merging it into the code.

I did extensive testing with huge mailboxes successfully- just insert the lines marked with '++' and recompile.

-- code snipped of ostream-lzma.c, around line 124 --

    do {
            len = sizeof(zstream->outbuf) - zs->avail_out;
            if (len != 0) {
                    zs->next_out = zstream->outbuf;
                    zs->avail_out = sizeof(zstream->outbuf);

                    zstream->outbuf_used = len;
                    if ((ret = o_stream_zlib_send_outbuf(zstream)) <= 

0.                          return ret;
                    if (done)
                            break;
            }
   
            ret = lzma_code(zs, LZMA_FINISH);
            switch (ret) {
            case LZMA_STREAM_END:
                    done = TRUE;
                    break;

   ++ /* LZMA_FINISH should be called until LZMA_STREAM_END */ ++ /* got LZMA_OK on high server load when syncing through imap */ ++ case LZMA_OK: ++ break; case LZMA_MEM_ERROR: i_fatal_status(FATAL_OUTOFMEM, "lzma.write(%s): Out of memory", o_stream_get_name(&zstream->ostream.ostream)); default: ++ /* log the unexpected error code if any */ ++ i_fatal("lzma returncode when flushing : %d", ret); i_unreached(); } } while (zs->avail_out != sizeof(zstream->outbuf));

-- /code snipped --

yours sincerely

Ing. Robert Nowotny Rotek GmbH Vienna, Austria