Re: charset_to_utf8 assertion of remaining src_size

On 21 May 2015, at 18:47, Cassidy Larson wrote:

I've been seeing this same error as well now:

Fatal: master: service(indexer-worker): child 20979 killed with signal 6 (core not dumped - set service indexer-worker { drop_priv_before_exec=yes }) Panic: file charset-iconv.c: line 132 (charset_to_utf8): assertion failed: (*src_size - pos <= CHARSET_MAX_PENDING_BUF_SIZE)

It's only shown up on one particular user, and after tracking down the message(s) that caused it I found out that there were two identical messages with the same exact MD5/SHA hashes from our storage server in 2012 or so. Removing one of the duplicates fixed it for me. Not sure how that happened, but wonder if it's worth checking for?

What OS or Linux distro are you using?

Thanks,

-c

On Sat, May 16, 2015 at 3:32 AM, Timo Sirainen mailto:tss@iki.fi> wrote: On 16 May 2015, at 09:39, Xin Li mailto:delphij@delphij.net> wrote:

...

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512

Hi,

On 5/15/15 10:58, Xin Li wrote:

...

Hi,

I have seen the following assertion fails on my own mail server (indexer-worker):

Panic: file charset-iconv.c: line 132 (charset_to_utf8): assertion failed: (*src_size - pos <= CHARSET_MAX_PENDING_BUF_SIZE)

I've managed to narrow down the issue to one mail, and can reliably trigger the assertion with 'doveadm index -u user INBOX' with only that email.

Can you send me some mail that triggers the crash? I'm having trouble reproducing.

...

Maybe we should reset the iconv descriptor after each full conversion? i.e.:

diff -r 91bbd6753689 src/lib-mail/message-decoder.c - --- a/src/lib-mail/message-decoder.c Thu May 14 23:49:12 2015 +0300 +++ b/src/lib-mail/message-decoder.c Fri May 15 22:52:41 2015 -0700 @@ -347,6 +347,8 @@ sizeof(ctx->translation_buf)); memcpy(ctx->translation_buf, data + pos, ctx->translation_size); + } else { + charset_to_utf8_reset(ctx->charset_trans); }

I think the fix probably belongs to charset-iconv.c