v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed) [proposed patch]

[John Fawcett]

On 31/10/2020 17:55, John Fawcett wrote:
> On 31/10/2020 04:57, PGNet Dev wrote:
>> On 10/18/20 10:28 PM, Aki Tuomi wrote:
>>>>     doveadm(myuser at example.com): Panic: file mail-storage.c: line
>>>> 2112 (mailbox_get_open_status): assertion failed: (box->opened)
>> ...
>>
>>> I can reproduce your problem with the `fts lookup` command. Luckily
>>> it's equivalent to running `doveadm search`. I'll open a bug about this.
>> Can you provide any status on the bug/fix?
>>
>> Thanks.
> I can contribute a patch that solves the segfault. Unfortunately though
> fts search may be more broken than this. It does not give me search
> results, even though I see it querying solr and getting hits.
>
> diff -ur dovecot-2.3.11.3-orig/src/plugins/fts/doveadm-fts.c
> dovecot-2.3.11.3-patch/src/plugins/fts/doveadm-fts.c
> --- dovecot-2.3.11.3-orig/src/plugins/fts/doveadm-fts.c    2020-08-12
> 14:20:41.000000000 +0200
> +++ dovecot-2.3.11.3-patch/src/plugins/fts/doveadm-fts.c    2020-10-31
> 17:52:09.019388695 +0100
> @@ -47,6 +47,14 @@
>      i_array_init(&result.scores, 16);
>  
>      box = mailbox_alloc(info->ns->list, info->vname, 0);
> +    mailbox_set_reason(box,"fts search");
> +    if (mailbox_open(box) < 0) {
> +        i_error("Couldn't open mailbox: %s",
> +            mailbox_get_last_internal_error(box, NULL));
> +        doveadm_mail_failed_error(ctx, MAIL_ERROR_TEMP);
> +        return -1;
> +    }
> +
>      if (fts_backend_lookup(backend, box, ctx->search_args->args,
>                        FTS_LOOKUP_FLAG_AND_ARGS, &result) < 0) {
>          i_error("fts lookup failed");
>
> On a more minor issue, with this patch if you search for a non existent
> mailbox, it does give a segfault for a different assert, in mail-user.c
> (*user)->refcount == 1.
>
> doveadm(john at voipsupport.it): Error: Couldn't open mailbox: Mailbox
> doesn't exist: inboxx
> doveadm(john at voipsupport.it): Panic: file mail-user.c: line 229
> (mail_user_deinit): assertion failed: ((*user)->refcount == 1)
> doveadm(john at voipsupport.it): Error: Raw backtrace:
> /usr/local/lib/dovecot/libdovecot.so.0(backtrace_append+0x42)
> [0x7f35c44c3ee2] ->
> /usr/local/lib/dovecot/libdovecot.so.0(backtrace_get+0x1e)
> [0x7f35c44c3fee] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xec53e)
> [0x7f35c44ce53e] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xec581)
> [0x7f35c44ce581] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
> [0x7f35c44254ea] ->
> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x56d87)
> [0x7f35c47e4d87] -> doveadm(+0x2cb28) [0x55c0eaa57b28] ->
> doveadm(+0x2d77c) [0x55c0eaa5877c] ->
> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x21d) [0x55c0eaa5960d] ->
> doveadm(doveadm_cmd_run_ver2+0x472) [0x55c0eaa6a372] ->
> doveadm(doveadm_cmd_try_run_ver2+0x37) [0x55c0eaa6a497] ->
> doveadm(main+0x1d4) [0x55c0eaa47c54] ->
> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f35c3d34555] ->
> doveadm(+0x1d0ef) [0x55c0eaa480ef]
> Aborted
>
> John
>
After further testing, I believe the fix for doveadm fts lookup command
is ok (at least on a virtual users setup - I don't have system users to
test this with). What I stated above is wrong: it does produce search
results - I missed them in the debugging because the format is a bit
different to doveadm search, but they are there.

So unless someone with system users reports something not working, I'd
recommend the above patch to resolve the mailbox not open
assert/segfault issue.

John