Re: fts_solr: Error: fts_solr: received invalid uid '0'

On 10/10/2019 04:05, Larry Rosenman via dovecot wrote:

Is there a way to get this as a patchset against 2.3.8?

https://github.com/stephanbosch/dovecot-core/tree/fix-solr-xml-parser-2.3.8

This hasn't seen any QA yet, so handle with care.

Regards,

Stephan.

On Tue, Oct 8, 2019 at 12:03 PM Stephan Bosch via dovecot mailto:dovecot@dovecot.org> wrote:

On 08/10/2019 02:52, Stephan Bosch via dovecot wrote:
>
>
> On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:
>> Hi,
>>
>>> Am 23.09.2019 um 09:36 schrieb Filip Hanes
<filip.hanes@ringier.sk <mailto:filip.hanes@ringier.sk>
>>> <mailto:filip.hanes@ringier.sk <mailto:filip.hanes@ringier.sk>>>:
>>>
>>>
>>> št 19. 9. 2019 o 15:30 Fabian via dovecot <dovecot@dovecot.org
<mailto:dovecot@dovecot.org>
>>> <mailto:dovecot@dovecot.org <mailto:dovecot@dovecot.org>>>
napísal(a):
>>>
>>>     Are there any other hints or tips regarding this „invalid uid
>>>     ‚0‘"-message?
>>>
>>> from fts-solr source:
>>>     case SOLR_XML_CONTENT_STATE_UID:
>>>
        if (uint32_parse(str, len, &ctx->uid) < 0 || ctx->uid == 0) {
>>>
            i_error("fts_solr: received invalid uid '%s'", t_strndup(str, len));

>>>
>>> It means that plugin tries to parse '0' when in state of
parsing uid
>>> in xml response from solr or there is a bug in xml response
parser.
>>> I would enable rawlog_dir in config and look there for unusual
>>> response or extra tags near usual uid in xml response. For
example:
>>> |plugin { fts = solr fts_solr = url=...
>>> rawlog_dir=/var/log/fts-solr/ }|
>>>
>>
>> Thanks for this tip! The rawlog_dir option is not yet supported in
>> our version 2.3.4. What we could provide now would be the
>> XML-Response from Solr. The corresponding search query sometimes
>> causes this error. But Solr's answer seems to always remain the
same
>> - in my opinion this already indicates a bug of Dovecot?
>>
>> The XML response can be downloaded here:
>> https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr
>
> Based on the XML response above, I investigated this problem
> thoroughly and determined that this is a pretty severe bug in
the Solr
> XML response parsing code. This occurs only when the response is
> rather large and the boundary between two read chunks falls in the
> middle of a numeric value (that happens to end in '0').
>
> Tracking internally as DOP-1470.
>
> I think this may also explain some of the weird reports about
invalid
> UIDs that we've seen in recent history on this mailing list.

Preliminary fix is here:
https://github.com/stephanbosch/dovecot-core/commits/fix-solr-xml-parser

It's big due to some code restructuring for making a proper unit
test. I
am not sure when this will be in a release.

Regards,

Stephan.

-- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx@gmail.com mailto:larryrtx@gmail.com US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106