fts_solr: Error: fts_solr: received invalid uid '0'

[Stephan Bosch]

On 08/10/2019 02:52, Stephan Bosch via dovecot wrote:
>
>
> On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:
>> Hi,
>>
>>> Am 23.09.2019 um 09:36 schrieb Filip Hanes <filip.hanes at ringier.sk 
>>> <mailto:filip.hanes at ringier.sk>>:
>>>
>>>
>>> št 19. 9. 2019 o 15:30 Fabian via dovecot <dovecot at dovecot.org 
>>> <mailto:dovecot at dovecot.org>> napísal(a):
>>>
>>>     Are there any other hints or tips regarding this „invalid uid
>>>     ‚0‘"-message?
>>>
>>> from fts-solr source:
>>>     case SOLR_XML_CONTENT_STATE_UID:
>>>         if (uint32_parse(str, len, &ctx->uid) < 0 || ctx->uid == 0) {
>>>             i_error("fts_solr: received invalid uid '%s'", t_strndup(str, len)); 
>>>
>>> It means that plugin tries to parse '0' when in state of parsing uid 
>>> in xml response from solr or there is a bug in xml response parser.
>>> I would enable rawlog_dir in config and look there for unusual 
>>> response or extra tags near usual uid in xml response. For example:
>>> |plugin { fts = solr fts_solr = url=... 
>>> rawlog_dir=/var/log/fts-solr/ }|
>>>
>>
>> Thanks for this tip! The rawlog_dir option is not yet supported in 
>> our version 2.3.4. What we could provide now would be the 
>> XML-Response from Solr. The corresponding search query sometimes 
>> causes this error. But Solr's answer seems to always remain the same 
>> - in my opinion this already indicates a bug of Dovecot?
>>
>> The XML response can be downloaded here: 
>> https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr
>
> Based on the XML response above, I investigated this problem 
> thoroughly and determined that this is a pretty severe bug in the Solr 
> XML response parsing code. This occurs only when the response is 
> rather large and the boundary between two read chunks falls in the 
> middle of a numeric value (that happens to end in '0').
>
> Tracking internally as DOP-1470.
>
> I think this may also explain some of the weird reports about invalid 
> UIDs that we've seen in recent history on this mailing list.

Preliminary fix is here: 
https://github.com/stephanbosch/dovecot-core/commits/fix-solr-xml-parser

It's big due to some code restructuring for making a proper unit test. I 
am not sure when this will be in a release.

Regards,

Stephan.