fts_solr: Error: fts_solr: received invalid uid '0'

[Stephan Bosch]

On 10/10/2019 04:05, Larry Rosenman via dovecot wrote:
> Is there a way to get this as a patchset against 2.3.8?
>

https://github.com/stephanbosch/dovecot-core/tree/fix-solr-xml-parser-2.3.8

This hasn't seen any QA yet, so handle with care.

Regards,

Stephan.

>
> On Tue, Oct 8, 2019 at 12:03 PM Stephan Bosch via dovecot 
> <dovecot at dovecot.org <mailto:dovecot at dovecot.org>> wrote:
>
>
>
>     On 08/10/2019 02:52, Stephan Bosch via dovecot wrote:
>     >
>     >
>     > On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:
>     >> Hi,
>     >>
>     >>> Am 23.09.2019 um 09:36 schrieb Filip Hanes
>     <filip.hanes at ringier.sk <mailto:filip.hanes at ringier.sk>
>     >>> <mailto:filip.hanes at ringier.sk <mailto:filip.hanes at ringier.sk>>>:
>     >>>
>     >>>
>     >>> št 19. 9. 2019 o 15:30 Fabian via dovecot <dovecot at dovecot.org
>     <mailto:dovecot at dovecot.org>
>     >>> <mailto:dovecot at dovecot.org <mailto:dovecot at dovecot.org>>>
>     napísal(a):
>     >>>
>     >>>     Are there any other hints or tips regarding this „invalid uid
>     >>>     ‚0‘"-message?
>     >>>
>     >>> from fts-solr source:
>     >>>     case SOLR_XML_CONTENT_STATE_UID:
>     >>>
>             if (uint32_parse(str, len, &ctx->uid) < 0 || ctx->uid == 0) {
>     >>>
>                 i_error("fts_solr: received invalid uid '%s'", t_strndup(str, len));
>
>     >>>
>     >>> It means that plugin tries to parse '0' when in state of
>     parsing uid
>     >>> in xml response from solr or there is a bug in xml response
>     parser.
>     >>> I would enable rawlog_dir in config and look there for unusual
>     >>> response or extra tags near usual uid in xml response. For
>     example:
>     >>> |plugin { fts = solr fts_solr = url=...
>     >>> rawlog_dir=/var/log/fts-solr/ }|
>     >>>
>     >>
>     >> Thanks for this tip! The rawlog_dir option is not yet supported in
>     >> our version 2.3.4. What we could provide now would be the
>     >> XML-Response from Solr. The corresponding search query sometimes
>     >> causes this error. But Solr's answer seems to always remain the
>     same
>     >> - in my opinion this already indicates a bug of Dovecot?
>     >>
>     >> The XML response can be downloaded here:
>     >> https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr
>     >
>     > Based on the XML response above, I investigated this problem
>     > thoroughly and determined that this is a pretty severe bug in
>     the Solr
>     > XML response parsing code. This occurs only when the response is
>     > rather large and the boundary between two read chunks falls in the
>     > middle of a numeric value (that happens to end in '0').
>     >
>     > Tracking internally as DOP-1470.
>     >
>     > I think this may also explain some of the weird reports about
>     invalid
>     > UIDs that we've seen in recent history on this mailing list.
>
>     Preliminary fix is here:
>     https://github.com/stephanbosch/dovecot-core/commits/fix-solr-xml-parser
>
>     It's big due to some code restructuring for making a proper unit
>     test. I
>     am not sure when this will be in a release.
>
>     Regards,
>
>     Stephan.
>
>
>
> -- 
> Larry Rosenman http://www.lerctr.org/~ler
> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com 
> <mailto:larryrtx at gmail.com>
> US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106