Doveadm fetch slow and 100%CPU with a specific message-id

[Ron Garret]

On Oct 25, 2021, at 10:05 AM, Aki Tuomi <aki.tuomi at open-xchange.com> wrote:

> 
>> On 25/10/2021 19:40 Alessio Cecchi <alessio at skye.it> wrote:
>> 
>> 
>> Hi,
>> I'm using doveadm fetch in order to find the mailbox where a messagge is stored:
>> doveadm fetch -u alessio at domain.com "mailbox" HEADER Message-ID '1234 at domain.com'
>> If the messagge-id is ... long more than? ... I don't know, the lookup is very very slow, here an example:
>> with message-id 9c102380c557e7e146a33cb4b49ab802 at cbweb.cecchi.net respons time: 3 secs
>> with message-id kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E at www.myxmail.com response time: 80 secs and java/solr use the 100% CPU
>> Both messages are in the same folder (Trash)
>> 
>> If I add -D to doveadm it stuck some seconds every time connect to Solr:
>> 
>> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: queue http://127.0.0.1:8983: Connection to peer 127.0.0.1:8983 claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com] 
>> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: conn 127.0.0.1:8983 [1]: Claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com]
>> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: request [Req1: GET http://127.0.0.1/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain...: Sent header
>> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: peer 127.0.0.1:8983: No more requests to service for this peer (1 connections exist, 0 pending)
>> [...]
>> Oct 25 18:30:08 doveadm(alessio at domain.com): Debug: http-client: queue http://127.0.0.1:8983
>> My dovecot version is 2.3.16 and Solr 7.7.
>> 
>> Why?
>> Thanks
>> 
>> -- 
>> Alessio Cecchi
>> Postmaster @ http://www.qboxmail.it
>> https://www.linkedin.com/in/alessice
> 
> 
> I wasn't able to reproduce this issue locally. Maybe your solr config has issues?

I also tried and failed to reproduce this problem (in a folder with 1500 messages).

Note that message-ids are not guaranteed to be unique.  During my test I found groups of as many as 20 different messages with the same message ID.  (Turns out this makes quite a reliable spam signal!)

rg