Doveadm fetch slow and 100%CPU with a specific message-id
Hi,
I'm using doveadm fetch in order to find the mailbox where a messagge is stored:
doveadm fetch -u alessio@domain.com "mailbox" HEADER Message-ID '1234@domain.com'
If the messagge-id is ... long more than? ... I don't know, the lookup is very very slow, here an example:
with message-id 9c102380c557e7e146a33cb4b49ab802@cbweb.cecchi.net respons time: 3 secs
with message-id kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com response time: 80 secs and java/solr use the 100% CPU
Both messages are in the same folder (Trash)
If I add -D to doveadm it stuck some seconds every time connect to Solr:
Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: queue http://127.0.0.1:8983: Connection to peer 127.0.0.1:8983 claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com]
Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: conn 127.0.0.1:8983 [1]: Claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: request [Req1: GET http://127.0.0.1/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain...: Sent header Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: peer 127.0.0.1:8983: No more requests to service for this peer (1 connections exist, 0 pending)
[...]
Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: queue http://127.0.0.1:8983
My dovecot version is 2.3.16 and Solr 7.7.
Why?
Thanks
-- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
On 25/10/2021 19:40 Alessio Cecchi alessio@skye.it wrote:
Hi, I'm using doveadm fetch in order to find the mailbox where a messagge is stored: doveadm fetch -u alessio@domain.com "mailbox" HEADER Message-ID '1234@domain.com' If the messagge-id is ... long more than? ... I don't know, the lookup is very very slow, here an example: with message-id 9c102380c557e7e146a33cb4b49ab802@cbweb.cecchi.net respons time: 3 secs with message-id kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com response time: 80 secs and java/solr use the 100% CPU Both messages are in the same folder (Trash)
If I add -D to doveadm it stuck some seconds every time connect to Solr:
Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: queue http://127.0.0.1:8983: Connection to peer 127.0.0.1:8983 claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: conn 127.0.0.1:8983 [1]: Claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: request [Req1: GET http://127.0.0.1/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain...: Sent header Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: peer 127.0.0.1:8983: No more requests to service for this peer (1 connections exist, 0 pending) [...] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: queue http://127.0.0.1:8983 My dovecot version is 2.3.16 and Solr 7.7.
Why? Thanks
-- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
I wasn't able to reproduce this issue locally. Maybe your solr config has issues?
Aki
On Oct 25, 2021, at 10:05 AM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 25/10/2021 19:40 Alessio Cecchi alessio@skye.it wrote:
Hi, I'm using doveadm fetch in order to find the mailbox where a messagge is stored: doveadm fetch -u alessio@domain.com "mailbox" HEADER Message-ID '1234@domain.com' If the messagge-id is ... long more than? ... I don't know, the lookup is very very slow, here an example: with message-id 9c102380c557e7e146a33cb4b49ab802@cbweb.cecchi.net respons time: 3 secs with message-id kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com response time: 80 secs and java/solr use the 100% CPU Both messages are in the same folder (Trash)
If I add -D to doveadm it stuck some seconds every time connect to Solr:
Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: queue http://127.0.0.1:8983: Connection to peer 127.0.0.1:8983 claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: conn 127.0.0.1:8983 [1]: Claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: request [Req1: GET http://127.0.0.1/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain...: Sent header Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: peer 127.0.0.1:8983: No more requests to service for this peer (1 connections exist, 0 pending) [...] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: queue http://127.0.0.1:8983 My dovecot version is 2.3.16 and Solr 7.7.
Why? Thanks
-- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
I wasn't able to reproduce this issue locally. Maybe your solr config has issues?
I also tried and failed to reproduce this problem (in a folder with 1500 messages).
Note that message-ids are not guaranteed to be unique. During my test I found groups of as many as 20 different messages with the same message ID. (Turns out this makes quite a reliable spam signal!)
rg
On Mon, 25 Oct 2021, Ron Garret wrote:
Note that message-ids are not guaranteed to be unique. During my test I found groups of as many as 20 different messages with the same message ID. (Turns out this makes quite a reliable spam signal!)
It's by far not a rare situation: duplicate message-ids happen whenever the sender names more than one local recipient during SMTP. It's a wholly unreliable way to indicates spaminess. However, if a high proportion of those recipients do not exist, ...
Joseph Tam jtam.home@gmail.com
On Oct 25, 2021, at 2:23 PM, Joseph Tam jtam.home@gmail.com wrote:
On Mon, 25 Oct 2021, Ron Garret wrote:
Note that message-ids are not guaranteed to be unique. During my test I found groups of as many as 20 different messages with the same message ID. (Turns out this makes quite a reliable spam signal!)
It's by far not a rare situation: duplicate message-ids happen whenever the sender names more than one local recipient during SMTP. It's a wholly unreliable way to indicates spaminess. However, if a high proportion of those recipients do not exist, ...
I think you may have misunderstood. What you say isn’t wrong, but in the case of multiple local SMTP recipients, all of the duplicate messages will have the same content. What I have found is the same message ID in messages with (very) *different* content (and often sent to the same user). All of that has been spam (and it is hard to imagine any situation in which it would not be).
rg
On Mon, 25 Oct 2021, Ron Garret wrote:
Note that message-ids are not guaranteed to be unique. During my test I found groups of as many as 20 different messages with the same message ID. (Turns out this makes quite a reliable spam signal!)
It's by far not a rare situation: duplicate message-ids happen whenever the sender names more than one local recipient during SMTP. It's a wholly unreliable way to indicates spaminess. However, if a high proportion of those recipients do not exist, ...
I think you may have misunderstood. What you say isn?t wrong, but in the case of multiple local SMTP recipients, all of the duplicate messages will have the same content. What I have found is the same message ID in messages with (very) *different* content (and often sent to the same user). All of that has been spam (and it is hard to imagine any situation in which it would not be).
Ah, that is a different situation. It could happen if the same message tooks different paths to your user e.g. via mailing list processor, but that is less common and would probably break DKIM.
Joseph Tam jtam.home@gmail.com
On Oct 25, 2021, at 4:10 PM, Joseph Tam jtam.home@gmail.com wrote:
On Mon, 25 Oct 2021, Ron Garret wrote:
Note that message-ids are not guaranteed to be unique. During my test I found groups of as many as 20 different messages with the same message ID. (Turns out this makes quite a reliable spam signal!)
It's by far not a rare situation: duplicate message-ids happen whenever the sender names more than one local recipient during SMTP. It's a wholly unreliable way to indicates spaminess. However, if a high proportion of those recipients do not exist, ...
I think you may have misunderstood. What you say isn?t wrong, but in the case of multiple local SMTP recipients, all of the duplicate messages will have the same content. What I have found is the same message ID in messages with (very) *different* content (and often sent to the same user). All of that has been spam (and it is hard to imagine any situation in which it would not be).
Ah, that is a different situation. It could happen if the same message tooks different paths to your user e.g. via mailing list processor, but that is less common and would probably break DKIM.
I see I still haven’t made myself clear. I’m talking about messages with the same message ID with *radically* different content. Different subject lines. Different senders. Different lengths. Different MIME message structures. All with the same message-id and all sent to the same user.
rg
Il 25/10/21 19:05, Aki Tuomi ha scritto:
On 25/10/2021 19:40 Alessio Cecchi alessio@skye.it wrote:
Hi, I'm using doveadm fetch in order to find the mailbox where a messagge is stored: doveadm fetch -u alessio@domain.com "mailbox" HEADER Message-ID '1234@domain.com' If the messagge-id is ... long more than? ... I don't know, the lookup is very very slow, here an example: with message-id 9c102380c557e7e146a33cb4b49ab802@cbweb.cecchi.net respons time: 3 secs with message-id kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com response time: 80 secs and java/solr use the 100% CPU Both messages are in the same folder (Trash)
If I add -D to doveadm it stuck some seconds every time connect to Solr:
Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: queue http://127.0.0.1:8983: Connection to peer 127.0.0.1:8983 claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: conn 127.0.0.1:8983 [1]: Claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: request [Req1: GET http://127.0.0.1/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain...: Sent header Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: peer 127.0.0.1:8983: No more requests to service for this peer (1 connections exist, 0 pending) [...] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: queue http://127.0.0.1:8983 My dovecot version is 2.3.16 and Solr 7.7.
Why? Thanks
-- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
I wasn't able to reproduce this issue locally. Maybe your solr config has issues?
Aki
Yes, could be my Solr setup. I'll investigate.
Is there a way to disable fts/solr when using doveadm? Like -o "plugin/fts="
Thanks
-- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
Il 25/10/21 21:33, Alessio Cecchi ha scritto:
Il 25/10/21 19:05, Aki Tuomi ha scritto:
On 25/10/2021 19:40 Alessio Cecchi alessio@skye.it wrote:
Hi, I'm using doveadm fetch in order to find the mailbox where a messagge is stored: doveadm fetch -u alessio@domain.com "mailbox" HEADER Message-ID '1234@domain.com' If the messagge-id is ... long more than? ... I don't know, the lookup is very very slow, here an example: with message-id 9c102380c557e7e146a33cb4b49ab802@cbweb.cecchi.net respons time: 3 secs with message-id kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com response time: 80 secs and java/solr use the 100% CPU Both messages are in the same folder (Trash)
If I add -D to doveadm it stuck some seconds every time connect to Solr:
Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: queue http://127.0.0.1:8983: Connection to peer 127.0.0.1:8983 claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: conn 127.0.0.1:8983 [1]: Claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: request [Req1: GET http://127.0.0.1/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain...: Sent header Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: peer 127.0.0.1:8983: No more requests to service for this peer (1 connections exist, 0 pending) [...] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: queue http://127.0.0.1:8983 My dovecot version is 2.3.16 and Solr 7.7.
Why? Thanks
-- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
I wasn't able to reproduce this issue locally. Maybe your solr config has issues?
Aki
Yes, could be my Solr setup. I'll investigate.
Is there a way to disable fts/solr when using doveadm? Like -o "plugin/fts="
Yes, with:
doveadm -o "plugin/fts=" fetch -u alessio@...
the time needed to lookup the message is now around 3 seconds for all query.
But, is possible to use the -o "plugin/fts=" in doveadm http API?
Because I run doveadm via http API with curl.
Thanks
-- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
On 25/10/2021 22:43 Alessio Cecchi alessio@skye.it wrote:
Il 25/10/21 21:33, Alessio Cecchi ha scritto:
Il 25/10/21 19:05, Aki Tuomi ha scritto:
On 25/10/2021 19:40 Alessio Cecchi alessio@skye.it wrote:
Hi, I'm using doveadm fetch in order to find the mailbox where a messagge is stored: doveadm fetch -u alessio@domain.com "mailbox" HEADER Message-ID '1234@domain.com' If the messagge-id is ... long more than? ... I don't know, the lookup is very very slow, here an example: with message-id 9c102380c557e7e146a33cb4b49ab802@cbweb.cecchi.net respons time: 3 secs with message-id kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com response time: 80 secs and java/solr use the 100% CPU Both messages are in the same folder (Trash)
If I add -D to doveadm it stuck some seconds every time connect to Solr:
Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: queue http://127.0.0.1:8983: Connection to peer 127.0.0.1:8983 claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: conn 127.0.0.1:8983 [1]: Claimed request [Req1: GET http://127.0.0.1:8983/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain.com] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: request [Req1: GET http://127.0.0.1/solr/dovecot/select?wt=xml&fl=uid,score&rows=26&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dhdr:kZ1ZoAa8QnsfZ64hTe9P3K0OOjL24xTq7vUMB3Qw3E@www.myxmail.com&fq=%2Bbox:13c0e32ee6430860201f0000c5b62527+%2Buser:alessio@domain...: Sent header Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: peer 127.0.0.1:8983: No more requests to service for this peer (1 connections exist, 0 pending) [...] Oct 25 18:30:08 doveadm(alessio@domain.com): Debug: http-client: queue http://127.0.0.1:8983 My dovecot version is 2.3.16 and Solr 7.7.
Why? Thanks
-- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
I wasn't able to reproduce this issue locally. Maybe your solr config has issues?
Aki
Yes, could be my Solr setup. I'll investigate.
Is there a way to disable fts/solr when using doveadm? Like -o "plugin/fts="
Yes, with:
doveadm -o "plugin/fts=" fetch -u alessio@...
the time needed to lookup the message is now around 3 seconds for all query.
But, is possible to use the -o "plugin/fts=" in doveadm http API?
Because I run doveadm via http API with curl.
Thanks
-- Alessio Cecchi
It's not possible.
Aki
participants (4)
-
Aki Tuomi
-
Alessio Cecchi
-
Joseph Tam
-
Ron Garret