On 3/20/23 09:52, Alessio Cecchi wrote:
I'm running a mail server with Dovecot 2.3.20 and Apache Solr as FTS backend.
It seems that iPhone/iPad Mail clients are generating some IMAP searches, especially on header Message-ID, that are increasing our Solr I/O load, especially during the night.
Are these kind of queries normal? What are they?
I don't know whether they are normal or not. That requires knowledge about IMAP that I do not have. It does seem odd that a mail client would be doing header searches unless the user is actually doing a search. I would expect it to look messages up by the ID using other imap commands, not with a header search. But my knowledge about IMAP is not complete enough to say for sure.
You can find some logs below.
Those Solr logs show what SHOULD be low-impact queries on a system that's appropriately sized.
I suspect what's happening is that the Solr server doesn't have enough free memory (memory not allocated by ANY program) to effectively cache the index.
If there is enough free memory for the OS disk cache and the cache is reasonably warmed up, queries like the ones you showed should produce very little disk I/O, because the data that they need will be sitting in memory and an actual disk read is unnecessary.
Once the search completes, the record retrievals might result in a cache miss, but the amount of data that gets retrieved for a dovecot search result is quite small. The data is compressed, so a few CPU cycles are required to decompress it.
Memory-related performance issues with Solr come up frequently enough that I wrote a wiki article about it:
https://cwiki.apache.org/confluence/display/solr/solrperformanceproblems
Thanks, Shawn