On 4/14/2019 7:59 AM, John Fawcett via dovecot wrote:
From dovecot point of view I can see the following as potentially useful features:
- a configurable batch size would enable to tune the number of emails per request and help stay under the 60 seconds hard coded http request timeout. A configurable http timeout would be less useful, since this will potentially run into other timeouts on solr side.
Even if several thousand emails are sent per batch, unless they're incredibly large, I can't imagine indexing them taking more than a few seconds. Does dovecot send attachments to Solr as well as the email itself? Hopefully it doesn't. If it does, then you would want a smaller batch size.
But if the heap size for Solr is not big enough, that can cause major delays no matter what requests are being sent, because Java will be spending most of its time doing garbage collection.
I'm also assuming that the Solr server is on the same LAN as dovecot and that transferring the update data does not take a long time.
Thanks, Shawn