Hello,
I started full reindex of whole emails base (doveadm index -A '*') and solr crashed due to out of memory. I had plenty of such messages:
doveadm(gzt): Error: fts_solr: Indexing failed: Connection lost: read((conn:127.0.0.1:8983,id=621)) failed: EOF (Request queued 40.832 secs ago, 1 send attempts in 39.359 secs, 0.002 in http ioloop, 0.000 in other ioloops, connected 704.138 secs ago) doveadm(gzt): Error: Mailbox Sent: Precache for UID=4036 failed: Internal error occurred. Refer to server log for more information. [2023-05-20 06:01:40] doveadm(gzt): Error: Mailbox Sent: Transaction commit failed: FTS transaction commit failed: backend deinit
I increased solr memory and restarted it. Do I have to restart dovedam index? The "request queued" message suggest that Dovecot is smart and I don't need to do anything. Am I right? (Any way to confirm that indeed everything was indexed?)
-- Przemyslaw Kwiatkowski
On 5/20/23 00:13, Przemysław Kwiatkowski via dovecot wrote:
I started full reindex of whole emails base (doveadm index -A '*') and solr crashed due to out of memory. I had plenty of such messages:
If you are seeing Java's "OutOfMemoryError" exception, you must figure out what resource was depleted. It actually might not be memory. If it's not, then increasing the Solr heap will not help.
I increased solr memory and restarted it. Do I have to restart dovedam index? The "request queued" message suggest that Dovecot is smart and I don't need to do anything. Am I right?
I would guess that you DO need to restart the reindex. I have no idea how long dovecot will queue the updates it sends. I would bet on it being something like 30 or 60 seconds.
Here is a script that I wrote to do a full dovecot reindex to Solr:
https://paste.elyograg.org/view/f183f91c
One caveat: I had to comment lines 50 through 52 because SolrCloud refuses to reload an alias. It has to be an actual collection. If you're not in cloud mode you don't need to worry about it all. If you are in cloud mode but using an actual collection name, then you can uncomment those lines and delete line 53.
This script has been tested with 9.x Solr versions. I am not sure whether it will work properly on earlier Solr versions.
Thanks, Shawn
participants (2)
-
Przemysław Kwiatkowski
-
Shawn Heisey