The search time was no better with it on than off.
So I'm thinking I got something misconfigured somewhere. It seems IMAP may not be using solr to fetch results. But this would be odd since I definitely do see a big improvements in times with fts plugins turned on when using roundcube.
OK, I'm finally getting somewhere. Found this lead here: https://dovecot.org/pipermail/dovecot/2012-February/081514.html
"doveadm fts optimize" throws an error:
doveadm(root): Error: Couldn't drop privileges: User is missing UID (see mail_uid setting)
Checking /var/log/mail.err, I see a ton of these entries:
83353 Aug 25 14:53:22 email dovecot: imap(s)<26536><gJ1Iw2bKKLh/AAAB>: Error: fts_solr: received invalid uid ' 83354 Aug 25 14:53:22 email dovecot: imap: Error: ' 83355 Aug 25 14:58:05 email dovecot: imap(s)<26637><Awma1GbKprh/AAAB>: Error: fts_solr: received invalid uid ' 83356 Aug 25 14:58:05 email dovecot: imap: Error: ' 83357 Aug 25 15:26:58 email dovecot: imap(s)<27217><TF0qOGfKort/AAAB>: Error: fts_solr: received invalid uid ' 83358 Aug 25 15:26:58 email dovecot: imap: Error: ' 83359 Aug 25 15:27:11 email dovecot: imap(s)<27217><TF0qOGfKort/AAAB>: Error: fts_solr: received invalid uid ' 83360 Aug 25 15:27:11 email dovecot: imap: Error: ' 83361 Aug 25 15:27:36 email dovecot: imap(s)<27217><TF0qOGfKort/AAAB>: Error: fts_solr: received invalid uid ' 83362 Aug 25 15:27:36 email dovecot: imap: Error: '
I think this will be nailed once I figure out this issue.