What kind of search response time are you setting with solr full text search?
MY SETUP: I have apache solr full text search enabled with dovecot. I have an inbox with about 40 subfolders. I'm using the roundcube web-based mail client. The find command is showing 15823 email files and apache solr reports the same number. I'm running a dedicated mail server with a 1 GB of ram. The solr machine is running on a separate machine with 4 GB ram.
THE PROBLEM: When I do a full text search through all my inbox and all subfolders on a single word, search results are returned in about 10 to 15 seconds. This is better than the 40 seconds or so I'm getting when I turn off the fts and fts_solr plugins but still a little disappointing.
WHAT I'D LIKE: Now, I don't expect instant search results like gmail, but getting the search results to display in less than 5 seconds would be a lot less painful that 10 to 15 seconds.
WHAT I'VE TRIED: I have reindexed the emails on solr to try to speed things upit up.
ANY ADVICE? Curious to know what response times others are seeing and if there is anything else I might try to speed things up. Maybe the number of subfolders is slowing things down?
THE PROBLEM: When I do a full text search through all my inbox and all subfolders on a single word, search results are returned in about 10 to 15 seconds. This is better than the 40 seconds or so I'm getting when I turn off the fts and fts_solr plugins but still a little disappointing.
I did some experimenting. I noticed that if the word I'm searching on is fairly rare, results will pop up quickly, like in around 3 to 5 seconds. Words that don't exist at all in any email returns nothing almost instantly.
But words that appear in several hundred emails are the ones that are take a much longer time.
Not sure if this just might be a slow email client or due to Dovecot itself or if maybe 1 GB of ram isn't enough and my machine is underpowered.
On 8/25/21 9:19 AM, Steve Dondley wrote:
I did some experimenting. I noticed that if the word I'm searching on is fairly rare, results will pop up quickly, like in around 3 to 5 seconds. Words that don't exist at all in any email returns nothing almost instantly.
But words that appear in several hundred emails are the ones that are take a much longer time.
Can this be reduced to: getting more search results takes longer?
On 8/24/2021 7:19 PM, Steve Dondley wrote:
THE PROBLEM: When I do a full text search through all my inbox and all subfolders on a single word, search results are returned in about 10 to 15 seconds. This is better than the 40 seconds or so I'm getting when I turn off the fts and fts_solr plugins but still a little disappointing.
I did some experimenting. I noticed that if the word I'm searching on is fairly rare, results will pop up quickly, like in around 3 to 5 seconds. Words that don't exist at all in any email returns nothing almost instantly.
But words that appear in several hundred emails are the ones that are take a much longer time.
This is offtopic for this list, but I will try to help you. If I am unsuccessful, you should raise the issue on the solr-users mailing list.
How much of the total server memory of 4GB did you give to Solr for its heap? Is there other software running on that server besides Solr?
What's the total size of all the Solr indexes on the Solr server?
Can you get the screenshot mentioned at the following URL, put it on a file-sharing site, and give me the URL?
https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems#Sol...
(disclaimer: I wrote that Solr wiki page)
You should read the entire page, the link above is to the section describing useful screenshots)
General note: A Solr search that takes 3 seconds (let alone 15) would have me concerned. If the system is sized appropriately, I would expect a search even on a massive index to complete in less than a second.
I happen to be using Solr for dovecot myself. If I search my index for "the" which is very common in English text, the query takes 19 milliseconds, and that is searching on five fields, as well as doing a facet on the user field. My Solr index has 150048 messages (122K of those are in my personal mailbox) and takes up 628 megabytes of disk space. The total size of the email that is indexed is 7 gigabytes.
|<str name="parsedquery_toString">+(cc:the | from:the | to:the | body:the | subject:the)</str> |
My index is using the stopword filter but the list of stopwords is empty.
The following response may interest you:
This is a search for "a" which I had run several times, so Solr was serving it from its cache, and this time it only took 6 milliseconds. It also shows what a facet can do. The longest time I got for the "a" search was 15 milliseconds, before the query was in the cache.
Thanks, Shawn
This is offtopic for this list, but I will try to help you. If I am unsuccessful, you should raise the issue on the solr-users mailing list.
Thanks so much. Greatly appreciated.
How much of the total server memory of 4GB did you give to Solr for its heap? Is there other software running on that server besides Solr?
I'm not sure and didn't know I could assign memory to solr. This is the first time I've used solr; I'm totally inexperienced with it. I've attached a screenshot that hopefully helps answer your question.
This machine is dedicated to solr and runs no other software as a daemon not related to the OS.
What's the total size of all the Solr indexes on the Solr server?
du -shc on /var/solr/data reports 781 M of total data.
du -sch on the core I'm using is only 116 M. As of now, I have 15K documents with about 2K deleted in the core.
Can you get the screenshot mentioned at the following URL, put it on a file-sharing site, and give me the URL?
https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems#Sol...
Attached.
(disclaimer: I wrote that Solr wiki page)
You should read the entire page, the link above is to the section describing useful screenshots)
OK, I will.
General note: A Solr search that takes 3 seconds (let alone 15) would have me concerned. If the system is sized appropriately, I would expect a search even on a massive index to complete in less than a second.
I happen to be using Solr for dovecot myself. If I search my index for "the" which is very common in English text, the query takes 19 milliseconds, and that is searching on five fields, as well as doing a facet on the user field. My Solr index has 150048 messages (122K of those are in my personal mailbox) and takes up 628 megabytes of disk space. The total size of the email that is indexed is 7 gigabytes.
Interesting and good to know.
|<str name="parsedquery_toString">+(cc:the | from:the | to:the | body:the | subject:the)</str> |
My index is using the stopword filter but the list of stopwords is empty.
The following response may interest you:
This is a search for "a" which I had run several times, so Solr was serving it from its cache, and this time it only took 6 milliseconds. It also shows what a facet can do. The longest time I got for the "a" search was 15 milliseconds, before the query was in the cache.
I think they queries themselves are returning very quickly, at least they were when I did a query on a single inbox. That's why I'm wondering if doing a search across 40 different inboxes via roundcube might be the issue. I'm thinking that each time a new mailbox is selected, it slows things down. But I have no idea how the IMAP search is performed across multiple inboxes so that's just a wild guess.
https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems#Sol...
Attached.
I uploaded the wrong screenshot. Brain was on a badly flawed autopilot system.
This is a search for "a" which I had run several times, so Solr was serving it from its cache, and this time it only took 6 milliseconds. It also shows what a facet can do. The longest time I got for the "a" search was 15 milliseconds, before the query was in the cache.
I think they queries themselves are returning very quickly, at least they were when I did a query on a single inbox. That's why I'm wondering if doing a search across 40 different inboxes via roundcube might be the issue. I'm thinking that each time a new mailbox is selected, it slows things down. But I have no idea how the IMAP search is performed across multiple inboxes so that's just a wild guess.
OK, I take this back. I did an imap search via telnet and solr reports the search takes about 3 to 4 seconds. Here's the output:
a search text "maynez"
- SEARCH 5 6 7 32 61 64 69 70 117 118 119 120 121 122 123 124 126 127 129 165 197 202 203 204 205 206 207 216 231 259 451 452 453 454 455 456 482 730 731 810 811 812 813 814 815 816 817 818 819 820 829 830 831 832 852 853 854 855 867 868 869 870 871 872 873 874 875 886 887 888 889 891 904 908 909 910 911 912 913 920 924 925 926 927 928 931 936 938 940 941 944 946 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 969 970 971 972 973 974 975 976 977 986 987 988 989 993 1012 1013 1014 1015 1016 1017 1019 1020 1021 1022 1023 1024 1025 1026 1027 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1047 1048 1069 1091 1112 1113 1114 1121 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1144 1145 1146 1147 1148 1150 1159 1160 1161 1162 1175 1176 1177 1178 1179 1180 1181 1186 1187 1188 1189 1190 1192 1193 1197 1198 1202 1204 1205 1208 1214 1215 1216 1217 1218 1228 1229 1231 1232 1234 1236 1237 1243 1244 1245 1246 1247 1248 1249 1250 1252 1256 1257 1297 1298 1299 1300 1301 1302 1303 1304 1307 1308 1316 1317 1319 1320 1327 1328 1331 1332 1335 1336 1348 1349 1352 1355 1356 1358 1359 1361 1417 1418 1419 1420 1421 1423 1424 1443 1444 1445 1446 1447 1453 1460 1461 1462 1463 1464 1500 1501 1502 1503 1504 1505 1507 1508 1509 1510 1513 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1539 1541 1542 1543 1544 1594 1596 1597 1598 1599 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1628 1629 1631 1632 1633 1634 1635 1636 1637 1638 1639 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1660 1661 1663 1664 1665 1666 1674 1675 1676 1677 1679 1680 1688 1691 1693 1694 1695 1696 a OK Search completed (3.029 + 0.001 + 0.228 secs).
On 8/25/2021 11:33 AM, Steve Dondley wrote:
OK, I take this back. I did an imap search via telnet and solr reports the search takes about 3 to 4 seconds. Here's the output:
a search text "maynez"
Try this in on the commandline of the Solr server:
time curl "http://localhost:YYY/solr/dovecot/select?q=maynez&defType=edismax&qf=body+to..."
Where YYY is the port Solr is listening on.
In the response, QTime is how long it took for Solr to do the search. This does not include time for the result set to be gathered or the time to transfer the response over the network, which is where the "time" part of that comes in. I see this doing a similar query on my Solr:
real 0m0.026s user 0m0.011s sys 0m0.010s
QTime for that request was 1 millisecond -- so it took 25 milliseconds for the stuff that is not counted in QTime.
You could do the query remotely by changing "localhost" to the hostname or IP address of the Solr server.
Thanks, Shawn
Try this in on the commandline of the Solr server:
time curl "http://localhost:YYY/solr/dovecot/select?q=maynez&defType=edismax&qf=body+to..."
OK I had to modify the query path slightly to get it to work with my core to:
time curl http://localhost:8983/solr/dondley/select?q=maynez&defType=edismax&qf=body+t...
But it didn't return any results:
Results:
"responseHeader":{ "status":0, "QTime":7, "params":{ "q":"maynez"}}, "response":{"numFound":0,"start":0,"numFoundExact":true,"docs":[] }}
real 0m0.018s user 0m0.004s sys 0m0.006s
I only have emails for this person in a subfolder of my main Inbox folder so maybe it's only searching the top level folder?
You could do the query remotely by changing "localhost" to the hostname or IP address of the Solr server.
From remote host, I got similar numbers (no results):
real 0m0.017s user 0m0.009s sys 0m0.002s
On 8/25/2021 12:05 PM, Steve Dondley wrote:
OK I had to modify the query path slightly to get it to work with my core to:
time curl http://localhost:8983/solr/dondley/select?q=maynez&defType=edismax&qf=body+t...
But it didn't return any results:
<snip>
I only have emails for this person in a subfolder of my main Inbox folder so maybe it's only searching the top level folder?
That query should search ALL emails that dovecot has indexed to Solr. There is no restriction for mailbox or folder.
Try replacing "maynez" with something else that you know will be in the index.
Note that you can ask dovecot to completely reindex everybody's email with this command run as root, and then you can try searching again a few minutes later:
doveadm index -A -q '*'
It took me forever to work out the right options for that command. I think it should probably get added to the wiki page about fts_solr.
Question for experts on the dovecot side of this: What does it take for dovecot to index new messages delivered with dovecot's LDA by postfix? Is it a matter of waiting a certain amount of time? I saw in Solr that the last update to the index was 3 hours earlier, but I had received email only a few minutes before looking at Solr's stats.
Thanks, Shawn
That query should search ALL emails that dovecot has indexed to Solr. There is no restriction for mailbox or folder.
OK.
Try replacing "maynez" with something else that you know will be in the index.
Did a search on "the". Still nothing. Very, very weird. What would explain why my email client is still returning results I get nothing from the command line?
Here is the exact command I'm running:
time curl http://172.30.0.94:8983/solr/dondley/select?q=the&defType=edismax&qf=body+to...
Do you see any typos in there?
Note that you can ask dovecot to completely reindex everybody's email with this command run as root, and then you can try searching again a few minutes later:
doveadm index -A -q '*'
Yeah, tried this yesterday. Didn't help. If you think I should try again, let me know.
On 2021-08-25 02:05 PM, Steve Dondley wrote:
Try this in on the commandline of the Solr server:
time curl "http://localhost:YYY/solr/dovecot/select?q=maynez&defType=edismax&qf=body+to..."
OK I had to modify the query path slightly to get it to work with my core to:
time curl http://localhost:8983/solr/dondley/select?q=maynez&defType=edismax&qf=body+t...
OK! found the issue. My command line mangled the URL when I edited the url. There is a stray ":" in the query string.
So I am now seeing a ton of results and the following query time:
real 0m0.118s user 0m0.003s sys 0m0.011s
So this looks really good and fast. So I think we can say with confidence solr is doing its job. So why is roundcube/dovecot taking so long to show the results?
So this looks really good and fast. So I think we can say with confidence solr is doing its job. So why is roundcube/dovecot taking so long to show the results?
Random guess... Buffering? Whatever is sending to the browser isn't sending enough bytes to flush the buffer so the data is left in limbo until enough time goes by the buffer gets flushed anyways. Maybe a apache/nginx thing, php thing or browser thing. Remember its solr > dovecot > php > web server > browser. Tried other browsers?
Random guess... Buffering? Whatever is sending to the browser isn't sending enough bytes to flush the buffer so the data is left in limbo until enough time goes by the buffer gets flushed anyways. Maybe a apache/nginx thing, php thing or browser thing. Remember its solr > dovecot > php > web server > browser. Tried other browsers?
I'm inclined to believe the problem is not that high up the food chain. Because when I query IMAP on a single folder over telnet following the instructions found here: https://doc.dovecot.org/configuration_manual/fts/solr/, imap reports that it's taking 3 to 4 seconds to return results:
a search text "maynez" a OK Search completed (3.386 + 0.001 + 0.250 secs).
This particular search returns a few hundred results.
The speed of the query changed depending on whether a lot of results are returned. So if I search on a nonsensical word like "zyzzix", imap reports results nearly instantaneously:
a search text "zyzzix" a OK Search completed (0.012 + 0.000 + 0.006 secs).
I'm inclined to believe the problem is not that high up the food chain. Because when I query IMAP on a single folder over telnet following the instructions found here: https://doc.dovecot.org/configuration_manual/fts/solr/, imap reports that it's taking 3 to 4 seconds to return results:
a search text "maynez" a OK Search completed (3.386 + 0.001 + 0.250 secs).
This particular search returns a few hundred results.
The speed of the query changed depending on whether a lot of results are returned. So if I search on a nonsensical word like "zyzzix", imap reports results nearly instantaneously:
a search text "zyzzix" a OK Search completed (0.012 + 0.000 + 0.006 secs).
I just did a quick test. I did a search over imap over telnet with fts plugins turned off and with them turned on.
The search time was no better with it on than off.
So I'm thinking I got something misconfigured somewhere. It seems IMAP may not be using solr to fetch results. But this would be odd since I definitely do see a big improvements in times with fts plugins turned on when using roundcube.
The search time was no better with it on than off.
So I'm thinking I got something misconfigured somewhere. It seems IMAP may not be using solr to fetch results. But this would be odd since I definitely do see a big improvements in times with fts plugins turned on when using roundcube.
OK, I'm finally getting somewhere. Found this lead here: https://dovecot.org/pipermail/dovecot/2012-February/081514.html
"doveadm fts optimize" throws an error:
doveadm(root): Error: Couldn't drop privileges: User is missing UID (see mail_uid setting)
Checking /var/log/mail.err, I see a ton of these entries:
83353 Aug 25 14:53:22 email dovecot: imap(s)<26536><gJ1Iw2bKKLh/AAAB>: Error: fts_solr: received invalid uid ' 83354 Aug 25 14:53:22 email dovecot: imap: Error: ' 83355 Aug 25 14:58:05 email dovecot: imap(s)<26637><Awma1GbKprh/AAAB>: Error: fts_solr: received invalid uid ' 83356 Aug 25 14:58:05 email dovecot: imap: Error: ' 83357 Aug 25 15:26:58 email dovecot: imap(s)<27217><TF0qOGfKort/AAAB>: Error: fts_solr: received invalid uid ' 83358 Aug 25 15:26:58 email dovecot: imap: Error: ' 83359 Aug 25 15:27:11 email dovecot: imap(s)<27217><TF0qOGfKort/AAAB>: Error: fts_solr: received invalid uid ' 83360 Aug 25 15:27:11 email dovecot: imap: Error: ' 83361 Aug 25 15:27:36 email dovecot: imap(s)<27217><TF0qOGfKort/AAAB>: Error: fts_solr: received invalid uid ' 83362 Aug 25 15:27:36 email dovecot: imap: Error: '
I think this will be nailed once I figure out this issue.
On 2021-08-25 04:05 PM, Steve Dondley wrote:
The search time was no better with it on than off.
So I'm thinking I got something misconfigured somewhere. It seems IMAP may not be using solr to fetch results. But this would be odd since I definitely do see a big improvements in times with fts plugins turned on when using roundcube.
OK, I'm finally getting somewhere. Found this lead here: https://dovecot.org/pipermail/dovecot/2012-February/081514.html
"doveadm fts optimize" throws an error:
doveadm(root): Error: Couldn't drop privileges: User is missing UID (see mail_uid setting)
Checking /var/log/mail.err, I see a ton of these entries:
83353 Aug 25 14:53:22 email dovecot: imap(s)<26536><gJ1Iw2bKKLh/AAAB>: Error: fts_solr: received invalid uid ' 83354 Aug 25 14:53:22 email dovecot: imap: Error: ' 83355 Aug 25 14:58:05 email dovecot: imap(s)<26637><Awma1GbKprh/AAAB>: Error: fts_solr: received invalid uid ' 83356 Aug 25 14:58:05 email dovecot: imap: Error: ' 83357 Aug 25 15:26:58 email dovecot: imap(s)<27217><TF0qOGfKort/AAAB>: Error: fts_solr: received invalid uid ' 83358 Aug 25 15:26:58 email dovecot: imap: Error: ' 83359 Aug 25 15:27:11 email dovecot: imap(s)<27217><TF0qOGfKort/AAAB>: Error: fts_solr: received invalid uid ' 83360 Aug 25 15:27:11 email dovecot: imap: Error: ' 83361 Aug 25 15:27:36 email dovecot: imap(s)<27217><TF0qOGfKort/AAAB>: Error: fts_solr: received invalid uid ' 83362 Aug 25 15:27:36 email dovecot: imap: Error: '
I think this will be nailed once I figure out this issue.
And it looks like I'm running into a major bug in the slightly dated version of dovecot debian uses:
https://www.mail-archive.com/dovecot@dovecot.org/msg78825.html
I think this will be nailed once I figure out this issue.
And it looks like I'm running into a major bug in the slightly dated version of dovecot debian uses:
https://www.mail-archive.com/dovecot@dovecot.org/msg78825.html
And this:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970692
Though my error is slightly different. Instead of uid '0' I have no uid at all.
On 8/25/2021 2:10 PM, Steve Dondley wrote:
And it looks like I'm running into a major bug in the slightly dated version of dovecot debian uses:
https://www.mail-archive.com/dovecot@dovecot.org/msg78825.html
Recently I did a fairly major upgrade. I had an older Ubuntu release with Dovecot 1.x and an older Postfix version, wanted to upgrade it to v20, which had significantly newer versions of both programs.
Before I did the Ubuntu upgrade, I took advice received here and installed the dovecot repo, upgrading it first. I looked at the following link and only found a couple of things in my config I needed to change. I didn't even use the conversion command at the top of the page. Everything still worked after I upgraded, which I found a little surprising. Postfix also worked after the upgrade, with no config changes required.
https://doc.dovecot.org/installation_guide/upgrading/from-1.2-to-2.0/
I did the fts_solr integration *after* I upgraded dovecot.
You can find the dovecot repos here, there are options for Debian:
Thanks, Shawn
On 2021-08-25 04:32 PM, Shawn Heisey wrote:
On 8/25/2021 2:10 PM, Steve Dondley wrote:
And it looks like I'm running into a major bug in the slightly dated version of dovecot debian uses:
https://www.mail-archive.com/dovecot@dovecot.org/msg78825.html
Recently I did a fairly major upgrade. I had an older Ubuntu release with Dovecot 1.x and an older Postfix version, wanted to upgrade it to v20, which had significantly newer versions of both programs.
Before I did the Ubuntu upgrade, I took advice received here and installed the dovecot repo, upgrading it first. I looked at the following link and only found a couple of things in my config I needed to change. I didn't even use the conversion command at the top of the page. Everything still worked after I upgraded, which I found a little surprising. Postfix also worked after the upgrade, with no config changes required.
https://doc.dovecot.org/installation_guide/upgrading/from-1.2-to-2.0/
I did the fts_solr integration *after* I upgraded dovecot.
You can find the dovecot repos here, there are options for Debian:
Thanks, Shawn
Fixed!
I just upgraded to Debian bullseye and I'm not enjoying lightning fast search! Woohoo!
Figuring out what *isn't* wrong is a good strategy to be sure you aren't chasing ghosts. So I greatly appreciate your time and patience. I will be sure to pay it forward.
On 25/08/2021 23:05 Steve Dondley <s@dondley.com> wrote:
The search time was no better with it on than off.
So I'm thinking I got something misconfigured somewhere. It seems IMAP may not be using solr to fetch results. But this would be odd since I definitely do see a big improvements in times with fts plugins turned on when using roundcube.
OK, I'm finally getting somewhere. Found this lead here: https://dovecot.org/pipermail/dovecot/2012-February/081514.html
"doveadm fts optimize" throws an error:
doveadm(root): Error: Couldn't drop privileges: User is missing UID (see mail_uid setting)
Checking /var/log/mail.err, I see a ton of these entries:
83353 Aug 25 14:53:22 email dovecot: imap(s)<26536><gJ1Iw2bKKLh/AAAB>: Error: fts_solr: received invalid uid ' 83354 Aug 25 14:53:22 email dovecot: imap: Error: ' 83355 Aug 25 14:58:05 email dovecot: imap(s)<26637><Awma1GbKprh/AAAB>: Error: fts_solr: received invalid uid ' 83356 Aug 25 14:58:05 email dovecot: imap: Error: ' 83357 Aug 25 15:26:58 email dovecot: imap(s)<27217><TF0qOGfKort/AAAB>: Error: fts_solr: received invalid uid ' 83358 Aug 25 15:26:58 email dovecot: imap: Error: ' 83359 Aug 25 15:27:11 email dovecot: imap(s)<27217><TF0qOGfKort/AAAB>: Error: fts_solr: received invalid uid ' 83360 Aug 25 15:27:11 email dovecot: imap: Error: ' 83361 Aug 25 15:27:36 email dovecot: imap(s)<27217><TF0qOGfKort/AAAB>: Error: fts_solr: received invalid uid ' 83362 Aug 25 15:27:36 email dovecot: imap: Error: '
I think this will be nailed once I figure out this issue.
doveadm fts optimize -u TARGETUSER
Otherwise, as any other doveadm mail command, it assumes -u $USER
Aki
One other data point from my experimenting that might shed some light on the problem:
If I limit a search to a single folder instead of across all folders, it still takes 5 or 6 seconds for the results to appear. So that kind of destroys my theory that the problem might be caused by having too many inbox folders.
On 2021-08-24 08:53 PM, Steve Dondley wrote:
MY SETUP: I have apache solr full text search enabled with dovecot. I have an inbox with about 40 subfolders. I'm using the roundcube web-based mail client. The find command is showing 15823 email files and apache solr reports the same number. I'm running a dedicated mail server with a 1 GB of ram. The solr machine is running on a separate machine with 4 GB ram.
THE PROBLEM: When I do a full text search through all my inbox and all subfolders on a single word, search results are returned in about 10 to 15 seconds. This is better than the 40 seconds or so I'm getting when I turn off the fts and fts_solr plugins but still a little disappointing.
WHAT I'D LIKE: Now, I don't expect instant search results like gmail, but getting the search results to display in less than 5 seconds would be a lot less painful that 10 to 15 seconds.
WHAT I'VE TRIED: I have reindexed the emails on solr to try to speed things upit up.
ANY ADVICE? Curious to know what response times others are seeing and if there is anything else I might try to speed things up. Maybe the number of subfolders is slowing things down?
OK, I figured it out. It was a bug in Debian's version of dovecot: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970692
Upgrading to "Bulleyse" fixed things up nicely and I now have super fast search.
Now I have to go tackle the plugin errors that are crashing the roundcube install. :)
Citát Steve Dondley <s@dondley.com>:
On 2021-08-24 08:53 PM, Steve Dondley wrote:
MY SETUP: I have apache solr full text search enabled with dovecot. I have an inbox with about 40 subfolders. I'm using the roundcube web-based mail client. The find command is showing 15823 email files and apache solr reports the same number. I'm running a dedicated mail server with a 1 GB of ram. The solr machine is running on a separate machine with 4 GB ram.
THE PROBLEM: When I do a full text search through all my inbox and all subfolders on a single word, search results are returned in about 10 to 15 seconds. This is better than the 40 seconds or so I'm getting when I turn off the fts and fts_solr plugins but still a little disappointing.
WHAT I'D LIKE: Now, I don't expect instant search results like gmail, but getting the search results to display in less than 5 seconds would be a lot less painful that 10 to 15 seconds.
WHAT I'VE TRIED: I have reindexed the emails on solr to try to speed things upit up.
ANY ADVICE? Curious to know what response times others are seeing and if there is anything else I might try to speed things up. Maybe the number of subfolders is slowing things down?
OK, I figured it out. It was a bug in Debian's version of dovecot:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970692Upgrading to "Bulleyse" fixed things up nicely and I now have super
fast search.Now I have to go tackle the plugin errors that are crashing the
roundcube install. :)
Look also here: https://repo.dovecot.org/
participants (6)
-
Aki Tuomi
-
azurit＠pobox.sk
-
dovecot＠ptld.com
-
Gedalya
-
Shawn Heisey
-
Steve Dondley