Solr, Dovecot & macOS / iOS
I’m thinking of getting Solr working with my Dovecot server. Server is new 6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of thousands of messages.
But I’m not sure if:
a) it will make enough of a difference and
b) does Mail.app and other mail clients on Macs or iOS devices perform searches on their local copy of mail or does it just send a search request to the server?
I’m guessing the searches are done locally so no point in Solr?
Does anyone know?
Thanks, James.
On 13 Aug 2019, at 14.58, James Brown via dovecot <dovecot@dovecot.org> wrote:
I’m thinking of getting Solr working with my Dovecot server. Server is new 6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of thousands of messages.
But I’m not sure if:
a) it will make enough of a difference and
Choose mailbox format wisely. sdbox preferred unless HFS+ has problems with 100s of thousands of small files in same directory. If so, then use mdbox with periodic purges.
b) does Mail.app and other mail clients on Macs or iOS devices perform searches on their local copy of mail or does it just send a search request to the server?
None of the apple devices use IMAP SEARCH. They ALL maintain and use their own local search database on the device. Also they seem to refresh the database every now and then redownloading all emails.
I’m guessing the searches are done locally so no point in Solr?
No point if only apple client devices.
Sami
Le 13 août 2019 à 14:16, Sami Ketola via dovecot <dovecot@dovecot.org> a écrit :
On 13 Aug 2019, at 14.58, James Brown via dovecot <dovecot@dovecot.org> wrote:
I’m thinking of getting Solr working with my Dovecot server. Server is new 6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of thousands of messages.
But I’m not sure if:
a) it will make enough of a difference and
Choose mailbox format wisely. sdbox preferred unless HFS+ has problems with 100s of thousands of small files in same directory. If so, then use mdbox with periodic purges.
b) does Mail.app and other mail clients on Macs or iOS devices perform searches on their local copy of mail or does it just send a search request to the server?
None of the apple devices use IMAP SEARCH. They ALL maintain and use their own local search database on the device. Also they seem to refresh the database every now and then redownloading all emails.
Do you have a source for that. My experience is that without server search support, iOS is very slow at returning result. Moreover, it keep only latest messages and never download message until you read them.
On 13 Aug 2019, at 15.37, Jean-Daniel via dovecot <dovecot@dovecot.org> wrote:
Le 13 août 2019 à 14:16, Sami Ketola via dovecot <dovecot@dovecot.org> a écrit :
On 13 Aug 2019, at 14.58, James Brown via dovecot <dovecot@dovecot.org> wrote:
I’m thinking of getting Solr working with my Dovecot server. Server is new 6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of thousands of messages.
But I’m not sure if:
a) it will make enough of a difference and
Choose mailbox format wisely. sdbox preferred unless HFS+ has problems with 100s of thousands of small files in same directory. If so, then use mdbox with periodic purges.
b) does Mail.app and other mail clients on Macs or iOS devices perform searches on their local copy of mail or does it just send a search request to the server?
None of the apple devices use IMAP SEARCH. They ALL maintain and use their own local search database on the device. Also they seem to refresh the database every now and then redownloading all emails.
Do you have a source for that. My experience is that without server search support, iOS is very slow at returning result. Moreover, it keep only latest messages and never download message until you read them.
I'm a apple device user myself. I have couple of iPhones, couple of iPads, couple if MacBooks and Mail.app on any of them is not using IMAP SEARCH. And I cannot find any configuration option to enable it. Only spotlight index is used. On Mac OS Mail.App seems to store the indexed data to:
samik@samikworkmac:~>ls -1 Library/Mail/V6/MailData/Envelope\ Index* Library/Mail/V6/MailData/Envelope Index Library/Mail/V6/MailData/Envelope Index-shm Library/Mail/V6/MailData/Envelope Index-wal
if those files are removed or spotlight search for mails is disabled Mail.App can't find anything anymore. It does not fall back to IMAP SEARCH.
Sami
Le 13 août 2019 à 14:53, Sami Ketola <Sami.Ketola@Open-Xchange.com> a écrit :
On 13 Aug 2019, at 15.37, Jean-Daniel via dovecot <dovecot@dovecot.org <mailto:dovecot@dovecot.org>> wrote:
Le 13 août 2019 à 14:16, Sami Ketola via dovecot <dovecot@dovecot.org <mailto:dovecot@dovecot.org>> a écrit :
On 13 Aug 2019, at 14.58, James Brown via dovecot <dovecot@dovecot.org <mailto:dovecot@dovecot.org>> wrote:
I’m thinking of getting Solr working with my Dovecot server. Server is new 6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of thousands of messages.
But I’m not sure if:
a) it will make enough of a difference and
Choose mailbox format wisely. sdbox preferred unless HFS+ has problems with 100s of thousands of small files in same directory. If so, then use mdbox with periodic purges.
b) does Mail.app and other mail clients on Macs or iOS devices perform searches on their local copy of mail or does it just send a search request to the server?
None of the apple devices use IMAP SEARCH. They ALL maintain and use their own local search database on the device. Also they seem to refresh the database every now and then redownloading all emails.
Do you have a source for that. My experience is that without server search support, iOS is very slow at returning result. Moreover, it keep only latest messages and never download message until you read them.
I'm a apple device user myself. I have couple of iPhones, couple of iPads, couple if MacBooks and Mail.app on any of them is not using IMAP SEARCH. And I cannot find any configuration option to enable it. Only spotlight index is used. On Mac OS Mail.App seems to store the indexed data to:
samik@samikworkmac:~>ls -1 Library/Mail/V6/MailData/Envelope\ Index* Library/Mail/V6/MailData/Envelope Index Library/Mail/V6/MailData/Envelope Index-shm Library/Mail/V6/MailData/Envelope Index-wal
if those files are removed or spotlight search for mails is disabled Mail.App can't find anything anymore. It does not fall back to IMAP SEARCH.
My question was more about iOS. I know that macOS Mail does not rely on any way on remote indexing and has it’s own local index, but as it also store all messages locally, it’s an easy requirement. For iOS that only download messages meta-data by default, I was not so sure.
I’m accessing my mail server using Apple devices only, and see some imap SEARCH requests in dovecot stats, but can’t figure out where they came from though. So you may be right.
On 13 Aug 19, at 05:58 , James Brown <jlbrown@bordo.com.au> wrote:
b) does Mail.app and other mail clients on Macs or iOS devices perform searches on their local copy of mail or does it just send a search request to the server?
Mail.app uses spotlight on the local data, so if your users are all Mac then solar is pointless.
participants (4)
-
＠lbutlr
-
James Brown
-
Jean-Daniel
-
Sami Ketola