Solr -> Xapian ?

[Aki Tuomi]

I hope you are aware that "linking with Xapian" requires somewhat more work than just -lxapian in linker? If you or someone feels like writing fts_xapian, go for it. 

Aki

> On 04 January 2019 at 08:20 Joan Moreau via dovecot <dovecot at dovecot.org> wrote:
> 
> 
> What about consedering linking Dovecot with Xapian librairies instead of
> going to nightmare Solr ? 
> 
> https://xapian.org/features
> 
> On 2019-01-02 17:10, John Tulp wrote:
> 
> > On Wed, 2019-01-02 at 00:59 -0800, M. Balridge wrote: The main problem is : After some time of indexing from Dovecot, Dovecot
> > returns errors (invalid SID, etc...) and Solr return "out of range
> > indexes" errors 
> > I've been watching the progress of this thread with no small concern, mainly
> > because I've been tasked with providing a server-side email search facility
> > with a budget and manpower level that comes down to mainly *1*, i.e., me.
> > 
> > I was expecting, given the strongly worded language about "just use
> > lucene/SOLR" and "ignore squat", that I should invest time + effort into this
> > JAVA nightmare that is SOLR.
> > 
> > I started with squat and another word-indexor system that used out-of-band
> > (not a dovecot plugin) software to provide rapid (sub-second) searches through
> > tens-of-GB-scale mailboxes.
> > 
> > Unlike what I was led to believe, the squat indexes worked surprisingly well,
> > once you sorted out the odd resource size (ulimit-related) issues (vsz &
> > friends) limitations. I did notice the "worst-case" search performance have
> > worryingly high O(x) increases in time, but I'd not seen anything that was a
> > dealbreaker. It goes without saying that various substring searches worked as
> > expected, for the most part.
> > 
> > My experiences with SOLR were similar to Messr. Moreau's: lots of startup
> > errors with provided schemata files. Lots of JAVA nonsense issues. Lots of
> > sensitivity to WHICH Java runtime, etc, etc. I finally fixated a specific JVM,
> > version of SOLR, and dovecot to find the "best" working combination, only to
> > find that the searches didn't work out as expected. I expected to be able to
> > do date-ranging based searches. Didn't work. I expected to search CONTENTS of
> > emails, and despite many days of tweaks, I couldn't get it to index even the
> > basics like filenames/types of attachments, so I could exposed
> > attachment-based searching to my users.
> > 
> > So, without rancour or antipathy, I ask the entire list: has ANYONE gotten a
> > Dovecot/solr-fts-plugin setup to work that provides as a BASELINE, all of the
> > following functionality:
> > 
> > 1) The ability to search for a string within any of the structured fields
> > (from/subject) that returns correct results?
> > 
> > 2) The ability to search for any string within the BODY of emails, including
> > the MIME attachment boundaries?
> > 
> > 3) The ability to do "ranging" searches for structures within emails that
> > decompose to "dates" or other simple-numeric data?
> > 
> > OPTIONALLY, and this is probably way outside of the scope of the above,
> > despite the fact that it's listed as a "selling point" of SOLR versus other
> > full text search engines:
> > 
> > 4) The ability to do searches against any attachments that are able to be
> > post-processed and hyper-indexed by SOLR+Tika?
> > 
> > -------------
> > 
> > SOLR seems to have "brand cachet", so presumably it actually works (for somebody).
> > 
> > Dovecot has not a little "brand cachet", and for me, I have innate faith and
> > trust in Timo and his software. I am no stranger to the "costs" of "free"
> > software, in that you sacrifice your own blood, sweat, and tears just to get
> > these disparate pieces to work together.
> > 
> > I *DO* respect that Timo has to keep the lights (and sauna) on in Finland.
> > Maybe there's a super-secret (no advertised prices, "carrier-only" price list)
> > with _Dovecot, Oy_ wherein the above ARE actually available for something less
> > than 6.022 x 10^23 Euros per centi-second of licencing fees.
> > 
> > But please, level with us faithful users.  Does this morass of Java B.S.
> > actually work, and if not, please just deprecate and remove this moribund
> > software, and stop trying to bury the only FTS plugin many of us HAVE actually
> > gotten to work.  (Pretty please?)
> > 
> > I respect that Messr. Moreau has made an earnest effort to get this JAVA B.S.
> > to actually work, as I have. 
> > 
> > He persevered where I'd given up. He's vocal about it, and now I'm chiming in
> > that this ornate collection of switchblades only cuts those who try to use them.
> > 
> > Respectfully,
> > =M=
>  Fascinating...
> 
> SOLR says the following are powered by SOLR...
> 
> https://wiki.apache.org/solr/PublicServers
> 
> Perhaps if you could find out from that list which of them are using
> SOLR in conjunction with Dovecot...
> 
> food for thought...