Am Dienstag, dem 31.08.2021 um 10:33 +0300 schrieb Aki Tuomi:

...

On 31/08/2021 00:11 Joan Moreau jom@grosjo.net wrote:

Hi There seems to be 2 plugins doing the same thins

• https://github.com/slusarz/dovecot-fts-flatcurve/
• https://github.com/grosjo/fts-xapian/ (mine) Both are in the doc of dovecot https://doc.dovecot.org/configuration_manual/fts/

I am currently working hard to push it to RPM package, and plugin is already approved by ArchLinux and Debian

Isn't there double work here ? Thanks JM

If you look closer, you can see they are not exactly duplicates. Flatcurve works differently than your plugin.

Aki

Is there somewhere a direct comparison of them? I currenty use fts-xapian from Joan without problems. But what would be the advantages of fts-flatcurve over fts-xapian?

On 31-08-2021 12:01, Aki Tuomi wrote:

...

On 31/08/2021 10:56 Felix Zielcke fzielcke@z-51.de wrote:

Am Dienstag, dem 31.08.2021 um 10:33 +0300 schrieb Aki Tuomi:

...

...

On 31/08/2021 00:11 Joan Moreau jom@grosjo.net wrote:

Hi There seems to be 2 plugins doing the same thins

• https://github.com/slusarz/dovecot-fts-flatcurve/
• https://github.com/grosjo/fts-xapian/ (mine) Both are in the doc of dovecot https://doc.dovecot.org/configuration_manual/fts/

I am currently working hard to push it to RPM package, and plugin is already approved by ArchLinux and Debian

Isn't there double work here ? Thanks JM

If you look closer, you can see they are not exactly duplicates. Flatcurve works differently than your plugin.

Aki

Is there somewhere a direct comparison of them? I currenty use fts-xapian from Joan without problems. But what would be the advantages of fts-flatcurve over fts-xapian?

fts_flatcurve does only full word searching, although you can use fts_filters and fts_tokenizers settings to affect stemming and other matching to make it work with plurals and such.

Both plugins have their merits.

I still think it's weird to see that Open-Xchange starts a FTS Xapian plugin with mostly the same basic functionality that is already available in an existing plugin maintained by someone in the community Especially if that happens without any (apparent) communication with the existing plugin developer to find out whether fixing the issues that slusarz/Open-Xchange seem to have with the existing plugin, can be fixed.

Combining forces just seems a better way to spend scarce development resources than building something similar (but different) without any communication.

(Note: I don't use any of these plugins).

My 2 cents,

Tom

See below.

On 08/31/2021 5:35 AM Aki Tuomi aki.tuomi@open-xchange.com wrote:

...

On 31/08/2021 13:59 Tom Hendrikx tom@whyscream.net wrote:

On 31-08-2021 12:01, Aki Tuomi wrote:

...

...

On 31/08/2021 10:56 Felix Zielcke fzielcke@z-51.de wrote:

Am Dienstag, dem 31.08.2021 um 10:33 +0300 schrieb Aki Tuomi:

...

...

On 31/08/2021 00:11 Joan Moreau jom@grosjo.net wrote:

Hi There seems to be 2 plugins doing the same thins

• https://github.com/slusarz/dovecot-fts-flatcurve/
• https://github.com/grosjo/fts-xapian/ (mine)

...

...

...

...

...

...

Isn't there double work here ?

FYI, Joan also reached out to me on the Github project page, so you can see my response to this question there:

https://github.com/slusarz/dovecot-fts-flatcurve/issues/5

Short answer: I see the plugins as "different". Yes, they both use the same underlying indexing technology (Xapian), but the design goals, features, and implementations are unique for each project. Thus, I don't see them as duplicate work.

...

...

...

Is there somewhere a direct comparison of them? I currenty use fts-xapian from Joan without problems. But what would be the advantages of fts-flatcurve over fts-xapian?

I was asked a similar question previously. See https://github.com/slusarz/dovecot-fts-flatcurve/issues/4#issuecomment-90242...

...

...

fts_flatcurve does only full word searching, although you can use fts_filters and fts_tokenizers settings to affect stemming and other matching to make it work with plurals and such.

Aki's knowledge is a bit dated - fts_flatcurve now supports both prefix matching (this is what Xapian provides by default) AND full substring matching (this is what is technically required by the IMAP RFC).

Although the current fts_flatcurve default is for full substring matching, it is debatable whether that is the correct choice. Users tend to think of search in terms of "prefix" matching - that's how Google works - and substring matching requires significant additional storage since Xapian doesn't support it natively.

...

I still think it's weird to see that Open-Xchange starts a FTS Xapian plugin with mostly the same basic functionality that is already available in an existing plugin maintained by someone in the community

As Aki noted, Open-Xchange didn't "start" anything. fts_flatcurve is entirely a project I have developed on my personal time.

As a Dovecot CE user, I had the same complaint that most people have - in current versions of Dovecot, there is no easy-to-use FTS option that works out-of-the-box. Solr is the only search driver still actively supported, and it requires significant additional resources and configuration to setup.

Unfortunately, we don't have the resources within Open-Xchange to create a CE-only search option as our customers generally use our proprietary, object storage aware FTS system that is part of OX Dovecot Pro. Without commercial demand, it is difficult to justify paid work on features that do not produce revenue in return. (fts_flatcurve does use Dovecot core features that are maintained for the Pro FTS system, like stemming, so we can take advantage of trickle down benefits from the commercial code maintenance.)

My goal was to give back to the community by providing an easy-to-use, resource sensitive, no-maintenance option to address this current product limitation.

Hopefully, fts_flatcurve will fit the requirements for most smaller installations that run Dovecot that don't require the carrier/enterprise grade FTS solutions large customers demand.

...

Especially if that happens without any (apparent) communication with the existing plugin developer to find out whether fixing the issues that slusarz/Open-Xchange seem to have with the existing plugin, can be fixed.

There were fundamental disagreements about design goals, and Joan (or anybody, really) would likely not have appreciated someone coming in and demanding that the entire architecture and design of the project needed to be changed based on my personal opinions.

This is open source. Everybody should be free to participate as they wish, as long as you are not negatively affecting someone else. I don't think that has happened here.

...

Combining forces just seems a better way to spend scarce development resources than building something similar (but different) without any communication.

I disagree, but that's ok. People can do that.

michael

On 09-01-2021 2:21 pm, Joan Moreau wrote:

...

Just for clarity, Open-Xchange has not written any xapian plugin whatsoever.

Yes but the doc says that Open Xchaneg "supports" one over the other. Honestly, I am doing this over my free time, begin very reactive to user requests, and have this confirmed by Debian, Archlinux and now Fedora in their core packages This is not very encouraging despite all the efforts achieved.

What is your goal for making software? To put out something good in the world that others can use if they choose to? Or a pat on the back? Can't users choose which ever plugin they prefer to use? Isn't that mission accomplished? Sounds like you are upset no one gave you a trophy. You will be satisfied with a thank you for creating open source software just like the thousands of others before you? Thank you.