<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /></head><body style='font-size: 9pt; font-family: Verdana,Geneva,sans-serif'>
<p>Can you send the log part that includes the "init" of the plugins (something similar as below) ?</p>
<p>WHich version of Xapian are you on ?</p>
<p>Jan 14 09:10:04 gjserver dovecot[31082]: indexer-worker(admin@grosjo.net)<14725><apZfHexRPFy9QAAA0thIag:UL+TNOxRPFyFOQAA0thIag>: FTS Xapian: Partial=2, Full=20 DB_PATH=/data/mail/grosjo.net/admin/xapian-indexes<br />Jan 14 09:10:04 gjserver dovecot[31082]: indexer-worker(admin@grosjo.net)<14725><apZfHexRPFy9QAAA0thIag:UL+TNOxRPFyFOQAA0thIag>: FTS Xapian : Mailbox Mail : Last UID=815055<br />Jan 14 09:10:04 gjserver dovecot[31082]: indexer-worker(admin@grosjo.net)<14725><apZfHexRPFy9QAAA0thIag:UL+TNOxRPFyFOQAA0thIag>: FTS Xapian : Mailbox Mail : Last UID=815055<br />Jan 14 09:10:04 gjserver dovecot[31082]: indexer-worker(admin@grosjo.net)<14725><apZfHexRPFy9QAAA0thIag:UL+TNOxRPFyFOQAA0thIag>: Opening RW /data/mail/grosjo.net/admin/xapian-indexes/db_5c935034609bc14c0e550000d6a3092d</p>
<p><br /></p>
<div id="signature"> </div>
<p><br /></p>
<p id="reply-intro">On 2019-01-14 10:08, Paul Hecker via dovecot wrote:</p>
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><!-- html ignored --><!-- head ignored --><!-- meta ignored -->
<div class="pre" style="margin: 0; padding: 0; font-family: monospace">Hi,<br /><br />I installed and tested your version, but the indexer process crashes reproducible with the following command after about 2000 messages were indexed:<br /><br />doveadm index -u <a href="mailto:paul@iwascoding.com" rel="noreferrer">paul@iwascoding.com</a> -q \*<br /><br />Jan 14 09:26:15 mail dovecot: indexer-worker(<a href="mailto:paul@iwascoding.com" rel="noreferrer">paul@iwascoding.com</a>)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Mailbox sent: UID 2038: Opened mail because: fts indexing<br />Jan 14 09:26:15 mail dovecot: indexer-worker: Error: terminate called after throwing an instance of 'std::bad_alloc'<br />Jan 14 09:26:15 mail dovecot: indexer-worker: Error: what(): std::bad_alloc<br />Jan 14 09:26:15 mail dovecot: indexer: Error: Indexer worker disconnected, discarding 48 requests for <a href="mailto:paul@iwascoding.com" rel="noreferrer">paul@iwascoding.com</a><br />Jan 14 09:26:15 mail dovecot: indexer-worker(<a href="mailto:paul@iwascoding.com" rel="noreferrer">paul@iwascoding.com</a>)<16777><IKpfOqBHPFyJQQAADYqDFA>: Fatal: master: service(indexer-worker): child 16777 killed with signal 6 (core dumps disabled - <a href="https://dovecot.org/bugreport.html#coredumps" target="_blank" rel="noopener noreferrer">https://dovecot.org/bugreport.html#coredumps</a>)<br /><br />I tried to delete the message, but this does not help (crashes e.g. after message 2029 or 2044). Other folders with fewer messages were successfully indexed before.<br /><br />Sorry, could not convince dovecot to create core dumps (read the docs, changed /proc/sys/kernel/core_pattern, added LimitCORE=unlimited/infinity, even created /etc/systemd/system/dovecot.service.d/coredump.conf to no avail). Custom Dovecot 2.3.4 on Debian Stretch.<br /><br />Thanks,<br />Paul<br /><br /><br />
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0">On 14. Jan 2019, at 07:42, Joan Moreau via dovecot <<a href="mailto:dovecot@dovecot.org" rel="noreferrer">dovecot@dovecot.org</a>> wrote:<br /><br />Thank you Stephan.<br /><br />The version here shall be up and running : <a href="https://github.com/grosjo/fts-xapian" target="_blank" rel="noopener noreferrer">https://github.com/grosjo/fts-xapian</a><br /><br /><br /><br /> <br /><br /><br />On 2019-01-14 00:07, Stephan Bosch wrote:<br /><br />
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><br /><br />Op 13/01/2019 om 21:25 schreef Joan Moreau via dovecot:
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><br />I tried to combined it, the "autoreconf" errors are solved<br /><br />Now, when I type "make install", the lib is not pushed into dovecot folder, but somewhere in /usr/local/...<br /><br />How to adjust this to have it arriving in the proper folder ?<br /><br /></blockquote>
<br />Depends on your system. It mostly a matter of setting a proper --prefix directory for configure, but other paths are configurable as well. I usually check what the official distribution package for Dovecot is doing and use that as a basis.<br /><br />For Debian I use the following configure command:<br /><br />./configure --with-ldap=plugin --with-ssl=openssl --with-sql=plugin --with-lua=plugin --with-pgsql --with-mysql --with-sqlite \<br /> --with-gssapi=plugin --with-solr --with-ioloop=best --enable-maintainer-mode \<br /> --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib --localstatedir=/var --mandir=/usr/share/man \<br /> --infodir=/usr/share/info --with-moduledir=/usr/lib/dovecot/modules --disable-rpath --disable-static<br /><br />Regards,<br /><br />Stephan<br /><br />
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><br />On 2019-01-13 21:01, Tuomi, Aki wrote:<br /><br />
<blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0">You copied your Makefile.am there. Stephan made you a working version, can you try that?<br />(sorry for dup)<br />Aki<br />-------- Original message --------<br />From: Joan Moreau <<a href="mailto:jom@grosjo.net" rel="noreferrer">jom@grosjo.net</a>><br />Date: 13/01/2019 21:39 (GMT+02:00)<br />To: Stephan Bosch <<a href="mailto:stephan@rename-it.nl" rel="noreferrer">stephan@rename-it.nl</a>><br />Cc: Aki Tuomi <<a href="mailto:aki.tuomi@open-xchange.com" rel="noreferrer">aki.tuomi@open-xchange.com</a>><br />Subject: Re: [FTS Xapian] Beta release<br /><br />I used the skeleton from Aki : <a href="https://github.com/grosjo/fts-xapian" target="_blank" rel="noopener noreferrer">https://github.com/grosjo/fts-xapian</a><br /><br />However, when I try to act as a visitor, I reach teh follwoing error:<br /><br /># autoreconf -vi<br />autoreconf: Entering directory `.'<br />autoreconf: configure.ac: not using Gettext<br />autoreconf: running: aclocal -I m4<br />autoreconf: configure.ac: tracing<br />autoreconf: running: libtoolize --copy<br />libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, '.'.<br />libtoolize: copying file './ltmain.sh'<br />libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.<br />libtoolize: copying file 'm4/libtool.m4'<br />libtoolize: copying file 'm4/ltoptions.m4'<br />libtoolize: copying file 'm4/ltsugar.m4'<br />libtoolize: copying file 'm4/ltversion.m4'<br />libtoolize: copying file 'm4/lt~obsolete.m4'<br />autoreconf: running: /usr/bin/autoconf<br />autoreconf: running: /usr/bin/autoheader<br />autoreconf: running: automake --add-missing --copy --no-force<br />configure.ac:9: installing './compile'<br />configure.ac:11: installing './config.guess'<br />configure.ac:11: installing './config.sub'<br />configure.ac:7: installing './install-sh'<br />configure.ac:7: installing './missing'<br />src/Makefile.am: installing './depcomp'<br />/usr/share/automake-1.16/am/depend2.am: error: am__fastdepCXX does not appear in AM_CONDITIONAL<br />/usr/share/automake-1.16/am/depend2.am: The usual way to define 'am__fastdepCXX' is to add 'AC_PROG_CXX'<br />/usr/share/automake-1.16/am/depend2.am: to 'configure.ac' and run 'aclocal' and 'autoconf' again<br />src/Makefile.am: error: C++ source seen but 'CXX' is undefined<br />src/Makefile.am: The usual way to define 'CXX' is to add 'AC_PROG_CXX'<br />src/Makefile.am: to 'configure.ac' and run 'autoconf' again.<br />src/Makefile.am:11: warning: variable 'NOPLUGIN_LDFLAGS' is defined but no program or<br />src/Makefile.am:11: library has 'NOPLUGIN' as canonical name (possible typo)<br />autoreconf: automake failed with exit status: 1<br /><br /><br /><br />On 2019-01-13 20:24, Stephan Bosch wrote:<br /><br /> Oh, right, a distribution tarball doesn't include some of the<br /> necessary files for your repository like autogen.sh and<br /> .gitignore. The attached tarball includes all those and is ready<br /> for `git init`. The previous tarball was made with `make<br /> distcheck` from this one.<br /><br /> Regards,<br /><br /> Stephan.<br /><br /> Op 13/01/2019 om 20:14 schreef Stephan Bosch:<br /><br /> Hi Joan,<br /><br /> Op 13/01/2019 om 19:03 schreef Aki Tuomi:<br /><br /> Yes, from compiling point of view it is done.<br /><br /> Unfortunately what is not done is all the other work<br /> involved, such as fixing all the inevitable bugs it has<br /> and maintaining it. We do not want, at this moment, take<br /> up maintaining and developing yet another FTS plugin as<br /> we have plenty of things to do already.<br /><br /> I invite you to setup your own repository and provide<br /> this plugin from there, being the maintainer of this<br /> plugin. We can add a link to your plugin on our FTS page<br /> so people can also find it.<br /><br /> There are other plugins like this, e.g.<br /> <a href="https://github.com/st3fan/dovecot-xaps-plugin" target="_blank" rel="noopener noreferrer">https://github.com/st3fan/dovecot-xaps-plugin</a><br /><br /><br /> I turned the code you provided into a separate plugin<br /> package. The distribution tarball is attached.<br /><br /> Notable changes:<br /><br /> - Added example copyright headers and COPYING and AUTHORS<br /> files. You should modify those to your preference.<br /> - Added README and INSTALL files (in markdown using Pandoc).<br /> Those need to be amended with details.<br /> - Amended the plugin code to display a debug message with the<br /> plugin name and version upon plugin load.<br /><br /> I advise you to turn this into a git repository and continue from there.<br /><br /> I do not recommend releasing this plugin with the<br /> -fpermissive flag and the resulting warning as it is now. But<br /> I'm assuming this is still a work in progress, so that is OK.<br /><br /> Regards,<br /><br /> Stephan.<br /><br /><br /> On 13 January 2019 at 19:52 Joan Moreau <<a href="mailto:jom@grosjo.net" rel="noreferrer">jom@grosjo.net</a><br /> <mailto:<a href="mailto:jom@grosjo.net" rel="noreferrer">jom@grosjo.net</a>>> wrote:<br /><br /><br /> The only point here of this fts-xapian is to get rid<br /> of solr (because it<br /> is just a nightmare to setup) and squat (because it is considere<br /> obsolete).<br /><br /> I already sent the changed in configure.ac, makefile.am, etc.. in order<br /> to include it in the dovecot, and it compiles properly<br /><br /> The only remaining point is to push it in hte git (yes, everything is<br /> already done)<br /><br /> On 2019-01-13 18:45, Aki Tuomi wrote:<br /><br /> On 13 January 2019 at 17:05 Joan Moreau via<br /> dovecot <<a href="mailto:dovecot@dovecot.org" rel="noreferrer">dovecot@dovecot.org</a><br /> <mailto:<a href="mailto:dovecot@dovecot.org" rel="noreferrer">dovecot@dovecot.org</a>>> wrote:<br /><br /> Hi<br /><br /> Please find attached the beta release of FTS<br /> Xapian, with the objective<br /> to replace fts_squat that is being deprecated.<br /><br /> Configuration is exactly the same as for fts_squat:<br /><br /> plugin {<br /><br /> plugin = fts fts_xapian (...)<br /> fts = xapian<br /> fts_autoindex = yes<br /> fts_enforced = yes<br /> fts_xapian = partial=2 full=20<br /><br /> This is installed on my production server<br /> (>120Gb of mailboxes), and I<br /> will observe it during the coming days.<br /><br /> I will definitely appreciate that this is added in the core git of<br /> docevot, in order to have a versionning of<br /> it, to remove squat and let<br /> basic users able to avoid Solr alternative as much as possible.<br /><br /> Thanks<br /><br /> JM<br /><br /> Hi!<br /><br /> I still recommend you setup a, say, github<br /> repository for your plugin. We are not able to<br /> currently include your work in dovecot core as it<br /> is more work than just pushing the code into the<br /> repo. Maybe it can be included in the future.<br /><br /> If you want, I can help you in setting up the<br /> required configuration scripts and such to make<br /> it possible to compile it as plugin.<br /><br /> Then anyone can download it and install it for<br /> their dovecot, even if dovecot itself has been<br /> installed from packages, and also makes it<br /> possible for package maintainers to consider<br /> including it in distributions.<br /><br /> Aki<br /><br /></blockquote>
</blockquote>
<br /><br /></blockquote>
</blockquote>
<br /><br />Paul Hecker<br /><a href="mailto:paul@iwascoding.com" rel="noreferrer">paul@iwascoding.com</a><br /><a href="http://www.iwascoding.com" target="_blank" rel="noopener noreferrer">http://www.iwascoding.com</a><br /><br />iwascoding GmbH<br />Weserstr 18, 10247 Berlin<br />HRB 97269 B Amtsgericht Berlin-Charlottenburg<br />Geschäftsführer: Paul Hecker, Ilja Iwas<br /><br /><br /><br /><br /><br /><br /><br /><br /></div>
</blockquote>
</body></html>