Re: [FTS Xapian] Beta release

Here it is:

Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Effective uid=8, gid=8, home=/var/spool/mail/iwascoding/paul Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Quota root: name=User quota backend=dict args=:file:/var/spool/mail/iwascoding/paul/dovecot-quota Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Quota rule: root=User quota mailbox=* bytes=2147483648 messages=0 Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Quota rule: root=User quota mailbox=* bytes=2147483648 messages=60000 Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Quota grace: root=User quota bytes=214748364 (10%) Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: dict quota: user=paul@iwascoding.com mailto:user=paul@iwascoding.com, uri=file:/var/spool/mail/iwascoding/paul/dovecot-quota, noenforcing=0 Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:~/mdbox Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: fs: root=/var/spool/mail/iwascoding/paul/mdbox, index=, indexpvt=, control=, inbox=, alt= Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: FTS Xapian: Partial=2, Full=20 DB_PATH=/var/spool/mail/iwascoding/paul/mdbox/xapian-indexes Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: quota: quota_over_flag check: quota_over_script unset - skipping Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Mailbox sent: Mailbox opened because: indexing Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: FTS Xapian : Mailbox sent : Last UID=0 Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: FTS Xapian : Mailbox sent : Last UID=0 Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Namespace : Using permissions from /var/spool/mail/iwascoding/paul/mdbox: mode=0700 gid=default Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Mailbox sent: UID 1: Opened mail because: fts indexing Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Opening RW /var/spool/mail/iwascoding/paul/mdbox/xapian-indexes/db_9ddfe10d8a8a8a568c120000654d370e Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Mailbox sent: UID 2: Opened mail because: fts indexing Jan 14 09:26:08 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Mailbox sent: UID 3: Opened mail because: fts indexing

Thank you!

On 14. Jan 2019, at 10:11, Joan Moreau via dovecot mailto:dovecot@dovecot.org> wrote:

Can you send the log part that includes the "init" of the plugins (something similar as below) ?

WHich version of Xapian are you on ?

Jan 14 09:10:04 gjserver dovecot[31082]: indexer-worker(admin@grosjo.net mailto:admin@grosjo.net)<14725>apZfHexRPFy9QAAA0thIag:UL+TNOxRPFyFOQAA0thIag: FTS Xapian: Partial=2, Full=20 DB_PATH=/data/mail/grosjo.net/admin/xapian-indexes http://grosjo.net/admin/xapian-indexes Jan 14 09:10:04 gjserver dovecot[31082]: indexer-worker(admin@grosjo.net mailto:admin@grosjo.net)<14725>apZfHexRPFy9QAAA0thIag:UL+TNOxRPFyFOQAA0thIag: FTS Xapian : Mailbox Mail : Last UID=815055 Jan 14 09:10:04 gjserver dovecot[31082]: indexer-worker(admin@grosjo.net mailto:admin@grosjo.net)<14725>apZfHexRPFy9QAAA0thIag:UL+TNOxRPFyFOQAA0thIag: FTS Xapian : Mailbox Mail : Last UID=815055 Jan 14 09:10:04 gjserver dovecot[31082]: indexer-worker(admin@grosjo.net mailto:admin@grosjo.net)<14725>apZfHexRPFy9QAAA0thIag:UL+TNOxRPFyFOQAA0thIag: Opening RW /data/mail/grosjo.net/admin/xapian-indexes/db_5c935034609bc14c0e550000d6a3092d http://grosjo.net/admin/xapian-indexes/db_5c935034609bc14c0e550000d6a3092d

On 2019-01-14 10:08, Paul Hecker via dovecot wrote:

...

Hi,

I installed and tested your version, but the indexer process crashes reproducible with the following command after about 2000 messages were indexed:

doveadm index -u paul@iwascoding.com mailto:paul@iwascoding.com -q \*

Jan 14 09:26:15 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Debug: Mailbox sent: UID 2038: Opened mail because: fts indexing Jan 14 09:26:15 mail dovecot: indexer-worker: Error: terminate called after throwing an instance of 'std::bad_alloc' Jan 14 09:26:15 mail dovecot: indexer-worker: Error: what(): std::bad_alloc Jan 14 09:26:15 mail dovecot: indexer: Error: Indexer worker disconnected, discarding 48 requests for paul@iwascoding.com mailto:paul@iwascoding.com Jan 14 09:26:15 mail dovecot: indexer-worker(paul@iwascoding.com mailto:paul@iwascoding.com)<16777><IKpfOqBHPFyJQQAADYqDFA>: Fatal: master: service(indexer-worker): child 16777 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps https://dovecot.org/bugreport.html#coredumps)

I tried to delete the message, but this does not help (crashes e.g. after message 2029 or 2044). Other folders with fewer messages were successfully indexed before.

Sorry, could not convince dovecot to create core dumps (read the docs, changed /proc/sys/kernel/core_pattern, added LimitCORE=unlimited/infinity, even created /etc/systemd/system/dovecot.service.d/coredump.conf to no avail). Custom Dovecot 2.3.4 on Debian Stretch.

Thanks, Paul