[Dovecot] Inconsistent search results and crash on force-resync

[Joe Beaubien]

Hi,

I am seeing inconsistencies in search results (finding 2 emails when only 1
exists, finding the email when it has been moved to another folder, etc).

I figured I should run force-resync to fix any issues. I ran the following:
doveadm -v force-resync -u <user> <mailbox> and I got some worrysome logs.
  - I should mention that I have been seeing some crashes of fts-lucene in
my logs. I sent a traceback of this on the mailing list 1-2 days ago under
the subject "[Dovecot] fts_lucene crashing".
  - I should also mention that all the problems I am having are only in 1
email account. This email account contains folders of over 100k emails. Do
I need to tweak dovecot somehow for this? Up until now all I did was change
vsz_limit to 1024 MB for "service imap".


Here are the logs:

Jun  1 11:15:01 XXXXX dovecot: imap(form): Error: Recent flags state
corrupted for mailbox INBOX2
Jun  1 11:15:01 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/INBOX2/dbox-Mails/dovecot.index reset, view is
now inconsistent
Jun  1 11:15:01 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/INBOX2/dbox-Mails/dovecot.index reset, view is
now inconsistent
Jun  1 11:15:01 XXXXX dovecot: imap(form): Error: Recent flags state
corrupted for mailbox contrat
Jun  1 11:15:01 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/contrat/dbox-Mails/dovecot.index reset, view is
now inconsistent
Jun  1 11:15:01 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/rep_Immigation
soi-m&AOo-me/dbox-Mails/dovecot.index reset, view is now inconsistent
Jun  1 11:15:01 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/Templates/dbox-Mails/dovecot.index reset, view
is now inconsistent
Jun  1 11:15:02 XXXXX dovecot: imap(form): Error: Recent flags state
corrupted for mailbox rep_eval_positive
Jun  1 11:15:02 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/rep_eval_positive/dbox-Mails/dovecot.index
reset, view is now inconsistent
Jun  1 11:15:02 XXXXX dovecot: imap(form): Error: Recent flags state
corrupted for mailbox Sent
Jun  1 11:15:02 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/Sent/dbox-Mails/dovecot.index reset, view is
now inconsistent
Jun  1 11:15:02 XXXXX dovecot: imap(form): Error: Recent flags state
corrupted for mailbox form_positif
Jun  1 11:15:02 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/form_positif/dbox-Mails/dovecot.index reset,
view is now inconsistent
Jun  1 11:15:02 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/Archives/contrat/dbox-Mails/dovecot.index
reset, view is now inconsistent
Jun  1 11:15:02 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/form_positif/dbox-Mails/dovecot.index reset,
view is now inconsistent
Jun  1 11:15:02 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/form_positif/dbox-Mails/dovecot.index reset,
view is now inconsistent
Jun  1 11:15:03 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/Archives/form_indetermine/dbox-Mails/dovecot.index
reset, view is now inconsistent
Jun  1 11:15:03 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/Archives/form_indetermine/dbox-Mails/dovecot.index
reset, view is now inconsistent
Jun  1 11:15:15 XXXXX dovecot: imap(form): Error: Recent flags state
corrupted for mailbox form_indetermine
Jun  1 11:15:15 XXXXX dovecot: imap(form): Error:
/data/emails/form/mailboxes/form_indetermine/dbox-Mails/dovecot.index
reset, view is now inconsistent
Jun  1 11:15:25 XXXXX dovecot: indexer-worker: Error: indexer-worker:
/home/jd/work/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:210:
std::string lucene::index::DocumentsWriter::closeDocStore(): Assertion
`numDocsInStore*8 == directory->fileLength( (docStoreSegment + "." +
IndexFileNames::FIELDS_INDEX_EXTENSION).c_str() )' failed.
Jun  1 11:15:25 XXXXX dovecot: indexer: Error: Indexer worker disconnected,
discarding 28 requests for form
Jun  1 11:15:25 XXXXX dovecot: indexer-worker(form): Fatal: master:
service(indexer-worker): child 9909 killed with signal 6 (core not dumped)


I have 3 questions:

  1) When the log says "/mailboxes/INBOX2/dbox-Mails/dovecot.index reset,
view is now inconsistent" should I be worried, or this will fix itself?

  2) Should I expect to see "Error: Recent flags state corrupted for
mailbox Sent"??? I ran the force-resync 3 times and I still see this
message.

  3) Any idea why clucene is crashing?


Regards,


-Joe