[Dovecot] Segmentation fault running doveadm index (lucene) on a big mailbox
Hi,
on a server running dovecot 2.2.12 I have a user with a quite big mailbox (~37000 Mails in the INBOX).
I tried to enable full text search using the fts_lucene backend (dovecot 2.2.12).
However, the doveadm -v index -u <username> INBOX
always runs into a
segmentation fault after mail number 24200:
I tried to get a backtrace using gdb:
doveadm(<username>): Info: INBOX: Caching mails seq=1..37578 24200/37578 Program received signal SIGSEGV, Segmentation fault. 0x00007ffff6612151 in lucene_index_build_more () from /usr/lib64/dovecot/lib21_fts_lucene_plugin.so (gdb) bt #0 0x00007ffff6612151 in lucene_index_build_more () from /usr/lib64/dovecot/lib21_fts_lucene_plugin.so #1 0x00007ffff6610b01 in ?? () from /usr/lib64/dovecot/lib21_fts_lucene_plugin.so #2 0x00007ffff6820d5b in fts_build_mail () from /usr/lib64/dovecot/lib20_fts_plugin.so #3 0x00007ffff6825b90 in ?? () from /usr/lib64/dovecot/lib20_fts_plugin.so #4 0x000000000041650a in ?? () #5 0x00000000004123a7 in _start ()
ulimit
reports "unlimited", so ulimits shouldn't be the culprit. Is
there another limit in dovecot?
Unfortunately, I can't "resume" the process, the indexing always starts again from the beginning.
Florian
doveconf -n:
# 2.2.12: /etc/dovecot/dovecot.conf # OS: Linux 3.14.0-gentoo x86_64 Gentoo Base System release 2.2 ext4 auth_mechanisms = plain login cram-md5 ntlm mail_gid = 900 mail_location = maildir:/home/vmail/%u mail_plugins = acl quota fts fts_lucene mail_uid = 900 maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { hidden = no list = children location = maildir:/home/vmail/%%u/:INDEX=/home/vmail/%%u/ prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = file:/home/vmail/shared-mailboxes fts = lucene fts_lucene = whitespace_chars=@. quota = maildir quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /home/vmail/%u/.dovecot.sieve sieve_dir = /home/vmail/%u/sieve } postmaster_address = postmaster@<domain> protocols = imap pop3 lmtp sieve sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-master { group = vmail mode = 0600 user = vmail } } service imap-postlogin { executable = script-login /usr/local/bin/imap-postlogin.sh user = $default_internal_user } service imap { executable = imap imap-postlogin } service managesieve-login { inet_listener sieve { port = 4190 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = dovecot } ssl = required ssl_cert =
On 04/15/2014 04:35 PM, Florian Klink wrote:
Hi,
on a server running dovecot 2.2.12 I have a user with a quite big mailbox (~37000 Mails in the INBOX).
I tried to enable full text search using the fts_lucene backend (dovecot 2.2.12). This patch (to be included in 2.2.13) addresses a different Lucene error, but might mitigate your issue as well. http://hg.dovecot.org/dovecot-2.2/rev/d63b209737be
If the issue remains and if possible, install dovecot dbg packages to get symbols and run a "bt full" instead of "bt".
br, Teemu Huovila
Am 15.04.2014 15:45, schrieb Teemu Huovila:
On 04/15/2014 04:35 PM, Florian Klink wrote:
Hi,
on a server running dovecot 2.2.12 I have a user with a quite big mailbox (~37000 Mails in the INBOX).
I tried to enable full text search using the fts_lucene backend (dovecot 2.2.12). This patch (to be included in 2.2.13) addresses a different Lucene error, but might mitigate your issue as well. http://hg.dovecot.org/dovecot-2.2/rev/d63b209737be
If the issue remains and if possible, install dovecot dbg packages to get symbols and run a "bt full" instead of "bt".
Unfortunately, the patch didn't fix the problem. However, I attached a full backtrace (with the applied patch, btw) including debug symbols.
It is attached, but I had to wipe some urls from the output. Hope thats not a problem.
I hope this helps!
Florian
On 15.4.2014, at 20.18, Florian Klink flokli@flokli.de wrote:
on a server running dovecot 2.2.12 I have a user with a quite big mailbox (~37000 Mails in the INBOX).
I tried to enable full text search using the fts_lucene backend (dovecot 2.2.12). This patch (to be included in 2.2.13) addresses a different Lucene error, but might mitigate your issue as well. http://hg.dovecot.org/dovecot-2.2/rev/d63b209737be
If the issue remains and if possible, install dovecot dbg packages to get symbols and run a "bt full" instead of "bt".
Unfortunately, the patch didn't fix the problem. However, I attached a full backtrace (with the applied patch, btw) including debug symbols.
It is attached, but I had to wipe some urls from the output. Hope thats not a problem.
http://hg.dovecot.org/dovecot-2.2/rev/2ef62e1f44f2 should help.
Am 15.04.2014 22:30, schrieb Timo Sirainen:
On 15.4.2014, at 20.18, Florian Klink flokli@flokli.de wrote:
on a server running dovecot 2.2.12 I have a user with a quite big mailbox (~37000 Mails in the INBOX).
I tried to enable full text search using the fts_lucene backend (dovecot 2.2.12). This patch (to be included in 2.2.13) addresses a different Lucene error, but might mitigate your issue as well. http://hg.dovecot.org/dovecot-2.2/rev/d63b209737be
If the issue remains and if possible, install dovecot dbg packages to get symbols and run a "bt full" instead of "bt".
Unfortunately, the patch didn't fix the problem. However, I attached a full backtrace (with the applied patch, btw) including debug symbols.
It is attached, but I had to wipe some urls from the output. Hope thats not a problem.
http://hg.dovecot.org/dovecot-2.2/rev/2ef62e1f44f2 should help.
doveadm(<username>): Info: INBOX: Caching mails seq=1..37596 *** Error in `/usr/bin/doveadm': malloc(): memory corruption: 0x000000000071c290 ***
Nope ;-)
On 15.4.2014, at 22.39, Florian Klink flokli@flokli.de wrote:
http://hg.dovecot.org/dovecot-2.2/rev/2ef62e1f44f2 should help.
doveadm(<username>): Info: INBOX: Caching mails seq=1..37596 *** Error in `/usr/bin/doveadm': malloc(): memory corruption: 0x000000000071c290 ***
Nope ;-)
Uh oh. http://hg.dovecot.org/dovecot-2.2/rev/5e42a8c18b51 makes it better.
Am 16.04.2014 00:14, schrieb Timo Sirainen:
On 15.4.2014, at 22.39, Florian Klink flokli@flokli.de wrote:
http://hg.dovecot.org/dovecot-2.2/rev/2ef62e1f44f2 should help.
doveadm(<username>): Info: INBOX: Caching mails seq=1..37596 *** Error in `/usr/bin/doveadm': malloc(): memory corruption: 0x000000000071c290 ***
Nope ;-)
Uh oh. http://hg.dovecot.org/dovecot-2.2/rev/5e42a8c18b51 makes it better.
Yes. This one fixed the memory corruption, allowed the indexing to complete and FTS now seems to work in this mailbox. Thank you! :-)
Florian
participants (3)
-
Florian Klink
-
Teemu Huovila
-
Timo Sirainen