Bug in dovecot 2.3 virtual plugin
Aki Tuomi
aki.tuomi at dovecot.fi
Mon Feb 5 14:25:17 EET 2018
Yeah, sorry I forgot to report back, it's now fixed in
https://github.com/dovecot/core/commit/5c1837529e6957da3e389683c43bd006859395e5.patch
Aki
On 05.02.2018 14:04, Marco Giunta wrote:
> Hi,
> did you have time to investigate about 'Panic: file unichar.c' bug ?
> Because I have the same problem with a 2.3.0 installation without
> virtual plugin.
>
> Thanks,
> Marco
>
>
> On 2018-01-03 16:52, Aki Tuomi wrote:
>> This is not a bug in virtual plugin, but in some email which contains
>> invalid unicode sequence somehow. Can you send me a core file? This
>> should not have occured ofc but would be nice to know how it ended up
>> here.
>>
>> Aki
>>
>>> On January 3, 2018 at 5:35 PM Jakobus Schürz
>>> <wertstoffe at nurfuerspam.de> wrote:
>>>
>>>
>>> Hi there!
>>>
>>> I compiled dovecot 2.3 from git. Because there is already a bug in
>>> virtual-plugin, and i hoped, it get fixed... but it doesn't. So this
>>> is the error-message from the log
>>>
>>> Jän 03 16:27:08 aldebaran dovecot[26460]:
>>> indexer-worker(jakob)<26476><qQ6g1+BhIJvAqAAO:sjYhMTH2TFpsZwAAk1Mx3g>:
>>> Panic: file unichar.c: line 160 (uni_ucs4_to_utf8_c): assertion
>>> failed: (uni_is_valid_ucs4(chr))
>>> Jän 03 16:27:08 aldebaran dovecot[26460]:
>>> indexer-worker(jakob)<26476><qQ6g1+BhIJvAqAAO:sjYhMTH2TFpsZwAAk1Mx3g>:
>>> Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0xc6021)
>>> [0x7f8299f7a021] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xc60ed)
>>> [0x7f8299f7a0ed] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
>>> [0x7f8299eec481] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(uni_ucs4_to_utf8_c+0xa0)
>>> [0x7f8299fb1500] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xa75e0)
>>> [0x7f8299f5b5e0] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(mail_html2text_more+0xc5)
>>> [0x7f8299f5b775] ->
>>> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xcfcc) [0x7f82990aefcc]
>>> -> /usr/local/lib/dovecot/lib20_fts_plugin.so(fts_parser_more+0x27)
>>> [0x7f82990aeca7] ->
>>> /usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x5e9)
>>> [0x7f82990acc39] ->
>>> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x1122d) [0x7f82990b322d]
>>> -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x916a)
>>> [0x7f82958e316a] ->
>>> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x10f5d) [0x7f82990b2f5d]
>>> -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x916a)
>>> [0x7f82958e316a] ->
>>> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x10f5d) [0x7f82990b2f5d]
>>> -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x2e)
>>> [0x7f829a2641be] -> dovecot/indexer-worker [jakob
>>> Synoptic/AKTUELL](+0x2533) [0x562227882533] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69)
>>> [0x7f8299f91bf9] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109)
>>>
>>> [0x7f8299f93499] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x52)
>>> [0x7f8299f91d02] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)
>>> [0x7f8299f91f18] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
>>> [0x7f8299f0f1e3] -> dovecot/indexer-worker [jakob
>>> Synoptic/AKTUELL](main+0xe7) [0x562227881f47] ->
>>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)
>>> [0x7f8299b352b1] -> dovecot/indexer-worker [jakob
>>> Synoptic/AKTUELL](_start+0x2a) [0x56222788201a]
>>> Jän 03 16:27:08 aldebaran dovecot[26460]: indexer: Error: Indexer
>>> worker disconnected, discarding 1 requests for jakob
>>> Jän 03 16:27:08 aldebaran dovecot[26460]:
>>> imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Error: indexer failed to index
>>> mailbox Synoptic/AKTUELL
>>> Jän 03 16:27:08 aldebaran dovecot[26460]:
>>> indexer-worker(jakob)<26476><qQ6g1+BhIJvAqAAO:sjYhMTH2TFpsZwAAk1Mx3g>:
>>> Fatal: master: service(indexer-worker): child 26476 killed with signal
>>> 6 (core dumps disabled)
>>> Jän 03 16:27:09 aldebaran dovecot[26460]:
>>> indexer-worker(jakob)<26484><qQ6g1+BhIJvAqAAO:J6mUIEz2TFp0ZwAAk1Mx3g>:
>>> Error: lucene index
>>> /var/lib/dovecot/db/indexes/Maildir/jakob/lucene-indexes:
>>> IndexWriter() failed (#1): Lock obtain timed out
>>> Jän 03 16:27:10 aldebaran dovecot[26460]:
>>> indexer-worker(jakob)<26484><qQ6g1+BhIJvAqAAO:J6mUIEz2TFp0ZwAAk1Mx3g>:
>>> Error: Mailbox Synoptic/AKTUELL: Transaction commit failed: BUG:
>>> Unknown internal error (attempted to index 1488 messages (UIDs
>>> 15214..16775))
>>> Jän 03 16:27:10 aldebaran dovecot[26460]:
>>> imap(jakob)<26480><lP5r2OBhKJvAqAAO>: Error: indexer failed to index
>>> mailbox Synoptic/AKTUELL
>>> Jän 03 16:27:11 aldebaran dovecot[26460]:
>>> imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Panic: file mail-index.c: line
>>> 793 (mail_index_close): assertion failed: (index->open_count > 0)
>>> Jän 03 16:27:11 aldebaran dovecot[26460]:
>>> imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Error: Raw backtrace:
>>> /usr/local/lib/dovecot/libdovecot.so.0(+0xc6021) [0x7fb0fbd3a021] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(+0xc60ed) [0x7fb0fbd3a0ed] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fb0fbcac481] ->
>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xf39a8)
>>> [0x7fb0fc0d99a8] ->
>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xd92d1)
>>> [0x7fb0fc0bf2d1] ->
>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xd9363)
>>> [0x7fb0fc0bf363] ->
>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_storage_mailbox_free+0x9)
>>>
>>> [0x7fb0fc0b0f39] ->
>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xd05a4)
>>> [0x7fb0fc0b65a4] ->
>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_free+0x19)
>>> [0x7fb0fc030c99] ->
>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xabdc)
>>> [0x7fb0f729bbdc] ->
>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xacb9)
>>> [0x7fb0f729bcb9] ->
>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a)
>>> [0x7fb0fc030c0a] ->
>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xac60)
>>> [0x7fb0f729bc60] ->
>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xacb9)
>>> [0x7fb0f729bcb9] ->
>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a)
>>> [0x7fb0fc030c0a] ->
>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_free+0x13)
>>> [0x7fb0fc030c93] -> dovecot/imap [jakob 192.168.0.14
>>> LOGOUT](imap_client_close_mailbox+0x5b) [0x56514d31305b] ->
>>> dovecot/imap [jakob 192.168.0.14 LOGOUT](cmd_logout+0x35)
>>> [0x56514d309485] -> dovecot/imap [jakob 192.168.0.14
>>> LOGOUT](command_exec+0x64) [0x56514d312ca4] -> dovecot/imap [jakob
>>> 192.168.0.14 LOGOUT](+0x1b0b2) [0x56514d3110b2] -> dovecot/imap [jakob
>>> 192.168.0.14 LOGOUT](+0x1b13c) [0x56514d31113c] -> dovecot/imap [jakob
>>> 192.168.0.14 LOGOUT](client_handle_input+0x1a5) [0x56514d311565] ->
>>> dovecot/imap [jakob 192.168.0.14 LOGOUT](client_input+0x82)
>>> [0x56514d311be2] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69)
>>> [0x7fb0fbd51bf9] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109)
>>>
>>> [0x7fb0fbd53499] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x52)
>>> [0x7fb0fbd51d02] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)
>>> [0x7fb0fbd51f18] ->
>>> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
>>> [0x7fb0fbccf1e3]
>>> Jän 03 16:27:11 aldebaran dovecot[26460]:
>>> imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Fatal: master: service(imap):
>>> child 26472 killed with signal 6 (core dumps disabled)
>>>
>>>
>>> I get this errors when i open or search in a virtual folder. i have
>>> enabled fts_lucene.
>>>
>>> My config is:
>>>
>>> # dovecot -n
>>> # 2.3.devel (393bfbea8): /usr/local/etc/dovecot/dovecot.conf
>>> # Pigeonhole version 0.5.0.rc1 (d68c23a1)
>>> # OS: Linux 4.13.0-trunk-amd64 x86_64 Debian 9.3 btrfs
>>> auth_mechanisms = plain login cram-md5 digest-md5
>>> auth_socket_path = /usr/local/var/run/dovecot/auth-userdb
>>> first_valid_uid = 1000
>>> imap_capability = +XDOVECOT
>>> imap_client_workarounds = tb-extra-mailbox-sep
>>> lda_mailbox_autocreate = yes
>>> lda_mailbox_autosubscribe = yes
>>> listen = *, ::1
>>> login_trusted_networks = 127.0.0.1/8 192.168.0.0/24 192.168.1.0/24
>>> 172.17.0.0/24 172.18.0.0/24
>>> mail_gid = vmail
>>> mail_home = /var/mail/%u
>>> mail_location =
>>> maildir:/var/mail/%u/Maildir:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/Maildir/%u:INDEXPVT=/var/lib/dovecot/db/indexes/Maildir/%u:CONTROL=/var/lib/dovecot/db/control/Maildir/%u
>>>
>>> mail_plugins = zlib quota acl listescape mail_log notify virtual fts
>>> fts_lucene
>>> mail_privileged_group = vmail
>>> mail_server_admin = mailto:jakob at xundeenergie.at
>>> mail_shared_explicit_inbox = yes
>>> mail_uid = vmail
>>> managesieve_notify_capability = mailto
>>> managesieve_sieve_capability = fileinto reject envelope
>>> encoded-character vacation subaddress comparator-i;ascii-numeric
>>> relational regex imap4flags copy include variables body enotify
>>> environment mailbox date index ihave duplicate mime foreverypart
>>> extracttext
>>> namespace {
>>> list = children
>>> location =
>>> maildir:/var/mail/public/:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/public/%u:INDEXPVT=/var/lib/dovecot/db/indexes/public/%u
>>>
>>> prefix = Roseggergasse/
>>> separator = /
>>> subscriptions = no
>>> type = public
>>> }
>>> namespace Geteilt {
>>> hidden = no
>>> inbox = no
>>> list = children
>>> location =
>>> maildir:/var/mail/%%u/Maildir:LAYOUT=fs:INDEXPVT=/var/lib/dovecot/db/indexes/shared/%u/%%u:INDEX=/var/lib/dovecot/db/indexes/shared/%u/%%u
>>>
>>> prefix = Geteilt/%%n/
>>> separator = /
>>> subscriptions = no
>>> type = shared
>>> }
>>> namespace Mailarchiv {
>>> hidden = no
>>> inbox = no
>>> list = children
>>> location =
>>> maildir:/var/mail/mailarchiv/%u/:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/mailarchiv/%u:INDEXPVT=/var/lib/dovecot/db/indexes/mailarchiv/%u
>>>
>>> mailbox incoming {
>>> auto = create
>>> }
>>> mailbox outgoing {
>>> auto = create
>>> }
>>> prefix = Mailarchiv/
>>> separator = /
>>> subscriptions = no
>>> type = private
>>> }
>>> namespace Real {
>>> hidden = yes
>>> list = no
>>> location =
>>> virtual:/var/mail/real:INDEX=/var/lib/dovecot/db/indexes/real/%u
>>> prefix = Real/
>>> separator = /
>>> subscriptions = no
>>> }
>>> namespace Synoptic {
>>> hidden = no
>>> list = children
>>> location =
>>> virtual:/var/mail/virtual:INDEX=/var/lib/dovecot/db/indexes/virtual/%u
>>> mailbox INBOX/Archives {
>>> auto = no
>>> special_use = \Archive
>>> }
>>> mailbox INBOX/Drafts {
>>> auto = no
>>> special_use = \Drafts
>>> }
>>> mailbox INBOX/Entwürfe {
>>> auto = no
>>> special_use = \Drafts
>>> }
>>> mailbox INBOX/Junk {
>>> auto = no
>>> special_use = \Junk
>>> }
>>> mailbox INBOX/Sent {
>>> auto = no
>>> special_use = \Sent
>>> }
>>> mailbox INBOX/Spam {
>>> auto = no
>>> special_use = \Junk
>>> }
>>> prefix = Synoptic/
>>> separator = /
>>> subscriptions = no
>>> }
>>> namespace inbox {
>>> hidden = no
>>> inbox = yes
>>> location =
>>> mailbox Archiv {
>>> auto = no
>>> special_use = \Archive
>>> }
>>> mailbox Archive {
>>> auto = no
>>> special_use = \Archive
>>> }
>>> mailbox Archives {
>>> auto = no
>>> special_use = \Archive
>>> }
>>> mailbox "Deleted Messages" {
>>> auto = no
>>> autoexpunge = 30 days
>>> special_use = \Trash
>>> }
>>> mailbox Drafts {
>>> auto = no
>>> special_use = \Drafts
>>> }
>>> mailbox Entwürfe {
>>> auto = no
>>> special_use = \Drafts
>>> }
>>> mailbox "Gelöschte Elemente" {
>>> auto = no
>>> autoexpunge = 30 days
>>> special_use = \Trash
>>> }
>>> mailbox "Gelöschte Objekte" {
>>> auto = no
>>> autoexpunge = 30 days
>>> special_use = \Trash
>>> }
>>> mailbox Gesendet {
>>> auto = no
>>> special_use = \Sent
>>> }
>>> mailbox "Gesendete Elemente" {
>>> auto = no
>>> special_use = \Sent
>>> }
>>> mailbox "Gesendete Objekte" {
>>> auto = no
>>> special_use = \Sent
>>> }
>>> mailbox Important {
>>> auto = no
>>> }
>>> mailbox Junk {
>>> auto = subscribe
>>> autoexpunge = 30 days
>>> special_use = \Junk
>>> }
>>> mailbox Mistkübel {
>>> auto = no
>>> autoexpunge = 30 days
>>> special_use = \Trash
>>> }
>>> mailbox Papierkorb {
>>> auto = no
>>> autoexpunge = 30 days
>>> special_use = \Trash
>>> }
>>> mailbox Sent {
>>> auto = subscribe
>>> special_use = \Sent
>>> }
>>> mailbox "Sent Messages" {
>>> auto = no
>>> special_use = \Sent
>>> }
>>> mailbox Spam {
>>> auto = no
>>> autoexpunge = 30 days
>>> special_use = \Junk
>>> }
>>> mailbox Synoptic/Alle {
>>> auto = no
>>> comment = All my messages
>>> special_use = \All
>>> }
>>> mailbox Trash {
>>> auto = no
>>> autoexpunge = 30 days
>>> special_use = \Trash
>>> }
>>> mailbox Wichtig {
>>> auto = create
>>> }
>>> prefix = INBOX/
>>> separator = /
>>> subscriptions = no
>>> type = private
>>> }
>>> namespace subscriptions {
>>> hidden = yes
>>> list = no
>>> location =
>>> prefix =
>>> subscriptions = yes
>>> }
>>> passdb {
>>> args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/users
>>> driver = passwd-file
>>> }
>>> plugin {
>>> acl = vfile:/etc/dovecot/dovecot-acl:cache_secs=300
>>> acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db
>>> fts = lucene
>>> fts_autoindex = yes
>>> fts_autoindex_max_recent_msgs = 80
>>> fts_index_timeout = 90s
>>> fts_lucene = whitespace_chars=@. normalize no_snowball
>>> mail_home = /var/mail/%u
>>> setting_name = sieve, managedsieve
>>> sieve = file:/var/mail/%u/sieve/;active=/var/mail/%u/sieve/%u.sieve
>>> }
>>> postmaster_address = postmaster at localhost
>>> protocols = imap pop3 lmtp imap lmtp sieve pop3 sieve
>>> service anvil {
>>> unix_listener anvil-auth-penalty {
>>> mode = 00
>>> }
>>> }
>>> service auth {
>>> unix_listener auth-client {
>>> group = vmail
>>> mode = 0666
>>> user = Debian-exim
>>> }
>>> unix_listener auth-userdb {
>>> group = vmail
>>> mode = 0666
>>> user = vmail
>>> }
>>> }
>>> service imap-login {
>>> inet_listener imap {
>>> port = 143
>>> }
>>> inet_listener imaps {
>>> port = 993
>>> ssl = yes
>>> }
>>> process_min_avail = 1
>>> service_count = 0
>>> }
>>> service imap {
>>> executable = imap postlogin
>>> process_limit = 1024
>>> vsz_limit = 400 M
>>> }
>>> service lmtp {
>>> executable = lmtp -L
>>> user = vmail
>>> vsz_limit = 400 M
>>> }
>>> service postlogin {
>>> executable = script-login -d rawlog
>>> }
>>> ssl = required
>>> ssl_cert = </usr/local/etc/dovecot/dovecot.crt
>>> ssl_dh = # hidden, use -P to show it
>>> ssl_key = # hidden, use -P to show it
>>> userdb {
>>> args = username_format=%u /usr/local/etc/dovecot/users
>>> default_fields = home=/var/mail/%u
>>> driver = passwd-file
>>> }
>>> verbose_proctitle = yes
>>> protocol lmtp {
>>> auth_username_format = %n
>>> mail_plugins = zlib quota acl listescape mail_log notify virtual
>>> fts fts_lucene quota sieve acl
>>> }
>>> protocol lda {
>>> mail_plugins = zlib quota acl listescape mail_log notify virtual
>>> fts fts_lucene quota sieve acl
>>> }
>>> protocol imap {
>>> mail_max_userip_connections = 10
>>> mail_plugins = zlib quota acl listescape mail_log notify virtual
>>> fts fts_lucene imap_quota imap_acl
>>> }
>>>
>>>
>>> The virtual Folder-definitions are:
>>>
>>> # find /var/mail/virtual/ /var/mail/real/
>>> /var/mail/virtual/
>>> /var/mail/virtual/GINBOX
>>> /var/mail/virtual/GINBOX/dovecot-virtual
>>> /var/mail/virtual/Neue
>>> /var/mail/virtual/Neue/dovecot-virtual
>>> /var/mail/virtual/Markierte
>>> /var/mail/virtual/Markierte/dovecot-virtual
>>> /var/mail/virtual/Ungelesene
>>> /var/mail/virtual/Ungelesene/dovecot-virtual
>>> /var/mail/virtual/Sent
>>> /var/mail/virtual/Sent/dovecot-virtual
>>> /var/mail/virtual/Drafts
>>> /var/mail/virtual/Drafts/dovecot-virtual
>>> /var/mail/virtual/Trash
>>> /var/mail/virtual/Trash/dovecot-virtual
>>> /var/mail/virtual/AKTUELL
>>> /var/mail/virtual/AKTUELL/dovecot-virtual
>>> /var/mail/virtual/Archives
>>> /var/mail/virtual/Archives/dovecot-virtual
>>> /var/mail/virtual/dovecot-acl-list
>>> /var/mail/virtual/WICHTIG
>>> /var/mail/virtual/WICHTIG/dovecot-virtual
>>> /var/mail/real/
>>> /var/mail/real/Alle
>>> /var/mail/real/Alle/dovecot-virtual
>>> /var/mail/real/dovecot-acl-list
>>> /var/mail/real/einjahr
>>> /var/mail/real/einjahr/dovecot-virtual
>>> /var/mail/real/zweijahr
>>> /var/mail/real/zweijahr/dovecot-virtual
>>> /var/mail/real/dreijahr
>>> /var/mail/real/dreijahr/dovecot-virtual
>>>
>>> # cat /var/mail/virtual/AKTUELL/dovecot-virtual
>>> Real/Alle
>>> inthread refs younger 2592000
>>>
>>> # cat /var/mail/real/Alle/dovecot-virtual
>>> !INBOX
>>> INBOX/*
>>> -INBOX/Trash
>>> -INBOX/Trash/*
>>> -INBOX/Archives
>>> -INBOX/Archives/*
>>> -INBOX/Drafts
>>> -INBOX/Drafts/*
>>> -INBOX/Junk
>>> -INBOX/Junk/*
>>> -INBOX/SYSTEM
>>> -INBOX/SYSTEM/*
>>> all
>>>
>>> The others are similar.
>>>
>>> Greez
>>>
>>> Jakob
>>
>
More information about the dovecot
mailing list