Bug in dovecot 2.3 virtual plugin

Aki Tuomi aki.tuomi at dovecot.fi
Wed Jan 3 17:52:55 EET 2018


This is not a bug in virtual plugin, but in some email which contains invalid unicode sequence somehow. Can you send me a core file? This should not have occured ofc but would be nice to know how it ended up here.

Aki

> On January 3, 2018 at 5:35 PM Jakobus Schürz <wertstoffe at nurfuerspam.de> wrote:
> 
> 
> Hi there!
> 
> I compiled dovecot 2.3 from git. Because there is already a bug in  
> virtual-plugin, and i hoped, it get fixed... but it doesn't. So this  
> is the error-message from the log
> 
> Jän 03 16:27:08 aldebaran dovecot[26460]:  
> indexer-worker(jakob)<26476><qQ6g1+BhIJvAqAAO:sjYhMTH2TFpsZwAAk1Mx3g>:  
> Panic: file unichar.c: line 160 (uni_ucs4_to_utf8_c): assertion  
> failed: (uni_is_valid_ucs4(chr))
> Jän 03 16:27:08 aldebaran dovecot[26460]:  
> indexer-worker(jakob)<26476><qQ6g1+BhIJvAqAAO:sjYhMTH2TFpsZwAAk1Mx3g>:  
> Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0xc6021)  
> [0x7f8299f7a021] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xc60ed)  
> [0x7f8299f7a0ed] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)  
> [0x7f8299eec481] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(uni_ucs4_to_utf8_c+0xa0)  
> [0x7f8299fb1500] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xa75e0)  
> [0x7f8299f5b5e0] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(mail_html2text_more+0xc5)  
> [0x7f8299f5b775] ->  
> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0xcfcc) [0x7f82990aefcc]  
> -> /usr/local/lib/dovecot/lib20_fts_plugin.so(fts_parser_more+0x27)  
> [0x7f82990aeca7] ->  
> /usr/local/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x5e9)  
> [0x7f82990acc39] ->  
> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x1122d) [0x7f82990b322d]  
> -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x916a)  
> [0x7f82958e316a] ->  
> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x10f5d) [0x7f82990b2f5d]  
> -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x916a)  
> [0x7f82958e316a] ->  
> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x10f5d) [0x7f82990b2f5d]  
> -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x2e)  
> [0x7f829a2641be] -> dovecot/indexer-worker [jakob  
> Synoptic/AKTUELL](+0x2533) [0x562227882533] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69)  
> [0x7f8299f91bf9] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109)  
> [0x7f8299f93499] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x52)  
> [0x7f8299f91d02] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)  
> [0x7f8299f91f18] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)  
> [0x7f8299f0f1e3] -> dovecot/indexer-worker [jakob  
> Synoptic/AKTUELL](main+0xe7) [0x562227881f47] ->  
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)  
> [0x7f8299b352b1] -> dovecot/indexer-worker [jakob  
> Synoptic/AKTUELL](_start+0x2a) [0x56222788201a]
> Jän 03 16:27:08 aldebaran dovecot[26460]: indexer: Error: Indexer  
> worker disconnected, discarding 1 requests for jakob
> Jän 03 16:27:08 aldebaran dovecot[26460]:  
> imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Error: indexer failed to index  
> mailbox Synoptic/AKTUELL
> Jän 03 16:27:08 aldebaran dovecot[26460]:  
> indexer-worker(jakob)<26476><qQ6g1+BhIJvAqAAO:sjYhMTH2TFpsZwAAk1Mx3g>:  
> Fatal: master: service(indexer-worker): child 26476 killed with signal  
> 6 (core dumps disabled)
> Jän 03 16:27:09 aldebaran dovecot[26460]:  
> indexer-worker(jakob)<26484><qQ6g1+BhIJvAqAAO:J6mUIEz2TFp0ZwAAk1Mx3g>:  
> Error: lucene index  
> /var/lib/dovecot/db/indexes/Maildir/jakob/lucene-indexes:  
> IndexWriter() failed (#1): Lock obtain timed out
> Jän 03 16:27:10 aldebaran dovecot[26460]:  
> indexer-worker(jakob)<26484><qQ6g1+BhIJvAqAAO:J6mUIEz2TFp0ZwAAk1Mx3g>:  
> Error: Mailbox Synoptic/AKTUELL: Transaction commit failed: BUG:  
> Unknown internal error (attempted to index 1488 messages (UIDs  
> 15214..16775))
> Jän 03 16:27:10 aldebaran dovecot[26460]:  
> imap(jakob)<26480><lP5r2OBhKJvAqAAO>: Error: indexer failed to index  
> mailbox Synoptic/AKTUELL
> Jän 03 16:27:11 aldebaran dovecot[26460]:  
> imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Panic: file mail-index.c: line  
> 793 (mail_index_close): assertion failed: (index->open_count > 0)
> Jän 03 16:27:11 aldebaran dovecot[26460]:  
> imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Error: Raw backtrace:  
> /usr/local/lib/dovecot/libdovecot.so.0(+0xc6021) [0x7fb0fbd3a021] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(+0xc60ed) [0x7fb0fbd3a0ed] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fb0fbcac481] ->  
> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xf39a8)  
> [0x7fb0fc0d99a8] ->  
> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xd92d1)  
> [0x7fb0fc0bf2d1] ->  
> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xd9363)  
> [0x7fb0fc0bf363] ->  
> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_storage_mailbox_free+0x9)  
> [0x7fb0fc0b0f39] ->  
> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xd05a4)  
> [0x7fb0fc0b65a4] ->  
> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_free+0x19)  
> [0x7fb0fc030c99] ->  
> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xabdc)  
> [0x7fb0f729bbdc] ->  
> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xacb9)  
> [0x7fb0f729bcb9] ->  
> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a)  
> [0x7fb0fc030c0a] ->  
> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xac60)  
> [0x7fb0f729bc60] ->  
> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xacb9)  
> [0x7fb0f729bcb9] ->  
> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a)  
> [0x7fb0fc030c0a] ->  
> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_free+0x13)  
> [0x7fb0fc030c93] -> dovecot/imap [jakob 192.168.0.14  
> LOGOUT](imap_client_close_mailbox+0x5b) [0x56514d31305b] ->  
> dovecot/imap [jakob 192.168.0.14 LOGOUT](cmd_logout+0x35)  
> [0x56514d309485] -> dovecot/imap [jakob 192.168.0.14  
> LOGOUT](command_exec+0x64) [0x56514d312ca4] -> dovecot/imap [jakob  
> 192.168.0.14 LOGOUT](+0x1b0b2) [0x56514d3110b2] -> dovecot/imap [jakob  
> 192.168.0.14 LOGOUT](+0x1b13c) [0x56514d31113c] -> dovecot/imap [jakob  
> 192.168.0.14 LOGOUT](client_handle_input+0x1a5) [0x56514d311565] ->  
> dovecot/imap [jakob 192.168.0.14 LOGOUT](client_input+0x82)  
> [0x56514d311be2] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69)  
> [0x7fb0fbd51bf9] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109)  
> [0x7fb0fbd53499] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x52)  
> [0x7fb0fbd51d02] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)  
> [0x7fb0fbd51f18] ->  
> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)  
> [0x7fb0fbccf1e3]
> Jän 03 16:27:11 aldebaran dovecot[26460]:  
> imap(jakob)<26472><qQ6g1+BhIJvAqAAO>: Fatal: master: service(imap):  
> child 26472 killed with signal 6 (core dumps disabled)
> 
> 
> I get this errors when i open or search in a virtual folder. i have  
> enabled fts_lucene.
> 
> My config is:
> 
> # dovecot -n
> # 2.3.devel (393bfbea8): /usr/local/etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.0.rc1 (d68c23a1)
> # OS: Linux 4.13.0-trunk-amd64 x86_64 Debian 9.3 btrfs
> auth_mechanisms = plain login cram-md5 digest-md5
> auth_socket_path = /usr/local/var/run/dovecot/auth-userdb
> first_valid_uid = 1000
> imap_capability = +XDOVECOT
> imap_client_workarounds = tb-extra-mailbox-sep
> lda_mailbox_autocreate = yes
> lda_mailbox_autosubscribe = yes
> listen = *, ::1
> login_trusted_networks = 127.0.0.1/8 192.168.0.0/24 192.168.1.0/24  
> 172.17.0.0/24 172.18.0.0/24
> mail_gid = vmail
> mail_home =  /var/mail/%u
> mail_location =  
> maildir:/var/mail/%u/Maildir:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/Maildir/%u:INDEXPVT=/var/lib/dovecot/db/indexes/Maildir/%u:CONTROL=/var/lib/dovecot/db/control/Maildir/%u
> mail_plugins = zlib quota acl listescape mail_log notify virtual fts  
> fts_lucene
> mail_privileged_group = vmail
> mail_server_admin = mailto:jakob at xundeenergie.at
> mail_shared_explicit_inbox = yes
> mail_uid = vmail
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope  
> encoded-character vacation subaddress comparator-i;ascii-numeric  
> relational regex imap4flags copy include variables body enotify  
> environment mailbox date index ihave duplicate mime foreverypart  
> extracttext
> namespace {
>    list = children
>    location =  
> maildir:/var/mail/public/:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/public/%u:INDEXPVT=/var/lib/dovecot/db/indexes/public/%u
>    prefix = Roseggergasse/
>    separator = /
>    subscriptions = no
>    type = public
> }
> namespace Geteilt {
>    hidden = no
>    inbox = no
>    list = children
>    location =  
> maildir:/var/mail/%%u/Maildir:LAYOUT=fs:INDEXPVT=/var/lib/dovecot/db/indexes/shared/%u/%%u:INDEX=/var/lib/dovecot/db/indexes/shared/%u/%%u
>    prefix = Geteilt/%%n/
>    separator = /
>    subscriptions = no
>    type = shared
> }
> namespace Mailarchiv {
>    hidden = no
>    inbox = no
>    list = children
>    location =  
> maildir:/var/mail/mailarchiv/%u/:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/mailarchiv/%u:INDEXPVT=/var/lib/dovecot/db/indexes/mailarchiv/%u
>    mailbox incoming {
>      auto = create
>    }
>    mailbox outgoing {
>      auto = create
>    }
>    prefix = Mailarchiv/
>    separator = /
>    subscriptions = no
>    type = private
> }
> namespace Real {
>    hidden = yes
>    list = no
>    location = virtual:/var/mail/real:INDEX=/var/lib/dovecot/db/indexes/real/%u
>    prefix = Real/
>    separator = /
>    subscriptions = no
> }
> namespace Synoptic {
>    hidden = no
>    list = children
>    location =  
> virtual:/var/mail/virtual:INDEX=/var/lib/dovecot/db/indexes/virtual/%u
>    mailbox INBOX/Archives {
>      auto = no
>      special_use = \Archive
>    }
>    mailbox INBOX/Drafts {
>      auto = no
>      special_use = \Drafts
>    }
>    mailbox INBOX/Entwürfe {
>      auto = no
>      special_use = \Drafts
>    }
>    mailbox INBOX/Junk {
>      auto = no
>      special_use = \Junk
>    }
>    mailbox INBOX/Sent {
>      auto = no
>      special_use = \Sent
>    }
>    mailbox INBOX/Spam {
>      auto = no
>      special_use = \Junk
>    }
>    prefix = Synoptic/
>    separator = /
>    subscriptions = no
> }
> namespace inbox {
>    hidden = no
>    inbox = yes
>    location =
>    mailbox Archiv {
>      auto = no
>      special_use = \Archive
>    }
>    mailbox Archive {
>      auto = no
>      special_use = \Archive
>    }
>    mailbox Archives {
>      auto = no
>      special_use = \Archive
>    }
>    mailbox "Deleted Messages" {
>      auto = no
>      autoexpunge = 30 days
>      special_use = \Trash
>    }
>    mailbox Drafts {
>      auto = no
>      special_use = \Drafts
>    }
>    mailbox Entwürfe {
>      auto = no
>      special_use = \Drafts
>    }
>    mailbox "Gelöschte Elemente" {
>      auto = no
>      autoexpunge = 30 days
>      special_use = \Trash
>    }
>    mailbox "Gelöschte Objekte" {
>      auto = no
>      autoexpunge = 30 days
>      special_use = \Trash
>    }
>    mailbox Gesendet {
>      auto = no
>      special_use = \Sent
>    }
>    mailbox "Gesendete Elemente" {
>      auto = no
>      special_use = \Sent
>    }
>    mailbox "Gesendete Objekte" {
>      auto = no
>      special_use = \Sent
>    }
>    mailbox Important {
>      auto = no
>    }
>    mailbox Junk {
>      auto = subscribe
>      autoexpunge = 30 days
>      special_use = \Junk
>    }
>    mailbox Mistkübel {
>      auto = no
>      autoexpunge = 30 days
>      special_use = \Trash
>    }
>    mailbox Papierkorb {
>      auto = no
>      autoexpunge = 30 days
>      special_use = \Trash
>    }
>    mailbox Sent {
>      auto = subscribe
>      special_use = \Sent
>    }
>    mailbox "Sent Messages" {
>      auto = no
>      special_use = \Sent
>    }
>    mailbox Spam {
>      auto = no
>      autoexpunge = 30 days
>      special_use = \Junk
>    }
>    mailbox Synoptic/Alle {
>      auto = no
>      comment = All my messages
>      special_use = \All
>    }
>    mailbox Trash {
>      auto = no
>      autoexpunge = 30 days
>      special_use = \Trash
>    }
>    mailbox Wichtig {
>      auto = create
>    }
>    prefix = INBOX/
>    separator = /
>    subscriptions = no
>    type = private
> }
> namespace subscriptions {
>    hidden = yes
>    list = no
>    location =
>    prefix =
>    subscriptions = yes
> }
> passdb {
>    args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/users
>    driver = passwd-file
> }
> plugin {
>    acl = vfile:/etc/dovecot/dovecot-acl:cache_secs=300
>    acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db
>    fts = lucene
>    fts_autoindex = yes
>    fts_autoindex_max_recent_msgs = 80
>    fts_index_timeout = 90s
>    fts_lucene = whitespace_chars=@. normalize no_snowball
>    mail_home =  /var/mail/%u
>    setting_name = sieve, managedsieve
>    sieve = file:/var/mail/%u/sieve/;active=/var/mail/%u/sieve/%u.sieve
> }
> postmaster_address = postmaster at localhost
> protocols = imap pop3 lmtp imap lmtp sieve pop3 sieve
> service anvil {
>    unix_listener anvil-auth-penalty {
>      mode = 00
>    }
> }
> service auth {
>    unix_listener auth-client {
>      group = vmail
>      mode = 0666
>      user = Debian-exim
>    }
>    unix_listener auth-userdb {
>      group = vmail
>      mode = 0666
>      user = vmail
>    }
> }
> service imap-login {
>    inet_listener imap {
>      port = 143
>    }
>    inet_listener imaps {
>      port = 993
>      ssl = yes
>    }
>    process_min_avail = 1
>    service_count = 0
> }
> service imap {
>    executable = imap postlogin
>    process_limit = 1024
>    vsz_limit = 400 M
> }
> service lmtp {
>    executable = lmtp -L
>    user = vmail
>    vsz_limit = 400 M
> }
> service postlogin {
>    executable = script-login -d rawlog
> }
> ssl = required
> ssl_cert = </usr/local/etc/dovecot/dovecot.crt
> ssl_dh =  # hidden, use -P to show it
> ssl_key =  # hidden, use -P to show it
> userdb {
>    args = username_format=%u /usr/local/etc/dovecot/users
>    default_fields = home=/var/mail/%u
>    driver = passwd-file
> }
> verbose_proctitle = yes
> protocol lmtp {
>    auth_username_format = %n
>    mail_plugins = zlib quota acl listescape mail_log notify virtual  
> fts fts_lucene quota sieve acl
> }
> protocol lda {
>    mail_plugins = zlib quota acl listescape mail_log notify virtual  
> fts fts_lucene quota sieve acl
> }
> protocol imap {
>    mail_max_userip_connections = 10
>    mail_plugins = zlib quota acl listescape mail_log notify virtual  
> fts fts_lucene imap_quota imap_acl
> }
> 
> 
> The virtual Folder-definitions are:
> 
> # find /var/mail/virtual/ /var/mail/real/
> /var/mail/virtual/
> /var/mail/virtual/GINBOX
> /var/mail/virtual/GINBOX/dovecot-virtual
> /var/mail/virtual/Neue
> /var/mail/virtual/Neue/dovecot-virtual
> /var/mail/virtual/Markierte
> /var/mail/virtual/Markierte/dovecot-virtual
> /var/mail/virtual/Ungelesene
> /var/mail/virtual/Ungelesene/dovecot-virtual
> /var/mail/virtual/Sent
> /var/mail/virtual/Sent/dovecot-virtual
> /var/mail/virtual/Drafts
> /var/mail/virtual/Drafts/dovecot-virtual
> /var/mail/virtual/Trash
> /var/mail/virtual/Trash/dovecot-virtual
> /var/mail/virtual/AKTUELL
> /var/mail/virtual/AKTUELL/dovecot-virtual
> /var/mail/virtual/Archives
> /var/mail/virtual/Archives/dovecot-virtual
> /var/mail/virtual/dovecot-acl-list
> /var/mail/virtual/WICHTIG
> /var/mail/virtual/WICHTIG/dovecot-virtual
> /var/mail/real/
> /var/mail/real/Alle
> /var/mail/real/Alle/dovecot-virtual
> /var/mail/real/dovecot-acl-list
> /var/mail/real/einjahr
> /var/mail/real/einjahr/dovecot-virtual
> /var/mail/real/zweijahr
> /var/mail/real/zweijahr/dovecot-virtual
> /var/mail/real/dreijahr
> /var/mail/real/dreijahr/dovecot-virtual
> 
> # cat /var/mail/virtual/AKTUELL/dovecot-virtual
> Real/Alle
>   inthread refs younger 2592000
> 
>   # cat /var/mail/real/Alle/dovecot-virtual
> !INBOX
> INBOX/*
> -INBOX/Trash
> -INBOX/Trash/*
> -INBOX/Archives
> -INBOX/Archives/*
> -INBOX/Drafts
> -INBOX/Drafts/*
> -INBOX/Junk
> -INBOX/Junk/*
> -INBOX/SYSTEM
> -INBOX/SYSTEM/*
>   all
> 
> The others are similar.
> 
> Greez
> 
> Jakob


More information about the dovecot mailing list