Bug in dovecot 2.3 virtual plugin
Jakobus Schürz
wertstoffe at nurfuerspam.de
Wed Feb 7 12:22:47 EET 2018
Hi there!
I compiled dovecot from git on 5.2.2018.
The id of the last commit is:
$ git log --format="%H" -n 1
83b1300b96e753206f9226e3738a7ad9f5428a75
From https://github.com/dovecot/core.git
There is a new error, when i try to open a virtual folder for my archiv
(almost 30.000 mails) in thunderbird, also in k9-mail on android.
Feb 07 11:09:55 aldebaran dovecot[7455]:
imap(jakob)<7472><U2Uvf5xkCQRcPAkS>: Panic: Leaked view for index
/var/lib/dovecot/db/indexes/archiv/jakob/Archiv/2016/Mailinglisten/stvkr-a/dovecot.index:
Opened in index-storage.c:336
Feb 07 11:09:55 aldebaran dovecot[7455]:
imap(jakob)<7472><U2Uvf5xkCQRcPAkS>: Error: Raw backtrace:
/usr/local/lib/dovecot/libdovecot.so.0(+0xc9431) [0x7f6edc3dd431] ->
/usr/local/lib/dovecot/libdovecot.so.0(+0xc94fa) [0x7f6edc3dd4fa] ->
/usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f6edc34fbe8] ->
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0xf44f2)
[0x7f6edc77b4f2] ->
/usr/local/lib/dovecot/libdovecot-storage.so.0(index_storage_mailbox_close+0x72)
[0x7f6edc752d32] ->
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a)
[0x7f6edc6d227a] ->
/usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xa700) [0x7f6ed78e8700]
->
/usr/local/lib/dovecot/lib20_virtual_plugin.so(virtual_backend_box_open+0x2b)
[0x7f6ed78e889b] ->
/usr/local/lib/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0x12a9)
[0x7f6ed78eba79] ->
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x51)
[0x7f6edc6d4781] ->
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37)
[0x7f6edc6d4837] -> dovecot/imap [jakob 92.60.9.18
select](cmd_select_full+0x17e) [0x55bde8d2a08e] -> dovecot/imap [jakob
92.60.9.18 select](command_exec+0x64) [0x55bde8d318d4] -> dovecot/imap
[jakob 92.60.9.18 select](+0x1bd82) [0x55bde8d2fd82] -> dovecot/imap
[jakob 92.60.9.18 select](+0x1be1c) [0x55bde8d2fe1c] -> dovecot/imap
[jakob 92.60.9.18 select](client_handle_input+0x1b5) [0x55bde8d30225] ->
dovecot/imap [jakob 92.60.9.18 select](client_input+0xa4)
[0x55bde8d30844] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69)
[0x7f6edc3f51c9] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12e)
[0x7f6edc3f6a3e] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c)
[0x7f6edc3f52cc] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)
[0x7f6edc3f54d8] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f6edc3725d3] -> dovecot/imap [jakob 92.60.9.18 select](main+0x339)
[0x55bde8d22599] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7f6edbf7ef2a]
-> dovecot/imap [jakob 92.60.9.18 select](_start+0x2a) [0x55bde8d2277a]
Feb 07 11:09:55 aldebaran dovecot[7455]:
imap(jakob)<7472><U2Uvf5xkCQRcPAkS>: Fatal: master: service(imap): child
7472 killed with signal 6 (core not dumped)
Feb 07 11:09:55 aldebaran dovecot[7455]: imap-login: Login:
user=<jakob>, method=CRAM-MD5, rip=1.2.3.4, lip=192.168.0.14, mpid=7474,
TLS, session=<tHw8f5xkCwRcPAkS>
Feb 07 11:09:56 aldebaran dovecot[7455]:
imap(jakob)<7474><tHw8f5xkCwRcPAkS>: Panic: Leaked view for index
/var/lib/dovecot/db/indexes/archiv/jakob/Archiv/2016/Mailinglisten/stvkr-a/dovecot.index:
Opened in index-storage.c:336
Feb 07 11:09:56 aldebaran dovecot[7455]:
imap(jakob)<7474><tHw8f5xkCwRcPAkS>: Error: Raw backtrace:
/usr/local/lib/dovecot/libdovecot.so.0(+0xc9431) [0x7fc44d019431] ->
/usr/local/lib/dovecot/libdovecot.so.0(+0xc94fa) [0x7fc44d0194fa] ->
/usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fc44cf8bbe8] ->
/usr/local/lib/dovecot/libdovecot-storage.so.0(+0xf44f2)
[0x7fc44d3b74f2] ->
/usr/local/lib/dovecot/libdovecot-storage.so.0(index_storage_mailbox_close+0x72)
[0x7fc44d38ed32] ->
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a)
[0x7fc44d30e27a] ->
/usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xa700) [0x7fc448524700]
->
/usr/local/lib/dovecot/lib20_virtual_plugin.so(virtual_backend_box_open+0x2b)
[0x7fc44852489b] ->
/usr/local/lib/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0x12a9)
[0x7fc448527a79] ->
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x51)
[0x7fc44d310781] ->
/usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37)
[0x7fc44d310837] -> dovecot/imap [jakob 92.60.9.18
select](cmd_select_full+0x17e) [0x55770c32408e] -> dovecot/imap [jakob
92.60.9.18 select](command_exec+0x64) [0x55770c32b8d4] -> dovecot/imap
[jakob 92.60.9.18 select](+0x1bd82) [0x55770c329d82] -> dovecot/imap
[jakob 92.60.9.18 select](+0x1be1c) [0x55770c329e1c] -> dovecot/imap
[jakob 92.60.9.18 select](client_handle_input+0x1b5) [0x55770c32a225] ->
dovecot/imap [jakob 92.60.9.18 select](client_input+0xa4)
[0x55770c32a844] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69)
[0x7fc44d0311c9] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x12e)
[0x7fc44d032a3e] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c)
[0x7fc44d0312cc] ->
/usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38)
[0x7fc44d0314d8] ->
/usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7fc44cfae5d3] -> dovecot/imap [jakob 92.60.9.18 select](main+0x339)
[0x55770c31c599] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7fc44cbbaf2a]
-> dovecot/imap [jakob 92.60.9.18 select](_start+0x2a) [0x55770c31c77a]
Feb 07 11:09:56 aldebaran dovecot[7455]:
imap(jakob)<7474><tHw8f5xkCwRcPAkS>: Fatal: master: service(imap): child
7474 killed with signal 6 (core not dumped)
The systemd-Unit for my self-compiled dovecot has set some ulimits,
because i got problems on a large amount of mails with doveadm,
increasing the ulimit avoid this problems:
# systemctl cat xe-dovecot.service |grep -v '^#'
[Unit]
Description=IMAP server (dovecot)
OnFailure=status-email-root@%n.service
PartOf=mailserver.service xe-dovecot.socket
ReloadPropagatedFrom=mailserver.service
[Service]
ExecStart=/usr/local/sbin/dovecot -F -c /usr/local/etc/dovecot/dovecot.conf
ExecReload=/usr/local/sbin/dovecot reload
ExecStop=/usr/local/sbin/dovecot stop
Restart=on-success
RestartSec=2
LimitAS=infinity
LimitRSS=infinity
LimitCORE=infinity
LimitNOFILE=infinity
PrivateTmp=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
NoNewPrivileges=true
[Install]
WantedBy=mailserver.target
My dovecot-conf is:
# doveconf -n
# 2.3.devel (83b1300b9): /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.0.rc1 (d33dca20)
# OS: Linux 4.14.0-3-amd64 x86_64 Debian buster/sid btrfs
# Hostname: aldebaran.xundeenergie.at
auth_mechanisms = plain login cram-md5 digest-md5
auth_socket_path = /usr/local/var/run/dovecot/auth-userdb
first_valid_uid = 1000
imap_capability = +XDOVECOT
imap_client_workarounds = tb-extra-mailbox-sep
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
listen = *, ::1
login_trusted_networks = 127.0.0.1/8 192.168.0.0/24 192.168.1.0/24
172.17.0.0/24 172.18.0.0/24
mail_gid = vmail
mail_home = /var/mail/%u
mail_location =
maildir:/var/mail/%u/Maildir:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/Maildir/%u:INDEXPVT=/var/lib/dovecot/db/indexes/Maildir/%u:CONTROL=/var/lib/dovecot/db/control/Maildir/%u
mail_plugins = zlib quota acl listescape mail_log notify virtual fts
fts_lucene
mail_privileged_group = vmail
mail_server_admin = mailto:jakob at xundeenergie.at
mail_shared_explicit_inbox = yes
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart extracttext
mmap_disable = yes
namespace {
list = children
location =
maildir:/var/mail/public/:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/public/%u:INDEXPVT=/var/lib/dovecot/db/indexes/public/%u
prefix = Roseggergasse/
separator = /
subscriptions = no
type = public
}
namespace Archiv {
hidden = no
inbox = no
list = children
location =
maildir:/var/mail/archiv/%u/:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/archiv/%u:INDEXPVT=/var/lib/dovecot/db/indexes/archiv/%u:CONTROL=/var/lib/dovecot/db/control/archiv/%u
mailbox Archiv {
auto = subscribe
special_use = \Archive
}
mailbox Archive {
auto = no
special_use = \Archive
}
mailbox Archives {
auto = no
special_use = \Archive
}
prefix = Archiv/
separator = /
subscriptions = no
type = private
}
namespace Geteilt {
hidden = no
inbox = no
list = children
location =
maildir:/var/mail/%%u/Maildir:LAYOUT=fs:INDEXPVT=/var/lib/dovecot/db/indexes/shared/%u/%%u:INDEX=/var/lib/dovecot/db/indexes/shared/%u/%%u
prefix = Geteilt/%%n/
separator = /
subscriptions = no
type = shared
}
namespace Mailarchiv {
hidden = no
inbox = no
list = children
location =
maildir:/var/mail/mailarchiv/%u/:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/mailarchiv/%u:INDEXPVT=/var/lib/dovecot/db/indexes/mailarchiv/%u:CONTROL=/var/lib/dovecot/db/control/mailarchiv/%u
mailbox incoming {
auto = create
}
mailbox outgoing {
auto = create
}
prefix = Mailarchiv/
separator = /
subscriptions = no
type = private
}
namespace Real {
hidden = yes
ignore_on_failure = no
list = no
location =
virtual:/var/mail/real:INDEX=/var/lib/dovecot/db/indexes/real/%u
prefix = Real/
separator = /
subscriptions = no
}
namespace Synoptic {
hidden = no
ignore_on_failure = no
list = children
location =
virtual:/var/mail/virtual:INDEX=/var/lib/dovecot/db/indexes/virtual/%u
mailbox AKTUELL {
auto = subscribe
}
mailbox INBOX/Archives {
auto = no
special_use = \Archive
}
mailbox INBOX/Drafts {
auto = no
special_use = \Drafts
}
mailbox INBOX/Entwürfe {
auto = no
special_use = \Drafts
}
mailbox INBOX/Junk {
auto = no
special_use = \Junk
}
mailbox INBOX/Sent {
auto = no
special_use = \Sent
}
mailbox INBOX/Spam {
auto = no
special_use = \Junk
}
mailbox Neue {
auto = subscribe
}
mailbox Real/Alle {
auto = no
}
mailbox WICHTIG {
auto = subscribe
}
prefix = Synoptic/
separator = /
subscriptions = no
}
namespace inbox {
hidden = no
inbox = yes
location =
mailbox "Deleted Messages" {
auto = no
autoexpunge = 30 days
special_use = \Trash
}
mailbox Drafts {
auto = no
special_use = \Drafts
}
mailbox Entwürfe {
auto = no
special_use = \Drafts
}
mailbox "Gelöschte Elemente" {
auto = no
autoexpunge = 30 days
special_use = \Trash
}
mailbox "Gelöschte Objekte" {
auto = no
autoexpunge = 30 days
special_use = \Trash
}
mailbox Gesendet {
auto = no
special_use = \Sent
}
mailbox "Gesendete Elemente" {
auto = no
special_use = \Sent
}
mailbox "Gesendete Objekte" {
auto = no
special_use = \Sent
}
mailbox Junk {
auto = subscribe
autoexpunge = 30 days
special_use = \Junk
}
mailbox Mistkübel {
auto = no
autoexpunge = 30 days
special_use = \Trash
}
mailbox Papierkorb {
auto = no
autoexpunge = 30 days
special_use = \Trash
}
mailbox SYSTEM {
auto = subscribe
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox "Sent Messages" {
auto = no
special_use = \Sent
}
mailbox Spam {
auto = no
autoexpunge = 30 days
special_use = \Junk
}
mailbox Synoptic/Alle {
auto = no
comment = All my messages
special_use = \All
}
mailbox Trash {
auto = no
autoexpunge = 30 days
special_use = \Trash
}
prefix = INBOX/
separator = /
subscriptions = no
type = private
}
namespace subscriptions {
hidden = yes
list = no
location =
prefix =
subscriptions = yes
}
passdb {
args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/users
driver = passwd-file
}
plugin {
acl = vfile:/usr/local/etc/dovecot/dovecot-acl:cache_secs=300
acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db
fts = lucene
fts_autoindex = yes
fts_autoindex_max_recent_msgs = 80
fts_index_timeout = 90
fts_lucene = whitespace_chars=@. normalize no_snowball
mail_home = /var/mail/%u
setting_name = sieve, managedsieve
sieve = file:/var/mail/%u/sieve/;active=/var/mail/%u/sieve/%u.sieve
sieve_before = /usr/local/etc/dovecot/sieve/global.sieve
sieve_global_dir = /usr/local/etc/dovecot/sieve/
}
postmaster_address = postmaster at localhost
protocols = imap pop3 lmtp sieve
service anvil {
client_limit = 199999
unix_listener anvil-auth-penalty {
mode = 0700
}
}
service auth {
unix_listener auth-client {
group = vmail
mode = 0666
user = Debian-exim
}
unix_listener auth-userdb {
group = vmail
mode = 0666
user = vmail
}
}
service imap-login {
inet_listener imap {
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
process_min_avail = 1
service_count = 0
}
service imap {
executable = imap postlogin
process_limit = 1024
vsz_limit = 400 M
}
service lmtp {
executable = lmtp -L
user = vmail
vsz_limit = 400 M
}
service postlogin {
executable = script-login -d rawlog
}
ssl = required
ssl_cert = </usr/local/etc/dovecot/dovecot.crt
ssl_dh = # hidden, use -P to show it
ssl_key = # hidden, use -P to show it
userdb {
args = username_format=%u /usr/local/etc/dovecot/users
default_fields = home=/var/mail/%u
driver = passwd-file
}
verbose_proctitle = yes
protocol lmtp {
auth_username_format = %n
mail_plugins = zlib quota acl listescape mail_log notify virtual fts
fts_lucene quota sieve acl
}
protocol lda {
mail_plugins = zlib quota acl listescape mail_log notify virtual fts
fts_lucene quota sieve acl
}
protocol imap {
mail_max_userip_connections = 30
mail_plugins = zlib quota acl listescape mail_log notify virtual fts
fts_lucene imap_quota imap_acl
}
This is unchanged from my original bug-report:
> The virtual Folder-definitions are:
>
> # find /var/mail/virtual/ /var/mail/real/
> /var/mail/virtual/
> /var/mail/virtual/GINBOX
> /var/mail/virtual/GINBOX/dovecot-virtual
> /var/mail/virtual/Neue
> /var/mail/virtual/Neue/dovecot-virtual
> /var/mail/virtual/Markierte
> /var/mail/virtual/Markierte/dovecot-virtual
> /var/mail/virtual/Ungelesene
> /var/mail/virtual/Ungelesene/dovecot-virtual
> /var/mail/virtual/Sent
> /var/mail/virtual/Sent/dovecot-virtual
> /var/mail/virtual/Drafts
> /var/mail/virtual/Drafts/dovecot-virtual
> /var/mail/virtual/Trash
> /var/mail/virtual/Trash/dovecot-virtual
> /var/mail/virtual/AKTUELL
> /var/mail/virtual/AKTUELL/dovecot-virtual
> /var/mail/virtual/Archives
> /var/mail/virtual/Archives/dovecot-virtual
> /var/mail/virtual/dovecot-acl-list
> /var/mail/virtual/WICHTIG
> /var/mail/virtual/WICHTIG/dovecot-virtual
> /var/mail/real/
> /var/mail/real/Alle
> /var/mail/real/Alle/dovecot-virtual
> /var/mail/real/dovecot-acl-list
> /var/mail/real/einjahr
> /var/mail/real/einjahr/dovecot-virtual
> /var/mail/real/zweijahr
> /var/mail/real/zweijahr/dovecot-virtual
> /var/mail/real/dreijahr
> /var/mail/real/dreijahr/dovecot-virtual
>
> # cat /var/mail/virtual/AKTUELL/dovecot-virtual
> Real/Alle
> inthread refs younger 2592000
>
> # cat /var/mail/real/Alle/dovecot-virtual
> !INBOX
> INBOX/*
> -INBOX/Trash
> -INBOX/Trash/*
> -INBOX/Archives
> -INBOX/Archives/*
> -INBOX/Drafts
> -INBOX/Drafts/*
> -INBOX/Junk
> -INBOX/Junk/*
> -INBOX/SYSTEM
> -INBOX/SYSTEM/*
> all
>
> The others are similar.
>
> Greez
>
> Jakob
More information about the dovecot
mailing list