I tried with 32k messages, and I wasn't able to provoke it to fail.

Hmmm...

 

I changed the dovecot-virtual from Real/Alle from

/var/mail/real/Alle: # cat dovecot-virtual
!INBOX
INBOX/*
-INBOX/Trash
-INBOX/Trash/*
-INBOX/Archives
-INBOX/Archives/*
-INBOX/Drafts
-INBOX/Drafts/*
-INBOX/Junk
-INBOX/Junk/*
-INBOX/SYSTEM
-INBOX/SYSTEM/*
 all

 

 (which gives the error)

 

to

/var/mail/real/Alle: # cat dovecot-virtual
!INBOX
INBOX/*
-INBOX/Trash
-INBOX/Trash/*
-INBOX/Archives
-INBOX/Archives/*
-INBOX/Drafts
-INBOX/Drafts/*
-INBOX/Junk
-INBOX/Junk/*
-INBOX/SYSTEM
-INBOX/SYSTEM/*
 all younger 525600

 

And thunderbird gets all this emails without error in dovecot.

 

Maybe there is a Problem with the amount of emails? Memory-restrictions? Settings for the process? ulimits? Or some restrictions in dovecot for the count of emails?

 

Jakob

 

Gesendet: Donnerstag, 26. Juli 2018 um 11:26 Uhr
Von: "Aki Tuomi" <aki.tuomi@dovecot.fi>
An: wertstoffe@nurfuerspam.de, dovecot@dovecot.org
Betreff: Re: Aw: Re: Panic: Leaked view for index - virtual mailbox

I am pretty sure Synoptic/INBOX/Archives will not be same as INBOX/Archives.

a SELECT Synoptic/INBOX/Archives
a NO Mailbox doesn't exist: Synoptic/INBOX/Archives (0.001 + 0.000 secs).
 

a SELECT INBOX/Archives
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
* 45 EXISTS

So as you can see, specifying 'INBOX/Archives' under 'Synoptic' namespace means it will not be the same as INBOX/Archives. It is a separate mailbox, and the only way it can work is if you have INBOX/Archives/dovecot-virtual file which specifies that it reads & writes to INBOX/Archives. This is not the way to make some kind of fancy aliasing.

I took a stab at reproducing your issue, but so far no luck. I have to try including the Archiv namespace as well.

Aki
 

On 26.07.2018 12:01, wertstoffe@nurfuerspam.de wrote:

Hello!

 

(and sorry for double mailings... i have some troubles with my mailprovider 1&1... :-/ )
Yes, i have for all of them similar Folders and Configurations. But i thought, it was to much to send you, because, they all are similar.

There is one Namespace "Real" with according Folder in /var/mail/real.
This Namespace is hidden to the User. It is used to "prepare" the views to the emails. All Emails, only all emails from the last month, halfyear, year, 2 years and so on. The Archive in the dovecot-virtual config in "Archiv" is a separate Namespace. (look below)

/var/mail/real: # l
insgesamt 0
drwxr-sr-x 1 vmail vmail 30 Jul 12 17:47 Alle
drwxr-sr-x 1 vmail vmail 30 Jul 10 00:06 Archiv
drwxr-sr-x 1 vmail vmail 30 Jul 10 00:06 Archiv_zweijahr
-rw-r--r-- 1 vmail vmail  0 Nov 27  2017 dovecot-acl-list
drwxr-sr-x 1 vmail vmail 30 Jul 12 11:06 dreijahr
drwxr-sr-x 1 vmail vmail 30 Jul 12 11:06 einjahr
drwxr-sr-x 1 vmail vmail 30 Jul 10 00:06 einmonat
drwxr-sr-x 1 vmail vmail 30 Jul 12 17:46 halbjahr
drwxr-sr-x 1 vmail vmail 30 Jul 12 11:06 zweijahr
drwxr-sr-x 1 vmail vmail 30 Jul 10 00:06 zweimonat

For example:
/var/mail/real: # cat Alle/dovecot-virtual
!INBOX
INBOX/*
-INBOX/Trash
-INBOX/Trash/*
-INBOX/Archives
-INBOX/Archives/*
-INBOX/Drafts
-INBOX/Drafts/*
-INBOX/Junk
-INBOX/Junk/*
-INBOX/SYSTEM
-INBOX/SYSTEM/*
 all

The "All-Mailbox" for only one year:
/var/mail/real: # cat einjahr/dovecot-virtual
!Real/Alle
 all younger 31536000

The others "zweijahr", "halbjahr" and so on keep the same definition, but with other count of seconds... this is for playing around, which is the best period to hold in the "AKTUELLE" Mailbox in Synoptic. And to choose in Thunderbird or other Mailclients, for faster syncing or more emails.

and the Archiv:
/var/mail/real: # cat Archiv/dovecot-virtual
Archiv/*
 all

And now the other Folder /var/mail/virtual, which contains the Namespace "SYNOPTIC". Here i collect different Views for my emails. This Namespace is shown to the User, and i can subscribe them in the MUA.

/var/mail/virtual: # cat AKTUELL/dovecot-virtual
!Real/halbjahr
 all

/var/mail/virtual: # cat ARCHIV/dovecot-virtual
Archiv
Archiv/*
 all younger 1261440000

/var/mail/virtual: # cat Ungelesene/dovecot-virtual
Real/Alle
 unseen

/var/mail/virtual: # cat Sent/dovecot-virtual
!INBOX/Sent
INBOX/Sent/*
 all

and the Namespace "Archive" is located in
/var/mail/archiv
 and contains all archieved emails. (There are really a lot Emails).
I declare this Namespace as "Special Folder", so Thunderbird picks it up as the Archive-Folder for automatic archieving old emails.

The thing is, I get the error on the "Alle", "Send", "ungelesene"... Mailboxes. The Archiv-Mailbox is working correctly!

Another funny thing is... I got this error on older versions and also on version 2.4 (=master in git). But not all the time.
Some day it worked without error... It worked until i deleted the whole index-directories a few days ago... then i got the error. Without changing the config!!

Uhmm... Yes, there is this Synoptic/INBOX/Archives Mailbox. This is for a special Mailclient "K9Mail" (on Android). Because i used there only the Synoptic-Namespace. And K9Mail created itself an own Archive-Folder, if there is no one below INBOX. With this solution, K9Mail shows the same Archive as Thunderbird shows ist as Special folder.


The Gmail-App handles different Namespaces at the same time, but cannot handle different Identities.
K9Mail can handle different Identities (for Alias-Emailadresses) for the same Account, but only one Namespace...

Jakob

 

Gesendet: Donnerstag, 26. Juli 2018 um 07:54 Uhr
Von: "Aki Tuomi" <aki.tuomi@dovecot.fi>
An: "Jakobus Schürz" <jakobus.schuerz@ebcont.com>, dovecot@dovecot.org
Betreff: Re: Panic: Leaked view for index - virtual mailbox

Hi!

Was looking at your config and I am puzzled why you are adding

  mailbox INBOX/Archives {
    auto = no
    special_use = \Archive
  }

under namespace 'Synoptic'? Do you actually have a virtual folder config
for all those folders? You can't just "move" folders into a namespace by
declaring them there, those become Synoptic/INBOX/Archives, and require
a dovecot-virtual file to work.

If the only mailbox in Synoptic, that has a dovecot-virtual file is
'AKTUELL', you should remove all folders that do not have a
dovecot-virtual file under /var/mail/virtual.

Aki


On 25.07.2018 10:46, Jakobus Schürz wrote:
> Hi Aki!
>
> The config is in the attachment.
>
> The output from gdb is this:
>
>
> # gdb /usr/lib/dovecot/imap /var/core/24491
> GNU gdb (Debian 8.1-3) 8.1
> Copyright (C) 2018 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-linux-gnu".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>.
> Find the GDB manual and other documentation resources online at:
> <http://www.gnu.org/software/gdb/documentation/>.
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> /usr/lib/dovecot/imap: Datei oder Verzeichnis nicht gefunden.
> [New LWP 24491]
> Core was generated by `dovecot/imap [jakob 213'.
> Program terminated with signal SIGABRT, Aborted.
> #0  0x00007f5c8ac4cf3b in ?? ()
> (gdb) bt full
> #0  0x00007f5c8ac4cf3b in ?? ()
> No symbol table info available.
> #1  0x0000000000000000 in ?? ()
> No symbol table info available.
> (gdb) quit
>
> And i have a complex setup for virtual mailboxes...
>
> For Synoptic/AKTUELL it is:
>
> # cat /var/mail/virtual/AKTUELL/dovecot-virtual
> !Real/halbjahr
>  all
>
> And for Real/halbjahr it is:
>
> # cat /var/mail/real/halbjahr/dovecot-virtual
> !Real/Alle
>  all younger 15768000
>
> And for Real/Alle it is:
>
> # cat /var/mail/real/Alle/dovecot-virtual
> !INBOX
> INBOX/*
> -INBOX/Trash
> -INBOX/Trash/*
> -INBOX/Archives
> -INBOX/Archives/*
> -INBOX/Drafts
> -INBOX/Drafts/*
> -INBOX/Junk
> -INBOX/Junk/*
> -INBOX/SYSTEM
> -INBOX/SYSTEM/*
>  all
>
>
> jakob
>
>
> Am 25.07.2018 um 09:11 schrieb Aki Tuomi:
>> So the core file is not very useful...
>>
>> can you, instead, attach your doveconf -n and your virtual folder
>> configurations and also
>>
>> gdb /usr/lib/dovecot/imap /path/to/core
>> bt full
>>
>> Aki
>>
>>> On 25 July 2018 at 09:55 Jakobus Schürz <jakobus.schuerz@ebcont.com>
>>> wrote:
>>>
>>>
>>> Hi Aki!
>>>
>>> This is debian buster, but a self-compiled dovecot from git!
>>>
>>> /usr/local/sbin/dovecot --version
>>> 2.3.2.1 (0719df592)
>>>
>>> jakob
>>>
>>>
>>> Am 25.07.2018 um 08:30 schrieb Aki Tuomi:
>>>> And can you provide dovecot --version too, please.
>>>>
>>>> Aki
>>>>
>>>>> On 25 July 2018 at 09:18 Aki Tuomi <aki.tuomi@dovecot.fi> wrote:
>>>>>
>>>>>
>>>>> Hi!
>>>>>
>>>>> Which debian release is this?
>>>>>
>>>>> Aki
>>>>>
>>>>>> On 25 July 2018 at 05:18 Jakobus Schürz
>>>>>> <jakobus.schuerz@ebcont.com> wrote:
>>>>>>
>>>>>>
>>>>>> Hi There!
>>>>>>
>>>>>> I run into this problem with virtual Mailboxes on release-2.3.2 (and
>>>>>> also in master and in older versions of dovecot)
>>>>>>
>>>>>> I added the core-file.
>>>>>>
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]:
>>>>>> imap(jakob)<11292><rg2GZclxHaPAqADc>: Panic: Leaked view for index
>>>>>> /var/lib/dovecot/db/indexes/Maildir/jakob/&ANY-BB/dovecot.index:
>>>>>> Opened
>>>>>> in index-storage.c:338
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]:
>>>>>> imap(jakob)<11292><rg2GZclxHaPAqADc>: Error: Raw backtrace:
>>>>>> /usr/local/lib/dovecot/libdovecot.so.0(+0xcedf1) [0x7ffad9ac0df1] ->
>>>>>> /usr/local/lib/dovecot/libdovecot.so.0(+
>>>>>> 0xceeba) [0x7ffad9ac0eba] ->
>>>>>> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
>>>>>> [0x7ffad9a2efe8] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xf7022)
>>>>>> [0x7ffad9c66022] -> /usr/local/lib/dovecot/libdovecot-s
>>>>>> torage.so.0(index_storage_mailbox_close+0x9a) [0x7ffad9c3d72a] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a)
>>>>>> [0x7ffad9bbb1aa] ->
>>>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xab80) [0
>>>>>> x7ffad9792b80] ->
>>>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(virtual_backend_box_open+0x2b)
>>>>>>
>>>>>> [0x7ffad9792d3b] ->
>>>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0x12a9)
>>>>>>
>>>>>> [0x7ffad9795f1
>>>>>> 9] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x51)
>>>>>>
>>>>>> [0x7ffad9bbd6d1] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37)
>>>>>> [0x7ffad9bbd787] -> /usr/local/lib/dovecot/libdo
>>>>>> vecot-storage.so.0(index_storage_get_status+0x31)
>>>>>> [0x7ffad9c3bed1] ->
>>>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x9ead)
>>>>>> [0x7ffad9791ead]
>>>>>> -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xb0f83) [0x7ffad
>>>>>> 9c1ff83] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x10602)
>>>>>> [0x7ffad97b4602] ->
>>>>>> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0xf813)
>>>>>> [0x7ffad99e9813] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox
>>>>>> _get_status+0x31) [0x7ffad9bbdb91] ->
>>>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0xf0b)
>>>>>>
>>>>>> [0x7ffad9795b7b] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x51)
>>>>>> [0
>>>>>> x7ffad9bbd6d1] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37)
>>>>>> [0x7ffad9bbd787] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x31)
>>>>>>
>>>>>> [0x7ffad9c3bed1] -> /usr/loca
>>>>>> l/lib/dovecot/lib20_virtual_plugin.so(+0x9ead) [0x7ffad9791ead] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xb0f83)
>>>>>> [0x7ffad9c1ff83] ->
>>>>>> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x10602)
>>>>>> [0x7ffad97b460
>>>>>> 2] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0xf813)
>>>>>> [0x7ffad99e9813] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x31)
>>>>>>
>>>>>> [0x7ffad9bbdb91] -> /usr/local/lib/dovecot/lib20_virtual_plug
>>>>>> in.so(virtual_storage_sync_init+0xf0b) [0x7ffad9795b7b] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x51)
>>>>>>
>>>>>> [0x7ffad9bbd6d1]
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]:
>>>>>> imap(jakob)<11292><rg2GZclxHaPAqADc>: Fatal: master: service(imap):
>>>>>> child 11292 killed with signal 6 (core dumped)
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]: auth: Debug: client in:
>>>>>> AUTH        19        PLAIN        service=imap secured=tls
>>>>>> session=9KKNZclxHqPAqADc lip=192.168.0.14        rip=192.168.0.2
>>>>>> 20        lport=143        rport=41758
>>>>>> local_name=aldebaran.xundeenergie.at
>>>>>> ssl_cipher=ECDHE-RSA-AES128-GCM-SHA256 ssl_cipher_bits=128
>>>>>> ssl_pfs=KxECDHE ssl_protocol=TLSv1.2
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]: auth: Debug: client passdb
>>>>>> out: CONT        19
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]: auth: Debug: client in:
>>>>>> CONT<hidden>
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]: auth: Debug:
>>>>>> passwd-file(jakob,192.168.0.220,<9KKNZclxHqPAqADc>): lookup:
>>>>>> user=jakob
>>>>>> file=/usr/local/etc/dovecot/users
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]: auth: Debug: client passdb
>>>>>> out: OK        19        user=jakob
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]: auth: Debug: master in:
>>>>>> REQUEST        2436366337        10570        19
>>>>>> dd4c468bbfe617d4ef06a1b8ce6c111d        session_pid=11298
>>>>>> request_auth_token
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]: auth: Debug:
>>>>>> passwd-file(jakob,192.168.0.220,<9KKNZclxHqPAqADc>): lookup:
>>>>>> user=jakob
>>>>>> file=/usr/local/etc/dovecot/users
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]: auth: Debug: master userdb
>>>>>> out: USER        2436366337        jakob home=/var/mail/jakob
>>>>>> acl_groups=home,admin auth_token=3ea3b630e131
>>>>>> 3bf5beb0f26ff740450189f63e3e
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]: imap-login: Login:
>>>>>> user=<jakob>, method=PLAIN, rip=192.168.0.220, lip=192.168.0.14,
>>>>>> mpid=11298, TLS, session=<9KKNZclxHqPAqADc>
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]:
>>>>>> imap(jakob)<11298><9KKNZclxHqPAqADc>: Panic: Leaked view for index
>>>>>> /var/lib/dovecot/db/indexes/Maildir/jakob/&ANY-BB/dovecot.index:
>>>>>> Opened
>>>>>> in index-storage.c:338
>>>>>> Jul 25 04:11:29 aldebaran dovecot[10573]:
>>>>>> imap(jakob)<11298><9KKNZclxHqPAqADc>: Error: Raw backtrace:
>>>>>> /usr/local/lib/dovecot/libdovecot.so.0(+0xcedf1) [0x7fbd982b2df1] ->
>>>>>> /usr/local/lib/dovecot/libdovecot.so.0(+
>>>>>> 0xceeba) [0x7fbd982b2eba] ->
>>>>>> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0)
>>>>>> [0x7fbd98220fe8] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xf7022)
>>>>>> [0x7fbd98458022] -> /usr/local/lib/dovecot/libdovecot-s
>>>>>> torage.so.0(index_storage_mailbox_close+0x9a) [0x7fbd9842f72a] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a)
>>>>>> [0x7fbd983ad1aa] ->
>>>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0xab80) [0
>>>>>> x7fbd97f84b80] ->
>>>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(virtual_backend_box_open+0x2b)
>>>>>>
>>>>>> [0x7fbd97f84d3b] ->
>>>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0x12a9)
>>>>>>
>>>>>> [0x7fbd97f87f1
>>>>>> 9] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x51)
>>>>>>
>>>>>> [0x7fbd983af6d1] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37)
>>>>>> [0x7fbd983af787] -> /usr/local/lib/dovecot/libdo
>>>>>> vecot-storage.so.0(index_storage_get_status+0x31)
>>>>>> [0x7fbd9842ded1] ->
>>>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x9ead)
>>>>>> [0x7fbd97f83ead]
>>>>>> -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xb0f83) [0x7fbd9
>>>>>> 8411f83] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x10602)
>>>>>> [0x7fbd97fa6602] ->
>>>>>> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0xf813)
>>>>>> [0x7fbd981db813] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox
>>>>>> _get_status+0x31) [0x7fbd983afb91] ->
>>>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0xf0b)
>>>>>>
>>>>>> [0x7fbd97f87b7b] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x51)
>>>>>> [0
>>>>>> x7fbd983af6d1] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37)
>>>>>> [0x7fbd983af787] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x31)
>>>>>>
>>>>>> [0x7fbd9842ded1] -> /usr/loca
>>>>>> l/lib/dovecot/lib20_virtual_plugin.so(+0x9ead) [0x7fbd97f83ead] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xb0f83)
>>>>>> [0x7fbd98411f83] ->
>>>>>> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x10602)
>>>>>> [0x7fbd97fa660
>>>>>> 2] -> /usr/local/lib/dovecot/lib01_acl_plugin.so(+0xf813)
>>>>>> [0x7fbd981db813] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x31)
>>>>>>
>>>>>> [0x7fbd983afb91] -> /usr/local/lib/dovecot/lib20_virtual_plug
>>>>>> in.so(virtual_storage_sync_init+0xf0b) [0x7fbd97f87b7b] ->
>>>>>> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x51)
>>>>>>
>>>>>> [0x7fbd983af6d1]
>>>>>> Jul 25 04:11:30 aldebaran dovecot[10573]:
>>>>>> imap(jakob)<11298><9KKNZclxHqPAqADc>: Fatal: master: service(imap):
>>>>>> child 11298 killed with signal 6 (core dumped)
>>>>>>
>>>>>>
>>>>>> jakob
>>>>>>
>