Inbox quota usage doubled when mailbox_list_index enabled, under some circumstances

[Mark Moseley]

On Tue, Aug 14, 2018 at 12:31 PM Chris Dillon <chris.dillon at cpanel.net>
wrote:

> I’ve had the opportunity to test the same configuration with a fresh build
> of the git master branch (2.4.devel) and the issue also occurs there.  I
> see that "mailbox_list_index = yes" is now enabled by default.  It can
> still be disabled via "mailbox_list_index = no" which allows the quota to
> be calculated correctly.
>
> ==========
> root at ubuntu1804:~# dovecot -n
> # 2.4.devel (44282aeeb): /usr/local/etc/dovecot/dovecot.conf
> # OS: Linux 4.15.0-30-generic x86_64 Ubuntu 18.04.1 LTS
> # Hostname: ubuntu1804
> mail_location = maildir:~/Maildir
> mail_plugins = quota
> namespace inbox {
>   inbox = yes
>   location =
>   prefix = INBOX.
>   separator = .
> }
> passdb {
>   driver = pam
> }
> plugin {
>   quota = maildir:Mailbox
> }
> userdb {
>   driver = passwd
> }
> ==========
>
> (To summarize from my previous message -- other than "mailbox_list_index =
> yes", second most important part of replication is that there is at least
> one email in the real inbox and at least one sub-folder named "INBOX" in
> maildir format)
>
> root at ubuntu1804:~# ls -ld
> /home/myuser/Maildir/cur/1532529376.M543965P58007.centos7.local\,S\=12712627\,W\=12877782\:2\,S
> /home/myuser/Maildir/.INBOX.Test/
> -rw-rw-r-- 1 myuser myuser 12712627 Aug 14 18:28
> '/home/myuser/Maildir/cur/1532529376.M543965P58007.centos7.local,S=12712627,W=12877782:2,S'
> drwxrwxr-x 5 myuser myuser       87 Aug 14 18:56
> /home/myuser/Maildir/.INBOX.Test/
> =========
>
> (In the following example usage is doubled, there is only one email)
>
> root at ubuntu1804:~# doveadm quota recalc -u myuser; doveadm quota get -u
> myuser
> Quota name Type    Value Limit
>                   %
> Mailbox    STORAGE 24830     -
>                   0
> Mailbox    MESSAGE     2     -
>                   0
> ==========
>
> (In the following example it works correctly with mailbox_list_index
> disabled)
>
> root at ubuntu1804:~# doveadm -o 'mailbox_list_index=no' quota recalc -u
> myuser; doveadm quota get -u myuser
> Quota name Type    Value Limit
>                   %
> Mailbox    STORAGE 12415     -
>                   0
> Mailbox    MESSAGE     1     -
>                   0
> ==========
>
> Best Regards




We recently upgraded from 2.2.32 to 2.2.36.1 and ran into the same issue as
above. I was about to start compiling all the intermediate versions to
pinpoint where it started and happened upon this post first. We are in the
same boat where some users have *sub*folders starting with INBOX, resulting
in directory names like ".INBOX.Event" (and our namespace root is INBOX)
and quota calculation double-counts INBOX's vsize.

Just to be clear too: At least in my case, it's *not* double counting,
e.g., INBOX.Event. But if the above conditions are met, it's
double-counting *INBOX*, because it now sees a folder called INBOX.INBOX
(which does *not* exist on the filesystem).

I hadn't gotten as far as Chris did (this just bubbled up today), but his
solution works here too, i.e. passing -o 'mailbox_list_index=no'  to
doveadm quota recalc.

Also, if I rename the directories to something else (e.g. from above,
rename ".INBOX.Event" to ".notINBOX.Event"), a quota recalc works just
fine. The presence of a directory called .INBOX.<something> is triggering
this.

I'm able to create new subfolders called INBOX.<something> with clients
like Apple Mail, which was a bit surprising (Apple Mail however choked when
I tried to just create a subfolder called 'INBOX'). We've got millions of
mailboxes, so educating users is a non-starter :)

Any fix for this from the dovecot devs?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dovecot.org/pipermail/dovecot/attachments/20190312/4cadcf11/attachment-0001.html>