<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Apr 9, 2019 at 9:52 PM Aki Tuomi <<a href="mailto:aki.tuomi@open-xchange.com" target="_blank">aki.tuomi@open-xchange.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><u></u>
<div>
<div>
<br>
</div>
<blockquote type="cite">
<div>
On 10 April 2019 05:00 Mark Moseley via dovecot <<a href="mailto:dovecot@dovecot.org" target="_blank">dovecot@dovecot.org</a>> wrote:
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
On Wed, Apr 3, 2019 at 9:37 PM Mark Moseley <
<a href="mailto:moseleymark@gmail.com" target="_blank">moseleymark@gmail.com</a>> wrote:
<br>
</div>
<div class="gmail_quote">
<blockquote>
<div dir="ltr">
<div dir="ltr">
<br>
</div>
<div class="gmail_quote">
<div class="gmail_attr" dir="ltr">
On Wed, Mar 20, 2019 at 2:13 PM Mark Moseley <
<a href="mailto:moseleymark@gmail.com" rel="noopener" target="_blank">moseleymark@gmail.com</a>> wrote:
<br>
</div>
<blockquote>
<div dir="ltr">
<div dir="ltr">
<div style="margin:0px;padding:0px 0px 20px;width:1270px;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif" class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-gs">
<div class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-">
<div style="font-size:12.800000190734863px;margin:8px 0px 0px;padding:0px" class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-ii m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-gt" id="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-:gj1">
<div class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-a3s m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-aXjCH" id="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-:gi4">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-adM">
<div class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-im">
<div class="gmail_attr">
Just hoping to get some dev eyes on this. I'm incredibly reluctant to throw the word 'bug' around
</div>
<div class="gmail_attr">
(since 99 times out of 100, it's not -- it's almost always the config), but I can't think of any way
</div>
<div class="gmail_attr">
that this could be a config issue, esp when the pre-2.2.34 version works as expected.
</div>
<div class="gmail_attr">
<br>
</div>
<div class="gmail_attr">
I noticed during troubleshooting that dovecot errors out if I try to create a subfolder called
</div>
<div class="gmail_attr">
'INBOX' but it'll happily create a subfolder called INBOX.SomethingElse (i.e. a folder called
</div>
<div class="gmail_attr">
INBOX.INBOX.SomethingElse - resulting in a directory called .INBOX.SomethingElse on the
</div>
<div class="gmail_attr">
filesystem, and leading to the problem described below). Is that sub-subfolder creation (where
</div>
<div class="gmail_attr">
the top level subfolder matches the namespace name) supposed to be allowed? It seems
</div>
<div class="gmail_attr">
odd that 'INBOX' (as a subfolder of INBOX) would be blocked but INBOX.SomethingElse (as
</div>
<div class="gmail_attr">
a subfolder of INBOX) would be allowed. I'd expect INBOX.SomethingElse (i.e.
</div>
<div class="gmail_attr">
INBOX.INBOX.SomethingElse) would be blocked as well.
</div>
<div class="gmail_attr">
<br>
</div>
<div class="gmail_attr" dir="ltr">
<br>
</div>
<div class="gmail_attr" dir="ltr">
On Wed, Mar 13, 2019 at 4:46 AM Bernd Wurst via dovecot <
<a href="mailto:dovecot@dovecot.org" rel="noopener" target="_blank">dovecot@dovecot.org</a>> wrote:
<br>
</div>
<blockquote>
Hello,
<br>
<br>we're operating dovecot on a small server. Some years ago, we migrated
<br>from courier IMAP to dovecot. Therefore, we defined our default
<br>Namespace "inbox" with prefix "INBOX." to have this compatible. I found
<br>this in some migration docs those days. Generally, everything worked as
<br>expected.
<br>
<br>Our only namespace is configured like this:
<br>
<br>namespace inbox {
<br> separator = .
<br> prefix = INBOX.
<br> inbox = yes
<br>}
<br>
<br>Regularly, there is no folder named INBOX or .INBOX in the file system,
<br>I suppose this is correct. But I found a special corner case today when
<br>it comes to quota calculation.
<br>
<br>When - for whatever reason - a folder .INBOX.foo (for arbitrary values
<br>of foo) exists, the whole mailbox is counted twice in quota
<br>recalculation. Just creating .INBOX does nothing but a subfolder
<br>triggers the problem.
<br>
<br>This is my shell view (replaced username and file path and deleted
<br>unnecessary debug output)
<br>
<br>$ cat maildirsize
<br>268435456S
<br>14697 17
<br>$ maildirmake .INBOX.foo
<br>$ sudo doveadm -D quota recalc -u <user>
<br>[...]
<br>doveadm(<user>): Debug: Namespace inbox: type=private, prefix=INBOX.,
<br>sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes
<br>location=maildir:/home/.../test
<br>doveadm(<user>): Debug: maildir++: root=/home/.../test, index=,
<br>indexpvt=, control=, inbox=/home/.../test, alt=
<br>doveadm(<user>): Debug: Namespace : type=private, prefix=, sep=,
<br>inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none
<br>doveadm(<user>): Debug: none: root=, index=, indexpvt=, control=,
<br>inbox=, alt=
<br>doveadm(<user>): Debug: quota: quota_over_flag check: quota_over_script
<br>unset - skipping
<br>doveadm(<user>): Debug: Quota root User quota: Recalculated relative
<br>rules with bytes=268435456 count=0. Now grace=26843545
<br>doveadm(<user>): Debug: Namespace INBOX.: Using permissions from
<br>/home/.../test: mode=0700 gid=default
<br>
<br>$ cat maildirsize
<br>268435456S
<br>29394 34
<br>
<br>
<br>So the used quota has exactly been doubled by just creating an empty
<br>subfolder.
<br>
<br>Do you have any pointers for fixing my configuration or is this a bug in
<br>dovecot?
<br>
<br>
</blockquote>
<div>
<br>
</div>
</div>
</div>
<div>
I coincidentally resurrected a months-old thread with this same issue a few days ago. I'm seeing the exact same after upgrading from 2.2.32 to 2.2.36.
</div>
<div>
<br>
</div>
<div>
The original poster (who also narrowed it down to something in 2.2.34) mentioned a workaround that does indeed work, namely setting mailbox_list_index=no:
</div>
<div>
<br>
</div>
<div>
> doveadm -o 'mailbox_list_index=no' quota recalc -u myuser
<br>
</div>
<div>
<br>
</div>
<div>
I've been staring at diffs of 2.2.33 and 2.2.34 without anything jumping out at me (not a C guy, sadly). Maybe src/lib-storage/index/index-storage.c or src/lib-storage/list/mailbox-list-fs-iter.c or src/lib-storage/list/mailbox-list-index-iter.c or src/lib-storage/list/mailbox-list-index.c?
</div>
<div>
<br>
</div>
<div>
The latter few have some added strcmp's against "INBOX". Then again, there's a lot of new code in the diffs under src/lib-storage that references INBOX specifically.
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div>
<br>
</div>
<div>
Can the Dovecot team confirm whether this is indeed a bug or not? I've not yet been able to test 2.3.x to see if the problem exists there as well.
<br>
</div>
</div>
</div>
</blockquote>
<div>
<br>
</div>
<div>
I've bisected this down to this commit:
</div>
<div>
<br>
</div>
<div>
git diff 7620195ceeea805137cbd1bae104e385eee474a9..97473a513feb2bbd763051869c8b7b83e24b37fa
</div>
<div>
<br>
</div>
<div>
diff --git a/src/lib-storage/list/mailbox-list-index-iter.c b/src/lib-storage/list/mailbox-list-index-iter.c
</div>
<div>
index c9afc7a..49cd941 100644
</div>
<div>
--- a/src/lib-storage/list/mailbox-list-index-iter.c
</div>
<div>
+++ b/src/lib-storage/list/mailbox-list-index-iter.c
</div>
<div>
@@ -90,13 +90,18 @@ mailbox_list_index_update_info(struct mailbox_list_index_iterate_context *ctx)
</div>
<div>
if (strcmp(ctx->info.vname, "INBOX") != 0) {
</div>
<div>
/* non-INBOX */
</div>
<div>
ctx->info.vname = p_strdup(ctx->info_pool, ctx->info.vname);
</div>
<div>
- } else {
</div>
<div>
+ } else if (!ctx->prefix_inbox_list) {
</div>
<div>
/* listing INBOX itself */
</div>
<div>
ctx->info.vname = "INBOX";
</div>
<div>
if (mail_namespace_is_inbox_noinferiors(ctx->info.ns)) {
</div>
<div>
ctx->info.flags &= ~(MAILBOX_CHILDREN|MAILBOX_NOCHILDREN);
</div>
<div>
ctx->info.flags |= MAILBOX_NOINFERIORS;
</div>
<div>
}
</div>
<div>
+ } else {
</div>
<div>
+ /* listing INBOX/INBOX */
</div>
<div>
+ ctx->info.vname = p_strconcat(ctx->info_pool,
</div>
<div>
+ ctx->ctx.list->ns->prefix, "INBOX", NULL);
</div>
<div>
+ ctx->info.flags |= MAILBOX_NONEXISTENT;
</div>
<div>
}
</div>
<div>
if ((node->flags & MAILBOX_LIST_INDEX_FLAG_NONEXISTENT) != 0)
</div>
<div>
ctx->info.flags |= MAILBOX_NONEXISTENT;
</div>
<div>
@@ -126,6 +131,14 @@ mailbox_list_index_update_next(struct mailbox_list_index_iterate_context *ctx,
</div>
<div>
{
</div>
<div>
struct mailbox_list_index_node *node = ctx->next_node;
</div>
<div>
<br>
</div>
<div>
+ if (!ctx->prefix_inbox_list && ctx->ctx.list->ns->prefix_len > 0 &&
</div>
<div>
+ strcmp(node->name, "INBOX") == 0 && node->parent == NULL &&
</div>
<div>
+ node->children != NULL) {
</div>
<div>
+ /* prefix/INBOX has children */
</div>
<div>
+ ctx->prefix_inbox_list = TRUE;
</div>
<div>
+ return;
</div>
<div>
+ }
</div>
<div>
+
</div>
<div>
if (node->children != NULL && follow_children) {
</div>
<div>
ctx->parent_len = str_len(ctx->path);
</div>
<div>
ctx->next_node = node->children;
</div>
<div>
diff --git a/src/lib-storage/list/mailbox-list-index.h b/src/lib-storage/list/mailbox-list-index.h
</div>
<div>
index 9dc1c48..e4b042a 100644
</div>
<div>
--- a/src/lib-storage/list/mailbox-list-index.h
</div>
<div>
+++ b/src/lib-storage/list/mailbox-list-index.h
</div>
<div>
@@ -138,6 +138,7 @@ struct mailbox_list_index_iterate_context {
</div>
<div>
struct mailbox_list_index_node *next_node;
</div>
<div>
<br>
</div>
<div>
unsigned int failed:1;
</div>
<div>
+ unsigned int prefix_inbox_list:1;
</div>
<div>
};
</div>
<div>
<br>
</div>
<div>
extern MODULE_CONTEXT_DEFINE(mailbox_list_index_module,
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
Prior to this commit, anything updating the quota would do the right thing for any .INBOX.<something> folders (i.e. not double count the contents of "INBOX" against the quota). After this commit, anything updating quota (new mail, quota recalc, etc) does the double counting of INBOX.
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div>
<br>
</div>
<div>
Thank you for the bisect! We'll look into this.
</div>
<div class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867io-ox-signature">
<pre>---
Aki Tuomi</pre></div></div></blockquote><div><br></div><div><br></div><div>Hi. I was curious if there were any fixes for this? We're still affected by this (and I imagine others are too but don't realize it). Thanks!</div></div></div>