<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Apr 9, 2019 at 9:52 PM Aki Tuomi <<a href="mailto:aki.tuomi@open-xchange.com" target="_blank">aki.tuomi@open-xchange.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><u></u>

  
   
 
 <div>
  <div>
   <br>
  </div>
  <blockquote type="cite">
   <div>
    On 10 April 2019 05:00 Mark Moseley via dovecot <<a href="mailto:dovecot@dovecot.org" target="_blank">dovecot@dovecot.org</a>> wrote:
   </div>
   <div>
    <br>
   </div>
   <div>
    <br>
   </div>
   <div dir="ltr">
    <div dir="ltr">
     <div dir="ltr">
      <div dir="ltr">
       On Wed, Apr 3, 2019 at 9:37 PM Mark Moseley <
       <a href="mailto:moseleymark@gmail.com" target="_blank">moseleymark@gmail.com</a>> wrote:
       <br>
      </div>
      <div class="gmail_quote">
       <blockquote>
        <div dir="ltr">
         <div dir="ltr">
          <br>
         </div>
         <div class="gmail_quote">
          <div class="gmail_attr" dir="ltr">
           On Wed, Mar 20, 2019 at 2:13 PM Mark Moseley <
           <a href="mailto:moseleymark@gmail.com" rel="noopener" target="_blank">moseleymark@gmail.com</a>> wrote:
           <br>
          </div>
          <blockquote>
           <div dir="ltr">
            <div dir="ltr">
             <div style="margin:0px;padding:0px 0px 20px;width:1270px;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif" class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-gs">
              <div class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-">
               <div style="font-size:12.800000190734863px;margin:8px 0px 0px;padding:0px" class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-ii m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-gt" id="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-:gj1">
                <div class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-a3s m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-aXjCH" id="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-:gi4">
                 <div dir="ltr">
                  <div dir="ltr">
                   <div dir="ltr">
                    <div dir="ltr">
                     <div dir="ltr">
                      <div dir="ltr">
                       <div dir="ltr">
                        <div dir="ltr">
                         <div class="gmail_quote">
                          <div class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-adM">
                           <div class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867gmail-m_4417296991923282657gmail-m_5214552142079555927gmail-im">
                            <div class="gmail_attr">
                             Just hoping to get some dev eyes on this. I'm incredibly reluctant to throw the word 'bug' around
                            </div>
                            <div class="gmail_attr">
                             (since 99 times out of 100, it's not -- it's almost always the config), but I can't think of any way 
                            </div>
                            <div class="gmail_attr">
                             that this could be a config issue, esp when the pre-2.2.34 version works as expected.
                            </div>
                            <div class="gmail_attr">
                             <br>
                            </div>
                            <div class="gmail_attr">
                             I noticed during troubleshooting that dovecot errors out if I try to create a subfolder called
                            </div>
                            <div class="gmail_attr">
                             'INBOX' but it'll happily create a subfolder called INBOX.SomethingElse (i.e. a folder called 
                            </div>
                            <div class="gmail_attr">
                             INBOX.INBOX.SomethingElse - resulting in a directory called .INBOX.SomethingElse on the 
                            </div>
                            <div class="gmail_attr">
                             filesystem, and leading to the problem described below). Is that sub-subfolder creation (where
                            </div>
                            <div class="gmail_attr">
                             the top level subfolder matches the namespace name) supposed to be allowed? It seems
                            </div>
                            <div class="gmail_attr">
                             odd that 'INBOX' (as a subfolder of INBOX) would be blocked but INBOX.SomethingElse (as
                            </div>
                            <div class="gmail_attr">
                             a subfolder of INBOX) would be allowed. I'd expect INBOX.SomethingElse (i.e. 
                            </div>
                            <div class="gmail_attr">
                             INBOX.INBOX.SomethingElse) would be blocked as well.
                            </div>
                            <div class="gmail_attr">
                             <br>
                            </div>
                            <div class="gmail_attr" dir="ltr">
                             <br>
                            </div>
                            <div class="gmail_attr" dir="ltr">
                             On Wed, Mar 13, 2019 at 4:46 AM Bernd Wurst via dovecot <
                             <a href="mailto:dovecot@dovecot.org" rel="noopener" target="_blank">dovecot@dovecot.org</a>> wrote:
                             <br>
                            </div>
                            <blockquote>
                             Hello,
                             <br>
                             <br>we're operating dovecot on a small server. Some years ago, we migrated
                             <br>from courier IMAP to dovecot. Therefore, we defined our default
                             <br>Namespace "inbox" with prefix "INBOX." to have this compatible. I found
                             <br>this in some migration docs those days. Generally, everything worked as
                             <br>expected.
                             <br>
                             <br>Our only namespace is configured like this:
                             <br>
                             <br>namespace inbox {
                             <br> separator = .
                             <br>  prefix = INBOX.
                             <br>  inbox = yes
                             <br>}
                             <br>
                             <br>Regularly, there is no folder named INBOX or .INBOX in the file system,
                             <br>I suppose this is correct.  But I found a special corner case today when
                             <br>it comes to quota calculation.
                             <br>
                             <br>When - for whatever reason - a folder .INBOX.foo (for arbitrary values
                             <br>of foo) exists, the whole mailbox is counted twice in quota
                             <br>recalculation. Just creating .INBOX does nothing but a subfolder
                             <br>triggers the problem.
                             <br>
                             <br>This is my shell view (replaced username and file path and deleted
                             <br>unnecessary debug output)
                             <br>
                             <br>$ cat maildirsize
                             <br>268435456S
                             <br>14697 17
                             <br>$ maildirmake .INBOX.foo
                             <br>$ sudo doveadm -D quota recalc -u <user>
                             <br>[...]
                             <br>doveadm(<user>): Debug: Namespace inbox: type=private, prefix=INBOX.,
                             <br>sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes
                             <br>location=maildir:/home/.../test
                             <br>doveadm(<user>): Debug: maildir++: root=/home/.../test, index=,
                             <br>indexpvt=, control=, inbox=/home/.../test, alt=
                             <br>doveadm(<user>): Debug: Namespace : type=private, prefix=, sep=,
                             <br>inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none
                             <br>doveadm(<user>): Debug: none: root=, index=, indexpvt=, control=,
                             <br>inbox=, alt=
                             <br>doveadm(<user>): Debug: quota: quota_over_flag check: quota_over_script
                             <br>unset - skipping
                             <br>doveadm(<user>): Debug: Quota root User quota: Recalculated relative
                             <br>rules with bytes=268435456 count=0. Now grace=26843545
                             <br>doveadm(<user>): Debug: Namespace INBOX.: Using permissions from
                             <br>/home/.../test: mode=0700 gid=default
                             <br>
                             <br>$ cat maildirsize
                             <br>268435456S
                             <br>29394 34
                             <br>
                             <br>
                             <br>So the used quota has exactly been doubled by just creating an empty
                             <br>subfolder.
                             <br>
                             <br>Do you have any pointers for fixing my configuration or is this a bug in
                             <br>dovecot?
                             <br>
                             <br>
                            </blockquote>
                            <div>
                             <br>
                            </div>
                           </div>
                          </div>
                          <div>
                           I coincidentally resurrected a months-old thread with this same issue a few days ago. I'm seeing the exact same after upgrading from 2.2.32 to 2.2.36.
                          </div>
                          <div>
                           <br>
                          </div>
                          <div>
                           The original poster (who also narrowed it down to something in 2.2.34) mentioned a workaround that does indeed work, namely setting mailbox_list_index=no:
                          </div>
                          <div>
                           <br>
                          </div>
                          <div>
                           > doveadm -o 'mailbox_list_index=no' quota recalc -u myuser
                           <br>
                          </div>
                          <div>
                           <br>
                          </div>
                          <div>
                           I've been staring at diffs of 2.2.33 and 2.2.34 without anything jumping out at me (not a C guy, sadly). Maybe src/lib-storage/index/index-storage.c or src/lib-storage/list/mailbox-list-fs-iter.c or src/lib-storage/list/mailbox-list-index-iter.c or src/lib-storage/list/mailbox-list-index.c?
                          </div>
                          <div>
                           <br>
                          </div>
                          <div>
                           The latter few have some added strcmp's against "INBOX". Then again, there's a lot of new code in the diffs under src/lib-storage that references INBOX specifically.
                          </div>
                         </div>
                        </div>
                       </div>
                      </div>
                     </div>
                    </div>
                   </div>
                  </div>
                 </div>
                </div>
               </div>
              </div>
             </div>
            </div>
           </div>
          </blockquote>
          <div>
           <br>
          </div>
          <div>
           Can the Dovecot team confirm whether this is indeed a bug or not?  I've not yet been able to test 2.3.x to see if the problem exists there as well.
           <br>
          </div>
         </div>
        </div>
       </blockquote>
       <div>
        <br>
       </div>
       <div>
        I've bisected this down to this commit: 
       </div>
       <div>
        <br>
       </div>
       <div>
        git diff 7620195ceeea805137cbd1bae104e385eee474a9..97473a513feb2bbd763051869c8b7b83e24b37fa
       </div>
       <div>
        <br>
       </div>
       <div>
        diff --git a/src/lib-storage/list/mailbox-list-index-iter.c b/src/lib-storage/list/mailbox-list-index-iter.c
       </div>
       <div>
        index c9afc7a..49cd941 100644
       </div>
       <div>
        --- a/src/lib-storage/list/mailbox-list-index-iter.c
       </div>
       <div>
        +++ b/src/lib-storage/list/mailbox-list-index-iter.c
       </div>
       <div>
        @@ -90,13 +90,18 @@ mailbox_list_index_update_info(struct mailbox_list_index_iterate_context *ctx)
       </div>
       <div>
                if (strcmp(ctx->info.vname, "INBOX") != 0) {
       </div>
       <div>
                        /* non-INBOX */
       </div>
       <div>
                        ctx->info.vname = p_strdup(ctx->info_pool, ctx->info.vname);
       </div>
       <div>
        -       } else {
       </div>
       <div>
        +       } else if (!ctx->prefix_inbox_list) {
       </div>
       <div>
                        /* listing INBOX itself */
       </div>
       <div>
                        ctx->info.vname = "INBOX";
       </div>
       <div>
                        if (mail_namespace_is_inbox_noinferiors(ctx->info.ns)) {
       </div>
       <div>
                                ctx->info.flags &= ~(MAILBOX_CHILDREN|MAILBOX_NOCHILDREN);
       </div>
       <div>
                                ctx->info.flags |= MAILBOX_NOINFERIORS;
       </div>
       <div>
                        }
       </div>
       <div>
        +       } else {
       </div>
       <div>
        +               /* listing INBOX/INBOX */
       </div>
       <div>
        +               ctx->info.vname = p_strconcat(ctx->info_pool,
       </div>
       <div>
        +                       ctx->ctx.list->ns->prefix, "INBOX", NULL);
       </div>
       <div>
        +               ctx->info.flags |= MAILBOX_NONEXISTENT;
       </div>
       <div>
                }
       </div>
       <div>
                if ((node->flags & MAILBOX_LIST_INDEX_FLAG_NONEXISTENT) != 0)
       </div>
       <div>
                        ctx->info.flags |= MAILBOX_NONEXISTENT;
       </div>
       <div>
        @@ -126,6 +131,14 @@ mailbox_list_index_update_next(struct mailbox_list_index_iterate_context *ctx,
       </div>
       <div>
         {
       </div>
       <div>
                struct mailbox_list_index_node *node = ctx->next_node;
       </div>
       <div>
        <br>
       </div>
       <div>
        +       if (!ctx->prefix_inbox_list && ctx->ctx.list->ns->prefix_len > 0 &&
       </div>
       <div>
        +           strcmp(node->name, "INBOX") == 0 && node->parent == NULL &&
       </div>
       <div>
        +           node->children != NULL) {
       </div>
       <div>
        +               /* prefix/INBOX has children */
       </div>
       <div>
        +               ctx->prefix_inbox_list = TRUE;
       </div>
       <div>
        +               return;
       </div>
       <div>
        +       }
       </div>
       <div>
        +
       </div>
       <div>
                if (node->children != NULL && follow_children) {
       </div>
       <div>
                        ctx->parent_len = str_len(ctx->path);
       </div>
       <div>
                        ctx->next_node = node->children;
       </div>
       <div>
        diff --git a/src/lib-storage/list/mailbox-list-index.h b/src/lib-storage/list/mailbox-list-index.h
       </div>
       <div>
        index 9dc1c48..e4b042a 100644
       </div>
       <div>
        --- a/src/lib-storage/list/mailbox-list-index.h
       </div>
       <div>
        +++ b/src/lib-storage/list/mailbox-list-index.h
       </div>
       <div>
        @@ -138,6 +138,7 @@ struct mailbox_list_index_iterate_context {
       </div>
       <div>
                struct mailbox_list_index_node *next_node;
       </div>
       <div>
        <br>
       </div>
       <div>
                unsigned int failed:1;
       </div>
       <div>
        +       unsigned int prefix_inbox_list:1;
       </div>
       <div>
         };
       </div>
       <div>
        <br>
       </div>
       <div>
         extern MODULE_CONTEXT_DEFINE(mailbox_list_index_module, 
       </div>
       <div>
        <br>
       </div>
       <div>
        <br>
       </div>
       <div>
        Prior to this commit, anything updating the quota would do the right thing for any .INBOX.<something> folders (i.e. not double count the contents of "INBOX" against the quota). After this commit, anything updating quota (new mail, quota recalc, etc) does the double counting of INBOX.
       </div>
      </div>
     </div>
    </div>
   </div>
  </blockquote>
  <div>
   <br>
  </div>
  <div>
   Thank you for the bisect! We'll look into this.
  </div>
  <div class="m_3191683122879777388m_6576098063468962263gmail-m_-9004329805236409867io-ox-signature">
   <pre>---
Aki Tuomi</pre></div></div></blockquote><div><br></div><div><br></div><div>Hi. I was curious if there were any fixes for this? We're still affected by this (and I imagine others are too but don't realize it). Thanks!</div></div></div>