[Dovecot] 1.0rc25: double INBOX folder (weird)
Let me start off by saying the initial cause was *not* a Dovecot bug. :)
If you have the default namespace "INBOX." (Courier emulation mode from the Wiki) and an actual folder in your ~/Maildir/ with INBOX in the name (~/Maildir/.INBOX.Miami), then Dovecot presents them both to the client as the real Inbox in Thunderbird:
Account Inbox (3) Drafts Sent INBOX (3)
...where (3) is me representing a new email account; if you click on either INBOX folder (parent or child) they show the same contents of the single Inbox. It was really confusing until I figured out the Maildir was messed up on the backend.
Sometime back in 2004 the Courier-IMAP server let one of my (dovecot) testers create a folder called .INBOX.Miami; until the 1.0rc25 upgrade we never even knew it was there - it has been invisible this whole time. I can only assume that a bug in Courier allowed that folder to be created in the first place, I'm not bothering to track it down.
Once I upgraded to 1.0rc25 this morning all the sudden there it was -- obviously not Dovecot's fault for putting it there and it was easy enough to fix by hand, but I thought I'd report it in case it exposed some logic gap in the code. If nothing else Dovecot shouldn't have duplicated the INBOX view to the client, and instead just showed it as a regular folder - yes?
-te
-- Troy Engel | Systems Engineer Fluid, Inc | http://www.fluid.com
Oliver Schulze L. wrote:
Whats your subscriptions files says?
It was clean, the INBOX.Miami wasn't listed or anything obvious like that.
-te
-- Troy Engel | Systems Engineer Fluid Inc. | http://www.fluid.com
On 2.3.2007, at 2.19, Troy Engel wrote:
If you have the default namespace "INBOX." (Courier emulation mode
from the Wiki) and an actual folder in your ~/Maildir/ with INBOX
in the name (~/Maildir/.INBOX.Miami), then Dovecot presents them
both to the client as the real Inbox in Thunderbird: ..Account Inbox (3) Drafts Sent INBOX (3)
...where (3) is me representing a new email account; if you click
on either INBOX folder (parent or child) they show the same
contents of the single Inbox. It was really confusing until I
figured out the Maildir was messed up on the backend.
Hmm. To me this sounds like a bug in Thunderbird. It should have
shown Miami under the INBOX.
Sometime back in 2004 the Courier-IMAP server let one of my
(dovecot) testers create a folder called .INBOX.Miami; until the
1.0rc25 upgrade we never even knew it was there - it has been
invisible this whole time. I can only assume that a bug in Courier
allowed that folder to be created in the first place, I'm not
bothering to track it down.
I don't think it's a bug really, because creating such mailbox is
perfectly valid.
Once I upgraded to 1.0rc25 this morning all the sudden there it was
-- obviously not Dovecot's fault for putting it there and it was
easy enough to fix by hand, but I thought I'd report it in case it
exposed some logic gap in the code. If nothing else Dovecot
shouldn't have duplicated the INBOX view to the client, and instead
just showed it as a regular folder - yes?
Pre-rc25 was buggy in handling that INBOX.INBOX.box case, but as far
as I know now it works as IMAP protocol requires it to work:
x list "" INBOX.IN%
- LIST (\Noselect \HasChildren) "." "INBOX.INBOX" x OK List completed.
x list "" INBOX.IN*
- LIST (\HasNoChildren) "." "INBOX.INBOX.foo" x OK List completed.
Timo Sirainen wrote:
Hmm. To me this sounds like a bug in Thunderbird. It should have shown Miami under the INBOX.
I agree -- TB 1.5 probably just has a (another) bug that this exposed.
I don't think it's a bug really, because creating such mailbox is perfectly valid.
Ahh. I didn't think it was, but I guess why not. :) It's just not very logical, but technically valid.
Pre-rc25 was buggy in handling that INBOX.INBOX.box case, but as far as I know now it works as IMAP protocol requires it to work:
This makes sense then why it magically appeared - rc25 started doing the right thing, but Thunderbird is still broken in the display of it.
Thanks for looking at it, -te
-- Troy Engel | Systems Engineer Fluid, Inc | http://www.fluid.com
I found a weird Thunderbird bug that sounds similar to this, so I'll describe it. It affected the Windows version of Thunderbird, not the Mac (didn't have a chance to test out the Linux version).
I had migrated from Courier to Dovecot. Courier had things in the INBOX namespace, and I got rid of that via the courier2dovecot script. I had people running Thunderbird on Windows adjust their namespace and close Thunderbird, then delete their IMAP cache. Then they restarted.
What Thunderbird then did was instead of using the Trash folder that was not in the INBOX namespace, it instead re-created the Trash folder as a subfolder of INBOX (Drafts and Sent did not have this problem). It also added a subscription to this folder. Both the Trash folders now (the one as a subfolder of Inbox and the one that wasn't) had the "trash" icon in Thunderbird, but Trash went to the one that was the subfolder of Inbox.. Attempting to go into the Thunderbird settings and change which of the two it was using did not help.
To fix this I had to have people close Thunderbird and delete their IMAP cache again. On the server side, I then had to rm -rf the INBOX.Trash folder in their account, and manually edit their subscriptions file to get rid of the extra INBOX.Trash entry. When they fired up Thunderbird again, it would start using the correct Trash folder.
IIRC this was the behavior on Thunderbird 1.5. On 1.0 I believe it did the same thing but didn't re-add the INBOX.Trash folder to subscriptions (again, IIRC...it was a bit ago and I've tried to forget this nightmare). Just another day in the life, and why I use KMail. It may crash more, but it ain't crazy.
--Jeff
Timo Sirainen wrote:
Pre-rc25 was buggy in handling that INBOX.INBOX.box case, but as far as I know now it works as IMAP protocol requires it to work:
OK I think I have new info -- 1.0rc25 "breaks" the Wiki migration and upgrade process that works with rc24 and earlier; I just logged onto my own account from home, and I now have a double INBOX in my Thunderbird.
A courier-imap migration changes this:
courierimap-subscribed:
INBOX INBOX.Trash INBOX.Sent
...into this:
subscriptions:
INBOX Trash Sent
pre-rc25 this was working perfectly; I had one Inbox as intended. If you look at the courier2dovecot.sh script everyone uses, this is a 'proper' conversion:
find $dir -name subscriptions -print0 | xargs -0r sed -i 's/INBOX\.//'
So, what I have now is a phantom INBOX in my Thunderbird with rc25; there is no actual folder .INBOX in my ~/Maildir/, but it is listed in my subscriptions file. So visually in TBird I see the contents in both Inbox and Inbox->INBOX now.
Obviously the fix is easy, delete the phantom INBOX from my subscriptions; however, this seems to point at rc25 code changes not being backwards compatible with all the previous migrations (what if you had migrated 1000 mailboxes? Oh the horror :)) and should probably be addressed somehow.
Maybe rc26+ should "fix" a phantom INBOX for the user? (listed in subscriptions but not actually present on disk)
-te
-- Troy Engel | Systems Engineer Fluid Inc. | http://www.fluid.com
Or, you're hitting some kind of bug in Tbird like what I described in my earlier post.
I'd try deleting Thunderbird's on-disk cache of your IMAP folders. Dovecot may be working just fine, and Tbird may not be handling its out-of-date cache correctly (which it doesn't).
--Jeff
On Saturday 03 March 2007, Troy Engel wrote:
Timo Sirainen wrote:
Pre-rc25 was buggy in handling that INBOX.INBOX.box case, but as far as I know now it works as IMAP protocol requires it to work:
OK I think I have new info -- 1.0rc25 "breaks" the Wiki migration and upgrade process that works with rc24 and earlier; I just logged onto my own account from home, and I now have a double INBOX in my Thunderbird.
A courier-imap migration changes this:
courierimap-subscribed:
INBOX INBOX.Trash INBOX.Sent
...into this:
subscriptions:
INBOX Trash Sent
pre-rc25 this was working perfectly; I had one Inbox as intended. If you look at the courier2dovecot.sh script everyone uses, this is a 'proper' conversion:
find $dir -name subscriptions -print0 | xargs -0r sed -i 's/INBOX\.//'
So, what I have now is a phantom INBOX in my Thunderbird with rc25; there is no actual folder .INBOX in my ~/Maildir/, but it is listed in my subscriptions file. So visually in TBird I see the contents in both Inbox and Inbox->INBOX now.
Obviously the fix is easy, delete the phantom INBOX from my subscriptions; however, this seems to point at rc25 code changes not being backwards compatible with all the previous migrations (what if you had migrated 1000 mailboxes? Oh the horror :)) and should probably be addressed somehow.
Maybe rc26+ should "fix" a phantom INBOX for the user? (listed in subscriptions but not actually present on disk)
-te
Jeff Mitchell wrote:
Or, you're hitting some kind of bug in Tbird like what I described in my earlier post.
I'd try deleting Thunderbird's on-disk cache of your IMAP folders. Dovecot may be working just fine, and Tbird may not be handling its out-of-date cache correctly (which it doesn't).
Hi Jeff,
While it may be a bug in the way TBird presents it to me (I'm aware of the caches etc. and the many TBird bugs - been supporting Mozilla and TBird for years), a problem also lies in the fact that a change happened from rc24 to rc25 which is not backwards compatible.
If you are migrating from Courier-IMAP and used the Wiki (and linked scripts), you most likely have INBOX in your subscriptions file, like all of my testers. I wouldn't be posting about this problem if it hadn't all the sudden cropped up in rc25, rc24 was 100% fine.
-te
-- Troy Engel | Systems Engineer Fluid Inc. | http://www.fluid.com
On Sat, 2007-03-03 at 09:13 -0800, Troy Engel wrote:
subscriptions:
INBOX Trash Sent .. Obviously the fix is easy, delete the phantom INBOX from my subscriptions; however, this seems to point at rc25 code changes not being backwards compatible with all the previous migrations (what if you had migrated 1000 mailboxes? Oh the horror :)) and should probably be addressed somehow.
Maybe rc26+ should "fix" a phantom INBOX for the user? (listed in subscriptions but not actually present on disk)
You mean removing the "INBOX" from subscriptions fixed it? There wasn't "INBOX.INBOX"? Having INBOX in subscriptions file should be perfectly valid, and I don't see bugs related to it.. It'd be more helpful if you could show the IMAP commands/replies that cause something wrong to happen (use eg. rawlog).
Removing non-existing mailboxes from subscriptions anyway is an IMAP protocol violation.
Timo Sirainen wrote:
On Sat, 2007-03-03 at 09:13 -0800, Troy Engel wrote:
subscriptions:
INBOX
You mean removing the "INBOX" from subscriptions fixed it? There wasn't "INBOX.INBOX"? Having INBOX in subscriptions file should be perfectly valid, and I don't see bugs related to it.. It'd be more helpful if you
You are correct, there was an INBOX listed in the subscriptions file, but no physical folder of any sort on disk; this showed up visually as INBOX.INBOX to ThunderBird. By removing the entry from subscriptions (and the associated client cache file) it of course was fixed; really no different than any other folder repairs.
Removing non-existing mailboxes from subscriptions anyway is an IMAP protocol violation.
Aha, OK wasn't aware. Something still changed between rc24 and rc25 in that showing this (misguided, inappropriate) INBOX now pops up. I'll just have to chock it up to Courier-IMAP having done the wrong thing, and Dovecot rc25 now doing the right thing. As you mentioned rc24 and below had issues, so maybe it should have been showing me this all along! :)
thx, -te
-- Troy Engel | Systems Engineer Fluid, Inc | http://www.fluid.com
On Tue, 2007-03-06 at 09:13 -0800, Troy Engel wrote:
You mean removing the "INBOX" from subscriptions fixed it? There wasn't "INBOX.INBOX"? Having INBOX in subscriptions file should be perfectly valid, and I don't see bugs related to it.. It'd be more helpful if you
You are correct, there was an INBOX listed in the subscriptions file, but no physical folder of any sort on disk; this showed up visually as INBOX.INBOX to ThunderBird. By removing the entry from subscriptions (and the associated client cache file) it of course was fixed; really no different than any other folder repairs.
No, looks like the rc25 fix broke LSUB command a bit. This should fix it properly: http://dovecot.org/list/dovecot-cvs/2007-March/007945.html
Timo Sirainen wrote:
No, looks like the rc25 fix broke LSUB command a bit. This should fix it properly: http://dovecot.org/list/dovecot-cvs/2007-March/007945.html
Looks good - I patched my dovecot (new RPM build), restarted the daemon completely after upgrade. Exited ThunderBird, by-hand edited my ~/Maildir/subscriptions and added 'INBOX' at the top, then logged back in - no new phantom INBOX subfolder shows up.
Let me know if you'd like it tested some other way...
Thanks! -te
-- Troy Engel | Systems Engineer Fluid, Inc | http://www.fluid.com
participants (5)
-
Jeff Mitchell
-
Jeff Mitchell
-
Oliver Schulze L.
-
Timo Sirainen
-
Troy Engel