[Dovecot] case (non-)sensitivity for folder names
Is IMAP supposed to be case sensitive or case in-sensitive? It seemed it would be case sensitive because I've had different cases of folders. But today I found I had two folders "Spam" and "spam", with directories ".INBOX.Spam" and ".INBOX.spam" on the server. Messages existed in each directory on the server and they were different. The messages could be read from Evolution. However, the list of messages had info mixed up between the folders. I then selected all messages in one of the folders and trashed them (click on trash icon). Messages disappeared in BOTH folders. On the server, after a couple minutes delay, messages in BOTH directories were flagged "T". That doesn't seem right. But I don't know enough about what IMAP is supposed to do, or on which end the names of folders get remapped into directory names, to even guess which is messed up here. I do know Evolution keeps a track of what it thinks the folder has, because in the past I have just "rm -fr" on a folder directory and Evolution still showed each message in its list, although it could not access them.
Anyone know what's the deal with that?
Unfortunately, there seems to be no operation in Evolution to tell it to discard its own cache.
I'm also wondering if life would be any better if I used Thunderbird.
-- sHiFt HaPpEnS!
On Wed, 2011-01-05 at 10:09 -0500, Phil Howard wrote:
Is IMAP supposed to be case sensitive or case in-sensitive?
Case sensitive, except for INBOX. (Or if the server is using case-insensitive filesystem then they're case-insensitive.)
It seemed it would be case sensitive because I've had different cases of folders. But today I found I had two folders "Spam" and "spam", with directories ".INBOX.Spam" and ".INBOX.spam" on the server. Messages existed in each directory on the server and they were different. The messages could be read from Evolution. However, the list of messages had info mixed up between the folders. I then selected all messages in one of the folders and trashed them (click on trash icon). Messages disappeared in BOTH folders. On the server, after a couple minutes delay, messages in BOTH directories were flagged "T". That doesn't seem right.
My guess is that Evolution messes this up.
Unfortunately, there seems to be no operation in Evolution to tell it to discard its own cache.
rm -rf ~/.evolution/mail/imap/
On Wed, Jan 5, 2011 at 12:05, Timo Sirainen tss@iki.fi wrote:
On Wed, 2011-01-05 at 10:09 -0500, Phil Howard wrote:
Is IMAP supposed to be case sensitive or case in-sensitive?
Case sensitive, except for INBOX. (Or if the server is using case-insensitive filesystem then they're case-insensitive.)
So what will Dovecot do when both directories exist: .INBOX.Spam and .INBOX.spam
It seemed it would be case sensitive because I've had different cases of folders. But today I found I had two folders "Spam" and "spam", with directories ".INBOX.Spam" and ".INBOX.spam" on the server. Messages existed in each directory on the server and they were different. The messages could be read from Evolution. However, the list of messages had info mixed up between the folders. I then selected all messages in one of the folders and trashed them (click on trash icon). Messages disappeared in BOTH folders. On the server, after a couple minutes delay, messages in BOTH directories were flagged "T". That doesn't seem right.
My guess is that Evolution messes this up.
And it might be confused by odd data it gets due to both above directories existing.
Unfortunately, there seems to be no operation in Evolution to tell it to discard its own cache.
rm -rf ~/.evolution/mail/imap/
Thanks. I'll give that a try soon.
-- sHiFt HaPpEnS!
On 5.1.2011, at 19.33, Phil Howard wrote:
On Wed, Jan 5, 2011 at 12:05, Timo Sirainen tss@iki.fi wrote:
On Wed, 2011-01-05 at 10:09 -0500, Phil Howard wrote:
Is IMAP supposed to be case sensitive or case in-sensitive?
Case sensitive, except for INBOX. (Or if the server is using case-insensitive filesystem then they're case-insensitive.)
So what will Dovecot do when both directories exist: .INBOX.Spam and .INBOX.spam
Same as if they were called .INBOX.Spam and .INBOX.HelloWorld. They are two completely different mailboxes.
On Wed, Jan 5, 2011 at 13:20, Timo Sirainen tss@iki.fi wrote:
On 5.1.2011, at 19.33, Phil Howard wrote:
On Wed, Jan 5, 2011 at 12:05, Timo Sirainen tss@iki.fi wrote:
On Wed, 2011-01-05 at 10:09 -0500, Phil Howard wrote:
Is IMAP supposed to be case sensitive or case in-sensitive?
Case sensitive, except for INBOX. (Or if the server is using case-insensitive filesystem then they're case-insensitive.)
So what will Dovecot do when both directories exist: .INBOX.Spam and .INBOX.spam
Same as if they were called .INBOX.Spam and .INBOX.HelloWorld. They are two completely different mailboxes.
But if for INBOX names are case IN-sensitive, then there will be confusion because anything obeying that concept would see "Inbox >> spam" and "Inbox >> Spam" as being the same box.
There are different ways to deal with case IN-sensitive name matching. A lame way is to translate all names to a specific case and do a lookup at the OS layer (which on Unix is case sensitive, so which name is found, if any, depends on how the case-ify is done). A somewhat smarter way is to scan the list of names and case-ify each, and compare that the case-ified search name (where they are case-ified the same way, example always upper, or maybe always lower). When a match is found, that leaves two alternatives: either stop there and use that name (which means which is used depends on the order names were obtained), or continue for more. If continuing for more and more are found, are they merged is the next question. The logic can get more convoluted with this.
The behavior I have seen, through Evolution (I just installed Thunderbird and will be trying that in a while), is that when I had 3 existing, "foo", "Foo", and "FOO", all under inbox (seen as ".INBOX" on the server, and as "Inbox" in the client), there were various situations of getting things mixed up among them as if accessing one would really access another, though it was unclear which. But I definitely ran into the case where when I had ONE message in each of "foo", "Foo" and "FOO", and deleted the message in "foo" only, it deleted one message in all three, but left the new message count on "FOO" as "(1)" even though it showed now messages.
The subfolders are definitely NOT kept completely distinct. It's as if some software was merging them, and some was not. Since all 3 did show up, clearly Dovecot did not merge them to LIST them to the client.
My boss was using Apple Mail, then tried Thunderbird, and similar effects are happening. I'm wanting to believe that IMAP is just so poorly defined that everyone is doing things just a bit differently, and enough to sort of work, but mess things up.
-- sHiFt HaPpEnS!
On 5.1.2011, at 23.04, Phil Howard wrote:
But if for INBOX names are case IN-sensitive, then there will be confusion because anything obeying that concept would see "Inbox >> spam" and "Inbox >> Spam" as being the same box.
It's case insensitive only for the INBOX itself, not its children. So INBOX.spam and inbox.spam are the same, but INBOX.Spam isn't the same as either of them.
The behavior I have seen, through Evolution (I just installed Thunderbird and will be trying that in a while), is that when I had 3 existing, "foo", "Foo", and "FOO", all under inbox (seen as ".INBOX" on the server, and as "Inbox" in the client), there were various situations of getting things mixed up among them as if accessing one would really access another, though it was unclear which. But I definitely ran into the case where when I had ONE message in each of "foo", "Foo" and "FOO", and deleted the message in "foo" only, it deleted one message in all three, but left the new message count on "FOO" as "(1)" even though it showed now messages.
I'm pretty sure this confusion comes from Evolution internally.
My boss was using Apple Mail, then tried Thunderbird, and similar effects are happening. I'm wanting to believe that IMAP is just so poorly defined that everyone is doing things just a bit differently, and enough to sort of work, but mess things up.
It's possible that multiple clients do case insensitive comparing. I doubt many people have actually tested what happens then. I only know Dovecot's code, and there's about zero possibility of it mixing up mailboxes with different casing.
On Wed, Jan 5, 2011 at 16:10, Timo Sirainen tss@iki.fi wrote:
On 5.1.2011, at 23.04, Phil Howard wrote:
But if for INBOX names are case IN-sensitive, then there will be confusion because anything obeying that concept would see "Inbox >> spam" and "Inbox >> Spam" as being the same box.
It's case insensitive only for the INBOX itself, not its children. So INBOX.spam and inbox.spam are the same, but INBOX.Spam isn't the same as either of them.
Ah, I misinterpreted the original reference to be recursive.
The behavior I have seen, through Evolution (I just installed Thunderbird and will be trying that in a while), is that when I had 3 existing, "foo", "Foo", and "FOO", all under inbox (seen as ".INBOX" on the server, and as "Inbox" in the client), there were various situations of getting things mixed up among them as if accessing one would really access another, though it was unclear which. But I definitely ran into the case where when I had ONE message in each of "foo", "Foo" and "FOO", and deleted the message in "foo" only, it deleted one message in all three, but left the new message count on "FOO" as "(1)" even though it showed now messages.
I'm pretty sure this confusion comes from Evolution internally.
Given the above clarification, that appears to be the case.
My boss was using Apple Mail, then tried Thunderbird, and similar effects are happening. I'm wanting to believe that IMAP is just so poorly defined that everyone is doing things just a bit differently, and enough to sort of work, but mess things up.
It's possible that multiple clients do case insensitive comparing. I doubt many people have actually tested what happens then. I only know Dovecot's code, and there's about zero possibility of it mixing up mailboxes with different casing.
Some of this might actually be more a case of how clients handle cases where nothing is subscribed. Apparently Thunderbird has no such mode, and always subscribes to what it creates. Evolution lets you run unsubscribed, but behaves like it had some kind of phantom subscription state. Some things do clear up when .evolution state is wiped clean.
I'm moving over to Thunderbird in the next day or so.
-- sHiFt HaPpEnS!
participants (2)
-
Phil Howard
-
Timo Sirainen