[Dovecot] Differences between IDLE when Maildir vs. wel the "mail" folder is used
Are there any differences with how both IDLE and "UID FETCH n:n+x (FLAGS UID...)" are handled when Dovecot is using a "mail" vs. a "Maildir" user?
I have recently implemented support for IDLE (Push E-mail) in a client infrastructure that I'm building. I haven't in depth checked out what the problem might be, but when Dovecot was using the "mail" dir in $HOME something about the implementation wasn't working. When I switched to "Maildir" everything worked.
I only tested to the point of seeing the "* EXISTS n" response during the idle. I saw this response. It's for example possible that the n is wrong or that a bit later the "UID FETCH ..." ain't returning the right information. It's also possible that my implementation is wrong (of course) and I haven't yet had the time to really figure out what goes wrong.
I did so far, however, test with Cyrus, Exchange, Courier and Isode's M-Box. All these tests where successful. SO if it indeed is a discrepancy in my implementation .. it's going to be something very specialised or specific I fear :-)
I also setup testing infrastructure http://tinymail.org/trac/tinymail/wiki/TestImapServers
Also take a careful look at this manual test. "tnytest" is a user on imap2.tinymail.org that uses Maildir. The "tnytest2" user doesn't have a Maildir configured. By that I mean that "Dovecot" has created the folder for E-mail using whatever it uses by default.
Dovecot's IDLE (the EXISTS) responses also take a lot longer on tnytest2 than on tnytest.
Another question. Why not simply "always" use the Maildir solution if that one works better? ;-) (really, honest question. Hehe).
Take a look at the "UID FETCH" response: It's incorrect on tnytest2. The first attempt I asked for 1:1 and I got none. The second I asked for 1:4 and I got 3 of them, while IDLE clearly stated that there where 4 messages available. In reality 4 where indeed available (or copied in total).
Well, I'm not saying something is or isn't wrong, just wondering and testing a little bit and seeing some strange results...
Feel free to use imap2.tinymail.org for testing. And if you need root on that virtual machine, so that you can install a new version, go ahead. On http://imap2.tinymail.org you van find things like version information (it's a CVS snapshot).
pvanhoof@lort:~$ telnet imap2.tinymail.org Trying 86.39.154.77... telnet: Unable to connect to remote host: Connection refused pvanhoof@lort:~$ telnet imap2.tinymail.org 143 Trying 86.39.154.77... Connected to imap2.tinymail.org. Escape character is '^]'.
- OK Dovecot ready. ao1 login tnytest tnytest ao1 OK Logged in. a01 select INBOX
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
- 1 EXISTS
- 0 RECENT
- OK [UIDVALIDITY 1171049606] UIDs valid
- OK [UIDNEXT 2] Predicted next UID a01 OK [READ-WRITE] Select completed. a04 IDLE
- idling
- 2 EXISTS DONE a04 OK Idle completed. UID FETCH 1:2 (FLAGS UID)
- 1 FETCH (FLAGS (\Seen) UID 1)
- 2 FETCH (FLAGS (\Seen) UID 2) UID OK Fetch completed. ^]
telnet> quit Connection closed. pvanhoof@lort:~$ telnet imap2.tinymail.org 143 Trying 86.39.154.77... Connected to imap2.tinymail.org. Escape character is '^]'.
- OK Dovecot ready. a01 login tnytest2 tnytest a01 OK Logged in. a01 select INBOX
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
- 0 EXISTS
- 0 RECENT
- OK [UIDVALIDITY 1171107413] UIDs valid
- OK [UIDNEXT 1] Predicted next UID a01 OK [READ-WRITE] Select completed. a04 IDLE
- idling
- 1 EXISTS DONE a04 OK Idle completed. a05 UID FETCH 1:1 (FLAGS UID) a05 OK Fetch completed. a04 IDLE
- idling
- 4 EXISTS DONE a04 OK Idle completed. a06 UID FETCH 1:4 (FLAGS UID)
- 1 FETCH (FLAGS (\Seen) UID 2)
- 2 FETCH (FLAGS (\Seen) UID 3)
- 3 FETCH (FLAGS (\Seen) UID 4) a06 OK Fetch completed.
-- Philip Van Hoof, software developer home: me at pvanhoof dot be gnome: pvanhoof at gnome dot org http://www.pvanhoof.be/blog
On Sat, 2007-02-10 at 12:58 +0100, Philip Van Hoof wrote:
Dovecot's IDLE (the EXISTS) responses also take a lot longer on tnytest2 than on tnytest.
That is because with maildir Dovecot can use kernel's dnotify support to know immediately when a new mail arrives. Inotify apparently supports also waiting for file changes, so I think it should work if you configure --with-notify=inotify. I'll have to try this myself too..
- OK [UIDNEXT 1] Predicted next UID a01 OK [READ-WRITE] Select completed. a04 IDLE
- idling
- 1 EXISTS DONE
How did you get those messages into the mailbox? Just wondering why it's not giving RECENT reply.
a04 OK Idle completed. a05 UID FETCH 1:1 (FLAGS UID) a05 OK Fetch completed.
Looks like I can reproduce this. At least with empty mailboxes it seems that Dovecot gives a new UID to a mail that already had it. I'll go and fix this.
However, Dovecot's behavior is perfectly valid from IMAP protocol's point of view. You shouldn't be assuming that the new UID is the same as UIDNEXT. For example another session could have just appended two messages and expunged the first one. In that case you'll see exactly the above behavior from all the IMAP servers.
So you really should be using message sequence numbers instead of UIDs before you're completely certain what the UID is. And even still using sequence numbers is a more "correct" IMAP way to do it, plus it's slightly faster :) I think the UIDs are mostly useful only for some initial local cache syncing when the mailbox is opened.
a06 UID FETCH 1:4 (FLAGS UID)
- 1 FETCH (FLAGS (\Seen) UID 2)
- 2 FETCH (FLAGS (\Seen) UID 3)
- 3 FETCH (FLAGS (\Seen) UID 4) a06 OK Fetch completed.
This is because the fourth message had UID 5 and you exluded it from the fetch.
participants (2)
-
Philip Van Hoof
-
Timo Sirainen