[Dovecot] LIST-STATUS issue
Looked at the hg commits since 2.1.1, but didn't see anything that was
exactly on-point to this...
UNSEEN counts seem to be broken when done in a LIST-STATUS call. I
see this, for example:
Timestamp: Thu, 01 Mar 2012 10:55:47 -0700 [...] C: 3 LIST "" (INBOX IN.dovecot) RETURN (STATUS (UNSEEN)) S: * LIST () "." "IN.dovecot" S: * STATUS "IN.dovecot" (UNSEEN 0) S: * LIST () "." "INBOX" S: * STATUS "INBOX" (UNSEEN 3) S: 3 OK List completed.
3 seconds later...
Timestamp: Thu, 01 Mar 2012 10:55:50 -0700 S: * OK [CAPABILITY IMAP4rev1 LITERAL+ LOGIN-REFERRALS ID ENABLE
AUTH=LOGIN XIMAPPROXY] Dovecot ready. C: [LOGIN Command - username: slusarz] S: * OK [XPROXYREUSE] IMAP connection reused by squirrelmail-imap_proxy S: 1 OK User logged in C: 2 EXAMINE IN.dovecot (QRESYNC (1254166849 25800
18484,20260,20393,20705,20860:20868)) S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft impflag1 impflag2
impflag0 Junk seen NonJunk) S: * OK [PERMANENTFLAGS ()] Read-only mailbox. S: * 13 EXISTS S: * 0 RECENT S: * OK [UNSEEN 5] First unseen. S: * OK [UIDVALIDITY 1254166849] UIDs valid S: * OK [UIDNEXT 20869] Predicted next UID S: * OK [HIGHESTMODSEQ 25800] Highest S: 2 OK [READ-ONLY] Select completed. [...] C: 7 SEARCH RETURN (COUNT) UNSEEN S: * ESEARCH (TAG "7") COUNT 8 S: 7 OK Search completed (0.000 secs).
I can confirm that I did not receive 8 dovecot list messages in 3 seconds. :)
Once the mailbox is accessed, I am seeing correct counts in
LIST-STATUS return again:
Timestamp: Thu, 01 Mar 2012 10:57:28 -0700 [...] C: 3 LIST "" (INBOX IN.dovecot) RETURN (STATUS (UNSEEN)) S: * LIST () "." "IN.dovecot" S: * STATUS "IN.dovecot" (UNSEEN 8) S: * LIST () "." "INBOX" S: * STATUS "INBOX" (UNSEEN 3) S: 3 OK List completed.
I am using imapproxy for these connections, so all access are
happening in the same Dovecot access.
michael
On 1.3.2012, at 20.03, Michael M Slusarz wrote:
Looked at the hg commits since 2.1.1, but didn't see anything that was exactly on-point to this...
UNSEEN counts seem to be broken when done in a LIST-STATUS call. I see this, for example:
Only with LIST-STATUS, or also STATUS itself? And with what kind of config (doveconf -n)? Can you reproduce this?
LIST-STATUS should use the exact same code as STATUS. STATUS should use pretty much the same code as SELECT. mailbox_list_index=yes makes a difference, but other than that I can't really think of a reason.
Quoting Timo Sirainen <tss@iki.fi>:
On 1.3.2012, at 20.03, Michael M Slusarz wrote:
Looked at the hg commits since 2.1.1, but didn't see anything that
was exactly on-point to this...UNSEEN counts seem to be broken when done in a LIST-STATUS call. I
see this, for example:Only with LIST-STATUS, or also STATUS itself? And with what kind of
config (doveconf -n)? Can you reproduce this?
I've been seeing this intermittently the last few days. When I notice
it, I try to reproduce and never can. Then it eventually comes back,
as in the first time I access the next morning. Maybe its some kind
of cache/timeout issue. (e.g. - I can't reproduce right now)
LIST-STATUS should use the exact same code as STATUS. STATUS should
use pretty much the same code as SELECT. mailbox_list_index=yes
makes a difference, but other than that I can't really think of a
reason.
# 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 3.2.1-1-ARCH x86_64 auth_mechanisms = plain login namespace { inbox = yes location = maildir:~/Maildir prefix = separator = . type = private } namespace { location = maildir:~/mailtest/mboxtesting2 prefix = "#shared." separator = . type = shared } namespace { location = mbox:~/mailtest/mail prefix = "#public2." separator = . type = public } passdb { args = dovecot driver = pam } plugin { acl = vfile } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } ssl_cert = </etc/ssl/newcerts/100001.pem ssl_key = </etc/ssl/private/key.pem userdb { driver = passwd } protocol imap { mail_plugins = acl imap_acl }
michael
Quoting Michael M Slusarz <slusarz@curecanti.org>:
Quoting Timo Sirainen <tss@iki.fi>:
On 1.3.2012, at 20.03, Michael M Slusarz wrote:
Looked at the hg commits since 2.1.1, but didn't see anything that
was exactly on-point to this...UNSEEN counts seem to be broken when done in a LIST-STATUS call.
I see this, for example:Only with LIST-STATUS, or also STATUS itself? And with what kind of
config (doveconf -n)? Can you reproduce this?I've been seeing this intermittently the last few days. When I
notice it, I try to reproduce and never can. Then it eventually
comes back, as in the first time I access the next morning. Maybe
its some kind of cache/timeout issue. (e.g. - I can't reproduce
right now)
It happened again. Once it got around midnight local time I stopped
seeing correct STATUS updates with the UNSEEN count. Running two
MUAs: I opened a mailbox with new messages in one, and the next time
the other MUA (webmail) polled it magically started seeing unseen
messages again. And now if I change the seen flag in one MUA, it will
continue to be updated (as normal) in the other MUA.
michael
On Fri, 2012-03-02 at 00:53 -0700, Michael M Slusarz wrote:
It happened again. Once it got around midnight local time I stopped
seeing correct STATUS updates with the UNSEEN count. Running two
MUAs: I opened a mailbox with new messages in one, and the next time
the other MUA (webmail) polled it magically started seeing unseen
messages again. And now if I change the seen flag in one MUA, it will
continue to be updated (as normal) in the other MUA.
Next time try talking IMAP protocol directly? First giving the same LIST STATUS command, and if it doesn't show the updated counts, try STATUS directly and finally EXAMINE.
I don't see anything in the code that could explain why this could be happening. What filesystem are you using?
Quoting Timo Sirainen <tss@iki.fi>:
On Fri, 2012-03-02 at 00:53 -0700, Michael M Slusarz wrote:
It happened again. Once it got around midnight local time I stopped seeing correct STATUS updates with the UNSEEN count. Running two MUAs: I opened a mailbox with new messages in one, and the next time the other MUA (webmail) polled it magically started seeing unseen messages again. And now if I change the seen flag in one MUA, it will continue to be updated (as normal) in the other MUA.
Next time try talking IMAP protocol directly? First giving the same LIST STATUS command, and if it doesn't show the updated counts, try STATUS directly and finally EXAMINE.
Weird. In the MUA I was seeing this:
C: 4 LIST "" (INBOX IN.dovecot) RETURN (STATUS (UNSEEN)) S: * LIST () "." "IN.dovecot" S: * STATUS "IN.dovecot" (UNSEEN 0) S: * LIST () "." "INBOX" S: * STATUS "INBOX" (UNSEEN 1) S: 4 OK List completed.
Then I did this:
slusarz@bigworm % dovecot
- PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID
ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS
MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED
I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN
CONTEXT=SEARCH LIST-STATUS SEARCH=FUZZY SPECIAL-USE ACL RIGHTS=texk]
Logged in as slusarz 1 LIST "" (INBOX IN.dovecot) RETURN (STATUS (UNSEEN)) - LIST () "." "IN.dovecot"
- STATUS "IN.dovecot" (UNSEEN 7)
- LIST () "." "INBOX"
- STATUS "INBOX" (UNSEEN 1) 1 OK List completed.
Sure enough, I went back to the MUA and now see this:
C: 4 LIST "" (INBOX IN.dovecot) RETURN (STATUS (UNSEEN)) S: * LIST () "." "IN.dovecot" S: * STATUS "IN.dovecot" (UNSEEN 7) S: * LIST () "." "INBOX" S: * STATUS "INBOX" (UNSEEN 1) S: 4 OK List completed.
The only difference... in the MUA I am enabling QRESYNC. This is what
the previous commands look like (before reaching the LIST-STATUS
command):
Timestamp: Fri, 02 Mar 2012 11:17:56 -0700 S: * OK [CAPABILITY IMAP4rev1 LITERAL+ LOGIN-REFERRALS ID ENABLE
AUTH=LOGIN XIMAPPROXY] Dovecot ready. C: [LOGIN Command - username: slusarz] S: 1 OK User logged in C: 2 ENABLE QRESYNC S: * ENABLED QRESYNC S: 2 OK Enabled. C: 3 STATUS IN.horde.cvs (MESSAGES UIDNEXT UIDVALIDITY HIGHESTMODSEQ) S: * STATUS "IN.horde.cvs" (MESSAGES 11 UIDNEXT 16767 UIDVALIDITY
1239077891 HIGHESTMODSEQ 31409) S: 3 OK Status completed.
Next time I see this issue, I will try to go back and issue ENABLE
QRESYNC from the command line.
I don't see anything in the code that could explain why this could be happening. What filesystem are you using?
Nothing fancy: jfs on Archlinux. I've been using it for years - it's
not something I have recently changed.
michael
On 2.3.2012, at 20.27, Michael M Slusarz wrote:
I don't see anything in the code that could explain why this could be happening. What filesystem are you using?
Nothing fancy: jfs on Archlinux. I've been using it for years - it's not something I have recently changed.
jfs used to have a problem with not updating directory's mtime when link()ing files to it, which caused Dovecot not to notice new mails. This got fixed a few years ago though. But it is a bit suspicious that this is happening with jfs..
Quoting Timo Sirainen <tss@iki.fi>:
On 2.3.2012, at 20.27, Michael M Slusarz wrote:
I don't see anything in the code that could explain why this could be happening. What filesystem are you using?
Nothing fancy: jfs on Archlinux. I've been using it for years -
it's not something I have recently changed.jfs used to have a problem with not updating directory's mtime when
link()ing files to it, which caused Dovecot not to notice new mails.
This got fixed a few years ago though. But it is a bit suspicious
that this is happening with jfs..
I can now verify that QRESYNC is triggering this behavior.
- PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID
ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS
MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED
I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN
CONTEXT=SEARCH LIST-STATUS SEARCH=FUZZY SPECIAL-USE ACL RIGHTS=texk]
Logged in as slusarz 1 ENABLE QRESYNC - ENABLED QRESYNC 1 OK Enabled. 2 LIST "" (IN.horde.dev) RETURN (STATUS (UNSEEN))
- LIST () "." "IN.horde.dev"
- STATUS "IN.horde.dev" (UNSEEN 0) 2 OK List completed. 3 EXAMINE IN.horde.dev
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded NonJunk
impflag0 impflag1) - OK [PERMANENTFLAGS ()] Read-only mailbox.
- 3 EXISTS
- 1 RECENT
- OK [UNSEEN 3] First unseen.
- OK [UIDVALIDITY 1255685339] UIDs valid
- OK [UIDNEXT 2805] Predicted next UID
- OK [HIGHESTMODSEQ 8266] Highest 3 OK [READ-ONLY] Select completed.
(The unseen message arrived about 7 minutes before I issued these
commands, so it didn't sneak in between command #2 and #3).
michael
On 3.3.2012, at 3.48, Michael M Slusarz wrote:
I can now verify that QRESYNC is triggering this behavior.
Fixed:
http://hg.dovecot.org/dovecot-2.1/rev/8cbc130c2b72 http://hg.dovecot.org/dovecot-2.1/rev/31ae11fe18b2
participants (2)
-
Michael M Slusarz
-
Timo Sirainen