[Dovecot] Re: New mail sometimes fails to appear

[Tom Alsberg]

Some more new information...

1. First, I was wrong in the last noted observation (probably my eyes
   got confused between the lines in the log).  When the first call to
   mbox_sync_has_changed returns 1 and the next ones return 0,
   Thunderbird is not notified of the new mail that has arrived (no
   EXISTS is sent), even the first time.  So I can't tell in advance what
   will be with the next message that arrives.

2. Also, I put some more debugging into mbox_sync_has_changed, and can
   say the following:  When everything is right, mbox_sync_has_changed
   returns 1 (the second and third times too) because
   mbox->mbox_dirty_stamp is different from st->st_mtime, in the last
   statement of mbox_sync_has_changed:

	return st->st_mtime != mbox->mbox_dirty_stamp ||
		st->st_size != mbox->mbox_dirty_size;

   While when it does not work, mbox->mbox_dirty_stamp equals
   st->st_mtime, and it returns 0.  In both cases, hdr->sync_stamp is
   different from st->st_mtime.

   (The same applies to the sizes as well, that is, they are different
   where the timestamps are different and equal where the timestamps are
   equal).

3. A more interesting revalation:  It is not only that EXISTS is not
   sent, and not only to do with IDLE.  When new mail arrives, even when
   Thunderbird issues a SELECT (by doing a check for new mail), it does
   not see the new messages.

4. Furthermore:  If the last message seen from within Thunderbird was
   not yet read, and the problem occurs (new mail arrives that is not
   seen), this last message will then seem corrupted from Thunderbird
   - it will contain at the end some garbage filling, and then part of
   the next message in the mailbox.

   This is while the mailbox itself is completely valid, has no such
   garbage in it, and the messages are validly separated by an empty
   line and a valid "From " line.  Perhaps this means there's some
   race-condition bug in the mbox reading/index syncing code?  That's
   the code I least understand as of now...

   Attached is an example of one such last message saved from
   Thunderbird after new mail arrived that was not seen.  The filler
   characters are bytes of 0x80, and then the next message from
   "n-Path: " is present.

Any hints?

  Cheers,
  -- Tom

On Mon, Feb 06, 2006 at 05:03:49PM +0200, I wrote:
><snip />
> However, it happens that on the first call mbox_sync_has_changed
> returns 1, but on the second call it returns 0.  Now, the first time
> this happens, EXISTS is still sent from imap_sync_deinit to the
> client (after the second call to mbox_sync_has_changed).
> 
> However after that, when new mail arrives, mbox_sync_has_changed
> returns 1 the first time and 0 the next times again, but in
> imap_sync_deinit, status.messages and ctx->messages_count are equal,
> and thus no EXISTS is sent to the client.
><snip />

-- 
  Tom Alsberg - hacker (being the best description fitting this space)
  Web page:	http://www.cs.huji.ac.il/~alsbergt/
DISCLAIMER:  The above message does not even necessarily represent what
my fingers have typed on the keyboard, save anything further.