Looks like a bug to me: Dovecot ignores Maildir/new timestamp
Hello!
I run Debian jessie and have an issue with the interaction between Dovecot (2.2.13), Mutt (1.5.23) and the Android Gmail (5.8.105868218) IMAP client, when using Maildir:
I use Dovecot LMTP for local delivery into Maildir and access these files both through Dovecot IMAP from the Android Gmail IMAP client and also directly through the filesystem by running Mutt locally on the mail server.
When a new mail arrives, Dovecot LMTP will add a new file in Maildir/new and also update the Dovecot index cache. If I then open Mutt and just delete this new message, then Mutt will just delete this file from Maildir/new and the Dovecot index cache will be out-of-sync.
(For testing, Mutt can be cut out of the equation entirely by instead just running "rm Maildir/new/*" from the command line to mimic what Mutt does when deleting a new message, but I mention the use of Mutt here to show why this is a real use-case for me.)
If I after this open the Gmail app and pull down the message index to refresh from the Dovecot IMAP server, it'll get the cached version of the index, in which the new message is still present (but only the headers, no body will be loaded and the app will show no body snippet). Pulling down the message index again to force a second refresh will cause the new/deleted message to disappear again.
This seems to be caused by Dovecot not looking at the Maildir/new timestamp to determine whether the index cache is up-to-date, but instead only looking at the Maildir/cur timestamp.
If I instead repeat all these testing steps from the beginning, sending a new mail message, deleting it with "rm Maildir/new/*" but then also running "touch Maildir/cur" before opening the Gmail app and doing the refresh, then Dovecot does update the cache and does not send any ghost messages to the IMAP client.
It does seem reasonable to me that Mutt doesn't touch Maildir/cur when just deleting new messages and I suspect that it might be a bug in Dovecot that it timestamp of Maildir/new isn't checked and the index cache updated if that timestamp is newer than the last index update done by the Dovecot LMTP service.
Does this analysis seem sound? Did I miss anything?
(Output of "doveconf -n" attached.)
Cheers // Fredrik Roubert
-- Forsterstrasse 64 | +41 78 8170377 CH-8044 Zürich | https://roubert.name/fredrik/
On 2015-12-01 11:30:10 +0100, Fredrik Roubert wrote:
I run Debian jessie and have an issue with the interaction between Dovecot (2.2.13), Mutt (1.5.23) and the Android Gmail (5.8.105868218) IMAP client, when using Maildir:
As a fellow mutt user. I use mutt with dovecot. mutt headercache was much slower for me than dovecot's caching. if you dont want to go via the tcp port, you can set /usr/lib/dovecot/imap as a tunnel in your muttrc.
hth
darix-- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org
On 01 Dec 2015, at 12:30, Fredrik Roubert fredrik@roubert.name wrote:
Hello!
I run Debian jessie and have an issue with the interaction between Dovecot (2.2.13), Mutt (1.5.23) and the Android Gmail (5.8.105868218) IMAP client, when using Maildir:
I use Dovecot LMTP for local delivery into Maildir and access these files both through Dovecot IMAP from the Android Gmail IMAP client and also directly through the filesystem by running Mutt locally on the mail server.
When a new mail arrives, Dovecot LMTP will add a new file in Maildir/new and also update the Dovecot index cache. If I then open Mutt and just delete this new message, then Mutt will just delete this file from Maildir/new and the Dovecot index cache will be out-of-sync.
(For testing, Mutt can be cut out of the equation entirely by instead just running "rm Maildir/new/*" from the command line to mimic what Mutt does when deleting a new message, but I mention the use of Mutt here to show why this is a real use-case for me.)
If I after this open the Gmail app and pull down the message index to refresh from the Dovecot IMAP server, it'll get the cached version of the index, in which the new message is still present (but only the headers, no body will be loaded and the app will show no body snippet). Pulling down the message index again to force a second refresh will cause the new/deleted message to disappear again.
This seems to be caused by Dovecot not looking at the Maildir/new timestamp to determine whether the index cache is up-to-date, but instead only looking at the Maildir/cur timestamp.
Dovecot does check Maildir/new timestamp, but it'll only perform a "partial sync" then, which means it adds any newly found mails but it doesn't delete any mails because it can't be sure that they weren't moved to cur/. One simple fix for this would be to scan cur/ also whenever new/ changes, but that would unnecessarily make the performance worse for most people. Another fix would be to remember that the mail was in new/ and when cur's timestamp hadn't changed and the mail wasn't in new/ anymore it would be removed from index. But this would require remembering that the mail is in new/, which isn't currently done and adding it would probably cause extra disk writes.
So I don't really see a way of fixing this in any nice way.
participants (3)
-
Fredrik Roubert
-
Marcus Rueckert
-
Timo Sirainen