[Dovecot] Deleting messages from MailDir

[Benjamin R. Haskell]

On Wed, 13 Feb 2008, Bill Cole wrote:

> At 12:21 AM -0500 2/13/08, Benjamin R. Haskell  imposed structure on a stream of electrons, yielding:
>> On Wed, 13 Feb 2008, Rody wrote:
>>> Op woensdag 13 februari 2008 00:43, schreef Bill Cole:
>>>> Yes, but you may also care that ctime is reset when a client has
>>>> Dovecot move a message from one subfolder to another within a
>>>> Maildir. I'm not sure why Dovecot does it, but a look at the messages
>>>> in the non-INBOX parts of my Maildir reveals that the ctime is always
>>>> later than the mtime, and the contents (Received headers) makes it
>>>> clear that Dovecot sets the mtime of messages to the original mtime
>>>> (i.e. original delivery time) when copying them.
>>>>
>>
>> I think the answer to "why Dovecot does it" is actually that Dovecot 
>> doesn't do anything with ctime. Under most *nix filesystems, ctime is 
>> the last time the inode underlying the file/dir was changed ('c' for 
>> "changed", not "created" -- [usually]). The inode gets changed when the 
>> file's moved from one directory to another.
>
> Not on all filesystems. Note what HFS+ (MacOS) does:
>
> [example of ctime not being updated]

Hence my qualifying it as "Under most *nix filesystems". But, good to know 
there's a commonly-used exception.


>>>> Hopefully Timo will speak up on this, but I have a vague recollection
>>>> of him saying that Dovecot never modifies message contents as a
>>>> matter of principle, and it seems to me that the design of Maildir
>>>> assumes that the mailstore server follows that principle rigorously.
>>>> That should make mtime quite static for an individual file, and it
>>>> looks to me like Dovecot even makes an effort to preserve the
>>>> delivery time of a message by replicating the mtime from the original
>>>> file to the new one when copying a message between subfolders.
>>
>> The delivery time can also be determined by the first part of the 
>> Maildir filename of the message:
>>
>> e.g. ls -l 1202878863.24522_1.myhost:2,
>> --rw------- 1 user group 551 2008-02-13 00:00 1202878863.24522_1.myhost:2,
>>
>> The 1202878863 part is the Unix timestamp corresponding to midnight, 
>> Feb 13, 2008 EST. So, it's not really a requirement of Maildir's design 
>> to keep the mtime updated. It's just easier+quicker than parsing the 
>> number out of the filename (where it *is* a requirement).
>
> Maildir DOES NOT require a timestamp in the filename, it's just common.
>

DJB's Maildir spec isn't RFC-esque (so it's not a MUST, in that sense). As 
far as I've seen, everything does what's described below. But, like you 
say, not a requirement. I think I skipped over the second and third 
sentences from this excerpt from http://cr.yp.to/proto/maildir.html 
(because of the first sentence) :

[""
Unless you're writing messages to a maildir, the format of a unique name 
is none of your business. A unique name can be anything that doesn't 
contain a colon (or slash) and doesn't start with a dot. Do not try to 
extract information from unique names.

Okay, so you're writing messages. A unique name has three pieces, 
separated by dots. On the left is the result of time() or the second 
counter from gettimeofday().
""]

Also note that that snippet expressly warns against doing what I said 
could be done (parsing the timestamp out of the filename).

I'm curious: do you have examples of mail software that doesn't use the 
timestamp? (I could see some run-once script not doing it, but I'd be 
surprised if widely-used software didn't.)


>
> On my system, the timestamps in the filenames match the ctime, NOT the
> mtime:
>
> ~/Maildir/.Tagged/cur $ ls -lcrt |tail -1
> -rw-------   1 wkc  staff    13293 Feb 12 23:45 1202877918.P26840Q0M183345.toaster.scconsult.com:2,S
> ~/Maildir/.Tagged/cur $ ls -l 1202877918.P26840Q0M183345.toaster.scconsult.com\:2\,S
> -rw-------   1 wkc  staff  13293 Feb 12 23:43 1202877918.P26840Q0M183345.toaster.scconsult.com:2,S
> ~/Maildir/.Tagged/cur wkc$ uepoch 1202877918
> 1202877918 is Tue Feb 12 23:45:18 2008
>
> The earlier mtime matches the delivery time.
>

I think it'd generally be the case that the filename timestamp will match 
the *initial* ctime (before any possible ctime changes), since that should 
be synonymous with creation time. And, mtime could vary, because Dovecot 
syncs the mtime to the delivery time.

But, I still don't quite follow your example. Call the mailfile X (so I 
don't have to type it):

ls -lc X shows 23:45 - so, ctime = 23:45
ls -l X shows 23:43 - so, mtime = 23:43
And, the filename timestamp == ctime.

But, you say the mtime matches the delivery time. So, the filename 
timestamp is two minutes *after* the delivery time? Did it take your LDA 
two minutes (or at least 1 min 19sec) to get the message from the MTA to 
the Maildir?

Also, I think (but, maybe not on HFS+) there'd be a ctime change on 
renaming the file from X to X,S (adding the 'Seen' flag). Do you know 
whether that's the case?

(e.g. try:)
touch foo
ls -lc foo
mv foo foo,S
ls -lc foo


>>> [...] It looks like the use of ctime or mtime depends on wether you 
>>> want the message removed x days after it was moved to say the trash 
>>> folder (ctime) or will be removed x days after it originally arrived 
>>> in the inbox (mtime). My personal opinion is currently that i would 
>>> like it removed x days after it was placed in a certain folder, hence 
>>> i use ctime.
>>
>> Depends on the context for me. [... blah blah. Me blabbering...]
>
> Again, that is FS-specific, and HFS+ is again a counter-example.
>
> (one can argue that HFS+ is Bad and Wrong in many ways, but it is in widespread use...)
>

I guess the take-home message is that ctime isn't quite as consistent 
cross-filesystem as mtime. But, I still think ctime has its place in 
non-"Bad and Wrong" filesystems.

And, as Charles pointed out, in Dovecot 1.1 there's a plugin to do this. 
Does the expunge plugin work without the user logging in, though? (not 
that that's an enormous problem -- more curiosity)

Best,
Ben