[Dovecot] Expire plugin / expire-tool redesign for doveadm

[Timo Sirainen]

On 14.4.2010, at 11.04, Steffen Kaiser wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Tue, 13 Apr 2010, Timo Sirainen wrote:
> 
>> The expire plugin keeps track of the oldest message in configured
>> mailboxes. Its only configuration is the list of tracked mailboxes.
>> There's no configuration like what the expire timeouts are or anything.
> 
> My first idea, that hit me after reading through the whole post, was to keep track of all mailboxes of all users and issue the commands later.
> The plugin saves just one value per physical mailbox, when a) the first message arrives an empty mailbox or messages are expunged? Or each time a message is received. What would be the impact of monitoring all mailboxes?

Looks like the current code does a dict lookup every time messages are saved. I guess this is mainly to fix the situation when expire plugin is enabled after messages already exist in mailbox. But this check could be moved to doveadm I think.. So the dict would only be updated when:

1) message arrives on empty mailbox
2) first message is expunged

But I don't think it's useful to do this for all mailboxes. Even if it doesn't impact normal performance, it wastes disk space on the database for no reason. But expire plugin already supports wildcards, so just telling it to monitor "*" mailboxes would get what you want.

>> There will be a new doveadm command:
>> 
>> doveadm expunge [-u user | -A] <mailbox> <search query>
> 
> What about shared mailboxes and namespaces? Can "<mailbox>" be a phyiscal path rather than a IMAP mailbox name?

No. By shared mailboxes I guess you mean public (i.e. not another user's mailboxes)? You can expunge them using some user that has rights to do that, and using e.g. public/blah as the mailbox name.

>> - When using multiple mailboxes it would be more optimal to handle all
>> mailboxes for a user at once, rather than using separate doveadm
>> commands. Maybe the command syntax needs some more thinking to support
>> this. Different mailboxes could have different rules though..
> 
> Hmm, does doveadm support to read commands from stdin / file?

I was actually thinking about that myself today too.

> If I imagine such scenario with some hundreds of users having configured expire for some folders, that leads to plenty of process creations etc.pp.
> 
> doveadm <<EOT
> expunge -A Trash savedbefore 1w
> expunge -u user1 INBOX savedbefore 1y
> expunge -u user2 INBOX savedbefore 1m
> EOT

I don't think the process creation is much of an issue. But something like this could work:

doveadm expunge -A <<EOT
Trash savedbefore 1w
Trash/* savedbefore 1w
Spam savedbefore 2mon
EOT

Another problem I had been thinking about was how to do the search query parsing. For example with fetch:

doveadm fetch (unseen or unanswered) subject "hello world"

That won't work, because shell compains about () characters. But they could be escaped with \( and \). The next problem is the quotes. If I know that a string is supposed to follow after subject, it't not really a problem. But I'd rather not build separate parsers for command line parsing and stdin parsing (because in stdin I need to handle "" internally). But without knowing the context, I don't know if:

doveadm fetch subject "(foo)"

should be translated to string "(foo)" or if there should be a list with foo as its only element. So maybe the whole thing should be instead:

doveadm fetch '(unseen or unanswered) subject "hello world"'

but that makes it more difficult to add variables, because if you do:

doveadm fetch 'subject "$foo"'

you need to make sure $foo escapes " and \ characters.

.. difficult..