[Dovecot] maildir maintenance?

[Brian Kroth]

Brian Kroth <bpkroth at gmail.com> 2010-11-24 13:28:
> Timo Sirainen <tss at iki.fi> 2010-11-24 19:04:
> > On 24.11.2010, at 18.59, Brian Kroth wrote:
> > 
> > >>> Hi, I'm running version 1.2.15 (so no doveadm)
> > >> You could build Dovecot v2.0 and only use doveadm binary from it.
> > > 
> > > Does it just issue the command via IMAP?  No direct filesystem
> > > operations?
> > 
> > It's all direct filesystem operations, no IMAP. But v1.2.15 can read v2.0's index files just fine.
> > 
> > >> Looks about ok. The main thing I'm worried is what happens if user creates mailboxes containing " or ' or ` characters.
> > > 
> > > Yeah, that was mostly me being lazy in my wanting to deal with escaping,
> > > so I just ignored them.
> > > 
> > > In what I originally wrote, I think it just won't touch them.
> > > 
> > > Or is the issue that the find command might remove them and then the
> > > indexes don't get fixed up?  I suppose I could just make sure that the
> > > find ignores those dirs, but I thought (from other maillist reading)
> > > that the next time their client SELECTs the folder it'll fix it up
> > > anyways.
> > 
> > I was more thinking what happens if the user creates a mailbox called `rm -rf /` or something.. Also if there are " or \ characters I think the LIST output will use literals and your parsing will break more or less badly.
> 
> That's certainly true.  I guess I was just hoping to skip over those
> mailboxes with unpleasant characters for the moment :}
> 
> More likely I'll rewrite this more carefully in Perl.
> 
> > > I suppose another spin on this would be for me to script the preauth
> > > imap client to figure out which mailboxes have messages marked for
> > > deletion of such and such an age and then try to use EXPUNGE to wipe
> > > just them out.  I'm not sure off hand if that's possible.
> > 
> > That would be a bit difficult at least to do via IMAP..
> 
> So I'm finding.  I guess I was thinking I could find the messages in a
> SELECTed mailbox via some parsing of either 
> - UID SEARCH DELETED X-SINCE $N_days_ago (where X-SINCE search X-SAVEDATE
>   instead of INTERNALDATE), or
> - UID FETCH 1:* (INTERNALDATE X-SAVEDATE FLAGS) as I've seen bantered about, or
> - combine the two and SEARCH DELETED, then
>   UID FETCH $initial_uid_list (X-SAVEDATE FLAGS) to refine the list.
> 
> Then use the (U?)IDs I get back from that to do
> - UID EXPUNGE $uid_list
> 
> Of course I've only started researching that avenue, so maybe that's not
> so reasonable.
> 
> I'm starting to see why so much effort has been expended on this front.
> 
> Thanks,
> Brian

So, I redid this in Perl to only use IMAP rather than any sudo or find
calls.  In theory then one doesn't need to worry about the indexes being
out of sync.  I still skipped over the "strange characters" mailboxes
for the moment.  I'm wondering what you think of this second rendition?

The only thing I'm not quite sure about is if there's some sort of race
between clients accessing/altering UIDs that may or may not get reused.
But I think this one is at least clear of the problem you mentioned
earlier.

In theory one would call it from cron like so:

30 0 * * * root /opt/cron/dovecot-maintenance.sh | logger -i -p mail.info -t dovecot-maintenance

Which loops over all the relevant users calling dovecot-maintenance.pl
on them in turn.  Could probably even fork off some small number to run
in parallel.

Thanks,
Brian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dovecot-maintenance.sh
Type: application/x-sh
Size: 903 bytes
Desc: not available
Url : http://dovecot.org/pipermail/dovecot/attachments/20101126/507d4bc0/attachment-0001.sh 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dovecot-maintenance.pl
Type: text/x-perl
Size: 8244 bytes
Desc: not available
Url : http://dovecot.org/pipermail/dovecot/attachments/20101126/507d4bc0/attachment-0002.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
Url : http://dovecot.org/pipermail/dovecot/attachments/20101126/507d4bc0/attachment-0003.bin