[Dovecot] Dovecot never release preallocated space in mdbox

[Timo Sirainen]

On 23.8.2013, at 15.32, Charles Marcus <CMarcus at Media-Brokers.com> wrote:

> On 2013-08-02 8:30 AM, Timo Sirainen <tss at iki.fi> wrote:
>> The problem is that there is no "rotation time". In normal operation Dovecot would be wasting time (=disk IO) looking at old files and figuring out if they would need hole punching. I guess the doveadm purge job could do that, but I'm not sure if that's always the best idea either. I remember some people using different mdbox settings for normal operations and for doveadm purge runs, so this could also unintentionally break things.
> 
> I'm considering migrating my users to mdbox, and thought that I had sent a follow-up to this, but found it in my Drafts folder...
> 
> This does give me a little pause for concern...
> 
> Shouldn't there be an internal, automatic mechanism in place for compacting (this I think is the more appropriate term, since it is what Thunderbird and Outlook both use, although it is used for local client files, as opposed to server based files) these files?
> 
> Personally I'd like it to happen immediately, when a user deletes a(some) message(es). Maybe add a new setting for 'mdbox_compact_trigger' or something like that, where it only does it if it will reclaim at least ## amount of space - although checking for that in and of itself might be as much or more work as just doing it immediately when the email(s) is(are) deleted.

There used to be some automatic purging done if enough mails were deleted, but that implementation made the performance much worse so I removed it. Some other better working automation could be added I guess.

> But, even if it wasn't fully automatic, maybe simplest would be just another option to pass to doveadm purge (ie, -C for 'compact)?

You mean for deleting the preallocated space? I guess it could look at the mdbox_rotate_interval and mdbox_preallocate_space settings and based on that remove the preallocated space from the old files.. But I'm still not convinced it's useful to combine those settings in any case, so doesn't seem like worth the trouble to implement it.

> Also - what happens currently when *all* messages in any given mdbox file are deleted? Is the mdbox file deleted?

When any messages are deleted from mdbox, the non-deleted messages are moved to another mdbox file and the old mdbox file is deleted.