On Mon, 2013-07-29 at 11:48 +0200, Stéphane BERTHELOT wrote:
mdbox_rotate_size = 128M mdbox_rotate_interval = 1d mdbox_preallocate_space = yes with virtual users and location like : mail_location = mdbox:~/mdbox
I don't think the remaining config is relevant but ask me if you need some other parts.
Using test accounts for 2 weeks now I've figured that the 128M preallocated space is never 'hole punched" (to use a similar term than "man fallocate" on Linux), even when rotating m.* files. Yeah, those settings weren't really intended to be used together. I am not sure to understand. Preallocation is based on rotate size actually, so you mean I shouldn't use a rotate interval with this kind of settings ? It would effectively "lose" less space since all files would be around
There would certainly be smart to use something similar to "FALLOC_FL_PUNCH_HOLE" on rotation (when doing close() ?) so that when we're sure there won't be anymore data appended to file that the allocated space == used space. The problem is that there is no "rotation time". In normal operation Dovecot would be wasting time (=disk IO) looking at old files and figuring out if they would need hole punching. I guess the doveadm purge job could do that, but I'm not sure if that's always the best idea either. I remember some people using different mdbox settings for normal operations and for doveadm purge runs, so this could also unintentionally break things.. Ok, I'm sorry I forgot to add that I'm using a cronjob to doveadm purge weekly. That effectively may be a good time to remove unneeded
Le 02/08/2013 14:30, Timo Sirainen a écrit : the preallocated size except the last one (current). preallocated sectors. I didn't have a look at the internals of the devecot process but I meant that this could happen when it chooses to switch to another file (because of rotation time, not purge). Effectively it may be at a time that we would want to avoid adding more IO...
When reading linux docs lately the more appropriate term may be "truncation" rather than "hole punching". That's what I did on command line (truncate with proper args) and it resetted the allocated size to "real" size.
I am pretty sure my understanding on those options was wrong so I already changed to 2M/no preallocation (since I didn't find any evidence that preallocation is making dovecot perform so much better ; there seemed to be a thread once on this list but noone concluded on that I think). My objective was to get a good compromise on performance (since I'm using ext4 anyways) and backup time (otherwise I wouldn't bother with rotation to "compensate" the potentially full rebuild of doveadm purge)
Maybe a documentation update should do the trick or a configuration warning stating that preallocation+rotate time would leave "wasted" space on disk.