[Dovecot] More efficient Deleting of a whole folder (or Purging Trash)

[Bill Cole]

At 4:03 PM +0000 1/29/08, Daniel Watts wrote:
>Hi,
>
>I've straced a few dovecot processes after hitting purge on a large 
>Trash folder (35,000 messages). It looks like it is going through 
>each message, one by one, and removing (unlinking) each one.
>
>Is there not a more efficient way to do this? If Dovecot knows the 
>whole folder is being deleted (ie a Trash purge), could it do 
>something clever with the filesystem to just remove the whole folder?

Not with most filesystems. Generally you need to unlink each file in 
a directory first before removing the directory. I'm not familiar 
with the details of XFS, but with most filesystems the directory is 
just a list of the names and inodes of the items in the directory, so 
deleting the directory node itself leaves all of those inodes and all 
of the data blocks they point to untouched: all you've done is lost 
track of them.

For Dovecot, the mailbox index also has to be updated for each 
deletion, and while that in itself is not terribly expensive per 
message, it has to be done one message at a time to maintain 
consistency between the index and the real mail.

>Incidently, the command that initially deleted those 35k messages to 
>Trash took about 1 hour to complete on my server and effectively 
>locked my account out for all that time. There were also an awlful 
>lot of these output from my strace:
>
>nanosleep({0, 131475000}, NULL)         = 0
>lstat("/home/virtual/xxx.com/home/admin/Maildir/.Folders.Porter/dovecot-uidlist.lock", 
>{st_mode=S_IFREG|0600, st_size=0, ...}) = 0
>
>Not sure if there's much one can do about speeding up deleting a lot 
>of messages though. My webmail client has a 'delete all messages' 
>button for each folder. If there was an efficient IMAP command for 
>deleting a whole folder that would be useful in those circumstances.
>
>Interested to know if there is anything that can be improved.

As a general issue, I doubt it.

However:

>Dovecot 1.0.9
>XFS Filesystem
>Maildir Directories

XFS historically has had very poor delete performance unless specially tuned.
-- 
Bill Cole                                  
bill at scconsult.com