At 4:03 PM +0000 1/29/08, Daniel Watts wrote:
Hi,
I've straced a few dovecot processes after hitting purge on a large Trash folder (35,000 messages). It looks like it is going through each message, one by one, and removing (unlinking) each one.
Is there not a more efficient way to do this? If Dovecot knows the whole folder is being deleted (ie a Trash purge), could it do something clever with the filesystem to just remove the whole folder?
Not with most filesystems. Generally you need to unlink each file in a directory first before removing the directory. I'm not familiar with the details of XFS, but with most filesystems the directory is just a list of the names and inodes of the items in the directory, so deleting the directory node itself leaves all of those inodes and all of the data blocks they point to untouched: all you've done is lost track of them.
For Dovecot, the mailbox index also has to be updated for each deletion, and while that in itself is not terribly expensive per message, it has to be done one message at a time to maintain consistency between the index and the real mail.
Incidently, the command that initially deleted those 35k messages to Trash took about 1 hour to complete on my server and effectively locked my account out for all that time. There were also an awlful lot of these output from my strace:
nanosleep({0, 131475000}, NULL) = 0 lstat("/home/virtual/xxx.com/home/admin/Maildir/.Folders.Porter/dovecot-uidlist.lock", {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
Not sure if there's much one can do about speeding up deleting a lot of messages though. My webmail client has a 'delete all messages' button for each folder. If there was an efficient IMAP command for deleting a whole folder that would be useful in those circumstances.
Interested to know if there is anything that can be improved.
As a general issue, I doubt it.
However:
Dovecot 1.0.9 XFS Filesystem Maildir Directories
XFS historically has had very poor delete performance unless specially tuned.
Bill Cole
bill@scconsult.com