[Dovecot] Cannot remove Maildir folder
Hi,
I have been using dovecot 1.0 beta 8 on Debian Sarge for a couple of days and I experienced some problems when removing folders..
As an IMAP client I use Thunderbird 1.5 and IMAP folders are on NFS. Here are the steps to re-produce the problem:
- Run Thunderbird as usual
- Create folder
- Remove folder - In my case folder is not removed but moved to Trash (Thunderbird setting)
- I go to Trash and remove it from there
- Folder is removed but I get additional ..DOVECOT-TRASHED directory - next time it won't be possible to remove it
Some more info:
- Just before removal I run lsof to see which files are in use:
bash# lsof /home/marcin ... imap 21360 marcin cwd DIR 0,15 344064 3307008 /home/marcin (filer:/vol/vol0/home/marcin) imap 21360 marcin 8u DIR 0,15 4096 4223651 /home/marcin/Maildir/.Trash.test/new (filer:/vol/vol0/home/marcin) imap 21360 marcin 9u DIR 0,15 4096 4223650 /home/marcin/Maildir/.Trash.test/cur (filer:/vol/vol0/home/marcin) imap 21360 marcin 11r REG 0,15 144 4223663 /home/marcin/Maildir/.Trash.test/dovecot.index (filer:/vol/vol0/home/marcin) imap 21360 marcin 12r REG 0,15 104 4223647 /home/marcin/Maildir/.Trash.test/dovecot.index.log (filer:/vol/vol0/home/marcin) imap 21360 marcin 13u REG 0,15 10272 4223657 /home/marcin/Maildir/.Trash.test/dovecot.index.cache (filer:/vol/vol0/home/marcin) ...
There are more files in use but in this case I'm interested with PID 21360 which has opened Trash/test (the one which I'm going to remove)
- Just after removal I see error message in Syslog:
dovecot: IMAP(marcin): unlink_directory(/home/marcin/Maildir/..DOVECOT-TRASHED) failed: Device or resource busy
Running lsof again gives the following:
imap 21360 marcin cwd DIR 0,15 344064 3307008 /home/marcin (filer:/vol/vol0/home/marcin) imap 21360 marcin 11r REG 0,15 144 4223663 /home/marcin/Maildir/..DOVECOT-TRASHED/.nfs004072af0002cb1b (filer:/vol/vol0/home/marcin) imap 21360 marcin 12r REG 0,15 104 4223647 /home/marcin/Maildir/..DOVECOT-TRASHED/.nfs0040729f0002cb1a (filer:/vol/vol0/home/marcin) imap 21360 marcin 13u REG 0,15 10272 4223657 /home/marcin/Maildir/..DOVECOT-TRASHED/.nfs004072a90002cb1c (filer:/vol/vol0/home/marcin)
Looking into some NFS docs we can read:
"A client has a file opened. Another process on that client deletes the
file. So the client NFS stack rather than deleting the file and
potentially hurting the other process, it renames the file to
.nfs
To me it looks like one imap process tries to delete .Trash.test directory (first rename and then unlink) whereas other imap process has still opened dovecot.* files. Any workaround/fix for that ? Regards,
Marcin Deranek
On Tue, 2006-05-30 at 16:22 +0200, Marcin Deranek wrote:
- Folder is removed but I get additional ..DOVECOT-TRASHED directory - next time it won't be possible to remove it
Dovecot should delete that directory automatically the next time LIST command is given.
dovecot: IMAP(marcin): unlink_directory(/home/marcin/Maildir/..DOVECOT-TRASHED) failed: Device or resource busy
Dovecot could just ignore that error also, but I don't know if that's a good idea..
To me it looks like one imap process tries to delete .Trash.test directory (first rename and then unlink) whereas other imap process has still opened dovecot.* files.
Most likely that Thunderbird has opened multiple connections, and in one it still has the mailbox opened while another one tries to delete it.
Any workaround/fix for that ?
Not really. Is it a real problem? :)
Pretty much the only way to fix this would be to make one imap process tell other imap processes to close the mailbox before it tries deleting it. Too much trouble to be worth it.
Another way would be to make one IMAP process handle all the connections from a user. That's something I've been thinking about doing optionally in Dovecot 2.0.
Timo Sirainen wrote:
Dovecot should delete that directory automatically the next time LIST command is given.
Hmm.. It seems it does not work: I restarted Thunderbird 3 times and directory is still there, issued LIST "" * and directory did not disappear :-(
Dovecot could just ignore that error also, but I don't know if that's a good idea..
Probably not.. People want to know when somehething goes wrong :-)
Most likely that Thunderbird has opened multiple connections, and in one it still has the mailbox opened while another one tries to delete it.
Yes, default is 5. I'll try to set it to 1 and see if it helps.
Not really. Is it a real problem? :)
For users: yes and that's the biggest group :-))
Pretty much the only way to fix this would be to make one imap process tell other imap processes to close the mailbox before it tries deleting it. Too much trouble to be worth it.
I can imagine..
Marcin
Hi Timo,
Marcin Deranek wrote:
Yes, default is 5. I'll try to set it to 1 and see if it helps.
It doesn't help, but I found the problem..
Short story: files dovecot.index, dovecot.index.log and dovecot.index.cache are unlinked before they are closed.. close-before-unlink should fix the problem for NFS.
Long story:
Just before removal: |fd|path| 10 -> /home/marcin/Maildir/.Trash.test/dovecot.index.cache 8 -> /home/marcin/Maildir/.Trash.test/dovecot.index 9 -> /home/marcin/Maildir/.Trash.test/dovecot.index.log
strace bash$ egrep '^(rmdir|unlink|close)' /tmp/out.txt close(12) = 0 close(11) = 0 close(11) = 0 unlink("cur") = -1 EISDIR (Is a directory) close(14) = 0 close(15) = 0 rmdir("cur") = 0 unlink("new") = -1 EISDIR (Is a directory) close(14) = 0 close(15) = 0 rmdir("new") = 0 unlink("tmp") = -1 EISDIR (Is a directory) close(14) = 0 close(15) = 0 rmdir("tmp") = 0 unlink("dovecot.index.log") = 0 unlink("dovecot.index") = 0 unlink("dovecot.index.cache") = 0 unlink(".nfs008b830b0002e5f4") = -1 EBUSY (Device or resource busy)
Just after |fd|path| 10 -> /home/marcin/Maildir/..DOVECOT-TRASHED/.nfs008b830d0002e5f6 8 -> /home/marcin/Maildir/..DOVECOT-TRASHED/.nfs008b83100002e5f5 9 -> /home/marcin/Maildir/..DOVECOT-TRASHED/.nfs008b830b0002e5f4
It seems when you unlink the file, before you close it causes NFS client to rename the file to .nfsXXX even if both operations were done by the very same process (tested with simple C program). In strace output I do not see any close(8-10), so files were opened when they were unlinked. I hope we can do something about it :-)) Cheers,
Marcin
P.S. Even with Thunderbird using 5 IMAP connections seems like all requests to single folder are handled by a single imap process.
On Wed, 2006-05-31 at 10:11 +0200, Marcin Deranek wrote:
Hi Timo,
Marcin Deranek wrote:
Yes, default is 5. I'll try to set it to 1 and see if it helps.
It doesn't help, but I found the problem..
Short story: files dovecot.index, dovecot.index.log and dovecot.index.cache are unlinked before they are closed.. close-before-unlink should fix the problem for NFS.
Looks like that's because Dovecot leaves the index files open for a while after closing the mailbox in case it gets reopened soon. I'll fix that.
Still if another process has the mailbox opened it'll fail, but I don't know if that's going to be a real problem.
Timo Sirainen wrote:
Looks like that's because Dovecot leaves the index files open for a while after closing the mailbox in case it gets reopened soon. I'll fix that.
Thanks.
Still if another process has the mailbox opened it'll fail, but I don't know if that's going to be a real problem.
Most probably not a 'real life' problem - as far as I've seen operations on a single folder are handled by the same imap process (including removal).
Marcin
participants (2)
-
Marcin Deranek
-
Timo Sirainen