[Dovecot] fstat no such file or directory error
Periodically, we will have users IMAP sessions get hung. Their client sees a message like:
The server responded: 'SERVERBUG] Internal error occurred. Refer to server log for more information
On the server, we see the following in the logs:
IMAP(<username>): Transaction log dovecot.index.log: duplicate transaction log sequence (37) IMAP(<username>): fscking index file dovecot.index IMAP(<username>): fstat() failed with index file dovecot.index: No such file or directory
After the fstat message shows up, there doesn't appear to be anything on the user side that can be done to fix the problem. We've seen multiple instances where the file dovecot is complaining about not existing is visible, readable, and writable by the user in question, but dovecot is convinced it's not. The only way we've found to fix this issue is to kill off all user connections. When the user reconnects, then all appears to work fine. We have not found a way to recreate this problem.
Some details:
# 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.26-2-686 i686 Debian 5.0.8
Mail files are stored on NFS. All imap connections for a given user are are mapped to a particular server at any given time.
We have a script in place that watches our logs for this order of events, so the problem self corrects relatively quickly. But it would certainly be nice not have this happen at all.
Thanks,
Mark
--
I'd rather be burning carbohydrates than hydrocarbons
On Thu, 2011-03-24 at 13:19 -0400, Mark Dieterich wrote:
IMAP(<username>): Transaction log dovecot.index.log: duplicate transaction log sequence (37)
This is the reason everything else messes up..
IMAP(<username>): fscking index file dovecot.index IMAP(<username>): fstat() failed with index file dovecot.index: No such file or directory
fstat() should never return ENOENT. With NFS if the file is gone, it should be ESTALE. See if the attached patch helps.
After the fstat message shows up, there doesn't appear to be anything on the user side that can be done to fix the problem. We've seen multiple instances where the file dovecot is complaining about not existing is visible, readable, and writable by the user in question, but dovecot is convinced it's not. The only way we've found to fix this issue is to kill off all user connections. When the user reconnects, then all appears to work fine. We have not found a way to recreate this problem.
I think the problem is that Dovecot doesn't know how to deal with the failing ENOENT, so it just keeps on giving failures.
Some details:
# 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.26-2-686 i686 Debian 5.0.8
Mail files are stored on NFS. All imap connections for a given user are are mapped to a particular server at any given time.
What about new mail deliveries? Are you using deliver?
Timo,
Thanks for the patch. It's been applied and I'll let you know whether it clears up the problems.
Mail files are stored on NFS. All imap connections for a given user are are mapped to a particular server at any given time.
What about new mail deliveries? Are you using deliver?
All our mail delivery is done with procmail. The majority of the users are using Maildir, if this matters.
Mark
Timo,
I just wanted to follow up on this "issue". We've had the patch in place now for a couple of weeks and it seems to be doing the trick. We are no longer seeing these issues. A bug has been filed with RedHat about the underlying issue, but who knows when it will actually be fixed.
I don't know if you want to roll this patch into an official release or not, given it shouldn't really need to be there. I plan on keeping the patch around, just in case it takes a while for RedHat (upstream?) to release a proper fix.
Thanks again for your help!
Mark
Timo Sirainen wrote:
On Thu, 2011-03-24 at 13:19 -0400, Mark Dieterich wrote:
IMAP(<username>): Transaction log dovecot.index.log: duplicate transaction log sequence (37)
This is the reason everything else messes up..
IMAP(<username>): fscking index file dovecot.index IMAP(<username>): fstat() failed with index file dovecot.index: No such file or directory
fstat() should never return ENOENT. With NFS if the file is gone, it should be ESTALE. See if the attached patch helps.
--
I'd rather be burning carbohydrates than hydrocarbons
OK, added to hg: http://hg.dovecot.org/dovecot-2.0/rev/d14b0fd0a423
On Fri, 2011-04-08 at 11:50 -0400, Mark Dieterich wrote:
Timo,
I just wanted to follow up on this "issue". We've had the patch in place now for a couple of weeks and it seems to be doing the trick. We are no longer seeing these issues. A bug has been filed with RedHat about the underlying issue, but who knows when it will actually be fixed.
I don't know if you want to roll this patch into an official release or not, given it shouldn't really need to be there. I plan on keeping the patch around, just in case it takes a while for RedHat (upstream?) to release a proper fix.
Thanks again for your help!
Mark
Timo Sirainen wrote:
On Thu, 2011-03-24 at 13:19 -0400, Mark Dieterich wrote:
IMAP(<username>): Transaction log dovecot.index.log: duplicate transaction log sequence (37)
This is the reason everything else messes up..
IMAP(<username>): fscking index file dovecot.index IMAP(<username>): fstat() failed with index file dovecot.index: No such file or directory
fstat() should never return ENOENT. With NFS if the file is gone, it should be ESTALE. See if the attached patch helps.
participants (2)
-
Mark Dieterich
-
Timo Sirainen