[Dovecot] 1.0.14 -> 1.1.1: dovecot-uidlist errors
I've just updated from 1.0.14 to 1.1.1 (Maildirs). According to <URL:http://wiki.dovecot.org/Upgrading/1.1>, "dovecot-uidlist file is in a new format. The old format is automatically converted to new one."
When I try to delete a message (which moves it to the IMAP Trash folder), I get the following looping sequence of errors until I cancel the delete operation in the mail client:
Jun 24 12:22:13 chicken-ac dovecot: IMAP(schmonz): fdatasync_path(/home/schmonz/Maildir/.Trash/cur) failed: Bad file descriptor Jun 24 12:22:13 chicken-ac dovecot: Panic: IMAP(schmonz): file maildir-uidlist.c: line 1217 (maildir_uidlist_sync_update): assertion failed: (ctx->first_unwritten_pos != (unsigned int)-1) Jun 24 12:22:13 chicken-ac dovecot: child 5170 (imap) killed with signal 6 Jun 24 12:22:14 chicken-ac dovecot: imap-login: Login: user=<schmonz>, method=PLAIN, rip=68.46.70.233, lip=160.39.240.17, TLS Jun 24 12:22:16 chicken-ac dovecot: IMAP(schmonz): fdatasync_path(/home/schmonz/Maildir/.Trash/cur) failed: Bad file descriptor Jun 24 12:22:16 chicken-ac dovecot: Panic: IMAP(schmonz): file maildir-uidlist.c: line 1217 (maildir_uidlist_sync_update): assertion failed: (ctx->first_unwritten_pos != (unsigned int)-1) Jun 24 12:22:16 chicken-ac dovecot: child 27601 (imap) killed with signal 6 Jun 24 12:22:17 chicken-ac dovecot: imap-login: Login: user=<schmonz>, method=PLAIN, rip=68.46.70.233, lip=160.39.240.17, TLS Jun 24 12:22:19 chicken-ac dovecot: IMAP(schmonz): fdatasync_path(/home/schmonz/Maildir/.Trash/cur) failed: Bad file descriptor Jun 24 12:22:19 chicken-ac dovecot: Panic: IMAP(schmonz): file maildir-uidlist.c: line 1217 (maildir_uidlist_sync_update): assertion failed: (ctx->first_unwritten_pos != (unsigned int)-1) Jun 24 12:22:19 chicken-ac dovecot: child 29637 (imap) killed with signal 6 Jun 24 12:22:20 chicken-ac dovecot: imap-login: Login: user=<schmonz>, method=PLAIN, rip=68.46.70.233, lip=160.39.240.17, TLS Jun 24 12:22:22 chicken-ac dovecot: IMAP(schmonz): fdatasync_path(/home/schmonz/Maildir/.Trash/cur) failed: Bad file descriptor Jun 24 12:22:22 chicken-ac dovecot: Panic: IMAP(schmonz): file maildir-uidlist.c: line 1217 (maildir_uidlist_sync_update): assertion failed: (ctx->first_unwritten_pos != (unsigned int)-1) Jun 24 12:22:22 chicken-ac dovecot: child 9143 (imap) killed with signal 6 Jun 24 12:22:22 chicken-ac dovecot: imap-login: Login: user=<schmonz>, method=PLAIN, rip=68.46.70.233, lip=160.39.240.17, TLS Jun 24 12:22:23 chicken-ac dovecot: IMAP(schmonz): Connection closed bytes=45/352132 Jun 24 12:24:23 chicken-ac dovecot: imap-login: Login: user=<schmonz>, method=PLAIN, rip=68.46.70.233, lip=160.39.240.17, TLS
Alternatively, sometimes I get an error dialog in the mail client: "The current command did not succeed. The mail server responded: BUG: Unknown internal error."
Have I done something wrong? Can I provide more information to help narrow this down?
Thanks,
- Amitai
On Jun 24, 2008, at 7:31 PM, Amitai Schlair wrote:
I've just updated from 1.0.14 to 1.1.1 (Maildirs). According to <URL:http://wiki.dovecot.org/Upgrading/1.1
, "dovecot-uidlist file is in a new format. The old format is
automatically converted to new one."When I try to delete a message (which moves it to the IMAP Trash
folder), I get the following looping sequence of errors until I
cancel the delete operation in the mail client:
Jun 24 12:22:13 chicken-ac dovecot: IMAP(schmonz): fdatasync_path(/ home/schmonz/Maildir/.Trash/cur) failed: Bad file descriptor
What OS and filesystem do you use? Apparently it doesn't like that I
try to fsync() a read-only directory file descriptor. But then again I
don't know if there's another alternative than to just not fsync() the
directory.
Jun 24 12:22:13 chicken-ac dovecot: Panic: IMAP(schmonz): file
maildir-uidlist.c: line 1217 (maildir_uidlist_sync_update):
assertion failed: (ctx->first_unwritten_pos != (unsigned int)-1)
Still shouldn't crash even if the fsync fails, I'll see about fixing
that.
A workaround for now would be to se fsync_disable=yes
Timo Sirainen wrote:
On Jun 24, 2008, at 7:31 PM, Amitai Schlair wrote:
Jun 24 12:22:13 chicken-ac dovecot: IMAP(schmonz): fdatasync_path(/home/schmonz/Maildir/.Trash/cur) failed: Bad file descriptor
What OS and filesystem do you use?
NetBSD 4.0, FFS with soft updates.
Jun 24 12:22:13 chicken-ac dovecot: Panic: IMAP(schmonz): file maildir-uidlist.c: line 1217 (maildir_uidlist_sync_update): assertion failed: (ctx->first_unwritten_pos != (unsigned int)-1)
Still shouldn't crash even if the fsync fails, I'll see about fixing that.
A workaround for now would be to se fsync_disable=yes
With the workaround, deleting works, and so copying sent messages to Sent will probably also work again (just about to find out :-).
Thanks for the quick response! If you need a fix tested, I can do that.
- Amitai
On Tue, Jun 24, 2008 at 6:50 PM, Amitai Schlair <schmonz@schmonz.com> wrote:
Timo Sirainen wrote:
On Jun 24, 2008, at 7:31 PM, Amitai Schlair wrote:
Jun 24 12:22:13 chicken-ac dovecot: IMAP(schmonz): fdatasync_path(/home/schmonz/Maildir/.Trash/cur) failed: Bad file descriptor
What OS and filesystem do you use?
NetBSD 4.0, FFS with soft updates.
Hello, I got the same problem on NetBSD 4.0 - Stable with FFS too. The fixed work for me too.
-- Jean-Edouard BABIN
Same Problem here: both the failed fsync and the Panic.
What OS and filesystem do you use? NetBSD 4.0, Mails on NFS (server is a Debian box)
Still shouldn't crash even if the fsync fails, I'll see about
fixing that. Yes, please.
A workaround for now would be to se fsync_disable=yes Same workaround works for me.
No need to hurry, it's just a test server (production just moved from
1.0rc18 to 1.0.13).
Apparently it doesn't like that I try to fsync() a read-only
directory file descriptor. I would argue that NetBSD is correct in this.
Posix says:
The fsync() function shall request that all data for the open file
descriptor named by fildes is to be transferred to the storage
device associated with the file described by fildes.
I read this a being about the file descriptor, not the file. Which
would imply that fsync()ing an O_RDONLY descriptor doesn't make sense.
However, the wording for fdatasync() is slightly different and not as
clear, whether intentional or not. But fdatasync() seems to be
conceptually about regular files anyway.
From an admittedly non-Linux perspective, I'm unsure why one would
need to perform a flush on a directory in the first place.
On Jul 9, 2008, at 3:48 PM, Edgar Fuß wrote:
Apparently it doesn't like that I try to fsync() a read-only
directory file descriptor. I would argue that NetBSD is correct in this.Posix says:
The fsync() function shall request that all data for the open file
descriptor named by fildes is to be transferred to the storage
device associated with the file described by fildes.I read this a being about the file descriptor, not the file. Which
would imply that fsync()ing an O_RDONLY descriptor doesn't make sense.However, the wording for fdatasync() is slightly different and not
as clear, whether intentional or not. But fdatasync() seems to be
conceptually about regular files anyway.
Dovecot isn't the only program trying to fsync directories. For
example sendmail does it too.
From an admittedly non-Linux perspective, I'm unsure why one would
need to perform a flush on a directory in the first place.
To make sure that if server crashes all the directory entries have
been written to disk by that time. So that:
- fsync(tmp/newmail)
- rename(tmp/newmail, new/newmail)
- fsync(new/)
- Reply OK to the sender
If there is no 3 and the server crashes just after 4, I don't think
the rename() was guaranteed to have reached the disk. The fsync(new/)
is supposed to guarantee this. If NetBSD doesn't support this, what
other way is there to do it? I'm guessing I could fsync(new/newmail)
again. But if I was creating a lot of mails, would I have to fsync()
all of the files?
(The reason why fsync(tmp/newmail) is done before rename is so that
half-written files won't show up in new/ directory in case of crashes,
so moving the fsync after rename doesn't solve the problem.)
If there is no 3 and the server crashes just after 4, I don't think
the rename() was guaranteed to have reached the disk. OK, I understand.
If NetBSD doesn't support this, what other way is there to do it? I don't know of a way of doing this. The problem probably is that
metadata writes used to be synchronous on FFS from the beginning.
However the goal was not to guarantee that any metadata change made
it to persistent storage before the system call returned so the
change would survive a crash. The goal was to always have the on-disk
metadata in a state, which, if not consistent, could be brought to a
consistent state by fsck. However, soft updates achieve this goal a
different way so UFS with soft updates doesn't do those synchronous
writes anymore.
I'll ask the question on tech-kern.
I'll ask the question on tech-kern. It looks like there is no method of explicitly syncing a drectory; you'll have to sync the individual files instead.
The reason is probably that before the advent of soft updates, metadata write on FFS used to be syncronous anyway.
On Tue, Jun 24, 2008 at 07:36:44PM +0300, Timo Sirainen wrote:
On Jun 24, 2008, at 7:31 PM, Amitai Schlair wrote:
Jun 24 12:22:13 chicken-ac dovecot: Panic: IMAP(schmonz): file
maildir-uidlist.c: line 1217 (maildir_uidlist_sync_update):
assertion failed: (ctx->first_unwritten_pos != (unsigned int)-1)Still shouldn't crash even if the fsync fails, I'll see about fixing that.
Apart from the design issue of fsync'ing read-only directory file handles, have you been able to look into fixing the assert panic? :-)
A workaround for now would be to se fsync_disable=yes
Personally, I don't like this since (I assume) it disables fsync'ing of tmp/ files (mails uploaded via imap) as well?
GeertOn Sat, 2008-07-19 at 22:57 +0200, Geert Hendrickx wrote:
On Tue, Jun 24, 2008 at 07:36:44PM +0300, Timo Sirainen wrote:
On Jun 24, 2008, at 7:31 PM, Amitai Schlair wrote:
Jun 24 12:22:13 chicken-ac dovecot: Panic: IMAP(schmonz): file
maildir-uidlist.c: line 1217 (maildir_uidlist_sync_update):
assertion failed: (ctx->first_unwritten_pos != (unsigned int)-1)Still shouldn't crash even if the fsync fails, I'll see about fixing that.
Apart from the design issue of fsync'ing read-only directory file handles,
I guess this will do for now: http://hg.dovecot.org/dovecot-1.1/rev/a8331972aee5
have you been able to look into fixing the assert panic? :-)
http://hg.dovecot.org/dovecot-1.1/rev/1edff123dd8f probably fixes it?
A workaround for now would be to se fsync_disable=yes
Personally, I don't like this since (I assume) it disables fsync'ing of tmp/ files (mails uploaded via imap) as well?
Right.
participants (5)
-
Amitai Schlair
-
Edgar Fuß
-
Geert Hendrickx
-
Jean-Edouard Babin
-
Timo Sirainen