On Fri, 28 Oct 2005, Timo Sirainen wrote:

On Thu, 2005-10-27 at 17:58 -0400, Todd Burroughs wrote:

...

Just installed the version from CVS as of Oct. 27.

I noticed three problems quite quickly:

My INBOX closed with "access error" after reading it for a bit. There's nothing in the logs or anything, but this hasn't happened for quite a while now. There where quite a lot of incoming messages at the time and I was marking messages as deleted using pine. After reverting to the last version of dovecot that I was using and restarting pine, it opened quickly (no reindexing) and the deleted emails where marked as deleted. I don't know if this helps much ;-)

Well, I don't know what "access error" means.. And do you mean you didn't try again with the new Dovecot, just reverted to old one?

I reverted back to the old one, as the new one was generating a lot of errors. As far as I can tell, "access error" means that the imap server dropped connection. Usually I see a crash when this happens. I'm running today's CVS version now and haven't seen any errors, although it is late Friday... This may have just been a coincidence.

...

dovecot: Oct 27 16:58:12 Error: 13481 imap(username): file mbox-sync-rewrite.c: line 393 (mbox_sync_read_and_move): assertion failed: (need_space == (uoff_t)-mails[idx].space)

Is this from older Dovecot too? I hoped I had fixed this yesterday.

This is from the newer one. The last Changelog entry was this:

2005-10-27 15:44 Timo Sirainen tss@iki.fi

     * configure.in: typofix

This user hasn't logged into the newest version yet. I see a couple entries in the new Changelog regarding mbox-sync-rewrite.c, so maybe it is fixed.

Todd

Todd Burroughs wrote:

I'm now running CVS version as of Oct 28. A couple users have gotten a very similar assertion failure. Here's the log entry and backtrace:

dovecot: Nov 01 05:07:18 Error: 8434 imap(username): file mbox-sync-rewrite.c: line 383 (mbox_sync_read_and_move): assertion failed: (seq == mail_ctx->seq)

I have reported this one before (running CVS from Oct 31) and sent backtraces as well. With this version of dovecot I've also seen assertion failures:

file buffer.c: line 230 (buffer_copy): assertion failed: (src_pos <= src->used)

file mbox-sync-rewrite.c: line 400 (mbox_sync_read_and_move): assertion failed: (need_space == (uoff_t)-mails[idx].space)

file mbox-sync.c: line 1297 (mbox_sync_handle_eof_updates): assertion failed: (file_size >= sync_ctx->expunged_space + trailer_size)

Have given backtraces of all these failures to the dovecot-list. Hope to see fixes in near future. ;-)

• Anders

-- Anders Lund Anders.Lund@uninett.no .~. UNINETT, N-7465 Trondheim, Norway / V
Phone: +47 73 55 79 08 | Fax: +47 73 55 79 01 /( )
^ ^

Tomi Hakala wrote:

Anders Lund wrote:

...

Have given backtraces of all these failures to the dovecot-list. Hope to see fixes in near future. ;-)

If you are not yet using Dovecot LDA could you please try it out and see if problems persist?

I will consider looking more into Dovecot LDA, but I'm a bit reluctant to "try it out" on my server to see if the problems disappear. In the first place I wanted to keep my delivery separate from Dovecot, and just use Dovecot for IMAP. I have thought of looking more into Dovecot LDA to see if SIEVE would give me better support for a simple way for my users to add vacation messages, filters, etc., but not until later. I still hope to see fixes for the assertion failures in Dovecot CVS, but if everything would be solved by running Dovecot LDA I guess I will have to look more at this solution. Anyhow, I don't understand what is so special about my setup, and why I should get these failures.

• Anders

-- Anders Lund Anders.Lund@uninett.no .~. UNINETT, N-7465 Trondheim, Norway / V
Phone: +47 73 55 79 08 | Fax: +47 73 55 79 01 /( )
^ ^

Tomi Hakala wrote:

Anders Lund wrote:

...

Anyhow, I don't understand what is so special about my setup, and why I should get these failures.

All I can think of is that you have some locking issue which is causing mbox corruption.

When it comes to locking I have this:

Postfix: mailbox_delivery_lock = fcntl, dotlock

Dovecot: lock_method = fcntl mbox_read_locks = fcntl dotlock mbox_write_locks = fcntl dotlock

"procmail -v" gives me: Locking strategies: dotlocking, fcntl()

I thought this should work fine, but if you spot any possible problems give me a hint.

• Anders

-- Anders Lund Anders.Lund@uninett.no .~. UNINETT, N-7465 Trondheim, Norway / V
Phone: +47 73 55 79 08 | Fax: +47 73 55 79 01 /( )
^ ^

Michael Paesold wrote:

For Postfix and Dovecot, the methods are the same for your configuration, but procmail uses them the other way round. So if you use procmail for mail delivery, this *could* cause problems, I guess.

Can you try to set locking for postfix and dovecot mbox to "dotlock fcntl" instead of "fcntl dotlock" and see if that improves your situation.

I've implemented this change now.

Anyone know it it's possible to change the way Procmail does locking without recompiling?

My setup is now:

Postfix: mailbox_delivery_lock = dotlock, fcntl

Dovecot: lock_method = dotlock mbox_read_locks = dotlock fcntl mbox_write_locks = dotlock fcntl

"procmail -v" gives me: Locking strategies: dotlocking, fcntl()

Had to set

mmap_disable=yes

for Dovecot for this to work, otherwise I get:

lock_method=dotlock and mmap_disable=no combination isn't supported. You don't _really_ want it anyway.

So, cross my fingers and hope for improvements. :)

• Anders

-- Anders Lund Anders.Lund@uninett.no .~. UNINETT, N-7465 Trondheim, Norway / V
Phone: +47 73 55 79 08 | Fax: +47 73 55 79 01 /( )
^ ^

On Thu, 3 Nov 2005, Anders Lund wrote:

Todd Burroughs wrote:

...

I'm now running CVS version as of Oct 28. A couple users have gotten a very similar assertion failure. Here's the log entry and backtrace: (seq == mail_ctx->seq)

I have reported this one before (running CVS from Oct 31) and sent backtraces as well. With this version of dovecot I've also seen assertion failures:

Anders,

Try running the current CVS version. Timo fixed a few things since Oct. 31 and I haven't seen an assert or any failure in 5 days now.

Todd

Anders Lund wrote:

Todd Burroughs wrote:

...

Anders,

Try running the current CVS version. Timo fixed a few things since Oct. 31 and I haven't seen an assert or any failure in 5 days now.

Just upgraded to nightly from 2005-11-09 now. Tailing my log for assertion failures now and crossing my fingers again. ;-)

Correcting my settings for locking in Postfix and Dovecot (to have the same priorities as Procmail) might have given some good results as well.

And for one user I discovered another problem: this at least made the mboxes readable for pine, which had previous

• running "file" on mboxes showed: "ISO-8859 mail text, with CRLF, LF line terminators" which I fixed with "dos2unix" on the mboxes (still some of these have "ISO-8859 mail text, with CR, LF line terminators")
• in addition some messages had wrong date stamp, which I fixed with "sed -i 's/Jan 1 /Jan 01 /' *"

problems. It also seemed to remove "assertion failed" with dovecot for this user.

Just a short update: Since upgrade to the nightly build from 2005-11-09 I've had no assertion failed. It seems a lot more stable on my system now, so the combination of:

• setting correct locking priority (dotlock before fcntl in Dovecot/Postfix, to comply with Procmail)
• correcting mboxes (typically this is mboxes moved from local Thunderbird folders on laptops running Windows to the IMAP server)
• upgrading to the latest nightly from 2005-11-09 seemed to be a good solution for me. Thanks for the suggestions I've got!

One final issue I've seen: Thunderbird 1.0.7 sometimes seems to go into a state of being unable to contact the IMAP server (when accessing a new folder). When this has happened I've quit Thunderbird, but can still see imap process running on the server. If I restart Thunderbird I get a new imap process, and the old one is still running. This is perhaps a client issue? Have others seen this?

• Anders

-- Anders Lund Anders.Lund@uninett.no .~. UNINETT, N-7465 Trondheim, Norway / V
Phone: +47 73 55 79 08 | Fax: +47 73 55 79 01 /( )
^ ^