[Dovecot] test80: assert/core debug info

Jeff A. Earickson

16 Aug 2005 16 Aug '05

6:14 p.m.

Timo,

Attached is gdb information from core dumps related to the following assert in test-80:

IMAP(username): file mbox-sync-update.c: line 442 (mbox_sync_update_header_from): assertion failed: (ctx->mail.uid == 0 || ctx->mail.uid_broken || ctx->mail.uid == mail->uid)

My setup: Solaris 9, mbox format. test-80 compiled with gcc 4.0.1 using the following configure options:

CC=gcc CFLAGS="-g -O" CPPFLAGS=-I/opt/openssl/include LDFLAGS=-L/opt/openssl/lib
./configure --prefix=/opt/dovecot.test80 --with-ssl=openssl
--with-ssldir=/opt/openssl/ssl --with-rawlog --enable-debug

Note that system mailboxes are NFS mounted from the mail server to the IMAP server, which may cause you to groan.

The attached debug session was done with gdb 6.3, also compiled with gcc 4.0.1. Note that some of the variables I queried were undefined (or otherwise unknown to gdb). The variable "first_nonexpunged" at line 470 of mbox-sync-rewrite.c is undefined, and I wondered why it was scoped so locally. The value of mails[idx] being undefined at mbox-sync-rewrite.c:353 also seems like bad news.

Hope this helps.

Jeff Earickson Colby College

Attachments:

typescript.txt (text/plain — 8.5 KB)

Show replies by date

Andrew Hutchings

16 Aug 16 Aug

7:56 p.m.

On Tue, 2005-08-16 at 11:14 -0400, Jeff A. Earickson wrote:

...

My setup: Solaris 9, mbox format. test-80 compiled with gcc 4.0.1 using the following configure options:

...

Note that system mailboxes are NFS mounted from the mail server to the IMAP server, which may cause you to groan.

...

Jeff Earickson Colby College

I could be wrong here, but as far as I am aware mbox is not NFS safe, that could be your problem.

Regards Andrew

Andrew Hutchings (A-Wing) - Linux Guru Netserve Consultants - http://www.domaincity.co.uk/ Linux CDs and DVDs - http://www.linuxiso.co.uk/ Random quote 196: Your password must be at least 18770 characters and cannot repeat any of your previous 30689 passwords. Please type a different password. Type a password that meets these requirements in both text boxes. - MicroSoft error message

Timo Sirainen

17 Aug 17 Aug

2:32 a.m.

On 16.8.2005, at 18:14, Jeff A. Earickson wrote:

...

Attached is gdb information from core dumps related to the following assert in test-80:

IMAP(username): file mbox-sync-update.c: line 442 (mbox_sync_update_header_from): assertion failed: (ctx->mail.uid == 0 || ctx->mail.uid_broken || ctx->mail.uid == mail->uid) ..

This patch will fix it I think? --- src/lib-storage/index/mbox/mbox-sync-parse.c 7 Aug 2005 11:28:13 -0000 1.44 +++ src/lib-storage/index/mbox/mbox-sync-parse.c 16 Aug 2005 23:30:00 -0000 @@ -342,7 +342,7 @@ return TRUE; } - if (ctx->sync_ctx->seq == 1 && !ctx->seen_imapbase) { + if (ctx->seq == 1 && !ctx->seen_imapbase) { /* Don't bother allowing X-UID before X-IMAPbase header. c-client doesn't allow it either, and this way the UID doesn't have to be reset if X-IMAPbase

Jeff A. Earickson

2:26 p.m.

Timo, Thank you, the patch solved the problem of core dumps and asserts. There have been none in the last eight hours since I applied the patch. The syslog output has much fewer complaints. There are occasional mail.error syslog warnings like: IMAP(user): UIDs broken with partial sync in mbox file /var/mail/user and one occurance of: IMAP(user): mbox sync: UID inserted in the middle of mailbox /var/mail/user (195 > 34, seq=2, idx_msgs=31) Anything to worry about? Jeff Earickson Colby College On Wed, 17 Aug 2005, Timo Sirainen wrote:

...

Date: Wed, 17 Aug 2005 02:32:20 +0300 From: Timo Sirainen To: Jeff A. Earickson Cc: dovecot@dovecot.org Subject: Re: [Dovecot] test80: assert/core debug info

On 16.8.2005, at 18:14, Jeff A. Earickson wrote:

...
Attached is gdb information from core dumps related to the following assert in test-80:

IMAP(username): file mbox-sync-update.c: line 442 (mbox_sync_update_header_from): assertion failed: (ctx->mail.uid == 0 || ctx->mail.uid_broken || ctx->mail.uid == mail->uid) ..

This patch will fix it I think?

--- src/lib-storage/index/mbox/mbox-sync-parse.c 7 Aug 2005 11:28:13 -0000 1.44 +++ src/lib-storage/index/mbox/mbox-sync-parse.c 16 Aug 2005 23:30:00 -0000 @@ -342,7 +342,7 @@ return TRUE; }

- if (ctx->sync_ctx->seq == 1 && !ctx->seen_imapbase) { + if (ctx->seq == 1 && !ctx->seen_imapbase) { /* Don't bother allowing X-UID before X-IMAPbase header. c-client doesn't allow it either, and this way the UID doesn't have to be reset if X-IMAPbase

Timo Sirainen

5:16 p.m.

On Wed, 2005-08-17 at 07:26 -0400, Jeff A. Earickson wrote:

...

Timo,

Thank you, the patch solved the problem of core dumps and asserts. There have been none in the last eight hours since I applied the patch. The syslog output has much fewer complaints. There are occasional mail.error syslog warnings like:

IMAP(user): UIDs broken with partial sync in mbox file /var/mail/user

and one occurance of:

IMAP(user): mbox sync: UID inserted in the middle of mailbox /var/mail/user (195 > 34, seq=2, idx_msgs=31)

Anything to worry about?

They aren't exactly good, but shouldn't really happen either. The first one is completely transparent to users, the second one probably send some error to client.

If you can figure out a way to reproduce any of these, I'd like to know. I haven't seen them myself for a long time.

Although if locking is set wrong, these can happen. Can your users access the mboxes directly or using some other software than Dovecot?

Jeff A. Earickson

5:45 p.m.

On Wed, 17 Aug 2005, Timo Sirainen wrote:

...

Date: Wed, 17 Aug 2005 17:16:51 +0300 From: Timo Sirainen tss@iki.fi To: Jeff A. Earickson jaearick@colby.edu Cc: dovecot@dovecot.org Subject: Re: [Dovecot] test80: assert/core debug info

On Wed, 2005-08-17 at 07:26 -0400, Jeff A. Earickson wrote:

...
Timo,

Thank you, the patch solved the problem of core dumps and asserts. There have been none in the last eight hours since I applied the patch. The syslog output has much fewer complaints. There are occasional mail.error syslog warnings like:

IMAP(user): UIDs broken with partial sync in mbox file /var/mail/user

and one occurance of:

IMAP(user): mbox sync: UID inserted in the middle of mailbox /var/mail/user (195 > 34, seq=2, idx_msgs=31)

Anything to worry about?

They aren't exactly good, but shouldn't really happen either. The first one is completely transparent to users, the second one probably send some error to client.

If you can figure out a way to reproduce any of these, I'd like to know. I haven't seen them myself for a long time.

Although if locking is set wrong, these can happen. Can your users access the mboxes directly or using some other software than Dovecot?

My lock setting is the default (the conf file is #lock_method = fcntl). Yes, users can hit their mailboxes by other means. The IMAP server (dovecot) is also the machine that people can login to interactively. System mailboxes are NFS mounted from the mail server, which people cannot login to. Pine users on IMAP server typically hit their mailbox directly via the NFS mount, but this can be easily changed so they go thru dovecot IMAP. The mail server is also the POP server, running qpopper (I should probably change to dovecot there). People are admonished to use either POP or IMAP but not both.

The reasons for this setup are historical and security-related. The IMAP and interactive login server has to remain different from the mail/POP server and the NFS mount must remain. Other than that, I am open to suggestions for improvement here. The two things I should do are:

make pine users go thru IMAP
change from qpopper POP to dovecot POP on the mail server

Jeff Earickson Colby College

Timo Sirainen

19 Aug 19 Aug

7:08 p.m.

On Wed, 2005-08-17 at 10:45 -0400, Jeff A. Earickson wrote:

...

...
...
IMAP(user): UIDs broken with partial sync in mbox file /var/mail/user IMAP(user): mbox sync: UID inserted in the middle of mailbox /var/mail/user (195 > 34, seq=2, idx_msgs=31)

Although if locking is set wrong, these can happen. Can your users access the mboxes directly or using some other software than Dovecot?

My lock setting is the default (the conf file is #lock_method = fcntl).

lock_method is for Dovecot's indexes. mbox_lock_read and mbox_lock_write are more important. But they default to dotlock + fcntl too.

...

Yes, users can hit their mailboxes by other means. The IMAP server (dovecot) is also the machine that people can login to interactively. System mailboxes are NFS mounted from the mail server, which people cannot login to. Pine users on IMAP server typically hit their mailbox directly via the NFS mount, but this can be easily changed so they go thru dovecot IMAP. The mail server is also the POP server, running qpopper (I should probably change to dovecot there). People are admonished to use either POP or IMAP but not both.

So you're running lockd too to get fcntl locking working for NFS I guess? Do you know if Pine uses fcntl locks or just dotlocks? Perhaps try adding dotlock to mbox_read_locks as well to see if the problems go away?

...

The reasons for this setup are historical and security-related. The IMAP and interactive login server has to remain different from the mail/POP server and the NFS mount must remain. Other than that, I am open to suggestions for improvement here. The two things I should do are:

make pine users go thru IMAP

change from qpopper POP to dovecot POP on the mail server

Dovecot should work fine with other software, as long as they all use the exact same mbox locking methods.

6977

Age (days ago)

6980

Last active (days ago)

List overview

6 comments

3 participants

participants (3)

Andrew Hutchings
Jeff A. Earickson
Timo Sirainen