[Dovecot] Broken Header uidvalidity problem after upgrade
Hi, we use a current Debian etch distribution as basis for our servers, which are running within a vmware server environment. Our current configuration consists of a dovecot server with imap enabled and a ldap based authentication. Mails are stored using the Maildir format on a central nfs store. We do not use any virtual mailboxes.
Since upgrading from dovecot version 1.0.rc15 (debian stable) to 1.0.13 (debian etch-backport) we got the following error logs.
Excerpt from mail.err: Apr 21 12:04:38 servera deliver(usera): /var/mail/Maildir/usera/dovecot-uidlist: Broken header (uidvalidity = 0, next_uid=9) Apr 21 15:37:50 servera deliver(userb): /var/mail/Maildir/userb/dovecot-uidlist: Broken header (uidvalidity = 0, next_uid=16) Apr 23 10:37:17 servera dovecot: IMAP(userc): /var/mail/Maildir/userc/.Sent/dovecot-uidlist: Broken header (uidvalidity = 0, next_uid=2) Apr 25 17:00:17 servera dovecot: IMAP(userd): /var/mail/Maildir/userd/dovecot-uidlist: Broken header (uidvalidity = 0, next_uid=8)
I did already some mailinglist reviews. They confirmed in some way the problem, but there is still no explanation where it comes from, and how it can be solved or if it may affect the proper operation of the system.
Here are some links with a short comment, which are related to this problem: http://www.dovecot.org/list/dovecot/2007-September/025541.html This post also uses nfs, but pam for authentication, error message is exactly the same as in our case.
http://www.dovecot.org/list/dovecot/2007-July/024001.html Version upgrade from 1.0.0 to one above, also problems with uidvalidity. They described, that it happens only once for each user after upgrading. This is similar to our observations. No user complains as well ;). So it would be only of interest if it is so.
http://wiki.dovecot.org/Clients/NegativeUIDs The number for uidvalidity is zero, so it is non-negative. But maybe it is correlated with these hints.
From my point of view, there could be some problems with time synchronization as well between nfs server and dovecot server machine. But we checked this and they differ never more than 2 seconds.
In total we are running two servers in parallel, each has it's own userspace, all accounts are strictly separated. One server is running the rc15 version, the other (more experimental) is running the backport 1.0.13. The described problem occurs only on the 1.0.13 server, which was previously updated. All other configurations are exaclty the same.
Any hints?
Best regards, Robert
-- Dipl.-Inform. Robert Henjes University of Wuerzburg, Institute of Computer Science, Department of Distributed Systems (Informatik III), Am Hubland, 97074 Wuerzburg, Germany
henjes@informatik.uni-wuerzburg.de Tel: +49 931 888-6652 Fax: +49 931 888-6632
On Apr 29, 2008, at 12:19 PM, Robert Henjes wrote:
Excerpt from mail.err: Apr 21 12:04:38 servera deliver(usera): /var/mail/Maildir/usera/ dovecot-uidlist: Broken header (uidvalidity = 0, next_uid=9) Apr 21 15:37:50 servera deliver(userb): /var/mail/Maildir/userb/ dovecot-uidlist: Broken header (uidvalidity = 0, next_uid=16)
To me these look like the older version just hadn't noticed broken
uidvalidity values. Newer version has sanity checks and fixes the
problem. Or does the error message happen more than once for the same
mailbox?
I think some versions of my Courier migration script could have
written zero uidvalidity values.
Timo Sirainen schrieb:
On Apr 29, 2008, at 12:19 PM, Robert Henjes wrote:
Excerpt from mail.err: Apr 21 12:04:38 servera deliver(usera): /var/mail/Maildir/usera/dovecot-uidlist: Broken header (uidvalidity = 0, next_uid=9) Apr 21 15:37:50 servera deliver(userb): /var/mail/Maildir/userb/dovecot-uidlist: Broken header (uidvalidity = 0, next_uid=16)
To me these look like the older version just hadn't noticed broken uidvalidity values. Newer version has sanity checks and fixes the problem. Or does the error message happen more than once for the same mailbox?
I think some versions of my Courier migration script could have written zero uidvalidity values.
Ok I think you're right. Your explanation does completley match our history of installing dovecot.
We installed the old dovecot version and converted the mails using mb2md (debian etch version 3.20) from mbox format to maildir format. Formerly we used uw-imap as mailserver, which seems after a short review not to care about this uidvalidity property at all. So after upgrading from 1.0.rc15 to 1.0.13 these sanity checks detected the problem and corrected them. So as observed until now, this happens only once for certain mailbox users either during deliver or while accessing using imap. But it doesn't happen necessarily to each mailbox user.
Is there a possibility to check the uidvalidity from administrators point of view on the server?
On Tue, 2008-04-29 at 17:10 +0200, Robert Henjes wrote:
Is there a possibility to check the uidvalidity from administrators point of view on the server?
Something like:
find . -name dovecot-uidlist|xargs head -1|grep '^1 0 '
The point is just that UIDVALIDITY must not be zero. Dovecot relies on this internally in some places, and also it's illegal to send such UIDVALIDITY using IMAP protocol. Strange that it worked with you before. :)
Timo Sirainen schrieb:
On Tue, 2008-04-29 at 17:10 +0200, Robert Henjes wrote:
Is there a possibility to check the uidvalidity from administrators point of view on the server?
Something like:
find . -name dovecot-uidlist|xargs head -1|grep '^1 0 '
The point is just that UIDVALIDITY must not be zero. Dovecot relies on this internally in some places, and also it's illegal to send such UIDVALIDITY using IMAP protocol. Strange that it worked with you before. :)
So after a while of testing and playing around I figured out the following:
If you create a new and empty directory with version 1.0.rc15 then there is no dovecot-uidlist file. After moving a new mail to this folder one is created
1 0 2 1 1209486091.P3655Q5.serverb:2,S
Moving another mail to this dir leads to:
1 0 3 1 1209486091.P3655Q5.serverb:2,S 2 1209486279.P3655Q6.serverb:2,S
tataaaa .. and there is the zero value, which should not be there according to your report. So it seems to be a bug of this debian version. After reviewing all the files of the users on our system, it seems that all new folders created between the migration to 1.0.rc15 and the upgrade to 1.0.13 had this problem. I didn't verify the 1.0.13 version yet due to the lack of time.
Any hints, to solve this problem, besides simply upgrading?
Best regards, Robert
1 0 2 1 1209486091.P3655Q5.serverb:2,S
Moving another mail to this dir leads to:
1 0 3 1 1209486091.P3655Q5.serverb:2,S 2 1209486279.P3655Q6.serverb:2,S
In the mean time I repeated the test with creating a new mailbox (folder) using version 1.0.13 (debian etch-backport). Using this version the uidvalidity seems to be ok. See the ecxcerpt of the log file:
1 1209544994 2 1 1209544994.P18795Q0M587290.servera:2,S
So I think simply upgrading to version 1.0.13 should be the savest way to repair this bug and correct the wrong uids.
Robert-- Dipl.-Inform. Robert Henjes University of Wuerzburg, Institute of Computer Science, Department of Distributed Systems (Informatik III), Am Hubland, 97074 Wuerzburg, Germany
henjes@informatik.uni-wuerzburg.de Tel: +49 931 888-6652 Fax: +49 931 888-6632
participants (2)
-
Robert Henjes
-
Timo Sirainen