- dovecot - dovecot.org

[Dovecot] Weird internaldate behavior with 1.0-test80 and mbox folder
by Jon Roma 03 Sep '05

03 Sep '05

Has anyone else encountered weird date behavior with mbox-style folders? I am nearly ready to bite the bullet and convert my UW imap hierarchy (which is in mbx format) to Dovecot. In so doing, I need to convert each UW-specific mbx-format mailboxes to the more portable mbox format, which Dovecot can read. (This is done with the 'mbxcvt' utility that's part of UW.) I am then planning to copy those mbox-format folders into a staging area in a directory I've defined to Dovecot as a mbox-format namespace. The plan was then to use my normal GUI to reorganize my old data and copy it into proper maildir folders into Dovecot's default namespace. Since I'm naturally paranoid, I did a dry run of a couple folders first and discovered a rather strange anomaly -- all the mail in the converted folder is displayed by Dovecot with a timestamp exactly 200 minutes (3 hours 20 minutes) into the future! Investigating this via telneting to the respective IMAP ports shows that UW imapd shows the correct timestamp both in the original mbx-format folder and the converted mbox-format folder -- so it appears that Dovecot is skewing the time. This doesn't look like a timezone issue since 200 minutes ahead from CST6CDT would be a pretty strange timezone -- not even Newfoundland has this odd a TZ value! :) (For the curious, Newfoundland is a Canadian province whose time zone is TZ=NST3:30NDT, therefore being a half-hour "off" from the rest of North America.) I have distilled this into a degenerate case of a single folder with a single message and can reproduce. As mentioned, I am running 1.0-test80. My platform is AIX 5.1. I created the test by sending a message to myself using date | mail -s "Test at $(date)" roma(a)uiuc.edu and using my MUA (Mulberry) to file it into the test folder, which defaults to mbx format in UW's namespace. Here's what UW shows when I select/fetch the "original" message: * namespace * NAMESPACE (("" "/")("#mhinbox" NIL)("#mh/" "/")) (("~" "/")) (("#shared/" "/")("#ftp/" "/")("#news." ".")("#public/" "/")) * OK NAMESPACE completed * select "~/Test" * 1 EXISTS * 0 RECENT * OK [UIDVALIDITY 1124811965] UID validity status * OK [UIDNEXT 3] Predicted next UID * FLAGS (\Answered \Flagged \Deleted \Draft \Seen) * OK [PERMANENTFLAGS (\* \Answered \Flagged \Deleted \Draft \Seen)] Permanent flags * OK [READ-WRITE] SELECT completed * fetch 1 internaldate * 1 FETCH (INTERNALDATE "23-Aug-2005 11:16:37 -0500") * OK FETCH completed * fetch 1 envelope * 1 FETCH (ENVELOPE ("Tue, 23 Aug 2005 11:16:30 -0500" "Test at Tue Aug 23 11:16:29 CDT 2005" (("Jon Roma" NIL "roma" "zippy.cso.uiuc.edu")) (("Jon Roma" NIL "roma" "zippy.cso.uiuc.edu")) (("Jon Roma" NIL "roma" "zippy.cso.uiuc.edu")) ((NIL NIL "roma" "uiuc.edu")) NIL NIL NIL "<200508231616.j7NGGUrc022618(a)zippy.cso.uiuc.edu>")) * OK FETCH completed Next I use the UW 'mbxcvt' to convert the folder, leaving it in mbox format in a different folder (~/Test-mbox) -- still in UW's namespace. Here's what the telnet connection shows: * namespace * NAMESPACE (("" "/")("#mhinbox" NIL)("#mh/" "/")) (("~" "/")) (("#shared/" "/")("#ftp/" "/")("#news." ".")("#public/" "/")) * OK NAMESPACE completed * select "~/Test-mbox" * 1 EXISTS * 1 RECENT * OK [UIDVALIDITY 1124813840] UID validity status * OK [UIDNEXT 2] Predicted next UID * FLAGS (\Answered \Flagged \Deleted \Draft \Seen) * OK [PERMANENTFLAGS (\* \Answered \Flagged \Deleted \Draft \Seen)] Permanent flags * OK [READ-WRITE] SELECT completed * fetch 1 internaldate * 1 FETCH (INTERNALDATE "23-Aug-2005 11:16:37 -0500") * OK FETCH completed * fetch 1 envelope * 1 FETCH (ENVELOPE ("Tue, 23 Aug 2005 11:16:30 -0500" "Test at Tue Aug 23 11:16:29 CDT 2005" (("Jon Roma" NIL "roma" "zippy.cso.uiuc.edu")) (("Jon Roma" NIL "roma" "zippy.cso.uiuc.edu")) (("Jon Roma" NIL "roma" "zippy.cso.uiuc.edu")) ((NIL NIL "roma" "uiuc.edu")) NIL NIL NIL "<200508231616.j7NGGUrc022618(a)zippy.cso.uiuc.edu>")) * OK FETCH completed Now, copying the same mbox-format folder into dovecot's namespace and telneting to my Dovecot port gives the following: * namespace * NAMESPACE (("" ".")("Stage/" "/")) NIL NIL * OK Namespace completed. * select "Stage/Test" * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS * 1 RECENT * OK [UIDVALIDITY 1124815188] UIDs valid * OK [UIDNEXT 2] Predicted next UID * OK [READ-WRITE] Select completed. * fetch 1 internaldate * 1 FETCH (INTERNALDATE "23-Aug-2005 14:36:37 -0500") * OK Fetch completed. * fetch 1 envelope * 1 FETCH (ENVELOPE ("Tue, 23 Aug 2005 11:16:30 -0500" "Test at Tue Aug 23 11:16:29 CDT 2005" (("Jon Roma" NIL "roma" "zippy.cso.uiuc.edu")) (("Jon Roma" NIL "roma" "zippy.cso.uiuc.edu")) (("Jon Roma" NIL "roma" "zippy.cso.uiuc.edu")) ((NIL NIL "roma" "uiuc.edu")) NIL NIL NIL "<200508231616.j7NGGUrc022618(a)zippy.cso.uiuc.edu>")) * OK Fetch completed. Notice the "internaldate" response. Dovecot reports the correct time zone, but the internaldate value is 200 minutes ahead of what it should be!!! This is an interesting mystery; I seem to be living three hours and 20 minutes in the future. I've attached the entire contents of the mbox-style mailbox that Dovecot misinterprets. Thanks in advance. >From MAILER-DAEMON Tue Aug 23 11:39:48 2005Date: 23 Aug 2005 11:39:48 -0500 From: Mail System Internal Data <MAILER-DAEMON(a)zippy.cso.uiuc.edu> Subject: DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA X-IMAP: 1124815188 0000000001 Status: RO This text is part of the internal format of your mail folder, and is not a real message. It is created automatically by the mail system software. If deleted, important folder data will be lost, and it will be re-created with the data reset to initial values. >From roma(a)zippy.cso.uiuc.edu Tue Aug 23 11:16:37 2005 -0500 Status: RO X-Status: X-Keywords: Received: via dmail-4.1(10) for +INBOX; Tue, 23 Aug 2005 11:16:37 -0500 (CDT) Return-Path: <roma(a)zippy.cso.uiuc.edu> Received: from zippy.cso.uiuc.edu (loopback [127.0.0.1]) by zippy.cso.uiuc.edu (8.12.10/8.12.10) with ESMTP id j7NGGUCQ027294 for <roma(a)uiuc.edu>; Tue, 23 Aug 2005 11:16:30 -0500 Received: (from roma@localhost) by zippy.cso.uiuc.edu (8.12.10/8.12.10/Submit) id j7NGGUrc022618 for roma(a)uiuc.edu; Tue, 23 Aug 2005 11:16:30 -0500 Date: Tue, 23 Aug 2005 11:16:30 -0500 From: Jon Roma <roma(a)zippy.cso.uiuc.edu> Message-Id: <200508231616.j7NGGUrc022618(a)zippy.cso.uiuc.edu> To: roma(a)uiuc.edu Subject: Test at Tue Aug 23 11:16:29 CDT 2005 X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on zippy.cso.uiuc.edu X-Spam-Level: X-Spam-Status: No, score=-5.9 required=2.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.1 X-UID: 1 Tue Aug 23 11:16:29 CDT 2005

2 2

[Dovecot] (no subject)
by Phillip Needham 02 Sep '05

02 Sep '05

I am trying to get away from courier imap because it is flaky the way it uses FAM (or gamin in my case), and the developers seem to have a chip on their collective shoulder, and it is too complex. I found dovecot because it is ships with CentOS 4 (ie Red Hat Enterprise 4). It looks like just what I need: simple, fast. But I can't use it! I can't authenticate. If any of these statements were untrue, I would use it happily: o it doesn't support simple bind authentication against LDAP o I don't want to make passwords readable, and this version (0.99.14) doesn't support SSHA anyway o it won't talk to the saslauthd I already have configured o I can't find how to implement a new auth module for dovecot I am wondering why, in the quest to fill a niche with a simple, fast, secure IMAP server, the authors felt the need to re-implement a robust and well-tested infrastructure (saslauthd) with a less-featureful and untested one?

3 3

[Dovecot] Dovecot imap with Postfix and using Thunderbird as client
by Ben Ware 02 Sep '05

02 Sep '05

Hi, I just recently installed dovecot, and I am having the following problem: If I send an email to my account on this new machine, I can log in via SSH and use the unix mail command, and instantly my email is visible. If I turn on Thunderbird, it will show my new message. If Thunderbird was already on and had already checked the mailbox, even if I click on get new mail for that specific mailbox, the new mail does not show up until I restart thunderbird. Is there something that I have set wrong in the config files that is preventing the IMAP part of dovecot from refreshing the messages quickly for me to see? I dont see anything in the log files at all except for it showing me logging in. Thanks, Ben

2 1

[Dovecot] 'Bind'-method authentication for LDAP
by Phillip Needham 02 Sep '05

02 Sep '05

Has there been any thought given to implementing authentication via an LDAP 'bind', rather than by a password lookup and hash comparison? doing a bind would obviate the need to make the passwords readable by the dovecot process and would lend itself to improved security. Phillip Needham Principal, iBright Consulting phillip(a)ibright.net 614-783-3301

2 1

[Dovecot] another assert/core with 1.0alpha1
by Jeff A. Earickson 01 Sep '05

01 Sep '05

Same as reported before: Assert: IMAP(user): file mbox-sync-update.c: line 442 (mbox_sync_update_header_from): assertion failed: (ctx->mail.uid == 0 || ctx->mail.uid_broken || ctx->mail.uid == mail->uid) Setup: 1.0alpha1 on Solaris 9, built with gcc 4.0.1 Debug output of the core is attached. Jeff Earickson Colby College

1 0

[Dovecot] Dovecot version for production server
by Andrew Velikoredchanin 01 Sep '05

01 Sep '05

Hi! What version of dovecot I can secure install in production server? -- Best regards, Andrew Velikoredchanin

1 0

Re: [Dovecot] Oddities with Debian unstable dovecot
by Lisandro Grullon 31 Aug '05

31 Aug '05

whaooo that is interesting, Thank you so much for the explanation. I guess my choice now is to go with maildir as Blairpoint out because I am running in single host mode. Probably in the future I will run a big enough enviroment as a cluster data center them I will op for mbox. Thank you again Blair and Tomi for educating me in the topic.Lisandro ----------------------------------------------------------------------------------------------------------------------- Lisandro Grullon New York City College of Technology Division of Continuing Education Director of Network Operations Lisandro Office: 1718-552-1178 Lisandro E-mail: lgrullon(a)citytech.cuny.edu "Unix is user-friendly. It's just very selective about who its friends are." >>> Tomi Hakala <tomi.hakala(a)clinet.fi> 08/31/05 3:08 PM >>> Lisandro Grullon wrote: > What method for mail boxes are you guys using mbox or maildir and which one > would you recommend? Thanks. I'd say it depends, I'm working with clustered filesystems in SAN environments where mbox is significantly faster than maildir since single mbox file generates much less overhead in form of distributed file locking accross cluster nodes. On single host systems with good file system I'd probably go with maildir. Maybe someday I'll manage to do some benchmarking between mbox and maildir with Dovecot. -- Tomi Hakala

1 0

Re: [Dovecot] Oddities with Debian unstable dovecot
by Lisandro Grullon 31 Aug '05

31 Aug '05

What method for mail boxes are you guys using mbox or maildir and which one would you recommend? Thanks. ----------------------------------------------------------------------------------------------------------------------- Lisandro Grullon New York City College of Technology Division of Continuing Education Director of Network Operations Lisandro Office: 1718-552-1178 Lisandro E-mail: lgrullon(a)citytech.cuny.edu "Unix is user-friendly. It's just very selective about who its friends are." >>> Blair Zajac <blair(a)orcaware.com> 08/31/05 12:43 PM >>> Tomi Hakala wrote: > Blair Zajac wrote: > >>dovecot: Aug 30 13:14:06 Error: IMAP(ashley): Corrupted index cache file >>/home/ashley/Maildir/dovecot.index.cache: record points outside file > > > Please try lastest CVS version, there are fixes for indexing > code which resolves this problem. > Thanks. I'll give that a shot. Regards, Blair

2 1

Re: [Dovecot] Oddities with Debian unstable dovecot
by Blair Zajac 31 Aug '05

31 Aug '05

Lisandro Grullon wrote: > What method for mail boxes are you guys using mbox or maildir and which one would you recommend? Thanks. I use maildir because it stores each message in a separate file. So this gets me: 1) Daily backups of a ~/Maildir are much smaller than a large mbox because most files don't change, while a single change to a large mbox file will require the entire mbox to be backed up. 2) Moving emails from folder to folder is much faster, as it's moving a single file and you don't have to delete an email from a middle of a mbox file. I have a 5 year old 450 Mhz Dell that easily and quickly manages over 2 Gbytes of email for several users. Regards, Blair

1 0

[Dovecot] Oddities with Debian unstable dovecot
by Blair Zajac 31 Aug '05

31 Aug '05

Hello, I'm running an Debian etch box with unstable and ever since the last upgrade to dovecot, I've seen odd things. At times, I would see emails would go missing from the Inbox or other folders, and I couldn't move emails that were moved from the Inbox to another folder back into the Inbox. These have seen gone away since I removed all dovecot.* files and .index* files from my Maildir's and let Dovecot regenerate them as needed. The most current thing are these errors: dovecot: Aug 30 11:38:33 Error: IMAP(blair): file mail-transaction-log-view.c: l ine 183 (mail_transaction_log_view_set): assertion failed: (max_file_offset == ( uoff_t)-1 || max_file_offset <= file->sync_offset) dovecot: Aug 30 13:05:21 Info: imap-login: Login: ashley [127.0.0.1] dovecot: Aug 30 13:14:06 Error: IMAP(ashley): Corrupted index cache file /home/ashley/Maildir/dovecot.index.cache: record points outside file Anything I can do to help here? I don't have a core file, even though I have 'mail_drop_priv_before_exec = yes'. The other odd thing I noticed is after running /etc/init.d/dovecot stop, I would have several imap processes left over in my uid. They don't do much: # strace -p 5014 Process 5014 attached - interrupt to quit gettimeofday({1125460345, 997748}, {420, 0}) = 0 gettimeofday({1125460345, 998465}, NULL) = 0 poll([{fd=8, events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}, {fd=0, events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}], 2, 410) = 0 gettimeofday({1125460346, 410768}, {420, 0}) = 0 gettimeofday({1125460346, 410890}, NULL) = 0 poll([{fd=8, events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}, {fd=0, events=POLLIN|POLLPRI|POLLERR|POLLHUP|POLLNVAL}], 2, 0) = 0 gettimeofday({1125460346, 411940}, {420, 0}) = 0 gettimeofday({1125460346, 412551}, NULL) = 0 However, lsof shows that the file descriptor 8 is a pipe back to fd 9 in the same process. Why would poll() be polling it's own file descriptor? Is this normal? Is this process waiting to do a fork later on and communicate with a child? # lsof -p 5014 COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME imap 5014 blair cwd DIR 8,4 4096 6619138 /home/blair imap 5014 blair rtd DIR 8,4 4096 2 / imap 5014 blair txt REG 8,4 420168 4572306 /usr/lib/dovecot/imap imap 5014 blair mem REG 0,0 0 [heap] (stat: No such file or directory) imap 5014 blair mem REG 8,4 1178624 6636723 /home/blair/Maildir/dovecot.index.cache imap 5014 blair mem REG 8,4 20148 6636742 /home/blair/Maildir/dovecot.index imap 5014 blair mem REG 8,4 1265136 6996193 /lib/tls/libc-2.3.5.so imap 5014 blair mem REG 8,4 9592 6996195 /lib/tls/libdl-2.3.5.so imap 5014 blair mem REG 8,4 21568 6636721 /home/blair/Maildir/dovecot.index.log imap 5014 blair mem REG 8,4 88924 6995972 /lib/ld-2.3.5.so imap 5014 blair 0u IPv4 160155 TCP localhost.localdomain:imap2->localhost.localdomain:47852 (ESTABLISHED) imap 5014 blair 1u IPv4 160155 TCP localhost.localdomain:imap2->localhost.localdomain:47852 (ESTABLISHED) imap 5014 blair 2w FIFO 0,5 160165 pipe imap 5014 blair 3r CHR 1,9 16396 /dev/urandom imap 5014 blair 4u REG 8,4 1178624 6636723 /home/blair/Maildir/dovecot.index.cache imap 5014 blair 5r DIR 8,4 77824 6930433 /home/blair/Maildir/new imap 5014 blair 6r DIR 8,4 118784 6900850 /home/blair/Maildir/cur imap 5014 blair 8r FIFO 0,5 160244 pipe imap 5014 blair 9w FIFO 0,5 160244 pipe imap 5014 blair 10u REG 8,4 20148 6636742 /home/blair/Maildir/dovecot.index imap 5014 blair 11u REG 8,4 21568 6636721 /home/blair/Maildir/dovecot.index.log Regards, Blair -- Blair Zajac, Ph.D. <blair(a)orcaware.com> Subversion and Orca training and consulting http://www.orcaware.com/svn/

2 2