November 2006 - dovecot

[Dovecot] Problem/workaround with X-Keywords and Apple's Mail.app
by Joey Coleman 17 Feb '07

17 Feb '07

Hello, I've just switched from UW-IMAP to Dovecot, and for the most part, things are good. However, I encountered the problem with the X-Keywords header not being written back into individual messages in an mbox file (originally described in [1], and Timo gave a clue as to maybe why in [2]). What was happening for me is that Apple's Mail.app would flag a message as spam, which happily sets a couple keywords in the message, but those weren't being written back (at least, I didn't see them in the file on the server). The "DO NOT DELETE..." message had a perfectly fine X-IMAP header, and up until I switched things were working great. The user visible symptom of this happened every time new spam arrived in my Junk folder [3] --- the new spam would arrive, be flagged as spam by Mail.app, and the earlier pieces of mail would promptly be unflagged! Anyway, Timo's clue in [2] prompted me to turn off both mbox_lazy_writes and mbox_dirty_syncs (the latter just on the off- chance... it's a low-volume server, so I can deal with the inefficiency). Lo and behold, Mail.app started acting sane again *and* the X-Keywords header appeared immediately. Oddly, during all of this, the Status and X-UID keywords would appear in the messages just fine; no clue as to why those and not X- Keywords. Version details are: - Dovecot 1.0rc13 [4] - FreeBSD 5.3-RELEASE - Apple Mail.app 2.1 (752/752.2) cheers, --joey [1] http://dovecot.org/list/dovecot/2006-March/011872.html [2] http://dovecot.org/list/dovecot/2006-June/014126.html [3] And it felt really strange while troubleshooting this as I found myself *hoping* for another piece of spam to arrive. Bleh. [4] Yes, two versions behind current, but in my defense, I did check the announcements and they didn't seem to mention the problem.

2 1

[Dovecot] Shared folders plans for the future
by Timo Sirainen 11 Feb '07

11 Feb '07

So while driving for several hours in my car recently and having nothing else to do, I started thinking how they could be done. I think the "public folders" configuration will stay as it is now. Does it really even need anything else? The problematic case is how the users can share their mailboxes to other users. There are two problems related to it: 1) How to get (quickly) a list of another user's mailboxes that I have access to? 2) How to get quickly a list of all users who have mailboxes that I have access to? The 1) problem is easier to solve. Create a directory listing all the mailboxes that are shared to at least someone (have ACLs set). It could be configured like: namespace shared { # IMAP namespace prefix, %u expands to shared user prefix = user/%u/ # Directory containing symlinks to user's all shared mailboxes share_dir = ~/Maildir/shared # Where to store indexes for other users' accessed mailboxes index_dir = ~/Maildir/others-indexes } The ~/Maildir/shared directory could then contain symlinks to all the shared mailboxes. The directory list could probably be in Maildir++ layout even though the symlinks would point to mboxes. This requires adding code that can handle mixed maildir/mbox directories, but it'll come anyway.. The 2) problem is more difficult. There are 3 choices: a) Use a shared database file. lib-dict could be used for this, so the database could be in SQL or in a berkeley db file. The downside to this is that the file needs to be readable and writable to everyone, so if users have shell access the users could break the file. They dictionary would contain data in format "<destination_user>/<sharing_user>" = 1 and "<destination_group>/<sharing_user>" = 1, ie. the value would be ignored and only the existence of the keys would be important. Then iterating through "<my_username>/*" and "<all_my_groups>/*" would list the users who probably have mailboxes where I have access to. b) Use a directories with sticky bit set. Well, basically the same idea as in a), except filesystem would be used. Actually perhaps this could be made another lib-dict backend.. c) Brute force: Just assume everyone potentially have shared mailboxes. Get the list of users by readdir()ing through /home (configurable). I guess each of these could be made an option.. So something like: namespace shared { .. # a) shared_dict = db:/var/lib/dovecot/shared-mailboxes.db # b) shared_dict = dir:/var/lib/dovecot/shared-mailboxes # c) shared_dict = bruteforce:/home } I'm currently also implementing a "mailbox list index", which contains a list of all the user's mailboxes and some useful metadata for them, such as number of messages and so on. So that STATUS command wouldn't need to open the mailbox index files if they haven't changed. The mailbox list index updating is normally done when directory's mtime has changed. For list of shared users that would also work in 2b) and 2c) cases, but for 2a) I'd either have to always sync everything or alternatively add some sequence/<dest_user/group>=# value which is grown for each update, and the syncing is done only when the sequence is updated.

6 17

[Dovecot] duplicate emails
by Alex 07 Feb '07

07 Feb '07

i'm working with dovecot-1.0.rc15 I have receive a lot of duplicate emails with microsoft outlook, i don't have this problem with outlook express This is in pop3

10 36

[Dovecot] beta8: core, signal 11
by Jeff A. Earickson 05 Feb '07

05 Feb '07

Timo, The gdb analysis is attached. It looks like memory got seriously stomped in this core dump. There was no assert or other useful information in the syslog. My setup: Solaris 9, imap and imaps, mbox format, INBOXes NFS mounted, index files on local ufs filesystem, beta8 configured like so: VERSION=1.0.beta8 CC=gcc CFLAGS="-g -O" CPPFLAGS=-I/opt/openssl/include LDFLAGS=-L/opt/openssl/lib \ ./configure --prefix=/opt/dovecot.$VERSION \ --with-ssl=openssl \ --with-ssldir=/opt/openssl/ssl \ --with-sql-drivers=no \ --disable-ipv6 \ > configure.out Jeff Earickson Colby College

6 9

[Dovecot] Connection to server time out when deleting (moving to Trash)
by Anders Lund 19 Jan '07

19 Jan '07

Hi, Running: * Dovecot 1.0 rc15 * Debian Sarge Linux Using Maildir and IMAP over SSL. When trying to delete a lot of mail (> 5000-6000 messages) in some folder, using Mozilla Thunderbird (1.5.0.7) on Ubuntu, I often experience "Connection to server <hostname> timed out." Any suggestions on how to fix this problem? "Empty Trash" seem to work fine, so I guess this is related to moving a lot of mail from one folder to another (Thrash)? - Anders -- Anders Lund <Anders.Lund(a)uninett.no> .~. UNINETT, N-7465 Trondheim, Norway / V \ Phone: +47 73 55 79 08 | Fax: +47 73 55 79 01 /( )\ ^ ^

4 10

[Dovecot] Index corruption causes child process to die
by Juha Saarinen 16 Jan '07

16 Jan '07

My recent migration from Courier-IMAP to Dovecot has been marred by these errors: Oct 24 12:12:40 server dovecot: IMAP(juha): Fixed index file ~user/Maildir/.Geekzone/dovecot.index: first_recent_uid_lowwater 431 -> 430 Oct 24 12:12:40 server dovecot: IMAP(juha): Corrupted index file ~user/Maildir/.Geekzone/dovecot.index: Duplicate header extension keywords Oct 24 12:12:40 server dovecot: child 5786 (imap) killed with signal 11 They happen with Thunderbird 1.5.0.7 and I'm using Dovecot 1.0RC7 from the FreeBSD ports. I think the index corruption may be caused by Tbird's message filtering - I'm asking it to filter certain messages from INBOX to subfolders. When the error occurs, Tbird pops up an error message saying it can't connect to the mail server, but I don't lose any messages it seems. Instead, I get duplicates and even triplicates when I restart Tbird. Any work around for this? -- Juha http://www.geekzone.co.nz/juha

7 35

[Dovecot] dovecot rc7 assert, core
by Jeff A. Earickson 09 Jan '07

09 Jan '07

Hi, Wow, I haven't had an assert/core in dovecot in a long time. Here goes: Setup: Solaris 10, dovecot built with gcc 4.1.1, imap and imaps only. User's home directory lives on a ZFS filesystem, but I don't think that makes a difference. So does every other user here (>3K). Assert: IMAP(mulepri): file message-body-search.c: line 393 (message_body_search_ctx): assertion failed: (input->v_offset <= part->physical_pos) gdb backtrace and other poking at the core is attached. Jeff Earickson Colby College

2 1

[Dovecot] NFS
by Timo Sirainen 23 Dec '06

23 Dec '06

I've finally managed to run Dovecot without errors in two computers with maildir and indexes stored in NFS. I added a page to wiki about this: http://wiki.dovecot.org/NFS Suggestions how to keep attribute cache enabled but to allow Dovecot to specifically request not to use a cached value (when it's important) would be welcome.

4 7

[Dovecot] considering dovecot
by Rick Hazey 15 Dec '06

15 Dec '06

Hi all. I'm making a case to use Dovecot over Courier and would like some feedback. I've searched the web extensively and have been unable to find any hard information regarding Dovecot's IMAP performance. I've got some pressure to use Courier but my gut tells me that Dovecot should be much faster. In particular, I will have typical users with mailboxes exceeding 10,000 emails and in some cases up to 100,000. Courier might be the "safe" choice but I'm not confident it can handle inboxes with that many messages. Dovecot, with its indexes, should be much, much faster. Can anyone point me to some benchmarks or production uses of dovecot that bear this out? From my research, Courier maildrop appears to be what I need for local delivery. Is it possible to use Maildrop for local delivery with Dovecot? Or is the Dovecot LDA needed to update the maildir indexes? I also need the capability for users to set their own vacation messages via a web page. Maildrop looks to handle this easily (by allowing me to link the maildrop filter to a file with the vacation message) but I don't see similar capability with the Sieve plugin. I do see where I can create a vacation message with Sieve but the text of the message seems to be required to reside within the Sieve script. Perhaps I'm missing something? Any feedback would be greatly appreciated.

12 21

[Dovecot] rc15 dumped core
by Hans Morten Kind 07 Dec '06

07 Dec '06

We have been running rc15 since Sunday experiencing no trouble, but today one user crashed. rc15 on Solaris-5.9, the server had app 6GB free memory at the time of the crash, app 70% idle running with an avg.load at 4 of 8 processors available. The USER's INBOX was quite smooth, app 13MB containing 320 messages. The core dump was accompanied with these log-entries: Nov 23 2006 12:27:00 [local3.error] IMAP(USER): Corrupted index cache file imapindex/USER/.imap/INBOX/dovecot.index.cache: invalid field header size 12:28:23 [local3.error] IMAP(USER): mremap_anon(1164206080) failed: Not enough space 12:28:23 [local3.error] IMAP(USER): mremap_anon(8192) failed: Invalid argument 12:28:24 [local3.error] child 17510 (imap) killed with signal 10 12:28:25 [local3.info] imap-login: Login: user=<USER>, method=PLAIN, rip=000.177.00.21, pid=19178 We have been observing a few of the 'invalid field header size' messages while running rc15. But as this USER, others seem to recover after the message has been logged. The messages from 'mremap_anon' have never been seen before. Actually '1164206080' looks very much like a timestamp, differing 75223 seconds from the time of the crash ... spooky I will start running the supplied code for memory debugging on this user and I enclose a back trace from the crash. The interessting stuff from the back trace is probably that hdr (and so cache->hdr) is 0xffffffff so that hdr->continued_record_count gives no sense (Cannot access memory at address 0xb) but cache->index->map->records_count is 320 Still on this and another server running rc15 (Sol5.8), we have thousands of happy users beeing served very fast by Dovecot. hmk

2 5