- dovecot - dovecot.org

[Dovecot] Fix for the kevent "Unrecognized event" problem.
by Václav Haisman 11 May '06

11 May '06

The attached patch should fix the problem with dying imap on "Unrecognized event". The problem is that when we register a handle for IO_ERROR only, we still can get readable/writable event without EV_EOF being set. This case was not handled. -- Vaclav Haisman *** ioloop-kqueue.c.~1.4.~ Tue May 2 14:36:56 2006 --- ioloop-kqueue.c Tue May 9 21:29:33 2006 *************** *** 74,80 **** { struct ioloop_handler_context *ctx = ioloop->handler_context; const int fd = io->fd; ! struct kevent ev = { fd, 0, EV_ADD | EV_EOF, 0, 0, NULL }; enum io_condition condition = io->condition & MASK; i_assert(io->callback != NULL); --- 74,80 ---- { struct ioloop_handler_context *ctx = ioloop->handler_context; const int fd = io->fd; ! struct kevent ev = { fd, 0, EV_ADD, 0, 0, NULL }; enum io_condition condition = io->condition & MASK; i_assert(io->callback != NULL); *************** *** 184,190 **** i_assert(ctx->evbuf[i].ident < ctx->fds_size); if ((ctx->fds[ctx->evbuf[i].ident].mode & IO_ERROR) && ! (ctx->evbuf[i].flags & EV_EOF)) { struct io *errio = ctx->fds[ctx->evbuf[i].ident].errio; t_id = t_push(); --- 184,190 ---- i_assert(ctx->evbuf[i].ident < ctx->fds_size); if ((ctx->fds[ctx->evbuf[i].ident].mode & IO_ERROR) && ! (ctx->evbuf[i].flags & (EV_EOF | EV_ERROR))) { struct io *errio = ctx->fds[ctx->evbuf[i].ident].errio; t_id = t_push(); *************** *** 203,216 **** " in I/O handler %p", (void *)io->callback); } } else i_panic("Unrecognized event: kevent {.ident = %u," " .filter = 0x%04x," " .flags = 0x%04x," " .fflags = 0x%08x," ! " .data = 0x%08x}", ctx->evbuf[i].ident, ctx->evbuf[i].filter, ctx->evbuf[i].flags, ! ctx->evbuf[i].fflags, ctx->evbuf[i].data); } } --- 203,224 ---- " in I/O handler %p", (void *)io->callback); } + } else if (ctx->fds[ctx->evbuf[i].ident].mode & IO_ERROR) { + /* + NO-OP. If the handle is registered only for + IO_ERROR, then we can get readable/writable event + but no IO_READ | IO_WRITE set. + */ } else i_panic("Unrecognized event: kevent {.ident = %u," " .filter = 0x%04x," " .flags = 0x%04x," " .fflags = 0x%08x," ! " .data = 0x%08x}\n" ! "mode: 0x%x04x", ctx->evbuf[i].ident, ctx->evbuf[i].filter, ctx->evbuf[i].flags, ! ctx->evbuf[i].fflags, ctx->evbuf[i].data, ! ctx->fds[ctx->evbuf[i].ident].mode); } }

3 6

[Dovecot] IMAP threading - the THREAD command
by Jan Kundrát 11 May '06

11 May '06

Hi folks, there's an extension [1] to IMAP specifying the THREAD command to return list of message sequence numbers grouped to indicate the threading of them. That draft currently defines two algorithms, one of them based solely on Subjects (which is rather dumb) and the second one, combining Subjects with In-Reply-To (etc) headers. The second method is actually advertised and supported by Dovecot. However, I've got quite a lot of friends that lack the knowledge of what the "Subject" field is for, so they just leave it empty. The problem comes when I want Dovecot to return the threads - it follows the ietf draft, so it doesn't check only for references from message headers, but also the subjects. The result is that especially those mails with empty subjects are incorrectly grouped together. So, I'd like to see a feature that would eliminate this problem. It seems to me that there are currently two methods: a) Don't look at the Subject field at all - altough it might increase the speead a bit, it has some disadvantages as well, namely that you'll lose threading if someone uses broken mail client (but broken only in such a way that it preserves subjects) b) Ignore the Subject field if it is empty (after stripping stuff like "re:" etc - just as that draft specifies). Seems reasonable, IMHO. Of course we can't modify the result of a THREAD command (that would break the standard and it generally isn't a proper way to go), so we'd have to introduce another one. I'm pretty sure that especially method a) is very simple (and even simpler if I decided to break the standard by "fixing" the THREAD command), but I really don't like <rant>portable assembler :)</rant> <politely>C</politely>, so are there any chances for this "new and fixed" command to be included into Dovecot? I mean, I might do that in a flawed and ugly way just for me and post a patch so that possible interested persons can pick it, or I migth persuade someone to do it, actually :). Do you think that it's worth the issue? Does anyone have any experiences with talking to the IETF about possible extension of their draft? Ideas? [1] http://www.ietf.org/internet-drafts/draft-ietf-imapext-sort-17.txt Cheers, -jkt -- cd /local/pub && more beer > /dev/mouth

3 4

[Dovecot] Large folders and timeouts
by Fran Fabrizio 11 May '06

11 May '06

I have been having persistent issues with server timeouts on operations on large folders on our Dovecot install (1.0beta2 RPM that came with Fedora Core 5). Some observations from today... I have a Maildir folder with 38,000 messages (it is my Spam folder, naturally :-). In Thunderbird 1.5.0.2, if I click on that folder, it slowly but steadily downloads the headers and after 3-4 minutes, shows me the contents. So far, so good. If I then select all the messages and click Delete to delete them, that operation times out after about 1 minute. If I then click on my Trash folder after some number of minutes has passed, I see that those messages are now both in my Trash and in the original folder - I believe this is because Thunderbird first asks for a COPY over to Trash, and then asks for a STORE +FLAG \\Deleted on the copy that is still in the original folder. It appears that the COPY commands are working but the STORE commands are not since the connection is timing out before they run. I thought I'd try on another mail client, so I tried Squirrelmail. I set it to show 1000 messages per page. It does this just fine. Then I set it to 4,000 messages per page. Then I click on a folder with 12,000 messages, and I get this: ERROR: Connection dropped by IMAP server. Query: SORT (ARRIVAL) ISO-8859-1 ALL My questions are 1. what is the likely cause? and 2. will upgrading to a newer beta address any of this? I'd prefer to wait until FC5 has an updated RPM if possible. Our mail is stored on an NFS store. (I know NFS is slow and that this might be part of the cause.) However, it seems that the timeouts I am having are more related to times when there is no data flowing between the server and client. Note that the timeouts happened when the server was doing a COPY and when the server was doing a SORT but not when the client was downloading 38,000 headers. I'm just at a loss for what to look into next. I don't want to just say "NFS is slow" and live with it if this really has some other cause. :-) So, anyone have any thoughts as to other possible causes? Thunderbird has no option for setting the length of timeout from the client side, so there's nothing I can do from that end. Thanks, Fran -- Fran Fabrizio Senior Systems Analyst Department of Computer and Information Sciences University of Alabama at Birmingham http://www.cis.uab.edu/ 205.934.0653

4 4

[Dovecot] dovecot 1.0beta7 dying on me
by Dave Alden 11 May '06

11 May '06

Hi, Dovecot 1.0 beta7 is dying on me. My logfile shows: May 10 15:54:28 hermes dovecot: IMAP(alden): UIDs broken with partial sync in mbox file /var/mail/alden May 10 15:54:28 hermes dovecot: IMAP(alden): file mbox-sync-rewrite.c: line 106 (mbox_sync_headers_add_space): assertion failed: (start_pos < data_size) May 10 15:54:28 hermes dovecot: child 14185 (imap) killed with signal 6 I'm running this on a RHEL 3U7 system. I created a core file, but it doesn't look like it's too helpful (I've attached the gdb output to this message). I tried recompiling with "--enable-debug", but I got: /usr/lib/mysql/libmysqlclient.a(sha1.o)(.text+0x60): In function `sha1_result': : multiple definition of `sha1_result' ../lib/liblib.a(sha1.o)(.text+0x7f4):/scratch/dovecot-1.0.beta7/src/lib/sha1.c:233: first defined here /usr/bin/ld: Warning: size of symbol `sha1_result' changed from 158 in ../lib/liblib.a(sha1.o) to 126 in ../lib/liblib.a(sha1.o) collect2: ld returned 1 exit status Help? ...dave -- Dave Alden The Ohio State University Department of Mathematics

3 3

[Dovecot] Sieve-Editor
by Andreas Martin 11 May '06

11 May '06

Hi all. I'm looking for an editor for sieve-scripts. I tried the Squirrelmail plugin "avelsieve", but it requires "timsieved", wich Dovecot doesn't use. I don't want my users to write sieve-scripts on the console and server-side-filtering is absolutely required. I know that there are some frontends available, but I can't figure out, which one really works with libsieve and is usable for non-geeks... Do you have any suggestions? Thanks, Andreas

4 5

[Dovecot] pop3 problem with small messages with no Subject: and no To: headers
by Ken A 11 May '06

11 May '06

Hello, Seeing a strange thing with dovecot 1.0 b7 pop3. If a user gets a particular _very_ short spam message (not sure what virus makes these..), with NO Subject and NO To: headers and NO body, (example available upon request), the user is unable to download the mail. The log entry is: May 10 10:51:35 mail dovecot: POP3(user): Disconnected top=0/0, retr=1/1344, del=0/75, size=16383381 There's no QUIT, so no "Logged out" disconnect reason. I did a tcpdump to see what was happening. I have not been able to duplicate this myself yet, even with an exact copy of the message that seems to be causing it, so it could be a client issue, but I've never seen this before, and we just switched to Dovecot, so I thought I'd check with the list.. ideas? Message #9 is requested (it's the one in the user's box with missing headers). Dovecot begins returning it, but then disconnects, or so it seems. 10:51:35.171916 IP customer.60181 > mailserver.pop3: P 40:48(8) ack 1633 win 65535 0x0000: 4500 0030 d326 4000 7106 fc20 4624 16a1 E..0.&@.q...F$.. 0x0010: ceaa 0f11 eb15 006e 7ee7 2a93 d845 fe0c .......n~.*..E.. 0x0020: 5018 ffff 3618 0000 5245 5452 2039 0d0a P...6...RETR.9.. 10:51:35.172177 IP mailserver.pop3 > customer.60181: P 1633:2980(1347) ack 48 win 5840 0x0000: 4500 056b ca0b 4000 4006 3101 ceaa 0f11 E..k..@.@.1..... 0x0010: 4624 16a1 006e eb15 d845 fe0c 7ee7 2a9b F$...n...E..~.*. 0x0020: 5018 16d0 3fde 0000 2b4f 4b20 3133 3237 P...?...+OK.1327 0x0030: 206f 6374 6574 730d 0a52 6574 7572 6e2d .octets..Return- 0x0040: 5061 7468 3a20 3c68 7469 3935 6b67 6576 Path:.<hti95kgev 0x0050: 4072 @r 10:51:35.416549 IP mailserver.pop3 > customer.60181: P 1633:2980(1347) ack 48 win 5840 0x0000: 4500 056b ca0d 4000 4006 30ff ceaa 0f11 E..k..@.@.0..... 0x0010: 4624 16a1 006e eb15 d845 fe0c 7ee7 2a9b F$...n...E..~.*. 0x0020: 5018 16d0 3fde 0000 2b4f 4b20 3133 3237 P...?...+OK.1327 0x0030: 206f 6374 6574 730d 0a52 6574 7572 6e2d .octets..Return- 0x0040: 5061 7468 3a20 3c68 7469 3935 6b67 6576 Path:.<hti95kgev 0x0050: 4072 @r 10:51:35.470688 IP customer.60181 > mailserver.pop3: F 48:48(0) ack 2980 win 64188 0x0000: 4500 0028 d329 4000 7106 fc25 4624 16a1 E..(.)@.q..%F$.. 0x0010: ceaa 0f11 eb15 006e 7ee7 2a9b d846 034f .......n~.*..F.O 0x0020: 5011 fabc 09fa 0000 0000 0000 0000 P............. 10:51:35.470997 IP mailserver.pop3 > customer.60181: F 2980:2980(0) ack 49 win 5840 0x0000: 4500 0028 ca0f 4000 4006 3640 ceaa 0f11 E..(..@.@.6@.... 0x0010: 4624 16a1 006e eb15 d846 034f 7ee7 2a9c F$...n...F.O~.*. 0x0020: 5011 16d0 ede5 0000 P....... 10:51:35.515279 IP customer.60181 > mailserver.pop3: . ack 2981 win 64188 0x0000: 4500 0028 d32e 4000 7106 fc20 4624 16a1 E..(..@.q...F$.. 0x0010: ceaa 0f11 eb15 006e 7ee7 2a9c d846 0350 .......n~.*..F.P 0x0020: 5010 fabc 09f9 0000 0000 0000 0000 P............. That's all, the connection is closed. Ken A Pacific.Net

1 1

[Dovecot] Recommended FS for Dovecot Maildir
by Daniel Watts 10 May '06

10 May '06

Hi, I've heard that for Dovecot/Mailir systems there are filesystems that are optimised for the situation of many small files in one folder. Could I possibly have some feedback on what the recommended filesystems are? I've heard of ReiserFS but was wondering what other options there are and how they compare. If I get a good comprehensive response I'll build a wiki summary page out of the data gathered. Best wishes, Daniel

17 28

[Dovecot] Re: Recommended FS for Dovecot Maildir
by Roger Weeks 10 May '06

10 May '06

> Date: Wed, 10 May 2006 00:24:23 +0100 > From: Daniel Watts <d(a)nielwatts.com> > Subject: Re: [Dovecot] Recommended FS for Dovecot Maildir > To: Les Mikesell <lesmikesell(a)gmail.com> > > Can you remember any general findings? > > Thanks to all for responding in this thread - I am reading them all > and > making notes. All very useful information. > > Can I phrase the question in a different way? Of those of you who are > running 10,000+ user system - what FS do you use? > Or do you all use special file servers (netapps, emc etc?) with > proprietary filesystems? > > Daniel We're probably right at your 10000 user threshold, but we're an ISP so we get a LOT of mail traffic. Our mail system consists of dual exim servers processing incoming mail, writing to maildirs on a home- built NFS system. The home-built system is a dual-xeon machine with lots of SATA disks in a RAID5. We then have two client-facing machines running dovecot and squirrelmail. These are all SuperMicro servers running RedHat ES 4. They all use SATA disks. We are using EXT3 for the RAID5 filesystem, and I have had no performance issues to speak of. I had considered Reiser or xfs, but as I have limited experience with either filesystem I didn't feel comfortable using them in production. -- Roger J. Weeks Systems & Network Administrator Mendocino Community Network

1 0

[Dovecot] Debugging with strace
by Fran Fabrizio 10 May '06

10 May '06

We have a dovecot install that is working fine for everyone except one user, who complains that it takes a minute or longer to delete a single message from the INBOX to Trash. This is happening for him from multiple IMAP clients so it appears to be server-side. I wanted to start up a test instance of dovecot alongside the production one and test there. So I copied dovecot.conf, put imap on port 10143, and started up the test server like this: # strace -tt dovecot -F -c /etc/dovecottest.conf (I could not find the docs for -F but without it dovecot would quickly detatch into a daemon and I could no longer strace it... so is -F for putting dovecot in the foreground like I think? :-) I noticed two things when I did this: 1. My screen just scrolled continuously with poll calls - does dovecot poll something every second even when it has no clients connected? 2. It appeared to interfere with the production install running alongside of it - as I could no longer authenticate to it or copy messages to my Sent folder when I sent something out through the production dovecot. I had to restart the production instance to get it to behave. So my questions are whether all that polling output is normal and also, is my testing method really interfering with the production server, and if so, how to avoid that? Thanks, Fran -- Fran Fabrizio Senior Systems Analyst Department of Computer and Information Sciences University of Alabama at Birmingham http://www.cis.uab.edu/ 205.934.0653

2 2

[Dovecot] moving from postfix+dovecot to postfix+dovecot+postfixadmin+mysql
by albi 10 May '06

10 May '06

hi, i know this is not really a dovecot-question, but (after searching for hours and hours) here goes : last week i've tested a setup with postfixadmin and dovecot, and i'm very happy with the result! it's running on a FreeBSD-server but i've used this howto (which describes the setup for debian) : http://bliki.rimuhosting.com/space/knowledgebase/linux/mail/postfixadmin+on… big question is : how can i convert the already existing passwords from the mail-users to the mysql-based auth in postfixadmin/dovecot ------------------------------------------ thanks for having dovecot available, it's great to work with ! :) -- grtjs, albi gpg-key: lynx -dump http://scii.nl/~albi/gpg.asc | gpg --import

1 0