- dovecot - dovecot.org

[dovecot] 0.99.9-test6
by Timo Sirainen 21 Apr '03

21 Apr '03

Just two fixes: - dovecot-uidlist file is now assumed to be changed if it's modify time is changed. before we relied on inode changing which isn't fully reliable. - mbox dotlocks sometimes incorrectly thought someone else had overridden them Known problems that I should fix: - mbox: we don't necessarily notice message flag changes by external programs (or another dovecot without index files). It's because we don't always reparse the whole mbox file when it has changed. I think I'll have to add some X-IMAP-Changelog header which describes what changes were made by other Dovecot processes. If the change isn't listed in that header, we'll fallback to reparsing the whole file. Although that's still a bit annoying since MTA just appends the new mails, we wouldn't really need to reparse the whole file then, just the new mails. - I should use link() instead of relying on O_EXCL which doesn't work with NFSv2 (or Linux's NFSv3 client). - .subscriptions and .customflags files still aren't NFS-safe

1 0

[dovecot] Folders with mail and subfolders
by Keith Edmunds 21 Apr '03

21 Apr '03

I have a maildir hierarchy (built originally with Courier-IMAP) where some folders contain both email and subfolders. Using Dovecot I cannot see the mail in the top folders: I can only see the subfolders. Is this an RFC restriction or a Dovecot restriction? If the latter, is it planned to change? Otherwise: just started using Dovecot, and I'm very impressed with the speed of it. Thanks! Keith

4 4

[dovecot] 0.99.9-test5
by Timo Sirainen 21 Apr '03

21 Apr '03

- Fixed startup crashing in POP3 server - Several fixes to opening index file which needs to be rebuilt. Before it didn't always rebuild the index but just fallbacked to using only memory, and that could have been quite slow since all mails in the mailbox were opened and parsed. - If there's nothing in mail_cache_fields, index rebuilding doesn't open the file anymore. This should make in-memory indexes a lot faster than before, but it could also mean some performance hits later. If client wants to fetch all mails from the mailbox, the same thing has to be done anyway.. This needs more thinking actually what is the optimal behaviour. - Support /etc/shadow-like file for passdb.

3 3

[dovecot] Message with lots of attachments breaks Dovecot
by Tim Hunt 21 Apr '03

21 Apr '03

I have a message with over a 1000 embedded attachments(!), and it breaks Dovecot. The issue appears to be buffer management. There is some logic in the function message_part_deserialize_part that seems to assume that the message part structure is in a contiguous array, but that doesn't seem to be the case. I haven't been able to figure out enough about the memory management in Dovecot to fix the problem. Below is a horrible workaround that just allocates a lot more memory for the message part array, so that it remains contiguous. *** mail-index-update.c.orig Wed Mar 26 10:41:55 2003 --- mail-index-update.c Sun Apr 20 16:53:45 2003 *************** *** 498,504 **** if (cache_fields & DATA_FIELD_MESSAGEPART) { t_push(); ! buf = buffer_create_dynamic(data_stack_pool, 2048, (size_t)-1); message_part_serialize(part, buf); --- 498,504 ---- if (cache_fields & DATA_FIELD_MESSAGEPART) { t_push(); ! buf = buffer_create_dynamic(data_stack_pool, 51200, (size_t)-1); message_part_serialize(part, buf);

2 1

[dovecot] Patch for zero content-length
by Tim Hunt 21 Apr '03

21 Apr '03

mbox-index.c uses a zero content length as a flag for an invalid Content-Length: header. Unfortunately zero is also a valid value for Content-Length. The patch below attempts to address the problem. *** mbox-index.h.orig Tue Apr 15 09:04:02 2003 --- mbox-index.h Sun Apr 20 11:42:12 2003 *************** *** 19,24 **** --- 19,25 ---- struct istream *input; uoff_t content_length; int set_read_limit; + unsigned int content_length_good:1; }; int mbox_set_syscall_error(struct mail_index *index, const char *function); *** mbox-index.c.orig Mon Mar 24 05:52:44 2003 --- mbox-index.c Sun Apr 20 12:41:16 2003 *************** *** 264,277 **** start_offset = ctx->input->v_offset; i_stream_set_read_limit(ctx->input, 0); ! end_offset = start_offset + ctx->content_length; ! if (ctx->content_length == 0 || !mbox_verify_end_of_body(ctx->input, end_offset)) { ! if (ctx->content_length != 0) i_stream_seek(ctx->input, start_offset); mbox_skip_message(ctx->input); end_offset = ctx->input->v_offset; ctx->content_length = end_offset - start_offset; } i_stream_seek(ctx->input, start_offset); --- 264,280 ---- start_offset = ctx->input->v_offset; i_stream_set_read_limit(ctx->input, 0); ! end_offset = start_offset; ! if ( ctx->content_length_good ) ! end_offset += ctx->content_length; ! if (!ctx->content_length_good || !mbox_verify_end_of_body(ctx->input, end_offset)) { ! if (ctx->content_length_good ) i_stream_seek(ctx->input, start_offset); mbox_skip_message(ctx->input); end_offset = ctx->input->v_offset; ctx->content_length = end_offset - start_offset; + ctx->content_length_good = TRUE; } i_stream_seek(ctx->input, start_offset); *************** *** 301,311 **** --- 304,316 ---- strcasecmp(hdr->name, "Content-Length") == 0) { /* manual parsing, so we can deal with uoff_t */ ctx->content_length = 0; + ctx->content_length_good = TRUE; for (i = 0; i < hdr->value_len; i++) { if (hdr->value[i] < '0' || hdr->value[i] > '9') { /* invalid */ ctx->content_length = 0; + ctx->content_length_good = FALSE; break; }

1 0

[dovecot] a few small bug
by Farkas Levente 18 Apr '03

18 Apr '03

during the ./autogen.sh there is a few problem: ----------------------------------------- configure.in: installing `./install-sh' configure.in: installing `./mkinstalldirs' configure.in: installing `./missing' Use of uninitialized value in concatenation (.) or string at /usr/bin/automake line 8449. : installing `./config.guess' Use of uninitialized value in concatenation (.) or string at /usr/bin/automake line 8449. : installing `./config.sub' configure.in:3: required file `./config.h.in' not found aclocal.m4:916: required file `./ltmain.sh' not found src/auth/Makefile.am: installing `./depcomp' configure.in:13: warning: AC_ARG_PROGRAM invoked multiple times autoheader: `config.h.in' is created ----------------------------------------- would it be possible to not use your own libtool in the cvs? in this case I can run autogen.sh during the rpm build and doesn't have to run manulay and patch the genereated makefile. ps. this is the patch I use: --- dovecot-1.0/configure.lfarkas 2003-03-20 15:53:04.000000000 +0100 +++ dovecot-1.0/configure 2003-03-20 15:54:37.000000000 +0100 @@ -8205,7 +8205,7 @@ LIBTOOL_DEPS="$ac_aux_dir/ltmain.sh" # Always use our own libtool. -LIBTOOL='$(SHELL) $(top_builddir)/libtool' +LIBTOOL='$(SHELL) libtool' # Prevent multiple expansion -- Levente "Si vis pacem para bellum!"

2 3

[dovecot] Corrupt mbox handling
by Kristian Hoffmann 17 Apr '03

17 Apr '03

I've found that if an mbox file has two consecutive 'From' delimiters, I get the error: pop3(user): Error indexing mbox file /var/spool/mail/user: LF not found where expected UofW apparently ignores the second 'From' silently. I've been trying to run dovecot in gdb to get more details on exactly what is happening, but I haven't quite mastered the art of attaching to the pop3 process between the login process completing and it exiting. Is there some trick to doing that? If I can't work around it in dovecot, I can just write something in perl to fix all the duplicate 'From' lines. Thanks, Kristian Hoffmann PC-InTOUCH/FireToWire System Administrator khoff(a)pc-intouch.com ---

2 1

[dovecot] Changing hierarchy separator to '/'
by Charlie Brady 17 Apr '03

17 Apr '03

3 4

[dovecot] PATCH Add support for kqueue in ioloop subsystem
by Dominic Marks 16 Apr '03

16 Apr '03

Hey, I noticed that there was an ioloop "module" (if we can call it that) for select and poll and decided to add one for kqueue (aka kevent) BSDs high performance descriptor multiplexing API. I haven't done any of the configure glue stuff but the code is complete and works well. kqueue is available on all recent versions of FreeBSD, NetBSD, OpenBSD and Darwin (and therefore MacOS X). I've tested both pop3 and imap with no problems. To get it working: (bodge method) > tar zxvf dovecot-0.99.8.1.tar.gz > cp ioloop-kevent.c dovecot-0.99.8.1/src/lib/ > cd dovecot-0.99.8.1/ > ./configure > vi config.h 1. replace IOLOOP_POLL (or IOLOOP_SELECT) with IOLOOP_KEVENT > vi src/lib/Makefile 1. find liblib_a_SOURCES add ioloop-kevent.c to it 2. find liblib_a_OBJECTS add ioloop-kevent.$(OBJEXT) to it > make There is a paper on kqueue which includes a performance evaulation here: http://people.freebsd.org/~jlemon/papers/kqueue.pdf Thanks, -- Dominic <dom at cus.org.uk> <d.marks at student.umist.ac.uk>

2 1

[dovecot] current cvs
by Farkas Levente 16 Apr '03

16 Apr '03

hi, so after upgrade to the current cvs all of our problem disappear:-) but we don't have time to test the ssl part... although after I compile and install the new dovecot I stop it I do a -------------------- find /home/*/Maildir -name '.imap*' -exec rm {} \; find /home/*/Maildir -name 'dovecot-uidlist' -exec rm {} \; find /home/*/Maildir -name 'msgid.cache' -exec rm {} \; find /home/*/Maildir -name '.customflags' -exec rm {}\; -------------------- just to be sure that we use a fresh version and no other problem can be caused by the old index or any other cached files. this was a BIG mistake!!! we've got about 300 users and about 100 online at the same time. after a two minutes break (while I reinstall and delete) every user check his mail which casue a load about 90!!! the cpu usage 1-2% but the avarage load was 90 (curently about 0.22)! the reason was IO. we have to stop imap and wait for a while... I don't know the exact reason bu I assume dovecot is regenerate everybody's index a other cache at the same time. is there any way to reduce the possible number of IO load? this would be useful if never happend again!:-) this happend around 4:40. the output of sar -b: -------------------- 03:00:00 PM tps rtps wtps bread/s bwrtn/s 03:10:00 PM 48.18 29.19 18.99 1456.53 771.29 03:20:00 PM 41.66 25.80 15.86 1054.88 830.05 03:30:00 PM 42.64 25.67 16.97 1223.03 759.77 03:40:00 PM 44.98 31.95 13.03 1959.15 467.40 03:50:00 PM 26.68 14.63 12.05 741.48 424.12 04:00:00 PM 25.80 11.70 14.10 516.21 450.04 04:10:00 PM 59.55 34.91 24.64 1755.07 939.11 04:20:00 PM 95.98 61.15 34.83 4453.24 1563.25 04:30:00 PM 95.08 55.45 39.63 2182.22 1591.56 04:40:00 PM 168.85 145.99 22.86 11155.73 1037.25 04:50:00 PM 135.98 106.50 29.48 7908.21 1604.76 05:00:00 PM 125.87 97.65 28.23 5876.03 1060.50 05:10:00 PM 80.83 62.92 17.90 4321.82 590.19 05:20:00 PM 102.04 77.65 24.40 4907.27 1076.92 05:30:00 PM 129.04 106.58 22.46 6387.18 883.13 05:40:00 PM 89.44 73.20 16.24 4905.64 782.94 05:50:00 PM 88.23 73.16 15.06 5539.93 965.34 06:00:00 PM 92.49 72.67 19.81 5486.25 692.18 06:10:00 PM 45.24 32.82 12.42 1711.35 618.31 06:20:00 PM 42.17 28.16 14.01 960.11 622.33 06:30:00 PM 29.90 20.24 9.66 1538.27 376.17 06:40:00 PM 25.89 12.14 13.74 796.73 833.25 06:50:00 PM 20.50 4.98 15.52 264.91 993.62 07:00:00 PM 15.08 7.57 7.52 453.41 282.63 -------------------- the output of sar -d -------------------- 04:10:00 PM dev22-3 0.00 0.00 04:10:00 PM dev33-0 1.76 109.55 04:10:00 PM dev34-0 8.51 919.94 04:20:00 PM dev3-0 8.56 183.93 04:20:00 PM dev22-2 67.62 3798.02 04:20:00 PM dev22-3 0.00 0.00 04:20:00 PM dev33-0 4.80 463.81 04:20:00 PM dev34-0 15.01 1570.73 04:30:00 PM dev3-0 4.40 126.63 04:30:00 PM dev22-2 77.02 2440.08 04:30:00 PM dev22-3 0.00 0.00 04:30:00 PM dev33-0 1.94 124.80 04:30:00 PM dev34-0 11.73 1082.28 04:40:00 PM dev3-0 4.78 124.56 04:40:00 PM dev22-2 137.27 10306.18 04:40:00 PM dev22-3 0.00 0.00 04:40:00 PM dev33-0 13.50 410.44 04:40:00 PM dev34-0 13.29 1351.80 04:50:00 PM dev3-0 6.33 194.27 04:50:00 PM dev22-2 111.50 7961.44 04:50:00 PM dev22-3 0.00 0.00 04:50:00 PM dev33-0 5.64 156.51 -------------------- -- Levente "Si vis pacem para bellum!"

2 5