- dovecot - dovecot.org

[Dovecot] deliver+quota assertion
by Andrew Garner 30 Nov '07

30 Nov '07

When stressing dovecot's delivery agent, I sporadically encountered the following assertion failures and the associated errors. I was trickling in ~50 messages per minute to this mailbox via postal. Nov 30 09:38:41 mail deliver(example(a)example.com): close(/home/vmail/domains//+/example.com/k/example//Maildir/maildirsize) failed: Bad file descriptor Nov 30 09:38:41 mail deliver(example(a)example.com): rename(/home/vmail/domains//+/example.com/k/example//Maildir/maildirsize.lo…, /home/vmail/domains//+/example.com/k/example//Maildir/maildirsize) failed : No such file or directory Nov 30 09:38:41 mail deliver(example(a)example.com): file_dotlock_replace(/home/vmail/domains//+/example.com/k/example//Maildir/… failed: No such file or directory Nov 30 09:38:41 mail deliver(example(a)example.com): Internal quota calculation error Nov 30 09:38:41 mail deliver(example(a)example.com): file index-transaction.c: line 54 (index_transaction_finish_rollback): assertion failed: (t->mail_ref_count == 0) Nov 30 09:38:41 mail deliver(example(a)example.com): Raw backtrace: /usr/lib/dovecot/deliver(i_syslog_fatal_handler+0x2b) [0x80bf0ab] -> /usr/lib/dovecot/deliver [0x80beeba] -> /usr/lib/dovecot/deliver [0x 808f243] -> /usr/lib/dovecot/deliver(index_transaction_rollback+0x1a) [0x808f06a] -> /usr/lib/dovecot/modules/lda/lib10_quota_plugin.so [0xb7ea2f13] -> /usr/lib/dovecot/deliver(deliver_save+0x212) [0x8059412] -> /usr/lib/dovecot/deliver(main+0x12cb) [0x805a79b] -> /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xc8) [0xb7ebcea8] -> /usr/lib/dovecot/deliver [0x8058611] # dovecot -n # 1.1.beta9: /etc/dovecot/dovecot.conf base_dir: /var/run/dovecot/ protocols: imap imaps pop3 pop3s login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_greeting: Nationwide Internet Mail Server ready. verbose_proctitle: yes first_valid_uid: 89 last_valid_uid: 89 first_valid_gid: 89 last_valid_gid: 89 mail_uid: 89 mail_gid: 89 mail_location: maildir:/home/vmail/domains/%h/Maildir mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes lock_method: dotlock mail_executable(default): /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/rawlog /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): delay-newmail imap_client_workarounds(imap): delay-newmail imap_client_workarounds(pop3): pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %f pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: login plain cram-md5 digest-md5 username_format: %Lu verbose: yes debug: yes passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail plugin: quota: maildir quota_rule: *:storage=102400 This is the latest hg sources, somewhat after beta9: $ hg tip changeset: 6877:a5a7d38b6df7 branch: HEAD tag: tip user: Timo Sirainen <tss(a)iki.fi> date: Thu Nov 29 19:38:06 2007 +0200

1 0

[Dovecot] Zimbra benchmarking
by Timo Sirainen 30 Nov '07

30 Nov '07

Now that I have a working kvm setup, I thought I'd finally try how Zimbra works. This is mainly some microbenchmarking, so it may not have much to do with actual performance in real life. Setup: - 1GB memory given to kvm (from host's 2GB) - Intel Core 2 6600 (kvm uses only one CPU) - CentOS 5 - 15GB qcow2 image on XFS filesystem - Zimbra 5.0 RC2 RHEL5 x86_64 - Dovecot latest hg, dbox format Both Zimbra and Dovecot were tested running on the same kvm image. All data could be cached in memory, so this didn't really test disk I/O. Dovecot was run with fsync_disable=no. Running with =yes would have made its performance even better. Zimbra features/bugs -------------------- Zimbra's SEARCH command doesn't support substring searching as IMAP requires: 1 search text performance * SEARCH 136 183 227 231 232 233 245 1 OK SEARCH completed 2 search text erformance * SEARCH 2 OK SEARCH completed 3 search text performanc * SEARCH 3 OK SEARCH completed It's possible to search "performanc" in a non-standard way though: 4 search text "performanc*" * SEARCH 136 183 227 231 232 233 245 4 OK SEARCH completed This means that the upcoming Dovecot v1.1 is probably the only IMAP server that supports IMAP-compatible full text search indexes with incremental updates. (Cyrus Squat requires rebuilding the whole index from scratch just for adding one mail, which makes it kind of impractical.) Zimbra doesn't support SORT or THREAD extensions. I had trouble benchmarking more than 5 simultaneous connections in one mailbox. I'm not sure if this is a configurable setting somewhere, or if there's just some bug. The error messages that my imaptest gave looked like Zimbra was buggy, but I didn't look at them too closely. STORE command also seemed to be buggy, giving lots of "STORE failed" errors, even with just one connection. Zimbra uses 500MB of memory just to start up, so it's not exactly for hosting small installations. Login+Logout ------------ Dovecot: ./imaptest - select=0 secs=10 seed=0 clients=1 Logi Logo 100% 100% 3439 6878 ./imaptest - select=0 secs=10 seed=0 clients=10 Logi Logo 100% 100% 4415 8830 Zimbra: ./imaptest - select=0 secs=10 seed=0 clients=1 Logi Logo 100% 100% 18032 36064 ./imaptest - select=0 secs=10 seed=0 clients=10 Logi Logo 100% 100% 25519 51056 Looks like Dovecot's imap process creation hurts it a lot compared to Zimbra's thread creation. Luckily IMAP connections are usually long-living, so this shouldn't matter much, except for webmails for which you can use imapproxy in the middle. LIST "" * --------- Dovecot: ./imaptest - select=0 logout=0 list=100 secs=10 seed=0 clients=1 Logi List 100% 100% 1 181411 ./imaptest - select=0 logout=0 list=100 secs=10 seed=0 clients=10 Logi List 100% 100% 10 133451 Zimbra: ./imaptest - select=0 logout=0 list=100 secs=10 seed=0 clients=1 Logi List 100% 100% 1 1021 ./imaptest - select=0 logout=0 list=100 secs=10 seed=0 clients=10 Logi List 100% 100% 10 972 Yes, Dovecot's LIST is over 100 times faster. STATUS INBOX (MESSAGES UNSEEN RECENT) ------------------------------------- 100 messages in INBOX dovecot: ./imaptest - logout=0 select=0 status=100 secs=10 seed=0 clients=1 Logi Stat 100% 100% 1 191003 ./imaptest - logout=0 select=0 status=100 secs=10 seed=0 clients=10 Logi Stat 100% 100% 10 171171 Zimbra: ./imaptest - logout=0 select=0 status=100 secs=10 seed=0 clients=1 Logi Stat 100% 100% 1 3697 ./imaptest - logout=0 select=0 status=100 secs=10 seed=0 clients=10 Logi Stat 100% 100% 10 7009 FETCH 1:* (UID FLAGS ENVELOPE INTERNALDATE BODYSTRUCTURE) --------------------------------------------------------- 100 messages in INBOX. dovecot: ./imaptest - logout=0 select=100 fetch=100 secs=10 seed=0 clients=1 Logi Sele Fetc 100% 100% 100% 1 1 2769 ./imaptest - logout=0 select=100 fetch=100 secs=10 seed=0 clients=5 Logi Sele Fetc 100% 100% 100% 5 5 2902 Zimbra: ./imaptest - logout=0 select=100 fetch=100 secs=10 seed=0 clients=1 Logi Sele Fetc 100% 100% 100% 1 1 203 ./imaptest - logout=0 select=100 fetch=100 secs=10 seed=0 clients=5 Logi Sele Fetc 100% 100% 100% 5 5 258 FETCH ? (BODY[]) ---------------- Randomly fetch message body from 100 messages Dovecot: ./imaptest - logout=0 select=100 fetch2=100 secs=10 seed=0 clients=1 Logi Sele Fet2 100% 100% 100% 30% 1 1 51958 ./imaptest - logout=0 select=100 fetch2=100 secs=10 seed=0 clients=5 Logi Sele Fet2 100% 100% 100% 30% 5 5 141397 Zimbra: ./imaptest - logout=0 select=100 fetch2=100 secs=10 seed=0 clients=1 Logi Sele Fet2 100% 100% 100% 30% 1 1 410 ./imaptest - logout=0 select=100 fetch2=100 secs=10 seed=0 clients=5 Logi Sele Fet2 100% 100% 100% 30% 5 5 2291 STORE ? FLAGS.SILENT (random flags and keywords) ------------------------------------------------ Dovecot: ./imaptest - logout=0 select=100 store=100 secs=10 seed=0 clients=1 Logi Sele Stor 100% 100% 100% 1 1 15236 ./imaptest - logout=0 select=100 store=100 secs=10 seed=0 clients=5 Logi Sele Stor 100% 100% 100% 5 5 5533 Zimbra: ./imaptest - logout=0 select=100 store=100 secs=10 seed=0 clients=1 Logi Sele Stor 100% 100% 100% 1 1 308 ./imaptest - logout=0 select=100 store=100 secs=10 seed=0 clients=5 Logi Sele Stor 100% 100% 100% 5 5 319 Zimbra gave lots of errors: Error: STORE failed: STORE failed APPEND ------ Appends to empty mailbox: Dovecot: 1 client, 5 seconds: 3959 mails 1 client, 5 seconds, fsync_disable=yes: 21889 mails Zimbra: 1 clients, 5 seconds: 222 mails 5 clients, 5 seconds: 322 mails Zimbra is apparently building full text search indexes while appending, so this test doesn't mean much until I can test Dovecot's performance with Squat indexing.

5 6

[Dovecot] squat plugin
by Joe Wong 29 Nov '07

29 Nov '07

Hello Timo, I have installed 1.1 beta 8 with fts-squat enabled. I have added the plugin section in dovecot.conf. How can I actually make use of the full text index? Regards, - Joe

2 2

[Dovecot] old mails
by Timo Sirainen 29 Nov '07

29 Nov '07

I just got a 6 months old mail to this list and wondered for a while why it happened. Looks like the old dovecot.org was rebooted and Postfix went up and started sending queued mails. So just ignore if you got those.

1 0

Re: [Dovecot] 1.1beta9 'make' fails on osx/Tiger
by Jerry Yeager 29 Nov '07

29 Nov '07

On Nov 28, 2007, at 1:39 PM, dovecot-request(a)dovecot.org wrote: > ------------------------------ > > Message: 7 > Date: Wed, 28 Nov 2007 10:38:19 -0800 > From: snowcrash <schneecrash+dovecot(a)gmail.com> > Subject: Re: [Dovecot] 1.1beta9 'make' fails on osx/Tiger > To: "Jerry Yeager" <jerry(a)scene-naturally.dyndns.org> > Cc: dovecot(a)dovecot.org > Message-ID: > <70f41ba20711281038q12499454o94a9a221e61c54b2(a)mail.gmail.com> > Content-Type: text/plain; charset=ISO-8859-1 > (stuff cut out) > > i'd gather tss would like to know about issues. (it shouldn't be > terribly surprising for a projects' beta release to at least be > _tried_ with _other_ projects' beta releases ...) > > cheers. > > That is usually a good idea, especially if you mention that you are using non standard stuff so that folks who are not aware of that can then get a better understanding of the issues. Or for the elucidation of the folks that are following the list trying to decide what efforts they will have to undertake to make the switch. p.s. The folks at AT&T also recommend that "you" use the newer version of gcc than the older-hacked-to-get-something-for-Tiger version of the Apple Leopard version they (AT&T) are providing -- it is further down the page where they provide links. Jerry

2 2

[Dovecot] autogen.sh fails -> "required file `./config.rpath' not found"
by snowcrash 29 Nov '07

29 Nov '07

if i run autogen.sh on fresh 1.1.beta9 source, it fails @ ./autogen.sh ... 100%[====================================>] 154,427 115.09K/s 08:45:56 (114.82 KB/s) - `wiki-export.tar.gz' saved [154427/154427] configure.in:16: required file `./config.rpath' not found configure.in:5: installing `./missing' configure.in:5: installing `./install-sh' src/auth/Makefile.am: installing `./depcomp' autoreconf: automake failed with exit status: 1 per advice, "gettext-related test failures: config.rpath presence" http://www.mail-archive.com/automake-patches@gnu.org/msg00426.html a simple touch config.rpath ./autogen.sh fixes the problem. hth

2 4

[Dovecot] spf record
by Jim Flowers 28 Nov '07

28 Nov '07

Your spf record is broken: dovecot.org. 39942 IN TXT "v=spf1 a -all" -- Jim Flowers <jflowers(a)ezo.net> -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.

12 20

Re: [Dovecot] 1.1beta9 'make' fails on osx/Tiger
by snowcrash 28 Nov '07

28 Nov '07

(Yes, Really .... w/ gcc v4.2.1, like i said in the first place) > Actually, the 1.1 beta v9 builds just fine under Tiger (PPC and > PPC64) if you use the official Apple sanctioned gcc v 4.01 they ship > with both Tiger and Leopard. If you do download and use gcc 4.2.1 from > the GNU folks you might want to pass along the error / bug reports to > them instead of Apple, but I suspect the GNU folks will tell you to > try the current 4.2.2 version or the new 4.3 version. i'm using Apple's gcc 4.2.1: http://www.opensource.apple.com/darwinsource/10.5/gcc_42-5531/ and the v4.2.1 Tiger equiv from the R-project provided at http://r.research.att.com/tools/ both Apple & ATT are _already_ well plugged into both .. er .. Apple & GNU. yes, 4.2.1 is "not distro" ... but it *is* up-n-coming, as evienced by the fact that it *is* offered on the darwinsource page. i'd gather tss would like to know about issues. (it shouldn't be terribly surprising for a projects' beta release to at least be _tried_ with _other_ projects' beta releases ...) cheers.

1 0

Re: [Dovecot] 1.1beta9 'make' fails on osx/Tiger (Not Really)
by Jerry Yeager 28 Nov '07

28 Nov '07

On Nov 28, 2007, at 10:59 AM, dovecot-request(a)dovecot.org wrote: > Message: 8 > Date: Wed, 28 Nov 2007 07:49:13 -0800 > From: snowcrash <schneecrash+dovecot(a)gmail.com> > Subject: Re: [Dovecot] 1.1beta9 'make' fails on osx/Tiger, but OK on > osx/Leopard (multiple definitions of symbol _hash_create) > To: "Timo Sirainen" <tss(a)iki.fi> > Cc: Dovecot Mailing List <dovecot(a)dovecot.org> > Message-ID: > <70f41ba20711280749k63a6b099obf09b687e5733422(a)mail.gmail.com> > Content-Type: text/plain; charset=ISO-8859-1 > >> (lots of messages from the digest snipped out) Actually, the 1.1 beta v9 builds just fine under Tiger (PPC and PPC64) if you use the official Apple sanctioned gcc v 4.01 they ship with both Tiger and Leopard. If you do download and use gcc 4.2.1 from the GNU folks you might want to pass along the error / bug reports to them instead of Apple, but I suspect the GNU folks will tell you to try the current 4.2.2 version or the new 4.3 version. Jerry

1 0

[Dovecot] 1.1beta9 'make' fails on osx/Tiger, but OK on osx/Leopard (multiple definitions of symbol _hash_create)
by snowcrash 28 Nov '07

28 Nov '07

i'm building dovecot 1.1.beta9 on both osx 10.4.11 osx 10.5.1 using Using built-in specs. Target: powerpc-apple-darwin8 Configured with: /Builds/apple/gcc_42-5531/obj/src/configure --disable-checking -enable-werror --prefix=/usr --mandir=/usr/share/man --enable-languages=c,objc,c++,obj-c++ --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ --with-gxx-include-dir=/usr/include/c++/4.0.0 --with-slibdir=/usr/lib --build=i686-apple-darwin8 --program-prefix= --host=powerpc-apple-darwin8 --target=powerpc-apple-darwin8 Thread model: posix gcc version 4.2.1 (Apple Inc. build 5531) on both boxes. configuring as Install prefix ...................... : /usr/local/dovecot File offsets ........................ : 64bit I/O loop method ..................... : kqueue File change notification method ..... : kqueue Building with SSL support ........... : yes (OpenSSL) Building with IPv6 support .......... : yes Building with pop3 server ........... : yes Building with mail delivery agent .. : yes Building with GSSAPI support ........ : no Building with user database modules . : static prefetch passwd passwd-file checkpassword sql Building with password lookup modules : passwd passwd-file pam checkpassword sql Building with SQL drivers ............:mysql sqlite is OK on both boxes. on Leopard, make completes without error. but, on Tiger, it fails @ make ... make[3]: Nothing to be done for `all'. Making all in master /bin/sh ../../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wstrict-aliasing=2 -I/usr/local/ssl/include -L/usr/local/db46/lib -L/usr/local/sqlite/lib -lsqlite3 -L/usr/local/mysql/lib/mysql -L/usr/local/lib -liconv -o dovecot auth-process.o askpass.o capabilities-posix.o child-process.o dict-process.o dup2-array.o listener.o log.o login-process.o mail-process.o main.o master-settings.o syslog-util.o ssl-init.o ../lib-settings/libsettings.a ../lib/liblib.a gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wstrict-aliasing=2 -I/usr/local/ssl/include -o dovecot auth-process.o askpass.o capabilities-posix.o child-process.o dict-process.o dup2-array.o listener.o log.o login-process.o mail-process.o main.o master-settings.o syslog-util.o ssl-init.o -L/usr/local/db46/lib -L/usr/local/sqlite/lib /usr/local/sqlite/lib/libsqlite3.dylib -L/usr/local/lib -lreadline -lncurses -lpthread -L/usr/local/mysql/lib/mysql /usr/local/lib/libiconv.dylib ../lib-settings/libsettings.a ../lib/liblib.a /usr/libexec/gcc/powerpc-apple-darwin8/4.2.1/ld: multiple definitions of symbol _hash_create /usr/lib/gcc/powerpc-apple-darwin8/4.2.1/../../../libpthread.dylib(strhash.So) definition of _hash_create ../lib/liblib.a(hash.o) definition of _hash_create in section (__TEXT,__text) /usr/libexec/gcc/powerpc-apple-darwin8/4.2.1/ld: multiple definitions of symbol _hash_destroy /usr/lib/gcc/powerpc-apple-darwin8/4.2.1/../../../libpthread.dylib(strhash.So) definition of _hash_destroy ../lib/liblib.a(hash.o) definition of _hash_destroy in section (__TEXT,__text) collect2: ld returned 1 exit status make[3]: *** [dovecot] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 looks like problems with ot at all sure what in Tiger's causing this ... hints/suggestions? thanks!

3 14

Re: [Dovecot] imap process killed with signal 11
by Daniel 28 Nov '07

28 Nov '07

On Mon, 26 Nov 2007 14:49:01 +0200 Timo Sirainen <tss(a)iki.fi> wrote: > On Sun, 2007-11-25 at 13:39 +0000, Daniel wrote: > > Using the latest cvs from HEAD, I'm experiencing problems after > > sending a message and the client is trying upload the message to the > > appropriate (Sent/Queue) folder. I get only one error in > > dovecot.log: dovecot: Nov 25 14:21:31 Error: child 19554 (imap) > > killed with signal 11 > > Could you get gdb backtrace from this crash? Otherwise I can't really > do anything about it. See http://dovecot.org/bugreport.html > Sure: (gdb) bt full #0 0x1c04af71 in mail_cache_field_exists () No symbol table info available. #1 0x1c04cb4c in mail_cache_field_can_add () No symbol table info available. #2 0x1c041b9d in index_mail_cache_add_idx () No symbol table info available. #3 0x1c041c5c in index_mail_cache_add_idx () No symbol table info available. #4 0x1c04259d in index_mail_close () No symbol table info available. #5 0x1c04298a in index_mail_free () No symbol table info available. #6 0x1c0629cc in mail_free () No symbol table info available. #7 0x035b06ac in quota_mailbox_transaction_commit (ctx=0x8acfa700, uid_validity_r=0xcfbf928c, first_saved_uid_r=0xcfbf9290, last_saved_uid_r=0xcfbf9294) at quota-storage.c:100 qbox = (struct quota_mailbox *) 0x7e27b1e0 qt = (struct quota_transaction_context *) 0x87187bc0 #8 0x1c064810 in mailbox_transaction_commit_get_uids () No symbol table info available. #9 0x1c010a04 in __register_frame_info () No symbol table info available. #10 0x1c010c1c in __register_frame_info () No symbol table info available. #11 0x1c0103e3 in __register_frame_info () No symbol table info available. #12 0x1c078858 in io_loop_handler_run () No symbol table info available. #13 0x1c077e75 in io_loop_run () No symbol table info available. #14 0x1c01c158 in main () No symbol table info available. Is this useful? Daniel

2 2

[Dovecot] How to find an invalid configuration in dovecot.conf
by Laurent PELLISSIER 28 Nov '07

28 Nov '07

Hi, I'm trying to test Dovecot as an IMAP ans POP3 server but I ran into a fatal problem. On the client side (telnet) I'm trying very basic IMAP commands : $ telnet localhost imap Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. * OK Dovecot ready. 1 LOGIN lavande secret 1 OK Logged in. Connection closed by foreign host. The login succeeds but the connection hangs ! On the server side it complains that the configuration file dovecot.conf is invalid (it was copied from dovecot-example.conf) but it never tells what line number is incorrect. dovecot -a or dovecot -n doesn't complain about a syntax error or something. It works for POP3 for the same user/pass. # /opt/dovecot/sbin/dovecot -F -c /etc/dovecot.conf dovecot: Nov 28 15:43:42 Info: Dovecot v1.0.8 starting up dovecot: Nov 28 15:43:53 Info: imap-login: Login: user=<lavande>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured dovecot: Nov 28 15:43:53 Error: IMAP(lavande): Error: Dovecot is already running with PID 4369 (read from /opt/dovecot-1.0.8/var/run/dovecot/master.pid) dovecot: Nov 28 15:43:53 Error: IMAP(lavande): Fatal: Invalid configuration in /etc/dovecot.conf dovecot: Nov 28 15:43:53 Error: IMAP(lavande): Fatal: EOF while reading environment from master dovecot: Nov 28 15:43:53 Error: child 4380 (imap) returned error 89 Best regards, -- Laurent PELLISSIER @ Ecole des Mines d'Alès

2 3

[Dovecot] quick question about fs quota overhead in plugin
by Adam McDougall 28 Nov '07

28 Nov '07

Last night I enabled imap_quota so dovecot could report usage reported by disk quota. I don't intend to actually use the quota plugin to place any limits anytime soon though. How much overhead does this add to normal operations that allocate disk space? Ideally I'd like a situation where the only overhead is incurred when the user uses the mail client to specifically check their usage. Is that possible, and/or is there a better way to do this? If it does cause general overhead on the NFS filer, I could just accept it until/unless I feel it becomes a burden I cannot bear. mail_plugins = acl fts fts_squat quota imap_quota ... quota = fs ...

2 1

[Dovecot] v1.0.8 released
by Timo Sirainen 28 Nov '07

28 Nov '07

http://dovecot.org/releases/1.0/dovecot-1.0.8.tar.gz http://dovecot.org/releases/1.0/dovecot-1.0.8.tar.gz.sig + Authentication: Added "password_noscheme" field that can be used instead of "password". "password" treats "{prefix}" as a password scheme while "password_noscheme" treats it as part of the password itself. So "password_noscheme" should be used if you're storing passwords as plaintext. Non-plaintext passwords never begin with "{", so this isn't a problem with them. - IMAP: Partial body fetching was sometimes non-optimal, causing the entire message to be read for every FETCH command. - deliver failed to save the message when envelope sender address contained spaces. - Maildir++ quota: We could have randomly recalculated quota when it wasn't necessary. - Login process could have crashed after logging in if client sent data before "OK Logged in" reply was sent (i.e. before master had replied that login succeeded). - Don't assert-crash when reading dovecot.index.logs generated by Dovecot v1.1. - Authentication: Don't assert-crash if password beings with "{" but doesn't contain "}". - Authentication cache didn't work when using settings that changed the username (e.g. auth_username_format).

1 0

[Dovecot] IMAP --> Creating Folders --> trailing slash
by Andre Hübner 28 Nov '07

28 Nov '07

Hello List, i use Dovecot 1.0.7 for pop and imap on Suse Linux 10.1. I need a little help in understanding something. If i want to create a folder with outlook express or thunderbird which can also create subfolders, then i have to create these folders with a trailing slash. folders created with trailing slash are in filesystem created als real folders, "normal" folders are created as file. Is it possible to change this behavior? I want to create a folders just by typing a name without trailing slash. It should be possible to store messages in this folder or create subfolders. Is there any workaround to achieve this goal? Thank you Andre

2 1

[Dovecot] little submissions feature request
by Netskin | Corin Langosch 28 Nov '07

28 Nov '07

Hi! I'm using dovecot for a while now and really like it. Actually dovecot 1.5 on Debian AMD64. Now I have to use a cifs mounted maildir directory and found it to be quite slow, though I already tuned it a lot. The only way to get it fast was to put INDEX and CONTROL files on the local disc and so outside of the maildirs. Performance is great but the problem is with submissions to maildirs. When a CONTROL directive is given, dovecot writes the submissions file to this directory too. Now as I have two dovecots in a loadbalancer my users have to do their modifications to their subscriptions always twice (for each server in the loadbalancer). I now modified dovecot on my own to put the submissions file into the Maildir, everything else is unchanged. This works just great and as desired. Anything I missed in my setup, locking issues, ...? Otherwise it may worth adding a switch for this? A directive like INDEX, CONTROL... :SUBMISSSIONS=path... Regards, Corin

2 1

[Dovecot] 1.1.beta8 crashes with segfault when SIGHUP
by arvids 28 Nov '07

28 Nov '07

Hello, dovecot crashes when it receives -HUP signal. It happens always if there was some activity - for example, if I start dovecot, check any account through POP3 and then send -HUP to dovecot process, it crashes with the following log entry: segfault at 00000008 eip 0804d3fb esp bfdd3860 error 4 If there were no activity at all since starting, it does not crash. my dovecot -n output: # 1.1.beta8: /usr/local/dovecot/etc/dovecot.conf base_dir: /var/run/dovecot/ syslog_facility: local0 protocols: pop3 ssl_disable: yes disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/local/dovecot/libexec/dovecot/pop3-login login_greeting: Server. login_process_per_connection: no login_process_size: 64 login_processes_count: 5 login_max_processes_count: 32 verbose_proctitle: yes first_valid_uid: 95 first_valid_gid: 95 mail_uid: 95 mail_gid: 95 mail_location: maildir:~/Maildir mail_debug: yes dotlock_use_excl: yes fsync_disable: yes mail_executable: /usr/local/dovecot/libexec/dovecot/pop3 mail_plugins: quota mail_plugin_dir: /usr/local/dovecot/lib/dovecot/pop3 pop3_enable_last: yes pop3_client_workarounds: outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login digest-md5 cram-md5 apop cache_size: 4096 cache_ttl: 28800 cache_negative_ttl: 60 user: doveauth verbose: yes debug: yes debug_passwords: yes passdb: driver: sql args: /usr/local/dovecot/etc/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /usr/local/dovecot/etc/dovecot-sql.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail plugin: quota: dict:::proxy:/var/run/dovecot/dict-server:quotadict dict: quotadict: mysql:/usr/local/dovecot/etc/dovecot-sql-dict-quota.conf Regards, Arvids

2 2

[Dovecot] 1.1.beta9 deliver crashes with segfault
by arvids 28 Nov '07

28 Nov '07

Hello, after upgrade from beta8 to beta9 deliver process crashes with segfault. Deleting old maildirs does not help. Here is backtrace: Program received signal SIGSEGV, Segmentation fault. 0x0808e36d in mail_cache_field_get_decision (cache=0x8114da0, field_idx=128) at mail-cache-lookup.c:301 301 i_assert(field_idx < cache->fields_count); (gdb) bt #0 0x0808e36d in mail_cache_field_get_decision (cache=0x8114da0, field_idx=128) at mail-cache-lookup.c:301 #1 0x0808f351 in mail_cache_field_can_add (ctx=0x810ca58, seq=1, field_idx=4) at mail-cache-transaction.c:997 #2 0x0808351f in index_mail_want_cache (mail=0x810ce00, field=<value optimized out>) at index-mail.c:597 #3 0x08084046 in index_mail_cache_sizes (mail=0x810ce00) at index-mail.c:618 #4 0x080845a7 in index_mail_close (_mail=0x810ce00) at index-mail.c:971 #5 0x0808510e in index_mail_free (_mail=0x810ce00) at index-mail.c:1166 #6 0x080a8e20 in mail_free (mail=0x810cdcc) at mail.c:18 #7 0xb7e3dd3c in quota_mailbox_transaction_commit (ctx=0x810c6f8, uid_validity_r=0xbfedf444, first_saved_uid_r=0xbfedf444, last_saved_uid_r=0xbfedf444) at quota-storage.c:100 #8 0x080aa525 in mailbox_transaction_commit (t=0xbfedf484) at mail-storage.c:661 #9 0x08059304 in deliver_save (namespaces=0x80f1210, storage_r=0xbfedf558, mailbox=0x80d6918 "INBOX", mail=0x80f8260, flags=0, keywords=0x0) at deliver.c:146 #10 0x0805a6cd in main (argc=1301916, argv=0xbfedf614) at deliver.c:898 my dovecot -n output: # 1.1.beta9: /usr/local/dovecot/etc/dovecot.conf base_dir: /var/run/dovecot/ syslog_facility: local0 protocols: pop3 ssl_disable: yes disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/local/dovecot/libexec/dovecot/pop3-login login_greeting: Server. login_process_per_connection: no login_processes_count: 5 login_max_processes_count: 32 verbose_proctitle: yes first_valid_uid: 95 first_valid_gid: 95 mail_uid: 95 mail_gid: 95 mail_location: maildir:~/Maildir mail_debug: yes dotlock_use_excl: yes fsync_disable: yes mail_executable: /usr/local/dovecot/libexec/dovecot/pop3 mail_plugins: quota mail_plugin_dir: /usr/local/dovecot/lib/dovecot/pop3 pop3_enable_last: yes pop3_client_workarounds: outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login digest-md5 cram-md5 apop cache_size: 1024 cache_ttl: 600 cache_negative_ttl: 60 user: doveauth verbose: yes debug: yes debug_passwords: yes passdb: driver: sql args: /usr/local/dovecot/etc/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /usr/local/dovecot/etc/dovecot-sql.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail plugin: quota: dict:::proxy:/var/run/dovecot/dict-server:quotadict dict: quotadict: mysql:/usr/local/dovecot/etc/dovecot-sql-dict-quota.conf Regards, Arvids

2 1

[Dovecot] Can't login from Mac OS X Mail.app
by Mike McGonagle 28 Nov '07

28 Nov '07

Hello all, I have been working on getting this machine -- Peace may sound simple—one beautiful word— but it requires everything we have, every quality, every strength, every dream, every high ideal. —Yehudi Menuhin (1916–1999), musician

2 1

[Dovecot] Embracing .99
by Joe Allesi -X (joallesi - Coyote Creek Consulting at Cisco) 28 Nov '07

28 Nov '07

We're stuck on RHEL 4.x, so we're currently working on a plan to live in harmony with .99 for another year or so. With that said, the plan is below. Let me know if you have any other recommendations. [ ]Convert to high-performance mode vice high-security to reduce overall resource usage. [ ]Implement iptables throttling for new connections to prevent denials of service. [ ]Covert from mbox to maildir to help reduce the likelihood of corruption that lead to mailbox outages. As our overall individual mailbox usage is low, maildir seems to be a good option as mbox corruption has severely impacted a few of our high-profile applications. However, does anyone know of a sane method to use maildir on RHEL 4.x without switching over to Postfix? Thanks!

5 4

[Dovecot] Various uidlist and index errors with 1.1 on NFS
by Adam McDougall 28 Nov '07

28 Nov '07

Two nights ago I took a leap and extended my testing of dovecot 1.1 by replacing 1.0 for the approx 15 users I had on 1.0. At that time I also for the first time tried dovecot 1.1 in a load balanced 2 server configuration with indexes on NFS. I was hoping I did this right, using the mail_nfs params and 1.1 so fchown etc would flush the access cache, but I am getting a number of messages and errors that indicate things aren't happy. Attaching error logs. Some of the errors were caused by me running multiple mail clients including mutt, Thunderbird, Outlook Express, other users may be using one or more copies of thunderbird, or possibly other clients. Most commonly thunderbird. Any help or advice would be appreciated. My goal is to have shared indexes between servers since I will eventually have 4. I'd prefer to avoid maintaining some kind of user->server preference mapping in a proxy (haven't looked into it yet). Thanks. # 1.1.beta6: /usr/local/etc/dovecot.conf ssl_cert_file: /usr/local/etc/apache2/ssl/mail.pem ssl_key_file: /usr/local/etc/apache2/ssl/mail.pem login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 1000 mail_extra_groups: postlocal mail_location: maildir:%h/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_plugins: acl mail_log_max_lines_per_sec: 0 imap_client_workarounds: delay-newmail netscape-eoh tb-extra-mailbox-sep namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: private separator: / prefix: mail/ hidden: yes subscriptions: yes namespace: type: private separator: / prefix: Mail/ hidden: yes subscriptions: yes namespace: type: public separator: / prefix: #shared/decs/ location: maildir:/egr/mail/shared/decs:CONTROL=%h/Maildir/dovecot/public/control/decs:INDEX=%h/Maildir/dovecot/public/indexes/decs list: yes subscriptions: yes auth default: passdb: driver: pam userdb: driver: passwd args: system_user= socket: type: listen client: path: /var/spool/postfix/private/auth mode: 384 user: postfix group: postfix plugin: acl: vfile:/usr/local/etc/dovecot-acls:cache_secs=10

2 8

[Dovecot] Fwd: Re: Fwd: dovecot bug report (64 bit file offsets)
by markevich＠inbox.lv 27 Nov '07

27 Nov '07

Yes, the problem is the compatibility with the mysql library but I stull can't figure out how this might affect reading/writing. ---------- Forwarded message ---------- No: markevich(a)inbox.lv Kam: Marcus Rueckert <darix(a)opensu.se> Atbildt: markevich(a)inbox.lv Datums: Tue, 27 Nov 2007 14:15:29 +0200 Temats: Re: [Dovecot] Fwd: dovecot bug report (64 bit file offsets) I agree that it’s kind strange behavior. Also I’ve googled that exactly the same errors (unexpected eof found … on file ‘UNOPENED’) were related to mysql. But how mysql compile options might affect configuration file parsing and read/write operations? (indexes for example) Does any one have any idea why this might happen? “any slight change to the main dovecot.conf file can prevent dovecot from starting (segfault with "Unexpected eof found when reading file 'UNOPENED' (Errcode: 0)" message), whereas i386 packages work somewhat better.” http://dovecot.org/list/dovecot/2006-April/012340.html Regards, Andris Quoting Marcus Rueckert : which is funny actually as on opensuse (10.1 -> 10.3 -> factory) has no problem with dovecot running on x86_64. darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org -- Tavs bezmaksas pasts Inbox.lv Links: ------ [1] mailto:darix@opensu.se ---------- End of forwarded message ---------- -- Tavs bezmaksas pasts Inbox.lv

1 0

[Dovecot] dovecot imap hangs
by russ 27 Nov '07

27 Nov '07

All, I recently did a fresh install/setup on centos 5 to replace my older courier-imap and all seemed to go well until today I started to notice that it simply hangs. I use thunderbird and squirrel mail and after a while it seems like I"m simply rejected. I'm almost guessing that it has to do with the session timing out and then dovecot is unwilling or unable to renew the session. I can't login and there is little or no information in the logs. I've set up a cron job to restart dovecot every 5 minutes to deal with this for now. I'm new to dovecot, what sort of info should I send to you guys? I'm using centos 5, qmail-ldap and Maildir for my mail format. OpenLDAP: slapd 2.3.27 dovecot-1.0-1.2.rc15.el5 my /etc/dovecot.conf =================== mail_location = maildir:%h namespace private { prefix = INBOX. inbox = yes } mail_debug = yes maildir_copy_with_hardlinks = yes auth default { mechanisms = plain passdb ldap { # Path for LDAP configuration file, see doc/dovecot-ldap.conf for example args = /etc/dovecot-ldap.conf } userdb ldap { args = /etc/dovecot-ldap.conf } } =================== my dovecot-ldap.conf =================== # This file is opened as root, so it should be owned by root and mode 0600. # # NOTE: If you're not using authentication binds, you'll need to give # dovecot-auth read access to userPassword field in the LDAP server. # With OpenLDAP this is done by modifying /etc/ldap/slapd.conf. There should # already be something like this: # access to attribute=userPassword # by dn="<dovecot's dn>" read # add this # by anonymous auth # by self write # by * none # Space separated list of LDAP hosts to use. host:port is allowed too. hosts =127.0.0.1:389 # LDAP URIs to use. You can use this instead of hosts list. Note that this # setting isn't supported by all LDAP libraries. #uris = # Distinguished Name - the username used to login to the LDAP server #dn = dn=cn=Manager,dc=cttechhosting,dc=net dnpass=secret # Password for LDAP server #dnpass = # Use SASL binding instead of the simple binding. Note that this changes # ldap_version automatically to be 3 if it's lower. Also note that SASL binds # and auth_bind=yes don't work together. #sasl_bind = no # SASL mechanism name to use. #sasl_mech = # SASL realm to use. #sasl_realm = # SASL authorization ID, ie. the dnpass is for this "master user", but the # dn is still the logged in user. Normally you want to keep this empty. #sasl_authz_id = # Use authentication binding for verifying password's validity. This works by # logging into LDAP server using the username and password given by client. # The pass_filter is used to find the DN for the user. Note that the pass_attrs # is still used, only the password field is ignored in it. Before doing any # search, the binding is switched back to the default DN. auth_bind = yes # If authentication binding is used, you can save one LDAP request per login # if users' DN can be specified with a common template. The template can use # the standard %variables (see user_filter). Note that you can't # use any pass_attrs if you use this setting. # # If you use this setting, it's a good idea to use a different # dovecot-ldap.conf for userdb (it can even be a symlink, just as long as the # filename is different in userdb's args). That way one connection is used only # for LDAP binds and another connection is used for user lookups. Otherwise # the binding is changed to the default DN before each user lookup. # # For example: # auth_bind_userdn = cn=%u,ou=people,o=org # auth_bind_userdn = uid=%u,ou=accounts,dc=cttechhosting,dc=net # LDAP protocol version to use. Likely 2 or 3. #ldap_version = 2 ldap_version=3 # LDAP base. %variables can be used here. base = ou=accounts,dc=cttechhosting,dc=net # Dereference: never, searching, finding, always #deref = never # Search scope: base, onelevel, subtree #scope = subtree # User attributes are given in LDAP-name=dovecot-internal-name list. The # internal names are: # uid - System UID # gid - System GID # home - Home directory # mail - Mail location # # There are also other special fields which can be returned, see # http://wiki.dovecot.org/UserDatabase/ExtraFields user_attrs = mailMessageStore=home,qmailUID=uid,qmailGID=gid,mailMessageStore=mail # Filter for user lookup. Some variables can be used (see # http://wiki.dovecot.org/Variables for full list): # %u - username # %n - user part in user@domain, same as %u if there's no domain # %d - domain part in user@domain, empty if user there's no domain user_filter = (&(objectClass=qmailUser)(uid=%u)) # Password checking attributes: # user: Virtual user name (user@domain), if you wish to change the # user-given username to something else # password: Password, may optionally start with {type}, eg. {crypt} # There are also other special fields which can be returned, see # http://wiki.dovecot.org/PasswordDatabase/ExtraFields pass_attrs = mail=user,userPassword=password # If you wish to avoid two LDAP lookups (passdb + userdb), you can use # userdb prefetch instead of userdb ldap in dovecot.conf. In that case you'll # also have to include user_attrs in pass_attrs field prefixed with "userdb_" # string. For example: #pass_attrs = uid=user,userPassword=password,homeDirectory=userdb_home,qmailUID=userdb_uid,qmailGID=userdb_gid # Filter for password lookups pass_filter = (&(objectClass=qmailUser)(uid=%u)) # Default password scheme. "{scheme}" before password overrides this. # List of supported schemes is in: http://wiki.dovecot.org/Authentication default_pass_scheme = LDAP-SHA # You can use same UID and GID for all user accounts if you really want to. # If the UID/GID is still found from LDAP reply, it overrides these values. #user_global_uid = ========================= #user_global_gid = ===================

4 6

[Dovecot] Fwd: dovecot bug report (64 bit file offsets)
by markevich＠inbox.lv 27 Nov '07

27 Nov '07

I have found similar problem: http://www.dovecot.org/list/dovecot/2007-January/018623.html "After configure remove HAVE_PREAD from config.h, recompile and see if it works." I think I figured out where might be the problem. If the binary is compiled in the 64bit environment and you want to use 64 bit file offset you must use 64 bit equivalents of the function. Of course if you are using open/pread (which are 32 bit compatible only) the whole program functionality will be broken. You must use open64/pread64 etc. Else macros _LARGE_FILE_API should be defined in order to use large files in other case nothing will work (random parsing errors and dead indexes will appear ). As I can see there is a big issue in the dovecot. http://publib.boulder.ibm.com/iseries/v5r2/ic2924/index.htm?info/apis/pread… Regards, Andris ---------- Forwarded message ---------- No: markevich(a)inbox.lv Kam: tss(a)iki.fi Reply-To: markevich(a)inbox.lv Date: Tue, 27 Nov 2007 10:45:23 +0200 Temats: dovecot bug report (64 bit file offsets) Hello. I found 2 issues while using dovecot in the 64-bit system. Before that I was using it on the Solaris 10 32bit and everything worked fine. First issue is that it's impossible to specify Openssl library directory, there is only possiblity to specify type gnutls/openssl. But if there are 2 versions of openssl installed it's hard to specify which one we want to use, we need to play with AUXLIBS and LD flags to prevent looking in un-needed locations. I'm suggesting to add new ./configure key like --with-ssl-dir like all other programs do (why dovecot hasn't it?). For example in FC8 openssl 0.9.8b is available in the packages but latest is 0.9.8.g, and it's impossible to compile dovecot with 0.9.8b because of the missing function in it which present in 0.9.8.g. Second and most serious bug is that it seems that dovecot is unable to use 64 file offsets correctly. First problem was that I as unable to launch dovecot at all - it gave a lot of configuration file (dovecot.conf) parsing errors (which is working perfectly on the 32 bit Solaris). It was very strange because he complained that he is expecting to see '=' in the line where only comments are located. Deleting some lines at the start of the configuration file produced the error on random lines (I can't describe it in other way) but with the comments too. For example, first error complained that line 42 is incorrect (only comments are there), after deletion of 2 empty lines at the beginning dovecot complained that line 172 has errors but any way there are only comments there too. I've deleted all of the comments from the configuration file but still dovecot randomly is giving such errors. Any way dovecot is UNABLE to parse all my file, I can see it from dovecot -n. He is starting normally (time to time) but only half of the configuration file is read. Similar problems goes with the indexes dovecot.index.log - dovecot is able to create the files but is unable to read them after any access to it dovecot generates errors like unexpected eof found and file is corrupted. From all this I can assume that 64bit file offsets on a 64 bit system aren't working at all. Latest dovecot 1.0.7 (also tried 1.0.5) Fedora core 8 64bit (with latest patches and updates) Intel core duo 6300 (x86_64) XFS (only local filesystem) Regards, Andris

2 3

[Dovecot] dovecot with ldap and allow_nets
by Marc Cuypers 27 Nov '07

27 Nov '07

Hi, I'm using dovecot on debian etch: ||/ Name Version ii dovecot-common 1.0.rc15-2etch1 ii dovecot-imapd 1.0.rc15-2etch1 ii dovecot-pop3d 1.0.rc15-2etch1 # dovecot --version 1.0.rc15 Now here is my question. Some of the mail users may only login from the LAN, while others can login from the LAN and the internet. I've read about allow_nets but i can't find very much info when dovecot is used with ldap. Can someone give me a direction (url, configuration file, ...). Thanks in advance. -- Best regards, Marc

3 9