December 2008 - dovecot

[Dovecot] DIGEST-MD5 user/realm mismatch with Postfix
by Darren Pilgrim 22 Feb '09

22 Feb '09

This is a follow-on to the "SQL field format for digest-md5?" thread. After some additional debugging, I've found the problem is a mismatch in what Dovecot expects and Postfix uses for the user and realm. For a username "user(a)example.com" and password "sekret", Dovecot expects: username = "user" realm = "example.com" but Postfix sends the checksum of: username = "user(a)example.com" realm = "" How do I fix this?

2 1

[Dovecot] unkillable imap process(es) with high CPU-usage
by Arno Wald 15 Feb '09

15 Feb '09

Hello, I am having a problem with my dovecot-daemon. It is forking one or more (I saw up to perhaps 8 of them) imap processes under my user name. These processes are consuming a lot of CPU time and are not killable: > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 8616 arno 20 0 2900 1600 1204 R 98 0.2 1196:38 imap Stopping dovecot does not quit these processes. Killing them (even "kill -9" as root) is not possible. The only solution to get rid of them is to reboot. I have found a mailing list post of one who seemed to have the same problem. He solved it by upgrading to version 1.1. But in my case this did not help. Is this a known problem? What could I check or do? My system: very up to date debian/sid (sidux) kernel 2.6.27-8.slh.1-sidux-686 CPU: Intel(R) Core(TM)2 CPU 4300 @ 1.80GHz Filesystem: ext3fs tried dovecot from sid: 1:1.0.15-2.3 tried dovecot from experimental: 1:1.1.2-3 My dovecot.conf is the original debian configuration with only one line changed into: protocols = imaps # dovecot -n # 1.1.2: /etc/dovecot/dovecot.conf log_timestamp: %Y-%m-%d %H:%M:%S protocols: imaps login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login mail_privileged_group: mail auth default: passdb: driver: pam userdb: driver: passwd I am using dovecot locally on my system with me as the only user. As client I am using thunderbird alias icedove 2.0.0.17-1. Icedove retrieves the Mails from another imap server and sorts them into Maildir-folders in dovecot. I do not know when the imap-process are going mad. It happens after the system (and the mail-client) is up for a while. Do you need more information? Thanks, Arno

10 35

[Dovecot] Dovecot performance on GFS clustered filesystem
by Allen Belletti 06 Feb '09

06 Feb '09

Hello All, We are using Dovecot 1.1.3 to serve IMAP on a pair of clustered Postfix servers which share a fiber array via the GFS clustered filesystem. This all works very well for the most part, with the exception that certain operations are so inefficient on GFS that they generate significant I/O load and hurt performance. We are using the Maildir format on disk. We're also using Dovecot's deliver from Postfix to handle local delivery. As best I can determine, the worst problems occur when certain users with very large Inboxes (~10k messages) receive new mail and their client looks up information about that message. GFS doesn't seem to efficiently handle the large directories that contain folders like this. As a result, lots of I/O ops are generated and performance suffers for everyone. I am beginning to wonder if it might be more efficient to revert to the old mbox format, with one file per folder (plus whatever indices are creates.) It seems that this ought to work better with GFS which is geared toward smaller numbers of larger files. Is anyone on the list currently doing that? Alternately, any thoughts regarding tuning or other options would be appreciated. Thanks, Allen -- Allen Belletti allen(a)isye.gatech.edu 404-894-6221 Phone Industrial and Systems Engineering 404-385-2988 Fax Georgia Institute of Technology

3 4

[Dovecot] Courier->dovecot migration script
by Warren Baker 22 Jan '09

22 Jan '09

HI All, We have the latest Dovecot 1.1.6 running and I need to migrate some POP3 users over from Courier to Dovecot and would need to convert the courierpop3dsizelist to maintain the UIDs. I just need to confirm that the script (http://www.dovecot.org/tools/courier-dovecot-migrate.pl) only works for Dovecot v1.0 and not for v1.1? As the dovecout-uidlist file that is created (by the script) is not in the same format as dovecot-uidlist file that is created by the 1.1.6 server. Thanks, Warren

5 8

[Dovecot] Patch to deliver
by Allan Cassaro 16 Jan '09

16 Jan '09

Hi Timo! I wrote a little patch to add the capability of the deliver to customize the "subject" of the rejection messages. Now is possibly to translate the body and subject! In dovecot.conf: rejection_reason_subject = Automatically rejected mail to %t rejection_reason = Your message to <%t> was automatically rejected:%n%r Please, review the patch! :D

4 13

[Dovecot] [dovecot] INDEX variable and mbox_snarf plugin
by Jonathan Siegle 16 Jan '09

16 Jan '09

Is there a way to tell the dovecot mbox_snarf plugin to use an alternate location for the index/cache files? It doesn't seem to want to use the INDEX variable. I'm guessing the answer is no because of all the ties in lib-storage/index/mbox/*c to the directory where the file(that is the inbox) lives. The filesystem where I keep the inbox file doesn't have a directory for the user to own. The user only owns the mbox file. Thanks, Jonathan

2 3

[Dovecot] Apple patch 9
by Mike Abbott 15 Jan '09

15 Jan '09

Patch #9 adds support for Open Directory and an encrypted SSL keystore. Notes about this patch: It adds --enable-apple and --with-od options to configure (after running autoconf). It depends on patch #7 (hash_table_create/destroy). Automake wipes out this patch's changes to Makefile.in. The - framework options should come from Makefile.am but the author didn't do that. It uses a tab-stops-every-4 indentation style. It may compile and run only on Mac OS X 10.6, currently available via Apple Developer Connection. This code is stable but Apple is still verifying its robustness. This is the patch that needs the user name as extra_fields[0] in deliver, to handle user name aliases. More patches to come, probably after the holidays.

3 12

[Dovecot] Panic 1.1.4
by Andre Hübner 14 Jan '09

14 Jan '09

Hi List, try 1.1.4 and have new Panics in Log. Hoped that 1.1.3 finishes this errors... Oct 8 21:56:28 dd10636 dovecot: IMAP(username): Loading modules from directory: /usr/lib/dovecot/imap Oct 8 21:56:28 dd10636 dovecot: IMAP(username): Module loaded: /usr/lib/dovecot/imap/lib20_mail_log_plugin.so Oct 8 21:56:28 dd10636 dovecot: IMAP(username): Effective uid=857, gid=628, home=/home/popuser/username Oct 8 21:56:28 dd10636 dovecot: IMAP(username): mbox: data=~/mail:INBOX=/var/spool/mail/username Oct 8 21:56:28 dd10636 dovecot: IMAP(username): fs: root=/home/popuser/username/mail, index=, control=, inbox=/var/spool/mail/username Oct 8 21:56:28 dd10636 dovecot: imap-login: Login: user=<username>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Oct 8 21:56:28 dd10636 dovecot: Panic: IMAP(username): file istream-limit.c: line 84 (i_stream_limit_read): assertion failed: (ret != -1 || stream->istream.e of || stream->istream.stream_errno != 0) Oct 8 21:56:28 dd10636 dovecot: IMAP(username): Raw backtrace: imap [0x80c3a30] -> imap [0x80c3a8a] -> imap [0x80c338c] -> imap [0x80c9d04] -> imap(i_stream_ read+0x28) [0x80c7e28] -> imap(i_stream_read_data+0x1d) [0x80c7edd] -> imap(message_parse_header_next+0x7c) [0x80bbcdc] -> imap [0x80b9385] -> imap [0x80b9aa7 ] -> imap [0x80b9bb4] -> imap [0x80b9c95] -> imap(i_stream_seek+0x69) [0x80c7db9] -> imap [0x8071763] -> /usr/lib/dovecot/imap/lib20_mail_log_plugin.so [0xb7f a3aec] -> /usr/lib/dovecot/imap/lib20_mail_log_plugin.so [0xb7fa3d56] -> imap(cmd_copy+0x1d2) [0x805a142] -> imap [0x805edfc] -> imap [0x805eea5] -> imap [0x8 05f665] -> imap(client_input+0x5e) [0x805f87e] -> imap(io_loop_handler_run+0x100) [0x80cb590] -> imap(io_loop_run+0x28) [0x80ca6a8] -> imap(main+0x4ac) [0x806 73bc] -> /lib/libc.so.6(__libc_start_main+0xdc) [0xb7e8d87c] -> imap [0x8059191] Oct 8 21:56:28 dd10636 dovecot: child 9120 (imap) killed with signal 6 dovecot -n # 1.1.4: /etc/dovecot.conf protocols: imap imaps pop3 pop3s ssl_ca_file: /path/to/*hostname.com.bundle.crt ssl_cert_file: /path/to/*.hostname.com.crt ssl_key_file: /path/to/*.hostname.com.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_greeting: hostname.com mailserver ready. login_process_per_connection: no login_processes_count: 1 max_mail_processes: 100 verbose_proctitle: yes mail_location: mbox:~/mail:INBOX=/var/spool/mail/%u mail_cache_min_mail_count: 30 mail_debug: yes lock_method: flock mbox_read_locks: dotlock mbox_very_dirty_syncs: yes mail_drop_priv_before_exec: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugins(default): mail_log mail_plugins(imap): mail_log mail_plugins(pop3): mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 imap_client_workarounds(default): outlook-idle netscape-eoh tb-extra-mailbox-sep delay-newmail imap_client_workarounds(imap): outlook-idle netscape-eoh tb-extra-mailbox-sep delay-newmail imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): oe-ns-eoh outlook-no-nuls auth default: verbose: yes passdb: driver: shadow userdb: driver: passwd gdb /usr/libexec/dovecot/imap /home/popuser/username/core GNU gdb 6.4 Copyright 2005 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i586-suse-linux"...Using host libthread_db library "/lib/libthread_db.so.1". Core was generated by `imap [username 127.0.0.1]'. Program terminated with signal 6, Aborted. warning: Can't read pathname for load map: Eingabe-/Ausgabefehler. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /usr/lib/dovecot/imap/lib20_mail_log_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_mail_log_plugin.so Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 Reading symbols from /lib/libgcc_s.so.1...done. Loaded symbols for /lib/libgcc_s.so.1 #0 0xffffe410 in __kernel_vsyscall () (gdb) bt full #0 0xffffe410 in __kernel_vsyscall () No symbol table info available. #1 0xb7ea07d0 in raise () from /lib/libc.so.6 No symbol table info available. #2 0xb7ea1ea3 in abort () from /lib/libc.so.6 No symbol table info available. #3 0x080c3a3d in default_fatal_finish (type=<value optimized out>, status=0) at failures.c:150 backtrace = 0x80f1ae8 "imap [0x80c3a30] -> imap [0x80c3a8a] -> imap [0x80c338c] -> imap [0x80c9d04] -> imap(i_stream_read+0x28) [0x80c7e28] -> imap(i_stream_read_data+0x1d) [0x80c7edd] -> imap(message_parse_header_next+0x7c"... #4 0x080c3a8a in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80d71ac "file %s: line %d (%s): assertion failed: (%s)", args=0xbf941da4 "Sj\016\bT") at failures.c:430 No locals. #5 0x080c338c in i_panic (format=0x80d71ac "file %s: line %d (%s): assertion failed: (%s)") at failures.c:197 args = 0xbf941da4 "Sj\016\bT" #6 0x080c9d04 in i_stream_limit_read (stream=0x8118cc8) at istream-limit.c:83 left = <value optimized out> ret = -1 pos = 65 __PRETTY_FUNCTION__ = "i_stream_limit_read" #7 0x080c7e28 in i_stream_read (stream=0x8118cf0) at istream.c:73 _stream = (struct istream_private *) 0x0 ret = <value optimized out> __PRETTY_FUNCTION__ = "i_stream_read" #8 0x080c7edd in i_stream_read_data (stream=0x8118cf0, data_r=0xbf941e68, size_r=0xbf941e64, threshold=65) at istream.c:299 ret = -1080811928 read_more = false __PRETTY_FUNCTION__ = "i_stream_read_data" #9 0x080bbcdc in message_parse_header_next (ctx=0x81118f0, hdr_r=0xbf941ec0) at message-header-parser.c:89 msg = ( const unsigned char *) 0x811292f "qan4iKSQ1jgszBF7yO1N1FgdflELgOSeaEzr+P+aEXdIbk/ooMFh+wew7C5rzd2W+AfY+P7e\n3el1/64bLf+hadyHUxkt3auVaT3Y6zcoxZaSmY1gEydp4JcyEVNapSNbsWFKYUTSJiju10VF\nkBLd+JYm8PvXkDSplAEe9MRASmX2Jzt/zG6b39O5byfU5IA6JiBj/5"... i = 6 size = 65 startpos = 64 colon_pos = 4294967295 parse_size = 64 value_pos = <value optimized out> ret = 6 continued = false continues = <value optimized out> crlf_newline = false #10 0x080b9385 in read_header (mstream=0x8118da8) at istream-header-filter.c:163 hdr = (struct message_header_line *) 0x0 highwater_offset = <value optimized out> pos = <value optimized out> ret = <value optimized out> matched = false hdr_ret = <value optimized out> __PRETTY_FUNCTION__ = "read_header" #11 0x080b9aa7 in i_stream_header_filter_read (stream=0x8118da8) at istream-header-filter.c:293 mstream = (struct header_filter_istream *) 0x0 ret = <value optimized out> pos = <value optimized out> #12 0x080b9bb4 in parse_header (mstream=0x8118da8) at istream-header-filter.c:333 pos = 73 #13 0x080b9c95 in i_stream_header_filter_seek (stream=0x8118da8, v_offset=1904, mark=false) at istream-header-filter.c:347 mstream = (struct header_filter_istream *) 0x0 #14 0x080c7db9 in i_stream_seek (stream=0x8118dd0, v_offset=1904) at istream.c:144 _stream = (struct istream_private *) 0x8118da8 ---Type <return> to continue, or q <return> to quit--- #15 0x08071763 in mbox_mail_get_physical_size (_mail=0x81142b8, size_r=0xbf942058) at mbox-mail.c:255 mail = <value optimized out> data = (struct index_mail_data *) 0x8114334 input = (struct istream *) 0x8118dd0 hdr_size = {physical_size = 1738, virtual_size = 1771, lines = 33} old_offset = 1904 body_offset = 151200709 body_size = <value optimized out> next_offset = 148393877 #16 0xb7fa3aec in mail_log_action (dest_trans=<value optimized out>, mail=0x81142b8, event=<value optimized out>, data=0x810f4f8 "Trash") at mail-log-plugin.c:305 lt = (struct mail_log_transaction_context *) 0x811c708 msgid = 0x8114bb4 "<20081008182128.D148E68E1BECD(a)relay1.senderdomain.net>" size = <value optimized out> str = (string_t *) 0x80f1858 pool = <value optimized out> #17 0xb7fa3d56 in mail_log_copy (t=0x810ff00, mail=0x81142b8, flags=MAIL_SEEN, keywords=0x0, dest_mail=0x0) at mail-log-plugin.c:400 _data_stack_cur_id = 4 name = <value optimized out> #18 0x0805a142 in cmd_copy (cmd=0x80fb9c8) at cmd-copy.c:66 client = (struct client *) 0x80fa548 storage = (struct mail_storage *) 0x80fb108 destbox = (struct mailbox *) 0x810f358 t = (struct mailbox_transaction_context *) 0x810ff00 search_arg = <value optimized out> messageset = 0x80ffac0 "737" mailbox = 0x80ffac8 "Trash" src_uidset = <value optimized out> msg = <value optimized out> sync_flags = <value optimized out> imap_flags = <value optimized out> copy_count = 1 uid_validity = <value optimized out> uid1 = <value optimized out> uid2 = <value optimized out> ret = 1 __PRETTY_FUNCTION__ = "cmd_copy" #19 0x0805edfc in client_command_input (cmd=0x80fb9c8) at client.c:580 client = (struct client *) 0x80fa548 command = <value optimized out> __PRETTY_FUNCTION__ = "client_command_input" #20 0x0805eea5 in client_command_input (cmd=0x80fb9c8) at client.c:629 client = (struct client *) 0x80fa548 command = (struct command *) 0x23a0 __PRETTY_FUNCTION__ = "client_command_input" #21 0x0805f665 in client_handle_input (client=0x80fa548) at client.c:670 _data_stack_cur_id = 3 ret = <value optimized out> remove_io = <value optimized out> handled_commands = false #22 0x0805f87e in client_input (client=0x80fa548) at client.c:725 cmd = <value optimized out> output = (struct ostream *) 0x80fb904 bytes = 23 __PRETTY_FUNCTION__ = "client_input" #23 0x080cb590 in io_loop_handler_run (ioloop=0x80f99b0) at ioloop-epoll.c:203 ctx = <value optimized out> event = (const struct epoll_event *) 0x80f9ae8 list = (struct io_list *) 0x80fa5b8 ---Type <return> to continue, or q <return> to quit--- io = (struct io_file *) 0x80fb960 tv = {tv_sec = 1799, tv_usec = 999127} t_id = 2 msecs = <value optimized out> ret = 1 i = 0 j = 0 call = <value optimized out> #24 0x080ca6a8 in io_loop_run (ioloop=0x80f99b0) at ioloop.c:320 No locals. #25 0x080673bc in main (argc=Cannot access memory at address 0x23a0 ) at main.c:293 No locals. Thanks Andre

2 2

[Dovecot] Maildir loses keywords in index beyond 26
by George Greer 14 Jan '09

14 Jan '09

(NOTE: I am not on the mailing list so please Cc: me.) dovecot-1.1.6-2.fc10.i386 (configuration not changed from stock) Fedora 10 AMD Athlon XP ext3 Maildir format doesn't store keywords in the index properly -- it goes up to 26 and then further keywords are dropped. I ran the tests with a simple script that feeds the lines from "commands" to Dovecot one at a time and waits for the "OK" before the next one. The IMAP command file is attached, as well as output of dovecot, idxview, and logview running against both mbox and maildir formats. Test mailbox created with: for i in `seq -f %02g 0 99`; do echo "Test message #$i" | mail -s "Test message #$i" sandbox@localhost; sleep 1; done For the maildir test I used the conversion script: Step 2: su sandbox -c 'perl /usr/share/doc/dovecot-1.1.6/UW-to-Dovecot-Migration/perfect_maildir.pl ~/Maildir < /var/spool/mail/sandbox' # 1.1.6: /etc/dovecot.conf # OS: Linux 2.6.25.6-55.fc9.i686 i686 Fedora release 10 (Cambridge) login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 auth default: passdb: driver: pam userdb: driver: passwd -- George Greer

2 1

[Dovecot] linuxconf migration story
by Thanos Chatziathanassiou 14 Jan '09

14 Jan '09

Hi, just subscribed after successfully (?) migrating from a somewhat old linuxconf/vimap config. dovecot-1.1.7, x86 A few issues: The index files seem to be created with the wrong permissions. I get numerous ``Dec 24 15:36:29 mail dovecot: POP3(user@domain): stat(/var/run/dovecot/imap-index/domain/user/.imap/INBOX) failed: Permission denied (euid=60107 egid=231 missing +x perm: /var/run/dovecot/imap-index/domain)'' errors. the directory /var/run/dovecot/imap-index is drwxrwxrwt 29 dovecot dovecot 4.0k Dec 24 15:57 imap-index but the directories inside it are created as drwx------ 15 60107 popusers 4.0k Dec 24 15:50 domain Questions: - Why is POP3 interested in the imap indices ? - IMAP clients do not complain all that much, as far as I can see, but what happens to POP3 ? - Is there something to configure to get rid of this ? Also, I have written a somewhat sloppy patch to determine the domain from the server's IP. That way it is not necessary to persuade all users to switch to logging in as ``user@domain'' (which I've tried with very limited success). Would this be the correct place to post the patch and discuss ? Best Regards, Thanos Chatziathanassiou

2 4