January 2007 - dovecot

[Dovecot] dovecot imap logging capabilities
by Michael A. Painter Jr. 18 Jan '07

18 Jan '07

Hello, I have searched but have not found any information on logging imap actions. I want to log when someone uses imap to delete, move, copy, etc. This would greatly help me when someone deletes an email and asks where it went. I get this fairly often from a few users. Any help would be greatly appreciated. Thank you, Michael A. Painter Jr.

4 4

[Dovecot] A few rc17 imap crashes
by Chris Wakelin 18 Jan '07

18 Jan '07

We upgraded from 1.0-beta7 to 1.0-rc17 on Tuesday evening and looking at today's logs, just four imap processes out of 90,000+ died. I managed to get core files (the server is running Solaris 8 on Sparc, and Dovecot was compiled with gcc 3.3.2). I hope they're useful! (I should also say that performance between the two versions is roughly similar according to Solaris' process accounting stats, looking at CPU usage and disk blocks accessed.) Best Wishes, Chris 1) Here's the last log entry for each > dovecot: Jan 11 11:39:59 Info: IMAP 5751 xxxxxxxx xxx.xxx.xxx.xxx : Opened Sent (18 msgs) > dovecot: Jan 11 11:39:59 Error: child 5751 (imap) killed with signal 11 > dovecot: Jan 11 12:28:49 Info: IMAP 12765 xxxxxxxx xxx.xxx.xxx.xxx : Disconnected: Internal error occurred. Refer to server log for more information. [2007-01-11 12:28:49] > dovecot: Jan 11 12:28:49 Error: child 12765 (imap) killed with signal 10 > dovecot: Jan 11 13:01:49 Info: IMAP 14234 xxxxxxxx xxx.xxx.xxx.xxx : Opened List Reading (124 msgs) > dovecot: Jan 11 13:01:54 Error: child 14234 (imap) killed with signal 10 > dovecot: Jan 11 19:13:55 Info: IMAP 18786 xxxxxxxx xxx.xxx.xxx.xxx : Opened Sent (45 msgs) > dovecot: Jan 11 19:13:57 Error: child 18786 (imap) killed with signal 11 (I have extra logging lines in cmd-select.c, hence the "Opened" lines) 2) backtraces! pid 5751 > #0 0xff1c2838 in t_delete () from /usr/lib/libc.so.1 > #1 0xff1c23e4 in realfree () from /usr/lib/libc.so.1 > #2 0xff1c1f48 in _malloc_unlocked () from /usr/lib/libc.so.1 > #3 0xff1c1ce0 in malloc () from /usr/lib/libc.so.1 > #4 0xff1b617c in calloc () from /usr/lib/libc.so.1 > #5 0x89c18 in pool_system_malloc (pool=0xb66f0, size=156) at mempool-system.c:67 > #6 0x84ab8 in i_malloc (size=156) at imem.c:9 > #7 0x43b2c in mbox_md5_init () at mbox-md5.c:94 > #8 0x3df20 in mbox_sync_parse_match_mail (mbox=0xcee58, view=0xcdde8, seq=2) at mbox-sync-parse.c:557 > #9 0x3966c in mbox_file_seek (mbox=0xcee58, view=0xcdde8, seq=2, deleted_r=0xffbeee47) at mbox-file.c:195 > #10 0x3b628 in mbox_mail_seek (mail=0xd0bf8) at mbox-mail.c:69 > #11 0x3b988 in mbox_mail_get_stream (_mail=0xd0bf8, hdr_size=0x0, body_size=0x0) at mbox-mail.c:184 > #12 0x70458 in mail_get_stream (mail=0x0, hdr_size=0x0, body_size=0x0) at mail.c:107 > #13 0x5321c in index_mail_get_header_stream (_mail=0xd0bf8, _headers=0xd1368) at index-mail-headers.c:638 > #14 0x70434 in mail_get_header_stream (mail=0x0, headers=0xd1368) at mail.c:98 > #15 0x529ec in index_mail_headers_get_envelope (mail=0xd0bf8) at index-mail-headers.c:408 > #16 0x515c8 in index_mail_get_special (_mail=0xd0bf8, field=MAIL_FETCH_IMAP_ENVELOPE) at index-mail.c:706 > #17 0x70474 in mail_get_special (mail=0xd0bf8, field=MAIL_FETCH_IMAP_ENVELOPE) at mail.c:114 > #18 0x254d0 in fetch_envelope (ctx=0xd0bf8, mail=0xd0bf8, context=0x0) at imap-fetch.c:450 > #19 0x24e80 in imap_fetch (ctx=0xc4a58) at imap-fetch.c:291 > #20 0x1f62c in cmd_fetch (cmd=0xc27a4) at cmd-fetch.c:163 > #21 0x23334 in client_handle_input (cmd=0xc27a4) at client.c:331 > #22 0x232b4 in client_handle_input (cmd=0xc27a4) at client.c:388 > #23 0x23430 in _client_input (context=0xc2760) at client.c:428 > #24 0x87924 in io_loop_handler_run (ioloop=0xbf838) at ioloop-poll.c:199 > #25 0x87184 in io_loop_run (ioloop=0xbf838) at ioloop.c:281 > #26 0x2c658 in main (argc=0, argv=0xffbef5ec, envp=0xffbef5fc) at main.c:280 pid 12765 > #0 0xff1c27f0 in t_delete () from /usr/lib/libc.so.1 > #1 0xff1c23e4 in realfree () from /usr/lib/libc.so.1 > #2 0xff1c2c24 in _free_unlocked () from /usr/lib/libc.so.1 > #3 0xff1c2b74 in free () from /usr/lib/libc.so.1 > #4 0x5b550 in mail_index_free (_index=0xcac54) at mail-index.c:62 > #5 0x552f0 in index_list_free (list=0xcac50) at index-storage.c:84 > #6 0x555fc in destroy_unrefed (all=true) at index-storage.c:162 > #7 0x379ac in mbox_free (_storage=0xc2598) at mbox-storage.c:369 > #8 0x711a4 in mail_storage_destroy (_storage=0xc1a2c) at mail-storage.c:204 > #9 0x2cbd8 in namespace_deinit (namespaces=0xc1a10) at namespace.c:156 > #10 0x22a00 in client_destroy (client=0xc2760, reason=0x0) at client.c:90 > #11 0x87924 in io_loop_handler_run (ioloop=0xbf838) at ioloop-poll.c:199 > #12 0x87184 in io_loop_run (ioloop=0xbf838) at ioloop.c:281 > #13 0x2c658 in main (argc=0, argv=0xffbef604, envp=0xffbef614) at main.c:280 pid 14234 > #0 0xff1c2838 in t_delete () from /usr/lib/libc.so.1 > #1 0xff1c23e4 in realfree () from /usr/lib/libc.so.1 > #2 0xff1c2cb8 in cleanfree () from /usr/lib/libc.so.1 > #3 0xff1c1dec in _malloc_unlocked () from /usr/lib/libc.so.1 > #4 0xff1c1ce0 in malloc () from /usr/lib/libc.so.1 > #5 0xff1b617c in calloc () from /usr/lib/libc.so.1 > #6 0x89c18 in pool_system_malloc (pool=0xb66f0, size=28) at mempool-system.c:67 > #7 0x86d7c in timeout_add (msecs=1740000, callback=0x1f920 <idle_timeout>, context=0xc4a18) at ioloop.c:146 > #8 0x1fda0 in cmd_idle (cmd=0xc27a4) at cmd-idle.c:245 > #9 0x23334 in client_handle_input (cmd=0xc27a4) at client.c:331 > #10 0x232b4 in client_handle_input (cmd=0xc27a4) at client.c:388 > #11 0x23430 in _client_input (context=0xc2760) at client.c:428 > #12 0x87924 in io_loop_handler_run (ioloop=0xbf838) at ioloop-poll.c:199 > #13 0x87184 in io_loop_run (ioloop=0xbf838) at ioloop.c:281 > #14 0x2c658 in main (argc=0, argv=0xffbef5fc, envp=0xffbef60c) at main.c:280 pid 18786 > #0 0xff1c2974 in t_splay () from /usr/lib/libc.so.1 > #1 0xff1c27e0 in t_delete () from /usr/lib/libc.so.1 > #2 0xff1c23e4 in realfree () from /usr/lib/libc.so.1 > #3 0xff1c2cb8 in cleanfree () from /usr/lib/libc.so.1 > #4 0xff1c1dec in _malloc_unlocked () from /usr/lib/libc.so.1 > #5 0xff1c1ce0 in malloc () from /usr/lib/libc.so.1 > #6 0xff1b617c in calloc () from /usr/lib/libc.so.1 > #7 0x89c18 in pool_system_malloc (pool=0xb66f0, size=28) at mempool-system.c:67 > #8 0x7fa58 in buffer_create_dynamic (pool=0xb66f0, init_size=20) at buffer.c:109 > #9 0x56924 in index_transaction_init (t=0xcf090, ibox=0xd03a8, flags=0) at index-transaction.c:13 > #10 0x38fa0 in mbox_transaction_begin (box=0xcf090, flags=0) at mbox-transaction.c:16 > #11 0x71954 in mailbox_transaction_begin (box=0xcf090, flags=0) at mail-storage.c:497 > #12 0x29f30 in imap_sync_init (client=0xc2760, box=0xd03a8, imap_flags=0, flags=28) at imap-sync.c:47 > #13 0x2a534 in cmd_sync (cmd=0xc27a4, flags=28, imap_flags=0, tagline=0x93120 "OK Fetch completed.") at imap-sync.c:243 > #14 0x1f470 in cmd_fetch_finish (ctx=0xc4a50) at cmd-fetch.c:103 > #15 0x1f66c in cmd_fetch (cmd=0xc27a4) at cmd-fetch.c:173 > #16 0x226b4 in cmd_uid (cmd=0xc27a4) at cmd-uid.c:19 > #17 0x23334 in client_handle_input (cmd=0xc27a4) at client.c:331 > #18 0x232b4 in client_handle_input (cmd=0xc27a4) at client.c:388 > #19 0x23430 in _client_input (context=0xc2760) at client.c:428 > #20 0x87924 in io_loop_handler_run (ioloop=0xbf838) at ioloop-poll.c:199 > #21 0x87184 in io_loop_run (ioloop=0xbf838) at ioloop.c:281 > #22 0x2c658 in main (argc=0, argv=0xffbef5ec, envp=0xffbef5fc) at main.c:280 -- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, c.d.wakelin(a)reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094

3 15

[Dovecot] default_mail_env/mail_location syntax changes
by Mike Brown 18 Jan '07

18 Jan '07

Sorry, I tried to find this info on the wiki and in the docs, but couldn't quite figure it out. Dovecot 1.0rc8 introduced this change: "mbox:/var/mail/%u" no longer works as the mail location. You'll have to specify the mail root explicitly, just like the examples always have: "mbox:~/mail:INBOX=/var/mail/%u" If I need a custom INBOX location but the default behavior of using the first of [/ or ~/mail or ~/Mail] having a .imap subdirectory, how do I go about specifying that? That is, on my system, - inbox file: ~/Mailbox - mail storage dir: ~/mail OR ~/Mail (it varies by user) - all files are in mbox format Thanks, Mike

2 3

[Dovecot] Dovecot Sieve capabilties - regex
by James Turnbull 18 Jan '07

18 Jan '07

Hi Does the Dovecot Sieve implementation support the regex capability? I get the following error when I try to add it: Jan 3 16:07:30 absinthe deliver(james): sieve parse error: line 1: Unsupported features in require line The line is: require "regex"; Thanks James Turnbull -- James Turnbull <james(a)lovedthanlost.net> --- Author of Pro Nagios 2.0 (http://www.amazon.com/gp/product/1590596099/) Hardening Linux (http://www.amazon.com/gp/product/1590594444/) --- PGP Key (http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x0C42DF40)

3 7

[Dovecot] some Maildir files not honoring dovecot-shared
by Glenn Leavell 18 Jan '07

18 Jan '07

I'm using RC15 on Solaris 9. The wiki indicates that when a dovecot-shared file exists, "Dovecot uses the same permissions and GID when it creates index files, dovecot-uidlist file, new mail files, etc." In my case, Dovecot *is* creating new index files based on dovecot-shared, but dovecot-uidlist and individual message files don't completely match the permissions/group template. Specifically, dovecot-uidlist isn't taking the group from dovecot-uidlist, and individual messages files don't seem to be honoring dovecot-shared at all (group or permissions). Here's an example; every file and directory below in .abusetest-in was created by Dovecot *except* for dovecot-shared (I let Dovecot create cur, new, tmp just to see what it would do, but I know that I can create and fix the group and permissions on these directories myself in a sharing situation): glenn@tony:~/Maildir/.abusetest-in$ ls -lR .: total 34 drwx------ 2 glenn adm 512 Jan 9 09:58 cur/ -rwxrwx--- 1 glenn abuse 0 Jan 9 09:57 dovecot-shared* -rw-rw---- 1 glenn adm 44 Jan 9 09:58 dovecot-uidlist -rw-rw---- 1 glenn abuse 156 Jan 9 09:58 dovecot.index -rw-rw---- 1 glenn abuse 10272 Jan 9 09:58 dovecot.index.cache -rw-rw---- 1 glenn abuse 156 Jan 9 09:58 dovecot.index.log drwx------ 2 glenn adm 512 Jan 9 09:58 new/ drwx------ 2 glenn adm 512 Jan 9 09:58 tmp/ ./cur: total 4 -rw------- 2 glenn adm 1438 Jan 5 09:59 1168354728.P15661Q0M891161.tony:2,S ./new: total 0 ./tmp: total 0 glenn@tony:~/Maildir/.abusetest-in$ I see in the RC16 release notes that there was a fix for the dovecot-keywords file, but it doesn't mention these other files. Is this a bug, or am I misunderstanding how the dovecot-shared file should work? Thanks very much, Glenn -- Glenn Leavell <glenn(a)usg.edu> Office of Information and Instructional Technology Board of Regents of the University System of Georgia

3 4

[Dovecot] assertion failed
by Shaun T. Erickson 18 Jan '07

18 Jan '07

I installed dovecot on a RH9 server this evening, via an RPM from atrpms.net (dovecot-1.0-0_34.rc15.RHL9.at.i386.rpm). The auth portion is working perfectly, as I'm using it to do SMTP AUTH with my postfix 2.3.5 installation. I have configured it to provide imaps. When I connect to it, it rapidly connects, disconnects, connects ... ad infinitum, reporting this, in the logs, each time: Dec 30 16:03:33 max dovecot: imap-login: Login: user=<ste>, method=plain, rip=192.168.32.168, lip=65.206.7.147, TLS Dec 30 16:03:33 max dovecot: IMAP(ste): file istream-raw-mbox.c: line 499 (istream_raw_mbox_get_body_size): assertion failed: (rstream->mail_size != (uoff_t)-1) Dec 30 16:03:33 max dovecot:child 19228 (imap) killed with signal 6 Here is my dovecot.conf file: listen = [*] protocols = imaps disable_plaintext_auth = yes ssl_cert_file = /etc/postfix/max.blackdogsoft.net.crt ssl_key_file = /etc/postfix/max.blackdogsoft.net.key ssl_ca_file = /etc/postfix/cacert.pem default_mail_env = mbox:%h:INBOX=/var/mail/%u mail_read_mmaped = yes verbose_proctitle = yes protocol lda { postmaster_address = postmaster(a)blackdogsoft.net } auth default { mechanisms = plain login passdb pam { } userdb passwd { } socket listen { client { path = /var/spool/postfix/provate/auth mode = 0660 user = postfix group = postfix } } } Suggestions? -- -ste

3 10

[Dovecot] POP3 timeout
by Jason Gretz 18 Jan '07

18 Jan '07

Hey guys I have a user's machine that won't drop his POP3 connection, is there a way to set a timeout? I'm not seeing it in the default config or in the webmin module for dovecot, any ideas? Thanks! Jason Gretz Network Admin Angstman Law 208-850-9284 https://companyweb/

2 1

[Dovecot] Correct usage of NFS mounted Maildirs
by Fran Fabrizio 18 Jan '07

18 Jan '07

Dovecot 1.0RC15 running on Fedora Core 5 kernel 2.6.18-1.2257.fc5smp. A SLES10 server (kernel 2.6.16.21-0.8-smp) has the home directories on an ext3 partition mounted with options (rw,acl,user_xattr,usrquota) which is then exposed as an NFS mount with server side options (rw,no_root_squash,no_acl,async). The home directories include Maildirs to which postfix delivers. The Fedora Core 5 dovecot&postfix machine then mounts this NFS mount with options (rw,hard,intr) Dovecot variables: mmap_disable=yes lock_method=dotlock default_mail_env = maildir:~/Maildir:INDEX=/usr/local/dovecot/indexes/%u (only one dovecot server and wanted to see if local indices helped) I didn't turn off the attribute caching of NFS since I only have one dovecot server, I think the wiki said it was safe to leave on in this case. With this configuration, I am seeing a lot of NFS Stale Filehandle errors, even after moving the index files to a local disk. Here's an excerpt from the log: dovecot: Dec 22 11:23:06 Info: Dovecot v1.0.rc15 starting up dovecot: Dec 22 11:23:22 Info: imap-login: Login: user=<nfsuser1>, method=plain, rip=138.26.64.50, lip=138.26.64.5 dovecot: Dec 22 11:23:24 Info: imap-login: Login: user=<nfsuser1>, method=plain, rip=138.26.64.50, lip=138.26.64.5 dovecot: Dec 22 11:24:48 Info: imap-login: Login: user=<nfsuser1>, method=plain, rip=138.26.64.50, lip=138.26.64.5 dovecot: Dec 22 11:26:43 Error: IMAP(nfsuser1): inotify_add_watch(/nethome/facstaff/nfsuser1/Maildir/.Trash/new) failed: Stale NFS file handle dovecot: Dec 22 11:26:43 Error: IMAP(nfsuser1): inotify_add_watch(/nethome/facstaff/nfsuser1/Maildir/new) failed: Stale NFS file handle To be honest, I'm not 100% sure this is a dovecot problem, because I also see that I am getting stale filehandles when postfix tries to deliver the mail to the Maildir, but I'm curious as to whether this is because of something dovecot did because I have it misconfigured. Does anything about my dovecot config provide any clues as to what I might have done wrong? The NFS mounts are completely stable everywhere except within the Maildirs. From the shell, I can run a loop to call 'ls' 1000 times outside of a Maildir and it works fine. Inside a Maildir, I will get Stale NFS Filehandle errors before long. Does this sound like it could have something to do with my Dovecot config, given that it is happening even after I moved indices to the local disk (still getting stale filehandle errors on files like dovecot-keywords). I performed the following procedure: - shut down dovecot - did a rm-rf Maildir on my test account - unmounted the NFS mount /nethome/facstaff - remounted the NFS mount /nethome/facstaff - set the index files to be local in dovecot.conf and created that directory (/usr/local/dovecot/index/) - sent a few test mails to the user (to recreate the Maildir) - started dovecot - logged in (via Thunderbird) to account through dovecot. - read test message - deleted test message - opened shell to dovecot server - cd'ed into user's Maildir - ran 'ls' a few times. Got Stale Filehandle errors after a few. - back in Thunderbird, tried to read/move/delete other test mails, got the above errors in the log. If it's not a dovecot issue I know it's off-topic, but has anyone else seen similar? I might be imagining this, but I think the problem started when I moved from SLES9 to SLES10 for my fileserver (newer kernel, newer version of NFS, perhaps an incompatibility here?) Thanks, Fran

2 1

[Dovecot] Custom mail directory with LDAP
by Maykel Moya 18 Jan '07

18 Jan '07

For certain users, I have their mail boxes under /srv/mpop/<username>/Maildir. For the rest, I have their mail boxes under /srv/vmail/<username>/Maildir. mpop/ users are stored in LDAP, vmail/ users are local users so I discard the idea of set a fixed mail_location. In dovecot-ldap.conf I can't get the picture to set /srv/mpop/<username>/Maildir as mail. I'd tried user_attrs = /srv/mpop/uid/Maildir=mail but watching the logs I saw that a '/srv/mpop/uid/Maildir' is queried to the LDAP Server. Would be nice to have a user_global_mail equivalent to user_global_uid for LDAP user lookups. Regards, maykel

2 1

[Dovecot] auth unix sockets
by Uldis Pakuls 18 Jan '07

18 Jan '07

Hi, I'm working on "avelsieve" SqurrelMail plug-in to implement SIEVE "file" back end for dovecot. 1. My be someone my explain me, how to use client auth socket to authenticate against dovecot-auth unix socket. As fas as I understood login must be sent in form "user\0\user\0password" as base64 encoded string. pp code (I'm using socket used by postfix auth): -------------------- <?php $authstr = base64_encode('test@virtual.net\0test@virtual.net\0test'); $fp = fsockopen("unix:///var/spool/postfix/private/auth",0, $errno, $errstr, 30); if (!$fp) { echo "$errstr ($errno)<br />\n"; } else { $out = "$authstr\r\n"; fwrite($fp, $out); while (!feof($fp)) { echo fgets($fp, 128); } fclose($fp); } ?> ----------------------- Dovecot auth complains about syntax It seems 'auth mech' also is required. thus; how to correctly use dovecot client auth socket to authenticate? (some FAQ?, php sample code?) 2. [most important] dovecot LDA uses auth maser sockets to read userdb and get home directory path to deliver mail and locate sieve scripts. I need some home path to locate users sieve script. Wow to obtain it using dovecot auth sockets? Tnx Uldis

3 2