- dovecot - dovecot.org

Re: [Dovecot] Search Problem (Was Namespace)
by Jeff Grossman 21 Oct '07

21 Oct '07

-----Original Message----- From: "Timo Sirainen" <tss(a)iki.fi> To: "Jeff Grossman" <jeff(a)stikman.com> Cc: "Dovecot Mailing List" <dovecot(a)dovecot.org> Sent: 10/20/2007 6:51 PM Subject: Re: [Dovecot] Search Problem (Was Namespace) On 21.10.2007, at 4.21, Jeff Grossman wrote: > Timo Sirainen wrote: >> On 21.10.2007, at 3.58, Jeff Grossman wrote: >> >>> 2 search since 6-Sep-2007 >> .. >>> Oct 20 17:54:47 apple IMAP(jeff): : since: 1190593264 vs 2147483647 >>> Oct 20 17:54:47 apple IMAP(jeff): : since: 1190829893 vs 2147483647 >> >> I guess utc_mktime() doesn't work correctly for some reason. What >> are TIME_T_MAX_BITS and TIME_T_SIGNED in your config.h? You've >> 32bit system? >> > Yes, this is a 32-bit system. I did test it with TIME_T_MAX_BITS=32, but looks like I never tested it with actual 32bit systems. This should fix it: http:// hg.dovecot.org/dovecot/rev/4fb613eb8ce9 It is all fixed. Thank you.

1 0

[Dovecot] Upgrade from 1.0.1 -> 1.0.5 failed (rc 75)
by Andy Shellam (Mailing Lists) 21 Oct '07

21 Oct '07

Hi, I'm using a mixture of Exim, Dovecot and Deliver. Exim receives the mail and validates the user against our database, then pipes the e-mail to deliver which puts it in the right mailbox. We were running Exim 4.67 and Dovecot 1.0.1, and today I did an in-place upgrade to Dovecot 1.0.5 and Exim 4.68. However, now Exim is failing deliveries saying deliver returned code 75. If I examine the dovecot log file, I'm getting errors like: deliver(andy.shellam-lists(a)mailnetwork.co.uk): Oct 21 17:19:49 Fatal: chroot(/users/mail/andy.shellam-lists(a)mailnetwork.co.uk) failed: Operation not permitted All mailboxes are owned by user "dovecot-mail" and group "localservice", with user and group having full permissions, and other having no access (0770). None of the config files (Exim nor Dovecot's) have changed, nor have permissions on the mailbox paths, and Dovecot can still access the mailboxes through IMAP just fine. If I delete the dovecot log-file and force a mail delivery, the log gets re-created with mode 0644 and owner dovecot-auth:localservice, which confirms the user running dovecot is the same group, so why doesn't it have permissions? I've read on another message that return code 75 is EX_TEMPFAIL, what does this mean? My sendmail_path is the correct path to Exim, verified by killing Exim and copying and pasting the command to start it back up again. Any advice would be appreciated, as I may have to restore back to 1.0.1. Thanks, Andy dovecot -n: # 1.0.5: /usr/local/dovecot/etc/dovecot.conf base_dir: /var/run/dovecot/ log_path: /mndata/logs/dovecot.current.log protocols: imap ssl_disable: yes disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/local/dovecot/libexec/dovecot/imap-login login_user: dovecot-login login_greeting: SystemVDM2 ready. login_processes_count: 5 valid_chroot_dirs: /mndata/mail mail_chroot: /mndata/mail first_valid_uid: 1013 last_valid_uid: 1013 first_valid_gid: 1002 last_valid_gid: 1002 mail_extra_groups: localservice mailbox_idle_check_interval: 60 maildir_copy_with_hardlinks: yes namespace: type: private separator: . prefix: INBOX. inbox: yes auth default: mechanisms: plain login default_realm: mailnetwork.co.uk user: dovecot-auth username_translation: +@ username_format: %Lu passdb: driver: sql args: /usr/local/dovecot/etc/dovecot-sql.conf userdb: driver: sql args: /usr/local/dovecot/etc/dovecot-sql.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 432 user: dovecot-auth group: localservice

3 3

[Dovecot] Problem building Dovecot from hg
by Asheesh Laroia 21 Oct '07

21 Oct '07

I did an hg pull; hg update, and now my Dovecot working copy doesn't build: ../../src/lib-storage/index/dbox/libstorage_dbox.a(dbox-file.o): In function `dbox_file_metadata_get': /home/paulproteus/dnlds/dovecot/src/lib-storage/index/dbox/dbox-file.c:820: undefined reference to `dbox_file_maildir_metadata_get' collect2: ld returned 1 exit status make[3]: *** [imap] Error 1 make[3]: Leaving directory `/home/paulproteus/dnlds/dovecot/src/imap' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/home/paulproteus/dnlds/dovecot/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/paulproteus/dnlds/dovecot' make: *** [all] Error 2 I made it compile on my system by commenting out the return on that line and making it return NULL, but presumably you want it to actually do something. (-: -- Asheesh. -- Virtue does not always demand a heavy sacrifice -- only the willingness to make it when necessary. -- Frederick Dunn

2 1

[Dovecot] Segfault by asking for a folder?
by Asheesh Laroia 21 Oct '07

21 Oct '07

I 'hg pull'ed and now my Dovecot segfaults when I select a particular folder. I've removed all files with 'index' in the name, so I don't think it's due to bad indexes. Oct 21 09:18:40 alchemy dovecot: Dovecot v1.1.beta3 starting up Oct 21 09:18:47 alchemy dovecot: imap-login: Login: user=<paulproteus>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, TLS Oct 21 09:18:47 alchemy dovecot: child 28714 (imap) killed with signal 11 is in syslog. I haven't stripped the binaries as far as I know, yet here's what seems to happen in gdb: paulproteus@alchemy:~/dovecot.rawlog $ gdb dovecot [snip] This GDB was configured as "i486-linux-gnu"... Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1". (gdb) set args --exec-mail imap (gdb) run Starting program: /usr/local/sbin/dovecot --exec-mail imap * PREAUTH [CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS UIDPLUS LIST-EXTENDED] Logged in as paulproteus 01 SELECT pinerc Program received signal SIGSEGV, Segmentation fault. Cannot remove breakpoints because program is no longer writable. It might be running in another process. Further execution is probably impossible. 0x08073d0d in ?? () (gdb) bt #0 0x08073d0d in ?? () #1 0xbff92f70 in ?? () #2 0x00000000 in ?? () (gdb) bt full #0 0x08073d0d in ?? () No symbol table info available. #1 0xbff92f70 in ?? () No symbol table info available. #2 0x00000000 in ?? () No symbol table info available. (gdb) Uh? It seems to also segfault the same way if I select INBOX. -- Asheesh. -- You'll never see all the places, or read all the books, but fortunately, they're not all recommended.

1 2

[Dovecot] dovecot? putting wrong seperator in subscriptions file
by Adam McDougall 21 Oct '07

21 Oct '07

My config has folders show up in the root namespace as well as hidden ones like mail/ for compat. In thunderbird when I set the namespace prefix to mail/, I effectively cannot subscribe to folders. You can pull up the folder list, select some folders to subscribe to, hit okay, and nothing changes in thunderbird: 3 subscribe "mail/Spam" 3 OK Subscribe completed. 4 lsub "" "mail/*" 4 OK Lsub completed. What gets put in my subscriptions file: mail.Spam If I manually change my subscriptions file to mail/Spam, it works as expected. # dovecot -n # 1.1.beta3: /usr/local/etc/dovecot.conf ssl_cert_file: /usr/local/etc/apache2/ssl/cert.pem ssl_key_file: /usr/local/etc/apache2/ssl/cert.pem disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 0 mail_location: maildir:%h/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_plugins: mail_log mail_log_max_lines_per_sec: 0 imap_client_workarounds: delay-newmail netscape-eoh tb-extra-mailbox-sep imap_logout_format: bbytes=%i/%o username=%u namespace: type: private separator: / prefix: mail/ location: maildir:%h/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes hidden: yes namespace: type: private separator: / inbox: yes list: yes auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 384 user: postfix group: postfix plugin: acl: vfile:/usr/local/etc/dovecot-acls

2 4

[Dovecot] two small low importance issues in 1.1b
by Adam McDougall 21 Oct '07

21 Oct '07

Item 1: I store the indexes and control in their own directory. I noticed in 1.1: 1192 ./private/indexes/.INBOX 32 ./private/indexes/.Trash 1228 ./private/indexes 4 ./private/control/.INBOX/.INBOX 84 ./private/control/.INBOX 4 ./private/control/.Trash/.Trash 12 ./private/control/.Trash 100 ./private/control 1332 ./private .INBOX/.INBOX and .Trash/.Trash are both empty. This was an empty directory, I logged in with thunderbird and looked at the inbox and Trash folders. Item 2: I wanted to try imap_logout_format, forgetting what params it uses and taking a stab at it, but it seems to be ignored? It shows up in dovecot -n. I made it 'bbytes' just to see if the intentional typo would come through, but no: imap_logout_format: bbytes=%i/%o username=%u dovecot: IMAP(mcdouga9): Disconnected: Logged out bytes=439/169567

2 2

[Dovecot] dovecot 1.1.beta3 crashes on NetBSD/sparc64 4.0_RC3 with Thunderbird 2.0.0.6
by Lloyd Parkes 21 Oct '07

21 Oct '07

Hi all, I'm getting core dumps from Dovecot when using Thunderbird as a client. Everything is fine when using the Apple Mail client. I've appended the debug dumps to this message so they are out of the way, but they should be read now. I think the problem might be that io->prev and io->next aren't reliably initialised in ioloop.c:io_add. The value that io->prev has when the program crashes is just completely bogus. It isn't aligned correctly, and it doesn't point to anywhere inside the process's memory map. I don't know what *io->next is, but it sure doesn't look like a struct io_file. It doesn't even look like a struct io. Part of the garbage in *io->next is the char sequence "/vol/home/lloyd/Maildir/new" which is part of my mail box. *io is not in the doubly linked list in current_io_look->io_files. I'm guessing that this object has been unlinked already. Some further digging (not shown here) shows that io->callback is notify_callback. It is set to NULL by the time imap crashes. Here are all the gory GDB details: Version: 1.1.beta3 OS: NetBSD 4.0_RC3/sparc64 (a 64 bit big endian architecture) Client: Mozilla Thunderbird 2.0.0.6 bt full: #0 0x00000000001e846c in io_file_unlink (io=0x32e2c0) at ioloop.c:54 No locals. #1 0x00000000001e8568 in io_remove (_io=0x3420a8) at ioloop.c:82 io_file = (struct io_file *) 0x32e2c0 io = (struct io *) 0x32e2c0 #2 0x000000000017a0a4 in index_mailbox_check_remove_all (ibox=0x331048) at index-mailbox-check.c:118 file = (struct index_notify_file *) 0x0 aio = (struct index_notify_io *) 0x3420a0 #3 0x0000000000139850 in maildir_notify_changes (box=0x331048) at maildir-storage.c:824 mbox = (struct maildir_mailbox *) 0x331048 #4 0x00000000001c0724 in mailbox_notify_changes (box=0x331048, min_interval=0, callback=0, context=0x0) at mail-storage.c:544 No locals. #5 0x00000000001c074c in mailbox_notify_changes_stop (box=0x331048) at mail-storage.c:550 No locals. #6 0x00000000001167fc in idle_finish (ctx=0x3380b0, done_ok=true, free_cmd=true) at cmd-idle.c:48 client = (struct client *) 0x334100 #7 0x0000000000116a04 in idle_client_input (ctx=0x3380b0) at cmd-idle.c:90 client = (struct client *) 0x334100 line = 0x33a075 "DONE" #8 0x00000000001ea2ac in io_loop_handler_run (ioloop=0x320080) at ioloop-kqueue.c:149 ctx = (struct ioloop_handler_context *) 0x32e080 events = (struct kevent *) 0x332000 event = (const struct kevent *) 0x332000 tv = {tv_sec = 0, tv_usec = 999441} ts = {tv_sec = 0, tv_nsec = 999441000} io = (struct io_file *) 0x320a80 events_count = 4 t_id = 2 msecs = 1000 ret = 1 i = 0 __PRETTY_FUNCTION__ = "io_loop_handler_run" #9 0x00000000001e8f28 in io_loop_run (ioloop=0x320080) at ioloop.c:303 No locals. #10 0x000000000012d710 in main (argc=1, argv=0xffffffffffffc978, envp=0xffffffffffffc988) at main.c:293 No locals. More GDB Info: (gdb) print *io $2 = {io = {condition = 0, callback = 0, context = 0x331048}, prev = 0x10000000d, next = 0x32e280, refcount = 0, fd = 0} (gdb) print *io->next $18 = {io = {condition = 0, callback = 0, context = 0x471a982100000000}, prev = 0x0, next = 0x2f766f6c2f686f6d, refcount = 1697606764, fd = 1870226479} (gdb) print *current_ioloop->io_files $35 = {io = {condition = IO_READ, callback = 0x1e9284 <event_callback>, context = 0x342080}, prev = 0x0, next = 0x320140, refcount = 1, fd = 11} (gdb) print *current_ioloop->io_files->next $36 = {io = {condition = IO_ERROR, callback = 0x12ca70 <log_error_callback>, context = 0x0}, prev = 0x320a00, next = 0x3200c0, refcount = 1, fd = 2} (gdb) print *current_ioloop->io_files->next->next $37 = {io = {condition = IO_READ, callback = 0x1ea6f0 <signal_read>, context = 0x0}, prev = 0x320140, next = 0x0, refcount = 1, fd = 5} (gdb) print io $1 = (struct io_file *) 0x32e2c0 (gdb) print *current_ioloop $2 = {prev = 0x0, io_files = 0x320a00, next_io_file = 0x0, timeouts = 0x320a40, handler_context = 0x32e080, notify_handler_context = 0x342080, running = 1}

3 2

[Dovecot] Problems with dovecot 1.1b3
by Adam McDougall 21 Oct '07

21 Oct '07

I am testing dovecot 1.1 betas on a testing system, hoping to take advantage of self repairing and NFS-capable indexes when 1.1 is sensible to put into production. I ran into a number of issues that I did not have with 1.0, and I think I'll just pick one or more at a time and report them rather than try to report every single issue at once, hoping that some of the fixes will solve more than one problem. I started with 1.1b2 just for record, but the issues seem to be present in 1.1b3 as well. I realized yesterday that using the acl plugin may be complicating my testing and results, so I have disabled it for now and am connecting with plaintext IMAP just so I can sniff connections and see what is happening. I am running it on FreeBSD 6.2-stable from Sep 19 2007. First my dovecot -n output: # 1.1.beta3: /usr/local/etc/dovecot.conf ssl_cert_file: /usr/local/etc/apache2/ssl/ssl.pem ssl_key_file: /usr/local/etc/apache2/ssl/ssl.pem disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 0 mail_location: maildir:%h/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_plugins: mail_log mail_log_max_lines_per_sec: 0 imap_client_workarounds: delay-newmail netscape-eoh tb-extra-mailbox-sep namespace: type: private separator: / prefix: mail/ location: maildir:%h/Maildir:CONTROL=%h/Maildir/dovecot/private2/control:INDEX=%h/Maildir/dovecot/private2/indexes hidden: yes list: yes namespace: type: private separator: / inbox: yes list: yes auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 384 user: postfix group: postfix plugin: acl: vfile:/usr/local/etc/dovecot-acls The first thing I did was to try to update my dovecot.conf from 1.0 to work with 1.1. When I started dovecot 1.1 for the first time, I noticed it was listening to 143 but not 993, even though my config was designed for ssl/tls in 1.0 without any listen ports specified: # netstat -a | grep LISTEN | grep imap tcp4 0 0 *.imap *.* LISTEN So, I thought I would try specifying the ports in the config: protocol imap { listen = *:143 ssl_listen = *:993 # .. } Success! # netstat -a | grep LISTEN | grep imap tcp4 0 0 *.imaps *.* LISTEN tcp4 0 0 *.imap *.* LISTEN Initially I was then testing using ACLs, but I disabled it to simplify. One of the more basic problems I've seen is the behavior from "list = no/yes". For the sake of simplicity and example, I've disabled most of the additional public and private namespaces from my 1.0 config. I have some private namespaces with prefix "mail/" and "Mail/" that I wish to provide for users for compatability with clients set with those prefixes. In dovecot 1.0, this worked as expected: namespace: type: private separator: / prefix: mail/ hidden: yes But in 1.1b2 or b3, I never seem to get a list of the contents under mail/ when I tell mutt to default to servername/mail/ as the default folder path. Cases: hidden = yes list = yes a0006 LIST "" "mail" * LIST (\Noselect \HasChildren) "/" "mail" a0006 OK List completed. a0007 LIST "" "mail/%" a0007 OK List completed. mutt lists contents of only "../" and selecting it brings me to the imap root with the expected folders. When ../ is followed, I see all my folders as well as all my folders repeated under mail/ prefix. hidden = yes list = no a0006 LIST "" "mail" a0006 OK List completed. a0007 LIST "" "mail/%" a0007 OK List completed. mutt claims "No such folder" hidden = no list = yes a0006 LIST "" "mail" * LIST (\Noselect \HasChildren) "/" "mail" a0006 OK List completed. a0007 LIST "" "mail/%" a0007 OK List completed. mutt lists contents of only "../" and selecting it brings me to the imap root with the expected folders. When ../ is followed, I see all my folders as well as all my folders repeated under mail/ prefix.

2 6

[Dovecot] Problem with fts found against Dovecot hg; examples + trace attached
by Asheesh Laroia 21 Oct '07

21 Oct '07

I've been wanting to try out Squat for full-text indexing for a while now, so I finally gave it a whirl. I did a fresh pull from hg. I have it enabled, but it's taking *forever* to do a query. (Using alpine, I issued a Select command for all messages with some word anywhere, which presumably caused the Squat indexer to run.) I think Dovecot is infinite looping. I thought something was wrong when I attached strace to it for five minutes and saw no calls to read() or open(). I can reproduce the "select takes forever" on a tiny Maildir with 5 messages: http://paulproteus.acm.jhu.edu/bug-report/2007-10-21/broken.tar.gz I've also attached the *.in and *.out files that I think are related to this IMAP conversation. On a different IMAP conversation with the same "SEARCH takes forever" problem, I attached a gdb to the running Dovecot and got this backtrace: (gdb) bt #0 parse_next_body_to_boundary (ctx=0x85746e8, block_r=0xbf92325c) at message-parser.c:332 #1 0x080c1b7d in message_parser_parse_next_block (ctx=0x85746e8, block_r=0xbf92325c) at message-parser.c:717 #2 0xb7f582c2 in fts_mailbox_search_next_nonblock (ctx=0x811c0c0, mail=0x811d380, tryagain_r=0xbf9232bb) at fts-storage.c:175 #3 0x0805f3e6 in cmd_search_more (cmd=0x80feec0) at cmd-search.c:74 #4 0x0805f768 in cmd_search_more_callback (cmd=0x80feec0) at cmd-search.c:112 #5 0x080ce8eb in io_loop_handle_timeouts (ioloop=0x80fb250, update_run_now=true) at ioloop.c:264 #6 0x080cf309 in io_loop_handler_run (ioloop=0x80fb250) at ioloop-poll.c:159 #7 0x080ce7c8 in io_loop_run (ioloop=0x80fb250) at ioloop.c:303 #8 0x08068c9c in main (argc=135213400, argv=0xbf923464, envp=0xbf92346c) at main.c:293 So I do know that it's using the fts system. Every few minutes, I've interrupted it in gdb, and I have gotten these messages back from gdb: Program received signal SIGINT, Interrupt. parse_next_body_to_boundary (ctx=0x85746e8, block_r=0xbf92325c) at message-parser.c:332 332 if (data[i] == '\n') { (gdb) cont Continuing. [wait a few minutes] Program received signal SIGINT, Interrupt. parse_next_body_to_boundary (ctx=0x85746e8, block_r=0xbf92325c) at message-parser.c:331 331 for (; i < block_r->size; i++) { (gdb) cont Continuing. [wait a few minutes] Program received signal SIGINT, Interrupt. parse_next_body_to_boundary (ctx=0x85746e8, block_r=0xbf92325c) at message-parser.c:332 332 if (data[i] == '\n') { The ctx and block_r are the same at each point I interrupt it. This sounds like an infinite loop to me if it can't exit this for minutes and minutes. Furthermore, for the past couple of minutes, when I do this: (gdb) print block_r[0] $8 = {part = 0x80f3158, hdr = 0x0, data = 0x8240d08 "\n<DIV><BR></DIV>Perhaps, but it is important to note that this is a dissenting<BR>opinion, not part of the actual text of the Holt bill.<BR><BR><BR>--- In <A href=\"mailto:TrueVote@yahoogroups.com\">Tru"..., size = 8192} I always get the same response back. I've linked to a fairly small Maildir (five messages) that reproduces the problem for me. It's possible that only one of those messages is actually at fault; I haven't tried with any Maildir smaller than five messages. Is there something more I can do to help fix this problem? I'm happy to do anything reasonable further if you deem it useful. -- Asheesh. -- Bolub's Fourth Law of Computerdom: Project teams detest weekly progress reporting because it so vividly manifests their lack of progress.

2 1

[Dovecot] Multiple server sections for SSL
by Wolfram Schlich 21 Oct '07

21 Oct '07

Hi, I would like to achieve the following with my dovecot server: - listen von FQDN 'mail.foo.example.com'/IP 'A' for imaps and pop3s with an SSL certificate for 'mail.foo.example.com' - listen von FQDN 'mail.bar.example.com'/IP 'B' for imaps and pop3s with an SSL certificate for 'mail.bar.example.com' I stumbled over a posting by Timo made in 2004 that got my attention: http://osdir.com/ml/mail.imap.dovecot/2004-07/msg00133.html Now, what is the correct nesting/structure for the server, protocol, auth, dict and plugin section as well as any other (by default) "non-sectional" configuration options (like 'login_greeting')? I failed to find *any* documentation references on this topic :-( TIA. -- Regards, Wolfram Schlich <wschlich(a)gentoo.org> Gentoo Linux * http://dev.gentoo.org/~wschlich/

2 3