- dovecot - dovecot.org

[Dovecot] Segfault by asking for a folder?
by Asheesh Laroia 21 Oct '07

21 Oct '07

I 'hg pull'ed and now my Dovecot segfaults when I select a particular folder. I've removed all files with 'index' in the name, so I don't think it's due to bad indexes. Oct 21 09:18:40 alchemy dovecot: Dovecot v1.1.beta3 starting up Oct 21 09:18:47 alchemy dovecot: imap-login: Login: user=<paulproteus>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, TLS Oct 21 09:18:47 alchemy dovecot: child 28714 (imap) killed with signal 11 is in syslog. I haven't stripped the binaries as far as I know, yet here's what seems to happen in gdb: paulproteus@alchemy:~/dovecot.rawlog $ gdb dovecot [snip] This GDB was configured as "i486-linux-gnu"... Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1". (gdb) set args --exec-mail imap (gdb) run Starting program: /usr/local/sbin/dovecot --exec-mail imap * PREAUTH [CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS UIDPLUS LIST-EXTENDED] Logged in as paulproteus 01 SELECT pinerc Program received signal SIGSEGV, Segmentation fault. Cannot remove breakpoints because program is no longer writable. It might be running in another process. Further execution is probably impossible. 0x08073d0d in ?? () (gdb) bt #0 0x08073d0d in ?? () #1 0xbff92f70 in ?? () #2 0x00000000 in ?? () (gdb) bt full #0 0x08073d0d in ?? () No symbol table info available. #1 0xbff92f70 in ?? () No symbol table info available. #2 0x00000000 in ?? () No symbol table info available. (gdb) Uh? It seems to also segfault the same way if I select INBOX. -- Asheesh. -- You'll never see all the places, or read all the books, but fortunately, they're not all recommended.

1 2

[Dovecot] dovecot? putting wrong seperator in subscriptions file
by Adam McDougall 21 Oct '07

21 Oct '07

My config has folders show up in the root namespace as well as hidden ones like mail/ for compat. In thunderbird when I set the namespace prefix to mail/, I effectively cannot subscribe to folders. You can pull up the folder list, select some folders to subscribe to, hit okay, and nothing changes in thunderbird: 3 subscribe "mail/Spam" 3 OK Subscribe completed. 4 lsub "" "mail/*" 4 OK Lsub completed. What gets put in my subscriptions file: mail.Spam If I manually change my subscriptions file to mail/Spam, it works as expected. # dovecot -n # 1.1.beta3: /usr/local/etc/dovecot.conf ssl_cert_file: /usr/local/etc/apache2/ssl/cert.pem ssl_key_file: /usr/local/etc/apache2/ssl/cert.pem disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 0 mail_location: maildir:%h/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_plugins: mail_log mail_log_max_lines_per_sec: 0 imap_client_workarounds: delay-newmail netscape-eoh tb-extra-mailbox-sep imap_logout_format: bbytes=%i/%o username=%u namespace: type: private separator: / prefix: mail/ location: maildir:%h/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes hidden: yes namespace: type: private separator: / inbox: yes list: yes auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 384 user: postfix group: postfix plugin: acl: vfile:/usr/local/etc/dovecot-acls

2 4

[Dovecot] two small low importance issues in 1.1b
by Adam McDougall 21 Oct '07

21 Oct '07

Item 1: I store the indexes and control in their own directory. I noticed in 1.1: 1192 ./private/indexes/.INBOX 32 ./private/indexes/.Trash 1228 ./private/indexes 4 ./private/control/.INBOX/.INBOX 84 ./private/control/.INBOX 4 ./private/control/.Trash/.Trash 12 ./private/control/.Trash 100 ./private/control 1332 ./private .INBOX/.INBOX and .Trash/.Trash are both empty. This was an empty directory, I logged in with thunderbird and looked at the inbox and Trash folders. Item 2: I wanted to try imap_logout_format, forgetting what params it uses and taking a stab at it, but it seems to be ignored? It shows up in dovecot -n. I made it 'bbytes' just to see if the intentional typo would come through, but no: imap_logout_format: bbytes=%i/%o username=%u dovecot: IMAP(mcdouga9): Disconnected: Logged out bytes=439/169567

2 2

[Dovecot] dovecot 1.1.beta3 crashes on NetBSD/sparc64 4.0_RC3 with Thunderbird 2.0.0.6
by Lloyd Parkes 21 Oct '07

21 Oct '07

Hi all, I'm getting core dumps from Dovecot when using Thunderbird as a client. Everything is fine when using the Apple Mail client. I've appended the debug dumps to this message so they are out of the way, but they should be read now. I think the problem might be that io->prev and io->next aren't reliably initialised in ioloop.c:io_add. The value that io->prev has when the program crashes is just completely bogus. It isn't aligned correctly, and it doesn't point to anywhere inside the process's memory map. I don't know what *io->next is, but it sure doesn't look like a struct io_file. It doesn't even look like a struct io. Part of the garbage in *io->next is the char sequence "/vol/home/lloyd/Maildir/new" which is part of my mail box. *io is not in the doubly linked list in current_io_look->io_files. I'm guessing that this object has been unlinked already. Some further digging (not shown here) shows that io->callback is notify_callback. It is set to NULL by the time imap crashes. Here are all the gory GDB details: Version: 1.1.beta3 OS: NetBSD 4.0_RC3/sparc64 (a 64 bit big endian architecture) Client: Mozilla Thunderbird 2.0.0.6 bt full: #0 0x00000000001e846c in io_file_unlink (io=0x32e2c0) at ioloop.c:54 No locals. #1 0x00000000001e8568 in io_remove (_io=0x3420a8) at ioloop.c:82 io_file = (struct io_file *) 0x32e2c0 io = (struct io *) 0x32e2c0 #2 0x000000000017a0a4 in index_mailbox_check_remove_all (ibox=0x331048) at index-mailbox-check.c:118 file = (struct index_notify_file *) 0x0 aio = (struct index_notify_io *) 0x3420a0 #3 0x0000000000139850 in maildir_notify_changes (box=0x331048) at maildir-storage.c:824 mbox = (struct maildir_mailbox *) 0x331048 #4 0x00000000001c0724 in mailbox_notify_changes (box=0x331048, min_interval=0, callback=0, context=0x0) at mail-storage.c:544 No locals. #5 0x00000000001c074c in mailbox_notify_changes_stop (box=0x331048) at mail-storage.c:550 No locals. #6 0x00000000001167fc in idle_finish (ctx=0x3380b0, done_ok=true, free_cmd=true) at cmd-idle.c:48 client = (struct client *) 0x334100 #7 0x0000000000116a04 in idle_client_input (ctx=0x3380b0) at cmd-idle.c:90 client = (struct client *) 0x334100 line = 0x33a075 "DONE" #8 0x00000000001ea2ac in io_loop_handler_run (ioloop=0x320080) at ioloop-kqueue.c:149 ctx = (struct ioloop_handler_context *) 0x32e080 events = (struct kevent *) 0x332000 event = (const struct kevent *) 0x332000 tv = {tv_sec = 0, tv_usec = 999441} ts = {tv_sec = 0, tv_nsec = 999441000} io = (struct io_file *) 0x320a80 events_count = 4 t_id = 2 msecs = 1000 ret = 1 i = 0 __PRETTY_FUNCTION__ = "io_loop_handler_run" #9 0x00000000001e8f28 in io_loop_run (ioloop=0x320080) at ioloop.c:303 No locals. #10 0x000000000012d710 in main (argc=1, argv=0xffffffffffffc978, envp=0xffffffffffffc988) at main.c:293 No locals. More GDB Info: (gdb) print *io $2 = {io = {condition = 0, callback = 0, context = 0x331048}, prev = 0x10000000d, next = 0x32e280, refcount = 0, fd = 0} (gdb) print *io->next $18 = {io = {condition = 0, callback = 0, context = 0x471a982100000000}, prev = 0x0, next = 0x2f766f6c2f686f6d, refcount = 1697606764, fd = 1870226479} (gdb) print *current_ioloop->io_files $35 = {io = {condition = IO_READ, callback = 0x1e9284 <event_callback>, context = 0x342080}, prev = 0x0, next = 0x320140, refcount = 1, fd = 11} (gdb) print *current_ioloop->io_files->next $36 = {io = {condition = IO_ERROR, callback = 0x12ca70 <log_error_callback>, context = 0x0}, prev = 0x320a00, next = 0x3200c0, refcount = 1, fd = 2} (gdb) print *current_ioloop->io_files->next->next $37 = {io = {condition = IO_READ, callback = 0x1ea6f0 <signal_read>, context = 0x0}, prev = 0x320140, next = 0x0, refcount = 1, fd = 5} (gdb) print io $1 = (struct io_file *) 0x32e2c0 (gdb) print *current_ioloop $2 = {prev = 0x0, io_files = 0x320a00, next_io_file = 0x0, timeouts = 0x320a40, handler_context = 0x32e080, notify_handler_context = 0x342080, running = 1}

3 2

[Dovecot] Problems with dovecot 1.1b3
by Adam McDougall 21 Oct '07

21 Oct '07

I am testing dovecot 1.1 betas on a testing system, hoping to take advantage of self repairing and NFS-capable indexes when 1.1 is sensible to put into production. I ran into a number of issues that I did not have with 1.0, and I think I'll just pick one or more at a time and report them rather than try to report every single issue at once, hoping that some of the fixes will solve more than one problem. I started with 1.1b2 just for record, but the issues seem to be present in 1.1b3 as well. I realized yesterday that using the acl plugin may be complicating my testing and results, so I have disabled it for now and am connecting with plaintext IMAP just so I can sniff connections and see what is happening. I am running it on FreeBSD 6.2-stable from Sep 19 2007. First my dovecot -n output: # 1.1.beta3: /usr/local/etc/dovecot.conf ssl_cert_file: /usr/local/etc/apache2/ssl/ssl.pem ssl_key_file: /usr/local/etc/apache2/ssl/ssl.pem disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 0 mail_location: maildir:%h/Maildir:CONTROL=%h/Maildir/dovecot/private/control:INDEX=%h/Maildir/dovecot/private/indexes mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_plugins: mail_log mail_log_max_lines_per_sec: 0 imap_client_workarounds: delay-newmail netscape-eoh tb-extra-mailbox-sep namespace: type: private separator: / prefix: mail/ location: maildir:%h/Maildir:CONTROL=%h/Maildir/dovecot/private2/control:INDEX=%h/Maildir/dovecot/private2/indexes hidden: yes list: yes namespace: type: private separator: / inbox: yes list: yes auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 384 user: postfix group: postfix plugin: acl: vfile:/usr/local/etc/dovecot-acls The first thing I did was to try to update my dovecot.conf from 1.0 to work with 1.1. When I started dovecot 1.1 for the first time, I noticed it was listening to 143 but not 993, even though my config was designed for ssl/tls in 1.0 without any listen ports specified: # netstat -a | grep LISTEN | grep imap tcp4 0 0 *.imap *.* LISTEN So, I thought I would try specifying the ports in the config: protocol imap { listen = *:143 ssl_listen = *:993 # .. } Success! # netstat -a | grep LISTEN | grep imap tcp4 0 0 *.imaps *.* LISTEN tcp4 0 0 *.imap *.* LISTEN Initially I was then testing using ACLs, but I disabled it to simplify. One of the more basic problems I've seen is the behavior from "list = no/yes". For the sake of simplicity and example, I've disabled most of the additional public and private namespaces from my 1.0 config. I have some private namespaces with prefix "mail/" and "Mail/" that I wish to provide for users for compatability with clients set with those prefixes. In dovecot 1.0, this worked as expected: namespace: type: private separator: / prefix: mail/ hidden: yes But in 1.1b2 or b3, I never seem to get a list of the contents under mail/ when I tell mutt to default to servername/mail/ as the default folder path. Cases: hidden = yes list = yes a0006 LIST "" "mail" * LIST (\Noselect \HasChildren) "/" "mail" a0006 OK List completed. a0007 LIST "" "mail/%" a0007 OK List completed. mutt lists contents of only "../" and selecting it brings me to the imap root with the expected folders. When ../ is followed, I see all my folders as well as all my folders repeated under mail/ prefix. hidden = yes list = no a0006 LIST "" "mail" a0006 OK List completed. a0007 LIST "" "mail/%" a0007 OK List completed. mutt claims "No such folder" hidden = no list = yes a0006 LIST "" "mail" * LIST (\Noselect \HasChildren) "/" "mail" a0006 OK List completed. a0007 LIST "" "mail/%" a0007 OK List completed. mutt lists contents of only "../" and selecting it brings me to the imap root with the expected folders. When ../ is followed, I see all my folders as well as all my folders repeated under mail/ prefix.

2 6

[Dovecot] Problem with fts found against Dovecot hg; examples + trace attached
by Asheesh Laroia 21 Oct '07

21 Oct '07

I've been wanting to try out Squat for full-text indexing for a while now, so I finally gave it a whirl. I did a fresh pull from hg. I have it enabled, but it's taking *forever* to do a query. (Using alpine, I issued a Select command for all messages with some word anywhere, which presumably caused the Squat indexer to run.) I think Dovecot is infinite looping. I thought something was wrong when I attached strace to it for five minutes and saw no calls to read() or open(). I can reproduce the "select takes forever" on a tiny Maildir with 5 messages: http://paulproteus.acm.jhu.edu/bug-report/2007-10-21/broken.tar.gz I've also attached the *.in and *.out files that I think are related to this IMAP conversation. On a different IMAP conversation with the same "SEARCH takes forever" problem, I attached a gdb to the running Dovecot and got this backtrace: (gdb) bt #0 parse_next_body_to_boundary (ctx=0x85746e8, block_r=0xbf92325c) at message-parser.c:332 #1 0x080c1b7d in message_parser_parse_next_block (ctx=0x85746e8, block_r=0xbf92325c) at message-parser.c:717 #2 0xb7f582c2 in fts_mailbox_search_next_nonblock (ctx=0x811c0c0, mail=0x811d380, tryagain_r=0xbf9232bb) at fts-storage.c:175 #3 0x0805f3e6 in cmd_search_more (cmd=0x80feec0) at cmd-search.c:74 #4 0x0805f768 in cmd_search_more_callback (cmd=0x80feec0) at cmd-search.c:112 #5 0x080ce8eb in io_loop_handle_timeouts (ioloop=0x80fb250, update_run_now=true) at ioloop.c:264 #6 0x080cf309 in io_loop_handler_run (ioloop=0x80fb250) at ioloop-poll.c:159 #7 0x080ce7c8 in io_loop_run (ioloop=0x80fb250) at ioloop.c:303 #8 0x08068c9c in main (argc=135213400, argv=0xbf923464, envp=0xbf92346c) at main.c:293 So I do know that it's using the fts system. Every few minutes, I've interrupted it in gdb, and I have gotten these messages back from gdb: Program received signal SIGINT, Interrupt. parse_next_body_to_boundary (ctx=0x85746e8, block_r=0xbf92325c) at message-parser.c:332 332 if (data[i] == '\n') { (gdb) cont Continuing. [wait a few minutes] Program received signal SIGINT, Interrupt. parse_next_body_to_boundary (ctx=0x85746e8, block_r=0xbf92325c) at message-parser.c:331 331 for (; i < block_r->size; i++) { (gdb) cont Continuing. [wait a few minutes] Program received signal SIGINT, Interrupt. parse_next_body_to_boundary (ctx=0x85746e8, block_r=0xbf92325c) at message-parser.c:332 332 if (data[i] == '\n') { The ctx and block_r are the same at each point I interrupt it. This sounds like an infinite loop to me if it can't exit this for minutes and minutes. Furthermore, for the past couple of minutes, when I do this: (gdb) print block_r[0] $8 = {part = 0x80f3158, hdr = 0x0, data = 0x8240d08 "\n<DIV><BR></DIV>Perhaps, but it is important to note that this is a dissenting<BR>opinion, not part of the actual text of the Holt bill.<BR><BR><BR>--- In <A href=\"mailto:TrueVote@yahoogroups.com\">Tru"..., size = 8192} I always get the same response back. I've linked to a fairly small Maildir (five messages) that reproduces the problem for me. It's possible that only one of those messages is actually at fault; I haven't tried with any Maildir smaller than five messages. Is there something more I can do to help fix this problem? I'm happy to do anything reasonable further if you deem it useful. -- Asheesh. -- Bolub's Fourth Law of Computerdom: Project teams detest weekly progress reporting because it so vividly manifests their lack of progress.

2 1

[Dovecot] Multiple server sections for SSL
by Wolfram Schlich 21 Oct '07

21 Oct '07

Hi, I would like to achieve the following with my dovecot server: - listen von FQDN 'mail.foo.example.com'/IP 'A' for imaps and pop3s with an SSL certificate for 'mail.foo.example.com' - listen von FQDN 'mail.bar.example.com'/IP 'B' for imaps and pop3s with an SSL certificate for 'mail.bar.example.com' I stumbled over a posting by Timo made in 2004 that got my attention: http://osdir.com/ml/mail.imap.dovecot/2004-07/msg00133.html Now, what is the correct nesting/structure for the server, protocol, auth, dict and plugin section as well as any other (by default) "non-sectional" configuration options (like 'login_greeting')? I failed to find *any* documentation references on this topic :-( TIA. -- Regards, Wolfram Schlich <wschlich(a)gentoo.org> Gentoo Linux * http://dev.gentoo.org/~wschlich/

2 3

[Dovecot] Homedir Access without exposing whole Homedir.
by Dan Mahoney, System Admin 21 Oct '07

21 Oct '07

Hello All, Dovecot 1.0.3 I am coming from UW IMAP, and I'm finding for some reason that mail is getting stored in a variety of places (which I believe, is because by default UW imap allowed access to the entire home directory). All files are mbox. My default delivery location is ~/.mail Thus: * At least a couple of my users have mail in ~/INBOX, as well as ~/INBOX.drafts (not many users so I don't really care about this). * Several of my users have their mail folders in their home directory, AND in a mail folder. For example, my "test case" user, had the following .mailboxlist file: mail/Sent mail/Drafts mail/Spam mail/Trash Sent Items Drafts Deleted Items Junk E-mail Now, here's the question. I'd like to make it so that everyone has what they had before, but that any new folders are created in ~/mail/ (so that if I ever DO decide to convert, I know where to look). What I thought would be useful for this is make the default namespace "", but set it hidden. Then define a secondary namespace (mail) and use that. However, when I did this, and got a mailbox list, it still recursively listed every file in the homedir. Is there a simple way to do what I want? -Dan Mahoney -- "Station!" -Bill & Ted's Bogus Journey --------Dan Mahoney-------- Techie, Sysadmin, WebGeek Gushi on efnet/undernet IRC ICQ: 13735144 AIM: LarpGM Site: http://www.gushi.org ---------------------------

2 2

[Dovecot] [Fwd: Re: Namespace Question]
by Jeff Grossman 21 Oct '07

21 Oct '07

Timo Sirainen wrote: > On Fri, 2007-10-05 at 17:27 -0700, Jeff Grossman wrote: > >> Timo Sirainen wrote: >> >>> On Tue, 2007-10-02 at 18:12 -0700, Jeff Grossman wrote: >>> >>> >>>> So, what would >>>> have changed between version 1.0.5 and 1.1beta2 to somehow make the >>>> phone think all of the messages are older than 30 days? >>>> >>>> >>> Is it using SORT command? Apparently there are some bugs still in it. >>> http://dovecot.org/bugreport.html#sniffing >>> >>> >>> >> I don't think so. Here is the first .in file from the rawlog >> >> A6 NAMESPACE >> A7 SELECT "Inbox" >> A8 SEARCH SINCE 6-Sep-2007 >> > > OK, this finds the messages based on their INTERNALDATE. With maildir > that means the files' mtime. Are all the mtimes older than 6 Sep? If > not, try deleting dovecot.index.cache file to see if it helps. > > Out of 13 messages in my inbox, 12 of them are newer than 6 Sep. I deleted the dovecot.index.cache file and it still will not display any messages in my inbox when I have the setting to only display the most current 30 days worth of mail. Jeff Timo, Just FYI, the above problem is still not fixed with beta 3. Jeff

2 7

[Dovecot] v1.1 QUOTA and inbox quota
by Kristian Vlahovicek 21 Oct '07

21 Oct '07

My mail agent is postfix and by default it imposes the INBOX size of 50 meg. I would like my imap mail clients (I'm using Thunderbird) to "see" this and warn me when my inbox fills up. I tried doing it with the old quota system but wasn't successful. With the new quota rule system it looks like per-mailbox quotas are possible, however when I do this: quota = maildir quota_rule = INBOX:storage=50000 I get an error in the log: Sep 28 02:18:31 discworld dovecot: IMAP(kristian): open((null)) failed: Bad address Sep 28 02:18:31 discworld dovecot: IMAP(kristian): Internal quota calculation error Sep 28 02:20:04 discworld dovecot: IMAP(kristian): open((null)) failed: Bad address Sep 28 02:20:04 discworld dovecot: IMAP(kristian): open((null)) failed: Bad address is there any way to get the inbox quota to work properly? I guess many systems use postfix and have 50M inbox limit so it would really be helpful if this information is pushed somehow through imap to the client. thanks, K

2 1