- dovecot - dovecot.org

[Dovecot] assertion failure in current hg, file istream.c: line 303 (i_stream_read_data): assertion failed: (stream->stream_errno != 0) (1.1.3 was working fine)
by Diego Liziero 16 Oct '08

16 Oct '08

Today I updated to current dovecot-1.1 hg tree and I got many of these assertion failures: file istream.c: line 303 (i_stream_read_data): assertion failed: (stream->stream_errno != 0) (gdb) bt full #0 0x00352402 in __kernel_vsyscall () No symbol table info available. #1 0x0043ed20 in raise () from /lib/libc.so.6 No symbol table info available. #2 0x00440631 in abort () from /lib/libc.so.6 No symbol table info available. #3 0x080f6968 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:150 backtrace = 0x906d950 "/usr/libexec/dovecot/imap [0x80f6946] -> /usr/libexec/dovecot/imap [0x80f7207] -> /usr/libexec/dovecot/imap(i_fatal+0) [0x80f6ac0] -> /usr/libexec/dovecot/imap(i_stream_read_data+0. #4 0x080f7207 in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x8124778 "file %s: line %d (%s): assertion failed: (%s)", args=0xbfe643a4 "mG\022\b/\001") at failures.c:430 No locals. #5 0x080f6ac0 in i_panic (format=0x8124778 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:197 args = 0xbfe643a4 "mG\022\b/\001" #6 0x080fcb64 in i_stream_read_data (stream=0x9094a08, data_r=0xbfe64444, size_r=0xbfe64440, threshold=0) at istream.c:303 ret = -1 read_more = false __PRETTY_FUNCTION__ = "i_stream_read_data" #7 0x08065ba1 in imap_fetch_send (ctx=0x90716e0, output=0x90714ec, input=0x9094a08, cr_skipped=false, virtual_size=169, add_missing_eoh=false, last_cr=0x9071734) at imap-fetch-body.c:132 msg = (const unsigned char *) 0x0 i = 3219539032 size = 0 vsize_left = 169 sent = 0 ret = 581237648214344776 add = 16 '\020' blocks = false #8 0x08065d7f in fetch_stream_send (ctx=0x90716e0) at imap-fetch-body.c:217 ret = 151603720 #9 0x080660e8 in fetch_stream (ctx=0x90716e0, size=0x908efc4) at imap-fetch-body.c:293 input = (struct istream *) 0x10b8df #10 0x08066225 in fetch_data (ctx=0x90716e0, body=0x9071cd8, size=0x908efc4) at imap-fetch-body.c:318 str = (string_t *) 0x906cc18 #11 0x08066ac7 in fetch_body_mime (ctx=0x90716e0, mail=0x908e9e8, body=0x9071cd8) at imap-fetch-body.c:555 part = (const struct message_part *) 0x908efb0 section = 0x9071d12 "MIME" __PRETTY_FUNCTION__ = "fetch_body_mime" #12 0x08064bad in imap_fetch_more (ctx=0x90716e0) at imap-fetch.c:309 h = (const struct imap_fetch_context_handler *) 0x90718c8 _data_stack_cur_id = 4 client = (struct client *) 0x9071368 handlers = (const struct imap_fetch_context_handler *) 0x9071850 count = 7 ret = 1 __PRETTY_FUNCTION__ = "imap_fetch_more" #13 0x08064dd4 in imap_fetch (ctx=0x90716e0) at imap-fetch.c:361 ret = 0 __PRETTY_FUNCTION__ = "imap_fetch" #14 0x0805c792 in cmd_fetch (cmd=0x9071600) at cmd-fetch.c:152 ctx = (struct imap_fetch_context *) 0x90716e0 args = (const struct imap_arg *) 0x9075688 search_arg = (struct mail_search_arg *) 0x9071678 messageset = 0x9075750 "4068" ret = 151455048 #15 0x08061173 in cmd_uid (cmd=0x9071600) at cmd-uid.c:26 command = (struct command *) 0x907068c cmd_name = 0x9075738 "fetch" #16 0x08062534 in client_command_input (cmd=0x9071600) at client.c:580 client = (struct client *) 0x9071368 command = (struct command *) 0x23 __PRETTY_FUNCTION__ = "client_command_input" #17 0x08062769 in client_command_input (cmd=0x9071600) at client.c:629 client = (struct client *) 0x9071368 command = (struct command *) 0x9070668 __PRETTY_FUNCTION__ = "client_command_input" #18 0x08062867 in client_handle_next_command (client=0x9071368, remove_io_r=0xbfe64765) at client.c:670 size = 132 #19 0x080628a3 in client_handle_input (client=0x9071368) at client.c:680 _data_stack_cur_id = 3 ret = 7 remove_io = false handled_commands = false #20 0x08062a31 in client_input (client=0x9071368) at client.c:725 cmd = (struct client_command_context *) 0x4e7408 output = (struct ostream *) 0x90714ec bytes = 132 __PRETTY_FUNCTION__ = "client_input" #21 0x08101115 in io_loop_handler_run (ioloop=0x906f9b0) at ioloop-epoll.c:203 ctx = (struct ioloop_handler_context *) 0x906faa8 events = (struct epoll_event *) 0x906fae8 event = (const struct epoll_event *) 0x906fae8 list = (struct io_list *) 0x9071568 io = (struct io_file *) 0x9071548 tv = {tv_sec = 1799, tv_usec = 999874} events_count = 3 t_id = 2 msecs = 1800000 ret = 1 i = 0 j = 0 call = true #22 0x081003ac in io_loop_run (ioloop=0x906f9b0) at ioloop.c:320 No locals. #23 0x0806debd in main (argc=1, argv=0xbfe648c4, envp=0xbfe648cc) at main.c:293 No locals.

2 9

[Dovecot] Ignore prefix given by client?
by Tim Bishop 15 Oct '08

15 Oct '08

Hi all, I'm new to Dovecot, but so far I've been really impressed with what I've seen. I hope someone can help with the following question. We're migrating from a setup where some of our users use mbox with UW imap. The problem is we've never enforced a folder prefix, so some have their folders in ~/, others in ~/mail and others in ~/Mail. We've also got some users using maildir with Courier imap. I decided the easiest way to get this working transparently for the users was to use the post-login scripting approach. My script manages to locate the user's mail and sets MAIL accordingly. This all works a treat. The problem is when a user specifies their own prefix in their mail client (if it's blank there are no problems). If they have ~/Mail none of the folders show up. I've had a play with namespaces in an attempt to fix that, but it doesn't seem to be doing what I want. So my question is: is it possible for Dovecot to simply ignore the folder prefix given by the client? Thanks, Tim. -- Tim Bishop http://www.bishnet.net/tim/ PGP Key: 0x5AE7D984

2 2

[Dovecot] Something weird happening...
by Claudio Prono 15 Oct '08

15 Oct '08

Hi to all, I use Dovecot dovecot11-1.1.3-5.1 from OpenSuSE 11, when i send a mail from the logs i read: Oct 8 10:36:31 mail postfix/pipe[17934]: A360A851: to=<mediatest(a)atpss.net>, relay=dovecot, delay=0.22, delays=0.18/0/0/0.04, dsn=4.3.0, status=deferred (temporary failure. Command output: Can't open log file /var/log/dovecot-err.log: Permission denied ) But, log files seems ok: -rw------- 1 777 root 56 Oct 8 10:35 dovecot-err.log -rw------- 1 777 root 302 Oct 8 10:35 dovecot-inf.log Also processess seems right: root 17965 0.0 0.0 2052 584 ? Ss 10:35 0:00 /usr/sbin/dovecot root 17966 0.0 0.2 8564 2172 ? S 10:35 0:00 dovecot-auth root 17967 0.0 0.2 8564 2356 ? S 10:35 0:00 dovecot-auth -w dovecot 17969 0.0 0.1 3476 1572 ? S 10:36 0:00 pop3-login dovecot 17970 0.0 0.1 3476 1572 ? S 10:36 0:00 pop3-login dovecot 17971 0.0 0.1 3476 1572 ? S 10:36 0:00 pop3-login dovecot 17972 0.0 0.1 3484 1584 ? S 10:36 0:00 imap-login dovecot 17973 0.0 0.1 3484 1584 ? S 10:36 0:00 imap-login dovecot 17974 0.0 0.1 3484 1584 ? S 10:36 0:00 imap-login Here is my config: log_path: /var/log/dovecot-err.log info_log_path: /var/log/dovecot-inf.log protocols: imap pop3 ssl_disable: yes disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login first_valid_uid: 51 last_valid_uid: 51 mail_location: maildir:/var/mail/virtual/%n@%d mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib64/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib64/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/modules/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %Mf auth default: passdb: driver: pam passdb: driver: sql args: /etc/dovecot/dovecot-mysql.conf userdb: driver: passwd userdb: driver: sql args: /etc/dovecot/dovecot-mysql.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 plugin: quota: maildir quota_warning: storage=80%% /usr/local/bin/quota-warning.sh 80 So, what can be the problem? Thanks, Claudio Prono. -- ------------ Claudio Prono Systems Development @ Atpss.net Srl, Divisione Implementazione Sistemi Via San Bernardino, 17 - 10137 Torino (TO) - IT Tel +39-011.32.72.100 Fax +39-011.32.46.497 PGP Fingerprint: 75C2 4049 E23D 2FBF A65F 40DB EA5C 11AC C2B0 3647 Disclaimer: http://atpss.net/disclaimer ------------

6 10

[Dovecot] Multiple IMAP connections.
by Roderick A. Anderson 15 Oct '08

15 Oct '08

There is a project (catmail) to build a webmail front end using the Catalyst Framework and the question came up about multiple connections per user. Specifically the question was: "The IMAP server can support multiple connections, but do you think the backend should support multiple connections for separate folders or one IMAP connection for one session?" Since my servers are too low volume to have hit any kind of resource limits, and I believe Thunderbird mostly does the "Right Thing", I'm asking how you, that have higher volume systems, would prefer connections handled. Also I believe there was a thread, or two :-), on the way some of the other webmail programs handle connections. Not good? Thanks for all insights and suggestions. Rod --

1 0

[Dovecot] Bug: MODSEQ FETCH return (?)
by Michael M. Slusarz 15 Oct '08

15 Oct '08

Hi Timo - hope things are well :) Playing around with the CONDSTORE (RFC 4551) stuff while testing the IMAP lib I am writing and I think I have found a bug in Dovecot 1.2. RFC 4551 [4] identifies the FETCH response to a MODSEQ request as follows: fetch-mod-resp = "MODSEQ" SP "(" permsg-modsequence ")" with 'permsg-modsequence' defined as: permsg-modsequence = mod-sequence-value ;; per message mod-sequence Unfortunately, Dovecot 1.2 is returning as follows: S: * 1 FETCH (UID 1 MODSEQ 3 FLAGS (\Flagged \Deleted \Recent)) Where, under the RFC, I believe it should be: S: * 1 FETCH (UID 1 MODSEQ (3) FLAGS (\Flagged \Deleted \Recent)) Here is a proposed patch (WARNING: my C skills = zero): # HG changeset patch # User root(a)bigworm.curecanti.org # Date 1224040830 21600 # Branch HEAD # Node ID 0c0615bc6bf74ccf109049e251a30b8ab5f8ea67 # Parent 40df4d4cced09fa72d37581d0c526416d2c6e3de per RFC 4551, MODSEQ FETCH responses need to be wrapped in (). diff -r 40df4d4cced0 -r 0c0615bc6bf7 src/imap/imap-fetch.c --- a/src/imap/imap-fetch.c Tue Oct 14 14:52:44 2008 +0300 +++ b/src/imap/imap-fetch.c Tue Oct 14 21:20:30 2008 -0600 @@ -749,7 +749,7 @@ modseq = mail_get_modseq(mail); if (ctx->client->highest_fetch_modseq < modseq) ctx->client->highest_fetch_modseq = modseq; - str_printfa(ctx->cur_str, "MODSEQ %llu ", + str_printfa(ctx->cur_str, "MODSEQ (%llu) ", (unsigned long long)modseq); return 1; } Hope this helps/is correct. michael

2 1

[Dovecot] Search results as virtual folders
by Daniel L. Miller 15 Oct '08

15 Oct '08

This may have been asked before - if so I apologize. Some mail clients - like Thunderbird - can provide a virtual folder or "saved search" to provide a list of mails that match certain criteria. Obviously, unless special client-side caching is implemented - which Thunderbird (my own client of choice) - does not, generation of these folders take time. Does Dovecot have any provision for generating such virtual folders? Possibly as the result of a sieve script? -- Daniel

3 2

Re: [Dovecot] Panic 1.1.4
by Timo Sirainen 14 Oct '08

14 Oct '08

On Oct 14, 2008, at 6:03 PM, Andre Hübner wrote: > Hi, > >> If you still have the core, could you do: > >> fr 6 >> p *stream oops, sorry, I meant actually: fr 6 p *stream.parent p *stream.parent.real_stream >> Does this happen often? Could you also try with the attached patch? >> It >> should trigger the assert earlier. > > yes, often. i patched the file, repackaged and 2 hours later i got > next core. i did gdb again: Hmm. The backtrace didn't change.

1 0

[Dovecot] Compatibility namespaces containing a leading ~ appear to be broken in 1.1.x
by David McBride 14 Oct '08

14 Oct '08

Greetings, I'm building a new 1.1.x-series Dovecot installation, and I have a problem setting up compatibility namespaces intended to handle old client-side settings. (See also: http://wiki.dovecot.org/Namespaces#head-0f3ab5ecab632eb517866c3508021e0605d…) I have produced a slightly cut-down example configuration which demonstrates the problem: # dovecot -n # 1.1.3: /etc/dovecot/dovecot.conf protocols: imap imaps pop3 pop3s ssl_cert_file(default): /etc/ssl/certs/imap.pem ssl_cert_file(imap): /etc/ssl/certs/imap.pem ssl_cert_file(pop3): /etc/ssl/certs/pop.pem ssl_key_file(default): /etc/ssl/private/imap.key ssl_key_file(imap): /etc/ssl/private/imap.key ssl_key_file(pop3): /etc/ssl/private/pop.key disable_plaintext_auth(default): yes disable_plaintext_auth(imap): yes disable_plaintext_auth(pop3): no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_max_processes_count: 1024 max_mail_processes: 32768 verbose_proctitle: yes first_valid_uid: 100 mail_privileged_group: mail mail_location: mbox:~/IMAP:INBOX=~/.email:INDEX=/data/dovecot-indexes/%1u/%u mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): zlib quota imap_quota mail_plugins(imap): zlib quota imap_quota mail_plugins(pop3): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): delay-newmail outlook-idle tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail outlook-idle tb-extra-mailbox-sep imap_client_workarounds(pop3): pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %08Xv%08Xu namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: private separator: / prefix: Mail/ location: mbox:~/Mail:INDEX=/data/dovecot-indexes/%1u/%u hidden: yes subscriptions: yes namespace: type: private separator: / prefix: ~/Mail/ location: mbox:~/Mail:INDEX=/data/dovecot-indexes/%1u/%u hidden: yes subscriptions: yes auth default: mechanisms: gssapi gss-spnego plain passdb: driver: pam userdb: driver: passwd plugin: quota: fs:Disk Quota Given this configuration, a Thunderbird client using a folder prefix of Mail/ is successfully able to enumerate (and access) non-INBOX folders: 5 list "" "Mail/%" * LIST (\Noselect \HasChildren) "/" "Mail/archive"^M * LIST (\NoInferiors \UnMarked) "/" "Mail/Trash"^M 5 OK List completed.^M 6 list "" "Mail/%/%" * LIST (\Noselect \HasChildren) "/" "Mail/archive/"^M * LIST (\NoInferiors \UnMarked) "/" "Mail/archive/INBOX-2008-10"^M * LIST (\NoInferiors \UnMarked) "/" "Mail/archive/INBOX-2008-09"^M [...] 6 OK List completed.^M 7 list "" "INBOX" * LIST (\NoInferiors \UnMarked) "/" "INBOX"^M 7 OK List completed.^M 8 list "" "Mail/archive" * LIST (\Noselect \HasChildren) "/" "Mail/archive"^M 8 OK List completed.^M ... Whereas a client using a folder prefix of ~/Mail/ is only able to see its INBOX: 5 list "" "~/Mail/%" 5 OK List completed.^M 6 list "" "~/Mail/%/%" 6 OK List completed.^M 7 list "" "INBOX" * LIST (\NoInferiors \UnMarked) "/" "INBOX"^M 7 OK List completed.^M 8 create "~/Mail/Trash" 8 NO Mailbox exists.^M Note that even though it was unable to execute a listing of the hidden namespace, it was clearly aware of it, and able to discover that the 'Trash' mailbox that Thunderbird automatically tries to create already existed in the underlying filesystem. Through experimentation, it appears that Dovecot 1.1.x doesn't correctly handle namespaces that *begin* with the character '~'; using ~ in the middle of a prefix, e.g. "flibble~/Mail", worked as documented, leading me to suspect a bug in the ~-expansion handling of Dovecot. (Dovecot 1.0.x worked as expected in this configuration.) Unfortunately, an hg bisect run on recent Dovecot-1.1 versions didn't expose the offending commit that introduced this problem, so this bug (assuming it is a bug) was probably inadvertently introduced during the 1.1-development series. Can anyone suggest where in the source tree I should start looking to correct this behaviour? Cheers, David -- David McBride <dwm(a)tastycake.net>

2 2

[Dovecot] Get an extra_field in login process
by Alexandre Estanislau 14 Oct '08

14 Oct '08

Hi. I have an auth plugin where an extra_field is filled. Somewhere, in login process I need to log the authentication command and the extra_field, I was trying to do it in login-common/client-common.c, adding a new variable to login_log_format_elements and trying to get the fileld from environment, but, I could not find any way to do it, the getenv returns null everytime. Anyone can suggest something to do it ? Is there a way that I get the extra_fields from request in login process ? Thanks -- Alexandre *Estanislau* Puhl - PX3D1363 alexandre at estanislau dot org http://estanislau.org Jabber: alexandre at estanislau dot org QAP 27,335 MHz AM - Unidade 08 do Agrupamento Sentinela

2 1

[Dovecot] busy loop?
by Cor Bosman 13 Oct '08

13 Oct '08

I just noticed that about 25% of our servers have imap processes hanging: 97996 xxxxxx 1 101 0 4096K 2052K CPU3 2 180:54 98.93% imap All with 98-100% cpu load, taking up 1 or more CPUs. dovecot 1.1.2, FreeBSD 6.2. is this caused by the deadlock issue thats fixed in 1.1.4? Cor

2 1