November 2009 - dovecot

[Dovecot] Two different issues with dovecot 1.1.13
by Stuart Rowan 20 Jan '10

20 Jan '10

Hi Timo, I'm using Debian backports.org dovecot-imapd, 1:1.1.13-2~bpo50+1. I've got two different issues, a reoccurring, seemingly harmless one about "Corrupted index cache file" (boring.txt) and one that went bonkers over the least few days (bt.txt) causing much spamming of the logs. I've made bt.txt stop happening by just rm'ing dovecot.index* : find Maildir/ -type f -name 'dovecot.index*' -print0 | xargs -0r rm ... which I realise may not be the most helpful thing to have done from your point of view. Anyway I just wanted to let you know about both of these. If one or both are already known about or already fixed by a later version of Dovecot, great and sorry for the noise! If not, and you want any more information etc., let me know. Many thanks, Stu. dovecot: 2009-07-28 10:23:16 Error: IMAP(strr): Corrupted index cache file /home/local/strr/Maildir/dovecot.index.cache: record continues outside its allocated size dovecot: 2009-07-28 10:23:56 Error: IMAP(strr): Corrupted index cache file /home/local/strr/Maildir/dovecot.index.cache: record continues outside its allocated size dovecot: 2009-07-28 08:05:54 Panic: IMAP(strr): file index-mail-headers.c: line 77 (index_mail_parse_header_finish): assertion failed: ((match[match_idx] & HEADER_MAT CH_FLAG_FOUND) == 0) dovecot: 2009-07-28 08:05:54 Error: IMAP(strr): Raw backtrace: imap [0x48dbf2] -> imap [0x48dc73] -> imap [0x48d346] -> imap(index_mail_parse_header+0x617) [0x4545f7] -> imap [0x483284] -> imap [0x483478] -> imap(i_stream_read+0x2c) [0x491d7c] -> imap(i_stream_read_data+0x46) [0x491f16] -> imap(message_get_body_size+0x108) [0x4885 a8] -> imap [0x4226c3] -> imap(imap_fetch+0x162) [0x420852] -> imap(cmd_fetch+0x23c) [0x41a27c] -> imap [0x41e46c] -> imap [0x41e51d] -> imap [0x41eb25] -> imap(clien t_input+0x63) [0x41efd3] -> imap(io_loop_handler_run+0xcb) [0x49515b] -> imap(io_loop_run+0x18) [0x4945c8] -> imap(main+0x3da) [0x426bea] -> /lib/libc.so.6(__libc_sta rt_main+0xe6) [0x7f0e851941a6] -> imap [0x418859] dovecot: 2009-07-28 08:05:54 Error: child 20007 (imap) killed with signal 6 (core dumps disabled) dovecot: 2009-07-28 08:05:56 Panic: IMAP(strr): file index-mail-headers.c: line 77 (index_mail_parse_header_finish): assertion failed: ((match[match_idx] & HEADER_MAT CH_FLAG_FOUND) == 0) dovecot: 2009-07-28 08:05:56 Error: IMAP(strr): Raw backtrace: imap [0x48dbf2] -> imap [0x48dc73] -> imap [0x48d346] -> imap(index_mail_parse_header+0x617) [0x4545f7] -> imap [0x483284] -> imap [0x483478] -> imap(i_stream_read+0x2c) [0x491d7c] -> imap(i_stream_read_data+0x46) [0x491f16] -> imap(message_get_body_size+0x108) [0x4885 a8] -> imap [0x4226c3] -> imap(imap_fetch+0x162) [0x420852] -> imap(cmd_fetch+0x23c) [0x41a27c] -> imap [0x41e46c] -> imap [0x41e51d] -> imap [0x41eb25] -> imap(clien t_input+0x63) [0x41efd3] -> imap(io_loop_handler_run+0xcb) [0x49515b] -> imap(io_loop_run+0x18) [0x4945c8] -> imap(main+0x3da) [0x426bea] -> /lib/libc.so.6(__libc_sta rt_main+0xe6) [0x7f2edf08b1a6] -> imap [0x418859] dovecot: 2009-07-28 08:05:56 Error: child 20984 (imap) killed with signal 6 (core dumps disabled)

2 5

[Dovecot] fts squat non-english search for 2 words
by vuser1＠test123.ru 19 Jan '10

19 Jan '10

Hello, It looks I encoutered a bug or misconfiguration. fts_squat search for subject and body works excellent for English mails. For non-English (in particular, Russian) it works only when query consists of 1 word. Phrases - 2 and more words - always returns nothing. Example: search for "planet" ("планета") returns results, search for "Earth" ("Земля") also returns results, but "planet Earth" ("планета Земля") returns nothing. But there are emails having exact phrase "planet Earth". This problem occurs only for non-English queries, both for search in subject and in email body. I tried web-mail Horde 3.2 and Thunderbird. I turned fts plugin off and it correctly finds phrases with 2 and more russian words! So problem is squat. Is it a bug or known config issue? OS: Debian 5.0, installed as openVZ container inside Ubuntu 8.04. Dovecot: 1.2.4, from backports.org Ext3 filesys dovecot -n -------------- # 1.2.4: /etc/dovecot/dovecot.conf # OS: Linux 2.6.24-24-openvz i686 Debian 5.0.3 simfs log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/test123/test123.full.crt ssl_key_file: /etc/ssl/test123/priv/test123.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login last_valid_uid: 500 mail_privileged_group: mail mail_location: maildir:/var/mail/%u mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): quota imap_quota fts fts_squat mail_plugins(imap): quota imap_quota fts fts_squat mail_plugins(pop3): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh lda: postmaster_address: postmaster(a)test123.ru hostname: test123.ru sendmail_path: /usr/sbin/sendmail auth_socket_path: /var/run/dovecot/auth-master mail_plugins: quota sieve log_path: info_log_path: auth default: mechanisms: plain login user: nobody passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: passwd userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: mail master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: mail plugin: acl: vfile:/etc/dovecot/acls trash: /etc/dovecot/trash.conf fts: squat fts_squat: partial=4 full=20 --------------- (with full=10 problem persists)

2 7

[Dovecot] Dovecot Proxy and environment variables
by BaQs 19 Jan '10

19 Jan '10

Dear list, First of all, I am French so excuse my bad english. I am currently using dovecot proxy feature, using sql backend. I "proxy" the imap/pop requests to another server, let's say server2. On server2, I use a script to keep up to date a "pop before smtp auth" database. I used mail_executable = /path/to/this/script.sh ... My problem is: the $IP environment variable on server2 is the proxy's IP. How could I give a variable from proxy to server2, or simply get the real client public IP ? Thanks a lot in advance for your answers Pierre. PS: let me also thank very much dovecot developers and community !

3 7

[Dovecot] problem with uidlist
by Cor Bosman 06 Jan '10

06 Jan '10

In both 1.2.6 and 1.2.7 (probably also before that, but dont have logs) im seeing quite a few of these: Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx): /var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist: Duplicate file entry at line 2650: 1257768394._smtp.mxdrop238.41874,S=17304:2,Sad (uid 34270 -> 37166) I never really looked at it because it doesnt seem to be causing many problems, but i just noticed something that probably hints at the cause.. Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx): /var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist: Duplicate file entry at line 2650: 1257768394._smtp.mxdrop238.41874,S=17304:2,Sad (uid 34270 -> 37166) Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx): /var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist: Duplicate file entry at line 2651: 1257769385._smtp.mxdrop155.53889,S=305996:2,Sad (uid 34271 -> 37167) Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx): /var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist: Duplicate file entry at line 2652: 1257769984._smtp.mxdrop107.23815,S=320781:2,Sabd (uid 34272 -> 37168) Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx): /var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist: Duplicate file entry at line 2670: 1257858287._smtp.mxdrop131.43563,S=26827:2,Sad (uid 34311 -> 37186) Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx): /var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist: Duplicate file entry at line 2671: 1257858303._smtp.mxdrop238.88513,S=2143:2,Sad (uid 34312 -> 37187) Nov 17 12:45:12 userimap10.xs4all.nl dovecot: IMAP(xxxxxxx): /var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist: Duplicate file entry at line 2672: 1257858868._smtp.mxdrop217.21298,S=8548:2,Sad (uid 34314 -> 37188) Nov 17 12:45:12 userimap7.xs4all.nl dovecot: IMAP(xxxxxxx): /var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist: Duplicate file entry at line 994: 1244800736._smtp.mxdrop237.59318,S=4573:2,Sad (uid 32467 -> 35510) Nov 17 12:45:12 userimap7.xs4all.nl dovecot: IMAP(xxxxxxx): /var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist: Duplicate file entry at line 995: 1244807927._smtp.mxdrop215.53275,S=439887:2,Sad (uid 32468 -> 35511) Nov 17 12:45:12 userimap7.xs4all.nl dovecot: IMAP(xxxxxxx): /var/spool/mail/dovecot-control/c/c0/xxxxxxx/INBOX/.INBOX/dovecot-uidlist: Duplicate file entry at line 996: 1244828590._smtp.mxdrop111.81795,S=3606:2,RSabd (uid 32469 -> 35512) This customer seems to be using our IMAP service from 2 machines at the same time. Possibly a webmail service and a desktop imap client. It may be that both machines are trying to manipulate the uidlist at the same time, causing who knows that kind of corruption. This could maybe explain some of the coredumps im seeing in 1.2.7? This is on Debian/Lenny, using NFS spool and NFS control/indexes. Timo, you asked in private mail a few days ago if we're sure the filenames are forever unique. Im sure. I double checked it in the source code. The above filenames are timestamp_hostname.pid, and that seems pretty unique right? Regards, Cor

3 2

[Dovecot] managesieve error
by Lampa 26 Dec '09

26 Dec '09

Hello, when using sieve-php and using LOGIN auth, getting: 2009-11-21T15:13:48+0100 managesieve-login: Panic: file client.c: line 449 (client_set_auth_waiting): assertion failed: (client->to_auth_waiting == NULL) 2009-11-21T15:13:48+0100 dovecot: Error: child 8474 (login) killed with signal 6 (core dumps disabled) (latest ip=127.0.0.1) I'm using dovecot 1.2.6 from package debian testing (1:1.2.6-1) -- Lampa

2 1

[Dovecot] Dovecot Deliver CPU usage
by Aravind Divakaran 23 Dec '09

23 Dec '09

Hi All, We have upgraded the dovecot from 1.0 to 1.2.8. After upgradation we have noticed that dovecot deliver process is utilizing a lot of CPU. As a result the mailsystem will stop working. When we limit the no of procs in postfix for dovecot deliver the CPU usage will come down. But we are not facing this problem with the previous version. Below is the log files for dovecot when the error occurs write_full(/home/vmail/dhanbank.co.in/vinukumarvm/temp.server.be3e5a849a909847) failed: No space left on device dovecot: deliver(user(a)example.com): stat(/tmp/Dovecot Delivery Mail) failed: No space left on device But my system has 54gb of free space for root filesystem where tmp residing. Can anyone help me to solve this issue. Rgds, Aravind

5 17

[Dovecot] Release candidates for Sieve v0.1.14 and ManageSieve v0.11.10 for Dovecot v1.2.8
by Stephan Bosch 19 Dec '09

19 Dec '09

Hello Dovecot users, I am getting closer to new releases for Sieve and ManageSieve for Dovecot v1.2.8. However, the release for Sieve includes a large restructuring of the sources to improve extension handling and to remove global state. That is why I first provide a few pre-releases for you to check first: Sieve http://www.rename-it.nl/dovecot/1.2/dovecot-1.2-sieve-0.1.14pre.tar.gz http://www.rename-it.nl/dovecot/1.2/dovecot-1.2-sieve-0.1.14pre.tar.gz.sig ManageSieve http://www.rename-it.nl/dovecot/1.2/dovecot-1.2-managesieve-0.11.10pre.tar.… http://www.rename-it.nl/dovecot/1.2/dovecot-1.2-managesieve-0.11.10pre.tar.… Check the NEWS files in the respective repositories for other changes included in these releases: http://hg.rename-it.nl/dovecot-1.2-sieve/file/216f58aae5c1/NEWS http://hg.rename-it.nl/dovecot-1.2-managesieve/file/a44ca0d89875/NEWS Feel free to test these release candidates and please notify me when you incur any problems. The real releases will follow this weekend. Regards, -- Stephan Bosch stephan(a)rename-it.nl

3 5

[Dovecot] commercial help
by Jim Pazarena 15 Dec '09

15 Dec '09

Contact info * Author: Timo Sirainen, tss(a)iki.fi <mailto:tss@iki.fi>. * Please use the Dovecot mailing list <http://www.dovecot.org/mailinglists.html> for questions about Dovecot. You don't have to subscribe to it. * Depending on your needs, commercial support may be available. Send a mail if you're interested. according to the above (from the dovecot.org webpage), "commercial support may be available". After spending the better part of today trying unsuccessfully, I am ready to "Send a mail if you're interested". But I do not know WHO to send it to. Likely Timo, but I do not want to be presumptuous. I am on FreeBSD with Dovecot 1.2.4 I need to have both POP3 & IMAP working to replace vm-pop3d (which was POP3 only). Local users are: mbox:/home/%u/mail/:INBOX=/mail/%u with password in /etc/passwd Virtual users (which vm-pop3d handled readily) are mbox:/home/VIRTUAL/%d/%u:INBOX/mail/VIRTUAL/%d/%u with password in /exim/etc/VIRTUAL/%d/passwd It is worth it to me to pay a knowledgeable person to create the correct conf file to make this happen. Who should I contact? Thanks, Jim Pazarena

4 4

[Dovecot] Problem using LDAP generating Userhome
by Thorsten Habich 11 Dec '09

11 Dec '09

Hi there, I'm trying to let dovecot generate the userhome directory dynamicly via user_attr. If "uid" is returned from LDAP before "mail" everything is fine, but if "mail" is delivered before "uid" the localpart of the mailaddress is used in home path. e.g.: Mail: testuser(a)domain.de UID : test123 sometimes home is /mail/store/testuser and sometimes /mail/store/test123 depending on the order the ldap server replies. I'm currently using version 1.2.7 of dovecot in testing and 1.1.6 in production. Both have the same "bug". Mails are stored on nfs, indexes are stored local. This is my dovecot-ldap.conf configuration: pass_filter = (|(&(objectClass=mailRecipient)(mail=%u))(&(objectClass=mailRecipient)(mailalternateAddress=%u))(&(objectClass=mailRecipient)(uid=%n))) pass_attrs = uid=user,userPassword=password,mailMessageStore=home=%$/telefonica.de/%Ln,mailFilterStore=sieve_dir=%$/telefonica.de/%Ln user_filter = (|(&(objectClass=mailRecipient)(mail=%u))(&(objectClass=mailRecipient)(mailalternateAddress=%u))(&(objectClass=mailRecipient)(uid=%n))) user_attrs = uid=user,mailMessageStore=home=%$/telefonica.de/%Ln,mailFilterStore=sieve_dir=%$/telefonica.de/%Ln The filter needs to be that way, because you need to be able to authenticate by uid or mailaddress. And the dovecot -n output: # 1.2.7: /etc/dovecot.conf # OS: Linux 2.6.18-92.1.13.el5 x86_64 Red Hat Enterprise Linux Server release 5.2 (Tikanga) base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot-info.log log_timestamp: %Y-%m-%d %H:%M:%S syslog_facility: local1 protocols: imap managesieve listen(default): *:143 listen(imap): *:143 listen(managesieve): *, [::] ssl: no login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(managesieve): /usr/libexec/dovecot/managesieve-login login_greeting: IMAP mail server ready. max_mail_processes: 200 mail_max_userip_connections(default): 50 mail_max_userip_connections(imap): 50 mail_max_userip_connections(managesieve): 10 verbose_proctitle: yes first_valid_uid: 1 mail_privileged_group: vmail mail_uid: 500 mail_gid: 500 mail_location: maildir:~/:INDEX=/mail/index/%Ln/ mailbox_idle_check_interval: 10 mail_debug: yes mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes lock_method: flock mbox_write_locks: dotlock fcntl mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(managesieve): /usr/libexec/dovecot/managesieve mail_plugins(default): expire autocreate mail_plugins(imap): expire autocreate mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(managesieve): /usr/lib64/dovecot/managesieve imap_client_workarounds(default): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(managesieve): managesieve_implementation_string(default): dovecot managesieve_implementation_string(imap): dovecot managesieve_implementation_string(managesieve): Cyrus timsieved v2.2.13 namespace: type: private separator: . prefix: INBOX. inbox: yes list: yes subscriptions: yes namespace: type: public separator: . prefix: Public. location: maildir:/mail/store/a.backend/shared list: yes subscriptions: yes lda: hostname: xxx mail_plugins: sieve expire mail_plugin_dir: /usr/lib64/dovecot/lda auth_socket_path: /var/run/dovecot/auth-master postmaster_address: xxx(a)telefonica.de sieve_global_dir: /mail/store/backend/sieve/ auth default: mechanisms: plain login username_format: %Lu verbose: yes debug: yes process_size: 512 passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf socket: type: listen client: path: /var/run/dovecot/auth-client mode: 432 user: postfix group: vmail master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail plugin: sieve: ~/%Ln.sieve sieve_dir: ~/sieve sieve_storage: ~/ expire: Trash 7 Trash/* 7 Junk 30 expire_dict: proxy::expire autocreate: INBOX.Trash autocreate2: INBOX.Drafts autocreate3: INBOX.Sent autosubscribe: INBOX.Trash autosubscribe2: INBOX.Drafts autosubscribe3: INBOX.Sent dict: expire: db:/var/lib/dovecot/expire.db Thanks in advanced. Thorsten Habich

2 2

[Dovecot] dovecot: Fatal: Time just moved backwards by 3909 seconds.
by Udo Rader 10 Dec '09

10 Dec '09

Hi, after setting up a KVM based virtual guest on one of our virtualization servers, we see dovecot die on that virtual guest regularly because of an alleged time shift. The virtual guest is Centos 5.4 based with dovecot 1.2.8 (at first we also tried with the original 1.0.7 (?) dovecot shipped with Centos). I wrote "alleged time shift" because there is no timeshift whatsoever, or at least I don't notice it anywhere else. ntp is - of course ;-) - up and running and no other applications have time troubles and I've even disabled the local clock as a time reference. As a proof that the time did not shift see this log excerpt: ------CUT------- Nov 29 14:02:22 athesia dovecot: IMAP(atester): Connection closed Nov 29 14:04:11 athesia dovecot: imap-login: Time just moved backwards by 4397 seconds. This might cause a lot of problems, so I'll just kill myself now. http://wiki.dovecot.org/TimeMovedBackwards ------CUT------- So as you see, dovecot successfully logged a message on 14:02 and then complained about the time having shifted 4397 seconds on 14:04 ... Thanks in advance -- Udo Rader, CTO http://www.bestsolution.at http://riaschissl.blogspot.com

4 12