- dovecot - dovecot.org

[Dovecot] Virtual mailboxes not always up to date?
by Patrick Nagel 10 Dec '09

10 Dec '09

Hi, I have been experimenting with the Virtual mailboxes plugin [1] recently. This is my setup: the following dovecot-virtual file exists in a virtual mailbox called "todo": ------ dovecot-virtual ------ * -Trash -Trash/* -Spam -Spam/* OR KEYWORD $TODO KEYWORD todo ----------------------------- so if any mail in any mailbox except Trash+Spam has the keyword "$TODO" (that's what KMail sets when you mark a message as "action item") or "todo" (that's what Thunderbird sets when you put the Todo tag on a message), it should be shown in that 'todo' virtual mailbox. This works for the first time after I edit the dovecot-virtual file and then access the virtual mailbox (all mails I expect to be shown in the todo mailbox are there) - but further accesses always show the same content, even though I removed the keyword from one of the mails (and I can confirm that the keyword has actually been removed by dovecot, since the filename doesn't contain the letter corresponding to the keyword anymore). It seems that changing the dovecot-virtual file triggers the execution of the "search program" - but shouldn't simply accessing the virtual mailbox also trigger it? Or is there a (undocumented?) timeout between search program executions? Patrick. [1]: http://wiki.dovecot.org/Plugins/Virtual # dovecot -n # 1.2.6: /etc/dovecot/dovecot.conf # OS: Linux 2.6.22-vs2.2.0.7-gentoo x86_64 Gentoo Base System release 2.0.1 protocols: imaps managesieve listen: x.x.x.x ssl: required ssl_ca_file: /etc/ssl/dovecot/sub.class1.server.ca.crt ssl_cert_file: /etc/ssl/dovecot/ssl.crt ssl_key_file: /etc/ssl/dovecot/ssl.key login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(managesieve): /usr/libexec/dovecot/managesieve-login mail_max_userip_connections(default): 20 mail_max_userip_connections(imap): 20 mail_max_userip_connections(managesieve): 10 mail_location: maildir:~/.maildir mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(managesieve): /usr/libexec/dovecot/managesieve mail_process_size: 2048 mail_plugins(default): virtual fts fts_squat mail_plugins(imap): virtual fts fts_squat mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(managesieve): /usr/lib64/dovecot/managesieve imap_client_workarounds(default): tb-extra-mailbox-sep imap_client_workarounds(imap): tb-extra-mailbox-sep imap_client_workarounds(managesieve): namespace: type: private separator: . inbox: yes list: yes subscriptions: yes namespace: type: private separator: . prefix: virtual. location: virtual:~/.maildir/virtual list: yes subscriptions: yes lda: postmaster_address: postmaster@x mail_plugins: sieve virtual auth default: passdb: driver: pam args: * userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix plugin: fts: squat fts_squat: partial=4 full=10 -- STAR Software (Shanghai) Co., Ltd. http://www.star-group.net/ Phone: +86 (21) 3462 7688 x 826 Fax: +86 (21) 3462 7779 PGP key: E883A005 https://stshacom1.star-china.net/keys/patrick_nagel.asc Fingerprint: E09A D65E 855F B334 E5C3 5386 EF23 20FC E883 A005

3 7

[Dovecot] 2.0.alpha3: imap-login segfault in hex_to_binary
by Mike Abbott 10 Dec '09

10 Dec '09

Dovecot-2.0.alpha3's imap-login segfaulted with this backtrace: 0 hex_to_binary 1 master_send_request 2 anvil_input 3 io_loop_handler_run 4 io_loop_run 5 master_service_run 6 main 7 start Looks to me like auth_client_get_cookie() returned NULL in master_send_request().

2 1

[Dovecot] Bug report
by Bryan Seitz 10 Dec '09

10 Dec '09

One of my users is causing dovecot to exit with signal 6 and generate a core file. Seems to be in the mbox code. Dovecot configuration: # 1.2.4: /usr/local/etc/dovecot.conf # OS: FreeBSD 7.2-STABLE amd64 protocols: imap imaps pop3s listen(default): localhost:143 listen(imap): localhost:143 listen(pop3): * ssl_listen(default): *:993 ssl_listen(imap): *:993 ssl_listen(pop3): *:995 ssl_cert_file: /etc/ssl/certs/mail.bsd-unix.net.pem ssl_key_file: /etc/ssl/certs/mail.bsd-unix.net.pem login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login verbose_proctitle: yes first_valid_uid: 1000 first_valid_gid: 100 mail_privileged_group: mail mail_location: mbox:~/mail/:INBOX=/var/mail/%u mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_process_size: 1024 mail_plugins(default): fts fts_squat mail_plugins(imap): fts fts_squat mail_plugins(pop3): mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh lda: postmaster_address: postmaster(a)example.com sendmail_path: /usr/sbin/sendmail auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 384 user: postfix group: postfix plugin: fts: squat FreeBSD triton.bsd-unix.net 7.2-STABLE FreeBSD 7.2-STABLE #17: Sat Oct 3 15:47:44 EDT 2009 root@triton.bsd-unix.net:/usr/obj/usr/src/sys/TRITON amd64 dovecot-1.2.4_1 Secure and compact IMAP and POP3 servers CPU Arch: x86_64 Filesystem: UFS (gdb) bt full #0 0x0000000040b2989c in kill () from /lib/libc.so.7 No symbol table info available. #1 0x0000000040b286bb in abort () from /lib/libc.so.7 No symbol table info available. #2 0x00000000004d9165 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:160 backtrace = 0x40d4d048 "" #3 0x00000000004d9f7e in i_internal_fatal_handler (type=Could not find the frame base for "i_internal_fatal_handler". ) at failures.c:440 No locals. #4 0x00000000004d944a in i_panic (format=Could not find the frame base for "i_panic". ) at failures.c:207 args = Could not find the frame base for "i_panic". (gdb) bt #0 0x0000000040b2989c in kill () from /lib/libc.so.7 #1 0x0000000040b286bb in abort () from /lib/libc.so.7 #2 0x00000000004d9165 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:160 #3 0x00000000004d9f7e in i_internal_fatal_handler (type=Could not find the frame base for "i_internal_fatal_handler". ) at failures.c:440 #4 0x00000000004d944a in i_panic (format=Could not find the frame base for "i_panic". ) at failures.c:207 #5 0x000000000045aea8 in istream_raw_mbox_get_body_size (stream=0x40d76070, expected_body_size=18446744073709551615) at istream-raw-mbox.c:578 #6 0x00000000004622f3 in mbox_sync_read_next_mail (sync_ctx=0x7fffffffe290, mail_ctx=0x7fffffffe0b0) at mbox-sync.c:151 #7 0x00000000004649f6 in mbox_sync_loop (sync_ctx=0x7fffffffe290, mail_ctx=0x7fffffffe0b0, partial=false) at mbox-sync.c:1004 #8 0x0000000000465c00 in mbox_sync_do (sync_ctx=0x7fffffffe290, flags=MBOX_SYNC_UNDIRTY) at mbox-sync.c:1535 #9 0x00000000004667b8 in mbox_sync_int (mbox=0x40d23848, flags=MBOX_SYNC_UNDIRTY, lock_id=0x7fffffffe4b8) at mbox-sync.c:1842 #10 0x0000000000466a26 in mbox_sync (mbox=0x40d23848, flags=MBOX_SYNC_UNDIRTY) at mbox-sync.c:1894 #11 0x0000000000466b8a in mbox_storage_sync_init (box=0x40d23848, flags=65) at mbox-sync.c:1936 #12 0x0000000000488d34 in mailbox_sync_init (box=0x40d23848, flags=65) at mail-storage.c:562 #13 0x0000000000488dfa in mailbox_sync (box=0x40d23848, flags=65, status_items=239, status_r=0x7fffffffe580) at mail-storage.c:593 #14 0x000000000041f77a in select_open (ctx=0x40d4d148, mailbox=0x40d52170 "Drafts", readonly=false) at cmd-select.c:273 #15 0x000000000041fba0 in cmd_select_full (cmd=0x40d4d048, readonly=false) at cmd-select.c:381 #16 0x000000000041fbda in cmd_select (cmd=0x40d4d048) at cmd-select.c:388 #17 0x0000000000423164 in client_command_input (cmd=0x40d4d048) at client.c:611 #18 0x00000000004233a1 in client_command_input (cmd=0x40d4d048) at client.c:660 #19 0x00000000004234e3 in client_handle_next_command (client=0x40d45030, remove_io_r=0x7fffffffe73d) at client.c:701 #20 0x0000000000423568 in client_handle_input (client=0x40d45030) at client.c:713 #21 0x00000000004236d4 in client_input (client=0x40d45030) at client.c:752 #22 0x00000000004e5da9 in io_loop_handler_run (ioloop=0x40d1f0c0) at ioloop-kqueue.c:149 #23 0x00000000004e4dfb in io_loop_run (ioloop=0x40d1f0c0) at ioloop.c:335 #24 0x000000000042f016 in main (argc=3, argv=0x7fffffffe8a0, envp=0x7fffffffe8c0) at main.c:327 Log messages: Nov 11 23:11:20 triton dovecot: IMAP(sam): Panic: file istream-raw-mbox.c: line 578 (istream_raw_mbox_get_body_size): assertion failed: (rstream->body_offset != (uoff_t)-1) Nov 11 23:11:20 triton kernel: pid 35530 (imap), uid 3033: exited on signal 6 (core dumped) Any help would be appreciated! -- Bryan G. Seitz

2 1

[Dovecot] expire_dict - Folder names with spaces
by michael＠euroinfissi.eu 10 Dec '09

10 Dec '09

Hi! How to deal with IMAP foldernames that include spaces when setting up the expire tool for them? I have the folder Elementy usuni&ARk-te but expire: Wiadomo&AVs-ci-&AVo-mieci 7 Elementy?usuni&ARk-te 7 INBOX 365 does not work... and expire: Wiadomo&AVs-ci-&AVo-mieci 7 Elementy usuni&ARk-te 7 INBOX 365 produces the following: Stopping Dovecot Imap: [ OK ] Starting Dovecot Imap: ☺ILoading modules from directory: /usr/lib64/dovecot/imap ☺IModule loaded: /usr/lib64/dovecot/imap/lib10_quota_plugin.so ☺IModule loaded: /usr/lib64/dovecot/imap/lib11_imap_quota_plugin.so ☺IModule loaded: /usr/lib64/dovecot/imap/lib20_autocreate_plugin.so ☺IModule loaded: /usr/lib64/dovecot/imap/lib20_expire_plugin.so ☺IEffective uid=99, gid=99, home=/tmp ☺IQuota root: name=user backend=dict args=:proxy::quota ☺IQuota rule: root=user mailbox=* bytes=1073741824 messages=0 ☺IQuota warning: bytes=1020054732 (95%) messages=0 command=/usr/local/bin/quota-warning.sh 95 ☺IQuota warning: bytes=858993459 (80%) messages=0 command=/usr/local/bin/quota-warning.sh 80 ☺Iexpire: pattern=Wiadomo&AVs-ci-&AVo-mieci type=expunge secs=604800 ☺Iexpire: pattern=Elementy type=expunge secs=0 ☺Iexpire: pattern=7 type=expunge secs=0 ☺Fexpire: Missing expire days for mailbox '365' Error: imap dump-capability process returned 89 Fatal: Invalid configuration in /etc/dovecot.conf [FAILED] My conf: # OS: Linux 2.6.30.9-96.fc11.x86_64 x86_64 Fedora release 11 (Leonidas) info_log_path: /var/log/maillog protocols: imaps ssl_listen: * ssl: required ssl_cert_file: /etc/pki/certs/dovecot/dovecot.pem ssl_key_file: /etc/pki/certs/dovecot/dovecot_key.pem ssl_cipher_list: ALL:!ADH!LOW:!SSLv2:!EXP:+HIGH:+MEDIUM disable_plaintext_auth: yes verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable: /usr/libexec/dovecot/imap-login mail_privileged_group: dovecot_mail mail_debug: yes mail_plugins: autocreate quota imap_quota expire lda: postmaster_address: root@localhost auth default: passdb: driver: sql args: /etc/dovecot-sql.conf userdb: driver: sql args: /etc/dovecot-sql.conf plugin: quota: dict:user::proxy::quota quota_rule: *:storage=1048576 quota_warning: storage=95%% /usr/local/bin/quota-warning.sh 95 quota_warning2: storage=80%% /usr/local/bin/quota-warning.sh 80 expire: Wiadomo&AVs-ci-&AVo-mieci 7 Elementy?usuni&ARk-te 7 INBOX 365 expire_dict: proxy::expire dict: quota: mysql:/etc/dovecot-dict-quota.conf expire: mysql:/etc/dovecot-dict-expire.conf Thanks! Michael Lattari

3 4

[Dovecot] Patch: event port-based ioloop and notify
by Emanuele Pucciarelli 10 Dec '09

10 Dec '09

Greetings, thanks to all of you who work on Dovecot! I have prepared a small patch to support Solaris 10 and Opensolaris' event port mechanism for both the ioloop and the notify subsystems. It seems to work fine for me, but I haven't conducted any extensive testing. It would be great if someone could review and/or test it (and if it could eventually enter the code base). I have uploaded a copy at http://cr.opensolaris.org/~e.p/dovecot/dovecot-01-fen.diff . Thanks! -- Emanuele

2 3

[Dovecot] Mail.app + dovecot 1.2 + POP3
by Rodolfo Gonzalez Gonzalez 10 Dec '09

10 Dec '09

Hello all, I have an user with MacOS X 10.5 Mail.app 3.5, which is connecting using POP3 to a dovecot 1.2 server. He has a 512 kbps link, which he saturates downloading large files. While the bandwidth is saturated, Mail.app shows the POP3 account as "offline" (I haven't used that client, but it seems that it doesn't retry the connection if it fails at some moment). He argues that he has some other POP3 and IMAP4 accounts configured in the same app, and those accounts never go "offline" even if the bw usage is at 100% (those other accounts are not on my servers and I don't know which sw are they using). My question is, could dovecot be somehow involved in this "offline" issue? (something like the past IDLE issues with Outlook 2007?) Thanks in advanced.

2 3

Re: [Dovecot] After upgrading to 1.2 - absolute path questions
by Richard Stockton 09 Dec '09

09 Dec '09

Hi Timo, Okay, I'm confused now... Here is what we are using with 1.1.19: password_query = SELECT password, home as userdb_home, 5000 as userdb_uid, 5000 as userdb_gid FROM mailbox where userid = '%n' mail_location = maildir:/mail/%h:INDEX=MEMORY The "/mail/" dir contains 26 NFS mounted partitions. Will this still work in 1.2? I have been holding off upgrading because it sounded like I would have to add the "/mail/" prefix to all the "home" directories in my user DB, and with 5 instances of Dovecot running off the same DB, I would have to shut down the pop/imap servers for the upgrade, something that would NOT make my boss happy. However, if I can leave the configs and the database as they are, the update will not be a big problem. Please let me know if I can use the old configs with Dovecot 1.2.x, I would love to be using the latest and greatest. And BTW, thanks again for all your hard work, Dovecot is a wonderful piece of software, and we really appreciate it! - Richard

2 12

[Dovecot] v2.0.alpha3 released
by Timo Sirainen 09 Dec '09

09 Dec '09

http://dovecot.org/releases/2.0/alpha/dovecot-2.0.alpha3.tar.gz http://dovecot.org/releases/2.0/alpha/dovecot-2.0.alpha3.tar.gz.sig Nothing horribly exciting since alpha2, but since I did other releases I thought I might as well release this one too. - Most of the stuff in master.conf is now built into binaries so the file itself is small. dovecot -n also doesn't show the defaults anymore. - Settings now have size and time types. So instead of always wondering if the units are bytes or kilobytes or megabytes, you can just use e.g. 100k, 100M values. And the same for 100 secs, 100 mins, 100 hours, etc. - verbose_proctitle=yes was pretty useless in alpha2 after the pre/post-login changes. I've now enabled Linux proctitle hack that allows changing process title (in ps). It now shows e.g. the current command being executed by imap process. There is also some hope of getting a proper setproctitle() function to Linux, the patch just got to -mm code tree today! - anvil process now stays alive across SIGHUPs. So "doveadm who" no longer forgets users after SIGHUP. - remote_ip IP {} and local_ip IP {} changed to remote host|IP {} and local host|IP {}, i.e. if it's not an IP Dovecot does host name lookup. - Using host names is also important to get TLS SNI working. With TLS SNI support if client supports it, you can serve multiple certificates with a single IP address. - LMTP server adds now Received: header - Fixes to LMTP server - Fixes to dsync Some of the larger changes left: - plugin API probably needs to be changed, so that it's possible to handle multiple users by same process but with different plugins - config process is too slow - rawlog no longer works because of how pre/post-login works now. it probably needs to be changed into a service and configured something like: service imap-rawlog { executable = rawlog imap-real /home/%u unix_listener imap { } } service imap { unix_listener imap-real { } } The point would be that when imap-login connects to imap, it actually connects to rawlog which then connects to imap-real and starts proxying the connection. A few problems: 1) userdb lookup is done by imap process, so rawlog doesn't know user's home directory. Perhaps more code could be added so that rawlog requests imap process to send it back, but maybe that's too much trouble. That's why I thought just adding a 3rd /home/%u parameter, where rawlog expands %u to username since that's at least known. 2) Currently it's not possible to remove default internal unix_listeners, so the above config wouldn't actually work anyway. I guess there needs to be some new syntax like "!unix_listener imap" or something that removes it.. Then again, a nice thing about this is that the rawlog will work with all types of services, even imap-logins, although that's going to be almost useless for SSL connections.

3 9

[Dovecot] How do i translate the old default_mail_env setting?
by Gary Kline 09 Dec '09

09 Dec '09

Hi, First, I am new to dovecot. Before my Jan '08 meltdown, sendmail was sufficient. A friend set up dovecot and since things just-worked, I was happy with that. Now I have a new mailserver and what was installed nearly two years ago fails. My pal installed things in dovecot.conf this way: default_mail_env = maildir:~/Maildir but the new mail environment is too different to be readily understood. --At least for me!-- The "%u" variable is confusing; so in the "%h" ($HOME [?]) variable... Would the following edit work on my old conf file: mail_location = maildir:~/Maildir ? tia, gary kline -- Gary Kline kline(a)thought.org http://www.thought.org Public Service Unix http://jottings.thought.org http://transfinite.thought.org The 7.31a release of Jottings: http://jottings.thought.org/index.php

2 9

Re: [Dovecot] fts squat non-english search for 2 words
by vuser1＠test123.ru 09 Dec '09

09 Dec '09

Timo Sirainen <tss(a)iki.fi>: > On Sun, 2009-11-22 at 20:35 +0700, vuser1(a)test123.ru wrote: >> Timo, thank you for answer. Meanwhile I was trying to setup >> horde+dovecot+search. Next step was dovecot 1.2.4 + solr 1.4. It >> works! Now it can find 2 non-latin words. >> 1) I cannot search by substrings - neither "plane" nor "plane*" does >> find "planet" > > Try if attached patch helps? > Quick answer is "no" 8)). Now the story. I debugged and realized that patched plugin generates - for search "xxx yyy": q=body:"XXX YYY*" It should be: q=body:XXX* +body:YYY* (not q=body:"XXX*" +body:"YYY*" - quotation does matter) But this does not work as expected. Prefix searches (with asterisk) are case-sensitive. I googled around and found this post - http://michaelkimsal.com/blog/solr-case-sensitivty/comment-page-1/#comment-… . It is old - 2007, but it looks SOLR is still case-sensitive for *. Because of dovecot capitalizes query (and this is right, I think), the search will never find a thing. I played with Solr admin for 3 evenings and I have to say - its behaviour is strange. For example, if I send several emails with body of 2 words: "xxx yyy", "yyy xxx", "xxX yYY" etc. - different case and different word order - it does not find "xxx" in all emails. Maybe Solr 1.4 is not production-ready yet and 1.3 is better. But it is enough for me, maybe I will return to it in next year. Now I will try to apply your changeset (fts-squat: Fixed searching multi-byte characters) to dovecot 1.2.4 (debian stable). If you think 1.2.8 is better, I will follow your recommendation.

2 1