July 2011 - dovecot - dovecot.org

[Dovecot] compressed mboxes very slow
by kjonca＠o2.pl 29 Jan '12

29 Jan '12

I have some archive mails in gzipped mboxes. I could use them with dovecot 1.x without problems. But recently I have installed dovecot 2.0.12, and they are slow. very slow. Creating index files takes about 10 minutes for ~20M file with 560 messages for bzipped mbox, for gzipped is little better but still unusable :( Stracing dovecot process shows that every ~ 20 messages it rereads complete mbox file. Am I doing something wrong? KJ -- http://modnebzdury.wordpress.com/2009/10/01/niewiarygodny-list-prof-majewsk…

4 13

[Dovecot] Managing public folder ACL files
by Keith Edmunds 14 Nov '11

14 Nov '11

We have a customer with a large public folder hierarchy. They occasionally make requests to have the public folder ACLs changed; for example: "please give user X access to all public folders" (that's nearly 1700 folders). Worse: "please give user Y access to all sales folders" (there are 1450 sales folders). The problem is that there are (naturally) spaces in the folder names, which makes command line manipulation challenging. We've ended up with some astonishingly hacky Python scripts that enter each folder starting with (for example) ".sales" and replacing the dovecot-acl file to try to fulfil the above requests. One day our script are going to get it wrong, or requests will become more complex ("give X access all sales/CustA folders, Y access to all sales/CustB folders, and Z access to all sales folders). There must be a Better Way. How do others manage divergent ACLs within large public folder hierarchies? Thanks, Keith

5 4

[Dovecot] LDAP expired password
by roberto palmarin 10 Nov '11

10 Nov '11

hi all, I'm using postfix, LDAP, dovecot and horde for webmail. user and password information is stored in LDAP. I'm attempting to get password aging working properly and am not having much luck. even if password has expired user can login, can i tell dovecot to control the LDAP field shadowexpired? or is there some other way to check properly that the password is expired before allowing the user log in? thank's /roby

4 5

[Dovecot] dovecot evaluation on a 30 gb mailbox
by Rajesh M 04 Nov '11

04 Nov '11

hi all i use qmail toaster. i recently changed over to dovecot. posting below my evaluation on the same my machine dual core dual xeon 5140 processor, with 4 gb ram, centos linux with qmailtoaster i had setup a qmail tap account which backups up data for a client of mine mailbox size over 30 gb setup dovecot with squirrelmail and logged in it had over one hundred and thirty nine thousand emails 139019 emails in the 30 gb mailbox Reports 1) first login : took 7 minutes to parse 139019 mails but never timed out. this was simply impossible with courier 2) the next login it took 5 seconds by which time the mail numbers increased to 139045 3) next i changed squirrelmail display preference to list 500 mails per page and then clicked on inbox ... expecting dovecot to take atleast around 1-2 mins to show the mails ... the mails were displayed in just 8 seconds .. awesome performance qmailtoaster is simply waiting for a dovecot integration i am evaluating the stability of dovecot now in the next 2-3 weeks i post stability report rajesh

19 43

[Dovecot] sieve vacation reply
by Wojciech Giel 04 Oct '11

04 Oct '11

Hi, I'm configuring postfix with dovecot and roundcube. I'm using flat files as passwdb userdb. Everything works fine except sieve vacation reply. when i send mail from ex. user dennis to tytus I get "discarding vacation response for message implicitly delivered to". What Am i doing wrong? thanks sieve: using sieve path for user's script: /home/tytus/.dovecot.sieve 2011-05-17 16:56:45 deliver(tytus): Info: sieve: opening script /home/tytus/.dovecot.sieve 2011-05-17 16:56:45 deliver(tytus): Info: sieve: executing compiled script /home/tytus/.dovecot.sieve 2011-05-17 16:56:45 deliver(tytus): Info: sieve: msgid=<20110517155641.GA3452(a)badger.example.com>: discarding vacation response for message implicitly delivered to <tytus@badger> /etc/postfix/main.cf ... mailbox_command = /usr/lib/dovecot/deliver -n -m "${EXTENSION}" ... # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.1 log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps managesieve listen(default): * listen(imap): * listen(managesieve): 127.0.0.1:2000 ssl_ca_file: /etc/postfix/ssl/cacert.pem ssl_cert_file: /etc/postfix/ssl/servercrt.pem ssl_key_file: /etc/postfix/ssl/serverkey.pem ssl_cipher_list: ALL:!LOW:!SSLv2 login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login mail_location: maildir:~/Maildir:INDEX=~/Maildir/indexes mail_debug: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): quota imap_quota autocreate mail_plugins(imap): quota imap_quota autocreate mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve imap_client_workarounds(default): tb-extra-mailbox-sep netscape-eoh delay-newmail outlook-idle imap_client_workarounds(imap): tb-extra-mailbox-sep netscape-eoh delay-newmail outlook-idle imap_client_workarounds(managesieve): lda: postmaster_address: postmaster mail_plugins: sieve quota autocreate quota_full_tempfail: yes deliver_log_format: msgid=%m: %$ rejection_reason: Your message to <%t> was automatically rejected:%n%r log_path: /var/log/sieve.log auth default: mechanisms: plain login cram-md5 verbose: yes debug: yes debug_passwords: yes passdb: driver: pam passdb: driver: passwd-file args: scheme=cram-md5 /etc/dovecot/passwd userdb: driver: passwd-file args: /etc/dovecot/passwd userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/dovecot-auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 plugin: sieve: ~/.dovecot.sieve sieve_dir: ~/sieve autocreate: INBOX autocreate2: Sent autocreate3: Trash autocreate4: Drafts autocreate5: Junk autosubscribe: INBOX autosubscribe2: Sent autosubscribe3: Trash autosubscribe4: Drafts autosubscribe5: Junk quota: maildir:User quota quota_rule: Trash:storage=10M quota_warning: storage=85%% /usr/bin/dovecot-quota-warning.sh 85 quota_warning2: storage=90%% /usr/bin/dovecot-quota-warning.sh 90 quota_warning3: storage=95%% /usr/bin/dovecot-quota-warning.sh 95

5 7

[Dovecot] Dovecot 1.2, problem with SQL quota and mail aliases
by Jacek Osiecki 22 Sep '11

22 Sep '11

Hello, I have a small server running dovecot 1.2 (still not balsy enough to switch to 2.0 - virtual sql accounts, managesieve etc.). There are no local users - only ones defined in the SQL database, and everything else (including quota) is defined in SQL db too. Dovecot is used as IMAP daemon, and dovecot-lda is responsible for delivery (in order to make use of sieve and managesieve). I know that my configuration might be a bit ancient, but this is not referring to any bug (at least I think so) and rather to errors I've made in configuration. Several users have reported, that they are running out of quota even though their mailboxes are empty. I thought these were some single cases, but recently I discovered that this situation occurs when: - user john(a)domain.com has a mail alias, eg. john.smith(a)domain.com - he logs in to the imap account (using MUA or webmail) using his alias (john.smith(a)domain.com). In effect, all incoming emails for john(a)domain.com are increasing the counters (both "bytes" and "messages"), but since user logs in using john.smith(a)domain.com - anything he deletes from the mailbox decreases the records for john.smith(a)domain.com. Finally, John has two records: +---------------------------+-----------+----------+ | username | bytes | messages | +---------------------------+-----------+----------+ | john.smith(a)domain.com | -5488133 | -13 | | john(a)domain.com | 341411029 | 1111 | +---------------------------+-----------+----------+ The record for john(a)domain.com finaly reaches the quota limit - and even removing all emails does not help. My /etc/dovecot/dovecot-dict-quota.conf looks like this: map { pattern = priv/quota/storage table = quota username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota username_field = username value_field = messages } And my question is: what is used as "username"? Always the username given wher user logs in? Is there a way to provide some dictionary to translate all aliases to the real username? Best regards, -- Jacek Osiecki joshua(a)ceti.pl GG:3828944 I don't want something I need. I want something I want.

3 4

[Dovecot] [PATCH] Indexing mail attachments with Dovecot + Solr
by Antonio Perez-Aranda 31 Aug '11

31 Aug '11

Indexing mail attachments with Dovecot + Solr. This patch has been tested with these versions: * dovecot 2.0.9 * apache-solr 1.4.1 This is a patch for the fts-solr plugin (that indexes mail messages for Dovecot with Solr). In main stream, the plugin does not index attachments; With this patch, you can index mails and their attachments (pdf, docs, openoffice docs...) . You can get others goodies with this patch and the Solr Config provided, like Synonyms and Stemming (Spanish by default). Attachment indexing is provided by Solr Cell and Tika (ExtractingRequestHandler) * http://wiki.apache.org/solr/ExtractingRequestHandler Synonyms and Stemming are provided by SnowballPorterFilterFactory from Solr Language Analysis: * http://wiki.apache.org/solr/LanguageAnalysis We have tested Solr with Tomcat and Jetty. Tomcat is better to handle UTF-8 and bigger POSTS. Attachments file format supported * http://tika.apache.org/0.9/formats.html At present, attachments in attachments (like, for example, attachments in fordwarded "eml" attachments) are not indexed. Also, keep in mind that there are many types of files, and many variants of the same file type. Per Example, some pdf files are "not readable" by solr pdf reader. Config: There are two new options added to fts_solr property: * index-attachments Enable attachments indexing. * manual-update Avoid index on user search. You can trigger indexing using doveadm search or doveadm index commands. There is a new property for the section plugin to filter the mimetypes that you want to index. * fts_solr_mimetype files with this mimetype will be sent to solr. After integrating solr directory in your solr config, and building Dovecot with fts-solr support and with fts-solr-attachments-r885.patch applied, you can update your dovecot config by adding to your dovecot.conf: ... mail_pluings = $mail_plugins fts fts_solr plugin { fts = solr fts_solr = url=http://solrhost:8983/solr/ break-imap-search index-attachments fts_solr_mimetype = application/x-pdf application/vnd.openxmlformats-officedocument.wordprocessingml.document } ... -- Antonio Pérez-Aranda Alcaide aperezaranda(a)yaco.es Yaco Sistemas S.L. http://www.yaco.es/ C/ Rioja 5, 41001 Sevilla Teléfono +34 954 50 00 57 Fax +34 954 50 09 29

3 5

[Dovecot] Bug report v2.0.13 - CentOS x86_64 - NFS - mbox
by Kostas Zorbadelos 24 Aug '11

24 Aug '11

Greetings to all. It's my first post to the list. We just completed a migration from qpopper to dovecot for our IMAP and POP3 services. We have a rather large mail environment (we are the biggest provider in Greece). So, here are the details: - Keep getting errors like these in our production environment Jul 22 00:18:21 pop01 dovecot: master: Error: service(pop3): child 4078 killed with signal 11 (core dumps disabled) Jul 22 00:19:31 pop03 dovecot: master: Error: service(pop3): child 18849 killed with signal 11 (core dumps disabled) --------------------------------------------------------------------- dovecot -n output --------------------------------------------------------------------- /opt/dovecot/sbin/dovecot -n # 2.0.13: /opt/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-92.1.22.el5 x86_64 CentOS release 5.5 (Final) auth_cache_negative_ttl = 10 mins auth_cache_size = 5 M auth_cache_ttl = 10 mins auth_verbose = yes default_client_limit = 5000 default_process_limit = 500 disable_plaintext_auth = no first_valid_uid = 200 listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = <COMPANY> ready mail_access_groups = mail otemail disk root mail_fsync = always mail_location = mbox:INDEX=/var/index/dovecot/%2.16Hn/%2.254Hn/%u mail_nfs_storage = yes mbox_lock_timeout = 2 mins mbox_min_index_size = 200 k mbox_read_locks = dotlock_try fcntl mbox_write_locks = dotlock_try fcntl passdb { args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = imap pop3 service auth-worker { user = dovenull } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl = no userdb { args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_max_userip_connections = 100 } protocol pop3 { mail_max_userip_connections = 100 pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_fast_size_lookups = yes pop3_lock_session = yes pop3_reuse_xuidl = yes pop3_uidl_format = %08Xu%08Xv } I enabled core dumps in one of our backend servers and here is the relevant gdb trace: [root@pop08 ~]# gdb /opt/dovecot/libexec/dovecot/pop3 <path_to_core_file>/core.9273 GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-32.el5_6.2) Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /opt/dovecot/libexec/dovecot/pop3...(no debugging symbols found)...done. Reading symbols from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0...(no debugging symbols found)...done. Loaded symbols for /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /opt/dovecot/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /opt/dovecot/lib/dovecot/libdovecot.so.0 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libpthread.so.0 Core was generated by `dovecot/pop3'. Program terminated with signal 11, Segmentation fault. #0 0x00002b52e1027e54 in istream_raw_mbox_get_start_offset () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 (gdb) bt full #0 0x00002b52e1027e54 in istream_raw_mbox_get_start_offset () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #1 0x00002b52e102b759 in ?? () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #2 0x00002b52e100a2c0 in index_mail_expunge () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #3 0x0000000000405e9c in client_update_mails () No symbol table info available. #4 0x00000000004061c1 in client_command_execute () No symbol table info available. #5 0x00000000004045b9 in client_handle_input () No symbol table info available. #6 0x00002b52e12df698 in io_loop_call_io () from /opt/dovecot/lib/dovecot/libdovecot.so.0 No symbol table info available. #7 0x00002b52e12e09d5 in io_loop_handler_run () from /opt/dovecot/lib/dovecot/libdovecot.so.0 No symbol table info available. #8 0x00002b52e12df62d in io_loop_run () from /opt/dovecot/lib/dovecot/libdovecot.so.0 No symbol table info available. #9 0x00002b52e12cdf13 in master_service_run () from /opt/dovecot/lib/dovecot/libdovecot.so.0 No symbol table info available. #10 0x0000000000403994 in main () No symbol table info available. (gdb) All traces of the crashes are identical, that is #0 0x00002b52e1027e54 in istream_raw_mbox_get_start_offset () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 #1 0x00002b52e102b759 in ?? () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 #2 0x00002b52e100a2c0 in index_mail_expunge () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 #3 0x0000000000405e9c in client_update_mails () #4 0x00000000004061c1 in client_command_execute () #5 0x00000000004045b9 in client_handle_input () #6 0x00002b52e12df698 in io_loop_call_io () from /opt/dovecot/lib/dovecot/libdovecot.so.0 #7 0x00002b52e12e09d5 in io_loop_handler_run () from /opt/dovecot/lib/dovecot/libdovecot.so.0 #8 0x00002b52e12df62d in io_loop_run () from /opt/dovecot/lib/dovecot/libdovecot.so.0 #9 0x00002b52e12cdf13 in master_service_run () from /opt/dovecot/lib/dovecot/libdovecot.so.0 #10 0x0000000000403994 in main () We have mboxes over NFS and we also have an ldap user backend. For now, I do not have a scenario that reproduces the problem. Any idea, or input are highly appreciated. Of course I can provide any information requested (without exposing restricted company or client data) to help trace the problem and lead to the solution. Thanks and keep up the good work! Regards, Kostas Zorbadelos

3 8

[Dovecot] dovecot and tcpwrappers
by Kees Lemmens 23 Aug '11

23 Aug '11

Hi, I used dovecot 1.x for quite a while and it worked fine. However, I used it through inetd and used hosts.allow/deny to restrict access to only certain groups of systems. Since yesterday I have dovecot 2.0.13. But in version 2.0.13 it seems that starting using inetd doesn't work anymore : I only get a strange error message if I try to connect using telnet : telnet localhost imap doveconf: Fatal: execvp(imap-login) failed: No such file or directory I tried a lot of things but couldn't get it going over inetd so I decided to start it directly using an init script, also because you say in the wiki this is (now) the preferred method. That indeed works fine, but I realised today that dovecot is now completely ignoring all my hosts.allow/deny settings and so is worldwide accessible ! Is dovecot indeed ignoring hosts.allow/deny ? And if so, how can I have the same amount of protection as I previously had with dovecot 1.x over inetd ? BTW: I tried to post on the mailinglist but it seems I can't access them it the moment, altough I did subscribe and login. Is there anything wrong ? Thanks for your reply ! -- With best regards, Kees Lemmens. ----------------------------------------------------------------------- Delft Institute of Applied Mathematics (DIAM), Faculty Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, Delft, The Netherlands,

3 5

[Dovecot] zlib plugin doesn't read concatenated .gz mailbox
by Dmitry Nezhevenko 23 Aug '11

23 Aug '11

Hi, I've observed that dovecot doesn't displays all mails from compressed via gzip mailbox (mbox). At the same time "mutt -f mailbox.gz" displays it correctly with all messages. I'm using archivemail to archive old mails. Internally it creates new "gzip" archive with "expired" messages and then concatenates it with archive. The easiest way to reproduce it without archivemail is to create two mailboxes, gzip them independently and then concatenate to one gzip archive. mutt, gunzip and other tools understands such archives. Also I've shared such mbox: http://dion.org.ua/uploads/2011/07/dovecot_test_gzip.gz It should contains ~20 messages. dovecot will "see" only two first. PS. % sudo dovecot --version 2.0.13 -- WBR, Dmitry

2 1