August 2011 - dovecot

[Dovecot] Bug report v2.0.13 - CentOS x86_64 - NFS - mbox
by Kostas Zorbadelos 24 Aug '11

24 Aug '11

Greetings to all. It's my first post to the list. We just completed a migration from qpopper to dovecot for our IMAP and POP3 services. We have a rather large mail environment (we are the biggest provider in Greece). So, here are the details: - Keep getting errors like these in our production environment Jul 22 00:18:21 pop01 dovecot: master: Error: service(pop3): child 4078 killed with signal 11 (core dumps disabled) Jul 22 00:19:31 pop03 dovecot: master: Error: service(pop3): child 18849 killed with signal 11 (core dumps disabled) --------------------------------------------------------------------- dovecot -n output --------------------------------------------------------------------- /opt/dovecot/sbin/dovecot -n # 2.0.13: /opt/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-92.1.22.el5 x86_64 CentOS release 5.5 (Final) auth_cache_negative_ttl = 10 mins auth_cache_size = 5 M auth_cache_ttl = 10 mins auth_verbose = yes default_client_limit = 5000 default_process_limit = 500 disable_plaintext_auth = no first_valid_uid = 200 listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = <COMPANY> ready mail_access_groups = mail otemail disk root mail_fsync = always mail_location = mbox:INDEX=/var/index/dovecot/%2.16Hn/%2.254Hn/%u mail_nfs_storage = yes mbox_lock_timeout = 2 mins mbox_min_index_size = 200 k mbox_read_locks = dotlock_try fcntl mbox_write_locks = dotlock_try fcntl passdb { args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocols = imap pop3 service auth-worker { user = dovenull } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl = no userdb { args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_max_userip_connections = 100 } protocol pop3 { mail_max_userip_connections = 100 pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_fast_size_lookups = yes pop3_lock_session = yes pop3_reuse_xuidl = yes pop3_uidl_format = %08Xu%08Xv } I enabled core dumps in one of our backend servers and here is the relevant gdb trace: [root@pop08 ~]# gdb /opt/dovecot/libexec/dovecot/pop3 <path_to_core_file>/core.9273 GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-32.el5_6.2) Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /opt/dovecot/libexec/dovecot/pop3...(no debugging symbols found)...done. Reading symbols from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0...(no debugging symbols found)...done. Loaded symbols for /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /opt/dovecot/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /opt/dovecot/lib/dovecot/libdovecot.so.0 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libpthread.so.0 Core was generated by `dovecot/pop3'. Program terminated with signal 11, Segmentation fault. #0 0x00002b52e1027e54 in istream_raw_mbox_get_start_offset () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 (gdb) bt full #0 0x00002b52e1027e54 in istream_raw_mbox_get_start_offset () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #1 0x00002b52e102b759 in ?? () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #2 0x00002b52e100a2c0 in index_mail_expunge () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #3 0x0000000000405e9c in client_update_mails () No symbol table info available. #4 0x00000000004061c1 in client_command_execute () No symbol table info available. #5 0x00000000004045b9 in client_handle_input () No symbol table info available. #6 0x00002b52e12df698 in io_loop_call_io () from /opt/dovecot/lib/dovecot/libdovecot.so.0 No symbol table info available. #7 0x00002b52e12e09d5 in io_loop_handler_run () from /opt/dovecot/lib/dovecot/libdovecot.so.0 No symbol table info available. #8 0x00002b52e12df62d in io_loop_run () from /opt/dovecot/lib/dovecot/libdovecot.so.0 No symbol table info available. #9 0x00002b52e12cdf13 in master_service_run () from /opt/dovecot/lib/dovecot/libdovecot.so.0 No symbol table info available. #10 0x0000000000403994 in main () No symbol table info available. (gdb) All traces of the crashes are identical, that is #0 0x00002b52e1027e54 in istream_raw_mbox_get_start_offset () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 #1 0x00002b52e102b759 in ?? () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 #2 0x00002b52e100a2c0 in index_mail_expunge () from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0 #3 0x0000000000405e9c in client_update_mails () #4 0x00000000004061c1 in client_command_execute () #5 0x00000000004045b9 in client_handle_input () #6 0x00002b52e12df698 in io_loop_call_io () from /opt/dovecot/lib/dovecot/libdovecot.so.0 #7 0x00002b52e12e09d5 in io_loop_handler_run () from /opt/dovecot/lib/dovecot/libdovecot.so.0 #8 0x00002b52e12df62d in io_loop_run () from /opt/dovecot/lib/dovecot/libdovecot.so.0 #9 0x00002b52e12cdf13 in master_service_run () from /opt/dovecot/lib/dovecot/libdovecot.so.0 #10 0x0000000000403994 in main () We have mboxes over NFS and we also have an ldap user backend. For now, I do not have a scenario that reproduces the problem. Any idea, or input are highly appreciated. Of course I can provide any information requested (without exposing restricted company or client data) to help trace the problem and lead to the solution. Thanks and keep up the good work! Regards, Kostas Zorbadelos

3 8

[Dovecot] Cannot delete subfolder in public folder
by Karsten Becker 24 Aug '11

24 Aug '11

Hi there, I have the problem that I'm unable to delete a subfolder (again) I created within a public folder. I've already read about configuring Thunderbird to delete immediately - which I did. But it still doesn't work. I delete -> the folder is gone -> I get a TB error message saying "The folder doesn't exist" -> after restarting TB the folders appear again. Maybe someone has a hint. My tip is that I may have a wrong understanding of the ACL mechanism... :-( Regards from Berlin/Germany Karsten ------------------------ Log: > Aug 24 00:01:37 mail01 dovecot: imap(karsten.becker(a)company.eu): Debug: acl vfile: file /etc/dovecot/global-acls//.DEFAULT not found > Aug 24 00:01:37 mail01 dovecot: imap(karsten.becker(a)company.eu): Debug: acl vfile: file /srv/vmail/user-mailboxes/company.eu/karsten.becker/mailboxes/dovecot-acl not found > Aug 24 00:01:37 mail01 dovecot: imap(karsten.becker(a)company.eu): Debug: acl vfile: file /etc/dovecot/global-acls/Folders/test01/aaa not found > Aug 24 00:01:37 mail01 dovecot: imap(karsten.becker(a)company.eu): Debug: acl vfile: reading file /srv/vmail/public_folders/test01/aaa/dovecot-acl Here's the filesystem structure in /srv/vmail: > root@mail01.compdmz.local:/srv/vmail# ls -l public_folders/test01/ > total 20 > drwxr-xr-x 5 vmail vmail 4096 2011-08-23 23:50 aaa > drwxr-xr-x 2 vmail vmail 4096 2011-08-23 21:45 cur > -rw-r--r-- 1 vmail vmail 25 2011-08-23 22:19 dovecot-acl > drwxr-xr-x 2 vmail vmail 4096 2011-08-23 21:45 new > drwxr-xr-x 2 vmail vmail 4096 2011-08-23 21:45 tmp > root@mail01.compdmz.local:/srv/vmail# Here's the content of dovecot-acl (there are another ones in subfolder aaa with the same content, inherited during creation): > authenticated lrwstipekx Here's my configuration of Dovecot: > # 2.0.13: /usr/local/dovecot-2.0.13/etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-33-server x86_64 Ubuntu 10.04.3 LTS ext4 > doveconf: Warning: Dovecot was last started using /etc/dovecot/dovecot.conf, but this config is /usr/local/dovecot-2.0.13/etc/dovecot/dovecot.conf > auth_cache_negative_ttl = 0 > auth_cache_size = 10 M > auth_mechanisms = plain login > base_dir = /usr/local/dovecot/var/run/dovecot > dict { > acl = mysql:/etc/dovecot/dovecot-dict-shared-mailboxes-mysql.conf > expire = mysql:/etc/dovecot/dovecot-dict-expire-mysql.conf > quota = mysql:/etc/dovecot/dovecot-dict-quota-mysql.conf > } > log_timestamp = "%Y-%m-%d %H:%M:%S " > login_greeting = Company Institute > mail_attachment_dir = /srv/vmail/attachments > mail_attachment_hash = %{sha256} > mail_cache_min_mail_count = 2 > mail_debug = yes > mail_location = mdbox:/srv/vmail/user-mailboxes/%d/%n > mail_plugins = acl expire quota > mailbox_idle_check_interval = 1 mins > namespace { > inbox = yes > location = > prefix = > separator = / > subscriptions = yes > type = private > } > namespace { > list = children > location = mdbox:/srv/vmail/user-mailboxes/%%d/%%n > prefix = Users/%%d/%%n/ > separator = / > subscriptions = no > type = shared > } > namespace { > list = children > location = maildir:/srv/vmail/public_folders:INDEX=/srv/vmail/user-mailboxes/%d/%n/public_folders-seen:LAYOUT=fs > prefix = Folders/ > separator = / > subscriptions = no > type = public > } > passdb { > args = /etc/dovecot/dovecot-mysql.conf > driver = sql > } > plugin { > acl = vfile:/etc/dovecot/global-acls:cache_secs=300 > acl_shared_dict = proxy::acl > expire = Trash > expire2 = Trash/* > expire3 = Junk > expire4 = Junk/* > expire_dict = proxy::expire > quota = dict:User quota::proxy::quota > quota_rule = *:storage=10485760 > quota_rule2 = Trash:storage=+1048576 > quota_warning = storage=95%% quota-warning 95 %u > quota_warning2 = storage=80%% quota-warning 80 %u > } > postmaster_address = postmaster(a)company.eu > protocols = imap pop3 > service auth { > unix_listener /var/spool/postfix/private/auth_dovecot { > group = postfix > mode = 0660 > user = postfix > } > unix_listener auth-master { > mode = 0600 > user = vmail > } > user = root > } > service dict { > unix_listener dict { > mode = 0600 > user = vmail > } > } > service imap-login { > inet_listener imaps { > port = 993 > ssl = yes > } > process_min_avail = 2 > } > service imap { > vsz_limit = 512 M > } > service pop3-login { > inet_listener pop3s { > port = 995 > ssl = yes > } > } > service quota-warning { > executable = script /usr/local/bin/quota-warning.sh > user = vmail > } > ssl = required > ssl_cert = </etc/postfix/ssl/mail01.compdmz.local.crt > ssl_key = </etc/postfix/ssl/mail01.compdmz.local.key > userdb { > args = /etc/dovecot/dovecot-mysql.conf > driver = sql > } > verbose_proctitle = yes > protocol lda { > auth_socket_path = auth-master > postmaster_address = postmaster(a)company.eu > } > protocol imap { > imap_client_workarounds = delay-newmail > imap_max_line_length = 128 k > mail_plugins = acl expire quota imap_acl imap_quota > } > protocol pop3 { > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > }

2 3

[Dovecot] acl with hierarchy separators mismatched config
by YAEGASHI Takeshi 24 Aug '11

24 Aug '11

Hi there, I'm testing dovecot 2.0.13 on Debian squeeze (deb from http://xi.rename-it.nl/debian) with the following doveconf -n. -------- # 2.0.13 (1449a2e2c1f5): /etc/dovecot/dovecot.conf # OS: Linux 2.6.26-2-openvz-amd64 i686 Debian 6.0.2 first_valid_uid = 8 mail_debug = yes mail_location = maildir:~/Maildir mail_plugins = listescape mail_log notify acl namespace { inbox = yes location = prefix = separator = / type = private } namespace { list = children location = maildir:/var/mail/public:INDEX=~/Maildir/public prefix = Public/ separator = / subscriptions = no type = public } passdb { driver = pam } plugin { acl = vfile } protocols = " imap" ssl = no userdb { args = uid=mail gid=mail home=/var/mail/private/%u driver = static } protocol imap { mail_plugins = listescape mail_log notify acl imap_acl } -------- My primary interest is acl and listescape enabled folders in the public namespace. I've chosen "/" as the hierarchy separator to support folder names with dots ("."). /var/mail/public is a maildir with the maildir++ layout where the separator is a dot ("."). So I set up intial acls and folders as follows: -------- # mkdir /var/mail/public # echo 'anyone lra' >/var/mail/public/dovecot-acl # maildirmake.dovecot /var/mail/public/.aaa # echo 'anyone lrwstipekxa' >/var/mail/public/.aaa/dovecot-acl # chown -R mail.mail /var/mail/public -------- But my attempt to create a mailbox under Public/aaa fails with "Permission denied". -------- # imtest -a yaegashi localhost S: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. C: C01 CAPABILITY S: * CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN S: C01 OK Pre-login capabilities listed, post-login capabilities have more. Please enter your password: C: A01 AUTHENTICATE PLAIN ?????????????????/ S: * CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS ACL RIGHTS=texk S: A01 OK Logged in Authenticated. Security strength factor: 0 . getacl Public * ACL "Public" "anyone" alr . OK Getacl completed. . getacl Public/aaa * ACL "Public/aaa" "anyone" akxeilprwtscd . OK Getacl completed. . create Public/aaa/bbb . NO [NOPERM] Permission denied -------- If the layout of /var/mail/public switched to "fs" where the separator is "/", mailbox creation succeeds as expected. -------- namespace { list = children location = maildir:/var/mail/public:INDEX=~/Maildir/public:LAYOUT=fs prefix = Public/ separator = / subscriptions = no type = public } -------- -------- # maildirmake.dovecot /var/mail/public/aaa # echo 'anyone lrwstipekxa' >/var/mail/public/aaa/dovecot-acl # chown -R mail.mail /var/mail/public # imtest -a yaegashi localhost .... . getacl Public * ACL "Public" "anyone" alr . OK Getacl completed. . getacl Public/aaa * ACL "Public/aaa" "anyone" akxeilprwtscd . OK Getacl completed. . create Public/aaa/bbb . OK Create completed. . getacl Public/aaa/bbb * ACL "Public/aaa/bbb" "anyone" akxeilprwtscd . OK Getacl completed. . create Public/aaa/1.2.3 . OK Create completed. . create Public/aaa/cur . NO Invalid mailbox name: Public/aaa/cur -------- Is this behavior expected? Misconfiguration or dovecot bug? I prefer the maildir++ layout with listescape as it's reserved-folder-name free (eg. cur new tmp). Regards, -- YAEGASHI Takeshi <yaegashi(a)debian.org>

2 2

[Dovecot] LDA and auth-userdb socket permissions
by a.smith＠ukgrid.net 24 Aug '11

24 Aug '11

Hi, just wanted to check this as the wiki seems to have contradictory information. With respect to running the LDA as multiple UIDs the wiki says: [QUOTE]If you're using more than one UID for users, you're going to have problems running dovecot-lda, as most MTAs won't let you run dovecot-lda as root[/QUOTE] But in the example for the config file the text reads: [QUOTE] service auth { unix_listener auth-userdb { mode = 0600 user = vmail # User running dovecot-lda #group = vmail # Or alternatively mode 0660 + dovecot-lda user in this group } } [/QUOTE] So it says you can stick the LDA user just in the (vmail or whatever) group and that is enough. So you aren't restricted to a single UID for access anymore... I tested this and the later did not work, that is if I put my LDA user in the group for the auth-userdb socket with permissions 0660 I got an error back from dovecot saying that the owner was incorrect. So, as it stands I guess the bit about setting group should be removed from the wiki? Secondly, why doesn't this currently work? Why is the owner all important? thanks Andy.

3 13

[Dovecot] dovecot and tcpwrappers
by Kees Lemmens 23 Aug '11

23 Aug '11

Hi, I used dovecot 1.x for quite a while and it worked fine. However, I used it through inetd and used hosts.allow/deny to restrict access to only certain groups of systems. Since yesterday I have dovecot 2.0.13. But in version 2.0.13 it seems that starting using inetd doesn't work anymore : I only get a strange error message if I try to connect using telnet : telnet localhost imap doveconf: Fatal: execvp(imap-login) failed: No such file or directory I tried a lot of things but couldn't get it going over inetd so I decided to start it directly using an init script, also because you say in the wiki this is (now) the preferred method. That indeed works fine, but I realised today that dovecot is now completely ignoring all my hosts.allow/deny settings and so is worldwide accessible ! Is dovecot indeed ignoring hosts.allow/deny ? And if so, how can I have the same amount of protection as I previously had with dovecot 1.x over inetd ? BTW: I tried to post on the mailinglist but it seems I can't access them it the moment, altough I did subscribe and login. Is there anything wrong ? Thanks for your reply ! -- With best regards, Kees Lemmens. ----------------------------------------------------------------------- Delft Institute of Applied Mathematics (DIAM), Faculty Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, Delft, The Netherlands,

3 5

[Dovecot] zlib plugin doesn't read concatenated .gz mailbox
by Dmitry Nezhevenko 23 Aug '11

23 Aug '11

Hi, I've observed that dovecot doesn't displays all mails from compressed via gzip mailbox (mbox). At the same time "mutt -f mailbox.gz" displays it correctly with all messages. I'm using archivemail to archive old mails. Internally it creates new "gzip" archive with "expired" messages and then concatenates it with archive. The easiest way to reproduce it without archivemail is to create two mailboxes, gzip them independently and then concatenate to one gzip archive. mutt, gunzip and other tools understands such archives. Also I've shared such mbox: http://dion.org.ua/uploads/2011/07/dovecot_test_gzip.gz It should contains ~20 messages. dovecot will "see" only two first. PS. % sudo dovecot --version 2.0.13 -- WBR, Dmitry

2 1

[Dovecot] dovecot's documentation dearth
by Postmaster 23 Aug '11

23 Aug '11

Hello. The first thing for me to say is thanks! I've been using dovecot since test 61 and it has basically performed flawlessly for me from day number 1. Dovecot makes me look like a genius sysadmin, especially compared to my colleagues struggling with exchange. So thanks, gracias, obrigado!! I'm working on a configuration for 2.0 and I'm finding the documentation somewhat difficult. I think it would be very helpful to me to have encountered a single page that detailed all available configuration sections. I don't know how to find out what the sections are, when to use them or what specifically they control. So far I've encountered... passdb plugin service protocol userdb local_name (and I think there is another one of these for ips) The service section itself would benefit from a single page detailing all of the possible types of service sections available. The next problem I've had is discovering that several parts of dovecot have no documentation at all even though they are standalone executables run by root. config, log, and ssl_params all run as root but there is also anvil and they look to me like they could support listening on a port if inet_interfaces is defined. You can say, "Relax fella, trust me. These programs are all part of dovecot and are run only if they're needed." Then I'll say, "Yes but I'm the administrator. I need to know how the parts fit together to know if the system's broken." Then you say, "Well that makes sense, but the things you're talking about aren't really configured. They are mostly internal to dovecot, they just happen to be broken out into external programs. You may as well be asking for documentation on a specific function in a library. If that's what you want, you can read through the source code." Well I guess that would be one solution. The bottom line is that it gives me an uncomfortableness to not be able to control or explain the operation of the software I'm supposedly administering. Take the program named log (which should be named dovecot-log or something less generic), it is launched even though I've specified syslog in the configuration. Logging is not interrupted when the process is killed. So, why is it running? What is it doing? Why does it need root? How do I control it? I think these are all good questions for an administrator to ask. That's my feedback for what it's worth, but mostly, thanks again for dovecot!

2 1

[Dovecot] Default and per-User sieve script
by Patrick Westenberg 23 Aug '11

23 Aug '11

Hi guys, is there any way to configure Dovecot to process the default sieve script and, after that, a user specific script? I have a default script to sort spam into a spam folder but if a user specific script is present, the default script is ignored. sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_path = /usr/local/etc/dovecot/sieve/default.sieve Regards Patrick

3 2

[Dovecot] dovecot deadlock with procmail
by kunal verma 23 Aug '11

23 Aug '11

I m using dovecot 1.0.7. I m having problems in mail delivery to my mail server locally. When a user sends a mail few mails are in mail Q for longer period of time. The local delivery agent(*procmail*) is trying to deliver the mail at regular interval but it is unable to deliver. But as soon as I *restart* *dovecot* the mails in the Q are delivered immediately. I suspect it is because of locking of *mbox* file of users by dovecot. How to overcome this problem as mails are some mails are taking hours to be delivered?? please let me know the solution. Kunal Verma

2 1

[Dovecot] Large Mailbox Slow
by Matt 23 Aug '11

23 Aug '11

Doubt if there is any answer to this but will ask anyway. Have a few pop3 accounts with thousands of messages. Its slow when checking email naturally. Are there any tweaks to speed it up? I imagine there is an exchange of the message and header list which is the slow down. Too bad the list could not be compressed with gzip or something first. I think http has an option similar to that. Just asking.

3 3