September 2011 - dovecot

[Dovecot] Quotas not recalculating
by Kyle Brantley 07 Sep '11

07 Sep '11

Hello all -- I have an interesting issue, where the quotas are refusing the recalculate for anyone on my server. # doveadm quota get -u kyle(a)averageurl.com Quota name Type Value Limit % storage=540672 STORAGE 0 - 0 storage=540672 MESSAGE 0 - 0 # doveadm quota recalc -u kyle(a)averageurl.com # doveadm quota get -u kyle(a)averageurl.com Quota name Type Value Limit % storage=540672 STORAGE 0 - 0 storage=540672 MESSAGE 0 - 0 # du -sh /var/mail/vhosts/averageurl.com/kyle 512M /var/mail/vhosts/averageurl.com/kyle The IMAP client (Thunderbird) state that the user has no quotas in place at all. Now, if I copy the maildirsize file from another user to this user account, the quotas display in the IMAP client. At this point, however, trying to recalc the quota does nothing. Any help with this would be appreciated, config below... --Kyle # doveconf -n # 2.0.beta6 (3156315704ef): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-71.29.1.el6.x86_64 x86_64 CentOS Linux release 6.0 (Final) ext4 auth_gssapi_hostname = imap.mail.averageurl.com auth_krb5_keytab = /etc/dovecot/krb5.keytab auth_mechanisms = plain login gssapi digest-md5 cram-md5 first_valid_uid = 502 last_valid_uid = 502 mail_debug = yes mail_location = maildir:/var/mail/vhosts/%d/%n mail_plugins = $mail_plugins quota mbox_write_locks = fcntl passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = maildir:Quota sieve = /var/mail/vhosts/%d/%n/.sieve sieve_dir = /var/mail/vhosts/%d/%n/sieve/ } protocols = imap lmtp sieve service auth { unix_listener auth-userdb { group = mailstore mode = 0660 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service imap { process_limit = 128 } service lmtp { unix_listener lmtp { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 } ssl_ca = </etc/pki/tls/certs/AverageURL.crt ssl_cert = </etc/pki/dovecot/certs/imap.mail.averageurl.com.crt ssl_key = </etc/pki/dovecot/private/imap.mail.averageurl.com.pem.insecure userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = $mail_plugins sieve } protocol imap { mail_max_userip_connections = 20 mail_plugins = $mail_plugins imap_quota } protocol lmtp { mail_plugins = $mail_plugins sieve } protocol sieve { mail_max_userip_connections = 2 managesieve_max_line_length = 65536 } # cat /etc/dovecot/dovecot-sql.conf.ext driver = mysql connect = host=/var/lib/mysql/mysql.sock user=<snip> password=<snip> dbname=<snip> user_query = SELECT 502 AS uid, 502 AS gid, "/var/mail/vhosts/%d/%n" AS home, concat("maildir:storage=", floor(m.quota/1024)) AS quota FROM vmailboxes m, vhosts h WHERE `m`.`username` = '%n' AND `m`.`domain` = `h`.`UUID` AND `h`.`vhost` = '%d' default_pass_scheme = PLAIN password_query = SELECT aes_decrypt(`m`.`password`, <snip>) AS password FROM vmailboxes m, vhosts h WHERE `m`.`username` = '%n' AND `m`.`domain` = `h`.`UUID` AND `h`.`vhost` = '%d'

3 6

[Dovecot] dumb Q: how to search for email hack attempts ?
by Voytek 07 Sep '11

07 Sep '11

one of the users thinks someone hacked his email, I don't have time this morning to analyze mail logs in detail, but does some one has some tips to simply searching mail logs for multiple log in attempts, etc, I'd appreciate some grepping for failed logins show give some clues ?

2 1

[Dovecot] error in logs, but system appears to work
by Chris Hoogendyk 06 Sep '11

06 Sep '11

I'm bringing up a system with dovecot that has been running for years with uw-imap. In making the switch, I put a tail on the log file, made the switch, and sent a test mail and received it. I was able to receive mail with no apparent difficulties, but the log file was disconcerting. A number of users connected during the few minutes I had it running, and for each of them there was an error, e.g., "chown ~user/mail/.imap/INBOX failed: Operation not permitted" and "mkdir ~user/mail/.imap/INBOX failed: not owner". But those directories and files appeared to have been made. The first of those errors finishes with "group based on /var/mail/user". That makes no sense. I'm hoping someone can give me an idea what might cause this. Here are the log entries in debug mode and the directories for one such user: Sep 2 17:31:33 eclogite dovecot: [ID 583609 local2.info] auth(default): new auth connection: pid=7248 Sep 2 17:31:43 eclogite dovecot: [ID 583609 local2.info] auth(default): client in: AUTH 1 PLAIN service=pop3 secured lip=128.119.45.20 rip=128.119.45.9 lport=995 rport=53865 resp=AGVicmFkbGV5AFJvd2luZzEzKg== Sep 2 17:31:43 eclogite dovecot: [ID 583609 local2.info] auth(default): shadow(ebradley,128.119.45.9): lookup Sep 2 17:31:43 eclogite dovecot: [ID 583609 local2.info] auth(default): client out: OK 1 user=ebradley Sep 2 17:31:43 eclogite dovecot: [ID 583609 local2.info] auth(default): master in: REQUEST 37 7248 1 Sep 2 17:31:43 eclogite dovecot: [ID 583609 local2.info] auth(default): passwd(ebradley,128.119.45.9): lookup Sep 2 17:31:43 eclogite dovecot: [ID 583609 local2.info] auth(default): master out: USER 37 ebradley system_groups_user=ebradley uid=6633 gid=100 home=/u1/home/grad/ebradley Sep 2 17:31:43 eclogite dovecot: [ID 583609 local2.info] POP3(ebradley): Effective uid=6633, gid=100, home=/u1/home/grad/ebradley Sep 2 17:31:43 eclogite dovecot: [ID 583609 local2.info] POP3(ebradley): mbox: data=~/mail:INBOX=/var/mail/ebradley Sep 2 17:31:43 eclogite dovecot: [ID 583609 local2.info] POP3(ebradley): fs: root=/u1/home/grad/ebradley/mail, index=, control=, inbox=/var/mail/ebradley Sep 2 17:31:43 eclogite dovecot: [ID 583609 local2.info] POP3(ebradley): Namespace : Using permissions from /u1/home/grad/ebradley/mail: mode=0700 gid=-1 Sep 2 17:31:43 eclogite dovecot: [ID 583609 local2.error] POP3(ebradley): chown(/u1/home/grad/ebradley/mail/.imap/INBOX, -1, 6(mail)) failed: Operation not permitted (egid=100(geolgrad), group based on /var/mail/ebradley) Sep 2 17:31:43 eclogite dovecot: [ID 583609 local2.error] POP3(ebradley): mkdir(/u1/home/grad/ebradley/mail/.imap/INBOX) failed: Not owner Sep 2 17:31:43 eclogite dovecot: [ID 583609 local2.info] pop3-login: Login: user=<ebradley>, method=PLAIN, rip=128.119.45.9, lip=128.119.45.20, TLS When I looked at their home directory, I found: drwx------ 4 ebradley geolgrad 512 Jul 28 2009 /u1/home/grad/ebradley drwx------ 4 ebradley geolgrad 512 Sep 2 17:31 /u1/home/grad/ebradley/mail drwx------ 3 ebradley geolgrad 512 Sep 2 17:31 /u1/home/grad/ebradley/mail/.imap drwxrwx--- 2 ebradley geolgrad 512 Sep 2 17:31 /u1/home/grad/ebradley/mail/.imap/INBOX -rw-rw---- 1 ebradley mail 9484323 Sep 6 15:16 /var/mail/ebradley My dovecot configuration is at the end. The other pieces are sendmail, milter-greylist, mimedefang, etc. -- --------------- Chris Hoogendyk - O__ ---- Systems Administrator c/ /'_ --- Biology& Geology Departments (*) \(*) -- 140 Morrill Science Center ~~~~~~~~~~ - University of Massachusetts, Amherst <hoogendyk(a)bio.umass.edu> --------------- Erdös 4 # dovecot -a # 1.2.17: /usr/local/etc/dovecot.conf Warning: fd limit 256 is lower than what Dovecot can use under full load (more than 768). Either grow the limit or change login_max_processes_count and max_mail_processes settings # OS: SunOS 5.9 sun4u base_dir: /var/run/dovecot/ log_path: info_log_path: log_timestamp: %b %d %H:%M:%S syslog_facility: local2 protocols: imaps pop3s listen: * ssl_listen: * ssl: required ssl_ca_file: /etc/mail/certs/ca-bundle.crt ssl_cert_file: /etc/mail/certs/sendmail.pem ssl_key_file: /etc/mail/certs/sendmail.pem ssl_key_password: ssl_parameters_regenerate: 168 ssl_cipher_list: ssl_cert_username_field: commonName ssl_verify_client_cert: no disable_plaintext_auth: yes verbose_ssl: no shutdown_clients: yes nfs_check: yes version_ignore: no login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_user: dovecot login_greeting: ready. login_log_format_elements: user=<%u> method=%m rip=%r lip=%l %c login_log_format: %$: %s login_process_per_connection: yes login_chroot: yes login_trusted_networks: login_process_size: 64 login_processes_count: 3 login_max_processes_count: 128 login_max_connections: 256 valid_chroot_dirs: mail_chroot: max_mail_processes: 512 mail_max_userip_connections(default): 10 mail_max_userip_connections(imap): 10 mail_max_userip_connections(pop3): 3 verbose_proctitle: no first_valid_uid: 1000 last_valid_uid: 0 first_valid_gid: 98 last_valid_gid: 0 mail_access_groups: mail_privileged_group: mail mail_uid: mail_gid: mail_location: mbox:~/mail:INBOX=/var/mail/%u mail_cache_fields: mail_never_cache_fields: imap.envelope mail_cache_min_mail_count: 0 mailbox_idle_check_interval: 30 mail_debug: yes mail_full_filesystem_access: no mail_max_keyword_length: 50 mail_save_crlf: no mmap_disable: no dotlock_use_excl: yes fsync_disable: no mail_nfs_storage: no mail_nfs_index: no mailbox_list_index_disable: yes lock_method: fcntl maildir_stat_dirs: no maildir_copy_with_hardlinks: yes maildir_copy_preserve_filename: no maildir_very_dirty_syncs: no mbox_read_locks: fcntl mbox_write_locks: dotlock fcntl mbox_lock_timeout: 300 mbox_dotlock_change_timeout: 120 mbox_min_index_size: 0 mbox_dirty_syncs: yes mbox_very_dirty_syncs: no mbox_lazy_writes: yes dbox_rotate_size: 2048 dbox_rotate_min_size: 16 dbox_rotate_days: 1 mail_drop_priv_before_exec: no mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_process_size: 256 mail_plugins: mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 mail_log_prefix: %Us(%u): mail_log_max_lines_per_sec: 10 imap_max_line_length: 65536 imap_capability: imap_client_workarounds: imap_logout_format: bytes=%i/%o imap_id_send: imap_id_log: imap_idle_notify_interval: 120 pop3_no_flag_updates: no pop3_enable_last: no pop3_reuse_xuidl(default): no pop3_reuse_xuidl(imap): no pop3_reuse_xuidl(pop3): yes pop3_save_uidl: no pop3_lock_session: no pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): %08Xv%08Xu pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh pop3_logout_format: top=%t/%p, retr=%r/%b, del=%d/%m, size=%s dict_db_config: dict_process_count: 1 auth default: mechanisms: plain realms: default_realm: cache_size: 0 cache_ttl: 3600 cache_negative_ttl: 3600 executable: /usr/local/libexec/dovecot/dovecot-auth user: root chroot: username_chars: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_ username_translation: username_format: master_user_separator: anonymous_username: anonymous krb5_keytab: gssapi_hostname: winbind_helper_path: /usr/bin/ntlm_auth failure_delay: 2 verbose: yes debug: yes debug_passwords: yes ssl_require_client_cert: no ssl_username_from_cert: no use_winbind: no count: 1 worker_max_count: 30 process_size: 256 passdb: driver: shadow args: deny: no pass: no master: no userdb: driver: passwd args: #

2 3

[Dovecot] Extdata plugin, MySQL
by Steve Fatula 06 Sep '11

06 Sep '11

Does someone have an example of configuring the Pigeonhole Sieve Extdata plugin with MySQL dict? Particularly, the map and what key might be used in this? The doc in the wiki isn;t clear at all to me when one wants to use MySQL store. The file in the doc folder of the mercurial repository is basically what I want to do, in that RFC, it's bascially looking for a discard_spam setting, I presume by user whose Sieve script is running. So, what would the sql mapping file look like? What lookup key is used? I can't really find any map documentation for the syntax, can a special key be used, perhaps, I need to append a value to the user if that is the key? Can this data be accessed via the dictionary proxy to make access more efficient? While I don't want to discard spam based on the setting in the database, I DO want to not move to the spam folder based on a setting stored in an existing MySQL database. So, very similar idea. I want to use an existing database, but, the key is username + a certain constant value for another field. Am hoping this can be mapped as well. Steve

1 0

[Dovecot] Mail files?
by Knute Johnson 05 Sep '11

05 Sep '11

I have a Ubuntu server running dovecot 1.2.9 with mail in mbox. There are mail files in /var/mail/user and there are mail files in /home/user/mail that correspond to the IMAP folders. In the config file details below it shows mail_location: mbox:~/mail:INBOX=/var/mail/%u, does that put the INBOX in /var/mail? Is INBOX in /var/mail/user to allow local delivery or something? If I was using POP3 instead would all the mail be in the INBOX? Can I backup these files by just copying and if a file is destroyed, copy them back? # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-33-generic i686 Ubuntu 10.04.3 LTS log_timestamp: %Y-%m-%d %H:%M:%S protocols: imaps ssl_cert_file: /etc/ssl/certs/ssl-cert-snakeoil.pem ssl_key_file: /etc/ssl/private/ssl-cert-snakeoil.key login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login mail_max_userip_connections: 16 mail_privileged_group: mail mail_location: mbox:~/mail:INBOX=/var/mail/%u mbox_write_locks: fcntl dotlock imap_capability: IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS auth default: passdb: driver: pam userdb: driver: passwd Thanks, -- Knute Johnson knute2011(a)knutejohnson.com

3 4

[Dovecot] 2.1.alpha1: SIGSEGV on EXPUNGE
by Lutz Preßler 05 Sep '11

05 Sep '11

Hi, I upgraded my test installation from Debian lenny with latest 2.0.14 auto builds to squeeze with v2.1.alpha1 (amd64) auto builds. (From /var/log/mail.log: Sep 5 12:34:45 pressler dovecot: master: Dovecot v2.1.alpha1 starting up (core dumps disabled) No hg hash tag in log output anymore?) Now the imap processes segfault on every EXPUNGE command: Excerpt from telnet session: 0 store 1585:1585 flags \Deleted * 1585 FETCH (FLAGS (\Deleted)) 0 OK Store completed. 0 expunge Log output: Sep 5 15:40:59 pressler dovecot: imap(2580, lpmail): delete: box=INBOX, uid=34722, msgid=<E1R0ZJW-00FrF7-0U(a)intern.SerNet.DE>, size=1980, vsize=2031, from=Lutz Pressler <Lutz.Pressler(a)SerNet.DE>, flags=(\Deleted) Sep 5 15:41:22 pressler dovecot: imap(2580, lpmail): expunge: box=INBOX, uid=34722, msgid=<E1R0ZJW-00FrF7-0U(a)intern.SerNet.DE>, size=1980, vsize=2031, from=Lutz Pressler <Lutz.Pressler(a)SerNet.DE>, flags=(\Deleted) Sep 5 15:41:22 pressler dovecot: master: Error: service(imap): child 2580 killed with signal 11 (core dumps disabled) Also happens with mail_log plugin disabled. Any idea? Configuration on request (or tests with core dumps enabled). Regards, Lutz -- Lutz Preßler <Lutz.Pressler(a)SerNet.DE> http://www.SerNet.DE/ SerNet Service Network GmbH, Bahnhofsallee 1b, D-37081 Göttingen Tel.: +49-551-370000-2, FAX: +49-551-370000-9 AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen

2 1

[Dovecot] v2.1.alpha1 released
by Timo Sirainen 05 Sep '11

05 Sep '11

http://dovecot.org/releases/2.1/alpha/dovecot-2.1.alpha1.tar.gz http://dovecot.org/releases/2.1/alpha/dovecot-2.1.alpha1.tar.gz.sig So it's time for the first alpha version of Dovecot v2.1. There are no huge intrusive changes, so I expect v2.1.0 to be released this year (maybe even in a few months?) The biggest changes are related to full text search handling. I'll probably still make some changes to it and its configuration. Also it wouldn't be difficult to add more backends, does anyone want to add Xapian support for Dovecot? :) Mailbox list indexes are intended to improve performance of IMAP STATUS command lookups, since they could be simply looked up from the index without even opening the mailbox. TODO related to this: 1) optimize virtual mailbox handling so that it doesn't need to actually open all of the mailboxes' index files, it can simply look at the mailbox list index for changes. 2) Implement IMAP NOTIFY extension using this. Statistics tracking can be used to look at what's happening currently in your system and also its history. You can look at what's happening in command/session/user/domain/ip level. "doveadm stats top" gives a list of currently running sessions and sorts them by whichever is using the most disk io/cpu/whatever. "doveadm stats dump" followed by one of the levels shows you a raw dump of everything stored in that level. It would be nice if someone wrote a pretty Ajaxy HTML stats browser where you could zoom in/out of levels. I'd also like to hear if anyone has suggestions on how to improve what stats are gathered or how they could be exported. (I've thought about SNMP, but I'm not entirely sure how to export the data nicely using it.) Finally here's the NEWS file: * Plugins now use UTF-8 mailbox names rather than mUTF-7: acl, autocreate, expire, trash, virtual * auth_username_format default changed to %Lu. If you really want case sensitive usernames, set it back to empty. * Solr full text search backend changed to use mailbox GUIDs instead of mailbox names, requiring reindexing everything. solr_old backend can be used with old indexes to avoid reindexing, but it doesn't support some newer features. + imapc (= IMAP client) storage allows using a remote IMAP server to be used as storage. This allows using Dovecot as a smart (caching) proxy or using dsync to do migration from remote IMAP server. + Mailbox indexing via queuing indexer service (required for Lucene) + Lucene full text search (FTS) backend rewritten with support for different languages + FTS finally supports "OR" search operation + FTS supports indexing attachments via external programs + IMAP FUZZY extension, supported by Lucene and Solr FTS backends + Mailbox list indexes + Statistics tracking via stats service. Exported via doveadm stats. + Autocreate plugin creates/subscribes mailboxes physically only when the mailbox is opened for the first time. Mailbox listing shows the autocreated mailboxes even if they don't physically exist. + Password and user databases now support default_fields and override_fields settings to specify template defaults/overrides. - listescape plugin works perfectly now

7 28

[Dovecot] Possible signed/unsigned bug in libdovecot-storage (2:2.0.14 )
by Kui Zhang 05 Sep '11

05 Sep '11

Hello We are getting following errors when moving mail from inbox to any folder. "Message has been copied too many times (32767+1)" lazy_expunge is enabled. Error messages seems to be from libdovecot-storage. Any one can confirm the bug? or knows of a work around? Thanks K.Z

2 5

[Dovecot] User tries to move folder, then looses it
by Spyros Tsiolis 05 Sep '11

05 Sep '11

Hello, Well, this is kind of difficult to explain. I have this user that keeps making mistakes on his pc and sometimes I don't know if I should believe her or not. Anywhoo, this time round, she managed to loose a whole folder on Thunderbird. She claims she tried to move the folder from a tree directory structure (see : sub-sub-folder) she created for herself to another place underneath "root". Additionaly, she managed to mangle things up since she used on TB characters like "/" "\" and "&" . So thunderbird refuses to access those sub-folders on the folder tree and I cannot rename them (in order to have later access). Here's dovecot info : -------------------------------------------------------------------------- # 1.2.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.17.4.el5 i686 CentOS release 5.5 (Final) ext3 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/dovecot.log info_log_path: /var/log/dovecot/dovecot-info.log ssl_parameters_regenerate: 48 verbose_ssl: yes login_dir: /var/run/dovecot//login login_executable: /usr/local/dovecot/libexec/dovecot/imap-login login_greeting: * Dovecot ready * login_max_processes_count: 64 mail_location: maildir:/var/MailRoot/domains/%d/%n/Maildir mail_plugins: zlib auth default: verbose: yes debug: yes debug_passwords: yes passdb: driver: passwd-file args: /etc/dovecot/passwd passdb: driver: pam userdb: driver: static args: uid=vmail gid=vmail home=/home/vmail/%u userdb: driver: passwd -------------------------------------------------------------------------- TIA, s. ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis

1 0

[Dovecot] Trash Plugin
by Aydın Demirel 05 Sep '11

05 Sep '11

ReHi; I have a question about trash plugin. I added following lines into dovecot.conf: quota_rule = *:storage=2048000 quota_rule2 = Trash:storage=1MB quota_rule3 = SPAM:ignore When I set *:storage line as default quota_rule , There is no problem.. But when I removed this line and set Trash:storage as default quota_rule, plugin is not working.. That I said, Do I have to add *:storage line as first default line in config line? Regards.. -- *Aydın Demirel Endersys Ltd. Sistem Destek Mühendisi/ System Support Engineer* * *<<Endersys is the first Advanced Business Partner of Red Hat in Turkey! http://www.europe.redhat.com/partners/endersys/>> Phone : +90 216 470 9423 | GSM : +90 530 401 8203 Fax : +90 216 470 9508 | Web : http://www.endersys.com <http://www.endersys.com/> Blog : http://blog.endersys.com <http://blog.endersys.com/> Twitter : http://www.twitter.com/endersys LPI : The #1 Linux Certification for IT Professionals LPI (Linux Professional Institute) Turkey http://www.lpi-turkey.com <http://www.lpi-turkey.com/>

2 2