December 2015 - dovecot

[PATCH] [dovecot 2.2.9] Quota warnings ignored with FS quotas
by Grzegorz Nosek 06 Dec '15

06 Dec '15

Hi all, I noticed that in some circumstances quota warnings are ignored. The bug arises when both of the following are used: 1. percentage-based quota warnings, i.e.: quota_warning = storage=1%% quota-warning 1 %u 2. filesystem quota backend (and probably others, except for quotas configured directly in dovecot config) Percentage-based quota warnings have rule.bytes_limit recalculated based on root_set->default_rule.bytes_limit, however this value is zero when FS quotas are in use. Real quota values (from quotactl) are fetched very late, in quota_warnings_execute() but at that point no recalculation happens. As the warning rules have bytes_limit==0, they're effectively ignored. The patch below enables quota warnings to be sent when using filesystem (and possibly maildirsize-based) quotas. Based and tested on Ubuntu 14.04's dovecot 2.2.9. Best regards, Grzegorz Nosek diff --git a/src/plugins/quota/quota.c b/src/plugins/quota/quota.c index adbd70d..8e4d7e0 100644 --- a/src/plugins/quota/quota.c +++ b/src/plugins/quota/quota.c @@ -1163,6 +1163,8 @@ static void quota_warnings_execute(struct quota_transaction_context *ctx, &count_current, &count_limit) < 0) return; + quota_root_recalculate_relative_rules(root->set, bytes_limit, count_limit); + bytes_before = bytes_current - ctx->bytes_used; count_before = count_current - ctx->count_used; for (i = 0; i < count; i++) {

3 3

SIS attachments compressed
by Juan Carlos Sanchez 06 Dec '15

06 Dec '15

Hello: In case it is defined to use zlib compression, does SIS save attachments compressed? I have found this question replied in the past (2012) and the answer was NO. I ask again just to know if there have been changes or if it is in the roadmap to add compression. I have been testing to migrate from maildir+zlib to mdbox+zlib+sis and the results, in terms of space used, and worst than using just mdbox+zlib (without SIS), and suppose the reason in the lack of compression in attachments. Thanks in advance. -- ------------------------------------------------------ Juan Carlos Sanchez Hernandez Responsable de Seguridad y Correo Electronico Servicio de Planificacion Informatica y Comunicaciones Universidad Politecnica de Madrid Rectorado Avda. Ramiro de Maeztu 7 28040 Madrid ------------------------------------------------------

2 1

[Dovecot] lmtp_rcpt_check_quota not working
by Bernhard Schmidt 06 Dec '15

06 Dec '15

Hi, Dovecot 2.2.9 (Debian package from backports) does not seem to respect lmtp_rcpt_check_quota # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.4 deliver_log_format = from=<%e>, size=%p, message-id=<%m>, status=%$ disable_plaintext_auth = no lmtp_rcpt_check_quota = yes mail_debug = yes mail_gid = mstore mail_location = maildir:~/Maildir:INDEX=/var/cache/dovecot/index/%-1.1u/%u mail_plugins = quota mail_uid = mstore managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { quota = maildir quota_rule = ?:storage=512M quota_rule2 = INBOX.Trash:ignore quota_status_nouser = DUNNO quota_status_overquota = 452 4.2.2 Mailbox is full quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=85%% quota-warning 85 %u sieve = ~/currently-active-script.sieve sieve_dir = ~/sieve } pop3_uidl_format = %08Xv%08Xu protocols = imap lmtp sieve pop3 quota_full_tempfail = yes service anvil { client_limit = 3000 unix_listener anvil { group = sudo mode = 0660 } } service auth { client_limit = 3000 unix_listener auth-userdb { group = mstore mode = 0660 user = mstore } } service imap-login { client_limit = 1024 inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 2500 process_min_avail = 4 service_count = 0 } service lmtp { inet_listener lmtp { address = xxx port = 24 } } service managesieve-login { inet_listener sieve { port = 2000 } service_count = 1 } service managesieve { process_limit = 1024 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-status { client_limit = 5 executable = quota-status -p postfix inet_listener { address = xxx port = 12340 } } service quota-warning { executable = script /etc/dovecot/quotawarnmsg.sh unix_listener quota-warning { user = mstore } user = mstore } ssl_cert = </etc/dovecot/dovecot.pem ssl_key = </etc/dovecot/private/dovecot.pem userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } verbose_proctitle = yes protocol imap { mail_plugins = quota imap_quota imap_zlib } protocol lda { mail_plugins = quota sieve } protocol lmtp { mail_plugins = quota sieve } so Dovecot should reject in RCPT TO phase, but according to the logs of the upstream Postfix it does so after END-OF-DATA (host xxx[2001:xxx] said: 452 4.2.2 <user(a)domain.de> Quota exceeded (mailbox for user is full) (in reply to end of DATA command)) I haven't seen any commits regarding to this in 2.2.10 or tip. Regards, Bernhard

3 6

Dovecot 2.2.18.2: Panic: file fs-api.c: line 756 (fs_copy): assertion failed: (src->fs == dest->fs)
by Frank Mehrtens 06 Dec '15

06 Dec '15

Hello, a colleague found some strange messages in the logs: Oct 27 13:08:43 mail postfix/qmgr[1618]: E83C8180F56: from=<user1(a)domain.de>, size=298937, nrcpt=2 (queue active) Oct 27 13:08:43 mail dovecot: lmtp(6740): Connect from local Oct 27 13:08:43 mail dovecot: lmtp(user2(a)domain.de): gYFbBEdpL1ZUGgAA0nA0dQ: message-id=<035901d110b0$3929bc20$ab7d3460$(a)domain.de>: saved mail to INBOX (from: user1(a)domain.de, subject: WG: Matthias, size: 295392) Oct 27 13:08:43 mail postfix/lmtp[6702]: E83C8180F56: to=<user2(a)domain.de>, relay=mail.domain.de[private/dovecot-lmtp], delay=0.41, delays=0.36/0/0/0.05, dsn=2.0.0, status=sent (250 2.0.0 <user2(a)domain.de> gYFbBEdpL1ZUGgAA0nA0dQ Saved) Oct 27 13:08:43 mail dovecot: lmtp(mailarchiv(a)domain.de): Panic: file fs-api.c: line 756 (fs_copy): assertion failed: (src->fs == dest->fs) Oct 27 13:08:43 mail dovecot: lmtp(mailarchiv(a)domain.de): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x7b53a) [0x7f10419e553a] -> /usr/lib64/dovecot/libdovecot.so.0(+0x7b5a6) [0x7f10419e55a6] -> /usr/lib64/dovecot/libdovecot.so.0(+0x281ea) [0x7f10419921ea] -> /usr/lib64/dovecot/libdovecot.so.0(+0x3163a) [0x7f104199b63a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(sdbox_copy+0x4c4) [0x7f1041cbad94] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_copy+0x7a) [0x7f1041c9f37a] -> /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver_save+0x1a7) [0x7f1041f891b7] -> /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver+0x11b) [0x7f1041f8956b] -> dovecot/lmtp() [0x406922] -> dovecot/lmtp() [0x40730d] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7f10419f7fbb] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xaf) [0x7f10419f918f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f10419f8049] -> /usr/lib64/dovecot/libdovec ot.so.0(io_loop_run+0x38) [0x7f10419f80c8] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f1041997043] -> dovecot/lmtp(main+0x18b) [0x404dbb] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f10415f4d5d] -> dovecot/lmtp() [0x404b49] Oct 27 13:08:43 mail dovecot: lmtp(mailarchiv(a)domain.de): Fatal: master: service(lmtp): child 6740 killed with signal 6 (core dumps disabled) Oct 27 13:08:43 mail postfix/lmtp[6702]: E83C8180F56: to=<mailarchiv(a)domain.de>, relay=mail.domain.de[private/dovecot-lmtp], delay=0.41, delays=0.36/0/0/0.05, dsn=4.4.2, status=deferred (lost connection with mail.domain.de[private/dovecot-lmtp] while sending end of data -- message may be sent more than once) Postfix always_bcc set to mailarchiv(a)domain.de. [root@mail ~]# rpm -qa | grep dove dovecot-ee-2.2.18.2-1.el6.x86_64 dovecot-ee-managesieve-0.4.8-1.el6.x86_64 dovecot-ee-pigeonhole-0.4.8-1.el6.x86_64 [root@mail ~]# doveconf -n # 2.2.18.2 (866bffbafde7): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (73d4f6cd676a) # OS: Linux 2.6.32-573.7.1.el6.x86_64 x86_64 CentOS release 6.7 (Final) ext4 auth_cache_negative_ttl = 0 auth_mechanisms = plain login deliver_log_format = message-id=%m: %$ (from: %f, subject: %s, size: %p) dict { quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no imap_client_workarounds = tb-extra-mailbox-sep mail_attachment_dir = /storage/mail/attachments mail_attachment_hash = %{sha512} mail_gid = vmail mail_location = sdbox:/storage/mail/%d/%n mail_plugins = acl quota mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate namespace { list = children location = sdbox:/storage/mail/%%d/%%n/ prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace { list = children location = maildir:/storage/mail/public:INDEX=/storage/mail/%d/%n/public prefix = public/ separator = / subscriptions = no type = public } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox "Gelöschte Objekte" { special_use = \Trash } mailbox "Gesendete Objekte" { special_use = \Sent } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = file:/storage/mail/shared-mailboxes sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service imap-login { inet_listener imaps { ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service pop3-login { inet_listener pop3s { ssl = yes } } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = acl quota sieve } protocol imap { mail_plugins = acl quota imap_acl imap_quota } Best regards Frank -- DECOIT GmbH Fahrenheitstraße 9 D-28359 Bremen Tel.: +49-421-596064-0 Fax: +49-421-596064-09 E-Mail: info(a)decoit.de <mailto:info@decoit.de> Web: http://www.decoit.de <http://www.decoit.de/> Amtsgericht Bremen, HRB 21533 Geschäftsführer: Prof. Dr.-Ing. Kai-Oliver Detken

2 1

mailbox_list_index and maildir_very_dirty_syncs are in conflicts?
by Alessio Cecchi 06 Dec '15

06 Dec '15

Hi, I’m running Dovecot 2.2.19 with Maildir as storage and LDA for delivery. I noticed that if I set mailbox_list_index=yes and maildir_very_dirty_syncs=yes when I login via IMAP the STATUS command don’t “see” new messages in sub-folders (like Spam). Example, a new message as arrived in Spam folder (I can see in Maildir/new/) but IMAP STATUS don’t see it: . STATUS Spam (MESSAGES UNSEEN) * STATUS Spam (MESSAGES 139 UNSEEN 0) (in this case the date of “dovecot.list.index” files is not updated) If I comment out mailbox_list_index=yes or maildir_very_dirty_syncs=yes I can see the new message: . STATUS Spam (MESSAGES UNSEEN) * STATUS Spam (MESSAGES 140 UNSEEN 1) (in this case the date of “dovecot.list.index” files is updated after the STATUS command) Instead the SELECT command works fine always. An importante note, my dovecot LDA configuration (on MX servers) don’t update index files: protocol lda { mail_location = maildir:~/Maildir:INDEX=MEMORY mail_plugins = quota acl expire fts fts_solr zlib sieve } These because I need to filter incoming email via Sieve but since I cannot use LMTP (and Director) on MX (but I have Director for POP/IMAP access) the only way for not corrupting dovecot.index files is not update their on delivery emails. But reading http://wiki2.dovecot.org/MailLocation/Maildir this shouldn’t be a problem (Optimizations “maildir_very_dirty_syncs=yes” … It's still safe to deliver new mails to new/ …) since MX deliver new emails in new/. Is a Dovecot limit or is a problem of my configuration? Thanks my configuration: # 2.2.19 # Pigeonhole version 0.4.9 auth_cache_negative_ttl = 5 mins auth_cache_size = 10 M auth_cache_ttl = 20 mins auth_master_user_separator = * auth_worker_max_count = 50 deliver_log_format = msgid=%m, from=%f, subject="%s": %$ dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags imap_idle_notify_interval = 29 mins imap_logout_format = in=%i out=%o session=<%{session}> last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = 10.96.3.156 login_trusted_networks = 10.0.0.0/24 mail_fsync = always mail_location = maildir:~/Maildir mail_plugins = quota acl expire fts fts_solr zlib mailbox_list_index = yes maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate vnd.dovecot.duplicate mmap_disable = yes namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-deny-sql.conf.ext deny = yes driver = sql } passdb { args = /etc/dovecot/extra/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl antispam_backend = mailtrain antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/bin/sa-learn antispam_mail_spam = --spam antispam_spam = Spam antispam_trash = Trash expire = Trash expire2 = Spam expire_dict = proxy::expire fts = solr fts_solr = url=http://10.0.0.1:8983/solr/ quota = maildir:UserQuota quota2 = dict:Quota Usage::noenforcing:proxy::sqlquota quota_grace = 10M quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /etc/dovecot/sieve/before.sieve sieve_dir = ~/sieve sieve_extensions = +vnd.dovecot.duplicate -vacation zlib_save = gz zlib_save_level = 6 } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_fast_size_lookups = yes pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s, bytes=%i/%o, session=<%{session}> protocols = imap pop3 sieve sendmail_path = /var/qmail/bin/sendmail service auth { client_limit = 6524 unix_listener auth-userdb { group = vchkpw mode = 0660 user = vpopmail } } service dict { process_limit = 500 unix_listener dict { group = vchkpw mode = 0660 user = vpopmail } } service imap-login { process_min_avail = 4 service_count = 0 } service imap-postlogin { executable = script-login /etc/dovecot/scripts/imap-postlogin.sh unix_listener imap-postlogin { group = vchkpw mode = 0660 user = vpopmail } user = vpopmail } service imap { executable = imap imap-postlogin process_limit = 5000 service_count = 100 vsz_limit = 384 M } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { process_min_avail = 4 service_count = 0 } service pop3-postlogin { executable = script-login /etc/dovecot/scripts/pop3-postlogin.sh unix_listener pop3-postlogin { group = vchkpw mode = 0660 user = vpopmail } user = vpopmail } service pop3 { executable = pop3 pop3-postlogin process_limit = 1024 service_count = 100 } service quota-warning { executable = script /etc/dovecot/scripts/quota-warning.sh unix_listener quota-warning { user = vpopmail } user = vpopmail } ssl_cert = </etc/dovecot/ssl/wildcard.pem ssl_key = </etc/dovecot/ssl/wildcard.key ssl_protocols = !SSLv2 !SSLv3 submission_host = 127.0.0.1 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lda { mail_location = maildir:~/Maildir:INDEX=MEMORY mail_plugins = quota acl expire fts fts_solr zlib sieve } protocol imap { mail_max_userip_connections = 25 mail_plugins = quota acl expire fts fts_solr zlib imap_quota imap_acl imap_zlib } protocol sieve { mail_max_userip_connections = 2 } protocol pop3 { mail_max_userip_connections = 5 } -- Alessio Cecchi Postmaster AT http://www.qboxmail.it http://www.linkedin.com/in/alessice

2 1

ACL Troubles
by Bobber 06 Dec '15

06 Dec '15

Using dovecot version 2.2.19. In the config file I have the following to set up acls: > mail_plugins = $mail_plugins acl > protocol imap { > mail_plugins = $mail_plugins imap_acl > } > plugin { > acl = vfile:/usr/local/etc/dovecot/acls:cache_secs=300 > } And here's my acl file: > * anyone rl I am getting the following error in the dovecot log file: > Error: Global ACL file /usr/local/etc/dovecot/acls line 1: Unknown ID 'anyone rl' Also, my subfolders in the shared mailbox area are no longer visible or accessible. Any ideas what I am doing wrong? -- *Bob Wooldridge* Blog: http://kc0dxf.net/blog/

2 1

LIST MANAGEMENT BROKEN
by Edwardo Garcia 06 Dec '15

06 Dec '15

Timo I have for two days try to unsubscribe from this list using email mailman. The list server does not send me confirmation request. Please fix your server and remove me

2 1

Released Pigeonhole v0.4.10.rc1 for Dovecot v2.2.20.rc1
by Stephan Bosch 05 Dec '15

05 Dec '15

Hello Dovecot users, Here's the Pigeonhole RC that goes with the Dovecot RC. There is a conflict with the t_str_trim() function. To be able to quickly release Dovecot, Timo has reverted the involved change so that 0.4.9 will still compile with the final Dovecot v2.2.20 release. This Pigeonhole RC will therefore not compile against the final Dovecot release (only against v2.2.20.rc1), which I will mend quickly once Dovecot v2.2.20 is released. Changelog v0.4.10.rc1: + Implemented the Sieve mime and foreverypart extensions (RFC 5703). These are fully implemented. The interaction with the editheader extension needs some work, but this should not influence most uses; i.e., changes by the editheader extension are not always visible using foreverypart/mime. + Sieve body extension: Properly implemented the `:text' body transform. It now extracts text for HTML message parts. + Sieve enotify extension: mailto method: Implemented the sieve_notify_mailto_envelope_from setting. This allows configuring the source of the notification sender address for e-mail notifications. This is similar to what already can be configured for redirect. + Added a sieve_enabled (defaults to 'yes') setting that allows explicitly disabling Sieve processing for particular users. This used to be possible by setting `sieve=', but ever since the sieve_before, sieve_after and sieve_default settings were added, this method was not reliable anymore. - variables extension: Fixed handling of empty string by the `:length' set modifier. An empty string yielded an empty string rather than "0". - Fixed memory leak in the Sieve script byte code dumping facility. Extension contexts were never actually freed. - Fixed handling of implicit keep when the last Sieve script is a global one. In that case the implicit keep action was executed in global context, which could mean that trivial (quota) errors ended up in the system log file, rather than the user log file. - doveadm sieve plugin: Fixed crashes caused by incorrect context allocation in the sieve command implementations. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.10… http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.10… Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this release candidate and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

1 0

Content-Enconding
by Leander Schäfer 05 Dec '15

05 Dec '15

Hi, With Apache one may use "mod_deflate" in order to reduce bandwidth by using e.g. gzip to compress the http traffic. I would like to use something similar for email traffic between mail clients to dovecote and postfix. My questions are: a) Is this established well enough in most mail clients? b) Can you recomment a good tutorial for this? Thank you very much

2 3

object storage
by Bradley Giesbrecht 04 Dec '15

04 Dec '15

Is the Dovecot Object Storage plugin still available for purchase? Clicking the store link [2] from this page [1] does not show any info about purchasing the plugin. [1] http://www.dovecot.fi/dovecot-object-storage-plugins-available-for-online-p… [2] http://shop.dovecot.fi/ Regards, Bradley Giesbrecht (pixilla)

7 7