July 2015 - dovecot - dovecot.org

doveadm-server does NOT know fts commands
by SATOH Fumiyasu 16 Dec '15

16 Dec '15

Hi, I'm using Dovecot 2.2.13 with Dovecot Director, Doveadm server and FTS. I want to run doveadm on Director hosts, but Dovecot server on backend hosts reject "fts *" commands. On Dovecot Director hosts: # doveadm fts optimize -u u0001(a)example.jp # echo $? 75 # doveadm fts rescan -u u0001(a)example.jp # echo $? 75 doveadm index -u u0001(a)example.jp INBOX # echo $? 0 On Dovecot backend hosts: # grep doveadm /var/log/maillog Jul 14 10:25:40 backend dovecot: doveadm: Error: doveadm: Client sent unknown command: fts optimize Jul 14 10:25:51 backend dovecot: doveadm: Error: doveadm: Client sent unknown command: fts rescan Is this doveadm-server restriction? On Dovecot backend hosts (2): # doveadm fts optimize -u u0001(a)example.jp -S /var/run/dovecot/doveadm-server # echo $? 75 # doveadm fts rescan -u u0001(a)example.jp -S /var/run/dovecot/doveadm-server # echo $? 75 # doveadm index -u u0001(a)example.jp -S /var/run/dovecot/doveadm-server INBOX # echo $? 0 # doveadm fts optimize -u u0001(a)example.jp # echo $? 0 # doveadm fts rescan -u u0001(a)example.jp # echo $? 0 # doveadm index -u u0001(a)example.jp INBOX # echo $? 0 -- -- Name: SATOH Fumiyasu @ OSS Technology Corp. (fumiyas @ osstech co jp) -- Business Home: http://www.OSSTech.co.jp/ -- GitHub Home: https://GitHub.com/fumiyas/ -- PGP Fingerprint: BBE1 A1C9 525A 292E 6729 CDEC ADC2 9DCA 5E1C CBCA

2 3

mmap_disable=yes not honored always
by Peter Benko 06 Dec '15

06 Dec '15

<Sorry for the resend, it seems that the formatting of my original mail got garbled> Hi all, I'm experimenting with having the mail store on a 9p file system that lacks mmap() functionality. So I disabled it in dovecot: mmap_disable = yes However, I keep getting the following error messages in my log: Oct 15 16:55:00 computer-name dovecot: imap user(a)domain.com[192.168.1.3] Error: mmap() failed with file /mail/mailboxes/domain.com/user/indexpvt-accounts/other-user/mailboxes/INBOX/dovecot.index.pvt.log: Invalid argument I only get those messages for the shared folder's private indexes. This setup works perfectly on an ext2/3/4 fs. Any opinions, is this a bug? Is mmap_disable not mandatory for all operations? Best regards, Peter

2 1

[PATCH] [dovecot 2.2.9] Quota warnings ignored with FS quotas
by Grzegorz Nosek 06 Dec '15

06 Dec '15

Hi all, I noticed that in some circumstances quota warnings are ignored. The bug arises when both of the following are used: 1. percentage-based quota warnings, i.e.: quota_warning = storage=1%% quota-warning 1 %u 2. filesystem quota backend (and probably others, except for quotas configured directly in dovecot config) Percentage-based quota warnings have rule.bytes_limit recalculated based on root_set->default_rule.bytes_limit, however this value is zero when FS quotas are in use. Real quota values (from quotactl) are fetched very late, in quota_warnings_execute() but at that point no recalculation happens. As the warning rules have bytes_limit==0, they're effectively ignored. The patch below enables quota warnings to be sent when using filesystem (and possibly maildirsize-based) quotas. Based and tested on Ubuntu 14.04's dovecot 2.2.9. Best regards, Grzegorz Nosek diff --git a/src/plugins/quota/quota.c b/src/plugins/quota/quota.c index adbd70d..8e4d7e0 100644 --- a/src/plugins/quota/quota.c +++ b/src/plugins/quota/quota.c @@ -1163,6 +1163,8 @@ static void quota_warnings_execute(struct quota_transaction_context *ctx, &count_current, &count_limit) < 0) return; + quota_root_recalculate_relative_rules(root->set, bytes_limit, count_limit); + bytes_before = bytes_current - ctx->bytes_used; count_before = count_current - ctx->count_used; for (i = 0; i < count; i++) {

3 3

SIS attachments compressed
by Juan Carlos Sanchez 06 Dec '15

06 Dec '15

Hello: In case it is defined to use zlib compression, does SIS save attachments compressed? I have found this question replied in the past (2012) and the answer was NO. I ask again just to know if there have been changes or if it is in the roadmap to add compression. I have been testing to migrate from maildir+zlib to mdbox+zlib+sis and the results, in terms of space used, and worst than using just mdbox+zlib (without SIS), and suppose the reason in the lack of compression in attachments. Thanks in advance. -- ------------------------------------------------------ Juan Carlos Sanchez Hernandez Responsable de Seguridad y Correo Electronico Servicio de Planificacion Informatica y Comunicaciones Universidad Politecnica de Madrid Rectorado Avda. Ramiro de Maeztu 7 28040 Madrid ------------------------------------------------------

2 1

[Dovecot] lmtp_rcpt_check_quota not working
by Bernhard Schmidt 06 Dec '15

06 Dec '15

Hi, Dovecot 2.2.9 (Debian package from backports) does not seem to respect lmtp_rcpt_check_quota # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.4 deliver_log_format = from=<%e>, size=%p, message-id=<%m>, status=%$ disable_plaintext_auth = no lmtp_rcpt_check_quota = yes mail_debug = yes mail_gid = mstore mail_location = maildir:~/Maildir:INDEX=/var/cache/dovecot/index/%-1.1u/%u mail_plugins = quota mail_uid = mstore managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { quota = maildir quota_rule = ?:storage=512M quota_rule2 = INBOX.Trash:ignore quota_status_nouser = DUNNO quota_status_overquota = 452 4.2.2 Mailbox is full quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=85%% quota-warning 85 %u sieve = ~/currently-active-script.sieve sieve_dir = ~/sieve } pop3_uidl_format = %08Xv%08Xu protocols = imap lmtp sieve pop3 quota_full_tempfail = yes service anvil { client_limit = 3000 unix_listener anvil { group = sudo mode = 0660 } } service auth { client_limit = 3000 unix_listener auth-userdb { group = mstore mode = 0660 user = mstore } } service imap-login { client_limit = 1024 inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 2500 process_min_avail = 4 service_count = 0 } service lmtp { inet_listener lmtp { address = xxx port = 24 } } service managesieve-login { inet_listener sieve { port = 2000 } service_count = 1 } service managesieve { process_limit = 1024 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-status { client_limit = 5 executable = quota-status -p postfix inet_listener { address = xxx port = 12340 } } service quota-warning { executable = script /etc/dovecot/quotawarnmsg.sh unix_listener quota-warning { user = mstore } user = mstore } ssl_cert = </etc/dovecot/dovecot.pem ssl_key = </etc/dovecot/private/dovecot.pem userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } verbose_proctitle = yes protocol imap { mail_plugins = quota imap_quota imap_zlib } protocol lda { mail_plugins = quota sieve } protocol lmtp { mail_plugins = quota sieve } so Dovecot should reject in RCPT TO phase, but according to the logs of the upstream Postfix it does so after END-OF-DATA (host xxx[2001:xxx] said: 452 4.2.2 <user(a)domain.de> Quota exceeded (mailbox for user is full) (in reply to end of DATA command)) I haven't seen any commits regarding to this in 2.2.10 or tip. Regards, Bernhard

3 6

Re: Questions about hardlinks, alternate storage and compression
by Gionatan Danti 26 Nov '15

26 Nov '15

Hi Javier, thanks for your reply. I already checked SIS and, while interesting, is not what I want, because: 1) it can be difficult to restore a single message/attachment from a backup 2) only the attachments, and not the entire messages, are deduped. Message-based hardlinks really exists for a reason. The good news is that I found _why_ they are not working: it depends from how dovecot and its sieve plugin (pigenhole) interact. Basically, if I define anything for the before_sieve and after_sieve variables, dovecot stops creating hardlinks for multiple copies of the same message. On the other hand, private (per-user) sieve file works without interfering with hardlinks. In a similar manner, disabling sieve also permits dovecot to create multiple hardlinks for a single message. Does someone know if newer dovecot versions change anything in this regard? Thank you all. On 13/07/15 11:10, Javier Miguel Rodríguez wrote: > Search about "single instance storage dovecot". This is what you need. > > Regards > > Javier > > On 27/06/2015 18:18, Gionatan Danti wrote: >> Hi all, >> I have some questions about hardlinks, alternate storage and >> compression. I already scanned the list for related information and I >> have an idea of how things works, but I would like to have a definite >> answer. >> >> System spec: >> - CentOS 6.6 x64 >> - dovecot-2.0.9-8.el6_6.4.x86_64 RPM package/version >> - sdbox mail store >> >> 1) About hardlinks: when sending the same message to two different >> recipients, I see that the two u.x files are created as two different >> files. Diffing them, I see that the only difference is a single char >> (see [1] for an example). My questions are: >> a) it is possible to tell dovecot to create a single file + a single >> hardlink (linkref=2)? As other IMAP servers support that features (eg: >> Cyrus, CommunigatePro, etc) I am wondering if I missed something in >> configuring dovecot... >> b) If it is not possible, can I run a script that compare the >> various files and substitute equal ones (minus the changing line) with >> hardlinks, or it will confuse dovecot? As a side note, why the >> changing line ever exists? >> >> 2) About alternate storage and compression: actually, I use a single >> mail_location without compression. I would like to have an alternate >> storage and to enable compression on it only, leaving the main >> location without compression. I if understand it correctly, it >> _should_ be done using a command similar to "doveadm -Dv -o >> "plugin/zlib_save=gz" altmove -uu testuser sentbefore 8d". I'm right >> thinking that it should work? I will really end with a primary >> uncompressed mail store and an alternate, zlib-compressed one? >> >> Thank you all and sorry if I did some naive questions. >> >> [1] >> 63c63 >> < G2fd0811c64be8e553d970000eaa8309f >> --- >>> G2ed0811c64be8e553d970000eaa8309f >> >> >> > -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti(a)assyoma.it - info(a)assyoma.it GPG public key ID: FF5F32A8

3 6

imap_logout_format improvements / ideas
by Timo Sirainen 03 Oct '15

03 Oct '15

http://hg.dovecot.org/dovecot-2.2/rev/93bba97afb2a http://hg.dovecot.org/dovecot-2.2/rev/e0a17714f0c9 So all in all, you can now add to imap_logout_format: # %i - total number of bytes read from client # %o - total number of bytes sent to client # %{fetch_hdr_count} - Number of mails with mail header data sent to client # %{fetch_hdr_bytes} - Number of bytes with mail header data sent to client # %{fetch_body_count} - Number of mails with mail body data sent to client # %{fetch_body_bytes} - Number of bytes with mail body data sent to client # %{deleted} - Number of mails where client added \Deleted flag # %{expunged} - Number of mails that client expunged # %{trashed} - Number of mails that client copied/moved to the # special_use=\Trash mailbox. Any thoughts on what else would be useful? I was also thinking about separating (deletes and) expunges from Trash vs non-Trash, but I'm not sure how useful that would be (and what would it be called?) It's also a bit annoying that a copy+expunge to a Trash will return expunged=1 trashed=1, but a move to Trash (via IMAP MOVE) will return expunged=0 trashed=1. But trying to fix that would be too much trouble.

4 4

FTS not indexing new folders (mailboxes)
by Francisco Wagner C. Freire 24 Sep '15

24 Sep '15

Hello, Im getting problem on FTS on new created folders (mailboxes). How to simulate: - Enable FTS (solr on our case) - Create a new folder (mailbox) - Move some message to that mailbox - Try to search for the message contents *doveadm search -u user@domain text sometext* # Found nothing, but if i move the message to another mailbox, the search catch the message. # I tried doveadm index command but no response as well # This works only after a full fts rescan *doveadm fts rescan -u user@domain* # And now, works *doveadm search -u user@domain text sometext3c7b60241443ba55af680000cc4d0d4b 1* # My dovecot version = 2.2.18 (EE) # My fts config: fts = solr fts_autoindex = yes fts_autoindex_max_recent_msgs = 50 fts_solr = url=http://127.0.0.1:8080/solr/ Anyone got this problem?

3 2

Multiple passwords for a user (SQL)
by Marco Fretz 22 Sep '15

22 Sep '15

Hi everyone, I'm not sure if this has been discuessed already as I coun't find anything in the archives - maybe I'm looking for the wrong thing. I want to build something similar to googles "app passwords" where your use a different password (strong, auto generated) for each device / app. In general I think this is pretty easy on the database side as I just need a password table and link say the passwords with an user_id to the corresponding user in the users table - one use has many passwords (1:n). for dovecot this means that it will get multiple rows with passwords back from the "password_query". is there a way to tell dovecot to check all those returned passwords and "pass" the request if one of those passwords match? thank you, best regards Marco

2 3

[Dovecot] Pigeonhole sieve re-filter extension?
by T.B. 22 Sep '15

22 Sep '15

Dear Stephan Bosch, dear Dovecot community, "sieve-filter" re-filtering of existing mails is already a great progress for everyone using sieve! But as far as I know this functionality is just accessible by the server administrator. In the meanwhile everyone knows the comfort of re-executing filter scripts on his own existing mailboxes - triggered by himself! This service is already very widespread. Examples are MUA's like Outlook or Thunderbird which provide this service just on the client side. Or almost every web-mail provider offers this. Is there already an managesieve protocol extension that allows client triggered server side re-filtering? If not, maybe you could design a RFC for this? What do you think? Best regards. T.B.

4 8