- dovecot - dovecot.org

Re: Replication issue EOF or unknown flag V
by Rick Romero 11 Aug '16

11 Aug '16

Quoting Rick Romero <rick(a)havokmon.com>: > Hi, I'm trying to sort out a replication error: It was working > initially, but now it no longer works. Dovecot 2.2.25, Maildir format > On the 'master' side I see : Aug 09 14:02:17 dsync-server(rick at > havokmon.com[1]): Error: read(172.16.1.86) failed: EOF (version not > received) Aug 09 14:02:17 dsync-local(rick at havokmon.com[1]): Error: > read(172.16.1.86) failed: EOF (version not received) On the 'slave' > side I see: Aug 09 14:02:10 doveadm(172.16.1.81,rick at > havokmon.com[1]): Error: doveadm client: Unknown flag: V Aug 09 14:02:17 > doveadm(172.16.1.81,rick at havokmon.com[1]): Error: doveadm client: > Unknown flag: V on FreeBSD 10.2. I changed my mailbox to be on the 'slave' server, and the error messages switched sides. # 2.2.25 (7be1766): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.15 (97b3da0) # OS: FreeBSD 10.2-RELEASE amd64 auth_master_user_separator = * auth_mechanisms = plain login auth_username_translation = %@ auth_verbose = yes debug_log_path = /var/log/dovecot-debug.log default_internal_user = vpopmail default_login_user = dovecot default_vsz_limit = 1536 M disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 12345 first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 log_path = /dev/stderr login_greeting = Dovecot ready. login_trusted_networks = 172.16.1.0/24 mail_fsync = never mail_plugins = " quota zlib stats notify replication" mail_privileged_group = mail mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace compat { alias_for = hidden = yes inbox = no list = no location = prefix = INBOX. separator = . } namespace inbox { inbox = yes location = prefix = separator = . } passdb { args = /usr/local/etc/dovecot/dovecot-master-sql.conf driver = sql master = yes pass = yes } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } plugin { mail_replica = tcp:172.16.1.81 quota = maildir quota_rule = Trash:storage=+10%% stats_refresh = 30 secs stats_track_cmds = yes } replication_max_conns = 15 service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vpopmail } unix_listener replication-notify { mode = 0666 user = vpopmail } } service anvil { client_limit = 4799 } service auth-worker { user = $default_internal_user } service auth { client_limit = 4996 unix_listener auth-master { mode = 0600 } } service config { unix_listener config { user = vpopmail } } service doveadm { inet_listener { port = 12345 } user = vpopmail } service imap-login { process_limit = 2548 process_min_avail = 25 service_count = 1 } service imap-postlogin { executable = script-login /usr/local/etc/dovecot/lastauth-imap.sh user = vpopmail } service imap { executable = /usr/local/libexec/dovecot/imap imap-postlogin process_limit = 2048 } service lmtp { executable = lmtp -L inet_listener lmtp { port = 24 } user = vpopmail } service pop-postlogin { executable = script-login /usr/local/etc/dovecot/lastauth-pop.sh user = vpopmail } service pop3-login { process_limit = 2048 process_min_avail = 15 service_count = 1 } service pop3 { executable = /usr/local/libexec/dovecot/pop3 pop-postlogin } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } service stats { fifo_listener stats-mail { mode = 0600 user = vpopmail } } shutdown_clients = no ssl_cert = </etc/ssl/mail.pem ssl_key = </etc/ssl/mail.key userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_max_userip_connections = 25 mail_plugins = " quota zlib stats notify replication imap_zlib quota imap_quota notify replication" } protocol pop3 { mail_max_userip_connections = 25 mail_plugins = quota notify replication namespace inbox { location = mailbox Spam { auto = no autoexpunge = 1 weeks special_use = \Junk } mailbox Trash { auto = no autoexpunge = 2 weeks special_use = \Trash } prefix = } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = quota notify replication userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql name = } } protocol lmtp { auth_debug = no info_log_path = /var/log/dovecot-lmtp.log mail_plugins = sieve quota notify replication namespace inbox { location = mailbox Spam { auto = no autoexpunge = 1 weeks special_use = \Junk } mailbox Trash { auto = no autoexpunge = 2 weeks special_use = \Trash } prefix = } postmaster_address = admin(a)domain.com userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql name = } } driver = mysql connect = host= user= password= dbname=vpopmail default_pass_scheme = CRYPT password_query = SELECT CONCAT(pw_name, '@', pw_domain) AS user, pw_passwd AS password, IF(host IS NULL or host = '','172.16.1.86', host ) as host, 'Y' AS proxy_maybe, pw_dir as userdb_home, 89 as userdb_uid, 89 as userdb_gid FROM vpopmail WHERE pw_name = '%n' AND pw_domain = '%d' AND !(pw_gid & 8) AND !(pw_gid & 2) AND ('%r'!='172.16.1.93' or !(pw_gid & 4)) AND ('%r'!='172.16.1.92' or !(pw_gid & 4)) user_query = SELECT CONCAT(pw_name, '@', pw_domain) AS user, "89" as uid, "89" as gid, IF(host IS NULL or host = '','172.16.1.86', host ) as host, 'Y' AS proxy_maybe, pw_dir as home, pw_dir as mail_home, CONCAT('maildir:', pw_dir , '/Maildir/' ) as mail_location FROM vpopmail WHERE pw_name = '%n' AND pw_domain = '%d' iterate_query = SELECT CONCAT(pw_name, '@', pw_domain) AS user from vpopmail Links: ------ [1] http://dovecot.org/cgi-bin/mailman/listinfo/dovecot

1 0

Replication issue EOF or unknown flag V
by Rick Romero 09 Aug '16

09 Aug '16

Hi, I'm trying to sort out a replication error: It was working initially, but now it no longer works. Dovecot 2.2.25, Maildir format Delivering via LMTP. If I run doveadm replicator replicate rick(a)havokmon.com On the 'master' side I see : Aug 09 14:02:17 dsync-server(rick(a)havokmon.com): Error: read(172.16.1.86) failed: EOF (version not received) Aug 09 14:02:17 dsync-local(rick(a)havokmon.com): Error: read(172.16.1.86) failed: EOF (version not received) On the 'slave' side I see: Aug 09 14:02:10 doveadm(172.16.1.81,rick(a)havokmon.com): Error: doveadm client: Unknown flag: V Aug 09 14:02:17 doveadm(172.16.1.81,rick(a)havokmon.com): Error: doveadm client: Unknown flag: V This seems to just happen with my mailbox, so I've completely removed it on both sides, to no avail. My mailbox was re-created on the slave side, but nothing else has occurred (no folders or new mail). #doveadm user rick(a)havokmon.com field value uid 89 gid 89 home /home/prime/domains/havokmon.com/rick mail maildir:/home/prime/domains/havokmon.com/rick/Maildir/ host 172.16.1.81 proxy_maybe Y mail_home /home/prime/domains/havokmon.com/rick mail_location maildir:/home/prime/domains/havokmon.com/rick/Maildir/ Any thoughts?

1 0

custom mail_location detection for dovecot lda
by Alexander 'Leo' Bergolth 09 Aug '16

09 Aug '16

Hi! Is it possible to do something like "post-login scripting" to detect and set a per-user custom mail location for dovecot LDA delivery? I tried to use a wrapper-script around dovecot-lda that sets the environment variables MAIL="..." and USERDB_KEYS="MAIL" just like overriding userdb fields may be done using imap postlogin scripts, but unfortunately this doesn't seem to be honored. (Also tried with dovecot-lda -k.) Woud be very useful for slow migration from one mailbox format to another. Is there any other way to supply a custom mail_location to dovecot-lda? Cheers, --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria

2 4

help me with sieve_pipe
by 句号先生。 | 08 Aug '16

08 Aug '16

dear all I have a question about sieve_pipe use. how to open the plugin,and then specify the sieve script. the above is my dovecot configuration,and the sieve script,help me to have a look at the right ,thank you. Thanks !! Waiting....

2 1

Questions about how fts-solr works
by Radu Gheorghe 08 Aug '16

08 Aug '16

Hello, I'm quite a Dovecot newbie, so please be gentle :) Though I did my homework as well as I could, I still have some questions regarding the Solr plugin. Specifically: - I understand that by default, a mailbox is indexed on the first search and then deltas are indexed in subsequent searches. Are Emails indexed in batches or one by one? Looking at the code, I see a hardcoded limit of 1000, and I'm guessing if the mailbox is done and there are <1000 Emails in the buffer, it just flushes them, right? - if I set fts_autoindex=yes, does it mean that as soon as the Email is delivered by the MTA, it will be indexed in Solr? or does it have to read by the user or touched in any way? - also, with fts_autoindex=yes, are Emails indexed in batches? if yes, is there also a time limit besides the size limit? e.g. if only 100 messages were received - I have the same question about deletes: when do they happen, and are they batched? - what happens if Solr is unavailable? I know Dovecot keeps track of what indexed in dovecot.indexed files, but does it retry? if yes, what's the retry policy and can it be configured? Also, does it behave the same if Solr is actually available and throws an error? - the same question is for attachments, though I think this is general FTS - what if Tika fails to parse the attachment? Does Dovecot still index the Email metadata? As a side, I'm also wondering if I could use the Tika that comes with Solr (https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+Solr+C…) Can fts-solr handle attachments at all? I'm asking because I don't see that field in the default schema.xml - can I add arbitrary URI parameters to the Solr request? I see that one could fiddle with the path, which I assume will let one have one collection per mailbox (though I'm curious how that works with batches - I'm guessing one batch/indexing thread per mailbox?). Specifically, I'm interested in using the mailbox as a routing value - if I read the code well, Dovecot does a soft commit when it's done with the specific mailbox. For indexing at search time, I see why it makes sense. If I do "autoindex", can I disable that and let Solr autoSoftCommit every N seconds? That should improve indexing throughput and reduce load. I see that one can already do this for hard commits (I'd use autoCommit there, though a hard commit is also triggered when ramBufferSizeMB gets hit) - when querying, can I sort by an arbitrary field, such as the date? I saw I can sort by score, but I can't find anything in the code that will suggest it's supported - also when querying, can I specify which fields to return? I see that the plugin asks for Email ID, so I'm guessing it fetches things like from/to from the Email itself. I'm thinking that if I want to sort by those values I need to set docValues=true on them, to save memory. In that case, I might as well retrieve the original string from docValues, which should be a whole lot faster My plan is to do all sorts of tests, but having a better background on how it works will certainly help. Any pointers, feedback, encouragement, etc is certainly welcome - thanks in advance! Best regards, Radu -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/

1 0

Dovecot book available again
by Peer Heinlein 08 Aug '16

08 Aug '16

Hi, after my publisher has to shut down his business at the end of last year, it took several months to organize everything. But it's done! I'm happy to announce: The Dovecot book is available again. You can order it at Createspace: https://www.createspace.com/5942312 Or Amazon: https://www.amazon.com/dp/1534895701 https://www.amazon.co.uk/dp/1534895701 https://www.amazon.de/dp/1534895701 And in some weeks in every book store by ISBN-13 978-1534895706. If you want to support the author and if you help us to make good books you should think about ordering it from Createspace instead of Amazon. There's a 10% discount for users of this mailinglist if you use the discount code VR5YF7W2 at Createspace. Valid until August, 21th. Feedback and comments by mail is appreciated, also comments and ratings at Createspace and Amazon :-) Thanks again to Timo and everybody from Dovecot Oy / Open-Xchange for their support. Peer P.S.: The German Dovecot-Book is also available at: http://www.dovecot-buch.de/online-shop/buch-bestellung/ -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

6 5

Dovecot password policy
by Robert Blayzor 06 Aug '16

06 Aug '16

Is there a way to configure Dovecot to perhaps filter/enforce which passwords are accepted before authenticating? Ie: Reject immediately (without a database lookup) if password is not X characters in length? ? -- Robert inoc.net!rblayzor XMPP: rblayzor.AT.inoc.net PGP Key: 78BEDCE1 @ pgp.mit.edu

4 5

Re: Dovecot password policy
by Joseph Tam 05 Aug '16

05 Aug '16

Robert Blayzor <rblayzor.bulk(a)inoc.net> writes: > Is there a way to configure Dovecot to perhaps filter/enforce which > passwords are accepted before authenticating? Ie: Reject immediately > (without a database lookup) if password is not X characters in length? Yes, use the checkpassword hook. http://wiki.dovecot.org/AuthDatabase/CheckPassword I think there also some PAM module that you can stack into your system that will enforce password policies. Joseph Tam <jtam.home(a)gmail.com>

1 0

Dovecot and Solr 6
by KSB 05 Aug '16

05 Aug '16

Hi! Dovecot 2.2.24 Had set up solr and new schema collection. Copied dovecot provided schema. There was an error with booleans (while getting schema via http), which I "solved" by removing "add-unknown-fields-to-the-schema" from solrconfig.xml. It is correct way to solve this? Anyway, I run tcpdump to see network activity between dovecot and solr: #tcpdump -i lo port 8983 and see nothing while doing: #doveadm fts rescan -u username(a)domain.tld no output also from doveadm. Noticed that dovecot.index.log reapears in user mail root if deleted. plugin { fts = solr fts_solr = url=http://localhost:8983/solr/dovecot/ debug .. } -- KSB

3 4

Problem with sieve configuration on dovecot+debian_jessie+ISPC3
by Filip Loncar [WarpMax | FederationServers] 05 Aug '16

05 Aug '16

I'm new-ish to dovecot and sieve (haven't used it extensively till now) and I'm not sure that I configured everything propperly. The problem is that we're trying to get the following sieve code to run: require ["fileinto", "regex", "date", "relational", "vacation", "duplicate"]; require "duplicate"; if duplicate :header "message-id" { discard; } else { redirect "<somemail1>"; redirect "<somemail2>"; redirect "<somemail3>"; redirect "<somemail4>"; redirect "<somemail5>"; redirect "<somemail6>"; keep; } keep; But we're constantly getting the following error: sieve: info: started log at Aug 02 08:13:21. main script: line 5: error: require command: unknown Sieve capability `duplicate'. main script: line 6: error: unknown test 'duplicate' (only reported once at first occurence). main script: error: validation failed. Both me and my collegue can't find what's wrong with the system. I would appreciate any kind of help on this matter. Below's my configuration. # dovecot --version 2.2.13 # dovecot -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-45-pve x86_64 Debian 8.5 auth_mechanisms = plain login disable_plaintext_auth = no listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave vnd.dovecot.duplicate passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = /var/vmail/%d/%n/.sieve sieve_duplicate_default_period = 14d sieve_extensions = +vnd.dovecot.duplicate sieve_max_redirects = 6 sieve_plugins = sieve_extprograms } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service imap-login { client_limit = 1000 process_limit = 500 } ssl_cert = </etc/postfix/smtpd.cert ssl_key = </etc/postfix/smtpd.key ssl_protocols = !SSLv2 !SSLv3 userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = sieve quota } # uname -a Linux fero 2.6.32-45-pve #1 SMP Wed Mar 30 12:22:08 CEST 2016 x86_64 GNU/Linux It's a Debain Jessie VM running ISPC3. -- Srdac(an pozdrav | Best regards Filip Lonc(ar ___________________________________________ WarpMax hosting *WarpMax hosting* *A* Tehnolos(ki park Varaz(din, Zagrebac(ka 89, Varaz(din, Croatia - Europe *T * +385 42 492 129, 490 090, +385 99 5896 123 ___________________________________ Ova elektronic(ka poruka, kao i svi eventualni privici, su povjerljivi i namijenjeni iskljuc(ivo naznac(enom primatelju(ima). Ukoliko niste naznac(eni primatelj(i) ove elektronic(ke poruke, molimo da odmah obavijestite pos(iljatelja, a sve kopije ove elektronic(ke poruke (ukljuc(ujuc'i i privitke) trajno obris(ete. Svaka neovlas(tena upotreba, distribucija, reprodukcija ili priopc'avanje ove poruke zabranjeno je. WarpMax hosting ne preuzima odgovornost za sadrz(aj ove poruke i eventualnu s(tetu nastalu primitkom ove poruke i priloga sadrz(anih u poruci. Ne jamc(imo da je ova elektronic(ka poruka slobodna od virusa. WarpMax hosting ne preuzima odgovornost za bilo kakav gubitak ili s(tetu koja nastane zbog prisutnosti virusa u elektronic(koj poruci. This e-mail, including all attached files, if any, is confidential and intended for the sole use by the person(s) mentioned as recipient(s). If you are not the named recipient(s) indicated in this message, please notify the sender immediately, and permanently delete all copies of this message (including any attachments). Any review, use, distribution, disclosure, printing or copying of this e-mail other than by the intended recipient(s) is prohibited. WarpMax hosting accepts no liability for content of this message and any potential damage caused by this message and files attached to it. No warranty is given that this email is free of viruses. WarpMax hosting accepts no liability for any loss or damage caused by presence of a virus.

4 3