April 2016 - dovecot - dovecot.org

Dovecot Proxy LTMP client connect to TCP port 0
by Wido den Hollander 09 Apr '16

09 Apr '16

Hi, I am trying to set up a Dovecot proxy which proxies through POP3, IMAP and LTMP towards a different Dovecot machine. On the proxy machine I use a MySQL database as a userdb and passwdb backend and it returns the proper information as described here: http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy IMAP and POP3 works just fine, but with LMTP I run into a problem. On the 'proxy' machine Postfix is also running and it deliver locally to LTMP via Socket: virtual_transport = lmtp:unix:private/dovecot-lmtp Dovecot there is also configured to proxy LTMP: lmtp_proxy = yes service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } So far so good. A telnet to localhost 110 or 143 allows me to log in to the POP3/IMAP. When Postfix delivers the message locally to Dovecot through LTMP it tries to proxy it though. When doing so it tries to connect to TCP port 0 (zero). dovecot: lmtp(22580): Error: lmtp client: connect(mbox01.XXXX.nl, 0) failed: Connection refused I know I can return the 'port' field in the userdb query, but the same query is used for POP3, IMAP and LMTP. So that can't be hardcoded. I fixed it for now with a CASE statement in SQL: password_query = SELECT b.hostname AS host, NULL AS password, \ 'Y' AS nopassword, u.email AS destuser, 'Y' AS proxy, \ CASE '%s' WHEN 'lmtp' THEN 2525 WHEN 'pop3' THEN 110 WHEN 'imap' THEN 143 END AS port \ FROM User u, Backend b, Domain d \ WHERE u.domainID = d.domainID \ AND b.backendID = d.backendID \ AND u.email = '%u' %s is a variable containing the service Dovecot is trying to look up. This is however rather hacky. On my destination machine LMTP is listening on port 2525. Is there any way to tell the Dovecot LTMP client to connect to port 2525 by default? Thanks, Wido

1 1

fts_solr: Lookup failed: 400 Bad Request / GET null null
by Chris Laif 08 Apr '16

08 Apr '16

Hi, the log file shows suspicious entries as follows: Apr 3 09:14:49 xxx dovecot: imap(user@domain): Error: fts_solr: Lookup failed: 400 Bad Request The tomcat7 log shows for this request: 10.0.0.234 - - [03/Apr/2016:09:14:49 +0200] "GET null null" 400 - Dovecot is latest 2.2.23 with a pretty standard FTS config copied from the wiki: mail_plugins = " acl fts fts_solr" fts = solr fts_solr = break-imap-search url=http://10.0.0.123:8080/solr/ Unfortunately I do not know (yet) which client action produces these log lines. "GET null null" seems not to be a sensible request in any case. Anyone seeing the same effect? Chris

3 5

Intermittent IMAP Login failures - about 25% fail
by Mobile Phone 08 Apr '16

08 Apr '16

New server & just added three domains: now I see "dovecot: imap-login: Aborted login (auth failed, 1 attempts in 5 secs): user=<username.companyname>, method=PLAIN, rip=1.1.1.1, lip=99.99.99.99, TLS, session=<kw6Y2NYviQBex5Un>" Plus all Outlook users keep being bounced and finally get in. I am going to kill all the users soon... Not too many files open, nor bandwidth, nor load. Any light/help appreciated. root@brazil:/var/log# dovecot --version 2.2.9 root@brazil:/var/log# dovecot -n # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.19.0-58-generic x86_64 Ubuntu 14.04.4 LTS auth_debug_passwords = yes auth_mechanisms = plain login mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } protocols = imap pop3 pop3 ssl_cert = </etc/dovecot/dovecot.pem ssl_key = </etc/dovecot/private/dovecot.pem userdb { driver = passwd } Thanks

4 7

Permissions for new folders
by David Cigánek 08 Apr '16

08 Apr '16

Hi, I have a problem with permissions for newly created folders in Dovecot. Everything worked fine until few days ago, when i did a little change in userdb lookup - but i don't know why it is causing problems. Server uses following directory structure: index - /data/mail/index mail - /data/mail/mail alt - /data/mail/archive all 3 on separate volumes, for performance reasons. Up until few days, i used following: # doveadm user dave(a)example.com field value user dave(a)example.com uid 13726 gid 1004 home /data/mail/mail/003/3726/da/dave mail mdbox:~/mbox/:DIRNAME=.00f-dbox-Mails:INDEX=/data/mail/stub/index/data/mail/mail/003/3726/da/dave:ALT=/data/mail/stub/archive/data/mail/mail/003/3726/da/dave config is quite simple: mail_location = mdbox:~/mbox/:DIRNAME=.00f-dbox-Mails:INDEX=/data/mail/stub/index%h:ALT=/data/mail/stub/archive%h passdb { driver = sql args = /data/mail/sql/dovecot-sql.conf.ext } userdb { driver = static args = gid=vmail } from SQL i get those fileds username domain password userdb_home userdb_uid userdb_quota_rule userdb_quota_over_flag For nicer paths, i created a symlinks /data/mail/stub/index/data/mail -> /data/mail/index /data/mail/stub/archive/data/mail -> /data/mail/archive so path /data/mail/stub/index/data/mail/mail/003/3726/da/dave is in fact in /data/mail/index/003/3726/da/dave In that setup, everything worked fine, but i didn't like those simlinks, so i altered the SQL query to return all paths in userdb_mail filed: # doveadm user dave(a)example.com field value user dave(a)example.com uid 13726 gid 1004 home /data/mail/mail/003/3726/da/dave mail mdbox:~/mbox/:DIRNAME=.00f-dbox-Mails:INDEX=/data/mail/index/003/3726/da/dave:ALT=/data/mail/archive/003/3726/da/dave So everything is in fact the same but little nicer. But since then, newly created directories doesn't get the same permissions like parent folder as day should (2770), but instead they are created with 2700: # stat -c "%a (%A)" /data/mail/mail/ 2770 (drwxrws---) # stat -c "%a (%A)" /data/mail/mail/003 2700 (drwx--S---) I need to have there 2770 perms, especialy because when other user like this is created: user dave(a)example.com uid 13999 gid 1004 home /data/mail/mail/003/3999/da/dave Dovecot is unable to create directory /data/mail/mail/003/3999, because it doesn't have write permissions for group on /data/mail/mail/003. I can't find any reason for this behavior - is it something in Dovecot or am I doing something wrong? Thank you, David Dovecot version: 2.2.21 # doveconf -n # 2.2.21 (5345f22): /usr/local/dovecot/2.2.21/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.12 (c1c0a23) # OS: Linux 3.10.0-327.4.5.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_master_user_separator = xxxxxx dict { sieve_movespam = mysql:/data/mail/sql/dovecot-movespam-sql.conf.ext sieve_redirect = mysql:/data/mail/sql/dovecot-redirects-sql.conf.ext } mail_location = mdbox:~/mbox/:DIRNAME=.00f-dbox-Mails:INDEX=/data/mail/stub/index%h:ALT=/data/mail/stub/archive%h mail_plugins = " quota notify replication" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mdbox_rotate_size = 15 M passdb { args = xxxxxx driver = passwd-file master = yes pass = yes skip = authenticated } passdb { args = /data/mail/sql/dovecot-sql.conf.ext driver = sql } passdb { args = /data/mail/sql/dovecot-catchall-sql.conf.ext driver = sql } plugin { mail_replica = tcp:xxx:xxx quota = dict:user-quota::file:~/user_quota.dat quota2 = dict:domain-quota:%d:file:~/../../domain_quota.dat quota2_over_flag_value = 1 quota2_over_script = quota-warning domain %d %n@%d %i missmatch2 quota2_rule = *:bytes=500M quota2_rule2 = Trash:bytes=+5%% quota2_warning = bytes=90%% quota-warning domain %d %n@%d %i warn 90 quota2_warning2 = bytes=100%% quota-warning domain %d %n@%d %i over 100 quota2_warning3 = -bytes=100%% quota-warning domain %d %n@%d %i under 100 quota_grace = 5%% quota_over_flag_value = 1 quota_over_script = quota-warning user %n %n@%d %i missmatch1 quota_rule = *:bytes=0 quota_rule2 = Trash:bytes=+5%% quota_warning = bytes=90%% quota-warning user %n %n@%d %i warn 90 quota_warning2 = bytes=100%% quota-warning user %n %n@%d %i over 100 quota_warning3 = -bytes=100%% quota-warning user %n %n@%d %i under 100 sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = dict:proxy::sieve_movespam;name=sieve_movespam;bindir=~ sieve_before2 = dict:proxy::sieve_redirect;name=sieve_before;bindir=~ sieve_max_actions = 500 sieve_max_redirects = 500 sieve_redirect_envelope_from = recipient } pop3_no_flag_updates = yes protocols = imap pop3 lmtp sieve replication_dsync_parameters = -1 -d -N -l 30 -U replication_max_conns = 5 service aggregator { fifo_listener replication-notify-fifo { group = vmail mode = 0660 user = vmail } unix_listener replication-notify { group = vmail mode = 0660 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service doveadm { inet_listener { port = xxxx } } service imap-login { client_limit = 10000 inet_listener imap { port = 143 } process_min_avail = 50 service_count = 0 vsz_limit = 256 M } service imap { client_limit = 1 process_limit = 20000 vsz_limit = 1 G } service lmtp { inet_listener lmtp { port = 2525 } } service pop3-login { inet_listener pop3 { port = 110 } } service quota-warning { executable = script /data/mail/scripts/quota-actions.sh unix_listener quota-warning { mode = 0666 } user = root } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } ssl = no ssl_client_ca_file = /etc/pki/tls/cert.pem syslog_facility = local5 userdb { args = gid=vmail driver = static } userdb { args = /data/mail/sql/dovecot-sql.conf.ext driver = sql override_fields = gid=vmail } protocol imap { mail_max_userip_connections = 1000 mail_plugins = " quota notify replication imap_quota" } protocol lmtp { mail_plugins = " quota notify replication sieve" } protocol lda { mail_plugins = " quota notify replication sieve" }

1 0

Broken pipe with own dict server. Timing issue?
by martijn.list 08 Apr '16

08 Apr '16

Hi, I have created my own dict server (proxy:[<dict path>]:<destination dict>) to access a proprietary database. The dict server basically creates a listing socket on a separate thread and then waits for incoming requests with accept(handle, NULL, 0). When a request comes in (i.e., when accept returns the accepted socket fd), the request is handed over to a handler thread (from a thread pool). The handler thread then read from the input. The issue that I have is that if the read is done too fast after the accept, Dovecot reports a broken pipe: dovecot: auth-worker(19904): Error: write(/usr/share/test-server/tmp/test-dict) failed: Broken pipe This happens for almost all processes that try to access the dict server. However If I add some sort of delay between the accept call and read (for example a sleep), everything works. No broken pipe even after testing with a large number of parallel requests for a long time. Somehow it looks like if the first read is done too fast, it takes a long time to read the data. During this time it looks like all other Dovecot processes are no longer able to connect and report the broken pipe. With the artificial delay, everything works ok and I can see that multiple incoming request are handled by multiple threads so it looks like Dovecot uses multiple auth worker processes. If however one connection hangs on the read it looks like no Dovecot proccess is able to connect Any idea what might be causing the broken pipe in relation to timing of the read? Kind regards, Martijn

2 2

Own implementation of a dict server. Is dict lookup multi threaded or single threaded?
by martijn.list 08 Apr '16

08 Apr '16

Hi, I have created my own dict server (proxy:[<dict path>]:<destination dict>) to access a proprietary database. What happens if the dict server lookup is slow? for example because the lookup process takes some time? Does Dovecot access the dict proxy from multiple threads or is this a single threaded process? Kind regards, Martijn Brinkers

2 1

ETOOMANYREFS related errors
by Luis Ugalde 07 Apr '16

07 Apr '16

Hi, I'm starting to see, on a pretty standard Debian Jessie installation, some error messages that are apparently related to the ETOOMANYREFS errno. Firstly, the mail log shows this: dovecot: pop3-login: Error: fd_send(pop3, 18) failed: Too many references: cannot splice And then the login process fails: dovecot: pop3-login: Internal login failure (pid=34388 id=1) (internal failure, 1 successful auths): user=<username>, method=PLAIN, rip=rip, lip=lip, session=<session> Is this something that Dovecot should be able to handle, or is it strictly Debian/libc/MySillyMistake related? #uname -a Linux server 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 (2016-01-17) x86_64 GNU/Linux #dovecot --version 2.2.13 #dpkg -l | grep -E 'dovecot-core|libc6|linux-image' ii dovecot-core 1:2.2.13-12~deb8u1 amd64 ii libc6:amd64 2.19-18+deb8u3 amd64 ii libc6-i386 2.19-18+deb8u3 amd64 ii linux-image-3.16.0-4-amd64 3.16.7-ckt20-1+deb8u3 amd64 Best Regards, Luis

2 3

lmtp_rcpt_check_quota = yes silently blocks mail if quota check fails
by Tom Sommer 07 Apr '16

07 Apr '16

I have a user which has a quota-check that fails with 'Internal error', if I enable lmtp_rcpt_check_quota, the mail is silently tempfailed. Actually it looks like lmtp_rcpt_check_quota=yes suppresses normal Info/Error logging, even on Director. The MTA just gets " 451 4.3.0 <xxxx(a)xxx.xxx> Temporary internal error (in reply to end of DATA command)" from the director, but nothing is logged with lmtp_rcpt_check_quota=yes, anywhere. -- Tom

1 0

doveadm sync -1 -R
by Lukas Kolbe 07 Apr '16

07 Apr '16

Hi, I’m migrating an old Zarafa-installation to a dovecot using mdbox and sis on Debian Jessie. To copy over the emails, I use doveadm backup on the new box - or more precisely: vmail# doveadm -v -o imapc_user=$user -o imapc_password=$pass -o imapc_host=$host -o imapc_features=rfc822.size -o imapc_port=993 -o mailbox_list_index=no -o imapc_ssl_verify=no backup -R -x "Public folders*" -x "Public folders/*" -u $user imapc: For each invoke of doveadm backup -R, the local (new) mailbox gets wiped and all mails are retransferred - so far, so good. I’d like to use "doveadm sync -1“ as per the manpage to keep the downtime short, however sync doesn’t like the -R option: sync: invalid option -- 'R' doveadm sync [-u <user>|-A] [-S <socket_path>] [-1fPU] [-l <secs>] [-r <rawlog path>] [-m <mailbox>] [-g <mailbox_guid>] [-n <namespace> | -N] [-x <exclude>] [-s <state>] -d|<dest> If I understand it correctly, without -R it would mean that doveadm sync would incorporate the changes from the local dovecot-mailbox into the remote imapc:-Mailbox, which is the opposite of what I want. Dovecot version is current head as of yesterday. Is what I’m trying to do even possible with doveadm sync? Kind regards, Lukas

2 2

[help]enotify "mail to" run fine edit via managesieve 0.4.4 complied failed
by wuwei 07 Apr '16

07 Apr '16

hi all edit via vi editor to change dovecot.sieve and enotify works fine.but when I telnet 4190 port via managesieve to edit it with the same code,the managesieve said "NO "Error in MANAGESIEVE command CHECKSCRIPT: "", find many topic in wiki and maillist but not solved. need you help!thanks!!dovecot version 2.2.15,pigonhole version 0.4.4 here are more infos below: *managesieve cmds:* "IMPLEMENTATION" "Dovecot Pigeonhole" "SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate vacation-seconds" "NOTIFY" "mailto" "SASL" "PLAIN LOGIN" "VERSION" "1.0" OK "Dovecot ready." listscripts NO "Error in MANAGESIEVE command received by server." AUTHENTICATE "PLAIN" "AGFuZ196aGFAdG9tLmNvbQAxMTExMTE=" OK "Logged in." listscripts "blackwhitelist" "rulefilter" ACTIVE OK "Listscripts completed." getscript "rulefilter" {471} require ["reject","fileinto","imap4flags","vacation","regex","encoded-character","include","copy","enotify","variables"]; if header :matches "Subject" "*" { set "Subject" "${1}"; } if header :matches "From" "*" { set "From" "${1}"; } if header :matches "Message-ID" "*" { set "msg_id" "${1}"; } if header :matches "To" "*" { set "To" "${1}"; } if header :matches "Date" "*" { set "Date" "${1}"; } notify "mailto:wuweiwwd@tom.com"; OK "Getscript completed." checkscript {471} require ["reject","fileinto","imap4flags","vacation","regex","encoded-character","include","copy","enotify","variables"]; if header :matches "Subject" "*" { set "Subject" "${1}"; } if header :matches "From" "*" { set "From" "${1}"; } if header :matches "Message-ID" "*" { set "msg_id" "${1}"; } if header :matches "To" "*" { set "To" "${1}"; } if header :matches "Date" "*" { set "Date" "${1}"; } notify "mailto:wuweiwwd@tom.com"; *NO "Error in MANAGESIEVE command CHECKSCRIPT: "* BYE "Disconnected for inactivity" Connection closed by foreign host. *sieve code:* require ["reject","fileinto","imap4flags","vacation","regex","encoded-character","include","copy","enotify","variables"]; if header :matches "Subject" "*" { set "Subject" "${1}"; } if header :matches "From" "*" { set "From" "${1}"; } if header :matches "Message-ID" "*" { set "msg_id" "${1}"; } if header :matches "To" "*" { set "To" "${1}"; } if header :matches "Date" "*" { set "Date" "${1}"; } notify "mailto:wuweiwwd@tom.com"; *dovecot.conf ./dovecot -n* # 2.2.15: /home/mail/dovecot/etc/dovecot/dovecot.conf doveconf: Warning: service auth { client_limit=80960 } is lower than required under max. load (300000) doveconf: Warning: service anvil { client_limit=60000 } is lower than required under max. load (240003) # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.2 auth_cache_negative_ttl = 1 days auth_cache_size = 1000 k auth_cache_ttl = 1 days auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login debug_log_path = /home/mail/dovecot/logs/dovecot.log default_client_limit = 60000 default_internal_user = vmail default_login_user = dovecotauth default_process_limit = 60000 disable_plaintext_auth = no first_valid_gid = 1007 first_valid_uid = 1007 imapc_features = rfc822.size fetch-headers imapc_host = 172.24.173.212 imapc_password = tom123VipIMAP587 imapc_user = %u info_log_path = /home/mail/dovecot/logs/dovecot.log log_path = /home/mail/dovecot/logs/dovecot.log mail_debug = yes mail_fsync = never mail_plugins = fts fts_lucene mail_log notify quota zlib mail_prefetch_count = 20 maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate vacation-seconds namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox "Junk E-mail" { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Items" { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = . } passdb { args = /home/mail/dovecot/etc/dovecot/passwd.masterusers driver = passwd-file master = yes } passdb { args = /home/mail/dovecot/bin/CheckPassword driver = checkpassword } plugin { fts = lucene fts_lucene = whitespace_chars=@. mail_debug = yes mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename append mail_log_fields = uid box msgid size quota = dict:%u::redis:host=127.0.0.1:prefix=user_quota/ quota_rule2 = Trash:storage=+80%% quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=80%% quota-warning 80 %u quota_warning3 = storage=70%% quota-warning 70 %u quota_warning4 = storage=60%% quota-warning 60 %u sieve = ~/sieve/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +vacation-seconds sieve_max_redirects = 10 sieve_redirect_envelope_from = orig_recipient sieve_vacation_default_period = 0 sieve_vacation_max_period = 0 sieve_vacation_min_period = 0 welcome_letter_path = /home/mail/dovecot/etc/welcome welcome_letter_url = http://202.108.252.162:8080/domainmail zlib_save = gz zlib_save_level = 6 } pop3_fast_size_lookups = yes pop3_no_flag_updates = yes postmaster_address = postmastar@%d protocols = imap pop3 sieve lmtp service auth-worker { user = dovecotauth } service auth { client_limit = 80960 unix_listener /home/mail/postfix/var/spool/postfix/private/auth-client { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 143 } service_count = 0 vsz_limit = 256 M } service imap { executable = imap postlogin process_limit = 60240 } service lmtp { inet_listener { port = 11000 } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 0 vsz_limit = 256 M } service managesieve { process_limit = 80960 } service pop { executable = imap postlogin process_limit = 60240 } service pop3-login { inet_listener pop3 { port = 110 } service_count = 0 vsz_limit = 256 M } service postlogin { executable = script-login -d rawlog -b -t } service quota-warning { executable = script /home/mail/dovecot/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = no submission_host = 172.24.140.52:26 userdb { driver = prefetch } userdb { args = /home/mail/dovecot/etc/dovecot/dovecot-dict-auth.conf.ext driver = dict } protocol imap { imap_logout_format = bytes=%i/%o mail_plugins = fts fts_lucene mail_log notify quota zlib imap_quota imap_zlib } protocol pop3 { pop3_logout_format = bytes=%i/%o, top=%t/%p, retr=%r/%b, del=%d/%m, size=%s } protocol lmtp { mail_plugins = fts fts_lucene mail_log notify quota zlib quota sieve } protocol sieve { mail_max_userip_connections = 1000 mail_plugins = fts fts_lucene mail_log notify quota zlib quota managesieve_implementation_string = Dovecot Pigeonhole managesieve_logout_format = bytes=%i/%o managesieve_max_compile_errors = 10 managesieve_max_line_length = 65536 } -- View this message in context: http://dovecot.2317879.n4.nabble.com/help-enotify-mail-to-run-fine-edit-via… Sent from the Dovecot mailing list archive at Nabble.com.

2 2