January 2017 - dovecot

fts-solr: Returning 400 on searches; unescaped braces
by Michael Welsh Duggan 17 Jan '17

17 Jan '17

Using Debian, dovecot-solr 1:2.2.26.0-4, and solr-tomcat 3.6.2+dfsg-9, I am getting 400 errors when doing searches. Here is an example search query from dovecot that failed (captured with wireshark): Frame 23: 338 bytes on wire (2704 bits), 338 bytes captured (2704 bits) on interface 0 Linux cooked capture Internet Protocol Version 6, Src: ::1, Dst: ::1 Transmission Control Protocol, Src Port: 56860, Dst Port: 8080, Seq: 1, Ack: 1, Len: 250 Hypertext Transfer Protocol GET /solr/select?fl=uid,score&rows=2664&sort=uid+asc&q={!lucene+q.op%3dAND}(hdr:test+OR+body:test)&fq=%2Bbox:6d5de009f991854df726000012cf7b9c+%2Buser:md5i HTTP/1.1\r\n Host: localhost:8080\r\n Date: Mon, 19 Dec 2016 00:25:56 GMT\r\n Connection: Keep-Alive\r\n \r\n [Full request URI: http://localhost:8080/solr/select?fl=uid,score&rows=2664&sort=uid+asc&q={!l…] [HTTP request 1/1] [Response in frame: 25] Here is the same query from firefox, which succeeds: Frame 66: 646 bytes on wire (5168 bits), 646 bytes captured (5168 bits) on interface 0 Linux cooked capture Internet Protocol Version 6, Src: ::1, Dst: ::1 Transmission Control Protocol, Src Port: 56862, Dst Port: 8080, Seq: 1, Ack: 1, Len: 558 Hypertext Transfer Protocol GET /solr/select?fl=uid,score&rows=2664&sort=uid+asc&q=%7B!lucene+q.op%3DAND%7D(hdr:test+OR+body:test)&fq=%2Bbox:6d5de009f991854df726000012cf7b9c+%2Buser:md5i HTTP/1.1\r\n Host: localhost:8080\r\n Connection: keep-alive\r\n Cache-Control: max-age=0\r\n Upgrade-Insecure-Requests: 1\r\n User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.75 Safari/537.36\r\n Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\n DNT: 1\r\n Accept-Encoding: gzip, deflate, sdch, br\r\n Accept-Language: en-US,en;q=0.8\r\n \r\n [Full request URI: http://localhost:8080/solr/select?fl=uid,score&rows=2664&sort=uid+asc&q=%7B…] [HTTP request 1/1] [Response in frame: 86] The salient difference seems to be the encoding of the braces. Indeed in the tomcat 8 logs, I find the following which seems to corroborate my hypothesis: java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986 at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:467) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:667) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:789) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1437) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Indeed the braces are listed in fts-backend-solr.c as part of solr_escape_chars, so don't know how the braces are making it through unencoded. -- Michael Welsh Duggan (md5i(a)md5i.com)

2 3

Dovecot source code audit
by Timo Sirainen 17 Jan '17

17 Jan '17

Mozilla sponsored source code audit for Dovecot. So thanks to them we have our first public code audit: https://wiki.mozilla.org/MOSS/Secure_Open_Source/Completed#dovecot Dates: October 2016 - January 2017 dovecot is a POP and IMAP mailserver; it is used in 68% of IMAP server deployments worldwide. The audit was performed by Cure53. The team found the following problems: • 3 Low The Cure53 team were extremely impressed with the quality of the dovecot code. They wrote: "Despite much effort and thoroughly all-encompassing approach, the Cure53 testers only managed to assert the excellent security-standing of Dovecot. More specifically, only three minor security issues have been found in the codebase, thus translating to an exceptionally good outcome for Dovecot, and a true testament to the fact that keeping security promises is at the core of the Dovecot development and operations."

9 9

Re: Remove empty milder folders
by Joseph Tam 16 Jan '17

16 Jan '17

@lbutlr <kremels(a)kreme.com> writes: > Not sure if anyone will find this useful, but this is how I deleted a > bunch (several hundred) of empty mail folders from a user account: > > # doveadm mailbox status -u ?user(a)example.com" messages "*" ALL | grep "=0" | awk -F= '{print $1}' | awk '{print "rm -rf ."$1}' > list > ... > I probably could do some research on how to combine those two awk pipes > but this worked, and it was fast. You can collapse all 3 downstream processes doveadm ... | awk '/messages=0$/{print "rm -rf ./" substr($1,9}' >list Needs shell escaping/sanitizing if you don't want to be the victim of mischief (e.g. maildir='X;cd ..;rm -rf *'). Joseph Tam <jtam.home(a)gmail.com>

1 0

Sieve: fileinto :create issue with Public Folder
by dovecot＠avv.solutions 16 Jan '17

16 Jan '17

Hello Community, I face the following issue: - using a sieve script to file into a existing private folder: ok - using a sieve script to file into new private folder using ":create": ok - using a sieve script to file into a existing public folder: ok - using a sieve script to file into new public folder using ":create": NOK ! The returned message is: "Permission denied" and the message correctly falls back to inbox. HOWEVER, if I run "doveadm mailbox create -u myuser@mydomain Public/TheNewFolder", the folder is created, and the delete command works as well. Any tip? Running dovecot 2.2.13 on a pi/debian jessie Thanks

2 1

Replication: Can't unsubscribe from shared mailbox
by Karsten Heiken 16 Jan '17

16 Jan '17

Hi, I am running two dovecot servers active/active. Everything runs pretty great, except for the replication of subscriptions in a shared namespace. When I unsubscribe from a folder the subscription instantly re-appears. The timestamp on the subscriptions file is updated, but the entry is still in there. If the other node is shut down, everything works as expected, which leads me to believe that the subscription is re-applied upon a replicator-run. I attached a tcpdump of a dsync run. The only action that was executed was to unsubscribe from the folder shared/weinrot(a)luis.uni-hannover.de/INBOX. Is there any other way I might help debugging this? Thanks, Karsten # 2.2.24 (a82c823): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 2.6.32-44-pve x86_64 Debian 8.5 default_vsz_limit = 512 M dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } imap_max_line_length = 2 M lmtp_rcpt_check_quota = yes mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_gid = 7777 mail_location = maildir:%h/Maildir:LAYOUT=fs:DIRNAME=maiLdir mail_plugins = " zlib quota acl notify replication " mail_shared_explicit_inbox = yes mail_uid = 7777 mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags notify namespace { hidden = no ignore_on_failure = no inbox = no list = children location = maildir:%%h/Maildir:INDEXPVT=%h/shared/%%u:LAYOUT=fs:DIRNAME=maiLdir prefix = shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { hidden = no inbox = yes list = yes location = mailbox 30dTrash { auto = subscribe autoexpunge = 30 days special_use = \Junk } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_shared_dict = proxy::acl mail_replica = tcp:10.6.1.10:24245 quota = maildir:Postfach-Limit quota_grace = 2%% quota_rule = *:storage=8G quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags sieve_max_actions = 250 sieve_max_redirects = 200 zlib_save = gz zlib_save_level = 6 } protocols = " imap lmtp sieve pop3 sieve" replication_dsync_parameters = -d -l 300 -U -N service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener auth-userdb { group = vmail user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service doveadm { inet_listener { port = 24245 } } service imap-login { process_min_avail = 16 service_count = 0 } service imap { executable = imap postlogin process_limit = 30000 } service lmtp { inet_listener lmtp { port = 24 } vsz_limit = 1 G } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 1024 } service pop3-login { process_min_avail = 4 service_count = 0 } service pop3 { process_limit = 2500 } service postlogin { executable = script-login -d rawlog /usr/local/bin/dovecot-postlogin.sh } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 12340 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } shutdown_clients = no ssl_cert = </etc/dovecot/private/fullchain.pem ssl_key = </etc/dovecot/private/privkey.pem syslog_facility = local6 userdb { args = /etc/dovecot/dovecot-ldap.conf.ext default_fields = home=/var/vmail/%Ld/%Ln quota_rule=*:bytes=8589934592 driver = ldap } verbose_proctitle = yes protocol lmtp { mail_plugins = " zlib quota acl notify replication sieve quota" } protocol lda { mail_plugins = " zlib quota acl notify replication sieve" } protocol imap { imap_metadata = yes mail_max_userip_connections = 100 mail_plugins = " zlib quota acl notify replication imap_quota imap_acl" } protocol pop3 { mail_max_userip_connections = 10 mail_plugins = " zlib quota acl notify replication" }

2 2

Panic: file istream-zlib.c: ...assertion failed: (ret == -1) / core dump on signal 6 / zlib
by C. Bernard 15 Jan '17

15 Jan '17

Hello! I have several gzip'ed mbox Files in my home dir and the problem occurs when I use one of the gzip'ed files as shown mbox in the gui and want to see one mail of it. It also happens on the extended search from Horde which then such a gzip'ed file. Example file: # file In_2014.gz In_2014.gz: gzip compressed data, was "In_2014", last modified: Wed Dec 2 10:53:22 2015, from Unix This happens with any gz Files. That's why I tried to sort out any problem else before submitting this. Directly connecting to dovecot imap (circumvention imapproxy) shows the same error. This setup was in place for 4 years with another server: It did not occur with dovecot 2.2.25 an up-imapproxy1.2.7 (Horde 5.2.12/imp5.2.16) on FreeBsd9.3-p55 (my old server). (I did not update to 2.2.27 on the last server as the maintainer just removed the ssl option from config I...and I knew I would update anyway soon,so I have no test of 2.2.27 with the old setup) The actual setup as follows: System is 10.3-RELEASE-p15 on amd64 Horde ist 5.2.13 / imp 6.2.17 / php-horde_lz4-1.0.7_1 (which is I guess unused here) Lz4 is liblz4-131 FS is UFS connection: Horde webmail -> 1143:imapproxyd:143 -> dovceot imap (or directly -> same) Jan 15 08:40:58 server dovecot: imap(xxx): Panic: file istream-zlib.c: line 416 (i_stream_zlib_seek): assertion failed: (ret == -1) Jan 15 08:40:59 server kernel: pid 73887 (imap), uid 1000: exited on signal 6 (core dumped) Jan 15 08:40:59 server dovecot: imap(xxx): Fatal: master: service(imap): child 73887 killed with signal 6 (core dumped) Jan 15 08:40:59 server HORDE: [imp] [fetch] IMAP error reported by server. [pid 69574 on line 730 of "/usr/local/www/htdocs/m/imp/lib/Imap.php"] Jan 15 08:40:59 server HORDE: [imp] [status] IMAP error reported by server. [pid 69574 on line 730 of "/usr/local/www/htdocs/m/imp/lib/Imap.php"] and Jan 15 08:40:56 server in.imapproxyd[912]: LOGOUT: 'xxx' from server sd [12] Jan 15 08:40:58 server in.imapproxyd[912]: LOGIN: 'xxx' (127.0.0.1:33269) on existing sd [12] Jan 15 08:40:59 server in.imapproxyd[912]: Get_Server_conn(): Unable to reuse server sd [12] for user 'xxx' (127.0.0.1:50504). Connection closed by server. # dovecot -n # 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 10.3-RELEASE-p11 amd64 auth_mechanisms = plain login disable_plaintext_auth = no first_valid_gid = 100 first_valid_uid = 1000 mail_location = mbox:~/:INBOX=/var/mail/%u mail_plugins = " zlib" mail_privileged_group = mail passdb { args = session=yes dovecot driver = pam } protocols = imap pop3 service auth { unix_listener auth-client { mode = 0660 } unix_listener auth-master { mode = 0600 } user = root } service imap { drop_priv_before_exec = yes } ssl = no userdb { args = blocking=yes driver = passwd } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } protocol lda { sendmail_path = /usr/sbin/sendmail } (gdb) bt full #0 0x000000001115c39a in thr_kill () from /lib/libc.so.7 #1 0x000000001115c386 in raise () from /lib/libc.so.7 #2 0x000000001115c309 in abort () from /lib/libc.so.7 #3 0x0000000010e1c9b4 in default_fatal_handler () from /usr/local/lib/dovecot/libdovecot.so.0 #4 0x0000000010e1d6e1 in i_set_failure_internal () from /usr/local/lib/dovecot/libdovecot.so.0 #5 0x0000000010e1cc59 in i_panic () from /usr/local/lib/dovecot/libdovecot.so.0 #6 0x000000001144ffae in i_stream_create_deflate () from /usr/local/lib/dovecot/lib20_zlib_plugin.so #7 0x0000000010e28051 in i_stream_seek_mark () from /usr/local/lib/dovecot/libdovecot.so.0 #8 0x0000000010ad7f15 in istream_raw_mbox_seek () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #9 0x0000000010ad864d in mbox_file_seek () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #10 0x0000000010ada7ad in mbox_dotlock_touch () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #11 0x0000000010ada367 in mbox_dotlock_touch () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #12 0x0000000010a9d365 in mail_get_hdr_stream_because () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #13 0x0000000010b0cbe5 in index_mail_parse_headers () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #14 0x0000000010b0f9d9 in index_mail_get_special () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #15 0x0000000010b0f7b9 in index_mail_get_special () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #16 0x0000000010a9d49f in mail_get_special () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #17 0x000000000041cf1d in imap_fetch_handlers_deinit () #18 0x000000000041c561 in imap_fetch_more () #19 0x000000000041c1f8 in imap_fetch_more () #20 0x000000000040ff4e in cmd_fetch () #21 0x000000000041a4e6 in command_exec () #22 0x0000000000419573 in clients_destroy_all () #23 0x000000000041974f in clients_destroy_all () #24 0x0000000000418d9e in client_handle_input () #25 0x000000000041786c in client_input () #26 0x0000000010e31619 in io_loop_call_io () from /usr/local/lib/dovecot/libdovecot.so.0 #27 0x0000000010e3338d in io_loop_handler_run_internal () from /usr/local/lib/dovecot/libdovecot.so.0 #28 0x0000000010e31b33 in io_loop_handler_run () from /usr/local/lib/dovecot/libdovecot.so.0 #29 0x0000000010e31908 in io_loop_run () from /usr/local/lib/dovecot/libdovecot.so.0 #30 0x0000000010dbca08 in master_service_run () from /usr/local/lib/dovecot/libdovecot.so.0 #31 0x00000000004253f4 in main () Thanks C.

1 0

TLS feature missing
by Tributh 13 Jan '17

13 Jan '17

Hi, i was using dovecot 2.2.25 compiled with opnessl 1.0.2 I realised with a cipherscan utility that i was able to support mulitiple TLS curves. Now i upgraded to 2.2.27 with opnessl1.1.0 and was falling back to historical stages where my server only servers one TLS-curve: secp384r1 right now. One big reason to compile the new ersion with openssl1.1.0 was to bring CHACHA20-POLY1305 ciphers and X25519 curves to modern clients. The ciphers i am estimating are working fine, but X25519 and also secp521r1 ist now longer supported, like it was in dovecot 2.2.25. Is there something broken? Or a new (know missing) config feature? Or is it a bug ? Regards Torsten

1 0

Re: doveadm mailbox cryptokey - not found
by Evgeniy Korneechev 12 Jan '17

12 Jan '17

> > You forgot to load mail_crypt plugin. > > mail_plugins = $mail_plugins mail_crypt > With these settings also. # dovecot -n ..... mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " mail_crypt" mbox_write_locks = fcntl ..... # service dovecot restart # doveadm mailbox cryptokey usage: doveadm [-Dv] [-f ......... # ls /usr/lib64/dovecot/modules/* | grep mail_crypt /usr/lib64/dovecot/modules/lib05_mail_crypt_acl_plugin.so /usr/lib64/dovecot/modules/lib10_mail_crypt_plugin.so /usr/lib64/dovecot/modules/libfs_mail_crypt.so /usr/lib64/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so -- WBR, BaseALT/ALTLinux Team

1 1

doveadm mailbox cryptokey - not found
by Evgeniy Korneechev 12 Jan '17

12 Jan '17

Hi, we have a problem. Command "doveadm mailbox cryptokey" does not work: # doveadm mailbox cryptokey usage: doveadm [-Dv] [-f <formatter>] mailbox <command> [<args>] create [-u <user>|-A] [-S <socket_path>] [-s] [-g <guid>] <mailbox> [...] delete [-u <user>|-A] [-S <socket_path>] [-e] [-r] [-s] [-Z] <mailbox> [...] ...... What is the problem? ------------------------------------------------ Install: http://wiki2.dovecot.org/CompilingSource#Compiling_Dovecot_From_Sources : ./configure && make && sudo make install # dovecot -n # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # OS: Linux 4.4.38-std-def-alt0.M80P.1 x86_64 ALT starter kit (Hypericum) mail_location = mbox:~/mail:INBOX=/var/mail/%u mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } ssl = required ssl_cert = </var/lib/ssl/certs/dovecot.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } -- WBR, BaseALT/ALTLinux Team

3 2

Unable to access apt.dovecot.fi
by Alexandre Ellert 12 Jan '17

12 Jan '17

Hi, I bought a Dovecot Enterprise repository acces in 2013 and since several months, I can't access it anymore. It seem that my credential are broken. When doing an apt-get update : W: Failed to fetch http://apt.dovecot.fi/3rdparty/debian/wheezy/dists/wheezy/main/binary-amd64… 401 Unauthorized W: Failed to fetch http://apt.dovecot.fi/stable-2.2/debian/wheezy/dists/wheezy/main/binary-amd… 401 Unauthorized And here is my sources.list : # cat /etc/apt/sources.list.d/dovecot.list deb http://USER:PASSWORD@apt.dovecot.fi/stable-2.2/debian/wheezy wheezy main # cat /etc/apt/sources.list.d/dovecot-3rdparty.list deb http://USER:PASSWORD@apt.dovecot.fi/3rdparty/debian/wheezy wheezy main Thank you for help. Alexandre

2 1