v2.2.17 release candidate released
http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz.sig
Quite a lot of changes, so here's a release candidate first. If no problems are reported I'll make v2.2.17 release early next week. Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16.
* Dovecot no longer checks or warns if a mountpoint is removed. This
was causing more trouble than it was worth. Make sure that all the
mountpoints that Dovecot accesses aren't writable by mail processes
when they're unmounted.
* dict server wasn't properly escaping/unescaping data. Fixing this
broke backwards compatibility with data that contains line feeds.
This hopefully affects only very few installations. If you're using
dict to save multiline data (Sieve scripts to SQL), you may be
affected.
* imap: SPECIAL-USE capability is no longer advertised if there are
no special_use flags specified for any mailboxes.
+ lmtp: Added lmtp_hdr_delivery_address setting to specify whether
to include email address in Delivered-To: and Received: headers.
+ Added initial version of full text search library, which includes
language-specific text normalization and filtering. This is still
in development, but it's already possible to use for testing with
fts-lucene and fts-solr.
+ lda, lmtp: deliver_log_format can now include %{delivery_time},
which expands to how many milliseconds it took to deliver the mail.
With LMTP %{session_time} also expands to how many milliseconds the
LMTP session took, not including the delivery time.
+ lmtp proxy: Mail delivery logging includes timing information.
+ imap: Most IMAP commands now include in the tagged reply how many
milliseconds it took to run the command (not counting the time spent
on waiting for the IMAP client to read/write data).
+ director: Implemented director_proxy_maybe passdb extra field to
be able to run director and backend in the same Dovecot instance.
(LMTP doesn't support mixed proxy/non-proxy destinations currently.)
+ doveadm: Added -F <file> parameter to read a list of users from the
given file and run the command for all the users. This is similar to
-A parameter reading the list of users from userdb lookup.
- auth: If auth_master_user_separator was set, auth process could be
crashed by trying to log in with empty master username.
- imap-login, pop3-login: Fixed crash on handshake failures with new
OpenSSL versions (v1.0.2) when SSLv3 was disabled.
- auth: If one passdb fails allow_nets check, it shouldn't have failed
all the other passdb checks later on.
- imap: Server METADATA couldn't be accessed
- imapc: Fixed \Muted label handling in gmail-migration.
- imapc: Various bugfixes and improvements.
- Trash plugin fixes by Alexei Gradinari
- mbox: Fixed crash/corruption in some situations when the first mail
was expunged.Compiled successfully on FreeBSD-{8.4|9.3|10.1}.
Running also on those and watching|listening for any complains.
Although on one of the 8.4, I have seen so many IMAP process started. I hope this is some uses connecting via IMAP. I have never seen that before.
root@gw:/usr/home/wash/Tools/Dovecot/2.2/dovecot-2.2.17.rc1 # ps ax | grep dove 47491 ?? I 0:00.00 supervise dovecot 47498 ?? S 0:00.13 /opt/dovecot2.2/sbin/dovecot -F -c /opt/dovecot2.2/etc/dovecot/dovecot.conf 47501 ?? I 0:00.01 /usr/local/bin/multilog t s1000000 n20 /var/log/dovecot/ 47509 ?? S 0:00.02 dovecot/anvil 47510 ?? S 0:00.02 dovecot/log 47570 ?? I 0:00.02 dovecot/imap 47571 ?? I 0:00.01 dovecot/imap 47572 ?? S 0:00.01 dovecot/imap 47573 ?? I 0:00.02 dovecot/imap 47574 ?? S 0:00.02 dovecot/imap 47575 ?? S 0:00.01 dovecot/imap 47576 ?? I 0:00.02 dovecot/imap 47577 ?? I 0:00.02 dovecot/imap 47578 ?? S 0:00.01 dovecot/imap 47579 ?? S 0:00.01 dovecot/imap 47580 ?? S 0:00.02 dovecot/imap 47581 ?? S 0:00.01 dovecot/imap 47584 ?? I 0:00.01 dovecot/imap 47587 ?? I 0:00.01 dovecot/imap 47591 ?? I 0:00.02 dovecot/imap 47594 ?? I 0:00.02 dovecot/imap 47597 ?? S 0:00.02 dovecot/imap 47598 ?? S 0:00.02 dovecot/imap 47600 ?? S 0:00.01 dovecot/imap 47601 ?? S 0:00.02 dovecot/imap 47848 ?? S 0:00.04 dovecot/config 47849 ?? S 0:00.03 dovecot/auth 47879 ?? I 0:00.00 dovecot/ssl-params 47908 ?? I 0:00.01 dovecot/pop3-login 48012 ?? I 0:00.01 dovecot/pop3-login 48057 ?? S 0:00.03 dovecot/auth -w 48087 0 S+ 0:00.00 grep dove root@gw:/usr/home/wash/Tools/Dovecot/2.2/dovecot-2.2.17.rc1 #
On 9 May 2015 at 20:41, Timo Sirainen tss@iki.fi wrote:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz.sig
Quite a lot of changes, so here's a release candidate first. If no problems are reported I'll make v2.2.17 release early next week. Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16.
* Dovecot no longer checks or warns if a mountpoint is removed.This was causing more trouble than it was worth. Make sure that all the mountpoints that Dovecot accesses aren't writable by mail processes when they're unmounted. * dict server wasn't properly escaping/unescaping data. Fixing this broke backwards compatibility with data that contains line feeds. This hopefully affects only very few installations. If you're using dict to save multiline data (Sieve scripts to SQL), you may be affected. * imap: SPECIAL-USE capability is no longer advertised if there are no special_use flags specified for any mailboxes.
+ lmtp: Added lmtp_hdr_delivery_address setting to specify whether to include email address in Delivered-To: and Received: headers. + Added initial version of full text search library, which includes language-specific text normalization and filtering. This is still in development, but it's already possible to use for testing with fts-lucene and fts-solr. + lda, lmtp: deliver_log_format can now include %{delivery_time}, which expands to how many milliseconds it took to deliver themail. With LMTP %{session_time} also expands to how many milliseconds the LMTP session took, not including the delivery time. + lmtp proxy: Mail delivery logging includes timing information. + imap: Most IMAP commands now include in the tagged reply how many milliseconds it took to run the command (not counting the time spent on waiting for the IMAP client to read/write data). + director: Implemented director_proxy_maybe passdb extra field to be able to run director and backend in the same Dovecot instance. (LMTP doesn't support mixed proxy/non-proxy destinations currently.) + doveadm: Added -F <file> parameter to read a list of users from the given file and run the command for all the users. This is similar to -A parameter reading the list of users from userdb lookup. - auth: If auth_master_user_separator was set, auth process could be crashed by trying to log in with empty master username. - imap-login, pop3-login: Fixed crash on handshake failures with new OpenSSL versions (v1.0.2) when SSLv3 was disabled. - auth: If one passdb fails allow_nets check, it shouldn't have failed all the other passdb checks later on. - imap: Server METADATA couldn't be accessed - imapc: Fixed \Muted label handling in gmail-migration. - imapc: Various bugfixes and improvements. - Trash plugin fixes by Alexei Gradinari - mbox: Fixed crash/corruption in some situations when the first mail was expunged.
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 "I can't hear you -- I'm using the scrambler."
Hi —
Timo Sirainen tss@iki.fi wrote:
Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16.
2.2.17.rc1 will compile on FBSD 10.1 only, if bash has been installed (see 554773D2.9060609@dovecot.fi from Teemu Huovila). This is from my point of view regarding portability issues "suboptimal". Both, word-break-data.sh and word-boundary-data.sh have comments about preferable use awk, perl, or … instead, IIRC. Teemu himself already mentioned in 554773D2.9060609@dovecot.fi that this is suboptimal, and that he is willing to solve that issue, thus, I am extremely far from blaming anyone involved in making this great software available. But, maybe it's worth solving this issue for those non-linux systems without bash installed per default before releasing dovecot v2.2.17, finally.
Besides that: Thanks for all of you involved making dovecot available!
Regards, Michael
On Sat, 9 May 2015 22:17:27 +0200, Michael Grimm stated:
Hi —
Timo Sirainen tss@iki.fi wrote:
Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16.
2.2.17.rc1 will compile on FBSD 10.1 only, if bash has been installed (see 554773D2.9060609@dovecot.fi from Teemu Huovila). This is from my point of view regarding portability issues "suboptimal". Both, word-break-data.sh and word-boundary-data.sh have comments about preferable use awk, perl, or … instead, IIRC. Teemu himself already mentioned in 554773D2.9060609@dovecot.fi that this is suboptimal, and that he is willing to solve that issue, thus, I am extremely far from blaming anyone involved in making this great software available. But, maybe it's worth solving this issue for those non-linux systems without bash installed per default before releasing dovecot v2.2.17, finally.
Besides that: Thanks for all of you involved making dovecot available!
Regards, Michael
For what its worth, I run FreeBSD 10.1 and I have Bash. Personally, I could not work without it. It is not a big deal to install it on FreeBSD. My question is does it have to be the default shell in dovecot v2.2.17 or just installed?
-- Jerry
On May 9, 2015, at 19:13 , Jerry jerry@seibercom.net wrote:
2.2.17.rc1 will compile on FBSD 10.1 only, if bash has been installed (see 554773D2.9060609@dovecot.fi from Teemu Huovila).[...]. But, maybe it's worth solving this issue for those non-linux systems without bash installed per default before releasing dovecot v2.2.17, finally.
Besides that: Thanks for all of you involved making dovecot available!
For what its worth, I run FreeBSD 10.1 and I have Bash. Personally, I could not work without it. It is not a big deal to install it on FreeBSD. My question is does it have to be the default shell in dovecot v2.2.17 or just installed?
I run FreeBSD 9, 10, and 10.1. And I don't have bash installed except on the one system I use to build lots of ports. Because, there are other ports that also presume the availability of bash, and therefore require it. I'd much rather be able to build dovecot without bash, because I don't have bash installed on my FreeBSD 10.1 mail server, where I run dovecot.
I'd also be willing to help by porting any of the scripts that require bash to something else that is available in the base installation of FreeBSD as well as the variety of Linux distributions.
I agree that it's not hard to install the bash port. But if you don't plan to use it for anything else, it's better not to have to.
- ChrisChris Ross cross+dovecot@distal.com wrote:
I'd much rather be able to build dovecot without bash, because I don't have bash installed on my FreeBSD 10.1 mail server, where I run dovecot.
I'd also be willing to help by porting any of the scripts that require bash to something else that is available in the base installation of FreeBSD as well as the variety of Linux distributions.
I started to learn awk today ;-) See an awk script attached that simulates "src/lib-fts/word-break-data.sh < src/lib-fts/PropList.txt > src/lib-fts/word-break-data.c". This script is pretty preliminary, although it does what it is supposed to do, if I am not mistaken. Please, give it a try.
It lacks a proper way of referencing stolen code in function mystrtonum (never wrote open source code); I did steal that code in order to stick to awk instead of gawk, which might be a portability issue as well. This script lacks any error handling, as well. Thus, consider it pre-alpha.
I do not have the time to continue for the coming days. Thus, anyone feeling to jump in: just do it ;-)
Regards, Michael
Michael Grimm trashcan@odo.in-berlin.de kirjoitti 9.5.2015 kello 23.17:
Hi —
Timo Sirainen tss@iki.fi wrote:
Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16.
2.2.17.rc1 will compile on FBSD 10.1 only, if bash has been installed
Dovecot hg requires bash currently, but the rc1 tarball should compile even without bash because the generated files are included in the tarball. Or should at least..
Timo Sirainen tss@iki.fi wrote:
Michael Grimm trashcan@odo.in-berlin.de kirjoitti 9.5.2015 kello 23.17:
Timo Sirainen tss@iki.fi wrote:
Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16.
2.2.17.rc1 will compile on FBSD 10.1 only, if bash has been installed
Dovecot hg requires bash currently, but the rc1 tarball should compile even without bash because the generated files are included in the tarball. Or should at least..
Yes, I know, that's what Teemu already pointed out. And yes, I can confirm, that http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz compiles without an installed bash under FBSD 10.1-STABLE
I can live with having to install bash in order to be able to follow hg development (what I do in a test jail), but it's from my point of view a little bit of "overkill", if one needs bash just for creating two c-files from two bash-scripts and their corresponding txt-files once in a while.
Again, please don't get me wrong, I am not complaining! I can live with the situation as is. I only wanted to report that issue because you asked for feedback from non-linux users ;-)
Regards, Michael
On Sun, 10 May 2015 10:27:25 +0300, Timo Sirainen stated:
Dovecot hg requires bash currently, but the rc1 tarball should compile even without bash because the generated files are included in the tarball. Or should at least..
Timo, IMHO, I would not worry too much about placating the FreeBSD crowd. They love to pretend that their way of doing things is superior to everyone else. I say this even though I use FreeBSD myself.
The FreeBSD port maintain for Dovecot can make bash a dependency or he can attempt to rewrite the Dovecot scripts himself. You work too hard improving Dovecot and should not have to tweak it for every OS out there. Believe me, I know from experience that it will come back to haunt you if you try to please everyone.
Again, thanks for all of your hard work on this software.
-- Jerry
I'm now seeing these crashes in the indexing code:
May 10 12:22:32 tornado.reub.net dovecot: master: Dovecot v2.2.17.rc1 (3903badc4ee0+) starting up for imap, lmtp, sieve ... May 10 13:07:50 tornado.reub.net dovecot: indexer: Error: Indexer worker disconnected, discarding 1 requests for reuben May 10 13:07:50 tornado.reub.net dovecot: indexer-worker(reuben): Fatal: master: service(indexer-worker): child 18532 killed with signal 11 (core not dumped)
Every time a delivery happens or mail is moved the indexer crashes out. It doesn't seem to have any visible end user impact though.
How do I go about debugging this further? How do I go about getting a backtrace on what is essentially a child process that doesn't produce a core? (Note that my installation normally does produce cores when it crashes, so there's something different about these indexer processes..)
Reuben
On 10/05/2015 3:41 AM, Timo Sirainen wrote:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz.sig
On 10 May 2015, at 06:19, Reuben Farrelly reuben-dovecot@reub.net wrote:
I'm now seeing these crashes in the indexing code:
May 10 12:22:32 tornado.reub.net dovecot: master: Dovecot v2.2.17.rc1 (3903badc4ee0+) starting up for imap, lmtp, sieve ... May 10 13:07:50 tornado.reub.net dovecot: indexer: Error: Indexer worker disconnected, discarding 1 requests for reuben May 10 13:07:50 tornado.reub.net dovecot: indexer-worker(reuben): Fatal: master: service(indexer-worker): child 18532 killed with signal 11 (core not dumped)
Every time a delivery happens or mail is moved the indexer crashes out. It doesn't seem to have any visible end user impact though.
What's your current doveconf -n output? I did a quick test with fts-lucene and it was working.
How do I go about debugging this further? How do I go about getting a backtrace on what is essentially a child process that doesn't produce a core? (Note that my installation normally does produce cores when it crashes, so there's something different about these indexer processes..)
One thing that often works is:
mkdir /var/core; chmod 0777 /var/core echo "/var/core/%p" > /proc/sys/kernel/core_pattern
If that doesn't help, you can also try: sysctl -w fs.suid_dumpable=2 (and disable afterwards)
Also if you can get indexer-worker to start without crashing (new mail delivery works?) you can attach gdb into it and wait for it to crash (gdb -p <indexer-worker pid>, cont).
On 10/05/2015 5:47 PM, Timo Sirainen wrote:
How do I go about debugging this further? How do I go about getting a backtrace on what is essentially a child process that doesn't produce a core? (Note that my installation normally does produce cores when it crashes, so there's something different about these indexer processes..)
One thing that often works is:
mkdir /var/core; chmod 0777 /var/core echo "/var/core/%p" > /proc/sys/kernel/core_pattern
If that doesn't help, you can also try: sysctl -w fs.suid_dumpable=2 (and disable afterwards)
Also if you can get indexer-worker to start without crashing (new mail delivery works?) you can attach gdb into it and wait for it to crash (gdb -p <indexer-worker pid>, cont).
Ok we have some (well, lots actually) of coredumps now:
tornado core # gdb /usr/libexec/dovecot/indexer-worker 26697 GNU gdb (Gentoo 7.9 vanilla) 7.9 Copyright (C) 2015 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: http://bugs.gentoo.org/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/libexec/dovecot/indexer-worker...done. Attaching to program: /usr/libexec/dovecot/indexer-worker, process 26697 ptrace: No such process. [New LWP 26697]
warning: Could not load shared library symbols for linux-vdso.so.1. Do you need "set solib-search-path" or "set sysroot"? [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `dovecot/indexer-worker'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007f4834fcc222 in fts_user_get_data_lang (user=0x16c7900) at fts-user.c:302 302 fts-user.c: No such file or directory. (gdb) info locals hdr = 0x1701f08 key = {uid = 8553, type = FTS_BACKEND_BUILD_KEY_HDR, part = 0x1696860, hdr_name = 0x1702120 "To", body_content_type = 0x0, body_content_disposition = 0x0}
(gdb) bt full
#0 0x00007f4834fcc222 in fts_user_get_data_lang (user=0x16c7900)
at fts-user.c:302
fuser = 0x0
lang = 0x0
error = 0xf
Thanks, Reuben
On 10 May 2015, at 11:47, Reuben Farrelly reuben-dovecot@reub.net wrote:
Ok we have some (well, lots actually) of coredumps now:
Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007f4834fcc222 in fts_user_get_data_lang (user=0x16c7900) at fts-user.c:302
Thanks, this should fix it: http://hg.dovecot.org/dovecot-2.2/rev/fddd3dbdf987
On 10/05/2015 7:06 PM, Timo Sirainen wrote:
On 10 May 2015, at 11:47, Reuben Farrelly reuben-dovecot@reub.net wrote:
Ok we have some (well, lots actually) of coredumps now:
Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007f4834fcc222 in fts_user_get_data_lang (user=0x16c7900) at fts-user.c:302
Thanks, this should fix it: http://hg.dovecot.org/dovecot-2.2/rev/fddd3dbdf987
Confirmed as fixed - all good now.
Thanks! Reuben
On May 9, 2015, at 10:41 AM, Timo Sirainen tss@iki.fi wrote:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz.sig
Quite a lot of changes, so here's a release candidate first. If no problems are reported I'll make v2.2.17 release early next week. Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16.
Between dovecot-2.2.17rc1 and 69630e6048fd a build issue on OS X/MacPorts appears to be gone.
Regards, Bradley Giesbrecht (pixilla)
- Timo Sirainen tss@iki.fi 2015.05.09 19:41:
- lda, lmtp: deliver_log_format can now include %{delivery_time}, which expands to how many milliseconds it took to deliver the mail. With LMTP %{session_time} also expands to how many milliseconds the LMTP session took, not including the delivery time.
This could use some nicer defaults similar to Postfix logging e.g.
deliver_log_format = msgid=%m, time=%{delivery_time}ms, status=%$
May 11 18:25:29 nihlus dovecot: lmtp(tlx@leuxner.net): CupIHNrXUFXUTAAAgUOSbA: sieve: msgid=5550D7CB.70909@comstyle.com, time=133ms, status=stored mail into mailbox 'Public/Mailing-Lists/Dovecot'
On 5/9/2015 10:41 AM, Timo Sirainen wrote:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz.sig
Quite a lot of changes, so here's a release candidate first. If no problems are reported I'll make v2.2.17 release early next week. Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16.
Hi Timo,
I tried building on Solaris 10 and get an error:
libtool: link: /opt/SUNWspro/bin/cc -G -z defs -h libfs_compress.so -o .libs/libfs_compress.so .libs/fs-compress.o -R/home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-compression/.libs -R/opt/csw/lib/dovecot -L/opt/csw/lib ../../lib-compression/.libs/libdovecot-compression.so -L/home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs -lrt -lnsl -lsocket -lsendfile -lc -m32 -xarch=pentium_pro -m32 -xarch=pentium_pro Undefined first referenced symbol in file iostream_temp_finish .libs/fs-compress.o (symbol belongs to implicit dependency /home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs/libdovecot.so.0) fs_write_stream .libs/fs-compress.o (symbol belongs to implicit dependency /home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs/libdovecot.so.0) fs_set_error .libs/fs-compress.o (symbol belongs to implicit dependency /home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs/libdovecot.so.0) o_stream_destroy .libs/fs-compress.o (symbol belongs to implicit dependency /home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs/libdovecot.so.0) fs_write_stream_finish .libs/fs-compress.o (symbol belongs to implicit dependency /home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs/libdovecot.so.0) fs_write_via_stream .libs/fs-compress.o (symbol belongs to implicit dependency /home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs/libdovecot.so.0)
participants (10)
-
Bradley Giesbrecht
-
Chris Ross
-
Jake Goerzen
-
Jerry
-
Michael Grimm
-
Odhiambo Washington
-
Reuben Farrelly
-
Santiago Vila
-
Thomas Leuxner
-
Timo Sirainen