On Mon, Dec 18, 2017 at 7:23 AM, Timo Sirainen <tss@iki.fi> wrote:
https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig
It's finally time for v2.3 release branch! There are several new and exciting features in it. I'm especially happy about the new logging and statistics code, which will allow us to generate statistics for just about everything. We didn't have time to implement everything we wanted for them yet, and there especially aren't all that many logging events yet that can be used for statistics. We'll implement those to v2.3.1, which might also mean that some of the APIs might still change in v2.3.1 if that's required.
We also have new lib-smtp server code, which was used to implement SMTP submission server and do a partial rewrite for LMTP server. Please test these before v2.3.0 to make sure we don't have any bad bugs left!
BTW. The v2.3.0 will most likely be signed with a new PGP key ED409DA1.
Some of the larger changes:
- Various setting changes, see https://wiki2.dovecot.org/Upgrading/2.3
- Logging rewrite started: Logging is now based on hierarchical events. This makes it possible to do various things, like: 1) giving consistent log prefixes, 2) enabling debug logging with finer granularity, 3) provide logs in more machine readable formats (e.g. json). Everything isn't finished yet, especially a lot of the old logging code still needs to be translated to the new way.
- Statistics rewrite started: Stats are now based on (log) events. It's possible to gather statistics about any event that is logged. See http://wiki2.dovecot.org/Statistics for details
- ssl_dh setting replaces the old generated ssl-parameters.dat
- IMAP: When BINARY FETCH finds a broken mails, send [PARSE] error instead of [UNKNOWNCTE]
- Linux: core dumping via PR_SET_DUMPABLE is no longer enabled by default due to potential security reasons (found by cPanel Security Team).
- Added support for SMTP submission proxy server, which includes support for BURL and CHUNKING extension.
- LMTP rewrite. Supports now CHUNKING extension and mixing of local/proxy recipients.
- auth: Support libsodium to add support for ARGON2I and ARGON2ID password schemes.
- auth: Support BLF-CRYPT password scheme in all platforms
- auth: Added LUA scripting support for passdb/userdb. See https://wiki2.dovecot.org/AuthDatabase/Lua
- Input streams are more reliable now when there are errors or when the maximum buffer size is reached. Previously in some situations this could have caused Dovecot to try to read already freed memory.
- Output streams weren't previously handling failures when writing a trailer at the end of the stream. This mainly affected encrypt and zlib compress ostreams, which could have silently written truncated files if the last write happened to fail (which shouldn't normally have ever happened).
- virtual plugin: Fixed panic when fetching mails from virtual mailboxes with IMAP BINARY extension.
- Many other smaller fixes
No issue compilng (and very very excited about this release, esp the Lua code, which is already super useful).
I did have this one issue so far with the RC. I was previously using a git checkout of ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015 with some of the Lua patches attached. That was working just fine (except for one thing I'll mention below). I rolled the RC and got this (and I was actually testing for the issue I had with ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015):
# doveadm -D acl set -u test1-shared@test.com INBOX user=test1@test.com read list Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib01_acl_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib02_lazy_expunge_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_virtual_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_lucene_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) doveadm(test1-shared@test.com)<37433><>: Debug: auth USER input: test1-shared@test.com uid=200 home=/m/mail//7/1/fe/test.com/test1-shared gid=200 quota=maildir:User quota mail_location=maildir:/m/mail//7/1/fe/ test.com/test1-shared/maildir doveadm(test1-shared@test.com)<37433><>: Debug: Added userdb setting: mail_location=maildir:/m/mail//7/1/fe/test.com/test1-shared/maildir doveadm(test1-shared@test.com)<37433><>: Debug: Added userdb setting: plugin/quota=maildir:User quota doveadm(test1-shared@test.com): Debug: Effective uid=200, gid=200, home=/m/mail//7/1/fe/test.com/test1-shared doveadm(test1-shared@test.com): Panic: file buffer.c: line 97 (buffer_check_limits): assertion failed: (buf->used <= buf->alloc) doveadm(test1-shared@test.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xc5d7a) [0x6ba69ea77d7a] -> /usr/lib/dovecot/libdovecot.so.0(+0xc5de9) [0x6ba69ea77de9] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x6ba69e9ea521] -> /usr/lib/dovecot/libdovecot.so.0(buffer_write+0x160) [0x6ba69ea72210] -> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so(+0x43fe) [0x6ba6997c33fe] -> /usr/lib/dovecot/libdovecot-storage.so.0(hook_mail_user_created+0x20a) [0x6ba69ed72a2a] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_user_init+0x220) [0x6ba69ed78870] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_storage_service_next_with_session_suffix+0x494) [0x6ba69ed76314] -> doveadm(+0x296ff) [0x5bde3fb56ff] -> doveadm(+0x2a3ab) [0x5bde3fb63ab] -> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x21d) [0x5bde3fb71bd] -> doveadm(doveadm_cmd_run_ver2+0x575) [0x5bde3fc6c15] -> doveadm(doveadm_cmd_try_run_ver2+0x37) [0x5bde3fc6c67] -> doveadm(main+0x1d2) [0x5bde3fa5f52] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x6ba69e609830] -> doveadm(_start+0x29) [0x5bde3fa6329] Aborted
This same command was working with ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015 earlier today -- this is run from command line history. The issue I had noticed with ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015 (and was going to ask about it today anyway) is that when I would use doveadm acl to add an ACL with *just* 'read', dovecot would create the ACL file correctly, but using a file-based acl_shared_dict, it would create a 0 byte acl_shared_dict file. However using anything else for the ACL, e.g. 'read list', it would create the acl_shared_dict file with what you'd expect. I don't know if that's expected behavior or not (but couldn't find anything on google to say so). So just to sum up: With just 'read', it'd create an empty acl_shared_dict file; with 'read list' or anything else, it'd create a working, non-empty acl_shared_dict file.
I was going to see if it was different in the RC but instead, there's just the above error.
This is just a test rig, so I can try out whatever would be useful to debugging. Happy to post doveconf -n, if helpful. The only config changes from ecfca41e9d998a0f21ce7a4bce1dc78c58c3e015 to RC1 that I made were just the things that needed to be done to let dovecot start up (removing some of the stats stuff, getting rid of ssl_protocols, etc).