v2.2.19 release candidate released
http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz.sig
A lot of changes since v2.2.18, so here's a release candidate first. If no bugs are reported, I'm planning on making the final release sometimes this week. The most interesting new features here are the imap-hibernate process, quota count backend and director/proxy improvements.
- "doveadm director flush" command has a changed meaning now: It safely moves users to their wanted backends, instead of simply forgetting the mapping entirely and leaving the existing connections untouched. Use -F parameter to get the original unsafe behavior.
- Added support for imap-hiberanate processes.
- Optimized tracking mailboxes' vsizes (= sum of all messages' size). If mailbox_list_index=yes, it's also stored in there. This makes it very efficient to look up vsizes for all mailboxes.
- Added a quota "count" backend, which uses the mailbox vsizes to get the current quota usage. It requires using the new quota_vsizes=yes setting, which tracks the messages' "virtual sizes" rather than "physical sizes". Their distiction is minor and mostly irrelevant nowadays (if mail sizes should be counted with LF or CRLF newlines).
- "doveadm director up/down" commands added. The monitoring script should be using these commands instead of changing the vhost count. This allows admin to manually disable a server by changing the vhost count to 0 without the monitoring script changing it back.
- Added support for HAProxy protocol: http://wiki2.dovecot.org/HAProxy
- Added push-notification plugin framework, which can be used to easily implement push notifications to various backends. Implemented "ox" backend for notifying Open-Xchange via HTTP/json.
- imap_logout_format supports more variables now, e.g. number of deleted messages.
- pop3: Added pop3_delete_type setting (related to pop3_deleted_flag).
- plugin { fts_enforced=yes } setting now fails body searches unless it can be done via the full text search engine.
- Added %{passdb:*} and %{userdb:*} variables to various places
- auth: Added ":protected" suffix for passdb and userdb fields. If used, the field doesn't overwrite an existing field.
- IMAP/POP3 proxy: If a backend server dies, avoid client reconnection spikes by slowly disconnecting clients over time. This is enabled by setting login_proxy_max_disconnect_delay=secs passdb extra field.
- imap: Added new read-only METADATA entries: /private/specialuse, /shared/comment, /shared/admin
- imap: If client disconnects in the middle of a command, log how long the command had been running.
- mdbox: Rebuilding could have caused message's reference count to overflow the 16bit number in some situations, causing problems when trying to expunge the duplicates.
- Various search fixes (fts, solr, tika, lib-charset, indexer)
- Various virtual plugin fixes
- Various fixes and optimizations to dsync, imapc and pop3-migration
- imap: Various RFC compliancy and crash fixes to NOTIFY
W dniu 23.09.2015 o 15:30, Timo Sirainen pisze:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz.sig
A lot of changes since v2.2.18, so here's a release candidate first. If no bugs are reported, I'm planning on making the final release sometimes this week. The most interesting new features here are the imap-hibernate process, quota count backend and director/proxy improvements.
Hi! I'm seeing: Sep 23 17:23:43 mail dovecot: dict: Panic: file driver-pgsql.c: line 463 (do_query): assertion failed: (SQL_DB_IS_READY(&db->api)) Sep 23 17:23:43 mail dovecot: dict: Fatal: master: service(dict): child 23430 killed with signal 6 (core dumped) Sep 23 17:23:43 mail dovecot: dict: Panic: file driver-pgsql.c: line 463 (do_query): assertion failed: (SQL_DB_IS_READY(&db->api)) Sep 23 17:23:43 mail dovecot: dict: Fatal: master: service(dict): child 23321 killed with signal 6 (core dumped)
# gdb -q /usr/libexec/dovecot/dict 'core-1443021920-97-6-!usr!libexec!dovecot!dict-23764' Reading symbols from /usr/libexec/dovecot/dict...Reading symbols from /usr/lib64/debug//usr/libexec/dovecot/dict.debug...done. done. [New LWP 23764]
warning: Could not load shared library symbols for linux-vdso.so.1. Do you need "set solib-search-path" or "set sysroot"? [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `dovecot/dict'. Program terminated with signal SIGABRT, Aborted. #0 0x00007f2623e6d096 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55 55 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt #0 0x00007f2623e6d096 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55 #1 0x00007f2623e6e680 in __GI_abort () at abort.c:89 #2 0x00007f26244918dd in default_fatal_finish (type=LOG_TYPE_PANIC, status=status@entry=0) at failures.c:201 #3 0x00007f2624491a31 in i_internal_fatal_handler (ctx=0x7fffb4a422f0, format=<optimized out>, args=<optimized out>) at failures.c:670 #4 0x00007f262442ca1e in i_panic (format=format@entry=0x7f262492c5c8 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 #5 0x00007f26249240b6 in sql_dict_transaction_has_nonexistent (ctx=0x7f2625e7bca0) at dict-sql.c:720 #6 0x00007f26249240f3 in sql_dict_transaction_commit_callback (error=<optimized out>, ctx=0x7f2625e7bca0) at dict-sql.c:734 #7 0x00007f262492844f in driver_sqlpool_commit_callback (error=<optimized out>, ctx=0x7f2625e726a0) at driver-sqlpool.c:720 #8 0x00007f262492a60f in transaction_commit_callback (result=0x7f2625e7b6a0, ctx=0x7f2625e66dd0) at driver-pgsql.c:880 #9 0x00007f2624929e8e in result_finish (result=0x7f2625e7b6a0) at driver-pgsql.c:388 #10 0x00007f26244a8c51 in io_loop_call_io (io=0x7f2625e66c30) at ioloop.c:559 #11 0x00007f26244aa40d in io_loop_handler_run_internal (ioloop=ioloop@entry=0x7f2625e60710) at ioloop-epoll.c:220 #12 0x00007f26244a8d26 in io_loop_handler_run (ioloop=ioloop@entry=0x7f2625e60710) at ioloop.c:607 #13 0x00007f26244a8f06 in io_loop_run (ioloop=0x7f2625e60710) at ioloop.c:583 #14 0x00007f2624433593 in master_service_run (service=0x7f2625e605b0, callback=<optimized out>) at master-service.c:640 #15 0x00007f2624921907 in main (argc=1, argv=0x7f2625e60390) at main.c:105 (gdb)
Marcin
On 23 Sep 2015, at 18:26, Marcin Mirosław marcin@mejor.pl wrote:
W dniu 23.09.2015 o 15:30, Timo Sirainen pisze:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz.sig
A lot of changes since v2.2.18, so here's a release candidate first. If no bugs are reported, I'm planning on making the final release sometimes this week. The most interesting new features here are the imap-hibernate process, quota count backend and director/proxy improvements.
Hi! I'm seeing: Sep 23 17:23:43 mail dovecot: dict: Panic: file driver-pgsql.c: line 463 (do_query): assertion failed: (SQL_DB_IS_READY(&db->api)) Sep 23 17:23:43 mail dovecot: dict: Fatal: master: service(dict): child 23430 killed with signal 6 (core dumped) Sep 23 17:23:43 mail dovecot: dict: Panic: file driver-pgsql.c: line 463 (do_query): assertion failed: (SQL_DB_IS_READY(&db->api)) Sep 23 17:23:43 mail dovecot: dict: Fatal: master: service(dict): child 23321 killed with signal 6 (core dumped)
It's because dict-sql started actually doing an asynchronous commits now, which were broken with pgsql. These should fix it:
http://hg.dovecot.org/dovecot-2.2/rev/59e4fcaa0f76 http://hg.dovecot.org/dovecot-2.2/rev/9ceeb1a5c492 http://hg.dovecot.org/dovecot-2.2/rev/beb3b8496b7d
I think this also means that you could have only a couple of dict processes, since all the queries should now be done asynchronously. So if you had for example:
quota = dict:User quota::proxy::sqlquota
you could now have:
quota = dict:User quota::proxy:dict-async:sqlquota
You'd also probably need to change the service dict-async { unix_listener dict-async { permissions } }.
Then again, it might not work. I haven't really tested this case. :)
W dniu 24.09.2015 o 00:07, Timo Sirainen pisze:
On 23 Sep 2015, at 18:26, Marcin Mirosław marcin@mejor.pl wrote:
W dniu 23.09.2015 o 15:30, Timo Sirainen pisze:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz.sig
A lot of changes since v2.2.18, so here's a release candidate first. If no bugs are reported, I'm planning on making the final release sometimes this week. The most interesting new features here are the imap-hibernate process, quota count backend and director/proxy improvements.
Hi! I'm seeing: Sep 23 17:23:43 mail dovecot: dict: Panic: file driver-pgsql.c: line 463 (do_query): assertion failed: (SQL_DB_IS_READY(&db->api)) Sep 23 17:23:43 mail dovecot: dict: Fatal: master: service(dict): child 23430 killed with signal 6 (core dumped) Sep 23 17:23:43 mail dovecot: dict: Panic: file driver-pgsql.c: line 463 (do_query): assertion failed: (SQL_DB_IS_READY(&db->api)) Sep 23 17:23:43 mail dovecot: dict: Fatal: master: service(dict): child 23321 killed with signal 6 (core dumped)
It's because dict-sql started actually doing an asynchronous commits now, which were broken with pgsql. These should fix it:
http://hg.dovecot.org/dovecot-2.2/rev/59e4fcaa0f76 http://hg.dovecot.org/dovecot-2.2/rev/9ceeb1a5c492 http://hg.dovecot.org/dovecot-2.2/rev/beb3b8496b7d
I think this also means that you could have only a couple of dict processes, since all the queries should now be done asynchronously. So if you had for example:
quota = dict:User quota::proxy::sqlquota
you could now have:
quota = dict:User quota::proxy:dict-async:sqlquota
You'd also probably need to change the service dict-async { unix_listener dict-async { permissions } }.
Then again, it might not work. I haven't really tested this case. :)
I didn't try to use dict-async yet. Those three patches solves problem with throwing core by dovecot-dict.
Thank you, Marcin
Hello;
Some issue with virtual storage and fts-lucene:
Sep 24 08:12:16 imap(jbv00@x jbv00@d1.dc.xandmail.comxx): Panic: file virtual-storage.c: line 370 (virtual_backend_box_close): assertion failed: (mbox->backends_open_count > 0) Sep 24 08:12:16 imap(jbv00@x jbv00@d1.dc.xandmail.comxx): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x7fb4a) [0x7fa31284fb4a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x7fbb6) [0x7fa31284fbb6] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fa3127f967f] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x9324) [0x7fa3111ba324] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x93d5) [0x7fa3111ba3d5] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x9429) [0x7fa3111ba429] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a) [0x7fa312b0a8ea] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_free+0x13) [0x7fa312b0c453] -> dovecot/imap(cmd_logout+0x40) [0x4113c0] -> dovecot/imap(command_exec+0x75) [0x4194f5] -> dovecot/imap() [0x417bb0] -> dovecot/imap() [0x417c46] -> dovecot/imap(client_handle_input+0x11d) [0x417f3d] -> dovecot/imap(client_input+0x6f) [0x4182bf] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7fa312861ecb] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xb7) [0x7fa312863457] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7fa312861f75] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fa312862118] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fa3127fe873] -> dovecot/imap(main+0x2cc) [0x4249ac] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fa31245ad5d] -> dovecot/imap() [0x40bf99] Sep 24 08:12:16 imap(jbv00@x jbv00@d1.dc.xandmail.comxx): Fatal: master: service(imap): child 16321 killed with signal 6 (core dumps disabled)
I think thys was introduced by http://hg.dovecot.org/dovecot-2.2/rev/251747c3afe5.
(gdb) r -u jbv00@xxx Starting program: /usr/local/libexec/dovecot/imap -u jbv00@xxx [Thread debugging using libthread_db enabled] process 16340 is executing new program: /usr/local/bin/doveconf Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.166.el6_7.1.x86_64 [Thread debugging using libthread_db enabled] process 16340 is executing new program: /usr/local/libexec/dovecot/imap Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.166.el6_7.1.x86_64 [Thread debugging using libthread_db enabled]
- PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SEARCH=FUZZY SPECIAL-USE QUOTA] Logged in as jbv00@xxx l select virtual.all
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent $Forwarded unknown-1)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent $Forwarded unknown-1 \*)] Flags permitted.
- 16480 EXISTS
- 0 RECENT
- OK [UNSEEN 626] First unseen.
- OK [UIDVALIDITY 1441363451] UIDs valid
- OK [UIDNEXT 16481] Predicted next UID
- OK [HIGHESTMODSEQ 1] Highest l OK [READ-WRITE] Select completed (0.000 + 0.000 secs). s search text test
- SEARCH 520 606 697 896 898 899 901 2118 2166 2871 2873 2878 2884 2885 2897 2928 2932 2935 2942 6286 7080 7082 7139 7339 7852 7866 7886 8433 8698 9373 9552 9578 9605 9609 9731 10053 10250 10320 10322 10323 10341 10493 10618 10887 11065 11069 11147 11169 11292 11297 11298 11351 11991 12197 12231 12428 12429 13889 15214 16464 s OK Search completed (0.393 + 0.000 secs). l logout imap(jbv00@xxx): Panic: file virtual-storage.c: line 370 (virtual_backend_box_close): assertion failed: (mbox->backends_open_count >
- imap(jbv00@xxx): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x7fb4a) [0x7ffff783bb4a] -> /usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7ffff783bc52] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x7ffff77e567f] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x9324) [0x7ffff61a6324] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x93d5) [0x7ffff61a63d5] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x9429) [0x7ffff61a6429] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a) [0x7ffff7af68ea] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_free+0x13) [0x7ffff7af8453] -> /usr/local/libexec/dovecot/imap(cmd_logout+0x40) [0x4113c0] -> /usr/local/libexec/dovecot/imap(command_exec+0x75) [0x4194f5] -> /usr/local/libexec/dovecot/imap() [0x417bb0] -> /usr/local/libexec/dovecot/imap() [0x417c46] -> /usr/local/libexec/dovecot/imap(client_handle_input+0x11d) [0x417f3d] -> /usr/local/libexec/dovecot/imap(client_input+0x6f) [0x4182bf] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7ffff784decb] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xb7) [0x7ffff784f457] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7ffff784df75] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7ffff784e118] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7ffff77ea873] -> /usr/local/libexec/dovecot/imap(main+0x2cc) [0x4249ac] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7ffff7446d5d] -> /usr/local/libexec/dovecot/imap() [0x40bf99]
Program received signal SIGABRT, Aborted. 0x00007ffff745a625 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.166.el6_7.1.x86_64 libgcc-4.4.7-16.el6.x86_64 libstdc++-4.4.7-16.el6.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) bt #0 0x00007ffff745a625 in raise () from /lib64/libc.so.6 #1 0x00007ffff745be05 in abort () from /lib64/libc.so.6 #2 0x00007ffff783bb40 in default_fatal_finish (type=<value optimized out>, status=0) at failures.c:201 #3 0x00007ffff783bc52 in default_fatal_handler (ctx=0x7fffffffe1d0, format=<value optimized out>, args=<value optimized out>) at failures.c:215 #4 0x00007ffff77e567f in i_panic (format=0x3fd4
) at failures.c:275 #5 0x00007ffff61a6324 in virtual_backend_box_close (mbox=0x664830, bbox=0x670838) at virtual-storage.c:370 #6 0x00007ffff61a63d5 in virtual_mailbox_close_internal (mbox=0x664830) at virtual-storage.c:401 #7 0x00007ffff61a6429 in virtual_mailbox_close (box=<value optimized out>) at virtual-storage.c:463 #8 0x00007ffff7af68ea in mailbox_close (box=0x664830) at mail-storage.c:1228 #9 0x00007ffff7af8453 in mailbox_free (_box=<value optimized out>) at mail-storage.c:1248 #10 0x00000000004113c0 in cmd_logout (cmd=0x65fb30) at cmd-logout.c:18 #11 0x00000000004194f5 in command_exec (cmd=0x65fb30) at imap-commands.c:169 #12 0x0000000000417bb0 in client_command_input (cmd=0x65fb30) at imap-client.c:869 #13 0x0000000000417c46 in client_command_input (cmd=0x65fb30) at imap-client.c:929 #14 0x0000000000417f3d in client_handle_next_command (client=0x65ef30) at imap-client.c:967 #15 client_handle_input (client=0x65ef30) at imap-client.c:979 #16 0x00000000004182bf in client_input (client=0x65ef30) at imap-client.c:1021 #17 0x00007ffff784decb in io_loop_call_io (io=0x65fa20) at ioloop.c:559 #18 0x00007ffff784f457 in io_loop_handler_run_internal (ioloop=<value optimized out>) at ioloop-epoll.c:220 #19 0x00007ffff784df75 in io_loop_handler_run (ioloop=0x63ec70) at ioloop.c:607 #20 0x00007ffff784e118 in io_loop_run (ioloop=0x63ec70) at ioloop.c:583 #21 0x00007ffff77ea873 in master_service_run (service=0x63eb10, callback=<value optimized out>) at master-service.c:640 #22 0x00000000004249ac in main (argc=3, argv=0x63e390) at main.c:442This can also be avoided by setting plugin/virtual_max_open_mailboxes=1000 (where 1000 is greater than the number of total folders of the mailbox).
/usr/local/sbin/dovecot -n # 2.2.19.rc1: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-504.el6.x86_64 x86_64 CentOS release 6.5 (Final) ext4 auth_debug = yes auth_mechanisms = plain cram-md5 apop auth_username_chars = auth_verbose = yes base_dir = /usr/local/var/run default_login_user = dovelog dict { path = /usr/local/var/run/dict-server quotadict = mysql:/usr/local/etc/dovecot/conf.d/dovecot-dict-quota.conf } disable_plaintext_auth = no first_valid_uid = 100 listen = * log_path = /home/backend/logs/dovecot/dovecot.log mail_gid = vmail mail_location = maildir:/home/backend/vmail/%d/%n mail_plugins = fts fts_lucene virtual mail_uid = vmail namespace { hidden = no inbox = yes location = mailbox Drafts { auto = create special_use = \Drafts } mailbox Sent { auto = create special_use = \Sent } mailbox Spam { auto = create special_use = \Junk } mailbox Trash { auto = create special_use = \Trash } prefix = INBOX. separator = . type = private } namespace { hidden = yes inbox = no list = no location = virtual:/usr/local/etc/dovecot/virtual:INDEX=~/virtual prefix = virtual. separator = . subscriptions = no } passdb { args = /usr/local/etc/dovecot/conf.d/dovecot-mysql.conf driver = sql } plugin { fts = lucene fts_decoder = decode2text fts_lucene = whitespace_chars=@. mail_log_events = delete undelete expunge copy mail_log_fields = uid box msgid size mail_log_group_events = no mail_log_max_lines_per_sec = 0 quota = dict:user::proxy::quotadict quota_rule = *:storage=100M quota_rule2 = *:messages=1000000 } protocols = imap pop3 service auth { unix_listener auth-client { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0600 user = vmail } user = dovecot } service decode2text { executable = script /usr/local/libexec/dovecot/decode2text.sh group = vmail unix_listener decode2text { group = vmail mode = 0666 user = vmail } user = vmail } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { user = dovelog } service pop3-login { user = dovelog } ssl_cert =
On Wed, Sep 23, 2015 at 3:30 PM, Timo Sirainen tss@iki.fi wrote:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz.sig
A lot of changes since v2.2.18, so here's a release candidate first. If no bugs are reported, I'm planning on making the final release sometimes this week. The most interesting new features here are the imap-hibernate process, quota count backend and director/proxy improvements.
- "doveadm director flush" command has a changed meaning now: It safely moves users to their wanted backends, instead of simply forgetting the mapping entirely and leaving the existing connections untouched. Use -F parameter to get the original unsafe behavior.
- Added support for imap-hiberanate processes.
- Optimized tracking mailboxes' vsizes (= sum of all messages' size). If mailbox_list_index=yes, it's also stored in there. This makes it very efficient to look up vsizes for all mailboxes.
- Added a quota "count" backend, which uses the mailbox vsizes to get the current quota usage. It requires using the new quota_vsizes=yes setting, which tracks the messages' "virtual sizes" rather than "physical sizes". Their distiction is minor and mostly irrelevant nowadays (if mail sizes should be counted with LF or CRLF newlines).
- "doveadm director up/down" commands added. The monitoring script should be using these commands instead of changing the vhost count. This allows admin to manually disable a server by changing the vhost count to 0 without the monitoring script changing it back.
- Added support for HAProxy protocol: http://wiki2.dovecot.org/HAProxy
- Added push-notification plugin framework, which can be used to easily implement push notifications to various backends. Implemented "ox" backend for notifying Open-Xchange via HTTP/json.
- imap_logout_format supports more variables now, e.g. number of deleted messages.
- pop3: Added pop3_delete_type setting (related to pop3_deleted_flag).
- plugin { fts_enforced=yes } setting now fails body searches unless it can be done via the full text search engine.
- Added %{passdb:*} and %{userdb:*} variables to various places
- auth: Added ":protected" suffix for passdb and userdb fields. If used, the field doesn't overwrite an existing field.
- IMAP/POP3 proxy: If a backend server dies, avoid client reconnection spikes by slowly disconnecting clients over time. This is enabled by setting login_proxy_max_disconnect_delay=secs passdb extra field.
- imap: Added new read-only METADATA entries: /private/specialuse, /shared/comment, /shared/admin
- imap: If client disconnects in the middle of a command, log how long the command had been running.
- mdbox: Rebuilding could have caused message's reference count to overflow the 16bit number in some situations, causing problems when trying to expunge the duplicates.
- Various search fixes (fts, solr, tika, lib-charset, indexer)
- Various virtual plugin fixes
- Various fixes and optimizations to dsync, imapc and pop3-migration
- imap: Various RFC compliancy and crash fixes to NOTIFY
On 09/24/2015 09:23 AM, Jean-Baptiste Vignaud wrote:
Hello;
Some issue with virtual storage and fts-lucene:
Sep 24 08:12:16 imap(jbv00@x jbv00@d1.dc.xandmail.comxx): Panic: file virtual-storage.c: line 370 (virtual_backend_box_close): assertion failed: (mbox->backends_open_count > 0)
Fixed: http://hg.dovecot.org/dovecot-2.2/rev/4b5b8e6a4a32
Although I think this code can now cause a lot of mailboxes to be opened again if the search result is found from all the mailboxes. So probably needs some more work.
On 09/24/2015 07:08 PM, Timo Sirainen wrote:
On 09/24/2015 09:23 AM, Jean-Baptiste Vignaud wrote:
Hello;
Some issue with virtual storage and fts-lucene:
Sep 24 08:12:16 imap(jbv00@x jbv00@d1.dc.xandmail.comxx): Panic: file virtual-storage.c: line 370 (virtual_backend_box_close): assertion failed: (mbox->backends_open_count > 0)
Fixed: http://hg.dovecot.org/dovecot-2.2/rev/4b5b8e6a4a32
Although I think this code can now cause a lot of mailboxes to be opened again if the search result is found from all the mailboxes. So probably needs some more work.
After a bit more testing, it looks like it would happen for all the searches regardless of the results. So, implemented it using another method that should always work: http://hg.dovecot.org/dovecot-2.2/rev/055d1e2d894c
On Thu, Sep 24, 2015 at 6:27 PM, Timo Sirainen tss@iki.fi wrote:
On 09/24/2015 07:08 PM, Timo Sirainen wrote:
On 09/24/2015 09:23 AM, Jean-Baptiste Vignaud wrote:
Hello;
Some issue with virtual storage and fts-lucene:
Sep 24 08:12:16 imap(jbv00@x jbv00@d1.dc.xandmail.comxx): Panic: file virtual-storage.c: line 370 (virtual_backend_box_close): assertion failed: (mbox->backends_open_count > 0)
Fixed: http://hg.dovecot.org/dovecot-2.2/rev/4b5b8e6a4a32
Although I think this code can now cause a lot of mailboxes to be opened again if the search result is found from all the mailboxes. So probably needs some more work.
After a bit more testing, it looks like it would happen for all the searches regardless of the results. So, implemented it using another method that should always work: http://hg.dovecot.org/dovecot-2.2/rev/055d1e2d894c
Problem fixed; thanks !
JB
On 23/09/2015 14:30, Timo Sirainen wrote:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz
Comiling with cc on Solaris 10 complains "void function cannot return value", patch attached. Dovecot is offering the equivalent of:
void bar() { }
void foo() { return bar(); }
which cc doesn't accept whereas gcc does.
Otherwise no problems, I have 2.2.19.rc1 running on a lightly loaded system.
Am 23.09.2015 um 15:30 schrieb Timo Sirainen:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz.sig
A lot of changes since v2.2.18, so here's a release candidate first. If no bugs are reported, I'm planning on making the final release sometimes this week. The most interesting new features here are the imap-hibernate process, quota count backend and director/proxy improvements.
rc1 compile and run on some smaller Debian hosts here without problems.
Andreas
W dniu 23.09.2015 o 15:30, Timo Sirainen pisze:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz.sig
A lot of changes since v2.2.18, so here's a release candidate first. If no bugs are reported, I'm planning on making the final release sometimes this week. The most interesting new features here are the imap-hibernate process, quota count backend and director/proxy improvements.
Hi! I'm getting compilation error using clang:
make[4]: Entering directory '/var/tmp/portage/net-mail/dovecot-2.2.19_rc1/work/dovecot-2.2.19.rc1/src/lib' /bin/sh ../../libtool --tag=CC --mode=compile clang -DHAVE_CONFIG_H -I. -I../.. -std=gnu99 -O2 -march=native -Wall -W -Wmissing-prototypes -Wmissing-declaratio ns -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -c -o guid.lo guid.c libtool: compile: clang -DHAVE_CONFIG_H -I. -I../.. -std=gnu99 -O2 -march=native -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscrip ts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -c guid.c -fPIC -DPIC -o .libs/guid.o guid.c:106:2: error: array size is negative buffer_create_from_data(&buf, guid_r, GUID_128_SIZE); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./buffer.h:23:8: note: expanded from macro 'buffer_create_from_data' (void)COMPILE_ERROR_IF_TRUE(__builtin_object_size((d),3) < ((s)?(s):1));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./macros.h:161:15: note: expanded from macro 'COMPILE_ERROR_IF_TRUE' (sizeof(char[1 - 2 * !!(condition)]) - 1) ^~~~~~~~~~~~~~~~~~~~~ 1 error generated.
/bin/sh ../../libtool --tag=CC --mode=compile clang -DHAVE_CONFIG_H -I. -I../.. -std=gnu99 -O2 -march=native -Wall -W -Wmissing-prototypes -Wmissing-declaratio ns -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -c -o istream-jsonstr.lo istream-jsonstr.c libtool: compile: clang -DHAVE_CONFIG_H -I. -I../.. -std=gnu99 -O2 -march=native -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscrip ts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -c istream-jsonstr.c -fPIC -DPIC -o .libs/istream-jsonstr.o istream-jsonstr.c:72:3: error: array size is negative buffer_create_from_data(&buf, dest, MAX_UTF8_LEN); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./buffer.h:23:8: note: expanded from macro 'buffer_create_from_data' (void)COMPILE_ERROR_IF_TRUE(__builtin_object_size((d),3) < ((s)?(s):1));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./macros.h:161:15: note: expanded from macro 'COMPILE_ERROR_IF_TRUE' (sizeof(char[1 - 2 * !!(condition)]) - 1) ^~~~~~~~~~~~~~~~~~~~~ 1 error generated. Makefile:1085: recipe for target 'istream-jsonstr.lo' failed make[4]: *** [istream-jsonstr.lo] Error 1
make[3]: Entering directory '/var/tmp/portage/net-mail/dovecot-2.2.19_rc1/work/dovecot-2.2.19.rc1/src/lib-mail' /bin/sh ../../libtool --tag=CC --mode=compile clang -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-charset -std=gnu99 -O2 -mar ch=native -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict -aliasing=2 -c -o istream-binary-converter.lo istream-binary-converter.c libtool: compile: clang -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-charset -std=gnu99 -O2 -march=native -Wall -W -Wmissing-prot otypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -c istream-binary-con verter.c -fPIC -DPIC -o .libs/istream-binary-converter.o istream-binary-converter.c:119:3: error: array size is negative buffer_create_from_data(&buf, dest, BASE64_BLOCK_SIZE); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../../src/lib/buffer.h:23:8: note: expanded from macro 'buffer_create_from_data' (void)COMPILE_ERROR_IF_TRUE(__builtin_object_size((d),3) < ((s)?(s):1));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../../src/lib/macros.h:161:15: note: expanded from macro 'COMPILE_ERROR_IF_TRUE' (sizeof(char[1 - 2 * !!(condition)]) - 1) ^~~~~~~~~~~~~~~~~~~~~ 1 error generated. Makefile:882: recipe for target 'istream-binary-converter.lo' failed
libtool: compile: clang -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-charset -I../../src/lib-dict -I../../src/lib-mail -I../../sr c/lib-storage -I../../src/lib-imap -I../../src/lib-imap-storage -std=gnu99 -O2 -march=native -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wch ar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -c imap-urlauth-backend.c -fPIC -DPIC -o .libs/imap-urlauth-backend.o imap-urlauth-backend.c:62:3: error: array size is negative buffer_create_from_data(&key_buf, mailbox_key_r, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../../src/lib/buffer.h:23:8: note: expanded from macro 'buffer_create_from_data' (void)COMPILE_ERROR_IF_TRUE(__builtin_object_size((d),3) < ((s)?(s):1));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../../src/lib/macros.h:161:15: note: expanded from macro 'COMPILE_ERROR_IF_TRUE' (sizeof(char[1 - 2 * !!(condition)]) - 1) ^~~~~~~~~~~~~~~~~~~~~ 1 error generated. Makefile:484: recipe for target 'imap-urlauth-bac
make[4]: Entering directory '/var/tmp/portage/net-mail/dovecot-2.2.19_rc1/work/dovecot-2.2.19.rc1/src/lib-storage' /bin/sh ../../libtool --tag=CC --mode=compile clang -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-auth -I../../src/lib-dict -I. ./../src/lib-sasl -I../../src/lib-ssl-iostream -I../../src/lib-fs -I../../src/lib-master -I../../src/lib-settings -I../../src/lib-charset -I../../src/lib-mail -I../.. /src/lib-imap -I../../src/lib-index -DPKG_RUNDIR=\""/var/run/dovecot"\" -DMODULEDIR=\""/usr/lib64/dovecot"\" -std=gnu99 -O2 -march=native -Wall -W -Wmissing-prototy pes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -c -o mail.lo mail.c libtool: compile: clang -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I../../src/lib-auth -I../../src/lib-dict -I../../src/lib-sasl -I../../src/l ib-ssl-iostream -I../../src/lib-fs -I../../src/lib-master -I../../src/lib-settings -I../../src/lib-charset -I../../src/lib-mail -I../../src/lib-imap -I../../src/lib-i ndex -DPKG_RUNDIR=\"/var/run/dovecot\" -DMODULEDIR=\"/usr/lib64/dovecot\" -std=gnu99 -O2 -march=native -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer- arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -c mail.c -fPIC -DPIC -o .libs/mail.o mail.c:419:3: error: array size is negative buffer_create_from_data(&buf, guid_128_r, GUID_128_SIZE); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../../src/lib/buffer.h:23:8: note: expanded from macro 'buffer_create_from_data' (void)COMPILE_ERROR_IF_TRUE(__builtin_object_size((d),3) < ((s)?(s):1));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../../src/lib/macros.h:161:15: note: expanded from macro 'COMPILE_ERROR_IF_TRUE' (sizeof(char[1 - 2 * !!(condition)]) - 1) ^~~~~~~~~~~~~~~~~~~~~ 1 error generated. Makefile:767: recipe for target 'mail.lo' failed
make[3]: Entering directory '/var/tmp/portage/net-mail/dovecot-2.2.19_rc1/work/dovecot-2.2.19.rc1/src/lib-otp' clang -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -std=gnu99 -O2 -march=native -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscrip ts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -c -o otp-parse.o otp-parse.c otp-parse.c:45:2: error: array size is negative buffer_create_from_data(&buf, hash, OTP_HASH_SIZE); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../../src/lib/buffer.h:23:8: note: expanded from macro 'buffer_create_from_data' (void)COMPILE_ERROR_IF_TRUE(__builtin_object_size((d),3) < ((s)?(s):1));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ../../src/lib/macros.h:161:15: note: expanded from macro 'COMPILE_ERROR_IF_TRUE' (sizeof(char[1 - 2 * !!(condition)]) - 1) ^~~~~~~~~~~~~~~~~~~~~ 1 error generated. Makefile:433: recipe for target 'otp-parse.o' failed
$ clang -v clang version 3.8.0 (cc0e5a2f90ee7a0d48042c4b5b66546521f77923) (11768ab0810142d431e9be949daa779a8809347b)
Marcin
On 24 Sep 2015, at 13:43, Marcin Mirosław marcin@mejor.pl wrote:
W dniu 23.09.2015 o 15:30, Timo Sirainen pisze:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz.sig
A lot of changes since v2.2.18, so here's a release candidate first. If no bugs are reported, I'm planning on making the final release sometimes this week. The most interesting new features here are the imap-hibernate process, quota count backend and director/proxy improvements.
Hi! I'm getting compilation error using clang:
guid.c:106:2: error: array size is negative buffer_create_from_data(&buf, guid_r, GUID_128_SIZE); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./buffer.h:23:8: note: expanded from macro 'buffer_create_from_data' (void)COMPILE_ERROR_IF_TRUE(__builtin_object_size((d),3) < ((s)?(s):1)); \
What happens if in src/lib/buffer.h line 23 you change the __builtin_object_size((d),3) to __builtin_object_size((d),1)? Does it work?
W dniu 24.09.2015 o 14:01, Timo Sirainen pisze:
On 24 Sep 2015, at 13:43, Marcin Mirosław marcin@mejor.pl wrote:
W dniu 23.09.2015 o 15:30, Timo Sirainen pisze:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz.sig
A lot of changes since v2.2.18, so here's a release candidate first. If no bugs are reported, I'm planning on making the final release sometimes this week. The most interesting new features here are the imap-hibernate process, quota count backend and director/proxy improvements.
Hi! I'm getting compilation error using clang:
guid.c:106:2: error: array size is negative buffer_create_from_data(&buf, guid_r, GUID_128_SIZE); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./buffer.h:23:8: note: expanded from macro 'buffer_create_from_data' (void)COMPILE_ERROR_IF_TRUE(__builtin_object_size((d),3) < ((s)?(s):1)); \
What happens if in src/lib/buffer.h line 23 you change the __builtin_object_size((d),3) to __builtin_object_size((d),1)? Does it work?
Yes, it works. Now dovecot-2.2.19.rc1 compiles fine.
Thanks, Marcin
P.S. Is such warnings something you care?
clang -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-settings -I../../src/lib-master -DPKG_RUNDIR=\""/var/run/dovecot"\" -DPKG_STATEDIR=\""/var/lib/dovecot"\" -DPKG_LIBEXECDIR=\""/usr/libexec/dovecot"\" -DBINDIR=\""/usr/bin"\" -std=gnu99 -O2 -march=native -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -c -o sd-daemon.o sd-daemon.c sd-daemon.c:56:19: warning: implicit declaration of function 'getenv' is invalid in C99 [-Wimplicit-function-declaration] if (!(e = getenv("LISTEN_PID"))) { ^ sd-daemon.c:56:17: warning: incompatible integer to pointer conversion assigning to 'const char *' from 'int' [-Wint-conversion] if (!(e = getenv("LISTEN_PID"))) { ^ ~~~~~~~~~~~~~~~~~~~~ sd-daemon.c:62:13: warning: implicit declaration of function 'strtoul' is invalid in C99 [-Wimplicit-function-declaration] l = strtoul(e, &p, 10); ^ sd-daemon.c:80:17: warning: incompatible integer to pointer conversion assigning to 'const char *' from 'int' [-Wint-conversion] if (!(e = getenv("LISTEN_FDS"))) { ^ ~~~~~~~~~~~~~~~~~~~~ sd-daemon.c:119:17: warning: implicit declaration of function 'unsetenv' is invalid in C99 [-Wimplicit-function-declaration] unsetenv("LISTEN_PID"); ^ sd-daemon.c:342:17: warning: incompatible integer to pointer conversion assigning to 'const char *' from 'int' [-Wint-conversion] if (!(e = getenv("NOTIFY_SOCKET"))) ^ ~~~~~~~~~~~~~~~~~~~~~~~ sd-daemon.c:411:9: warning: implicit declaration of function 'free' is invalid in C99 [-Wimplicit-function-declaration] free(p); ^ 7 warnings generated.
On 24 Sep 2015, at 15:11, Marcin Mirosław marcin@mejor.pl wrote:
What happens if in src/lib/buffer.h line 23 you change the __builtin_object_size((d),3) to __builtin_object_size((d),1)? Does it work?
Yes, it works. Now dovecot-2.2.19.rc1 compiles fine.
OK, committed the change.
P.S. Is such warnings something you care?
clang -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-settings -I../../src/lib-master -DPKG_RUNDIR=\""/var/run/dovecot"\" -DPKG_STATEDIR=\""/var/lib/dovecot"\" -DPKG_LIBEXECDIR=\""/usr/libexec/dovecot"\" -DBINDIR=\""/usr/bin"\" -std=gnu99 -O2 -march=native -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -c -o sd-daemon.o sd-daemon.c sd-daemon.c:56:19: warning: implicit declaration of function 'getenv' is invalid in C99 [-Wimplicit-function-declaration] if (!(e = getenv("LISTEN_PID"))) {
These were already fixed: http://hg.dovecot.org/dovecot-2.2/rev/c4b9e6606cbf
participants (5)
-
A. Schulze
-
James
-
Jean-Baptiste Vignaud
-
Marcin Mirosław
-
Timo Sirainen