Hello;
Some issue with virtual storage and fts-lucene:
Sep 24 08:12:16 imap(jbv00@x <jbv00@d1.dc.xandmail.com>xx): Panic: file virtual-storage.c: line 370 (virtual_backend_box_close): assertion failed: (mbox->backends_open_count > 0) Sep 24 08:12:16 imap(jbv00@x <jbv00@d1.dc.xandmail.com>xx): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x7fb4a) [0x7fa31284fb4a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x7fbb6) [0x7fa31284fbb6] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fa3127f967f] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x9324) [0x7fa3111ba324] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x93d5) [0x7fa3111ba3d5] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x9429) [0x7fa3111ba429] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a) [0x7fa312b0a8ea] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_free+0x13) [0x7fa312b0c453] -> dovecot/imap(cmd_logout+0x40) [0x4113c0] -> dovecot/imap(command_exec+0x75) [0x4194f5] -> dovecot/imap() [0x417bb0] -> dovecot/imap() [0x417c46] -> dovecot/imap(client_handle_input+0x11d) [0x417f3d] -> dovecot/imap(client_input+0x6f) [0x4182bf] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7fa312861ecb] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xb7) [0x7fa312863457] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7fa312861f75] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fa312862118] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fa3127fe873] -> dovecot/imap(main+0x2cc) [0x4249ac] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7fa31245ad5d] -> dovecot/imap() [0x40bf99] Sep 24 08:12:16 imap(jbv00@x <jbv00@d1.dc.xandmail.com>xx): Fatal: master: service(imap): child 16321 killed with signal 6 (core dumps disabled)
I think thys was introduced by http://hg.dovecot.org/dovecot-2.2/rev/251747c3afe5.
(gdb) r -u jbv00@xxx Starting program: /usr/local/libexec/dovecot/imap -u jbv00@xxx [Thread debugging using libthread_db enabled] process 16340 is executing new program: /usr/local/bin/doveconf Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.166.el6_7.1.x86_64 [Thread debugging using libthread_db enabled] process 16340 is executing new program: /usr/local/libexec/dovecot/imap Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.166.el6_7.1.x86_64 [Thread debugging using libthread_db enabled]
- PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SEARCH=FUZZY SPECIAL-USE QUOTA] Logged in as jbv00@xxx l select virtual.all
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent $Forwarded unknown-1)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent $Forwarded unknown-1 \*)] Flags permitted.
- 16480 EXISTS
- 0 RECENT
- OK [UNSEEN 626] First unseen.
- OK [UIDVALIDITY 1441363451] UIDs valid
- OK [UIDNEXT 16481] Predicted next UID
- OK [HIGHESTMODSEQ 1] Highest l OK [READ-WRITE] Select completed (0.000 + 0.000 secs). s search text test
- SEARCH 520 606 697 896 898 899 901 2118 2166 2871 2873 2878 2884 2885 2897 2928 2932 2935 2942 6286 7080 7082 7139 7339 7852 7866 7886 8433 8698 9373 9552 9578 9605 9609 9731 10053 10250 10320 10322 10323 10341 10493 10618 10887 11065 11069 11147 11169 11292 11297 11298 11351 11991 12197 12231 12428 12429 13889 15214 16464 s OK Search completed (0.393 + 0.000 secs). l logout imap(jbv00@xxx): Panic: file virtual-storage.c: line 370 (virtual_backend_box_close): assertion failed: (mbox->backends_open_count >
- imap(jbv00@xxx): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x7fb4a) [0x7ffff783bb4a] -> /usr/local/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7ffff783bc52] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x7ffff77e567f] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x9324) [0x7ffff61a6324] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x93d5) [0x7ffff61a63d5] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(+0x9429) [0x7ffff61a6429] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_close+0x1a) [0x7ffff7af68ea] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_free+0x13) [0x7ffff7af8453] -> /usr/local/libexec/dovecot/imap(cmd_logout+0x40) [0x4113c0] -> /usr/local/libexec/dovecot/imap(command_exec+0x75) [0x4194f5] -> /usr/local/libexec/dovecot/imap() [0x417bb0] -> /usr/local/libexec/dovecot/imap() [0x417c46] -> /usr/local/libexec/dovecot/imap(client_handle_input+0x11d) [0x417f3d] -> /usr/local/libexec/dovecot/imap(client_input+0x6f) [0x4182bf] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7ffff784decb] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xb7) [0x7ffff784f457] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7ffff784df75] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7ffff784e118] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7ffff77ea873] -> /usr/local/libexec/dovecot/imap(main+0x2cc) [0x4249ac] -> /lib64/libc.so.6(__libc_start_main+0xfd) [0x7ffff7446d5d] -> /usr/local/libexec/dovecot/imap() [0x40bf99]
Program received signal SIGABRT, Aborted. 0x00007ffff745a625 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.166.el6_7.1.x86_64 libgcc-4.4.7-16.el6.x86_64 libstdc++-4.4.7-16.el6.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) bt #0 0x00007ffff745a625 in raise () from /lib64/libc.so.6 #1 0x00007ffff745be05 in abort () from /lib64/libc.so.6 #2 0x00007ffff783bb40 in default_fatal_finish (type=<value optimized out>, status=0) at failures.c:201 #3 0x00007ffff783bc52 in default_fatal_handler (ctx=0x7fffffffe1d0, format=<value optimized out>, args=<value optimized out>) at failures.c:215 #4 0x00007ffff77e567f in i_panic (format=0x3fd4 <Address 0x3fd4 out of bounds>) at failures.c:275 #5 0x00007ffff61a6324 in virtual_backend_box_close (mbox=0x664830, bbox=0x670838) at virtual-storage.c:370 #6 0x00007ffff61a63d5 in virtual_mailbox_close_internal (mbox=0x664830) at virtual-storage.c:401 #7 0x00007ffff61a6429 in virtual_mailbox_close (box=<value optimized out>) at virtual-storage.c:463 #8 0x00007ffff7af68ea in mailbox_close (box=0x664830) at mail-storage.c:1228 #9 0x00007ffff7af8453 in mailbox_free (_box=<value optimized out>) at mail-storage.c:1248 #10 0x00000000004113c0 in cmd_logout (cmd=0x65fb30) at cmd-logout.c:18 #11 0x00000000004194f5 in command_exec (cmd=0x65fb30) at imap-commands.c:169 #12 0x0000000000417bb0 in client_command_input (cmd=0x65fb30) at imap-client.c:869 #13 0x0000000000417c46 in client_command_input (cmd=0x65fb30) at imap-client.c:929 #14 0x0000000000417f3d in client_handle_next_command (client=0x65ef30) at imap-client.c:967 #15 client_handle_input (client=0x65ef30) at imap-client.c:979 #16 0x00000000004182bf in client_input (client=0x65ef30) at imap-client.c:1021 #17 0x00007ffff784decb in io_loop_call_io (io=0x65fa20) at ioloop.c:559 #18 0x00007ffff784f457 in io_loop_handler_run_internal (ioloop=<value optimized out>) at ioloop-epoll.c:220 #19 0x00007ffff784df75 in io_loop_handler_run (ioloop=0x63ec70) at ioloop.c:607 #20 0x00007ffff784e118 in io_loop_run (ioloop=0x63ec70) at ioloop.c:583 #21 0x00007ffff77ea873 in master_service_run (service=0x63eb10, callback=<value optimized out>) at master-service.c:640 #22 0x00000000004249ac in main (argc=3, argv=0x63e390) at main.c:442
This can also be avoided by setting plugin/virtual_max_open_mailboxes=1000 (where 1000 is greater than the number of total folders of the mailbox).
/usr/local/sbin/dovecot -n # 2.2.19.rc1: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-504.el6.x86_64 x86_64 CentOS release 6.5 (Final) ext4 auth_debug = yes auth_mechanisms = plain cram-md5 apop auth_username_chars = auth_verbose = yes base_dir = /usr/local/var/run default_login_user = dovelog dict { path = /usr/local/var/run/dict-server quotadict = mysql:/usr/local/etc/dovecot/conf.d/dovecot-dict-quota.conf } disable_plaintext_auth = no first_valid_uid = 100 listen = * log_path = /home/backend/logs/dovecot/dovecot.log mail_gid = vmail mail_location = maildir:/home/backend/vmail/%d/%n mail_plugins = fts fts_lucene virtual mail_uid = vmail namespace { hidden = no inbox = yes location = mailbox Drafts { auto = create special_use = \Drafts } mailbox Sent { auto = create special_use = \Sent } mailbox Spam { auto = create special_use = \Junk } mailbox Trash { auto = create special_use = \Trash } prefix = INBOX. separator = . type = private } namespace { hidden = yes inbox = no list = no location = virtual:/usr/local/etc/dovecot/virtual:INDEX=~/virtual prefix = virtual. separator = . subscriptions = no } passdb { args = /usr/local/etc/dovecot/conf.d/dovecot-mysql.conf driver = sql } plugin { fts = lucene fts_decoder = decode2text fts_lucene = whitespace_chars=@. mail_log_events = delete undelete expunge copy mail_log_fields = uid box msgid size mail_log_group_events = no mail_log_max_lines_per_sec = 0 quota = dict:user::proxy::quotadict quota_rule = *:storage=100M quota_rule2 = *:messages=1000000 } protocols = imap pop3 service auth { unix_listener auth-client { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0600 user = vmail } user = dovecot } service decode2text { executable = script /usr/local/libexec/dovecot/decode2text.sh group = vmail unix_listener decode2text { group = vmail mode = 0666 user = vmail } user = vmail } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { user = dovelog } service pop3-login { user = dovelog } ssl_cert = </usr/local/etc/certs/dovecot-dev1.xxx_crt.pem ssl_key = </usr/local/etc/certs/dovecot-dev1.xxx_key.pem userdb { args = /usr/local/etc/dovecot/conf.d/dovecot-mysql.conf driver = sql } protocol imap { mail_plugins = quota imap_quota mail_log notify fts fts_lucene virtual } protocol pop3 { mail_plugins = quota virtual pop3_uidl_format = %08Xu%08Xv } protocol lda { auth_socket_path = /usr/local/var/run/auth-master info_log_path = /home/backend/logs/deliver/deliver.log log_path = /home/backend/logs/deliver/deliver.log mail_plugins = quota fts fts_lucene virtual postmaster_address = vignaud@xxx sendmail_path = /home/backend/postfix/usr/sbin/sendmail }
On Wed, Sep 23, 2015 at 3:30 PM, Timo Sirainen <tss@iki.fi> wrote:
http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.19.rc1.tar.gz.sig
A lot of changes since v2.2.18, so here's a release candidate first. If no bugs are reported, I'm planning on making the final release sometimes this week. The most interesting new features here are the imap-hibernate process, quota count backend and director/proxy improvements.
- "doveadm director flush" command has a changed meaning now: It safely moves users to their wanted backends, instead of simply forgetting the mapping entirely and leaving the existing connections untouched. Use -F parameter to get the original unsafe behavior.
- Added support for imap-hiberanate processes.
- Optimized tracking mailboxes' vsizes (= sum of all messages' size). If mailbox_list_index=yes, it's also stored in there. This makes it very efficient to look up vsizes for all mailboxes.
- Added a quota "count" backend, which uses the mailbox vsizes to get the current quota usage. It requires using the new quota_vsizes=yes setting, which tracks the messages' "virtual sizes" rather than "physical sizes". Their distiction is minor and mostly irrelevant nowadays (if mail sizes should be counted with LF or CRLF newlines).
- "doveadm director up/down" commands added. The monitoring script should be using these commands instead of changing the vhost count. This allows admin to manually disable a server by changing the vhost count to 0 without the monitoring script changing it back.
- Added support for HAProxy protocol: http://wiki2.dovecot.org/HAProxy
- Added push-notification plugin framework, which can be used to easily implement push notifications to various backends. Implemented "ox" backend for notifying Open-Xchange via HTTP/json.
- imap_logout_format supports more variables now, e.g. number of deleted messages.
- pop3: Added pop3_delete_type setting (related to pop3_deleted_flag).
- plugin { fts_enforced=yes } setting now fails body searches unless it can be done via the full text search engine.
- Added %{passdb:*} and %{userdb:*} variables to various places
- auth: Added ":protected" suffix for passdb and userdb fields. If used, the field doesn't overwrite an existing field.
- IMAP/POP3 proxy: If a backend server dies, avoid client reconnection spikes by slowly disconnecting clients over time. This is enabled by setting login_proxy_max_disconnect_delay=secs passdb extra field.
- imap: Added new read-only METADATA entries: /private/specialuse, /shared/comment, /shared/admin
- imap: If client disconnects in the middle of a command, log how long the command had been running.
- mdbox: Rebuilding could have caused message's reference count to overflow the 16bit number in some situations, causing problems when trying to expunge the duplicates.
- Various search fixes (fts, solr, tika, lib-charset, indexer)
- Various virtual plugin fixes
- Various fixes and optimizations to dsync, imapc and pop3-migration
- imap: Various RFC compliancy and crash fixes to NOTIFY