crash when trying to upgrade from 2.0.x to 2.3.x
Hello,
I'm trying to migrate dovecot mail database from 2.0.9 on Linux to 2.3.20 on FreeBSD. I've rsync'd the entire database, grabbed all the configs, changed the paths and permission, but when trying to access the mail from Thunderbird I have reproducible crash in the logs:
===Cut=== Feb 14 17:12:10 it-r-support dovecot[5888]: auth-worker(6378): Debug: mysql(localhost): Finished query 'SELECT maildir, 1143 AS uid, 1143 AS gid FROM mailbox WHERE username = 'zheganin-em@alamics.ru' AND active = '1'' in 0 msecs Feb 14 17:12:10 it-r-support dovecot[5888]: auth-worker(6378): Debug: conn unix:auth-worker (uid=143): auth-worker<16>: sql(zheganin-em@alamics.ru,192.168.57.177,<oE2zjKn0FTXAqDmx>): Finished userdb lookup Feb 14 17:12:10 it-r-support dovecot[5888]: auth-worker(6378): Debug: conn unix:auth-worker (uid=143): auth-worker<16>: Finished Feb 14 17:12:10 it-r-support dovecot[5888]: auth: Debug: sql(zheganin-em@alamics.ru,192.168.57.177,<oE2zjKn0FTXAqDmx>): Finished userdb lookup Feb 14 17:12:10 it-r-support dovecot[5888]: auth: Debug: master userdb out: USER 1282408449zheganin-em@alamics.ru maildir=alamics.ru/zheganin-em/ uid=1143 gid=1143 auth_mech=PLAIN auth_token=15cceab95e1251ea97a8e5ff62c70d1b7a76c96f Feb 14 17:12:10 it-r-support dovecot[5888]: imap-login: Login: user=<zheganin-em@alamics.ru>, method=PLAIN, rip=192.168.57.177, lip=172.16.11.13, mpid=6405, session=<oE2zjKn0FTXAqDmx> Feb 14 17:12:10 it-r-support dovecot[5888]: imap(zheganin-em@alamics.ru)<6405><oE2zjKn0FTXAqDmx>: Panic: file array.c: line 10 (array_idx_modifiable_i): assertion failed: (idx < array->buffer->used / array->element_size) Feb 14 17:12:10 it-r-support dovecot[5888]: imap(zheganin-em@alamics.ru)<6405><oE2zjKn0FTXAqDmx>: Fatal: master: service(imap): child 6405 killed with signal 6 (core dumped) ===Cut===
There's an older thread here mentioning that probably doveadm force-resync would help, but it doesn't fix this. Is there something that could be done here ? I've also reported this in the FreeBSD bugtracker, but just in case decided to ask here too.
Backtrace:
===Cut=== # lldb /usr/local/libexec/dovecot/imap --core imap-6404.core (lldb) target create "/usr/local/libexec/dovecot/imap" --core "imap-6404.core" Core file '/var/cores/imap-6404.core' (x86_64) was loaded. (lldb) bt
- thread #1, name = 'imap', stop reason = signal SIGABRT
- frame #0: 0x000000001151133a libc.so.7
__sys_thr_kill + 10 frame #1: 0x0000000011489c74 libc.so.7__raise + 52 frame #2: 0x000000001153b109 libc.so.7abort + 73 frame #3: 0x000000001137a336 libdovecot.so.0___lldb_unnamed_symbol5855 + 150 frame #4: 0x0000000011378ad1 libdovecot.so.0___lldb_unnamed_symbol5844 + 49 frame #5: 0x0000000011379881 libdovecot.so.0___lldb_unnamed_symbol5846 + 33 frame #6: 0x0000000011378ceb libdovecot.so.0i_panic + 187 frame #7: 0x000000001136a514 libdovecot.so.0array_idx_modifiable_i + 100 frame #8: 0x0000000012054a78 lib20_fts_plugin.softs_user_autoindex_exclude + 72 frame #9: 0x0000000012051b3b lib20_fts_plugin.softs_mailbox_allocated + 267 frame #10: 0x000000001113ad0b libdovecot-storage.so.0hook_mailbox_allocated + 155 frame #11: 0x0000000011134491 libdovecot-storage.so.0mailbox_alloc + 449 frame #12: 0x0000000001044311 imapcmd_select_full + 353 frame #13: 0x000000000104cd92 imapcommand_exec + 178 frame #14: 0x000000000104b64a imap___lldb_unnamed_symbol1265 + 58 frame #15: 0x000000000104b7b6 imap___lldb_unnamed_symbol1265 + 422 frame #16: 0x000000000104a151 imapclient_handle_input + 465 frame #17: 0x00000000010488a8 imapclient_input + 120 frame #18: 0x00000000113936df libdovecot.so.0io_loop_call_io + 111 frame #19: 0x0000000011396214 libdovecot.so.0io_loop_handler_run_internal + 388 frame #20: 0x0000000011393dea libdovecot.so.0io_loop_handler_run + 378 frame #21: 0x0000000011393c28 libdovecot.so.0io_loop_run + 88 frame #22: 0x00000000112f6208 libdovecot.so.0master_service_run + 24 frame #23: 0x000000000105af67 imapmain + 1383 frame #24: 0x000000000103c11d imap`_start + 269 (lldb) ===Cut===
- frame #0: 0x000000001151133a libc.so.7
Same backtrace in GDB:
===Cut=== # gdb /usr/local/libexec/dovecot/imap --core imap-6404.core GNU gdb (GDB) 12.1 [GDB v12.1 for FreeBSD] Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd13.1". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/local/libexec/dovecot/imap... (No debugging symbols found in /usr/local/libexec/dovecot/imap) [New LWP 100485] warning: Could not load shared library symbols for [vdso]. Do you need "set solib-search-path" or "set sysroot"? Core was generated by `dovecot/imap'. Program terminated with signal SIGABRT, Aborted. Sent by thr_kill() from pid 6404 and user 1143. #0 0x000000001151133a in thr_kill () from /lib/libc.so.7 (gdb) bt full #0 0x000000001151133a in thr_kill () from /lib/libc.so.7 No symbol table info available. #1 0x0000000011489c74 in raise () from /lib/libc.so.7 No symbol table info available. #2 0x000000001153b109 in abort () from /lib/libc.so.7 No symbol table info available. #3 0x000000001137a336 in ?? () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #4 0x0000000011378ad1 in ?? () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #5 0x0000000011379881 in ?? () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x0000000011378ceb in i_panic () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #7 0x000000001136a514 in array_idx_modifiable_i () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #8 0x0000000012054a78 in fts_user_autoindex_exclude () from /usr/local/lib/dovecot/lib20_fts_plugin.so No symbol table info available. #9 0x0000000012051b3b in fts_mailbox_allocated () from /usr/local/lib/dovecot/lib20_fts_plugin.so No symbol table info available. #10 0x000000001113ad0b in hook_mailbox_allocated () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #11 0x0000000011134491 in mailbox_alloc () from /usr/local/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #12 0x0000000001044311 in cmd_select_full () No symbol table info available. #13 0x000000000104cd92 in command_exec () No symbol table info available. #14 0x000000000104b64a in ?? () No symbol table info available. #15 0x000000000104b7b6 in ?? () No symbol table info available. #16 0x000000000104a151 in client_handle_input () No symbol table info available. #17 0x00000000010488a8 in client_input () No symbol table info available. #18 0x00000000113936df in io_loop_call_io () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #19 0x0000000011396214 in io_loop_handler_run_internal () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #20 0x0000000011393dea in io_loop_handler_run () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #21 0x0000000011393c28 in io_loop_run () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. --Type <RET> for more, q to quit, c to continue without paging-- #22 0x00000000112f6208 in master_service_run () from /usr/local/lib/dovecot/libdovecot.so.0 No symbol table info available. #23 0x000000000105af67 in main () No symbol table info available. (gdb) ===Cut===
Thanks. Eugene.
On 15/02/2023 08:29 EET Eugene M. Zheganin <eugene@zhegan.in> wrote:
Hello, I'm trying to migrate dovecot mail database from 2.0.9 on Linux to 2.3.20 on FreeBSD. I've rsync'd the entire database, grabbed all the configs, changed the paths and permission, but when trying to access the mail from Thunderbird I have reproducible crash in the logs:
Thank you for reporting this issue, which fts driver are you using? Can you provide doveconf -n output?
Aki
Hello,
On 15.02.2023 12:48, Aki Tuomi wrote:
Thank you for reporting this issue, which fts driver are you using? Can you provide
doveconf -noutput?
Seems like I'm using ... squat, and this, in turn, seems to be the old legacy piece from who knows how long ago installed instance. From your question - am I right supposing that if I will get rid of squat things will improve ? Also seems like it's worth mentioning that downgrading to dovecot 2.3.15 improved things radically - no crash at this time (with the same set of configuration files).
dovecot -n output:
===Cut===
# doveconf -n # 2.3.15 (0503334ab1): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.15 (e6a84e31) # OS: FreeBSD 13.1-RELEASE amd64 # Hostname: it-r-support auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login cram-md5 digest-md5 auth_verbose = yes auth_verbose_passwords = yes default_client_limit = 40963 default_process_limit = 10240 disable_plaintext_auth = no first_valid_gid = 1143 first_valid_uid = 1143 hostname = alamics.ru mail_home = /var/imap/%d/%n mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime for everypart extracttext mbox_write_locks = fcntl namespace { inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } namespace { location = maildir:/var/imap/%d/public prefix = Public/ separator = / subscriptions = no type = public } namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = file:/var/db/dovecot/shared-mailboxes fts = squat fts_autoindex = yes fts_squat = partial=4 full=10 quota = maildir:User quota quota_rule = *:storage=50G quota_rule2 = Trash:storage=+10%% quota_rule3 = Spam:storage=+20%% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_after = /var/imap/sieve/sieve-after sieve_before = /var/imap/sieve/sieve-before sieve_default = /var/imap/sieve/default.sieve sieve_dir = /var/imap/%d/%n/sieve sieve_global_dir = /var/imap/sieve sieve_max_script_size = 1M } postmaster_address = postmaster@alamics.ru protocols = imap pop3 lmtp sieve service auth { client_limit = 62464 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = dovecot mode = 0660 user = dovecot } } service imap-login { client_limit = 10000 inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 10240 process_min_avail = 1 service_count = 0 vsz_limit = 256 M } service imap { drop_priv_before_exec = yes process_limit = 10240 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = dovecot } user = dovecot } ssl_cert = </usr/local/etc/letsencrypt/live/alamics.ru/fullchain.pem ssl_key = # hidden, use -P to show it syslog_facility = local6 userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = " quota acl sieve" } protocol lda { mail_plugins = " quota acl sieve" } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_max_userip_connections = 512 mail_plugins = quota imap_quota fts fts_squat }
===Cut===
Thanks.
Eugene.
squat has been obsoleted since 2.3.0. You should use either solr or lucene (if it works), there is also xapian based flatcurve plugin available for 2.3 from community (will be in core for 2.4).
Aki
On 15/02/2023 14:07 EET Eugene M. Zheganin <eugene@zhegan.in> wrote:
Hello,
On 15.02.2023 12:48, Aki Tuomi wrote:
Thank you for reporting this issue, which fts driver are you using? Can you provide
doveconf -noutput?Seems like I'm using ... squat, and this, in turn, seems to be the old legacy piece from who knows how long ago installed instance. From your question - am I right supposing that if I will get rid of squat things will improve ? Also seems like it's worth mentioning that downgrading to dovecot 2.3.15 improved things radically - no crash at this time (with the same set of configuration files). dovecot -n output: ===Cut=== # doveconf -n # 2.3.15 (0503334ab1): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.15 (e6a84e31) # OS: FreeBSD 13.1-RELEASE amd64 # Hostname: it-r-support auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login cram-md5 digest-md5 auth_verbose = yes auth_verbose_passwords = yes default_client_limit = 40963 default_process_limit = 10240 disable_plaintext_auth = no first_valid_gid = 1143 first_valid_uid = 1143 hostname = alamics.ru mail_home = /var/imap/%d/%n mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime for everypart extracttext mbox_write_locks = fcntl namespace { inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } namespace { location = maildir:/var/imap/%d/public prefix = Public/ separator = / subscriptions = no type = public } namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = file:/var/db/dovecot/shared-mailboxes fts = squat fts_autoindex = yes fts_squat = partial=4 full=10 quota = maildir:User quota quota_rule = *:storage=50G quota_rule2 = Trash:storage=+10%% quota_rule3 = Spam:storage=+20%% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_after = /var/imap/sieve/sieve-after sieve_before = /var/imap/sieve/sieve-before sieve_default = /var/imap/sieve/default.sieve sieve_dir = /var/imap/%d/%n/sieve sieve_global_dir = /var/imap/sieve sieve_max_script_size = 1M } postmaster_address = postmaster@alamics.ru protocols = imap pop3 lmtp sieve service auth { client_limit = 62464 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = dovecot mode = 0660 user = dovecot } } service imap-login { client_limit = 10000 inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 10240 process_min_avail = 1 service_count = 0 vsz_limit = 256 M } service imap { drop_priv_before_exec = yes process_limit = 10240 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = dovecot } user = dovecot } ssl_cert = </usr/local/etc/letsencrypt/live/alamics.ru/fullchain.pem ssl_key = # hidden, use -P to show it syslog_facility = local6 userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = " quota acl sieve" } protocol lda { mail_plugins = " quota acl sieve" } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_max_userip_connections = 512 mail_plugins = quota imap_quota fts fts_squat }
===Cut===
Thanks. Eugene.
participants (2)
-
Aki Tuomi
-
Eugene M. Zheganin