Re: doveadm batch crash

On 08/12/2018 19:53, Jonathan Casiot via dovecot wrote:

Hi

I'm having an issue with doveadm batch - the following command always crashes:

doveadm batch : mailbox status all inbox

Whereas the following work as expected:

doveadm mailbox status all inbox doveadm batch : mailbox list

Any assistance gratefully received.

Debug output from the crash:

Debug: Loading modules from directory: /usr/lib64/dovecot Debug: Module loaded: /usr/lib64/dovecot/lib01_acl_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib10_mail_crypt_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib90_stats_plugin.so Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so doveadm(jc): Debug: Effective uid=1001, gid=100, home=/home/jc doveadm(jc): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled doveadm(jc): Debug: mail_crypt_plugin: mail_crypt_curve setting missing

• generating EC keys disabled doveadm(jc): Debug: Namespace inbox: type=private, prefix=, sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:~/mdbox:ALT=~/Archive/tape/mdbox doveadm(jc): Debug: fs: root=/home/jc/mdbox, index=, indexpvt=, control=, inbox=, alt=/home/jc/Archive/tape/mdbox doveadm(jc): Debug: acl: initializing backend with data: vfile doveadm(jc): Debug: acl: acl username = jc doveadm(jc): Debug: acl: owner = 1 doveadm(jc): Debug: acl vfile: Global ACLs disabled doveadm(jc): Panic: file mail-storage.c: line 875 (mailbox_set_reason): assertion failed: (reason != NULL) doveadm(jc): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xa0e9e) [0x7fc993023e9e] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7fc993023f0a] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7fc992fb3c97] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x49856) [0x7fc993301856] -> /usr/bin/doveadm(+0x348e2) [0x5631dbb5f8e2] -> /usr/bin/doveadm(+0x2e1ad) [0x5631dbb591ad] -> /usr/bin/doveadm(+0x2bd2c) [0x5631dbb56d2c] -> /usr/bin/doveadm(+0x2c92a) [0x5631dbb5792a] -> /usr/bin/doveadm(doveadm_mail_try_run+0x215) [0x5631dbb58285] -> /usr/bin/doveadm(main+0x46a) [0x5631dbb473fa] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fc992bd83d5] -> /usr/bin/doveadm(+0x1c565) [0x5631dbb47565] Aborted (core dumped)

And doveconf -n output:

# 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-957.1.3.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: ### redacted ### first_valid_gid = 100 first_valid_uid = 1000 last_valid_gid = 100 last_valid_uid = 1999 listen = * login_greeting = IMAP ready login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e session=<%{session}> %k mail_attribute_dict = file:%h/mdbox/dovecot-attributes mail_location = mdbox:~/mdbox:ALT=~/Archive/tape/mdbox mail_plugins = acl stats mail_crypt mailbox_list_index = yes mdbox_rotate_size = 16 M namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { driver = pam } plugin { acl = vfile mail_crypt_global_private_key = <### redacted ### mail_crypt_global_public_key = <### redacted ### mail_crypt_save_version = 2 stats_memory_limit = 16 M stats_refresh = 30 secs stats_track_cmds = yes } postmaster_address = ### redacted ### protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { process_limit = 200 process_min_avail = 4 service_count = 1 } service stats { fifo_listener stats-mail { group = users mode = 01224 } } ssl = required ssl_cert = <### redacted ### ssl_key =  # hidden, use -P to show it ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv2 !SSLv3 !TLSv1 userdb { driver = passwd } verbose_proctitle = yes protocol imap { imap_idle_notify_interval = 10 mins imap_metadata = yes mail_max_userip_connections = 20 mail_plugins = acl stats mail_crypt imap_acl imap_stats }

Regards,

Here is the gdb output for this issue - the status values in frame 6 look wrong:

GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/... Reading symbols from /usr/bin/doveadm...Reading symbols from /usr/lib/debug/usr/bin/doveadm.debug...done. done. (gdb) run Starting program: /usr/bin/doveadm batch : mailbox status all inbox process 9660 is executing new program: /usr/bin/doveconf process 9660 is executing new program: /usr/bin/doveadm [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". doveadm(jc): Panic: file mail-storage.c: line 875 (mailbox_set_reason): assertion failed: (reason != NULL) doveadm(jc): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xa0e9e) [0x7ffff73ade9e] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7ffff73adf0a] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7ffff733dc97] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x49856) [0x7ffff768b856] -> /usr/bin/doveadm(+0x348e2) [0x5555555888e2] -> /usr/bin/doveadm(+0x2e1ad) [0x5555555821ad] -> /usr/bin/doveadm(+0x2bd2c) [0x55555557fd2c] -> /usr/bin/doveadm(+0x2c92a) [0x55555558092a] -> /usr/bin/doveadm(doveadm_mail_try_run+0x215) [0x555555581285] -> /usr/bin/doveadm(main+0x46a) [0x5555555703fa] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7ffff6f623d5] -> /usr/bin/doveadm(+0x1c565) [0x555555570565]

Program received signal SIGABRT, Aborted. 0x00007ffff6f76207 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55 55 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig); (gdb) bt full #0 0x00007ffff6f76207 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55 resultvar = 0 pid = 9660 selftid = 9660 #1 0x00007ffff6f778f8 in __GI_abort () at abort.c:90 save_stage = 2 act = {__sigaction_handler = {sa_handler = 0x1, sa_sigaction = 0x1}, sa_mask = {__val = {7882532336211813423, 173, 140737344948160, 93824994941896, 140737341211075, 140737488346672, 93824994941896, 513, 5223750495495426816, 93824994941896, 140737341394611, 93824994941896, 140737488346768, 140737488347136, 140737341394953, 93824994941896}}, sa_flags = -147157558, sa_restorer = 0x7fffffffde01} sigs = {__val = {32, 0 }} #2 0x00007ffff73ade96 in default_fatal_finish (type=<optimized out>, status=status@entry=0) at failures.c:201 backtrace = 0x5555557e9c00 "/usr/lib64/dovecot/libdovecot.so.0(+0xa0e9e) [0x7ffff73ade9e] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7ffff73adf0a] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7ff"... #3 0x00007ffff73adf0a in default_fatal_handler (ctx=0x7fffffffdee0, format=<optimized out>, args=<optimized out>) at failures.c:215 status = 0 #4 0x00007ffff733dc97 in i_panic (format=format@entry=0x7ffff773c7c0 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0, timestamp_usecs = 0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fffffffdfe0, reg_save_area = 0x7fffffffdf20}} #5 0x00007ffff768b856 in mailbox_set_reason (box=<optimized out>, reason=<optimized out>) at mail-storage.c:875 __FUNCTION__ = "mailbox_set_reason" #6 0x00005555555888e2 in status_mailbox (info=0x555555838c58, ctx=0x5555557fe238) at doveadm-mail-mailbox-status.c:135 box = 0x555555838db8 status = {messages = 4151561466, recent = 32767, unseen = 1434453976, uidvalidity = 21845, uidnext = 0, first_unseen_seq = 0, first_recent_uid = 4294959296, last_cached_seq = 32767, highest_modseq = 93824995016992, highest_pvt_modseq = 93824995038873, keywords = 0x55555582cc50, permanent_flags = 4150865472, flags = 32767, permanent_keywords = 0, allow_new_keywords = 0, nonpermanent_modseqs = 0, no_modseq_tracking = 0, have_guids = 0, have_save_guids = 0, have_only_guid128 = 1} metadata = {guid = "p\214~UUU\000\000\300e\177UUU\000", virtual_size = 140737488347344, physical_size = 553903933854, first_save_date = 93824995035120, cache_fields = 0x20, precache_fields = (unknown: 0), backend_ns_prefix = 0x0, backend_ns_type = (unknown: 48)} #7 cmd_mailbox_status_run (_ctx=0x5555557fe238, user=<optimized out>) at doveadm-mail-mailbox-status.c:172 _data_stack_cur_id = 2 ctx = 0x5555557fe238 iter = 0x555555838c30 info = 0x555555838c58 ret = 0 #8 0x00005555555821ad in cmd_batch_run (_ctx=0x5555557f64c8, user=0x55555582caa8) at doveadm-mail-batch.c:41 cmdp__foreach_end = 0x5555557f66e0 ctx = 0x5555557f64c8 cmdp = 0x5555557f66d8 ret = 0 #9 0x000055555557fd2c in doveadm_mail_next_user (ctx=ctx@entry=0x5555557f64c8, cctx=cctx@entry=0x7fffffffe2b0, error_r=error_r@entry=0x7fffffffe240) at doveadm-mail.c:429 input = {module = 0x0, service = 0x5555555b9e2b "doveadm", username = 0x5555557ef7c3 "jc", session_id = 0x0, session_id_prefix = 0x0, session_create_time = 0, local_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = (unknown: 0), flags_override_remove = (unknown: 0), no_userdb_lookup = 0, debug = 0} error = 0xf9d0