dsync: assert panic on mbox // no INBOX sync on hybrid mbox/maildir

Wed Sep 27 13:47:37 EEST 2017

Hi,
 replying to my own issue:
The dsync crash seems to be triggered when there are emails to copy in
both directions, and it's fairly clear from comments in  the code that
it should be releasing a read lock before trying to grab a write lock.

A work-around for the crash was doing a one-way sync manually. This
allowed the process to complete and my setup is now replicating
properly. I guess as long as I don't trigger changes on both hosts

David

On 26 September 2017 at 22:39, David Gardner <somloieater at gmail.com> wrote:
> Hi,
>  This (hopefully!) might be a case of user error, as I'm new to
> dovecot... Alternatively it might be some confusion in an index, since
> I've been mucking about trying different things in my configuration
> files and had frequent crashes.
>
> I'm trying to sync 2 (LAN) hosts, ideally to have full automatic
> replication happening.
>
> They both have hybrid mbox INBOX and maildir++ folders, using
> namespaces, as suggested in the wiki. I might have deviated the config
> from the wiki version, in trying to get things working.
>
> Both hosts communicate perfectly with an external IMAP client (thunderbird).
> Replication is working, but only partially - it is syncing the folders
> but not INBOX. I've thus been trying various things to try to get the
> INBOX to sync.
>
> The crash is only happening one of the 2 hosts which I've named
> crashhost for this report. The process crashes no matter which host
> starts the sync.
> With a second user, the crash happens on the other host
>
> crashhost: Ubuntu 16.04, x86_64, laptop,  Dovecot 2.2.22, and Dovecot
> 2.2.32 (from debian sid package, locally compiled)
> otherhost: Ubuntu 16.04, x86_64, desktop,  Dovecot 2.2.22, and Dovecot
> 2.2.32 (from debian sid package, locally compiled)
>
>
> RESULTS:
>
> 1) doveadm sync -f -u david tcp:otherhost
> synchronises the folders, but does not touch INBOXes.
>
> 2) doveadm sync -f -m INBOX -u david tcp:otherhost
> Does not touch INBOXes
>
> 3) doveadm sync -n '#mbox/'  -f -u david tcp:otherhost
> dsync-local(david): Panic: file mbox-lock.c: line 800 (mbox_lock):
> assertion failed: (lock_type == F_RDLCK || mbox->mbox_lock_type !=
> F_RDLCK)
>
> I added some debug code, the assertion failure is being triggered by a
> call to mbox-lock() with:
> locktype=F_WRCLK  mbox->mbox_lock_type=F_RDLCK,  mbox->external_transactions=1
>
> The config files on the 2 hosts only differ by the hostnames they're
> supposed to be syncing to.
>
> $ dovecot -n
> # 2.2.32 (dfbe293d4): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.4.20 (7cd71ba)
> # OS: Linux 4.10.0-35-generic x86_64 Ubuntu 16.04.3 LTS
> doveadm_password =  # hidden, use -P to show it
> doveadm_port = 12346
> imapc_host = crashhost
> imapc_password =  # hidden, use -P to show it
> imapc_port = 993
> imapc_ssl = imaps
> imapc_ssl_verify = no
> imapc_user = %u
> mail_location = maildir:~/mail
> mail_plugins = " notify replication"
> mail_privileged_group = mail
> namespace {
>   inbox = no
>   list = yes
>   location =
>   prefix =
>   separator = /
>   subscriptions = yes
> }
> namespace inbox {
>   hidden = yes
>   inbox = yes
>   list = no
>   location = mbox:~/nomail:INBOX=/var/mail/%u:INDEX=/var/spool/mailindex/%u
>   mailbox Drafts {
>     special_use = \Drafts
>   }
>   mailbox Junk {
>     special_use = \Junk
>   }
>   mailbox Sent {
>     special_use = \Sent
>   }
>   mailbox "Sent Messages" {
>     special_use = \Sent
>   }
>   mailbox Trash {
>     special_use = \Trash
>   }
>   prefix = "#mbox/"
>   separator = /
>   type = private
> }
> passdb {
>   args = scheme=CRYPT username_format=%u /etc/dovecot/users
>   driver = passwd-file
> }
> plugin {
>   mail_replica = tcp:otherhost:12346
> }
> protocols = " imap"
> service aggregator {
>   fifo_listener replication-notify-fifo {
>     mode = 0666
>     user = vmail
>   }
>   unix_listener replication-notify {
>     mode = 0666
>     user = vmail
>   }
> }
> service config {
>   unix_listener config {
>     user = vmail
>   }
> }
> service doveadm {
>   inet_listener {
>     port = 12346
>   }
>   user = root
> }
> service replicator {
>   process_min_avail = 1
>   unix_listener replicator-doveadm {
>     mode = 0666
>   }
> }
> ssl = required
> ssl_cert = </etc/dovecot/dovecot.pem
> ssl_client_ca_dir = /etc/ssl
> ssl_key =  # hidden, use -P to show it
> userdb {
>   args = username_format=%u /etc/dovecot/users
>   driver = passwd-file
> }
>
> gdb) bt
> #0  0x00007f37f426e428 in __GI_raise (sig=sig at entry=6) at
> ../sysdeps/unix/sysv/linux/raise.c:54
> #1  0x00007f37f427002a in __GI_abort () at abort.c:89
> #2  0x00007f37f469f7aa in ?? () from /usr/lib/dovecot/libdovecot.so.0
> #3  0x00007f37f469f829 in default_fatal_handler () from
> /usr/lib/dovecot/libdovecot.so.0
> #4  0x00007f37f46302a1 in i_panic () from /usr/lib/dovecot/libdovecot.so.0
> #5  0x00007f37f49ac235 in mbox_lock () from
> /usr/lib/dovecot/libdovecot-storage.so.0
> #6  0x00007f37f49adb11 in mbox_save_begin () from
> /usr/lib/dovecot/libdovecot-storage.so.0
> #7  0x00007f37f49769a1 in mailbox_save_begin () from
> /usr/lib/dovecot/libdovecot-storage.so.0
> #8  0x00007f37f496a2e8 in mail_storage_copy () from
> /usr/lib/dovecot/libdovecot-storage.so.0
> #9  0x00007f37f3e332a5 in ?? () from
> /usr/lib/dovecot/modules/lib15_notify_plugin.so
> #10 0x00007f37f4976b73 in ?? () from /usr/lib/dovecot/libdovecot-storage.so.0
> #11 0x00007f37f4976d89 in mailbox_move () from
> /usr/lib/dovecot/libdovecot-storage.so.0
> #12 0x000055b45f0da448 in ?? ()
> #13 0x000055b45f0db35a in ?? ()
> #14 0x000055b45f0ddbc0 in dsync_mailbox_import_changes_finish ()
> #15 0x000055b45f0d8645 in dsync_brain_sync_mails ()
> #16 0x000055b45f0d3df8 in dsync_brain_run ()
> #17 0x000055b45f0d4140 in ?? ()
> #18 0x000055b45f0ea3ff in ?? ()
> #19 0x00007f37f46b4c82 in io_loop_call_io () from
> /usr/lib/dovecot/libdovecot.so.0
> #20 0x00007f37f46b6309 in io_loop_handler_run_internal () from
> /usr/lib/dovecot/libdovecot.so.0
> #21 0x00007f37f46b4d1c in io_loop_handler_run () from
> /usr/lib/dovecot/libdovecot.so.0
> #22 0x00007f37f46b4ec8 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0
> #23 0x000055b45f0bb4c2 in ?? ()
> #24 0x000055b45f0bd91c in ?? ()
> #25 0x000055b45f0be5aa in ?? ()
> #26 0x000055b45f0beeda in doveadm_mail_try_run ()
> #27 0x000055b45f0ae47c in main ()