Hi,
I'm getting an assertion failed on the receiving side, causing syncs to fail for one user. The servers are setup so that only one is receiving any traffic other than replication at any time. The one that's only receiving replications is the one that's failing.
I've tried deleting the user's home on the receiving server, but it still crashes during the sync. Oddly, the user's home is 7.4G on the sending server, but ends up at 42G on the receiving side, even after deleting and trying a fresh sync.
The mailbox implicated in the backtrace ("Spam") does have a very large number of messages in it. On sender: Spam messages=1217764 recent=0 uidnext=1218103 uidvalidity=1379509105 unseen=16 highestmodseq=744588 vsize=34468460093 guid=090ed93a7a055559abf10200fdf6807a firstsaved=1498744186 On receiver: Spam messages=1217766 recent=352 uidnext=1218105 uidvalidity=1379509105 unseen=16 highestmodseq=744589 vsize=34468496809 guid=090ed93a7a055559abf10200fdf6807a firstsaved=1519396172
Feb 23 14:57:20 dovecot: dsync-local(bgeels): Warning: fscking index file /gnoc/mail/home/bgeels/mail/storage/dovecot.map.index Feb 23 14:57:20 dovecot: dsync-local(bgeels): Warning: mdbox /gnoc/mail/home/bgeels/mail/storage: rebuilding indexes Feb 23 14:57:33 dovecot: dsync-local(bgeels): Panic: file mail-index-util.c: line 10 (mail_index_uint32_to_offset): assertion failed: (offset < 0x40000000) Feb 23 14:57:33 dovecot: dsync-local(bgeels): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x9f3de) [0x7feb584143de] -> /usr/lib64/dovecot/libdovecot.so.0(+0x9f4be) [0x7feb584144be] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7feb583a577c] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0xa0) [0x7feb587906d0] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_cache_compress+0x854) [0x7feb58774f34] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x25f) [0x7feb587884ff] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x10de) [0x7feb5870b3ae] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x858) [0x7feb5870ccd8] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mdbox_sync+0x4c) [0x7feb5870ce7c] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x4b) [0x7feb5870cf3b] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) [0x7feb586f2834] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) [0x7feb586f28d7] -> dovecot/doveadm-server(dsync_mailbox_import_deinit+0x475) [0x445495] -> dovecot/doveadm-server() [0x43edc0] -> dovecot/doveadm-server(dsync_brain_sync_mails+0x743) [0x43f653] -> dovecot/doveadm-server(dsync_brain_run+0x541) [0x43acf1] -> dovecot/doveadm-server() [0x43b070] -> dovecot/doveadm-server() [0x44fe5f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7feb58429cd2] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x10f) [0x7feb5842b3bf] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7feb58429d6c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7feb58429f28] -> dovecot/doveadm-server() [0x4209c5] -> dovecot/doveadm-server() [0x422df6] -> dovecot/doveadm-server() [0x4377f4] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7feb58429cd2] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x10f) [0x7feb5842b3bf] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7feb58429d6c] Feb 23 14:57:33 dsync-local(bgeels): Fatal: master: service(doveadm): child 82098 killed with signal 6 (core dumped)
I've attached the output of doveconf -n and the full backtrace from a core dump.
Dovecot 2.2.33.2 (GhettoForge package) CentOS 7 x86_64 XFS, no NFS.
-- Ian