Once you cache grows bigger than 0x4000000 you have problems
---Aki TuomiDovecot oy -------- Original message --------From: Ian Bobbitt <ibobbitt@globalnoc.iu.edu> Date: 23/02/2018 20:33 (GMT+02:00) To: dovecot@dovecot.org Subject: Re: Assertion during dsync receive
Thanks. I've had the user clear out that mailbox, and replication is
working fine for them again.
Is there a better way to catch this than watch for crashes and read
the backtrace to find what mailbox needs to be shrunk?
Where is the threshold for "too big"?
-- Ian
On 2/23/18 11:33 AM, Aki Tuomi wrote:
The mailbox is too big.
---
Aki Tuomi
Dovecot oy
-------- Original message --------
From: Ian Bobbitt <ibobbitt@globalnoc.iu.edu>
Date: 23/02/2018 17:52 (GMT+02:00)
To: dovecot@dovecot.org
Subject: Assertion during dsync receive
Hi,
I'm getting an assertion failed on the receiving side, causing
syncs to fail for one user. The servers are setup so that
only one is receiving any traffic other than replication at any
time. The one that's only receiving replications is the
one that's failing.
I've tried deleting the user's home on the receiving server, but
it still crashes during the sync. Oddly, the user's
home is 7.4G on the sending server, but ends up at 42G on the
receiving side, even after deleting and trying a fresh sync.
The mailbox implicated in the backtrace ("Spam") does have a very
large number of messages in it.
On sender:
Spam messages=1217764 recent=0 uidnext=1218103
uidvalidity=1379509105 unseen=16 highestmodseq=744588
vsize=34468460093
guid=090ed93a7a055559abf10200fdf6807a firstsaved=1498744186
On receiver:
Spam messages=1217766 recent=352 uidnext=1218105
uidvalidity=1379509105 unseen=16 highestmodseq=744589
vsize=34468496809
guid=090ed93a7a055559abf10200fdf6807a firstsaved=1519396172
Feb 23 14:57:20 dovecot: dsync-local(bgeels): Warning: fscking
index file
/gnoc/mail/home/bgeels/mail/storage/dovecot.map.index
Feb 23 14:57:20 dovecot: dsync-local(bgeels): Warning: mdbox
/gnoc/mail/home/bgeels/mail/storage: rebuilding indexes
Feb 23 14:57:33 dovecot: dsync-local(bgeels): Panic: file
mail-index-util.c: line 10 (mail_index_uint32_to_offset):
assertion failed: (offset < 0x40000000)
Feb 23 14:57:33 dovecot: dsync-local(bgeels): Error: Raw
backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x9f3de)
[0x7feb584143de] ->
/usr/lib64/dovecot/libdovecot.so.0(+0x9f4be) [0x7feb584144be]
->
/usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7feb583a577c]
->/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0xa0) [0x7feb587906d0] ->
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_cache_compress+0x854) [0x7feb58774f34] ->
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x25f) [0x7feb587884ff] ->
/usr/lib64/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x10de) [0x7feb5870b3ae] ->
/usr/lib64/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x858)
[0x7feb5870ccd8] ->
/usr/lib64/dovecot/libdovecot-storage.so.0(mdbox_sync+0x4c)
[0x7feb5870ce7c] ->/usr/lib64/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x4b) [0x7feb5870cf3b] ->
/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44)
[0x7feb586f2834] ->
/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37)
[0x7feb586f28d7] ->
dovecot/doveadm-server(dsync_mailbox_import_deinit+0x475)
[0x445495] -> dovecot/doveadm-server() [0x43edc0] ->
dovecot/doveadm-server(dsync_brain_sync_mails+0x743) [0x43f653]
-> dovecot/doveadm-server(dsync_brain_run+0x541)
[0x43acf1] -> dovecot/doveadm-server() [0x43b070] ->
dovecot/doveadm-server() [0x44fe5f] ->
/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x52)
[0x7feb58429cd2] ->/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x10f) [0x7feb5842b3bf] ->
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c)
[0x7feb58429d6c] ->
/usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38)
[0x7feb58429f28] -> dovecot/doveadm-server() [0x4209c5] ->
dovecot/doveadm-server() [0x422df6] -> dovecot/doveadm-server()
[0x4377f4] ->
/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x52)
[0x7feb58429cd2] ->/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x10f) [0x7feb5842b3bf] ->
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c)
[0x7feb58429d6c]
Feb 23 14:57:33 dsync-local(bgeels): Fatal: master:
service(doveadm): child 82098 killed with signal 6 (core dumped)
I've attached the output of `doveconf -n` and the full backtrace
from a core dump.
Dovecot 2.2.33.2 (GhettoForge package)
CentOS 7 x86_64
XFS, no NFS.
-- Ian