Assertion during dsync receive

[Ian Bobbitt]

Thanks. I've had the user clear out that mailbox, and replication is working fine for them again.

Is there a better way to catch this than watch for crashes and read the backtrace to find what mailbox needs to be shrunk?

Where is the threshold for "too big"?

-- Ian

On 2/23/18 11:33 AM, Aki Tuomi wrote:
> The mailbox is too big.
>
>
>
> ---
> Aki Tuomi
> Dovecot oy
>
> -------- Original message --------
> From: Ian Bobbitt <ibobbitt at globalnoc.iu.edu>
> Date: 23/02/2018 17:52 (GMT+02:00)
> To: dovecot at dovecot.org
> Subject: Assertion during dsync receive
>
> Hi,
>
> I'm getting an assertion failed on the receiving side, causing syncs to fail for one user. The servers are setup so that
> only one is receiving any traffic other than replication at any time. The one that's only receiving replications is the
> one that's failing.
>
> I've tried deleting the user's home on the receiving server, but it still crashes during the sync. Oddly, the user's
> home is 7.4G on the sending server, but ends up at 42G on the receiving side, even after deleting and trying a fresh sync.
>
> The mailbox implicated in the backtrace ("Spam") does have a very large number of messages in it.
> On sender:
> Spam messages=1217764 recent=0 uidnext=1218103 uidvalidity=1379509105 unseen=16 highestmodseq=744588 vsize=34468460093
> guid=090ed93a7a055559abf10200fdf6807a firstsaved=1498744186
> On receiver:
> Spam messages=1217766 recent=352 uidnext=1218105 uidvalidity=1379509105 unseen=16 highestmodseq=744589 vsize=34468496809
> guid=090ed93a7a055559abf10200fdf6807a firstsaved=1519396172
>
> Feb 23 14:57:20 dovecot: dsync-local(bgeels): Warning: fscking index file
> /gnoc/mail/home/bgeels/mail/storage/dovecot.map.index
> Feb 23 14:57:20 dovecot: dsync-local(bgeels): Warning: mdbox /gnoc/mail/home/bgeels/mail/storage: rebuilding indexes
> Feb 23 14:57:33 dovecot: dsync-local(bgeels): Panic: file mail-index-util.c: line 10 (mail_index_uint32_to_offset):
> assertion failed: (offset < 0x40000000)
> Feb 23 14:57:33 dovecot: dsync-local(bgeels): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x9f3de)
> [0x7feb584143de] -> /usr/lib64/dovecot/libdovecot.so.0(+0x9f4be) [0x7feb584144be] ->
> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7feb583a577c] ->
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0xa0) [0x7feb587906d0] ->
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_cache_compress+0x854) [0x7feb58774f34] ->
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x25f) [0x7feb587884ff] ->
> /usr/lib64/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x10de) [0x7feb5870b3ae] ->
> /usr/lib64/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x858) [0x7feb5870ccd8] ->
> /usr/lib64/dovecot/libdovecot-storage.so.0(mdbox_sync+0x4c) [0x7feb5870ce7c] ->
> /usr/lib64/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x4b) [0x7feb5870cf3b] ->
> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) [0x7feb586f2834] ->
> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) [0x7feb586f28d7] ->
> dovecot/doveadm-server(dsync_mailbox_import_deinit+0x475) [0x445495] -> dovecot/doveadm-server() [0x43edc0] ->
> dovecot/doveadm-server(dsync_brain_sync_mails+0x743) [0x43f653] -> dovecot/doveadm-server(dsync_brain_run+0x541)
> [0x43acf1] -> dovecot/doveadm-server() [0x43b070] -> dovecot/doveadm-server() [0x44fe5f] ->
> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7feb58429cd2] ->
> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x10f) [0x7feb5842b3bf] ->
> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7feb58429d6c] ->
> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7feb58429f28] -> dovecot/doveadm-server() [0x4209c5] ->
> dovecot/doveadm-server() [0x422df6] -> dovecot/doveadm-server() [0x4377f4] ->
> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7feb58429cd2] ->
> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x10f) [0x7feb5842b3bf] ->
> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7feb58429d6c]
> Feb 23 14:57:33 dsync-local(bgeels): Fatal: master: service(doveadm): child 82098 killed with signal 6 (core dumped)
>
> I've attached the output of `doveconf -n` and the full backtrace from a core dump.
>
> Dovecot 2.2.33.2 (GhettoForge package)
> CentOS 7 x86_64
> XFS, no NFS.
>
>
> -- Ian

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dovecot.org/pipermail/dovecot/attachments/20180223/98231781/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4090 bytes
Desc: S/MIME Cryptographic Signature
URL: <https://dovecot.org/pipermail/dovecot/attachments/20180223/98231781/attachment-0001.p7s>