[Dovecot] new crashes: is the index/mail cache endian neutral?

Part of our migration plan takes our users from one endianness to another (big to little). Will the index and mail cache files survive? I'm seeing some new core dumps as the first test user is migrated, which makes me think... not. What about if we want to build dovecot 64 bit in the future? Will that cause problems too?

Stack trace is below the log messages. I've edited the user's name out and replaced it with foobar, and cleansed out a few sensitive things.

    -dp

dovecot: Mar 01 08:46:49 Info: imap-login: Login: user=<foobar>, method=PLAIN, r ip=WWW.XXX.YYY.ZZ, lip=AAA.BBB.CC.DD, TLS dovecot: Mar 01 08:46:49 Info: IMAP(foobar): Effective uid=88442, gid=10, home=/ home/foobar dovecot: Mar 01 08:46:49 Info: IMAP(foobar): mbox: data=/home/foobar:INBOX=/var/ mail/foobar:INDEX=/home/foobar/Mail/imapd-indices dovecot: Mar 01 08:46:49 Info: IMAP(foobar): mbox: root=/home/foobar, index=/hom e/foobar/Mail/imapd-indices, inbox=/var/mail/foobar dovecot: Mar 01 08:46:50 Error: IMAP(foobar): Corrupted index file /home/foobar/ Mail/imapd-indices/.imap/INBOX/dovecot.index: messages_count too large (0 > 0) dovecot: Mar 01 08:46:50 Error: IMAP(foobar): Corrupted index file /home/foobar/ Mail/imapd-indices/.imap/INBOX/dovecot.index: messages_count too large (0 > 0) dovecot: Mar 01 08:46:50 Error: IMAP(foobar): Corrupted index cache file /home/f oobar/Mail/imapd-indices/.imap/INBOX/dovecot.index.cache: field header points ou tside file dovecot: Mar 01 08:46:51 Error: IMAP(foobar): Corrupted transaction log file /ho me/foobar/Mail/imapd-indices/.imap/INBOX/dovecot.index.log: unknown record type 0x10 dovecot: Mar 01 08:46:51 Warning: IMAP(foobar): fscking index file /home/foobar/ Mail/imapd-indices/.imap/INBOX/dovecot.index dovecot: Mar 01 08:46:51 Error: IMAP(foobar): Unexpected transaction log desync with index /home/foobar/Mail/imapd-indices/.imap/INBOX/dovecot.index dovecot: Mar 01 08:46:51 Info: IMAP(foobar): Disconnected: Mailbox is in inconsi stent state, please relogin. dovecot: Mar 01 08:48:06 Info: imap-login: Login: user=<foobar>, method=PLAIN, r ip=WWW.XXX.YYY.ZZ, lip=AAA.BBB.CC.DD, TLS dovecot: Mar 01 08:48:06 Info: IMAP(foobar): Effective uid=88442, gid=10, home=/ home/foobar dovecot: Mar 01 08:48:06 Info: IMAP(foobar): mbox: data=/home/foobar:INBOX=/var/ mail/foobar:INDEX=/home/foobar/Mail/imapd-indices dovecot: Mar 01 08:48:06 Info: IMAP(foobar): mbox: root=/home/foobar, index=/hom e/foobar/Mail/imapd-indices, inbox=/var/mail/foobar dovecot: Mar 01 09:07:09 Error: IMAP(foobar): Corrupted index file /home/foobar/ Mail/imapd-indices/Mail/.imap/zfs/dovecot.index: messages_count too large (27853 45536 > 1) dovecot: Mar 01 09:07:10 Error: IMAP(foobar): Corrupted index cache file /home/f oobar/Mail/imapd-indices/Mail/.imap/zfs/dovecot.index.cache: field header points outside file dovecot: Mar 01 09:20:44 Error: IMAP(foobar): Corrupted index file /home/foobar/ Mail/imapd-indices/Mail/.imap/NDMP/dovecot.index: messages_count too large (5200 93696 > 3) dovecot: Mar 01 09:20:44 Error: IMAP(foobar): Corrupted transaction log file /ho me/foobar/Mail/imapd-indices/Mail/.imap/NDMP/dovecot.index.log: start_offset (61 44) > file size (1768) dovecot: Mar 01 09:20:44 Warning: IMAP(foobar): fscking index file /home/foobar/ Mail/imapd-indices/Mail/.imap/NDMP/dovecot.index dovecot: Mar 01 09:20:44 Error: child 111112 (imap) killed with signal 11

    -dp

#0 0x0809eaa1 in mail_index_lock_exclusive (index=0x8117c70, lock_id_r=0x8047490) at mail-index-lock.c:326 ret = 0 __PRETTY_FUNCTION__ = "mail_index_lock_exclusive" #1 0x0809e15e in mail_index_fsck (index=0x8117c70) at mail-index-fsck.c:169 error = 0x80e9498 "" lock_id = 135173240 file_seq = 16777216 file_offset = 6144 ret = 134509928 lock_log = true #2 0x080a5c49 in mail_transaction_log_file_set_corrupted (file=0x8101c30, fmt=0x80d51a8 "start_offset (%llu) > file size (%llu)") at mail-transaction-log.c:64 No locals. #3 0x080a785e in mail_transaction_log_file_map (file=0x8101c30, start_offset=6144, end_offset=18446744073709551615) at mail-transaction-log.c:1313 index = (struct mail_index *) 0x8117c70 st = {st_dev = 47777187, st_pad1 = {0, 0, 0}, st_ino = 15761, st_mode = 33152, st_nlink = 1, st_uid = 88442, st_gid = 10, st_rdev = 4294967295, st_pad2 = {0, 0}, st_size = 1768, st_atim = { tv_sec = 1172769644, tv_nsec = 151892829}, st_mtim = {tv_sec = 1172522367, tv_nsec = 0}, st_ctim = {tv_sec = 1172732240, tv_nsec = 566025559}, st_blksize = 2048, st_blocks = 4, st_fstype = "lofs", '\0' , st_pad4 = {0, 0, 0, 0, 0, 0, 0, 0}} ret = 1768 use_mmap = 1 __PRETTY_FUNCTION__ = "mail_transaction_log_file_map" #4 0x080a7bca in mail_transaction_log_sync_lock (log=0x8101bd8, file_seq_r=0x0, file_offset_r=0x0) at mail-transaction-log.c:1373 __PRETTY_FUNCTION__ = "mail_transaction_log_sync_lock" #5 0x0809dad3 in mail_index_open (index=0x8117c70, flags=17, lock_method=MAIL_INDEX_LOCK_FCNTL) at mail-index.c:1423 i = 0 ret = 135294216 #6 0x08096241 in index_storage_mailbox_init (ibox=0x81164b8, name=0x0, flags=0, move_to_memory=false) at index-storage.c:360 storage = (struct mail_storage *) 0x80f62a0 index_flags = 17 lock_method = MAIL_INDEX_LOCK_FCNTL ret = 0 __PRETTY_FUNCTION__ = "index_storage_mailbox_init" #7 0x0807cbfc in mbox_alloc (storage=0x80f62a0, index=0x0, name=0x80f68b0 "Mail/NDMP", path=0x80e92f0 "/home/foobar/Mail/NDMP", flags=14) at mbox-storage.c:546 mbox = (struct mbox_mailbox *) 0x81164b8 pool = 0x1 #8 0x0807cd7c in mbox_open (storage=0x80f62a0, name=0x80f68b0 "Mail/NDMP", flags=14) at mbox-storage.c:662 mbox = (struct mbox_mailbox *) 0x80e9310 index = (struct mail_index *) 0x0 path = 0x80e92f0 "/home/foobar/Mail/NDMP" index_dir = 0x80e9310 "/home/foobar/Mail/imapd-indices/Mail/.imap/NDMP" #9 0x0807cf67 in mbox_mailbox_open (_storage=0x80f62a0, name=0x80f68b0 "Mail/NDMP", input=0x0, flags=14) at mbox-storage.c:773 path = 0x80e92d8 "/home/foobar/Mail/NDMP" st = {st_dev = 47777187, st_pad1 = {0, 0, 0}, st_ino = 15330, st_mode = 33152, st_nlink = 1, st_uid = 88442, st_gid = 10, st_rdev = 4294967295, st_pad2 = {0, 0}, st_size = 1189083, st_atim = { tv_sec = 1172732807, tv_nsec = 577163661}, st_mtim = {tv_sec = 1172679608, tv_nsec = 0}, st_ctim = {tv_sec = 1172731930, tv_nsec = 691932658}, st_blksize = 131072, st_blocks = 2224, st_fstype = "lofs", '\0' , st_pad4 = {0, 0, 0, 0, 0, 0, 0, 0}} #10 0x080acbf6 in mailbox_open (storage=0x0, name=0x80f68b0 "Mail/NDMP", input=0x0, flags=14) at mail-storage.c:365 No locals. #11 0x08067f63 in cmd_copy (cmd=0x80f65cc) at cmd-copy.c:83 client = (struct client *) 0x80f6588 storage = (struct mail_storage *) 0x80f62a0 destbox = (struct mailbox *) 0x8 t = (struct mailbox_transaction_context *) 0x806bec0 search_arg = (struct mail_search_arg *) 0x80f8800 messageset = 0x80f68a8 "542506" mailbox = 0x80f68b0 "Mail/NDMP" sync_flags = 0 ret = 134511032 #12 0x0806affd in cmd_uid (cmd=0x80f65cc) at cmd-uid.c:19 cmd_name = 0x80f68a0 "copy" #13 0x0806b9ce in client_handle_input (cmd=0x80f65cc) at client.c:331 client = (struct client *) 0x80f6588 __PRETTY_FUNCTION__ = "client_handle_input" #14 0x0806b962 in client_handle_input (cmd=0x80f65cc) at client.c:388 client = (struct client *) 0x80f6588 __PRETTY_FUNCTION__ = "client_handle_input" #15 0x0806bb10 in _client_input (context=0x80f6588) at client.c:428 client = (struct client *) 0x80f6588 cmd = (struct client_command_context *) 0x80f65cc ret = 2 #16 0x080bdc35 in io_loop_handler_run (ioloop=0x80f3d80) at ioloop-poll.c:199 ctx = (struct ioloop_handler_context *) 0x80f3db8 pollfd = (struct pollfd *) 0x2 tv = {tv_sec = 0, tv_usec = 617423} io = (struct io *) 0x8105460 t_id = 2 msecs = 0 ret = 0 call = true

-- Daniel Price - Solaris Kernel Engineering - dp@eng.sun.com - blogs.sun.com/dp