[Dovecot] Index corruption ?
Hi,
We are using dovecot for some times (a little before 1.0), for IMAP and (a little for the moment) deliver.
We have ~20000 logins/day, Maildirs are over NFS (with indexes too, for the moment...)
Up to 1.0.5, we sometime have this error in maillog:
Nov 1 13:55:41 tic dovecot: IMAP(login): file mail-index-view-sync.c: line 666 (mail_index_view_sync_end): assertion failed: (view->log_file_offset >= view->map->hdr.log_file_int_offset)
I didn't find any bits on google, nor on this list's archive...
# gdb (without debug, I may do it if required)
Program terminated with signal 6, Aborted. Reading symbols from /usr/local/lib/libiconv.so.3...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libiconv.so.3 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x182ad51f in kill () from /lib/libc.so.6 (gdb) bt #0 0x182ad51f in kill () from /lib/libc.so.6 #1 0x182ad4be in raise () from /lib/libc.so.6 #2 0x182ac0e8 in abort () from /lib/libc.so.6 #3 0x080a85da in i_set_failure_syslog () #4 0x080a80e1 in data_stack_deinit () #5 0x08092c97 in mail_index_view_sync_end () #6 0x0808229e in index_mailbox_sync_next () #7 0x0000001c in ?? () #8 0xbfbfe7e8 in ?? () #9 0x080d3140 in ?? () #10 0x080d8044 in ?? () #11 0x0000001c in ?? () #12 0xbfbfe828 in ?? () #13 0x08060761 in imap_sync_deinit () Previous frame inner to this frame (corrupt stack?)
dovecot -n # 1.0.5: /usr/local/etc/dovecot.conf ssl_ca_file: /etc/ssl/cacert.pem ssl_cert_file: /etc/ssl/private/imaps.ec-marseille.fr.pem ssl_key_file: /etc/ssl/private/imaps.ec-marseille.fr.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login login_greeting: IMAP Centrale ready. login_greeting_capability: yes login_process_size: 64 login_processes_count: 20 login_max_processes_count: 512 valid_chroot_dirs: /users:/home max_mail_processes: 2048 verbose_proctitle: yes first_valid_uid: 1000 last_valid_uid: 55000 first_valid_gid: 999 last_valid_gid: 55000 mail_extra_groups: mail mail_location: maildir:%h/Maildir/ mmap_disable: yes lock_method: flock maildir_stat_dirs: yes maildir_copy_with_hardlinks: yes mail_drop_priv_before_exec: yes mail_executable: /usr/local/libexec/dovecot/imap_convert imap_client_workarounds: delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep auth default: verbose: yes worker_max_count: 60 passdb: driver: pam args: imap passdb: driver: ldap args: /usr/local/etc/dovecot-ldap.conf userdb: driver: passwd userdb: driver: ldap args: /usr/local/etc/dovecot-uldap.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 plugin: quota: fs
# dovecot-ldap.conf (dovecot-uldap.conf is a symlink)
hosts = ldapr.serv.int:389 ldap:389
dn = <our binddn>
dnpass = <our pass>
auth_bind = yes
ldap_version = 3
base = ou=People,
--
___________________________________________________________
/ Geoffroy DESVERNAY |
/\ Service info
| Tel: (+33|0)4 91 05 45 24 /
\/ Ecole Centrale de Marseille | Fax: (+33|0)4 91 05 45 98 \/
\ (ex-EGIM) | Mail: dgeo@ec-marseille.fr /
-----------------------------------------------------------
On Fri, 2007-11-02 at 15:05 +0100, Geoffroy Desvernay wrote:
We are using dovecot for some times (a little before 1.0), for IMAP and (a little for the moment) deliver.
We have ~20000 logins/day, Maildirs are over NFS (with indexes too, for the moment...)
With more than one server accessing the same maildir at around the same time? See "attribute cache problems" (and the rest of the page as well) in http://wiki.dovecot.org/NFS
On Fri, Nov 02, 2007 at 05:52:49PM +0200, Timo Sirainen wrote:
On Fri, 2007-11-02 at 15:05 +0100, Geoffroy Desvernay wrote:
We are using dovecot for some times (a little before 1.0), for IMAP and (a little for the moment) deliver.
We have ~20000 logins/day, Maildirs are over NFS (with indexes too, for the moment...)
With more than one server accessing the same maildir at around the same time? See "attribute cache problems" (and the rest of the page as well) in http://wiki.dovecot.org/NFS
I'll bite, I keep hearing mention of "each user is assigned a specific computer which is used whenever possible" and I'm willing to try it out so my indexes are safe. The part where I get stuck is the "whenever possible".
I've looked at using perdition and/or dovecot, but both seem only offer a single IP to connect to per user, and the only workarounds I've heard of is to have an additional process monitor the imap servers and modify the mappings when a server is down. Is there a simpler way to have a proxy attempt a list of servers for each user?
I would just point it back at my hardware load balancer in essence, but for that to work, the server originating the connection cannot host the VIP of the target otherwise the connection will stay local since the VIP would be local (I don't have my traffic flow through the load balancer physically). I could put a bunch of "every server but me" VIPs in the LB, one for each server, but thats a messy hack. I've thought of a handful of different ways I could have the connections go through my planned 4 mail servers but none so far give me complete load balancing, automatic failover, and simple enough to make it worth trying to have only one copy of indexes.
Anyone have any input? Thanks
participants (3)
-
Adam McDougall
-
Geoffroy Desvernay
-
Timo Sirainen