[Dovecot] File locking problems and duplicate emails with the snarf plugin

[Adrian Barker]

Two weeks ago we switched all our IMAP users (around 25,000) from the
Washington IMAP server to Dovecot, and since then there have been
several occasions when reading new email just hangs, and duplicate
emails appear. We are using the 'snarf' plugin to copy new email to the
inbox, and it looks as though the problems are caused by file locking.
We have been using the Washington IMAP server for a number of years
without any file locking problems, so the underlying file locking
mechanism is robust. Has anyone else seen this problem ? Our IMAP
servers (5 of them) are a mixture of Sparc Solaris 8 and Solaris 10
machines, and they use NFS to a central mailspool running on AIX. We are
using a mixture of Dovecot 1.0.3 and 1.0.5.

Any help would be appreciated, as the problem has caused a lot of
disruption, but we are reluctant to revert to the Washington IMAP server
because of the performance improvements with Dovecot - there has been a
reduction of over 60% in NFS accesses to the filestore.


dovecot -n gives:

# 1.0.3: /usr/local/etc/dovecot.conf
base_dir: /var/run/dovecot/
listen: *:143
ssl_listen: *:993
ssl_ca_file: /usr/local/ssl/certs/sureserveredu.pem
ssl_cert_file: /usr/local/ssl/certs/imap-server_ucl_ac_uk.crt
ssl_key_file: /usr/local/ssl/certs/imap.key
ssl_cipher_list: MEDIUM:!LOW
disable_plaintext_auth: no
login_dir: /var/run/dovecot//login
login_executable: /usr/local/libexec/dovecot/imap-login
login_user: pp
login_process_per_connection: no
login_processes_count: 30
max_mail_processes: 1500
first_valid_uid: 200
mail_location: mbox:%h/imap-mail:INBOX=/nfs/rcs/mail-spool/mail/%u
mmap_disable: yes
mbox_lock_timeout: 55
mbox_dotlock_change_timeout: 50
mail_drop_priv_before_exec: yes
mail_executable: /usr/local/sbin/dovecot-login.csh
mail_plugins: mbox_snarf
mail_plugin_dir: /usr/local/lib/dovecot
namespace:
  type: private
  separator: /
  inbox: yes
auth default:
  passdb:
    driver: pam
  passdb:
    driver: passwd
  userdb:
    driver: passwd
plugin:
  mbox_snarf: ~/imap-mail/mbox




--

Adrian Barker,
Internet Technology Section
Information Systems
University College London, Gower Street, London WC1E 6BT
External phone: +44 20 7679 5140,  Fax (+44) 20 7388 5406
Internal phone: x 25140
Email: A.Barker at ucl.ac.uk