[Dovecot] Big sites using Dovecot

[David Lee]

On Fri, 22 Sep 2006, Joshua Goodall wrote:

> On Thu, Sep 21, 2006 at 06:14:14PM +0100, Jonathan wrote:
> > Are there any big Dovecot sites (say 50,000-100,000) users on this
> > list?
>
> Yes. Actually, more users than that.
>
> > What sort of hardware do you use.
>
> 2 x NetApp FAS2070c clustered filer heads w/NFS (fast fibrechannel
> disk shelves)
> IBM x336 SMTP/POP3/IMAP front-ends, 4GB RAM, 2x3GHz CPU.
> Load-balancing across front-ends via Cisco 6509 SLB.
> Gigabit Ethernet, jumbo frames.
>
> > What sort of problems do you see?
>
> You need a stable NFS client.  We're using Linux 2.6.16 with
> NFS client patches from Trond Myklebust.
>
> nfs mount options:
> 10.x.x.x:/vol/mailstore /var/mailstore nfs rw,tcp,nfsvers=3,rsize=8192,wsize=8192,nosuid,nodev,soft,intr,noac,actimeo=0 0 0

We, too, are hoping to put ~20,000 accounts onto a set of Linux IMAP
servers, NFS-mounting from NetApp.

Hitherto we have been using Washington IMAP on these machines; Washington
strongly discourages NFS access, but we seem to have been more-or-less OK
with such "noac"-like options (no known lost mail or mailbox corrpution).

Two main reasons prompt our thoughts to migrate from Washington to
dovecot:

1. Performance has been sluggish: high load average, probably caused by
NFS stat activity (itself because of "noac"?).

2. Although older Linuxes (e.g. Redhat 9, 2.4.20-43.9) have been OK, more
recent releases (e.g. FC5, 2.6.16-1) introduced some nasty deadlocking,
requiring machine reboot every day.  (Unacceptable!)


We hope dovecot will improve matters.

Any advice or comments or experiences?

Also, in such a set-up (multiple IMAP/Linux NFS-mouting from NetApp) where
should the dovecot index files be?  NFS from the NetApp?  Or on each Linux
machine (if so, on disk or ramdisk?)?

We plan to use dovecot LDA (rather than "mail" or Washington "tmail")
delivery.

Usually all activity (IMAP, delivery) for a given user should take place
within one machine.  But thi cannot be guaranteed.

Any other hints welcome!


> dovecot.conf
> -  mmap_disable=yes
> -  lock_method=fnctl

Thanks.  Got that!  Nice to have it confirmed.


> Note: without Trond M's NFS client patchset we see comedy VFS
> out-of-sync errors after a few days uptime, resulting in mistaken
> deadlocking of dovecot indices (usually only one user, but always
> a high-volume user) that needs a node reboot to fix, and some
> intervention with the "lock status / break" command on the OnTAP
> command line. With the patches it's been rock solid.

Could you provide more details?  (I wonder if these are related to
deadlock problems we see with Washington/IMAP on 2.6.16?)  Are these
patches in the processes of being pushed into the relevant source codes so
that they will ultimately be unnecessary?


-- 

:  David Lee                                I.T. Service          :
:  Senior Systems Programmer                Computer Centre       :
:                                           Durham University     :
:  http://www.dur.ac.uk/t.d.lee/            South Road            :
:                                           Durham DH1 3LE        :
:  Phone: +44 191 334 2752                  U.K.                  :