Director+NFS Experiences

[Mark Moseley]

On Thu, Feb 23, 2017 at 3:15 PM, Timo Sirainen <tss at iki.fi> wrote:

> On 24 Feb 2017, at 0.08, Mark Moseley <moseleymark at gmail.com> wrote:
> >
> > As someone who is about to begin the process of moving from maildir to
> > mdbox on NFS (and therefore just about to start the 'director-ization' of
> > everything) for ~6.5m mailboxes, I'm curious if anyone can share any
> > experiences with it. The list is surprisingly quiet about this subject,
> and
> > articles on google are mainly just about setting director up. I've yet to
> > stumble across an article about someone's experiences with it.
> >
> > * How big of a director cluster do you use? I'm going to have millions of
> > mailboxes behind 10 directors.
>
> I wouldn't use more than 10.
>
>
Cool



> > I'm guessing that's plenty. It's actually split over two datacenters.
>
> Two datacenters in the same director ring? This is dangerous. if there's a
> network connectivity problem between them, they split into two separate
> rings and start redirecting users to different backends.
>

I was unclear. The two director rings are unrelated and won't ever need to
talk to each other. I only mentioned the two rings to point out that all
6.5m mailboxes weren't behind one ring, but rather split between two



>
> > * Do you have consistent hashing turned on? I can't think of any reason
> not
> > to have it turned on, but who knows
>
> Definitely turn it on. The setting only exists because of backwards
> compatibility and will be removed at some point.
>
>
Out of curiosity (and possibly extremely naive), unless you've moved a
mailbox via 'doveadm director', if someone is pointed to a box via
consistent hashing, why would the directors need to share that mailbox
mapping? Again, assuming they're not moved (I'm also assuming that the
mailbox would always, by default, hash to the same value in the consistent
hash), isn't their hashing all that's needed to get to the right backend?
I.e. "I know what the mailbox hashes to, and I know what backend that hash
points at, so I'm done", in which case, no need to communicate to the other
directors. I could see that if you moved someone, it *would* need to
communicate that mapping. Then the only maps traded by directors would be
the consistent hash boundaries *plus* any "moved" mailboxes. Again, just
curious.