[Dovecot] Creating an IMAP repo for ~100 users need some advice

[Sven Hartge]

Kaya Saman <kayasaman at gmail.com> wrote:
> On 03/17/2012 07:36 PM, Sven Hartge wrote:
>> Kaya Saman<kayasaman at gmail.com>  wrote:

>>> I am currently in the process of setting up an IMAP repository for
>>> round 100 users....  Currently the user authentication method is
>>> being handled via a Windows Domain Controller.  The host OS for
>>> Dovecot will either be FreeBSD or CentOS.  Would Dovecot be able to
>>> authenticate to either the DC directly or would we need to go
>>> through LDAP??

>> Why not join the server to the domain and simply use PAM?

>> Using ActiveDirectory through LDAP is a bit of a pain so I would
>> avoid this if I where you.

> I don't actually have much AD/LDAP integration experience so I will
> try your method!

Question: do you need public or shared folders?

Using samba and winbindd to join a domain creates real users on your
server and as far as I know configuring shared folders with real users
is a bit of a pain, especially of you need shared flags (like Seen,
Replied, etc.) (Someone [Timo?] please correct me.)

>>> Additionally what would be the best method to store the **mail**
>>> information? - as in MySQL database or Maildir format; coinciding
>>> with this what is the best backup method in order to be able to do
>>> 'dump' backups or restore single emails??

>> Storing mails inside SQL? Not supported by dovecot and not very wise,
>> IMHO. DBmail does this, but to be honest, I never heard any good
>> feedback from admins using that product. From what I have been told, you
>> need quite the beefy server to get a decent performance out of DBmail,
>> compared to the needs of a "traditional" setup like with dovecot or
>> courier-mail, but I digress.
>>
>> To have a consistent backup, your mail storage should be able to
>> snapshot the volume the mail is stored on, so use LVM or an external
>> storage unit capable of snapshots.

> Hmm..... so FreeBSD coupled together with a ZFS repo for mail should 
> take care of 'Snapshot' issues.

Yes. Or using LVM on Linux.

>> Then backup the content of the snapshot using any program you like.
>> I use Bacula for long-term offsite storage and a local rsnapshot to
>> keep 7 days worth of mail for a quick restore.

> To be honest I was considering rsync'ing the dir containing users
> mailboxes to either another storage pool or server.

No need to rsync, if you use ZFS. Just create a new snapshot and you are
done. Bet thing about ZFS: you get deduplication for free, so the needed
space to store the backups will not grow as fast.

But you still may want to store the mails offsite/offserver for desaster
recovery.

Either use doveadm backup for that purpose or use rsnapshot, again
gaining you deduplication on the target server.

>> Whether you are able to restore single mails or the complete storage is
>> no property or feature of the mailbox format itself.
>>
>> Some formats are simpler to handle, like Maildir++, where you just drop
>> the file containing a mail into a directory.

> You mention Maildir++... is this Maildir format or something new which I 
> haven't heard about yet?

Maildir++ extends the original Maildir with things like Quota and ACLs
and was first implemented in Courier.
http://www.courier-mta.org/imap/README.maildirquota.html

All current MTAs and POP3/IMAP servers implement this variant.

Depending on the amount of mail a user collects inside a folder, Maildir
is not the best storage format. You may want to check into mdbox, if
your users are kind of "mail hoarders" (like some of my users are).

In my opinion, Maildir has outlived its usefullnes. It was fine when
users had 1,000 mails in some 10 folders, but today, users collect over
100,000 mails a year and Maildir is causing serious I/O trouble and the
need to heavily fine tune your storage and filesystems to cope with
those demands.

I cannot thank Timo enough for inventing mdbox, as this format breaks
this viciuos cycle and, as someone else said "it ends the battle at the
I/O front forever".

Grüße,
Sven.

-- 
Sigmentation fault. Core dumped.