[Dovecot] Creating an IMAP repo for ~100 users need some advice

[Kaya Saman]

Thanks so much Sven for your indepth and complete responses!



<snip>
> Question: do you need public or shared folders?

I don't need anything apart from an IMAP storage solution. I don't 
intend to tie in Dovecot with an MTA either as I will simply be using 
this for storage.

Long story but we don't have any control over our mail server which is 
handled by the parent company abroad and is on MS Exchange.

To use an IMAP storage solution is the only way to get rid of pesky MS 
.pst files which have been causing everyone grief and havoc.

>
> Using samba and winbindd to join a domain creates real users on your
> server and as far as I know configuring shared folders with real users
> is a bit of a pain, especially of you need shared flags (like Seen,
> Replied, etc.) (Someone [Timo?] please correct me.)

Actually we might have an LDAP server already taking care of the 
AD<->UNIX integration..... I don't know yet it's only my first week :-)

>
>>>> Additionally what would be the best method to store the **mail**
>>>> information? - as in MySQL database or Maildir format; coinciding
>>>> with this what is the best backup method in order to be able to do
>>>> 'dump' backups or restore single emails??
>>> Storing mails inside SQL? Not supported by dovecot and not very wise,
>>> IMHO. DBmail does this, but to be honest, I never heard any good
>>> feedback from admins using that product. From what I have been told, you
>>> need quite the beefy server to get a decent performance out of DBmail,
>>> compared to the needs of a "traditional" setup like with dovecot or
>>> courier-mail, but I digress.
>>>
>>> To have a consistent backup, your mail storage should be able to
>>> snapshot the volume the mail is stored on, so use LVM or an external
>>> storage unit capable of snapshots.
>> Hmm..... so FreeBSD coupled together with a ZFS repo for mail should
>> take care of 'Snapshot' issues.
> Yes. Or using LVM on Linux.

Yeah.... true but I specified ZFS as I'm a fan and also am quite 
comfortable with Solaris/*BSD too......


>
>>> Then backup the content of the snapshot using any program you like.
>>> I use Bacula for long-term offsite storage and a local rsnapshot to
>>> keep 7 days worth of mail for a quick restore.
>> To be honest I was considering rsync'ing the dir containing users
>> mailboxes to either another storage pool or server.
> No need to rsync, if you use ZFS. Just create a new snapshot and you are
> done. Bet thing about ZFS: you get deduplication for free, so the needed
> space to store the backups will not grow as fast.

Ok so that solves that! :-)

> But you still may want to store the mails offsite/offserver for desaster
> recovery.

They are currently being stored on the parent company mail server so 
this will be the/off-site/ disaster recovery system in a way :-P

>
> Either use doveadm backup for that purpose or use rsnapshot, again
> gaining you deduplication on the target server.

I will research this - thank you for that info :-)

>
>>> Whether you are able to restore single mails or the complete storage is
>>> no property or feature of the mailbox format itself.
>>>
>>> Some formats are simpler to handle, like Maildir++, where you just drop
>>> the file containing a mail into a directory.
>> You mention Maildir++... is this Maildir format or something new which I
>> haven't heard about yet?
> Maildir++ extends the original Maildir with things like Quota and ACLs
> and was first implemented in Courier.
> http://www.courier-mta.org/imap/README.maildirquota.html
>
> All current MTAs and POP3/IMAP servers implement this variant.
>
> Depending on the amount of mail a user collects inside a folder, Maildir
> is not the best storage format. You may want to check into mdbox, if
> your users are kind of "mail hoarders" (like some of my users are).
>
> In my opinion, Maildir has outlived its usefullnes. It was fine when
> users had 1,000 mails in some 10 folders, but today, users collect over
> 100,000 mails a year and Maildir is causing serious I/O trouble and the
> need to heavily fine tune your storage and filesystems to cope with
> those demands.
>
> I cannot thank Timo enough for inventing mdbox, as this format breaks
> this viciuos cycle and, as someone else said "it ends the battle at the
> I/O front forever".

So mdbox is a 'new' mailbox standard? ie. one can replace Maildir format 
with this and use mdbox instead. {Note to self: time to browse!}

Since where I'm implementing this is mainly an MS based environment they 
are concerned about /flat/ files.... which MS seems to typically do 
(although never used MS before so I wouldn't know). So there is some 
concern over performance, efficiency and manageability.

However, if like you say mdbox is the way to go then I will put a strong 
case together!

>
> Grüße,
> Sven.
>

Regards,


Kaya