[Dovecot] scalability and arhive ideas

[Stephen Usher]

fernando at dfcom.com.br wrote:
> I´m reading the past topics related to archive and scalability of dovecot,
> they are all very interesting. Here, I´m using two dovecot proxies in
> front of five storages pairs, and we split the domain´s accounts among
> those servers. So, we can share the i/o load and if one server goes down
> only few accounts of the domain stops (not all of them).
> 
> But, we began to have space problems - and the solution would be insert
> more and more storage servers. So I was searching for some archive
> solutions (hard links - at S.O level, or some dovecot extension). A friend
> told me that he knows an ISP that share even the mailbox of the users
> among many servers -
> 
> this is very weird and (at same time) very interesting approach. Instead
> of put all messages into one maildir and this maildir into one server,
> this "maildir" (?) is spplited among many servers - so, if one servers
> fails the account is still acessible and they move old/big messages to a
> new "cheap" storage - archiving transparently.

Surely, other than the possibility of archiving a copy in a separate location at 
delivery time, everything else here is better done by a high-availability 
clustered SAN and *not* by an application?

Archival is a valuable thing to have. Being able to, on delivery, deposit a 
separate copy elsewhere (without necessarily indexing it etc.) allows for 
near-line back-up or storage where legal or corporate regulation require.

(I'm currently doing this using a cron job and a program I've written which 
checks to see if there are any new messages in everyone's inbox Maildirs and 
then hard-links them into a separate directory structure once a minute. Messages 
which disappear from the true inbox are then kept for a further 90 days. This 
allows users to recover messages that they may have accidentally deleted from 
their inbox.)

Oh, and with reference to the second paragraph... hard links only work on a 
single filesystem, not across multiple filesystems or servers.

Steve
-- 
---------------------------------------------------------------------------
IT Systems Administrator,                     E-Mail:- steve at earth.ox.ac.uk
Department of Earth Sciences,                    Tel:-   +44 (0)1865 282110
University of Oxford, Parks Road, Oxford, UK.    Fax:-   +44 (0)1865 272072