[Dovecot] Backing Up
Roderick A. Anderson
raanders at acm.org
Thu Oct 30 22:34:50 EET 2008
mikkel at euro123.dk wrote:
>> On Oct 30, 2008, at 2:35 PM, mikkel at euro123.dk wrote:
>>> Maildir is nice compared to mbox but it really isn’t optimal. In days
>>> where IOPS is the most difficult resource to get into your server (and
>>> dovecot already using close to nothing in terms of CPU time and
>>> having one file per e-mail is less than sub-optimal especially when a
>>> large amount of users just downloads the whole mailbox using POP3
>>> (not to
>>> mention backing up Maildirs).
>> It seems to me that a database like Postgres or MySQL would be the
>> best bet.
> That's a matter of opinion. Moving mail storage to a database would
> probably be the last thing I would ever do (I'm not saying it's not the
> right thing for some people. I'm just not one of them).
> I'm using mysql for storing the users database but that’s another story.
> Adding a database is one additional level of complexity. One more program
> to govern. In my opinion it's nice to know that as long as the disk is
> readable nothing can go completely wrong.
I have to jump in here and go a bit tangential by saying there are
databases and want-to-be's.
> The database in my case would be roughly 400 GB holding some 60 million
Fair sized but not really big.
> Just imagine if one single byte got written to the wrong place. Power
> outage, OS crash, software bug or whatever could easily result in this (I
> regularly experience mysql tables that crash on their own from heavy use).
> Having to run a repair on a table of that size whilst all users are eager
> to get to their data must be a nightmare of proportions.
There is the difference between an enterprise database and MySQL. Yes,
yes, yes lots of /enterprises/ run applications that use MySQL but most
of those apps have throw away data or they are not using the free
version of MySQL.
> Just imagine backing the thing up, exporting 60.000.000 SQL queries.
> Not to say importing them again if something should go really wrong.
> Actually I'n not even sure it would be faster. When the index files grow
> to several gigabytes they kind of loose their purpose.
There are many businesses backing up way-more data than that and it it
isn't 60,000,000 queries -- it is one command. But if you use serious
hardware "backing up" isn't really needed. RAID, redundant/hot-swap
servers, etc. make backing up /extra redundancy/. :-)
And I bring this up because "Archiveopteryx"
<http://www.archiveopteryx.org/> uses a database - PostgreSQL.
> Maildir is very resilient to various errors. It is virtually impossible to
> corrupt a maildir (at least I've never experienced anything).
> Also you can backup up the thing without worrying about anything accessing
> it at the same time.
> Mbox less so but still a lot better than having one huge database.
> Dbox would be the ultimate compromise between crash resilience and a low
> number of files (not to mention the enormous potential for speed gains).
> Regards, Mikkel
More information about the dovecot