[Dovecot] Backing Up

[Roderick A. Anderson]

mikkel at euro123.dk wrote:
>> On Oct 30, 2008, at 2:35 PM, mikkel at euro123.dk wrote:
>>> Maildir is nice compared to mbox but it really isn’t optimal. In days
>>> where IOPS is the most difficult resource to get into your server (and
>>> dovecot already using close to nothing in terms of CPU time and
>>> memory)
>>> having one file per e-mail is less than sub-optimal especially when a
>>> large amount of users just downloads the whole mailbox using POP3
>>> (not to
>>> mention backing up Maildirs).
>>    It seems to me that a database like Postgres or MySQL would be the
>> best bet.
>>
> 
> That's a matter of opinion. Moving mail storage to a database would
> probably be the last thing I would ever do (I'm not saying it's not the
> right thing for some people. I'm just not one of them).
> I'm using mysql for storing the users database but that’s another story.

> Adding a database is one additional level of complexity. One more program
> to govern. In my opinion it's nice to know that as long as the disk is
> readable nothing can go completely wrong.
> 

I have to jump in here and go a bit tangential by saying there are 
databases and want-to-be's.

> The database in my case would be roughly 400 GB holding some 60 million
> records.

Fair sized but not really big.

> Just imagine if one single byte got written to the wrong place. Power
> outage, OS crash, software bug or whatever could easily result in this (I
> regularly experience mysql tables that crash on their own from heavy use).
> Having to run a repair on a table of that size whilst all users are eager
> to get to their data must be a nightmare of proportions.

There is the difference between an enterprise database and MySQL.  Yes, 
yes, yes lots of /enterprises/ run applications that use MySQL but most 
of those apps have throw away data or they are not using the free 
version of MySQL.

> Just imagine backing the thing up, exporting 60.000.000 SQL queries.
> Not to say importing them again if something should go really wrong.
> Actually I'n not even sure it would be faster. When the index files grow
> to several gigabytes they kind of loose their purpose.

There are many businesses backing up way-more data than that and it it 
isn't 60,000,000 queries -- it is one command.  But if you use serious 
hardware "backing up" isn't really needed.  RAID, redundant/hot-swap 
servers, etc. make backing up /extra redundancy/.  :-)

And I bring this up because  "Archiveopteryx" 
<http://www.archiveopteryx.org/> uses a database - PostgreSQL.


Rod
-- 
> 
> 
> Maildir is very resilient to various errors. It is virtually impossible to
> corrupt a maildir (at least I've never experienced anything).
> Also you can backup up the thing without worrying about anything accessing
> it at the same time.
> Mbox less so but still a lot better than having one huge database.
> 
> Dbox would be the ultimate compromise between crash resilience and a low
> number of files (not to mention the enormous potential for speed gains).
> 
> Regards, Mikkel
> 
>