mikkel@euro123.dk wrote:
On Oct 30, 2008, at 2:35 PM, mikkel@euro123.dk wrote:
Maildir is nice compared to mbox but it really isn’t optimal. In days where IOPS is the most difficult resource to get into your server (and dovecot already using close to nothing in terms of CPU time and memory) having one file per e-mail is less than sub-optimal especially when a large amount of users just downloads the whole mailbox using POP3 (not to mention backing up Maildirs). It seems to me that a database like Postgres or MySQL would be the best bet.
That's a matter of opinion. Moving mail storage to a database would probably be the last thing I would ever do (I'm not saying it's not the right thing for some people. I'm just not one of them). I'm using mysql for storing the users database but that’s another story.
Adding a database is one additional level of complexity. One more program to govern. In my opinion it's nice to know that as long as the disk is readable nothing can go completely wrong.
I have to jump in here and go a bit tangential by saying there are databases and want-to-be's.
The database in my case would be roughly 400 GB holding some 60 million records.
Fair sized but not really big.
Just imagine if one single byte got written to the wrong place. Power outage, OS crash, software bug or whatever could easily result in this (I regularly experience mysql tables that crash on their own from heavy use). Having to run a repair on a table of that size whilst all users are eager to get to their data must be a nightmare of proportions.
There is the difference between an enterprise database and MySQL. Yes, yes, yes lots of /enterprises/ run applications that use MySQL but most of those apps have throw away data or they are not using the free version of MySQL.
Just imagine backing the thing up, exporting 60.000.000 SQL queries. Not to say importing them again if something should go really wrong. Actually I'n not even sure it would be faster. When the index files grow to several gigabytes they kind of loose their purpose.
There are many businesses backing up way-more data than that and it it isn't 60,000,000 queries -- it is one command. But if you use serious hardware "backing up" isn't really needed. RAID, redundant/hot-swap servers, etc. make backing up /extra redundancy/. :-)
And I bring this up because "Archiveopteryx" http://www.archiveopteryx.org/ uses a database - PostgreSQL.
Rod
Maildir is very resilient to various errors. It is virtually impossible to corrupt a maildir (at least I've never experienced anything). Also you can backup up the thing without worrying about anything accessing it at the same time. Mbox less so but still a lot better than having one huge database.
Dbox would be the ultimate compromise between crash resilience and a low number of files (not to mention the enormous potential for speed gains).
Regards, Mikkel