Timo Sirainen tss@iki.fi wrote:

On 21.6.2004, at 21:04, Bill Moran wrote:

...

Subject says it all.

(I'm almost positive this is a FAQ, or that the information is online _somewhere_, but I just spent 20 minutes without finding it ... sorry if I didn't search hard enough)

Well, main Dovecot web page says it:

"Dovecot can work with standard mbox and maildir formats and it's fully compatible with UW-IMAP and Courier IMAP servers as well as mail clients accessing the mailboxes directly. I have also plans to support storing mails in SQL databases."

Ahh ... I had a feeling it would be right in front of my face ...

Still not in my near future plans, although I've had one guy contact me about making Dovecot use a library derived from dbmail as a backend.

I don't think it's really worth it to store mails in SQL database, unless you really need to do it for some reason. Performance is most likely never the reason to do it. Reliability and interoperability might be.

Reliability is the reason for me. We already have acceptable performance. I'm dealing with a client that provides IMAP services for multiple clients, and we're concerned about what happens in the event of a server failure on the machine with the HDDs. There's already RAID in place, but what if the mobo, PS, controller ... etc fails.

What I'd like to do is use Postgres with Slony to keep the IMAP folders on two completely seperate machines in real-time.

Anyway ... thanks for the great work you've done with Dovecot so far. Maybe I'll even have time to contribute something in the future.

-- Bill Moran Potential Technologies http://www.potentialtech.com

On Jun 22, 2004, at 11:39 AM, Xavier Beaudouin wrote:

With maildir and nfs you'll can get a very stable system... We are running with maildir + nfs now, and for a larger (200gb) mailstore over NFS we run into performance issues with maildir (lots of small files).

Smaller partitions might do the trick, but we didn't investigate yet. Furthermore: dovecot and nfs aren't playing nice yet, although the new dovecot-1.0-test releases look promising :)

Using database (eg postgres) as storage system can be a good idea (mostly when you'd like to search something), but when it get corrupted or fragmented... then you'll have troubles.... Same goes for plain filesystems.

/Savier

Kind regards, Maikel Verheijen.

On 22.6.2004, at 04:49, Tom Allison wrote:

I had a conversation with the same fellow a while back.

I'm told that dbmail is actually very fast in mail retrieval. But I was unable to confirm it myself.

It could be fast, but it couldn't be faster than doing "cat mailfile", which is basically what maildir clients do.

In short, they store the entire header in one field and the entire body in another field. Hardly optimized for the three or four most common sorts (date, subject, sender, threaded)

Sorting could be completely done by SQL server with ORDER BY, but threading would require just fetching a few headers and doing it internally. Dovecot 1.0's cache file btw. keeps the commonly used headers stored in it.

On Mon, 2004-06-28 at 22:04, Dan Stromberg wrote:

...

It could be fast, but it couldn't be faster than doing "cat mailfile", which is basically what maildir clients do.

Actually, in a really large folder, a database (with a decent organization) could be significantly faster than "cat mailfile" on more traditional filesystems.

However, as filesystems take on more database-like features, this distinction will likely become less important. (EG: dir_index on ext3).

Right. Directory indexes are really important with maildirs, especially large ones. I guess a database can be slightly faster even with those, depending on how much better it stores the data internally than filesystems. But SQL databases still have the overhead of parsing SQL commands, IPC and what else..

ReiserFS 4 promises very good performance with small files as well as other great things. I've been thinking about adding native support for it once it's more widely used. It should be able to at least partially replace Dovecot's indexes.