On 23.9.2005, at 18:04, John Peacock wrote:
Timo Sirainen wrote:
While the maildir format itself is simple, it's actually really difficult to handle correctly when the maildir is changing under us. Files can be renamed at any time so you'll have to be prepared to look for the file's new name at any time. Filesystems also don't work the way maildir assumes they do, so you have to work around their limitations too.
I thought that was the point of dovecot-lda, so that you know that dovecot is the only process touching the message files and can update the indexes as things change, rather than after the fact.
Dovecot still can't know that it's the only one, but I had been thinking about adding an option to make Dovecot assume it's the only program modifying the maildir. That should avoid scanning the cur/ directory constantly. Except that probably still requires some changes since up-to-date maildir filenames aren't stored anywhere, but maybe they could be guessed by looking at the base filename and mail's current flags from indexes and generating the most-likely filename based on them (or did I already have that code..? :)
This could possibly be also automatically set per mailbox. If user always expunges all mails at a time (POP3) or never expunges (mailing list archives), there would be only a single file.
If you are thinking that way, it might be useful to mark mailboxes as either:
a) sequential access (most POP3 and archives); b) random access (IMAP).
and provide different tuning based on usage expectations, rather than trying to actively tune. Then the system admin could choose which fits their needs best; we only have a few people using POP3 and everyone else is IMAP, and I frankly don't care what kind of performance the POP3 people have.
I haven't really thought about this yet, but a global setting wouldn't usually be enough for all users and I'm not sure how it can be set per-user or per-mailbox in any useful way.
I think SQL database as a mail store would have much worse performance than with any filesystem based mail store.
Except that with clustering, you could have multiple DB servers getting hit by multiple dovecot servers. SQL could potentially scale to much larger environments than filesystem support.
How did you think it would work?
One way to do it would be to distribute users to different SQL servers. That's just as possible with filesystems.
If you thought about some kind of database that is distributed across multiple computers.. Well, that would probably work, but I haven't heard much of them being in any actual use. Probably because they all cost too much. I don't think PostgreSQL or MySQL had support for these kind of clusters yet?
There are also clustered filesystems that are distributed across multiple computers. Dovecot already works with them.
Plus, think of all the neat fulltext indexing possibilities.
Unfortunately that doesn't help with IMAP at all, and might make it actually more difficult and slower to implement the kind of indexing that would benefit IMAP's SEARCH command.