On Mon, 2009-09-28 at 19:21 +0100, Ed W wrote:
In my mind this is more about what lib-storage API was supposed to abstract out, whereas my filesystem API would be used simply for binary data storage. The same FS API could be used to store both dbox files and index files.
I guess in this case it would be interesting to hear the kind of use cases you imagine that the storage API will be used for in practice? I think I might be kind of overthinking the problem?
lib-storage API has existed since Dovecot v1.0 and it's used to abstract out access to maildir, mbox, dbox, cydir, etc. SQL would fit right there with those.
Or did you mean FS API? For that my plans are to implement backends for:
- POSIX (just the way it works now)
- Async I/O (once Dovecot can do more things in parallel)
- Some kind of proxying to support shared mailboxes between different servers (or within same server when users are using different UIDs and don't have a common group)
- Massively distributed database storage for mails
- In-memory cache for index files, which permanently writes them using another storage. This is useful for any kind of multi-master setup like distributed database, NFS, clusterfs.
Seems like it's a very thin shim between a real file system and dovecot and would be mainly useful for supporting filesystems with non posix protocols, eg someone wants to store their mail files on mogile or DAV, but it doesn't address anything lower or higher than blocks of data?
Right, path/filename (or "key") -> binary byte stream.
Seems like it would be useful for:
- implementing very specific optimisations for example for NFS
- optimisation for filesystems with unusual strengths/weaknesses, eg GFS or Gluster?
In both of these I think the primary problem is that Dovecot tries to do IPC via filesystem (index files). So accessing the indexes via the in-memory cache that is guaranteed to be always up-to-date would get rid of all these ugly NFS cache flushing attempts etc.