[Dovecot] Documentation for "altpath" / "altmove" / ?"Alternate storage"

[Timo Sirainen]

On Fri, 2010-09-03 at 11:13 +0100, William Blunn wrote:

> We might want to have a name for the wiki page. Perhaps 
> "AlternateStorage"? Of course if it only applies to dbox, then another 
> idea is to make it a section on the page "MailboxFormat/dbox". All 
> suggestions gratefully received.

I think it should be either in dbox page or as a subpage under dbox,
other storages are very unlikely to get support for it.

> I would be interested in an overview of what alternate storage is. It 
> seems to be a way of transparently moving message data to alternare 
> storage, with the idea that the alternate storage may be on a different 
> filesystem which may be cheaper and/or slower than the main storage, 
> which may be an economic way to store messages where the fastest access 
> to them is less important. It is transparent in as much as IMAP/POP 
> users will not normally be able to tell if any given message has been 
> moved to alternate storage. A single mail folder can end up containing a 
> mixture of messages stored in main storage and alternate storage.

Right. Another way to use it is to store all new mails to a fast but
small SSD and then keep moving the mails to HDDs as often as necessary.

> (Another question begged here of what consistent term should be used to 
> refer to the main storage. Could be "main storage", "normal storage", 
> "ordinary storage".)

I think I have been calling it "primary storage".

> Also I would be interested to know the development/stability status of 
> alternate storage: experimental / alpha / beta / stable.

It's being used for the SSD -> HDD moves by one installation with
probably some hundreds of users if not more (I don't know).

> Also I would be interested to know the applicability against mailbox 
> formats: mbox / maildir / sdbox / mdbox / cydir.

For cydir it could be implemented, but there's not much point. For
mbox/maildir it would be complex.

> Also I would be interested to know which data gets moved to the 
> alternate storage, and which data stays in the main storage: 
> message-data / control-data / index-data / combined-control-and-index-data.

Only the message texts get moved to alt storage.

> Also I would be interested to know how data can be moved to alternate 
> storage. Seems to be that is only done by invocation of "doveadm 
> altmove". There is a page for that already so we can link to that.

With single-dbox the move operation can even be done manually by simply
running:

mv /home/user/dbox/mailboxes/INBOX/u.123 /altstorage/user/dbox/mailboxes/INBOX/

I guess it should work just fine, but still I don't recommend anyone
actually doing that. It might not work as easily some day in future, or
there may be some race conditions or something I can't think of right
now.

With multi-dbox the moving is more complex, but still it only copies the
data in m.* files.

> Also it would be interesting to have some idea of how it works. For 
> example there might be an "alternate storage" flag in the indexes, or 
> Dovecot tries the main location first and if not found there then it 
> tries the alternate storage. Just a brief overview, and any pertinent 
> ramifications of that.

Dovecot always tries to look for the u.123 or m.123 file from primary
storage and if it's not found there, it looks it up from alternate
storage. The state isn't stored in indexes. Really simple.