[Dovecot] Using a namespace for providing access to mail snapshots for user based on-demand restoration of email backups

[Joseph Tam]

A timely topic as I was just mulling over ways to provide this to my
users.

Charles Marcus wrote:

> The snapshots are stored with the following filesystem layout:
>
> /path/to/snapshotsdir/hourly.0
> ...

This is familiar to NetApp users.

> The 'names' (hourly, daily, weekly, monthly, yearly) are arbitrary (this
> is a bit confusing to people new to rsnapshot), and would *not* be used
> for displaying the mail folders to the users - it is the Date/Time
> stamps of each of the snapshot dirs above that would be used to display
> the folder names under the 'Time Machine' namespace. This is, I imagine,
> the part that will need some actual coding by Timo to get working -
> maybe just some new config variables added to the namespace code for
> mapping the date/time stamps of the directories to user friendly folder
> names in the namespace.
>
> That said, I'd like to design this and have it coded such that it will
> work with almost any type of backup storage that stores snapshots as
> date/time stamped directories like this (there must be others, right?).

One idea is to take this complexity entirely out of dovecot and create a
synthetic filesystem using hard or soft links (as rsnapshot has done) and
create your own, with whatever weird and wonderful naming scheme you want.

 	/path/to/TimeMachine/<friendlylabel>/<user>
 		-> /path/to/snapshotsdir/<snaplabel>/<user>

 	namespace {
 		prefix = TimeMachine
 		location = maildir:/path/to/TimeMachine:INDEX=MEMORY
 		...
 	}

This might not be very scalable depending on how big your userbase is.

I would probably define memory indices for this namespace, and take
the performance hit on the assumption that access will be a once-in-a-
while thing.  On-disk indices will probably get out of date with each
snapshot rollover, if if you have a lot of snapshots/mailboxes, it could
consume a non-trivial amount of space without a lot of benefits.

Or you could run a cron script to rename or remove old indices, but
that seems more trouble than it's worth.

One other consideration (at least for me) is if the INBOX and
personal mail folders are stored in two separate FS's.  It would be nice
to fuse the two sets of backups under the same namespace, but I don't
know how the namespace prefix matching works and whether you can define
hierarchical namespaces like

 	namespace {
 		prefix = backup/inbox
 		location = mbox:/path/to/inbox-snapdir/%u
 		...
 	}

 	namespace {
 		prefix = backup/mail
 		location = mbox:/path/to/mail-snapdir/%u
 		...
 	}

The above can also be accomplished with a synthetic filesystem.

Joseph Tam <jtam.home at gmail.com>