[Dovecot] SIS and restoring from backups

[Charles Marcus]

On 2012-03-24 9:16 AM, Timo Sirainen <tss at iki.fi> wrote:
> On 24.3.2012, at 14.54, Charles Marcus wrote:
>
>> On 2012-03-24 8:08 AM, Timo Sirainen<tss at iki.fi>  wrote:
>>> You can do full backups from a filesystem snapshot, which works
>>> "well enough" (might leave some unused attachments lying around in
>>> some rare cases, but that can also happen if Dovecot crashes/dies).
>>
>> But the problem isn't with backups, but with restores, right?
>
> Ah, right. Then it gets tricky.

Yeah, I seem to remember it was a comment like that that scared me about 
enabling it...

Can you expand on what exactly is 'tricky' about it? Also, have you 
given any thought to how to eliminate the 'trickiness'? I'm of the old 
school and like for my backups to not have any 'trickiness' about them - 
including performing restores... ;)

>> So, this would make the backup storage requirements larger - maybe
>> dramatically larger for sites that have a lot of large
>> attachments?

> Some backup systems can do internal deduplication.

Hmmm... and actually, rsnapshot (which uses rsync) does just that, which 
is *why* each additional snapshot only requires a small fraction of 
additional disk space (compared to the first main/full snapshot).

>> Am I correct that enabling SIS as it is currently implemented would
>> break this backup tool?

> I'm not sure. Are you running rsnapshot on live filesystem or on a
> snapshot? On live filesystem there would be race conditions.

I've been running it on a live system for a long time, and never had a 
problem beyond occasional messages like this:

file has vanished: 
"/var/vmail/example.com/username/cur/1332602593.Vfe02I9e7acdM308676.myhost.example.com:2,"
rsync warning: some files vanished before they could be transferred 
(code 24) at main.c(1052) [sender=3.0.9]

but the rsnapshot guys assured me this will and does not cause any real 
problems, other than those files don't get backed up.

I am however looking forward to migrating this to a VM so I can do 
snapshot for backups to get consistent point-in-time backups.

>> I was also thinking of asking about how to provide read-only access
>> to these backup snapshots to the users in some kind of special
>> namespace, so that they could all essentially go 'back in time' to
>> grab any emails that they may have inadvertently deleted...

> This should be possible, just point the namespace to such snapshot.
> You may need to point CONTROL dir to some temporary directory and
> index dir as well to either temp or to memory.

This is great news! I'm looking forward to getting this all working.

-- 

Best regards,

Charles