[Dovecot] SIS and restoring from backups

[Timo Sirainen]

On 25.3.2012, at 18.12, Charles Marcus wrote:

> On 2012-03-24 9:16 AM, Timo Sirainen <tss at iki.fi> wrote:
>> On 24.3.2012, at 14.54, Charles Marcus wrote:
>> 
>>> On 2012-03-24 8:08 AM, Timo Sirainen<tss at iki.fi>  wrote:
>>>> You can do full backups from a filesystem snapshot, which works
>>>> "well enough" (might leave some unused attachments lying around in
>>>> some rare cases, but that can also happen if Dovecot crashes/dies).
>>> 
>>> But the problem isn't with backups, but with restores, right?
>> 
>> Ah, right. Then it gets tricky.
> 
> Yeah, I seem to remember it was a comment like that that scared me about enabling it...
> 
> Can you expand on what exactly is 'tricky' about it? Also, have you given any thought to how to eliminate the 'trickiness'? I'm of the old school and like for my backups to not have any 'trickiness' about them - including performing restores... ;)

It's easy to restore a full backup. And it's easy to restore specific users if you have the full backup easily accessible (just run doveadm import with proper settings pointing to backup). What's difficult is if you just want to restore a specific user from the backup and can't easily do random access to all files. Then you'll first need to restore the user's dbox files and then somehow figure out which attachments to restore from the SIS directory.

>>> Am I correct that enabling SIS as it is currently implemented would
>>> break this backup tool?
> 
>> I'm not sure. Are you running rsnapshot on live filesystem or on a
>> snapshot? On live filesystem there would be race conditions.
> 
> I've been running it on a live system for a long time, and never had a problem beyond occasional messages like this:
> 
> file has vanished: "/var/vmail/example.com/username/cur/1332602593.Vfe02I9e7acdM308676.myhost.example.com:2,"
> rsync warning: some files vanished before they could be transferred (code 24) at main.c(1052) [sender=3.0.9]

I'd guess that with rsnapshot + Maildir you can get duplicate Maildir files if the rsnapshot is accessing a large maildir at the same time as user is changing a message flag. Dovecot usually notices these duplicates and logs a warning about them.