On Mon, Dec 05, 2011 at 01:51:05AM +0200, Timo Sirainen wrote:
I'm mainly wondering if it's common for backup programs to support using a separate program to generate the backups. For example if there was a "dovecot-backup" binary that just dumps all (or new-since-last-backup) of the users' mails into stdout, which the backup program can use. Or perhaps in that case there wouldn't really be much of anything for the backup to do except to write it to tape..
For databases, most commercial programs use some kind of application-specific plugins. Bakula has "Client Run Before Job" which permits indicating a command to be run before proceeding with the backup, maybe there is more. However I have not heard of any standardized way of doing it so that the application provider could provide a way to interact with multiple backup programs. It's more like each backup vendor boasting that his backup software can backup application XXX without downtime.
SIS was designed to work with hard links. They couldn't be replaced with symlinks without a redesign (which would be less efficient in normal operation).
Right, but if the backup program recognized this then maybe the replacement by a symlink could be done only in the backup. rsync has to keep in memory all the inodes and check the list every time a hard link is found. If it knew that "a hard link in attachments should link to an identical file name in the common attachment SiS store.
[Zimbra]
You mean you first create uncompressed zip files (why not just tar?) of all the mails to the filesystem and the backup software then backups those zip files?
Well, not I, this is Zimbra's backup system :-) The backups are the destination zips. Maybe zip is used because the extension and extraction method is the same whether compressed or not; that way compression is just an option to the backup program to be turned on or off.
Dovecot's mdbox files already contain multiple messages in each file, so it should be a lot more efficient to do backups on those. And each message in an mdbox file can be compressed if zlib plugin is enabled. So I think that sounds quite a lot like what you propose.
Is that combined or combinable with SiS? If attachments are in separate files, that means they are aligned on block boundaries, which make block-level SiS (like NetApp's) much more efficient. Think of an attachment sent to all department heads, all of whom forward the attachment to all their subordinates.