[Dovecot] Mail deduplication

[Timo Sirainen]

On 6.5.2013, at 18.03, Charles Marcus <CMarcus at Media-Brokers.com> wrote:

> On 2013-05-06 10:54 AM, Timo Sirainen <tss at iki.fi> wrote:
>> On 30.4.2013, at 12.22, Jan-Frode Myklebust <janfrode at tanso.net> wrote:
>>> Wasn't there also some issue with cleanup of attachments ? Not being able
>>> to delete the last copy, or something. I did some testing of using SIS on a
>>> backup dsync destination a year (or two) ago, and got quite confused..
>>> Don't quite remember the problems I had, but I did lose confidence in it
>>> and decided having the attachement together with the messages felt safest.
>>> 
>>> I would also love to hear from admins using it on large scale (100K+ active
>>> users). Maybe we should reconsider using it..
> 
>> I'm not aware of any bugs in SIS, but yeah, it can be a bit complicated. If you do things like dsync where destination is also mdbox/sdbox, it's going to keep using the same SIS directory and updating the refcounts, which you probably don't want for backups / temp directories (solution: give different parameters to the two sides of dsync where the other side disables SIS).
> 
> Hey Timo - so, how will rsync be affected as a backup app? Will it maintain the deduped state in the backup target?

Ideally you'd rsync from a filesystem snapshot instead of from live filesystem, otherwise the link counts might go wrong. And you need to use the -H parameter for rsync so it preserves hard links.