[Dovecot] (Single instance) attachment storage

[Daniel L. Miller]

On 7/19/2010 8:24 AM, Timo Sirainen wrote:
> Now that v2.0.0 is only waiting for people to report bugs (and me to
> figure out how to fix them), I've finally had time to start doing what I
> actually came here (Portugal Telecom/SAPO) to do. :)
>
> The idea is to have dbox and mdbox support saving attachments (or MIME
> parts in general) to separate files, which with some magic gives a
> possibility to do single instance attachment storage. Comments welcome.
>
>    
YAAAY!!!  Timo's gonna give us SIS!!!

Is it done yet :) ?

I'm just thinking out loud here - and I'm probably way off base.  Just 
tell me to shut up and I'll go away and hide until you're finished.

1.  You've already identified that enabling this feature needs to avoid 
introducing problems - including treating different-but-similar 
attachments as identical.  In your hashing choices, you only mentioned 
attachment body.  What about including size and date in the hash?

2.  You didn't explicitly define if SIS would be per-mailbox or 
system-wide.  Speaking for myself, and probably a few others, I'll take 
whatever implementation I can get - but I'd love to see it system-wide.

3.  Are you envisioning this as being handled totally within deliver, or 
would there be a server process for consolidating the messages?  I'm 
wondering about the impact to high-traffic sites (which mine is 
thankfully NOT) - if deliver needs to crunch on large messages, could 
this lead to time-out issues from the MTA's?

A possible alternative, have deliver write the message out as normal - 
but flag it for attachment processing.  Then have a secondary process 
awakened to check for attachments and perform accordingly.  So any SIS 
overhead becomes invisible to the MTA - other than needing available 
system resources for processing (and the attachment processing could be 
done at a lower priority).

-- 
Daniel