[Dovecot] Scalability plans: Abstract out filesystem and make it someone else's problem

[Charles Marcus]

On 8/12/2009, Ed W (lists at wildgooses.com) wrote:
> It also had a bunch of limitations, it was basically only single
> instance for CC recipients on a message (more or less).  Quite a lot
> of things such as certain types of virus scanning would (I think)
> easily disable the single instance storage also?
> 
> So I doubt it would help in most of the cases mentioned here, ie each
> time it was re-forwarded internally it would not be single instanced
> 
> I still think it would be instructive to do some benchmarks though -
> often these things look good on paper, but are surprisingly less
> effective (given the implementation cost)  when measured.  I'm not
> disagreeing, just would be interested to see some numbers...

Amazing... I mean, since Exchange is already a 'database', how hard
would it be to do it right (checksum each mime part, and use hardlinks
for subsequent duplicate checkummed mimeparts)? As long as everything
was properly and effectively indexed, it should be easily doable.

Make it do the work at delivery when the load is light enough, and have
a background task that does this for all messages that are not flagged
as having already been de-duped at delivery time when the load is light
enough.

-- 

Best regards,

Charles