Dovecot and data migration

Steffen Kaiser skdovecot at
Thu May 7 06:12:07 UTC 2015

Hash: SHA1

On Wed, 6 May 2015, Alain BERNARD wrote:

> Our legacy data store retains a single copy of a message regardless of the
> number of mailboxes in which that message resides. It does this by creating
> hard links to that message in the mailboxes containing that message.

> Thus, when we perform data migration to the server target (Dovecot), the
> copies of the same message are copied over with the migration process
> (imapsync). We use the storage format maildir.

> With a small message store, this means that a lot of messages are
> duplicated unnecessarily. How to reduce message store size due to duplicate
> storage of identical messages ?

There is no function in Dovecot doing that. For the synchronisation you 
can come up with some filesystem related script doing that easily.

> Does a relinking function exist and can be run in real-time mode ? how can
> we configure Dovecot to deduplicate for all users using a hash to determine
> whether the file could be already exist ?

In Dovecot v1 I did this with an external script, that hard linked equal 
files in cur and new directories that resides in more than 10 or so 

But in the production phase with Dovecot v2 you will face some culprit: 
with LMTP all messages are different now, because of the user-related 
Delivered-To and final Recieved header. With deliver this does not happen, 
but your MTA possibly adds different headers then, because usually LDAs 
are called per recipient. Dovecot deliver has the "-p" option to 
optionally hard link to file message file to the argument of -p. But then 
you must use some scripting to have your MTA call that script for all 
final recipients. You should also check, if Sieve is compatible with -p, 
because I remember some bug reports.

- -- 
Steffen Kaiser
Version: GnuPG v1


More information about the dovecot mailing list