doveadm deduplicate commands

[Peter]

Am 2015-02-13 um 17:47 schrieb Jiri Bourek:
> I'd try this (in shell):
>
> doveadm search -u user at domain.net mailbox inbox | wc -l
>
> Doveadm will print mailbox-guid and uid pair for every message in inbox,
> one per line, pipe it into "wc -l", which will count the lines and
> output a number - that's the count of messages in INBOX.
>
> Then try doveadm deduplicate and after that the search command above
> again. If the count changed, deduplicate is working - you may only need
> to run it multiple times.
>
> If the count doesn't change, dovecot is unable to recognize duplicates
> in your mailbox and you need to find another solution. Maybe check out
> the "-m" option in man doveadm-deduplicate

One should take great care, guids are not always unique, eg after 
consolidating several folders into one, that is, when deduplication 
might become really useful!

Below shell commands give a temptative view of what will be expunged, 
dont deduplicate, if you do not like what diff says:

> ## Beware - its not just duplicates sometimes…
> BOX="mailbox INBOX"
> USR="-u myname"

> # by guid
> doveadm -f table fetch $USR 'guid hdr.Message-ID hdr.Subject' $BOX | sort --stable -k1,1 > /tmp/F1A.txt
> doveadm -f table fetch $USR 'guid hdr.Message-ID hdr.Subject' $BOX | sort --stable --uniq -k1,1 > /tmp/F1B.txt
> diff -u /tmp/F1A.txt /tmp/F1B.txt | less -S
> doveadm deduplicate $USR $BOX

> # by Message-ID
> doveadm -f table fetch $USR 'guid hdr.Message-ID hdr.Subject' $BOX | sort --stable -k2,2 > /tmp/F2A.txt
> doveadm -f table fetch $USR 'guid hdr.Message-ID hdr.Subject' $BOX | sort --stable --uniq -k2,2 > /tmp/F2B.txt
> diff -u /tmp/F2A.txt /tmp/F2B.txt | less -S
> doveadm deduplicate -m $USR $BOX

-- 
peter