Remove "Duplicate" emails

[@lbutlr]

In a quest to remove “duplicate” messages sent to both me and lists I subscribe to I came up with this, which I think should clean out my Archive folder, but I’ve been unable to get it to work for scanning all on my list-user email.

$ doveadm -f table fetch -u kremels 'hdr.message-id guid uid hdr.x-listname' mailbox "Archive" | sort| awk 'cnt[$1]++{if (cnt[$1]==2) print prev[$1]; print} {prev[$1]=$0}' |grep -E "[0-9] +$" |awk '{print "doveadm expunge -u kremels MAILBOX-GUID "$2" UID "$3}’

X-listname is a header that my list user applies to each message that comes from a list, and the output is just text to the screen that I can then run manually (I am not confident enough to automatically delete the messages).

I include the X-listname header at the end so that I can exclude lines that don’t end in a number, which means the copies sent directly to me are the ones expunged and the ones from the list are preserved.

So far so good. But ther are issues.

First, even after expunging a message and running doveadm index -u kremels “Archive”, subsequent runs still show the same duplicate messages. 

Second, what I really want to do is run this over ALL the mailboxes, except for Junk and Sent but if that is possible I can’t find the right syntax.

-- 
"He has all the virtues I dislike and none of the vices I admire."
Winston Churchill