On 2018-02-23 (16:47 MST), Joseph Tam <jtam.home@gmail.com> wrote:
On Fri, 23 Feb 2018, @lbutlr wrote:
$ doveadm -f table fetch -u kremels 'hdr.message-id guid uid hdr.x-listname' mailbox "Archive" | sort| awk 'cnt[$1]++{if (cnt[$1]==2) print prev[$1]; print} {prev[$1]=$0}' |grep -E "[0-9] +$" |awk '{print "doveadm expunge -u kremels MAILBOX-GUID "$2" UID "$3}?
I was unaware of the syntax "hdr.{header}" -- all the reference materials I've seen only refers to "hdr" which returns the entire header block.
the error message from doveadm if you specify an invalid field is:
Available fetch fields: hdr.<name> body.<section> binary.<section> user mailbox mailbox-guid seq uid guid flags modseq hdr body body.snippet text text.utf8 size.physical size.virtual date.received date.sent date.saved date.received.unixtime date.sent.unixtime date.saved.unixtime imap.envelope imap.body imap.bodystructure pop3.uidl pop3.order refcount storageid
First, even after expunging a message and running doveadm index -u kremels ?Archive?, subsequent runs still show the same duplicate messages.
I suspect client side caching.
No, there is no client side involved. I am executing all of these these commands on the mail server. I expunge the messages, I index (or even force-resync) and the next search shows the same messages even though they are not in the Maildir anymore.
If {count} is what you expected, then dovecot has the correct information and it's likely some client-side caching issue.
I would have needed to check the count before doing this, and I did not.
Second, what I really want to do is run this over ALL the mailboxes, except for Junk and Sent but if that is possible I can?t find the right syntax.
You mean to remove duplicates from any 2 mailboxes, or remove duplicates in mailboxes also found in Archive?
I want to find any duplicates (based on msg ID) across all mailboxes, except Sent
doveadm -f table fetch -u kremels
'hdr.message-id guid uid'
NOT mailbox Archive
NOT mailbox Junk
NOT mailbox Sent
| sort -b >list1
Aha! Didn't know you could use NOT mailbox. That probably solves my issue on that score.
-- "It's unacceptable to think" - George W Bush 15/Sep/2006