Remove "Duplicate" emails (and documentation update)

[@lbutlr]

On 2018-02-23 (18:01 MST), @lbutlr <kremels at kreme.com> wrote:
> 
>>> First, even after expunging a message and running doveadm index -u
>>> kremels ?Archive?, subsequent runs still show the same duplicate
>>> messages.
>> 
>> I suspect client side caching.
> 
> No, there is no client side involved. I am executing all of these these commands on the mail server. I expunge the messages, I index (or even force-resync) and the next search shows the same messages even though they are not in the Maildir anymore.

OK, I did another run yesterday and expunged 732 message dupes. I checked for a few of them before the expunge and there were files there. I checked after the expunge (several minutes after) and the files were NOT there.

They still show up if I do another search.

just now: 
 # doveadm -f table fetch -u kremels 'hdr.message-id guid uid hdr.x-listname mailbox' mailbox 'Archive' | sort | awk 'cnt[$1]++{if (cnt[$1]>=2) print prev[$1]; print} {prev[$1]=$0}'  | sort -u -k1,1| awk '{print "doveadm expunge -u kremels MAILBOX-GUID "$2" UID "$3}' | grep 1514660575.66506_1
hdr.message-id     guid                uid hdr.x-listname mailbox
doveadm expunge -u kremels MAILBOX-GUID 1514660575.66506_1.mail.covisp.net UID 60608
 # find ~kremels/Maildir -name "1514660575.66506_1*"
 # doveadm fetch  -u kremels MAILBOX-GUID 1514660575.66506_1.mail.covisp.net UID 60608
Fatal: Invalid messageset

So, how do I get expunged messages to not show up in the fetch?

(Yes, I ran doveadm index -u kremels "*") 

# doveadm -Dv index -u kremels "*"
Debug: Loading modules from directory: /usr/local/lib/dovecot
Debug: Module loaded: /usr/local/lib/dovecot/lib20_virtual_plugin.so
Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm
Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so: Undefined symbol "acl_user_module" (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: Undefined symbol "expire_set_lookup" (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: Undefined symbol "quota_user_module" (this is usually intentional, so just ignore this message)
Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_sieve_plugin.so
Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: Undefined symbol "fts_filter_filter" (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: Undefined symbol "mail_crypt_user_get_public_key" (this is usually intentional, so just ignore this message)
doveadm(kremels): Debug: Effective uid=1004, gid=1004, home=/home/kremels
doveadm(kremels): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir
doveadm(kremels): Debug: maildir++: root=/home/kremels/Maildir, index=, indexpvt=, control=, inbox=/home/kremels/Maildir, alt=
doveadm(kremels): Debug: Archive: Mailbox opened because: index
doveadm(kremels): Info: Archive: Cache is already up to date
doveadm(kremels): Debug: Drafts: Mailbox opened because: index
doveadm(kremels): Info: Drafts: Cache is already up to date
doveadm(kremels): Debug: Junk: Mailbox opened because: index
doveadm(kremels): Info: Junk: Cache is already up to date
doveadm(kremels): Debug: Misc.not-to-me: Mailbox opened because: index
doveadm(kremels): Info: Misc.not-to-me: Cache is already up to date
doveadm(kremels): Debug: bind: Mailbox opened because: index
doveadm(kremels): Info: bind: Cache is already up to date
doveadm(kremels): Debug: Sent: Mailbox opened because: index
doveadm(kremels): Info: Sent: Cache is already up to date
doveadm(kremels): Debug: Trash: Mailbox opened because: index
doveadm(kremels): Info: Trash: Cache is already up to date
doveadm(kremels): Debug: dovecot: Mailbox opened because: index
doveadm(kremels): Info: dovecot: Cache is already up to date
doveadm(kremels): Debug: freebsd: Mailbox opened because: index
doveadm(kremels): Info: freebsd: Cache is already up to date
doveadm(kremels): Debug: httpd: Mailbox opened because: index
doveadm(kremels): Info: httpd: Cache is already up to date
doveadm(kremels): Debug: spamassassin: Mailbox opened because: index
doveadm(kremels): Info: spamassassin: Cache is already up to date
doveadm(kremels): Debug: bbedit: Mailbox opened because: index
doveadm(kremels): Info: bbedit: Cache is already up to date
doveadm(kremels): Debug: postfix: Mailbox opened because: index
doveadm(kremels): Info: postfix: Cache is already up to date
doveadm(kremels): Debug: macosx: Mailbox opened because: index
doveadm(kremels): Info: macosx: Cache is already up to date
doveadm(kremels): Debug: tidbits: Mailbox opened because: index
doveadm(kremels): Info: tidbits: Cache is already up to date
doveadm(kremels): Debug: INBOX: Mailbox opened because: index
doveadm(kremels): Info: INBOX: Cache is already up to date

-- 
How soon after the USPS issues the Calvin stamp will you send a letter with one
on the envelope?  Watterson: Immediately. I'm going to get in my horse and
buggy and snail-mail a check for my newspaper subscription.