[@ ~]# doveadm -D -o mail_location=mdbox_deleted:/home/mail-archive/testtest/mdbox:INDEX=/home/archiveindex/testtest/index mailbox status -u testtest 'messages vsize' "Archive/2018"

Debug: Loading modules from directory: /usr/lib64/dovecot Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib20_mail_log_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib20_replication_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib90_stats_plugin.so Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) doveadm(testtest): Debug: Effective uid=8267, gid=231, home=/home/popusers/testtest doveadm(testtest): Debug: replication: No mail_replica setting - replication disabled doveadm(testtest): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox_deleted:/home/mail-archive/testtest/mdbox:INDEX=/home/archiveindex/testtest/index doveadm(testtest): Debug: fs: root=/home/mail-archive/testtest/mdbox, index=/home/archiveindex/testtest/index, indexpvt=, control=, inbox=, alt= doveadm(testtest): Debug: Namespace 4archives: type=private, prefix=Archive/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=yes location=mdbox:/home/mail-archive/testtest/Archive/:CONTROL=/home/mail-archive/testtest/Archive/control:INDEX=/home/archiveindex/testtest/index doveadm(testtest): Debug: fs: root=/home/mail-archive/testtest/Archive, index=/home/archiveindex/testtest/index, indexpvt=, control=/home/mail-archive/testtest/Archive/control, inbox=, alt= doveadm(testtest): Debug: Archive/2018: Mailbox opened because: mailbox status Archive/2018 messages=70 vsize=6182960

[@ ~]# doveadm -D -o mail_location=mdbox:/home/mail-archive/testtest/mdbox:INDEX=/home/archiveindex/testtest/index mailbox status -u testtest 'messages vsize' "Archive/2018"

Debug: Loading modules from directory: /usr/lib64/dovecot Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib20_mail_log_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib20_replication_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib90_stats_plugin.so Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) doveadm(testtest): Debug: Effective uid=8267, gid=231, home=/home/popusers/testtest doveadm(testtest): Debug: replication: No mail_replica setting - replication disabled doveadm(testtest): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:/home/mail-archive/testtest/mdbox:INDEX=/home/archiveindex/testtest/index doveadm(testtest): Debug: fs: root=/home/mail-archive/testtest/mdbox, index=/home/archiveindex/testtest/index, indexpvt=, control=, inbox=, alt= doveadm(testtest): Debug: Namespace 4archives: type=private, prefix=Archive/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=yes location=mdbox:/home/mail-archive/testtest/Archive/:CONTROL=/home/mail-archive/testtest/Archive/control:INDEX=/home/archiveindex/testtest/index doveadm(testtest): Debug: fs: root=/home/mail-archive/testtest/Archive, index=/home/archiveindex/testtest/index, indexpvt=, control=/home/mail-archive/testtest/Archive/control, inbox=, alt= doveadm(testtest): Debug: Archive/2018: Mailbox opened because: mailbox status Archive/2018 messages=70 vsize=6182960

I am not sure if you wanted me to just add another -o, so I did these 3, but all with same results[1].

[1] [@ storage]# doveadm -D -o namespace/4archives/location=mdbox_deleted:/home/mail-archive/testtest/mdbox -o mail_location=mdbox_deleted:/home/mail-archive/testtest/mdbox:INDEX=/home/archiveindex/testtest/index mailbox status -u testtest 'messages vsize' "Archive/2018" Debug: Loading modules from directory: /usr/lib64/dovecot Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib20_mail_log_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib20_replication_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib90_stats_plugin.so Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) doveadm(testtest): Debug: Effective uid=8267, gid=231, home=/home/popusers/testtest doveadm(testtest): Debug: replication: No mail_replica setting - replication disabled doveadm(testtest): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox_deleted:/home/mail-archive/testtest/mdbox:INDEX=/home/archiveindex/testtest/index doveadm(testtest): Debug: fs: root=/home/mail-archive/testtest/mdbox, index=/home/archiveindex/testtest/index, indexpvt=, control=, inbox=, alt= doveadm(testtest): Debug: Namespace 4archives: type=private, prefix=Archive/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=yes location=mdbox_deleted:/home/mail-archive/testtest/mdbox doveadm(testtest): Debug: fs: root=/home/mail-archive/testtest/mdbox, index=, indexpvt=, control=, inbox=, alt= doveadm(testtest): Debug: Archive/2018: Mailbox opened because: mailbox status doveadm(testtest): Error: Mailbox Archive/2018: Failed to lookup mailbox status: Mailbox doesn't exist: Archive/2018

[@ storage]# doveadm -D -o namespace/4archives/location=mdbox_deleted:/home/mail-archive/testtest/mdbox:INDEX=/home/archiveindex/testtest/index mailbox status -u testtest 'messages vsize' "Archive/2018" Debug: Loading modules from directory: /usr/lib64/dovecot Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib20_mail_log_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib20_replication_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib90_stats_plugin.so Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) doveadm(testtest): Debug: Effective uid=8267, gid=231, home=/home/popusers/testtest doveadm(testtest): Debug: replication: No mail_replica setting - replication disabled doveadm(testtest): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:~/mdbox:INDEX=/home/popindex/testtest/index:CONTROL=~/mail/control doveadm(testtest): Debug: fs: root=/home/popusers/testtest/mdbox, index=/home/popindex/testtest/index, indexpvt=, control=/home/popusers/testtest/mail/control, inbox=, alt= doveadm(testtest): Debug: Namespace 4archives: type=private, prefix=Archive/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=yes location=mdbox_deleted:/home/mail-archive/testtest/mdbox:INDEX=/home/archiveindex/testtest/index doveadm(testtest): Debug: fs: root=/home/mail-archive/testtest/mdbox, index=/home/archiveindex/testtest/index, indexpvt=, control=, inbox=, alt= doveadm(testtest): Debug: Archive/2018: Mailbox opened because: mailbox status doveadm(testtest): Error: Mailbox Archive/2018: Failed to lookup mailbox status: Mailbox doesn't exist: Archive/2018

[@ storage]# doveadm -D -o namespace/4archives/location=mdbox_deleted:/home/mail-archive/testtest/mdbox mailbox status -u testtest 'messages vsize' "Archive/2018" Debug: Loading modules from directory: /usr/lib64/dovecot Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib20_mail_log_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib20_replication_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib90_stats_plugin.so Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) doveadm(testtest): Debug: Effective uid=8267, gid=231, home=/home/popusers/testtest doveadm(testtest): Debug: replication: No mail_replica setting - replication disabled doveadm(testtest): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:~/mdbox:INDEX=/home/popindex/testtest/index:CONTROL=~/mail/control doveadm(testtest): Debug: fs: root=/home/popusers/testtest/mdbox, index=/home/popindex/testtest/index, indexpvt=, control=/home/popusers/testtest/mail/control, inbox=, alt= doveadm(testtest): Debug: Namespace 4archives: type=private, prefix=Archive/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=yes location=mdbox_deleted:/home/mail-archive/testtest/mdbox doveadm(testtest): Debug: fs: root=/home/mail-archive/testtest/mdbox, index=, indexpvt=, control=, inbox=, alt= doveadm(testtest): Debug: Archive/2018: Mailbox opened because: mailbox status doveadm(testtest): Error: Mailbox Archive/2018: Failed to lookup mailbox status: Mailbox doesn't exist: Archive/2018

Hmmm, terribly sorry, I think it is related to that Archive is stored in

[@]# doveconf | grep location mail_location = mdbox:~/mdbox:INDEX=/home/popindex/%u/index:CONTROL=~/mail/control location = mdbox:/home/mail-archive/%u/Archive/:CONTROL=/home/mail-archive/%u/Archive/control:INDEX=/home/archiveindex/%u/index location =

-----Original Message----- From: Aki Tuomi aki.tuomi@open-xchange.com Sent: 15 March 2021 11:24 To: Marc Marc@f1-outsourcing.eu; dovecot@dovecot.org Subject: RE: archive namespace reports mdbox_deleted: same as mdbox: is that ok?

...

On 15/03/2021 12:12 Marc marc@f1-outsourcing.eu wrote:

I am not sure if you wanted me to just add another -o, so I did these 3, but all with same results[1].

That's because you did it slightly wrong..

doveadm -D -o namespace/4archives/location=mdbox_deleted:/home/mail- archive/testtest/Archive/:CONTROL=/home/mail- archive/testtest/Archive/control:INDEX=/home/archiveindex/testtest/index -o testtest 'messages vsize' "Archive/2018"

Aki

On 15/03/2021 13:56 Marc marc@f1-outsourcing.eu wrote:

Sorry Aki for asking again. I need to run the purge, but I want to make sure some user did not accedentally deleted a large folder.

I decided to keep tests limited for now to the main inbox namespace. But also here weird results.

Status of folders before the test[1], selected 10 messages from INBOX/test2 and shift+deleted them. Then restarted thunderbird (otherwise thunderbird kept showing 211 messages), INBOX/test2 indeed shows 10 fewer messages, now 201, but no deleted messages found INBOX/test2 and weirdly 1 in INBOX??? (result is the same after a force-resync)

If this surpasses public support let me know.

[1] [@ testtest]# doveadm -v purge -u testtest

[@ testtest]# doveadm -o mail_location=mdbox:/home/popusers/testtest/mdbox:INDEX=/home/popindex/testtest/index mailbox status -u testtest 'messages vsize' '*' Sent messages=17 vsize=7473 Drafts messages=6 vsize=91785 INBOX/test2 messages=211 vsize=13443443 Junk messages=2 vsize=1969 Deleted Messages messages=1 vsize=956 Trash messages=47 vsize=769284 INBOX messages=1117 vsize=207623033

[@ testtest]# doveadm -o mail_location=mdbox_deleted:/home/popusers/testtest/mdbox:INDEX=/home/popindex/testtest/index mailbox status -u testtest 'messages vsize' '*' Sent messages=0 vsize=0 Drafts messages=0 vsize=0 INBOX/test2 messages=0 vsize=0 Junk messages=0 vsize=0 Deleted Messages messages=0 vsize=0 Trash messages=0 vsize=0 INBOX messages=0 vsize=0

[2] [@ testtest]# doveadm -o mail_location=mdbox:/home/popusers/testtest/mdbox:INDEX=/home/popindex/testtest/index mailbox status -u testtest 'messages vsize' '*' Sent messages=17 vsize=7473 Drafts messages=5 vsize=3457 INBOX/test2 messages=201 vsize=12806313 Junk messages=2 vsize=1969 Deleted Messages messages=1 vsize=956 Trash messages=47 vsize=769284 INBOX messages=1117 vsize=207623033

[@ testtest]# doveadm -o mail_location=mdbox_deleted:/home/popusers/testtest/mdbox:INDEX=/home/popindex/testtest/index mailbox status -u testtest 'messages vsize' '*' Sent messages=0 vsize=0 Drafts messages=0 vsize=0 INBOX/test2 messages=0 vsize=0 Junk messages=0 vsize=0 Deleted Messages messages=0 vsize=0 Trash messages=0 vsize=0 INBOX messages=1 vsize=88328

[3] [@]# doveconf | grep mail_location mail_location = mdbox:~/mdbox:INDEX=/home/popindex/%u/index:CONTROL=~/mail/control

Can you please be more specific about the steps you've taken and provide some steps how you did this testing including deleting mails and such? Now it's bit random and hard to track.

Aki

Adding these steps

step 6.

check if we have still them message stored, ok as expected

[@ testtest]# grep 45765765870 /home/popusers/testtest/mdbox/storage/* /home/popusers/testtest/mdbox/storage/m.893:Subject: New file uploaded 45765765870

step 7.

purge mailbox with

[@ testtest]# doveadm -v purge -u testtest

step 8.

check if we have still them message stored, no, ok as expected

[@ testtest]# grep 45765765870 /home/popusers/testtest/mdbox/storage/* [@ testtest]#

On 15/03/2021 15:50 Marc marc@f1-outsourcing.eu wrote:

...

step 5.

verify refcount=0 messages in folders, not ok, INBOX/test2 should have 5

[@ testtest]# doveadm -o mail_location=mdbox_deleted:/home/popusers/testtest/mdbox:INDEX=/home/po pindex/testtest/index mailbox status -u testtest 'messages vsize' '*' Sent messages=0 vsize=0 Drafts messages=0 vsize=0 INBOX/test2 messages=0 vsize=0 Junk messages=0 vsize=0 Deleted Messages messages=0 vsize=0 Deleted Messages/test messages=0 vsize=0 Trash messages=0 vsize=0 INBOX messages=3 vsize=90243

This sould list this deleted emails not? If I do doveadm search -u testtest mailbox 'INBOX/test2' DELETED, it lists these emails.

Nope, expunged only. If they just have DELETED flag, they are not expunged yet.

Aki