Dovecot v2.3.13 reporting (very) incorrect vsize for some maildir folders

[Eirik Rye]

Hello!

Dovecot v2.3.13. Full, but anonymized, `doveconf -n` attached.

We are in the process of migrating maildir-backed users from filesystem quotes, to using dovecot's "count" quota backend.

This is not reflected in the doveconf output because we're overriding `quota`, `quota_rule` and `quota_vsizes` in userdb:

  {"quota":"count:User quota","quota_rule":"*:storage=15G","quota_vsizes":"yes"}

Afterwards, for a couple users, we received reports that mail delivery had failed because they were over quota. When looking into it for a specific user, we noticed that the vsize reported for a particular folder (with 47k messages) was reported as being nearly 50 times larger than it's on-disk size:

root at mail02:~# doveadm mailbox status -u anonymized_user 'messages recent unseen vsize' 'anonymized/folder/name'
[...]
anonymized/folder/name messages=47338 recent=0 unseen=0 vsize=14335366070

However, the filesystem itself reports a much smaller size (but correct message count):

root at mail02:~# du -hd1 /home/anonymized_user/Maildir/.anonymized.folder.name/
313M    /home/anonymized_user/Maildir/.anonymized.folder.name/cur
36K     /home/anonymized_user/Maildir/.anonymized.folder.name/tmp
4.0K    /home/anonymized_user/Maildir/.anonymized.folder.name/new
320M    /home/anonymized_user/Maildir/.anonymized.folder.name/

root at mail02:~# ls /home/anonymized_user/Maildir/.anonymized.folder.name/cur | wc -l
47338

I have tried:

  - `doveadm force-resync -u anonymized_user`
  - deleting the index files in the specific folder, and running `doveadm index -u anonymized_user '*'` as well as `doveadm mailbox status -u anonymized_user vsize '*'`
  - deleting all `*index*` files in maildir, and running `doveadm index -u anonymized_user '*'` as well as `doveadm mailbox status -u anonymized_user vsize '*'`
  - comparing all maildir-sizes (S=) with their actual sizes to see if there are discrepancies. There are none.
  - regardless of the above, set `maildir_broken_filename_sizes = yes`, deleted indexes and reindexed. 

Even after deleting the list index (dovecot.list.index) as well as the mailbox indexes, recalculating the vsize seems very quick so I feel like the incorrect vsize is being fetched from a cache somewhere instead of being recalculated..?

Any idea what is causing dovecot to report this wildly incorrect mailbox size? Are there any other files (not matching `*index*`) that are responsible? I'd rather not touch the user's control files, for obvious reasons.

On a side note: We're also experiencing some issues with `mail_vsize_bg_after_count`. It seems to work correctly by returning a temporary error and deferring to a background job, but the indexer-worker job often times never appears to start (or do anything) at all:

dovecot  29622  0.0  0.0   4120  1124 ?        S    15:22   0:00 dovecot/indexer [0 clients, 0 requests]
266248   29623  0.0  0.0   6008  4604 ?        S    15:22   0:00 dovecot/indexer-worker [idling]

Waiting a while, and attempting to refetch the quota returns the same temperror with no indexer-worker being started. However, this is a separate issue from the above and can be ignored for now.

Best regards,
Eirik Rye

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: doveconf.txt
URL: <https://dovecot.org/pipermail/dovecot/attachments/20210505/201c2b14/attachment.txt>