Hello everyone,
Our server has these installed:
dovecot-2.0.21-2.el6.x86_64 dovecot-pigeonhole-2.0.21-2.el6.x86_64 dovecot-mysql-2.0.21-2.el6.x86_64
...and has been running for quite a long time, with several hundred domains and thousands of accounts on it. My colleagues reported that it's been showing quota usage values that are more than actual disk usage, so I investigated and found that Dovecot shows a quota usage value roughly around 140% of actual disk usage. It's also valid on newly created accounts. My test account for example:
doveadm quota get -u test@example.local Quota name Type Value Limit % User quota STORAGE 4359 512000 0 User quota MESSAGE 7 - 0
du -sc /home/vmail/example.local/test/Maildir/{*,.[!.]*} 1044 /home/vmail/example.local/test/Maildir/cur 28 /home/vmail/example.local/test/Maildir/dovecot.index.cache 8 /home/vmail/example.local/test/Maildir/dovecot.index.log 4 /home/vmail/example.local/test/Maildir/dovecot.mailbox.log 4 /home/vmail/example.local/test/Maildir/dovecot-uidlist 4 /home/vmail/example.local/test/Maildir/dovecot-uidvalidity 0 /home/vmail/example.local/test/Maildir/dovecot-uidvalidity.56a4dc8e 4 /home/vmail/example.local/test/Maildir/new 4 /home/vmail/example.local/test/Maildir/subscriptions 4 /home/vmail/example.local/test/Maildir/tmp 24 /home/vmail/example.local/test/Maildir/.Junk 1932 /home/vmail/example.local/test/Maildir/.Sent 44 /home/vmail/example.local/test/Maildir/.Trash 3104 total
4359/3104=1,40431701
So it shows roughly around 1,4*actualDiskUsage.
The ratio is mostly the same for almost all the other accounts. It can vary between like 1,3-1,6. So, the gap gets insane when more disk space is used, say like with 2GB disk usage, Dovecot thinks 3,5GB quota is used...
dovecot quota recalc does not fix the issue, it only sets the same value again (I've checked with tcpdump and saw the query with the same quota usage value).
The method is Dictionary quota with SQL.
I'm attaching the dovecot -n output with some other config files.
I've tried setting messages and bytes value to -1 on the MariaDB database to force recalculation. But as soon as I run doveadm quota recalc, it gets the same wrong value again.
What can I do to fix this?
Thanks in advance.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, 24 Feb 2017, Umut Erol Kaçar wrote:
investigated and found that Dovecot shows a quota usage value roughly around 140% of actual disk usage. It's also valid on newly created accounts. My test account for example:
Quota does not count physical useage, but the amount of bytes allocated by the messages. Maildir may hardlink messages, hence, they count multiple times for the quota, but once for du.
The ratio is mostly the same for almost all the other accounts. It can vary between like 1,3-1,6. So, the gap gets insane when more disk space is used, say like with 2GB disk usage, Dovecot thinks 3,5GB quota is used...
Hmm, are you sure?
dovecot quota recalc does not fix the issue, it only sets the same value again (I've checked with tcpdump and saw the query with the same quota
if quota recalc re-creates the same value, please check the hard link stuff.
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1
iQEVAwUBWLBKfnz1H7kL/d9rAQK0ugf9EmPnt5zWLiWeCxsRi9iFHAcEFW6qE4be 98G2N5gzkyyQZZsPkaSQ05fVj81zijeArFn0DwtnXJzhDHkCzI4C2yd1LgwbgnFg /wakFkjW8jd7m58Xv45UEmKlZC83o7mOQPNYqp23v2M7k06y4XDnGpNf8V5Ipu/F IaoK7ZS9DkaIrF6Rv6+WGtlLjjUj4By0ZJk6mGD3eNsTnQmM6j+tEcv75WXJMhNP M7QkqwqPaj/oPVUfNut+e5VzB5Y/w1UAJpHMAnsfXK5MC9DnzU4OQtoxhWtcLzm/ HbTs7rfGbGtcnLh9tgMtMy43awqSHe9V4v0/DASL/0oCQbFFo0AEmg== =71mb -----END PGP SIGNATURE-----
Am 24.02.2017 um 16:00 schrieb Steffen Kaiser:
Quota does not count physical useage, but the amount of bytes allocated by the messages. Maildir may hardlink messages, hence, they count multiple times for the quota, but once for du.
And in your case dovecot even compressed the mails: According to your doveconf, you are using mail_plugins = [...] zlib.
Dovecot's quota is calculated using the uncompressed size, whereas du shows you the space actually allocated.
You're right, it's the zlib compression. It works as described here: http://wiki2.dovecot.org/Plugins/Zlib It appends the before-compression size to the file name like S=<size> and uses them for quota usage calculation.
So, when I run the following command on a Maildir, the result matches the value on the database. (size value on the quota table)
find . -type f | grep S= | awk -F, '{ gsub("S=","",$2); SUM+=$2 } END { print SUM }'
And the compression ratios relate to the difference between actual disk space usage and calculated quota usage values when I check like this, on a Maildir:
find . -type f -ls | grep S= | awk -F, '{ gsub("S=","",$2); print $2, $1 }' | awk '{ print "file: "$12,"\t orig: ",$1," comp.: ",$8," ratio: ",$1 / $8}'
Thanks a lot!
25 Şub 2017 Cmt, 00:07 tarihinde, Karsten Heiken < heiken@luis.uni-hannover.de> şunu yazdı:
Am 24.02.2017 um 16:00 schrieb Steffen Kaiser:
Quota does not count physical useage, but the amount of bytes allocated
by the messages. Maildir may hardlink messages, hence, they count multiple times for the quota, but once for du.
And in your case dovecot even compressed the mails: According to your doveconf, you are using mail_plugins = [...] zlib.
Dovecot's quota is calculated using the uncompressed size, whereas du shows you the space actually allocated.
participants (3)
-
Karsten Heiken
-
Steffen Kaiser
-
Umut Erol Kaçar