[Dovecot] mdbox-files over 2 MB
Hello,
I received an e-mail, this includes 9 forwarding, each with a file. These files are not removed from the e-mail and stored in the attachments-directory, although they are larger than 16 KB. For all files the "Content-Transfer-Encoding: base64" is set. The email is stored in completely in "m.00000034"
------------------------------------- doveconf:
# 2.2.12: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.4 ext4 mail_attachment_dir = /var/mail/attachments mail_attachment_hash = %{md5} mail_attachment_min_size = 16 k mail_location = mdbox:/var/mail/user/%n mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 2 M
------------------------------------- directory:
root@xxx:/var/mail/user/xxx/storage# ls -al ... -rw------- 1 vmail mail 14433784 Mär 16 13:51 m.00000034 ...
Quoting Reindl Harald h.reindl@thelounge.net:
Am 05.05.2014 16:10, schrieb Hardy Flor:
Is there really no one with this problem?
next time quote the problem instead demand others to seek for you in the archives
Even searching for him in the archives, I have no idea what he thinks
is a problem.
I know I have no issues using mdbox, with 2mb or 50mb rotate sizes.
On 5/5/2014 2:23 PM, Patrick Domack wrote:
Quoting Reindl Harald h.reindl@thelounge.net:
Am 05.05.2014 16:10, schrieb Hardy Flor:
Is there really no one with this problem?
next time quote the problem instead demand others to seek for you in the archives
Even searching for him in the archives, I have no idea what he thinks is a problem.
I know I have no issues using mdbox, with 2mb or 50mb rotate sizes.
He's wondering why many of his mdbox files are much smaller than 2 MB. His original post:
On 3/17/2014 1:29 AM, Hardy Flor wrote:
Hello,
there are copies with different size in 3 mailboxes of the user sequentially about 3800 emails. why not something 2MB files?
After the big file "m.00000034" with 14MB follow very many small ...
------------------------------------- doveconf:
# 2.2.12: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.4 ext4 mail_attachment_dir = /var/mail/attachments mail_attachment_hash = %{md5} mail_attachment_min_size = 16 k mail_location = mdbox:/var/mail/user/%n mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 2 M
------------------------------------- directory:
root@xxx:/var/mail/user/xxx/storage# ls -al insgesamt 76532 drwx--S--- 2 vmail mail 4096 Mär 16 13:52 . drwx--S--- 6 vmail mail 4096 Mär 16 12:10 .. -rw------- 1 vmail mail 74360 Mär 16 12:39 dovecot.map.index -rw------- 1 vmail mail 25204 Mär 16 13:52 dovecot.map.index.log -rw------- 1 vmail mail 63544 Mär 16 12:39 dovecot.map.index.log.2 -rw------- 1 vmail mail 2092921 Mär 16 12:15 m.00000001 -rw------- 1 vmail mail 2086746 Mär 16 12:15 m.00000002 -rw------- 1 vmail mail 2084182 Mär 16 12:15 m.00000003 -rw------- 1 vmail mail 2096826 Mär 16 12:15 m.00000004 -rw------- 1 vmail mail 1882666 Mär 16 12:15 m.00000005 -rw------- 1 vmail mail 1903965 Mär 16 12:15 m.00000006 -rw------- 1 vmail mail 2091169 Mär 16 12:15 m.00000007 -rw------- 1 vmail mail 2086396 Mär 16 12:15 m.00000008 -rw------- 1 vmail mail 507205 Mär 16 12:16 m.00000009 -rw------- 1 vmail mail 2031456 Mär 16 12:17 m.00000010 -rw------- 1 vmail mail 2095697 Mär 16 12:17 m.00000011 -rw------- 1 vmail mail 1689071 Mär 16 12:17 m.00000012 -rw------- 1 vmail mail 2092124 Mär 16 12:17 m.00000013 -rw------- 1 vmail mail 1950602 Mär 16 12:17 m.00000014 -rw------- 1 vmail mail 2092215 Mär 16 12:17 m.00000015 -rw------- 1 vmail mail 2087463 Mär 16 12:17 m.00000016 -rw------- 1 vmail mail 2079795 Mär 16 12:17 m.00000017 -rw------- 1 vmail mail 2014121 Mär 16 12:17 m.00000018 -rw------- 1 vmail mail 2081893 Mär 16 12:17 m.00000019 -rw------- 1 vmail mail 2092088 Mär 16 12:17 m.00000020 -rw------- 1 vmail mail 2090508 Mär 16 12:17 m.00000021 -rw------- 1 vmail mail 1929296 Mär 16 12:17 m.00000022 -rw------- 1 vmail mail 2067685 Mär 16 12:17 m.00000023 -rw------- 1 vmail mail 1745743 Mär 16 12:20 m.00000024 -rw------- 1 vmail mail 866452 Mär 16 12:20 m.00000025 -rw------- 1 vmail mail 296379 Mär 16 12:20 m.00000026 -rw------- 1 vmail mail 433541 Mär 16 12:39 m.00000027 -rw------- 1 vmail mail 92526 Mär 16 12:39 m.00000028 -rw------- 1 vmail mail 19094 Mär 16 12:39 m.00000029 -rw------- 1 vmail mail 106837 Mär 16 12:39 m.00000030 -rw------- 1 vmail mail 162012 Mär 16 12:39 m.00000031 -rw------- 1 vmail mail 412080 Mär 16 13:51 m.00000032 -rw------- 1 vmail mail 1715868 Mär 16 13:51 m.00000033 -rw------- 1 vmail mail 14433784 Mär 16 13:51 m.00000034 -rw------- 1 vmail mail 3280 Mär 16 13:51 m.00000035 -rw------- 1 vmail mail 6308 Mär 16 13:51 m.00000036 -rw------- 1 vmail mail 534274 Mär 16 13:51 m.00000037 -rw------- 1 vmail mail 378719 Mär 16 13:51 m.00000038 -rw------- 1 vmail mail 458528 Mär 16 13:51 m.00000039 -rw------- 1 vmail mail 192504 Mär 16 13:51 m.00000040 -rw------- 1 vmail mail 1277766 Mär 16 13:51 m.00000041 -rw------- 1 vmail mail 33417 Mär 16 13:51 m.00000042 -rw------- 1 vmail mail 142866 Mär 16 13:51 m.00000043 -rw------- 1 vmail mail 113529 Mär 16 13:51 m.00000044 -rw------- 1 vmail mail 55888 Mär 16 13:51 m.00000045 -rw------- 1 vmail mail 673504 Mär 16 13:51 m.00000046 -rw------- 1 vmail mail 430738 Mär 16 13:51 m.00000047 -rw------- 1 vmail mail 218494 Mär 16 13:51 m.00000048 -rw------- 1 vmail mail 96165 Mär 16 13:51 m.00000049 -rw------- 1 vmail mail 8370 Mär 16 13:51 m.00000050 -rw------- 1 vmail mail 29251 Mär 16 13:51 m.00000051 -rw------- 1 vmail mail 394680 Mär 16 13:51 m.00000052 -rw------- 1 vmail mail 674892 Mär 16 13:51 m.00000053 -rw------- 1 vmail mail 1076919 Mär 16 13:51 m.00000054 -rw------- 1 vmail mail 258806 Mär 16 13:51 m.00000055 -rw------- 1 vmail mail 60069 Mär 16 13:51 m.00000056 -rw------- 1 vmail mail 297891 Mär 16 13:51 m.00000057 -rw------- 1 vmail mail 92772 Mär 16 13:51 m.00000058 -rw------- 1 vmail mail 129272 Mär 16 13:51 m.00000059 -rw------- 1 vmail mail 56073 Mär 16 13:51 m.00000060 -rw------- 1 vmail mail 528260 Mär 16 13:51 m.00000061 -rw------- 1 vmail mail 29468 Mär 16 13:51 m.00000062 -rw------- 1 vmail mail 3250 Mär 16 13:51 m.00000063 -rw------- 1 vmail mail 96445 Mär 16 13:51 m.00000064 -rw------- 1 vmail mail 337663 Mär 16 13:51 m.00000065 -rw------- 1 vmail mail 187505 Mär 16 13:51 m.00000066 -rw------- 1 vmail mail 98839 Mär 16 13:51 m.00000067 -rw------- 1 vmail mail 5294 Mär 16 13:51 m.00000068 -rw------- 1 vmail mail 28276 Mär 16 13:51 m.00000069 -rw------- 1 vmail mail 67950 Mär 16 13:52 m.00000070 -rw------- 1 vmail mail 704457 Mär 16 13:52 m.00000071 -rw------- 1 vmail mail 1410205 Mär 16 13:52 m.00000072 -rw------- 1 vmail mail 736330 Mär 16 13:52 m.00000073 -rw------- 1 vmail mail 616992 Mär 16 13:52 m.00000074
I keep the quoted message complete because the original thread was posted six months ago. I am top-posting, no comments inside the quoted text.
I plan to write a blog about mdbox oddities but in the meantime I can tell you that if you store attachments "detached" in your mdbox storage the issue is this:
You have a 200Kbytes MB m.* file you are currently targeting.
You have a new email to store. Lets say it has a 3MB attachment.
You have a 2MB mdbox max file size .
Dovecot notices that 0.2+3 > 2, so it will create a new m.* file.
When storing the message, it will store the attachment outside of the mbox, so the new m.* file will be, lets say, 5Kbytes only.
Now you have 2 m.* files: one of 200 Kbytes and other of 5Kbytes.
Dovecot tries to reuse undersized m.* files, but it is quite forgiving. I have some private patches to increase its agressiveness.
If you don't detach attachment, you probably have the same issue: when a new message makes a m.* file to oversize, dovecot creates a new m.* file and leave the old one alone, so it will be undersized.
On 06/05/14 03:05, Stan Hoeppner wrote:
On 5/5/2014 2:23 PM, Patrick Domack wrote:
Quoting Reindl Harald h.reindl@thelounge.net:
Am 05.05.2014 16:10, schrieb Hardy Flor:
Is there really no one with this problem?
next time quote the problem instead demand others to seek for you in the archives
Even searching for him in the archives, I have no idea what he thinks is a problem.
I know I have no issues using mdbox, with 2mb or 50mb rotate sizes.
He's wondering why many of his mdbox files are much smaller than 2 MB. His original post:
On 3/17/2014 1:29 AM, Hardy Flor wrote:
Hello,
there are copies with different size in 3 mailboxes of the user sequentially about 3800 emails. why not something 2MB files?
After the big file "m.00000034" with 14MB follow very many small ...
------------------------------------- doveconf:
# 2.2.12: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.4 ext4 mail_attachment_dir = /var/mail/attachments mail_attachment_hash = %{md5} mail_attachment_min_size = 16 k mail_location = mdbox:/var/mail/user/%n mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 2 M
------------------------------------- directory:
root@xxx:/var/mail/user/xxx/storage# ls -al insgesamt 76532 drwx--S--- 2 vmail mail 4096 Mär 16 13:52 . drwx--S--- 6 vmail mail 4096 Mär 16 12:10 .. -rw------- 1 vmail mail 74360 Mär 16 12:39 dovecot.map.index -rw------- 1 vmail mail 25204 Mär 16 13:52 dovecot.map.index.log -rw------- 1 vmail mail 63544 Mär 16 12:39 dovecot.map.index.log.2 -rw------- 1 vmail mail 2092921 Mär 16 12:15 m.00000001 -rw------- 1 vmail mail 2086746 Mär 16 12:15 m.00000002 -rw------- 1 vmail mail 2084182 Mär 16 12:15 m.00000003 -rw------- 1 vmail mail 2096826 Mär 16 12:15 m.00000004 -rw------- 1 vmail mail 1882666 Mär 16 12:15 m.00000005 -rw------- 1 vmail mail 1903965 Mär 16 12:15 m.00000006 -rw------- 1 vmail mail 2091169 Mär 16 12:15 m.00000007 -rw------- 1 vmail mail 2086396 Mär 16 12:15 m.00000008 -rw------- 1 vmail mail 507205 Mär 16 12:16 m.00000009 -rw------- 1 vmail mail 2031456 Mär 16 12:17 m.00000010 -rw------- 1 vmail mail 2095697 Mär 16 12:17 m.00000011 -rw------- 1 vmail mail 1689071 Mär 16 12:17 m.00000012 -rw------- 1 vmail mail 2092124 Mär 16 12:17 m.00000013 -rw------- 1 vmail mail 1950602 Mär 16 12:17 m.00000014 -rw------- 1 vmail mail 2092215 Mär 16 12:17 m.00000015 -rw------- 1 vmail mail 2087463 Mär 16 12:17 m.00000016 -rw------- 1 vmail mail 2079795 Mär 16 12:17 m.00000017 -rw------- 1 vmail mail 2014121 Mär 16 12:17 m.00000018 -rw------- 1 vmail mail 2081893 Mär 16 12:17 m.00000019 -rw------- 1 vmail mail 2092088 Mär 16 12:17 m.00000020 -rw------- 1 vmail mail 2090508 Mär 16 12:17 m.00000021 -rw------- 1 vmail mail 1929296 Mär 16 12:17 m.00000022 -rw------- 1 vmail mail 2067685 Mär 16 12:17 m.00000023 -rw------- 1 vmail mail 1745743 Mär 16 12:20 m.00000024 -rw------- 1 vmail mail 866452 Mär 16 12:20 m.00000025 -rw------- 1 vmail mail 296379 Mär 16 12:20 m.00000026 -rw------- 1 vmail mail 433541 Mär 16 12:39 m.00000027 -rw------- 1 vmail mail 92526 Mär 16 12:39 m.00000028 -rw------- 1 vmail mail 19094 Mär 16 12:39 m.00000029 -rw------- 1 vmail mail 106837 Mär 16 12:39 m.00000030 -rw------- 1 vmail mail 162012 Mär 16 12:39 m.00000031 -rw------- 1 vmail mail 412080 Mär 16 13:51 m.00000032 -rw------- 1 vmail mail 1715868 Mär 16 13:51 m.00000033 -rw------- 1 vmail mail 14433784 Mär 16 13:51 m.00000034 -rw------- 1 vmail mail 3280 Mär 16 13:51 m.00000035 -rw------- 1 vmail mail 6308 Mär 16 13:51 m.00000036 -rw------- 1 vmail mail 534274 Mär 16 13:51 m.00000037 -rw------- 1 vmail mail 378719 Mär 16 13:51 m.00000038 -rw------- 1 vmail mail 458528 Mär 16 13:51 m.00000039 -rw------- 1 vmail mail 192504 Mär 16 13:51 m.00000040 -rw------- 1 vmail mail 1277766 Mär 16 13:51 m.00000041 -rw------- 1 vmail mail 33417 Mär 16 13:51 m.00000042 -rw------- 1 vmail mail 142866 Mär 16 13:51 m.00000043 -rw------- 1 vmail mail 113529 Mär 16 13:51 m.00000044 -rw------- 1 vmail mail 55888 Mär 16 13:51 m.00000045 -rw------- 1 vmail mail 673504 Mär 16 13:51 m.00000046 -rw------- 1 vmail mail 430738 Mär 16 13:51 m.00000047 -rw------- 1 vmail mail 218494 Mär 16 13:51 m.00000048 -rw------- 1 vmail mail 96165 Mär 16 13:51 m.00000049 -rw------- 1 vmail mail 8370 Mär 16 13:51 m.00000050 -rw------- 1 vmail mail 29251 Mär 16 13:51 m.00000051 -rw------- 1 vmail mail 394680 Mär 16 13:51 m.00000052 -rw------- 1 vmail mail 674892 Mär 16 13:51 m.00000053 -rw------- 1 vmail mail 1076919 Mär 16 13:51 m.00000054 -rw------- 1 vmail mail 258806 Mär 16 13:51 m.00000055 -rw------- 1 vmail mail 60069 Mär 16 13:51 m.00000056 -rw------- 1 vmail mail 297891 Mär 16 13:51 m.00000057 -rw------- 1 vmail mail 92772 Mär 16 13:51 m.00000058 -rw------- 1 vmail mail 129272 Mär 16 13:51 m.00000059 -rw------- 1 vmail mail 56073 Mär 16 13:51 m.00000060 -rw------- 1 vmail mail 528260 Mär 16 13:51 m.00000061 -rw------- 1 vmail mail 29468 Mär 16 13:51 m.00000062 -rw------- 1 vmail mail 3250 Mär 16 13:51 m.00000063 -rw------- 1 vmail mail 96445 Mär 16 13:51 m.00000064 -rw------- 1 vmail mail 337663 Mär 16 13:51 m.00000065 -rw------- 1 vmail mail 187505 Mär 16 13:51 m.00000066 -rw------- 1 vmail mail 98839 Mär 16 13:51 m.00000067 -rw------- 1 vmail mail 5294 Mär 16 13:51 m.00000068 -rw------- 1 vmail mail 28276 Mär 16 13:51 m.00000069 -rw------- 1 vmail mail 67950 Mär 16 13:52 m.00000070 -rw------- 1 vmail mail 704457 Mär 16 13:52 m.00000071 -rw------- 1 vmail mail 1410205 Mär 16 13:52 m.00000072 -rw------- 1 vmail mail 736330 Mär 16 13:52 m.00000073 -rw------- 1 vmail mail 616992 Mär 16 13:52 m.00000074
-- Jesús Cea Avión _/_/ _/_/_/ _/_/_/ jcea@jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ Twitter: @jcea _/_/ _/_/ _/_/_/_/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz
Sorry, I thought the tree display of threads is sufficient. I did not want to quote much.
Am 05.05.2014 16:16, schrieb Reindl Harald:
Am 05.05.2014 16:10, schrieb Hardy Flor:
Is there really no one with this problem? next time quote the problem instead demand others to seek for you in the archives
On May 6, 2014, at 5:17 AM, Hardy Flor HFlor@gmx.de wrote:
Sorry, I thought the tree display of threads is sufficient. I did not want to quote much.
Believe it or not, not all of us save every message or have the same MUA capabilities. Don’t assume how you read is the same way everyone does. Sometimes, what starts as something uninteresting and deleted immediately turns interesting. Therefore, quoting the relevant text from previous message is needed.
-- Larry Stone lstone19@stonejongleux.com http://www.stonejongleux.com/
Am 17.03.2014 07:38, schrieb Hardy Flor:
Hello,
I received an e-mail, this includes 9 forwarding, each with a file. These files are not removed from the e-mail and stored in the attachments-directory, although they are larger than 16 KB. For all files the "Content-Transfer-Encoding: base64" is set. The email is stored in completely in "m.00000034"
------------------------------------- doveconf:
# 2.2.12: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.4 ext4 mail_attachment_dir = /var/mail/attachments mail_attachment_hash = %{md5} mail_attachment_min_size = 16 k mail_location = mdbox:/var/mail/user/%n mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 2 M
------------------------------------- directory:
root@xxx:/var/mail/user/xxx/storage# ls -al ... -rw------- 1 vmail mail 14433784 Mär 16 13:51 m.00000034 ... Has this really not a solution? I've found some e-mails where the attachments are not stored separately.
participants (6)
-
Hardy Flor
-
Jesus Cea
-
Larry Stone
-
Patrick Domack
-
Reindl Harald
-
Stan Hoeppner