<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
color:black;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
color:black;
mso-fareast-language:EN-AU;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-AU" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:windowtext">Looks like its working. Haven’t had any recurrence of the issue since scheduling a move operation daily.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">Thanks for your help.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext">Cheers.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:windowtext"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="color:windowtext;mso-fareast-language:EN-AU">From:</span></b><span lang="EN-US" style="color:windowtext;mso-fareast-language:EN-AU"> Aki Tuomi [mailto:aki.tuomi@dovecot.fi]
<br>
<b>Sent:</b> Friday, 16 February 2018 3:17 PM<br>
<b>To:</b> Terence Lau <terence.lau@royhill.com.au>; dovecot@dovecot.org<br>
<b>Subject:</b> Re: Out of memory on lmtp vsz_limit<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p>How about you try moving the mail into another folder on daily basis, this way the INBOX would stay nice and empty.<span style="mso-fareast-language:EN-AU"><o:p></o:p></span></p>
<p>doveadm move -u <a href="mailto:archive@company.com">archive@company.com</a> Archive MAILBOX INBOX SENTBEFORE todays-date<o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Aki<o:p></o:p></p>
<div>
<p class="MsoNormal">On 16.02.2018 06:19, Terence Lau wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">Bump.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Any advice would be most appreciated.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Thanks.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="mso-fareast-language:EN-AU">From:</span></b><span lang="EN-US" style="mso-fareast-language:EN-AU"> Terence Lau
<br>
<b>Sent:</b> Wednesday, 24 January 2018 9:59 AM<br>
<b>To:</b> '<a href="mailto:dovecot@dovecot.org">dovecot@dovecot.org</a>' <a href="mailto:dovecot@dovecot.org">
<dovecot@dovecot.org></a><br>
<b>Subject:</b> Out of memory on lmtp vsz_limit</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">We’ve been getting these types or errors for quite a while now …<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Fatal: master: service(lmtp): child 63477 returned error 83 (Out of memory (service lmtp { vsz_limit=4096 MB }, you may need to increase it)<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">… and these errors have been decreasing in occurrence as we increased the default_vsz_limit. Which is good but I would like to get some advice on how I could possibly eliminate the errors.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">We have an internal smtp server (postfix 3.1.0) that has the config “<a href="mailto:always_bcc=archive@company.com">always_bcc=archive@company.com</a>” over lmtp. This mailbox is on a separate dovecot server with the following config
(please let me know if the full config is required):<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"># 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf<o:p></o:p></p>
<p class="MsoNormal"># Pigeonhole version 0.4.13 (7b14904)<o:p></o:p></p>
<p class="MsoNormal"># OS: Linux 4.4.0-109-generic x86_64 Ubuntu 16.04.1 LTS ext4<o:p></o:p></p>
<p class="MsoNormal">default_vsz_limit = 4 G<o:p></o:p></p>
<p class="MsoNormal">mail_location = maildir:/home/vmail/%d/%n<o:p></o:p></p>
<p class="MsoNormal">protocols = " imap lmtp pop3"<o:p></o:p></p>
<p class="MsoNormal">service lmtp {<o:p></o:p></p>
<p class="MsoNormal"> inet_listener lmtp {<o:p></o:p></p>
<p class="MsoNormal"> port = 24<o:p></o:p></p>
<p class="MsoNormal"> }<o:p></o:p></p>
<p class="MsoNormal">}<o:p></o:p></p>
<p class="MsoNormal">userdb {<o:p></o:p></p>
<p class="MsoNormal"> args = username_format=%u /etc/dovecot/users<o:p></o:p></p>
<p class="MsoNormal"> default_fields = uid=vmail gid=vmail home=/home/vmail/%d/%n<o:p></o:p></p>
<p class="MsoNormal"> driver = passwd-file<o:p></o:p></p>
<p class="MsoNormal">}<o:p></o:p></p>
<p class="MsoNormal">protocol lmtp {<o:p></o:p></p>
<p class="MsoNormal"> mail_plugins =<o:p></o:p></p>
<p class="MsoNormal">}<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Since we discovered the errors, we’ve been increasing the default_vsz_limit to 1G, then 2G and now 4G (Server has 6GB of memory). These errors occur whenever a large number of emails get sent around the same time to our smtp server. This
causes the dovecot server to start crunching CPU and Memory. Load average goes through the roof and takes some time to come back down as the smtp queue clears itself.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">This mailbox is obviously very large but we have a script that runs daily to delete any emails older than a month:<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">find /home/vmail/company.com/archive/new/ -type f -mtime +30 -exec rm {} \;<o:p></o:p></p>
<p class="MsoNormal">find /home/vmail/company.com/archive/cur/ -type f -mtime +30 -exec rm {} \;<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Still, the mailbox has on average of just under 300,000 emails. No one accesses this mailbox with an email client, not until we need to dig something up. And this has only happen once. So the emails pretty much never get read/process
by a user.<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Now that we’ve increased the default_vsz_limit to 4G, the occurrence of these errors has reduced considerably. But they still happen occasionally. Short of increasing the memory further, are there any other options I have?<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Thanks.<o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><span style="mso-fareast-language:EN-AU"><o:p> </o:p></span></p>
</div>
</body>
</html>