[Dovecot] Large subjects increase memory-usage and enlarge index-files

[Steve Litt]

On Sat, 06 Oct 2012 23:32:56 +0200, Peer Heinlein said:
> 
> Several times we already had the problems, that accounts with more the
> 1.3 or 1.7 billion e-mails in one folder run out-of-memory, even if
> vsize_limit of 750 MB is set.
> 
> In this case, the lmtpd-process haven't been able to allocate more
> memory to read/write/update the index-files and crashed (and the
> index-files become corrupted at the end.)
> 
> [Please -- don't discuss about the need of INBOXes with 1.7 million
> (unread) e-mails (don't discuss that with ME. Personally, I agree,
> that there's NO need for that...).]
> 
> But: We also noticed accounts with ~ 300.000 e-Mails running out of
> memory in the same situations. This happends, if the subject is very
> large (subject or some other header attributes).
> 
> And: We've been able to reproduce out-of-memory-Problems with just
> 13.000 e-mails with VERY long subjects (e.g.: network monitoring
> status informations), even with a vsize_limit of 750 MB (which is
> already very much).
> 
> 13.000 e-mails isn't very much. And it's easy to inject several
> thousands of prepared e-mails.
> 
> Having many mails for accounts with huge (and broken) index-files
> slows down the delivery rate VERY much and increases the need for
> memory and cpu resources and I/O very much.
> 
> So: This could be used for a very easy to do denial-of-service attac
> against Dovecot-based mailservers.
> 
> I don't have a clear solution for that, Dovecot needs the subject
> information in its index files. But it looks like, it isn't a good
> idea to put the whole subject into the index. Maybe it's
> better/necessary to use just the first 50-70 characters for that and
> to keep the rest away from the index?
> 
> I think I would prefer that even if that means, that accessing those
> folders with "special" e-mails will become slower because Dovecot has
> to get those informations directly from the e-mail.
> 
> This performance issue is just a problem for the user.
> 
> But crashing lmtpd-processes and lowering the delivery rate is a
> *real* problem for the whole IMAP-cluster.
> 
> Peer

While the real solution is being decided, can I avoid this possible DOS
attack by using procmail to /dev/null anything with more than a 256
byte subject, before it ever gets to Dovecot IMAP?

Thanks

SteveT

Steve Litt                *  http://www.troubleshooters.com/
                          *  http://twitter.com/stevelitt
Troubleshooting Training  *  Human Performance