[Dovecot] Large subjects increase memory-usage and enlarge index-files

[Peer Heinlein]

Several times we already had the problems, that accounts with more the
1.3 or 1.7 billion e-mails in one folder run out-of-memory, even if
vsize_limit of 750 MB is set.

In this case, the lmtpd-process haven't been able to allocate more
memory to read/write/update the index-files and crashed (and the
index-files become corrupted at the end.)

[Please -- don't discuss about the need of INBOXes with 1.7 million
(unread) e-mails (don't discuss that with ME. Personally, I agree, that
there's NO need for that...).]

But: We also noticed accounts with ~ 300.000 e-Mails running out of
memory in the same situations. This happends, if the subject is very
large (subject or some other header attributes).

And: We've been able to reproduce out-of-memory-Problems with just
13.000 e-mails with VERY long subjects (e.g.: network monitoring status
informations), even with a vsize_limit of 750 MB (which is already very
much).

13.000 e-mails isn't very much. And it's easy to inject several
thousands of prepared e-mails.

Having many mails for accounts with huge (and broken) index-files slows
down the delivery rate VERY much and increases the need for memory and
cpu resources and I/O very much.

So: This could be used for a very easy to do denial-of-service attac
against Dovecot-based mailservers.

I don't have a clear solution for that, Dovecot needs the subject
information in its index files. But it looks like, it isn't a good idea
to put the whole subject into the index. Maybe it's better/necessary to
use just the first 50-70 characters for that and to keep the rest away
from the index?

I think I would prefer that even if that means, that accessing those
folders with "special" e-mails will become slower because Dovecot has to
get those informations directly from the e-mail.

This performance issue is just a problem for the user.

But crashing lmtpd-processes and lowering the delivery rate is a *real*
problem for the whole IMAP-cluster.

Peer



-- 
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berlin

http://www.heinlein-support.de

Tel: 030 / 405051-42
Fax: 030 / 405051-19

Zwangsangaben lt. §35a GmbHG:
HRB 93818 B / Amtsgericht Berlin-Charlottenburg,
Geschäftsführer: Peer Heinlein -- Sitz: Berlin