[Dovecot] v2.1 memory usage

[Timo Sirainen]

On Tue, 2012-09-11 at 21:43 +0300, Timo Sirainen wrote:
> v2.1's imap/pop3 processes are linked with OpenSSL by default, because of imapc/pop3c storages. I wonder if you notice any actual (system-global) memory usage reduction when compiling without those and you have thousands of imap processes?
> 
> ./configure --with-storages=mdbox sdbox maildir mbox cydir
> 
> I guess I could try it myself also but maybe someone else wanting to reduce memory usage is quicker? :)
> 
> Related to that, you can look at where imap processes' memory goes with: http://dovecot.org/tools/linux-maps-parse-imap.pl
> I don't know how much memory those (writable!) file-backed memory areas actually use per-process. Maybe nothing, maybe something..

http://dovecot.org/tmp/proctest.sh can be used to easily test this.
Looks like simply linking a binary with -lcrypto increases each
process's memory usage by something like:

 * RHEL5: 50 kB
 * RHEL6/Debian/Ubuntu: 150-200 kB
 * Solaris: 100 kB
 * OpenBSD: 20 kB
 * OSX: 0 kB

Looks like it's the linker that uses up all of this memory. I'm not sure
why exactly though, I couldn't create a test library that had the same
effect. 

I also tried imap binary that was statically built vs. linked with
libdovecot*.so, and the difference was pretty small (.so used maybe
10-20k more).

Anyway, looks like Dovecot can't link OpenSSL to imap/pop3 processes
without wasting a ton of memory. In v2.2 I already moved imapc/pop3c
backend code to plugins to avoid this. Looks like similar ugliness is
needed for other features/backends also that may end up using SSL code.
(We were wondering with Stephan what to do about his new HTTP library
code that added support for SSL. It would be nice to keep it in the core
libdovecot.so, but not if it links with SSL. So looks like we'll need
some kind of a http-ssl plugin that is loaded only when needed.)