[Dovecot] v2.1 memory usage
v2.1's imap/pop3 processes are linked with OpenSSL by default, because of imapc/pop3c storages. I wonder if you notice any actual (system-global) memory usage reduction when compiling without those and you have thousands of imap processes?
./configure --with-storages=mdbox sdbox maildir mbox cydir
I guess I could try it myself also but maybe someone else wanting to reduce memory usage is quicker? :)
Related to that, you can look at where imap processes' memory goes with: http://dovecot.org/tools/linux-maps-parse-imap.pl I don't know how much memory those (writable!) file-backed memory areas actually use per-process. Maybe nothing, maybe something..
Only have 650 imap clients this weekend, have to give it a try during
a weekday.
Total(MB) Avg(kB) Name 2 4 /lib/ld-2.11.1.so 2 4 /lib/libbz2.so.1.0.4 2 4 /lib/libc-2.11.1.so 63 96 /lib/libcrypto.so.0.9.8 2 4 /lib/libdl-2.11.1.so 0 0 /lib/libnss_dns-2.11.1.so 0 0 /lib/libnss_files-2.11.1.so 2 4 /lib/libpthread-2.11.1.so 0 0 /lib/libresolv-2.11.1.so 2 4 /lib/librt-2.11.1.so 13 20 /lib/libssl.so.0.9.8 2 4 /lib/libz.so.1.2.3.3 2 4 /usr/lib/dovecot/imap 10 16 /usr/lib/dovecot/libdovecot-storage.so.0.0.0 2 4 /usr/lib/dovecot/libdovecot.so.0.0.0 2 4 /usr/lib/dovecot/modules/lib10_quota_plugin.so 2 4 /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so 2 4 /usr/lib/dovecot/modules/lib20_zlib_plugin.so 2 4 /usr/lib/dovecot/modules/lib30_imap_zlib_plugin.so 435 661 [anonymous] 640 973 [heap] 55 84 [stack] 1253 58352 TOTAL
Quoting Timo Sirainen tss@iki.fi:
v2.1's imap/pop3 processes are linked with OpenSSL by default,
because of imapc/pop3c storages. I wonder if you notice any actual
(system-global) memory usage reduction when compiling without those
and you have thousands of imap processes?./configure --with-storages=mdbox sdbox maildir mbox cydir
I guess I could try it myself also but maybe someone else wanting to
reduce memory usage is quicker? :)Related to that, you can look at where imap processes' memory goes
with: http://dovecot.org/tools/linux-maps-parse-imap.pl I don't know how much memory those (writable!) file-backed memory
areas actually use per-process. Maybe nothing, maybe something..
On Tue, 2012-09-11 at 21:43 +0300, Timo Sirainen wrote:
v2.1's imap/pop3 processes are linked with OpenSSL by default, because of imapc/pop3c storages. I wonder if you notice any actual (system-global) memory usage reduction when compiling without those and you have thousands of imap processes?
./configure --with-storages=mdbox sdbox maildir mbox cydir
I guess I could try it myself also but maybe someone else wanting to reduce memory usage is quicker? :)
Related to that, you can look at where imap processes' memory goes with: http://dovecot.org/tools/linux-maps-parse-imap.pl I don't know how much memory those (writable!) file-backed memory areas actually use per-process. Maybe nothing, maybe something..
http://dovecot.org/tmp/proctest.sh can be used to easily test this. Looks like simply linking a binary with -lcrypto increases each process's memory usage by something like:
- RHEL5: 50 kB
- RHEL6/Debian/Ubuntu: 150-200 kB
- Solaris: 100 kB
- OpenBSD: 20 kB
- OSX: 0 kB
Looks like it's the linker that uses up all of this memory. I'm not sure why exactly though, I couldn't create a test library that had the same effect.
I also tried imap binary that was statically built vs. linked with libdovecot*.so, and the difference was pretty small (.so used maybe 10-20k more).
Anyway, looks like Dovecot can't link OpenSSL to imap/pop3 processes without wasting a ton of memory. In v2.2 I already moved imapc/pop3c backend code to plugins to avoid this. Looks like similar ugliness is needed for other features/backends also that may end up using SSL code. (We were wondering with Stephan what to do about his new HTTP library code that added support for SSL. It would be nice to keep it in the core libdovecot.so, but not if it links with SSL. So looks like we'll need some kind of a http-ssl plugin that is loaded only when needed.)
On Mon, 2012-11-05 at 23:40 +0200, Timo Sirainen wrote:
Anyway, looks like Dovecot can't link OpenSSL to imap/pop3 processes without wasting a ton of memory. In v2.2 I already moved imapc/pop3c backend code to plugins to avoid this. Looks like similar ugliness is needed for other features/backends also that may end up using SSL code. (We were wondering with Stephan what to do about his new HTTP library code that added support for SSL. It would be nice to keep it in the core libdovecot.so, but not if it links with SSL. So looks like we'll need some kind of a http-ssl plugin that is loaded only when needed.)
Implemented it a bit easier way that also gets rid of imapc/pop3c plugins and simplifies other things: lib-ssl-iostream now loads OpenSSL dynamically: http://hg.dovecot.org/dovecot-2.2/rev/68d21f872fd7
This also provides a nice abstraction to OpenSSL, making it again possible to implement other backends like GnuTLS or NSS. (Except login process code doesn't use lib-ssl-iostream yet.)
On 05/11/2012 23:22, Timo Sirainen wrote:
On Mon, 2012-11-05 at 23:40 +0200, Timo Sirainen wrote:
Anyway, looks like Dovecot can't link OpenSSL to imap/pop3 processes without wasting a ton of memory. In v2.2 I already moved imapc/pop3c backend code to plugins to avoid this. Looks like similar ugliness is needed for other features/backends also that may end up using SSL code. (We were wondering with Stephan what to do about his new HTTP library code that added support for SSL. It would be nice to keep it in the core libdovecot.so, but not if it links with SSL. So looks like we'll need some kind of a http-ssl plugin that is loaded only when needed.)
Implemented it a bit easier way that also gets rid of imapc/pop3c plugins and simplifies other things: lib-ssl-iostream now loads OpenSSL dynamically: http://hg.dovecot.org/dovecot-2.2/rev/68d21f872fd7
This also provides a nice abstraction to OpenSSL, making it again possible to implement other backends like GnuTLS or NSS. (Except login process code doesn't use lib-ssl-iostream yet.)
Does libtomcrypt implement enough?
Ed
On 6.11.2012, at 17.26, Ed W wrote:
On 05/11/2012 23:22, Timo Sirainen wrote:
On Mon, 2012-11-05 at 23:40 +0200, Timo Sirainen wrote:
Anyway, looks like Dovecot can't link OpenSSL to imap/pop3 processes without wasting a ton of memory. In v2.2 I already moved imapc/pop3c backend code to plugins to avoid this. Looks like similar ugliness is needed for other features/backends also that may end up using SSL code. (We were wondering with Stephan what to do about his new HTTP library code that added support for SSL. It would be nice to keep it in the core libdovecot.so, but not if it links with SSL. So looks like we'll need some kind of a http-ssl plugin that is loaded only when needed.)
Implemented it a bit easier way that also gets rid of imapc/pop3c plugins and simplifies other things: lib-ssl-iostream now loads OpenSSL dynamically: http://hg.dovecot.org/dovecot-2.2/rev/68d21f872fd7
This also provides a nice abstraction to OpenSSL, making it again possible to implement other backends like GnuTLS or NSS. (Except login process code doesn't use lib-ssl-iostream yet.)
Does libtomcrypt implement enough?
It doesn't do SSL, which is all Dovecot cares about.
On 11/6/2012 12:30 PM, Timo Sirainen wrote:
On 6.11.2012, at 17.26, Ed W wrote:
On 05/11/2012 23:22, Timo Sirainen wrote:
On Mon, 2012-11-05 at 23:40 +0200, Timo Sirainen wrote: This also provides a nice abstraction to OpenSSL, making it again possible to implement other backends like GnuTLS or NSS. (Except login process code doesn't use lib-ssl-iostream yet.)
Does libtomcrypt implement enough?
It doesn't do SSL, which is all Dovecot cares about.
Can the GnuTLS OpenSSL compatibility layer be used safely?
Am 12.11.2012 02:11, schrieb Daniel L. Miller:
On 11/6/2012 12:30 PM, Timo Sirainen wrote:
On 6.11.2012, at 17.26, Ed W wrote:
On 05/11/2012 23:22, Timo Sirainen wrote:
On Mon, 2012-11-05 at 23:40 +0200, Timo Sirainen wrote: This also provides a nice abstraction to OpenSSL, making it again possible to implement other backends like GnuTLS or NSS. (Except login process code doesn't use lib-ssl-iostream yet.)
Does libtomcrypt implement enough?
It doesn't do SSL, which is all Dovecot cares about.
Can the GnuTLS OpenSSL compatibility layer be used safely?
where is the problem with openssl?
please leave us in peace with gnuTLS and see how it affects OpenVAS / Greenbone Sceurity Assistant on distributions like Fedora the whole year
what about config compatibility like ssl_cipher_list = ALL:!LOW:!MEDIUM:!SSLv2:!MD5:!aNULL:!eNUL:!ADH:!AESGCM:!EXP:HIGH
On 2012-11-11 17:20, Reindl Harald wrote:
Am 12.11.2012 02:11,
schrieb Daniel L. Miller:
On 11/6/2012 12:30 PM, Timo Sirainen
wrote:
On 6.11.2012, at 17.26, Ed W wrote:
On
05/11/2012 23:22, Timo Sirainen wrote:
On Mon, 2012-11-05
at 23:40 +0200, Timo Sirainen wrote: This also provides a nice abstraction to OpenSSL, making it again possible to implement other backends like GnuTLS or NSS. (Except login process code doesn't use lib-ssl-iostream yet.)
Does libtomcrypt implement enough?
It
doesn't do SSL, which is all Dovecot cares about.
Can the GnuTLS
OpenSSL compatibility layer be used safely?
where is the problem
with openssl?
I don't know what the problem is - I just know that I've heard from a number of developers (including the Postfix & Dovecot developers) that they don't like OpenSSL - but while GnuTLS looks interesting they aren't interested in working on the interface - though they're willing to accept patches. (My full apologies right now if Timo or Wietse are offended by my speaking out of turn).
I'm no security expert, but I do know that OpenSSL has had issues with version compatiblity. I had a very troubled time during an OpenSSL/Postfix upgrade that left me non-functional until I found the exact version pairings required.
The tiny bit of Googling I've done tells me GnuTLS seems to be a more standards-compliant implementation, and MAY be "safer" than OpenSSL. However, as OpenSSL is the de-facto standard used by most Linux programs, acceptance of GnuTLS is quite limited. I've been intrigued by what I've read about it, and took a quick look at enabling support in Dovecot for GnuTLS directly - but while it didn't seem overly heavy at first glance the fact that Timo doesn't want to do it tells me I'm underestimating the complexity.
On Sun, 11 Nov 2012 20:13:54 -0800 Daniel L. Miller articulated:
I don't know what the problem is - I just know that I've heard from a number of developers (including the Postfix & Dovecot developers) that they don't like OpenSSL - but while GnuTLS looks interesting they aren't interested in working on the interface - though they're willing to accept patches. (My full apologies right now if Timo or Wietse are offended by my speaking out of turn).
I'm no security expert, but I do know that OpenSSL has had issues with version compatiblity. I had a very troubled time during an OpenSSL/Postfix upgrade that left me non-functional until I found the exact version pairings required.
The tiny bit of Googling I've done tells me GnuTLS seems to be a more standards-compliant implementation, and MAY be "safer" than OpenSSL. However, as OpenSSL is the de-facto standard used by most Linux programs, acceptance of GnuTLS is quite limited. I've been intrigued by what I've read about it, and took a quick look at enabling support in Dovecot for GnuTLS directly - but while it didn't seem overly heavy at first glance the fact that Timo doesn't want to do it tells me I'm underestimating the complexity.
I have OpenSSL 1.0.1c 10 May 2012 installed on a FreeBSD machine that also runs Postfix and Dovecot. When I first updated to the new version from then 0.9x branch there were some minor problems. I believe that there was something Wietse had to do to get Postfix fully functional in the new environment, but it was done extremely quickly. The biggest problem I faced was that I discovered that I had to recompile every program on my system that depended on the new version of Openssl. Once that was done, virtually every problem I experienced disappeared.
I am not aware of any developer who fears using the new version of Openssl, although apparently you do. The fact that a newer version of any software is not totally compatible with an older version is nothing new. I am amazed when they are fully compatible. Openssl is the de facto standard and I think that making a concerted effort to work with it would be a wise choice.
I have also Googled and have not found any evidence that GnuTLS is more "standards compliant" nor "safer". I would be interested in those URLs. I would like to know who is making those claims and what their basis for them actually is.
On 12/11/2012 04:13, Daniel L. Miller wrote:
The tiny bit of Googling I've done tells me GnuTLS seems to be a more standards-compliant implementation, and MAY be "safer" than OpenSSL. However, as OpenSSL is the de-facto standard used by most Linux programs, acceptance of GnuTLS is quite limited. I've been intrigued by what I've read about it, and took a quick look at enabling support in Dovecot for GnuTLS directly - but while it didn't seem overly heavy at first glance the fact that Timo doesn't want to do it tells me I'm underestimating the complexity.
Openssl is a *massive* project and I'm unsure that gnutls is much smaller... We should assume that both are quite scary from a "security" point of view. Licensing is the main thing which divides them, gnutls is stated as GPL compatible (however, the nominal incompatibility of openssl seems difficult to understand?)
OpenVPN integrated with PolarSSL and got Dutch government official approval for the combined package. I think elsewhere it's stated that openssl would not have been approved because something like the codebase was too large to inspect and sign off http://polarssl.org/news?item=0132
I haven't worked with PolarSSL, so no idea, but it's massively smaller codebase is likely attractive if you are the kind of person who actually *does* security audits on the software you run in secure situations.
Openssl is just a complete swiss army knife of tools!
Ed W
On 2012-11-12 2:31 PM, Ed W lists@wildgooses.com wrote:
Openssl is a *massive* project and I'm unsure that gnutls is much smaller... We should assume that both are quite scary from a "security" point of view. Licensing is the main thing which divides them, gnutls is stated as GPL compatible (however, the nominal incompatibility of openssl seems difficult to understand?)
OpenVPN integrated with PolarSSL and got Dutch government official approval for the combined package. I think elsewhere it's stated that openssl would not have been approved because something like the codebase was too large to inspect and sign off http://polarssl.org/news?item=0132
I haven't worked with PolarSSL, so no idea, but it's massively smaller codebase is likely attractive if you are the kind of person who actually *does* security audits on the software you run in secure situations.
Openssl is just a complete swiss army knife of tools!
Very interesting... I'd certainly be interested in switching to it, as long as the most common tools are supported
Is it considered a drop in replacement for OpenSSL?
On 12.11.2012, at 6.13, Daniel L. Miller wrote:
where is the problem with openssl?
I don't know what the problem is - I just know that I've heard from a number of developers (including the Postfix & Dovecot developers) that they don't like OpenSSL - but while GnuTLS looks interesting they aren't interested in working on the interface - though they're willing to accept patches. (My full apologies right now if Timo or Wietse are offended by my speaking out of turn).
OpenSSL documentation is very bad. Its API has some annoying missing features. For example you can load certificates from a directory or a file but not from anything else like from a string in memory. I had to copy&paste a few functions from OpenSSL code just to be able to do them.
The tiny bit of Googling I've done tells me GnuTLS seems to be a more standards-compliant implementation, and MAY be "safer" than OpenSSL. However, as OpenSSL is the de-facto standard used by most Linux programs, acceptance of GnuTLS is quite limited. I've been intrigued by what I've read about it, and took a quick look at enabling support in Dovecot for GnuTLS directly - but while it didn't seem overly heavy at first glance the fact that Timo doesn't want to do it tells me I'm underestimating the complexity.
I already once wrote GnuTLS support for Dovecot, but GnuTLS changed its APIs since then and it was probably originally already buggy. I think the only somewhat "special" APIs that Dovecot needs nowadays are related to reading cert/keys from memory instead of from files. If GnuTLS can do that, I don't think there's anything special in supporting it. Although it might be a bit complex to make it work properly asynchronously. istream-openssl was a bit annoying in that way (all the data read from the fd must be parsed and decoded all the way through to the SSL istream, regardless of any max buffer limits).
On 2012-11-12 20:44, Timo Sirainen wrote:
On 12.11.2012, at 6.13, Daniel L. Miller wrote:
The tiny bit of Googling I've done tells me GnuTLS seems to be a more standards-compliant implementation, and MAY be "safer" than OpenSSL. However, as OpenSSL is the de-facto standard used by most Linux programs, acceptance of GnuTLS is quite limited. I've been intrigued by what I've read about it, and took a quick look at enabling support in Dovecot for GnuTLS directly - but while it didn't seem overly heavy at first glance the fact that Timo doesn't want to do it tells me I'm underestimating the complexity.
I already once wrote GnuTLS support for Dovecot, but GnuTLS changed its APIs since then and it was probably originally already buggy. I think the only somewhat "special" APIs that Dovecot needs nowadays are related to reading cert/keys from memory instead of from files. If GnuTLS can do that, I don't think there's anything special in supporting it. Although it might be a bit complex to make it work properly asynchronously. istream-openssl was a bit annoying in that way (all the data read from the fd must be parsed and decoded all the way through to the SSL istream, regardless of any max buffer limits).
A while ago, I started working on GnuTLS support for Dovecot. While I didn't finish the implementation due to time constraints (the "abstract" API of lib-ssl-iostream is made for OpenSSL and you have to emulate some things), I think it would be possible to add GnuTLS support and with GnuTLS it would be possible to load X.509 certificates from memory.
Regards, Matthias-Christian
participants (8)
-
Charles Marcus
-
Daniel L. Miller
-
Ed W
-
Jerry
-
Matthias-Christian Ott
-
Patrick Domack
-
Reindl Harald
-
Timo Sirainen