[Dovecot] SSL + compression?
OK, my interested is piqued now - does anyone have a recipe for how to make openssl compress the traffic before encrypting it? (Or perhaps it does by default?)
Ed W
Ed W wrote:
OK, my interested is piqued now - does anyone have a recipe for how to make openssl compress the traffic before encrypting it? (Or perhaps it does by default?)
Ed W
I don't think it does be default. The only what I know is to establish a compressed SSH tunnel to your server and then access the server over the tunnel. It will compress and give you an extra layer of encryption.
I don't think it does be default. The only what I know is to establish a compressed SSH tunnel to your server and then access the server over the tunnel. It will compress and give you an extra layer of encryption.
Umm, no. It will not compress. Think about it, encrypted data is fundamentally not compressible, that's the whole point.
johannes
On Fri, 20 Jun 2008, Johannes Berg wrote:
I don't think it does be default. The only what I know is to establish a compressed SSH tunnel to your server and then access the server over the tunnel. It will compress and give you an extra layer of encryption.
Umm, no. It will not compress. Think about it, encrypted data is fundamentally not compressible, that's the whole point.
x = length(compress(encrypt(data))) y = length(encrypt(compress(data))) z = length(encrypt(data))
Then, usually, x > y and z > y, but x is approximately the same as z. (That's speaking very generally; there may be optimizations in some case or another given your data.)
That is: encrypted data is less compressible than the original data, but if you compress first, then encrypt, it should be a win.
If I recall correctly, a "compressed SSH tunnel" would do 'y' (compress then encrypt). i.e., dovecot would see a non-SSL connection which gets compressed-then-encrypted, or decrypted-then-uncompressed by the endpoints of the tunnel.
Best, Ben
I don't think it does be default. The only what I know is to establish a compressed SSH tunnel to your server and then access the server over the tunnel. It will compress and give you an extra layer of encryption.
Umm, no. It will not compress. Think about it, encrypted data is fundamentally not compressible, that's the whole point.
x = length(compress(encrypt(data))) y = length(encrypt(compress(data))) z = length(encrypt(data))
Then, usually, x > y and z > y, but x is approximately the same as z. (That's speaking very generally; there may be optimizations in some case or another given your data.)
That is: encrypted data is less compressible than the original data, but if you compress first, then encrypt, it should be a win.
If I recall correctly, a "compressed SSH tunnel" would do 'y' (compress then encrypt). i.e., dovecot would see a non-SSL connection which gets compressed-then-encrypted, or decrypted-then-uncompressed by the endpoints of the tunnel.
Well, as far as I understood Marc, since he was saying "an extra layer of encryption" I understood him to mean that he wanted to
encrypt(compress(encrypt(imap stream)))
by building an ssh-tunnelled imaps (or imap/tls) connection. IOW, dovecot would see an SSL connection too.
johannes
On Fri, 20 Jun 2008, Johannes Berg wrote:
I don't think it does be default. The only what I know is to establish a compressed SSH tunnel to your server and then access the server over the tunnel. It will compress and give you an extra layer of encryption.
Umm, no. It will not compress. Think about it, encrypted data is fundamentally not compressible, that's the whole point.
[...me...]
Well, as far as I understood Marc, since he was saying "an extra layer of encryption" I understood him to mean that he wanted to
encrypt(compress(encrypt(imap stream)))
by building an ssh-tunnelled imaps (or imap/tls) connection. IOW, dovecot would see an SSL connection too.
Hmm, yes. I took it to mean that the 'encrypt' of
encrypt(compress(imap stream))
was the "extra layer". But, I think your interpretation is more easily arrived at, and if it's what Mark meant, you're absolutely right that the tunnel won't help.
A compressed SSH tunnel to regular, non-SSL IMAP should work to reduce traffic, though.
Best, Ben
IOW, dovecot would see an SSL connection too.
Hmm, yes. I took it to mean that the 'encrypt' of
encrypt(compress(imap stream))
was the "extra layer". But, I think your interpretation is more easily arrived at, and if it's what Mark meant, you're absolutely right that the tunnel won't help.
A compressed SSH tunnel to regular, non-SSL IMAP should work to reduce traffic, though.
Absolutely. And it'll even be considered 'secure' since local connections are secure.
OTOH, if you're going to the trouble to use ssh anyway, can Thunderbird do something like a "connect command"? I use that in evolution, and mine looks like something like this: ssh -C mailserver '/usr/sbin/dovecot --exec-mail imap'
where 'mailserver' really is an alias in my .ssh/config
johannes
Johannes Berg wrote:
I don't think it does be default. The only what I know is to establish a compressed SSH tunnel to your server and then access the server over the tunnel. It will compress and give you an extra layer of encryption.
Umm, no. It will not compress. Think about it, encrypted data is fundamentally not compressible, that's the whole point.
We are way off topic...
Back to the original question - discount SSH - how do we get compression
- SSL out of openssl..
Ed W
On Fri, 2008-06-20 at 23:04 +0100, Ed W wrote:
Johannes Berg wrote:
I don't think it does be default. The only what I know is to establish a compressed SSH tunnel to your server and then access the server over the tunnel. It will compress and give you an extra layer of encryption.
Umm, no. It will not compress. Think about it, encrypted data is fundamentally not compressible, that's the whole point.
We are way off topic...
:)
Back to the original question - discount SSH - how do we get compression + SSL out of openssl..
I don't think it's possible. OpenSSL says, in the NOTES section of SSL_COMP_add_compression_method(3):
The TLS standard (or SSLv3) allows the integration of
compression methods into the communication. The TLS RFC does
however not specify compression methods or their corresponding
identifiers, so there is currently no compatible way to
integrate compression with unknown peers. It is therefore
currently not recommended to integrate compression into
applications. Applications for non-public use may agree on
certain compression methods. Using different compression methods
with the same identifier will lead to connection failure.
johannes
Back to the original question - discount SSH - how do we get compression + SSL out of openssl..
I don't think it's possible. OpenSSL says, in the NOTES section of SSL_COMP_add_compression_method(3):
The TLS standard (or SSLv3) allows the integration of compression methods into the communication. The TLS RFC does however not specify compression methods or their corresponding identifiers, so there is currently no compatible way to integrate compression with unknown peers. It is therefore currently not recommended to integrate compression into applications. Applications for non-public use may agree on certain compression methods. Using different compression methods with the same identifier will lead to connection failure.
However, there is http://tools.ietf.org/html/draft-ietf-tls-compression, but openssl doesn't support that (only zlib and rle)
johannes
On Sat, 2008-06-21 at 00:13 +0200, Johannes Berg wrote:
Back to the original question - discount SSH - how do we get compression + SSL out of openssl..
I don't think it's possible. OpenSSL says, in the NOTES section of SSL_COMP_add_compression_method(3):
The TLS standard (or SSLv3) allows the integration of compression methods into the communication. The TLS RFC does however not specify compression methods or their corresponding identifiers, so there is currently no compatible way to integrate compression with unknown peers. It is therefore currently not recommended to integrate compression into applications. Applications for non-public use may agree on certain compression methods. Using different compression methods with the same identifier will lead to connection failure.
However, there is http://tools.ietf.org/html/draft-ietf-tls-compression, but openssl doesn't support that (only zlib and rle)
I'm way behind the times. http://www.faqs.org/rfc/rfc3749.txt
johannes
On Sat, 2008-06-21 at 00:17 +0200, Johannes Berg wrote:
On Sat, 2008-06-21 at 00:13 +0200, Johannes Berg wrote:
Back to the original question - discount SSH - how do we get compression + SSL out of openssl..
I don't think it's possible. OpenSSL says, in the NOTES section of SSL_COMP_add_compression_method(3):
The TLS standard (or SSLv3) allows the integration of compression methods into the communication. The TLS RFC does however not specify compression methods or their corresponding identifiers, so there is currently no compatible way to integrate compression with unknown peers. It is therefore currently not recommended to integrate compression into applications. Applications for non-public use may agree on certain compression methods. Using different compression methods with the same identifier will lead to connection failure.
However, there is http://tools.ietf.org/html/draft-ietf-tls-compression, but openssl doesn't support that (only zlib and rle)
I'm way behind the times. http://www.faqs.org/rfc/rfc3749.txt
Looking at OpenSSL code, I think the patch below will give 0.9.8 ability to support deflate compression. I'm not sure if I should include that to Dovecot though. At least not for v1.1. :) diff -r 68a0be847980 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Fri Jun 20 12:20:17 2008 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Sat Jun 21 04:29:51 2008 +0300 @@ -719,6 +719,7 @@ ssl_clean_free); SSL_library_init(); SSL_load_error_strings(); + (void)SSL_COMP_get_compression_methods(); extdata_index = SSL_get_ex_new_index(0, dovecot, NULL, NULL, NULL);
I'm way behind the times. http://www.faqs.org/rfc/rfc3749.txt
Looking at OpenSSL code, I think the patch below will give 0.9.8 ability to support deflate compression. I'm not sure if I should include that to Dovecot though. At least not for v1.1. :)
diff -r 68a0be847980 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Fri Jun 20 12:20:17 2008 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Sat Jun 21 04:29:51 2008 +0300 @@ -719,6 +719,7 @@ ssl_clean_free); SSL_library_init(); SSL_load_error_strings(); + (void)SSL_COMP_get_compression_methods();
extdata_index = SSL_get_ex_new_index(0, dovecot, NULL, NULL, NULL);
Huh? As far as I can tell that will just return NULL and have no side effects because you haven't previously added compression methods with SSL_COMP_add_compression_method. johannes
On Sat, 2008-06-21 at 09:28 +0200, Johannes Berg wrote:
I'm way behind the times. http://www.faqs.org/rfc/rfc3749.txt
Looking at OpenSSL code, I think the patch below will give 0.9.8 ability to support deflate compression. I'm not sure if I should include that to Dovecot though. At least not for v1.1. :)
diff -r 68a0be847980 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Fri Jun 20 12:20:17 2008 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Sat Jun 21 04:29:51 2008 +0300 @@ -719,6 +719,7 @@ ssl_clean_free); SSL_library_init(); SSL_load_error_strings(); + (void)SSL_COMP_get_compression_methods();
extdata_index = SSL_get_ex_new_index(0, dovecot, NULL, NULL, NULL);
Huh? As far as I can tell that will just return NULL and have no side effects because you haven't previously added compression methods with SSL_COMP_add_compression_method.
However, digging deeper, it appears that COMP_zlib _is_ actually "deflate", so adding SSL_COMP_add_compression_method(COMP_zlib()); should do the trick. Apparently it's compatible to gnutls too (see http://www.ietf.org/IESG/Implementations/rfc-3749-implementations.txt) johannes
On Sat, 2008-06-21 at 09:35 +0200, Johannes Berg wrote:
On Sat, 2008-06-21 at 09:28 +0200, Johannes Berg wrote:
I'm way behind the times. http://www.faqs.org/rfc/rfc3749.txt
Looking at OpenSSL code, I think the patch below will give 0.9.8 ability to support deflate compression. I'm not sure if I should include that to Dovecot though. At least not for v1.1. :)
diff -r 68a0be847980 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Fri Jun 20 12:20:17 2008 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Sat Jun 21 04:29:51 2008 +0300 @@ -719,6 +719,7 @@ ssl_clean_free); SSL_library_init(); SSL_load_error_strings(); + (void)SSL_COMP_get_compression_methods();
extdata_index = SSL_get_ex_new_index(0, dovecot, NULL, NULL, NULL);
Huh? As far as I can tell that will just return NULL and have no side effects because you haven't previously added compression methods with SSL_COMP_add_compression_method.
However, digging deeper, it appears that COMP_zlib _is_ actually "deflate", so adding
SSL_COMP_add_compression_method(COMP_zlib());
should do the trick. Apparently it's compatible to gnutls too (see http://www.ietf.org/IESG/Implementations/rfc-3749-implementations.txt)
At least OpenSSL 0.9.8[eg]'s SSL_COMP_get_compression_methods() adds all the compression methods on the first call. It doesn't return NULL with me.
On Sat, 2008-06-21 at 10:45 +0300, Timo Sirainen wrote:
On Sat, 2008-06-21 at 09:35 +0200, Johannes Berg wrote:
On Sat, 2008-06-21 at 09:28 +0200, Johannes Berg wrote:
I'm way behind the times. http://www.faqs.org/rfc/rfc3749.txt
Looking at OpenSSL code, I think the patch below will give 0.9.8 ability to support deflate compression. I'm not sure if I should include that to Dovecot though. At least not for v1.1. :)
diff -r 68a0be847980 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Fri Jun 20 12:20:17 2008 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Sat Jun 21 04:29:51 2008 +0300 @@ -719,6 +719,7 @@ ssl_clean_free); SSL_library_init(); SSL_load_error_strings(); + (void)SSL_COMP_get_compression_methods();
extdata_index = SSL_get_ex_new_index(0, dovecot, NULL, NULL, NULL);
Huh? As far as I can tell that will just return NULL and have no side effects because you haven't previously added compression methods with SSL_COMP_add_compression_method.
At least OpenSSL 0.9.8[eg]'s SSL_COMP_get_compression_methods() adds all the compression methods on the first call. It doesn't return NULL with me.
Hmm, ok, I was looking at 0.9.7 code. Maybe they did change the API then. johannes
On Sat, 2008-06-21 at 10:45 +0300, Timo Sirainen wrote:
However, digging deeper, it appears that COMP_zlib _is_ actually "deflate", so adding
SSL_COMP_add_compression_method(COMP_zlib());
should do the trick. Apparently it's compatible to gnutls too (see http://www.ietf.org/IESG/Implementations/rfc-3749-implementations.txt)
Oh, and the reason why I didn't even suggest the above was because the following code (0.9.8e):
int SSL_COMP_add_compression_method(int id, COMP_METHOD *cm) .. /* According to draft-ietf-tls-compression-04.txt, the compression number ranges should be the following:
0 to 63: methods defined by the IETF
64 to 192: external party methods assigned by IANA
193 to 255: reserved for private use */
if (id < 193 || id > 255)
{
SSLerr(SSL_F_SSL_COMP_ADD_COMPRESSION_METHOD,SSL_R_COMPRESSION_ID_NOT_WITHIN_PRIVATE_RANGE);
return 0;
}
That makes it impossible to add standard compression methods..
On Sat, 2008-06-21 at 10:50 +0300, Timo Sirainen wrote:
On Sat, 2008-06-21 at 10:45 +0300, Timo Sirainen wrote:
However, digging deeper, it appears that COMP_zlib _is_ actually "deflate", so adding
SSL_COMP_add_compression_method(COMP_zlib());
should do the trick. Apparently it's compatible to gnutls too (see http://www.ietf.org/IESG/Implementations/rfc-3749-implementations.txt)
Oh, and the reason why I didn't even suggest the above was because the following code (0.9.8e):
int SSL_COMP_add_compression_method(int id, COMP_METHOD *cm) .. /* According to draft-ietf-tls-compression-04.txt, the compression number ranges should be the following:
0 to 63: methods defined by the IETF 64 to 192: external party methods assigned by IANA 193 to 255: reserved for private use */
if (id < 193 || id > 255) { SSLerr(SSL_F_SSL_COMP_ADD_COMPRESSION_METHOD,SSL_R_COMPRESSION_ID_NOT_WITHIN_PRIVATE_RANGE); return 0; }
That makes it impossible to add standard compression methods..
Heh, yeah, obviously. Oh well. Apparently somebody was working on a patch to have the standard compression methods added automatically, so I suppose the best option might be to just wait for that (or see if it's already done) and hope everybody else adds them too...
johannes
On Sat, 2008-06-21 at 09:35 +0200, Johannes Berg wrote:
On Sat, 2008-06-21 at 09:28 +0200, Johannes Berg wrote:
I'm way behind the times. http://www.faqs.org/rfc/rfc3749.txt
Looking at OpenSSL code, I think the patch below will give 0.9.8 ability to support deflate compression. I'm not sure if I should include that to Dovecot though. At least not for v1.1. :)
diff -r 68a0be847980 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Fri Jun 20 12:20:17 2008 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Sat Jun 21 04:29:51 2008 +0300 @@ -719,6 +719,7 @@ ssl_clean_free); SSL_library_init(); SSL_load_error_strings(); + (void)SSL_COMP_get_compression_methods();
extdata_index = SSL_get_ex_new_index(0, dovecot, NULL, NULL, NULL);
Huh? As far as I can tell that will just return NULL and have no side effects because you haven't previously added compression methods with SSL_COMP_add_compression_method.
However, digging deeper,
and yet more, Thunderbird doesn't seem to support it: http://mxr.mozilla.org/seamonkey/source/security/nss/lib/ssl/ssl3con.c Neither does evolution, I think, since it too uses libnss. johannes
I don't think it does be default. The only what I know is to establish a compressed SSH tunnel to your server and then access the server over the tunnel. It will compress and give you an extra layer of encryption.
Umm, no. It will not compress. Think about it, encrypted data is fundamentally not compressible, that's the whole point.
johannes It will compress if you use a non-encrypted protocol while you are going
on 6-20-2008 12:46 PM Johannes Berg spake the following: through the tunnel.
-- MailScanner is like deodorant... You hope everybody uses it, and you notice quickly if they don't!!!!
On Fri, 2008-06-20 at 18:25 +0100, Ed W wrote:
OK, my interested is piqued now - does anyone have a recipe for how to make openssl compress the traffic before encrypting it? (Or perhaps it does by default?)
Incidentally, and you may complain about it being off-topic again, there's also an IMAP compression extension: http://tools.ietf.org/html/rfc4978
No idea if anybody implements it yet though.
johannes
participants (6)
-
Benjamin R. Haskell
-
Ed W
-
Johannes Berg
-
Marc Perkel
-
Scott Silva
-
Timo Sirainen