[Dovecot] Dovecot 2.0.rc3 doesn't compile with custom OpenSSL

Hi folks,

when building Dovecot 2.0.rc3 (never used any other 2.0.X before, so this issue might not be new to that specific release), then compilation fails on anything that is linked against the new "libdovecot-login.so" library.

For example, "imap-login" fails:

/bin/sh ../../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wstrict-aliasing=2 -I/usr/local/ssl/include -o imap-login client.o client-authenticate.o imap-login-settings.o imap-proxy.o ../../src/login-common/libdovecot-login.la ../../src/lib-dovecot/libdovecot.la -lrt libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wstrict-aliasing=2 -I/usr/local/ssl/include -o .libs/imap-login client.o client-authenticate.o imap-login-settings.o imap-proxy.o ../../src/login-common/.libs/libdovecot-login.so ../../src/lib-dovecot/.libs/libdovecot.so -lrt -Wl,-rpath -Wl,/usr/local/Dovecot-2.0.rc3/lib/dovecot /usr/bin/ld: warning: libcrypto.so.1.0.0, needed by ../../src/login-common/.libs/libdovecot-login.so, not found (try using -rpath or -rpath-link) /usr/bin/ld: warning: libssl.so.1.0.0, needed by ../../src/login-common/.libs/libdovecot-login.so, not found (try using -rpath or -rpath-link) ../../src/login-common/.libs/libdovecot-login.so: undefined reference to OBJ_txt2nid' [...] collect2: ld returned 1 exit status make[3]: *** [imap-login] Error 1 make[3]: Leaving directory /usr/local/src/dovecot-2.0.rc3/src/imap-login'

I'm not using system OpenSSL but a custom installation in /usr/local/ssl. According to the documentation, Dovecot is smart enough to get everything it needs from the /usr/local/ssl/lib/pkgconfig/*.pc files. Dovecot 1.2.X just compiles fine if configured like this:

./configure --prefix=/usr/local/${DOVECOTVER} --with-ssl=openssl

The problem with Dovecot 2.0.rc3 is, that libdovecot-login.so isn't given the correct pathes for the OpenSSL libraries:

$ ldd src/login-common/.libs/libdovecot-login.so
linux-gate.so.1 =>  (0x00b6e000)
libssl.so.1.0.0 => not found
libcrypto.so.1.0.0 => not found
libdl.so.2 => /lib/libdl.so.2 (0x00b2a000)
librt.so.1 => /lib/librt.so.1 (0x004d4000)
libc.so.6 => /lib/libc.so.6 (0x00110000)
/lib/ld-linux.so.2 (0x006e7000)
libpthread.so.0 => /lib/libpthread.so.0 (0x009c6000)

So everything depending on libdovecot-login.so will fail. Setting "SSL_CFLAGS" and "SSL_LIBS" manually doesn't help because Dovecot already sets these perfectly fine via pkg-config:

env SSL_CFLAGS="-I/usr/local/ssl/include" SSL_LIBS="-L/usr/local/ssl/lib -Wl,-R/usr/local/ssl/lib -lcrypto -lssl" ./configure --prefix=/usr/local/${DOVECOTVER} --with-ssl=openssl

Only the hard way works:

env CPPFLAGS="-I/usr/local/ssl/include" LDFLAGS="-L/usr/local/ssl/lib -Wl,-R/usr/local/ssl/lib -lcrypto -lssl" ./configure --prefix=/usr/local/${DOVECOTVER} --with-ssl=openssl

However, setting CPPFLAGS and LDFLAGS shouldn't be necessary because SSL_CFLAGS and SSL_LIBS already contain the information that is needed to build a proper library. They are just not used when linking libdovecot-login.so ;-)

Greetings, Andreas