Hi all, I've just upgraded my old Solaris 10 update 8 to Solaris 10 update 11 with the latest patches, but after the reboot with the new update I'm having a lot of problems with dovecot.
My version is 2.2.13 ( it was the last one, at the time of the first server setup ).
I have seen that ( it seems ) the new solaris don't honour the LD_LIBRARY_PATH.
The first error was a relocation error: relocation error: file /usr/local/dovecot/lib/dovecot/libdovecot-login.so.0: symbol EVP_PKEY_get1_EC_KEY: referenced symbol not found Just to bypass this phase I have linked the openssl-1.0.1e in /usr/local/lib ( libssl and libcrypto ) but then I got a new relocation error:
imap: Error: dlopen(/usr/local/dovecot/lib/dovecot/lib20_fts_plugin.so) failed: ld.so.1: imap: fatal: relocation error: file /usr/local/dovecot/lib/dovecot/lib20_fts_plugin.so: symbol http_url_parse: referenced symbol not found
Did someone has some idea on how to bypass those troubles ?
Thanks in advance
Pierluigi
On 29/12/2018 13:49, Pierluigi Frullani wrote:
I've just upgraded my old Solaris 10 update 8 to Solaris 10 update 11 with the latest patches, but after the reboot with the new update I'm having a lot of problems with dovecot.
My version is 2.2.13 ( it was the last one, at the time of the first server setup ).
2.2.13 is from around May 2014. It worked but I can't see why you wouldn't switch to the latest 2.3.4. (You might be seeing what I can't and your question hasn't explained.)
I have seen that ( it seems ) the new solaris don't honour the LD_LIBRARY_PATH.
I'm sure it does but you shouldn't need it anyway.
The first error was a relocation error: relocation error: file /usr/local/dovecot/lib/dovecot/libdovecot-login.so.0: symbol EVP_PKEY_get1_EC_KEY: referenced symbol not found Just to bypass this phase I have linked the openssl-1.0.1e in /usr/local/lib ( libssl and libcrypto ) but then I got a new relocation error:
imap: Error: dlopen(/usr/local/dovecot/lib/dovecot/lib20_fts_plugin.so) failed: ld.so.1: imap: fatal: relocation error: file /usr/local/dovecot/lib/dovecot/lib20_fts_plugin.so: symbol http_url_parse: referenced symbol not found
Did someone has some idea on how to bypass those troubles ?
Did you compile this yourself or are you using someone else's package? Solaris has no files in /usr/local, you must have added those.
Do not put your files in /use/local. You will clash with someone else thinking it is the place to put personal stuff. man filesystem: "/opt Root of a subtree for add-on application packages."
Do not use LD_LIBRARY_PATH in the run time environment. Instead use the runpath in binaries as set during linking.
On configure set the local paths --prefix=/opt/XXX
--sysconfdir=/etc/opt/XXX
--localstatedir=/var/opt/XXX
--mandir=/opt/XXX/share/man
--docdir=/opt/XXX/share/doc/dovecot \When linking use -L to point to the libraries. These need not be in the installation location and during build won't be (because you haven't installed yet because you are building new libraries).
When linking use -R to point to the installed location of the libraries.
It should work if the paths are set correctly in the binaries. A generic package can use $ORIGIN. It's possible libtool is doing its usual trick of making a simple task difficult - I take measures to undo its wrong doing and set -L and -R between libtool and ld (cc -G). I suggest building the new dovecot and getting it right but if you want to diagnose your current installation here are some command hints you might find useful.
-- find dovecot: # find /opt -type f -name dovecot /opt/XXX/sbin/dovecot
-- is there a package? # pkginfo | grep dovecot # pkginfo | nawk '/dovecot/{print $2}' | while read package do pkginfo -l ${package} done
-- what is its run path? # dump -Lv /opt/XXX/sbin/dovecot | grep RUNPATH
-- can the runtime linker find the libraries? # ldd -r /opt/XXX/sbin/dovecot
Answer inline.
On Sun, Dec 30, 2018 at 12:59 PM James <list@xdrv.co.uk> wrote:
On 29/12/2018 13:49, Pierluigi Frullani wrote:
My version is 2.2.13 ( it was the last one, at the time of the first server setup ).
2.2.13 is from around May 2014. It worked but I can't see why you wouldn't switch to the latest 2.3.4. (You might be seeing what I can't and your question hasn't explained.)
That's the date for installation. I was using the courier-imap and switched to dovecot. Not changed since then.
I have seen that ( it seems ) the new solaris don't honour the LD_LIBRARY_PATH.
I'm sure it does but you shouldn't need it anyway.
Believe me, it doesn't :(
Did you compile this yourself or are you using someone else's package? Solaris has no files in /usr/local, you must have added those.
Yes I did compile it by myself using the "configure; make " way, with the following command: CPPFLAGS="-I/usr/local/clucene/include -I/usr/local/libtextcat/include -I/usr/local/openssl-1.0.1e/include" LDFLAGS="-L/usr/local/clucene/lib -L/usr/local/libtextcat/lib -L/usr/local/openssl-1.0.1e/lib" ./configure --prefix=/usr/local/dovecot --with-ssl=yes --with-stemmer --with-lucene --with-zlib
( also tried with --without-shared-libs or without it )
The problem ( as usual ) arise with the openssl libs that solaris ships, as they are always missing some parts ( basically for the export laws solaris adhere :) ) If I compile with the internal (open)ssl libs it won't even finish the compilation :(
- Do not put your files in /use/local. You will clash with someone
else thinking it is the place to put personal stuff. man filesystem: "/opt Root of a subtree for add-on application packages."
Being that I'm the only one administering this machine I'm sure it would not ;)
- Do not use LD_LIBRARY_PATH in the run time environment. Instead use
the runpath in binaries as set during linking.
That's the important point, but please read on.
- When linking use -L to point to the libraries. These need not be in the installation location and during build won't be (because you haven't installed yet because you are building new libraries).
- When linking use -R to point to the installed location of the libraries.
That's fine with me, ( and eventually I will try to modify all the makefiles, but I hoped it would have already been done by configure ( wich obviously has not :)
It should work if the paths are set correctly in the binaries. A generic package can use $ORIGIN. It's possible libtool is doing its usual trick of making a simple task difficult - I take measures to undo its wrong doing and set -L and -R between libtool and ld (cc -G).
I will try too ( but wouldn't be nice if the configure tool would do that for us ? )
-- what is its run path? # dump -Lv /opt/XXX/sbin/dovecot | grep RUNPATH
Interesting: root@puma dump -Lv dovecot | grep RUN [10] RUNPATH /usr/local/dovecot/lib/dovecot:/usr/local/lib:/usr/ccs/lib:/lib:/usr/lib:/usr/sfw/lib so it's obviously missing the openssl libraries
-- can the runtime linker find the libraries? # ldd -r /opt/XXX/sbin/dovecot
Yes, although not the one I want: root@puma ldd -r imap-login | egrep "ssl|crypto" libssl.so.1.0.0 => /usr/lib/libssl.so.1.0.0 libcrypto.so.1.0.0 => /usr/lib/libcrypto.so.1.0.0
At the moment I've applied a workaround by creating a fake script that change the library path on execution:
root@puma cat imap-login #!/bin/sh LD_LIBRARY_PATH=/usr/local/openssl-1.0.1e/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH exec /usr/local/dovecot/libexec/dovecot/imap-login.orig
and it's working but is not something I really like.
Would you ( or could ) show me where to replace -R and -L on Makefile(s) to fix the runpath trouble ?
I've tried the following: CPPFLAGS="-I/usr/local/clucene/include -I/usr/local/libtextcat/include -I/usr/local/openssl-1.0.1e/include" LDFLAGS="-L/usr/local/clucene/lib -L/usr/local/libtextcat/lib -L/usr/local/openssl-1.0.1e/lib -R/usr/local/openssl-1.0.1e/lib " ./configure --prefix=/usr/local/dovecot --with-ssl=yes --with-stemmer --with-lucene --with-zlib and it seems to work:
root@puma dump -Lv ./src/imap-login/.libs/imap-login | grep RUN [14] RUNPATH /usr/local/dovecot/lib/dovecot:/usr/local/lib:/usr/local/openssl-1.0.1e/lib:/usr/ccs/lib:/lib:/usr/lib:/usr/sfw/lib
I will try this weekend to install the new binary and see if it has done the magic.
BTW, thanks in advance for your time
Pigi
On Mon, 31 Dec 2018, Pierluigi Frullani wrote:
- When linking use -R to point to the installed location of the libraries.
That's fine with me, ( and eventually I will try to modify all the makefiles, but I hoped it would have already been done by configure ( wich obviously has not :)
One way to I got around the shared library version snarl (and this is especially relevant for openssl where I had many versions installed) is to configure the default library runpath to a directory that should be searched first e.g.
crle -l /mylibs:/lib:/usr/lib:/usr/local/lib
so that /mylibs will be searched first, then the rest. You can symlink your libraries there. However, be aware that all executables (unless overidden by LD_LIBRARY_PATH) will be affected. I also did this to make dovecot NFS fault tolerant as it didn't depend on shared libraries located on NFS filesystems.
Joseph Tam <jtam.home@gmail.com>
On 31/12/2018 17:28, Pierluigi Frullani wrote:
> My version is 2.2.13 ( it was the last one, at the time of the first > server setup ). 2.2.13 is from around May 2014. It worked but I can't see why you wouldn't switch to the latest 2.3.4. (You might be seeing what I can't and your question hasn't explained.)
That's the date for installation. I was using the courier-imap and switched to dovecot. Not changed since then.
So use the latest version.
> I have seen that ( it seems ) the new solaris don't honour the > LD_LIBRARY_PATH. I'm sure it does but you shouldn't need it anyway.
Believe me, it doesn't :(
Test:
$ cat > sub.c << EOF void sub() {} EOF $ cat > main.c << EOF void sub(); int main() { sub(); } EOF $ cc -G -o libsub.so sub.c -- link with no run path in the executable, fails with no other help $ cc main.c -L. -lsub $ ./a.out ld.so.1: a.out: fatal: libsub.so: open failed: No such file or directory zsh: killed ./a.out -- succeeds with LD_LIBRARY_PATH set $ LD_LIBRARY_PATH=. ./a.out -- link with a run path, succeeds on its own $ cc main.c -L. -R. -lsub $ ./a.out $ dump -Lv a.out | grep RUNPATH [5] RUNPATH .
-- to see the logic when finding a library use "ldd -s ..." $ LD_LIBRARY_PATH=. ldd -s ./a.out
find object=libsub.so; required by a.out
search path=. (LD_LIBRARY_PATH)
trying path=./libsub.so
libsub.so => ./libsub.so
-- clean up: $ rm sub.c main.c libsub.so a.out
The problem ( as usual ) arise with the openssl libs that solaris ships,
Build your own openssl.
1. Do not put your files in /use/local. You will clash with someone else thinking it is the place to put personal stuff. man filesystem: "/opt Root of a subtree for add-on application packages."
Being that I'm the only one administering this machine I'm sure it would not ;)
You will appreciate the separation of system and add-ons. Solaris 10 sparse zones can make /usr read only. It really is the "right" way.
Would you ( or could ) show me where to replace -R and -L on Makefile(s) to fix the runpath trouble ?
I've tried the following: CPPFLAGS="-I/usr/local/clucene/include -I/usr/local/libtextcat/include -I/usr/local/openssl-1.0.1e/include" LDFLAGS="-L/usr/local/clucene/lib -L/usr/local/libtextcat/lib -L/usr/local/openssl-1.0.1e/lib -R/usr/local/openssl-1.0.1e/lib " ./configure --prefix=/usr/local/dovecot --with-ssl=yes --with-stemmer --with-lucene --with-zlib and it seems to work:
root@puma dump -Lv ./src/imap-login/.libs/imap-login | grep RUN [14] RUNPATH /usr/local/dovecot/lib/dovecot:/usr/local/lib:/usr/local/openssl-1.0.1e/lib:/usr/ccs/lib:/lib:/usr/lib:/usr/sfw/lib
It would be easier if you put openssl and your other software under a common prefix, $PREFIX/lib, (not a subdirectory). The SONAME should sort out versions.
If you are doing a simple build you will get away with the standard handling of -L and -R.
$ PREFIX=/opt/PREFIX $ export LDFLAGS="-R/opt/FIRST/lib:${PREFIX}/lib" $ ./configure --prefix=${PREFIX} ...
Gives RPATH: /opt/PREFIX/lib/dovecot:/opt/FIRST/lib:/opt/PREFIX/lib which I think will work for you.
Problems with libtool are when we require:
- /opt/PREFIX/lib/$ISALIST at the front before /opt/PREFIX/lib/
- a non standard lib is needed before the normal one instead of libtool using /opt/PREFIX/lib as the first
In general it is easier to let libtool think it is helping but ignore it. Create a wrapper script around cc to rewrite the args back to what they were before configure+libtool rearranged, in this case drop the -L and -R given and substitute the values in $LDFLAGS.
More reading: https://blogs.oracle.com/solaris/ldlibrarypath-just-say-no-v2 https://docs.oracle.com/cd/E86824_01/html/E54763/ld.so.1-1.html https://docs.oracle.com/cd/E19683-01/816-1386/chapter3-13312/ Avoid crle, you don't need it and it affects the whole system. Hint, you might need this to unlock your machine: # LD_LIBRARY_PATH=/usr/lib rm /var/ld/ld.config
On 31 Dec 2018, at 19.28, Pierluigi Frullani <pierluigi.frullani@gmail.com> wrote:
Answer inline.
On Sun, Dec 30, 2018 at 12:59 PM James <list@xdrv.co.uk <mailto:list@xdrv.co.uk>> wrote: On 29/12/2018 13:49, Pierluigi Frullani wrote:
My version is 2.2.13 ( it was the last one, at the time of the first server setup ).
2.2.13 is from around May 2014. It worked but I can't see why you wouldn't switch to the latest 2.3.4. (You might be seeing what I can't and your question hasn't explained.)
That's the date for installation. I was using the courier-imap and switched to dovecot. Not changed since then.
I have seen that ( it seems ) the new solaris don't honour the LD_LIBRARY_PATH.
I'm sure it does but you shouldn't need it anyway.
Believe me, it doesn't :(
Believe me it does. I used to work for Sun Microsystems for 14 years in Solaris support and sustaining and I can guarantee you that it does.
You problem is that Solaris has concept of Secure Runtime Linker, and for trusted applications most of LD_CONFIG and LD_LIBRARY_PATH is ignored for security reasons.
For secure applications LD_LIBRARY_PATH components are ignored for non-secure directories.
Your dovecot is probably setuid or setgid and considered as secure application and secure runtime linker rules are triggered for it. Then /usr/local is completely ignored from LD_LIBRARY_PATH.
Sami
participants (4)
-
James
-
Joseph Tam
-
Pierluigi Frullani
-
Sami Ketola