Dynamic linking a library - successful when executable is built, same setup fails when another .so is made

Question

I am writing a numerical code, in which I would like to use a third-party-written shared library. I am working on an x86_64 k8 architecture, under CentOS. The desired target that I would like to build would be either a Python or a Matlab extension module, which are, from what I understand, gcc-built dynamically-linked shared libraries, with extra Matlab/Python scaffolding built in. I'll focus on Python here, as the same problem happens, and it is probably more familiar to the community.

The library developer provided me originally with dynamic library and some test code written in C++. That's how he intended to distribute his library. I should make a note that I am trying to burden the original developer as little as possible, since the library probably being a side-project for him some time ago, and his test code ends up working OK. Therefore, I am trying to resolve issues on my end to the largest extent I can on my end. I managed to build his examples into executables, only with gcc 4.5.0. Usual gcc version that I use, 4.1.2, produced four "undefined reference to" errors during linking (with g++), specifically to _M_insert<long>(long), _M_widen_init(), _M_insert<double>(double) and __ostream_insert<char, std::char_traits<char> >. These are all part of std namespace. Compiling with g++ 4.5.0 resolved these undefined references and the example executables run correctly. Per E.R.'s comment, readelf -x.comment libmaxent_k8.so, indicates the original libraries were built with gcc 4.4.1.

To test whether linking to a Python extension works, I've built a small, Python extension function in C++ that just adds two numbers. Specifically, it doesn't use anything from the library I would like to use. The interface was SWIG 2.0 generated, compiled with g++ 4.50. and the code runs fine in Python 2.4.3. However, when I try to link to the original library, without ever referencing any symbols from it, the code again links fine, but then during runtime, while importing the extension, I get ImportError: libmaxent_k8.so: undefined symbol: _ZNSo9_M_insertIlEERSoT_, which, by c++filt, is the _M_insert(long), which is one of the original ones, that were undefined when the C++ code was linked using g++ 4.1.2.

I suspect the issue is with mismatched libstdc++ versions during linking and runtime of python, but I don't know how to resolve this. The best-case scenario for me would let me somehow get away without gcc 4.5.0 when linking the extensions, perhaps I was jumping ahead when resolving the original 4 missing references problem. Could the issue be resolved by bing built with somehow statically linking to libstdc++ 6.0.14 (which is a part of gcc 4.5.0) statically, while still retaining their character as dynamically linked libraries? Although, the Python has no problem of cooperating with gcc 4.5.0, Matlab does, and their support claims reliability only up to gcc 4.2.0. For this reason I would like to keep away from compiling with 4.5.0 as little as possible. My gcc comes with 6.0.8 version of libstdc++.

Here are some reports on the library in question. Remember, despite all these references, the code worked when compiled directly to an executable.

$ readelf -aD /home/mbudisic/lib64/libmaxent_k8.so | grep NEEDED
0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

$ ldd -d libmaxent_k8.so 
undefined symbol: _ZSt4cerr (./libmaxent_k8.so)
undefined symbol: _ZNSt8ios_base4InitD1Ev   (./libmaxent_k8.so)
undefined symbol: _ZSt4cout (./libmaxent_k8.so)
undefined symbol: __gxx_personality_v0  (./libmaxent_k8.so)
    libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002b92457f3000)
    libc.so.6 => /lib64/libc.so.6 (0x00002b9245a01000)
    /lib64/ld-linux-x86-64.so.2 (0x000000366e200000)

Running nm -uC libmaxent_k8.so results in 35 'U' labeled symbols, and two 'w' labeled symbols, from libm and libstdc++ libraries, which is far more than reported as undefined in linking/running.

Answer 1

Here is a very good write-up on why linking libstc++ statically may be not a very good idea:

http://www.trilithium.com/johan/2005/06/static-libstdc/

Answer 2

In general, glibc and libstdc++ try to provide backwards compatibility -- a binary built on an older system (or with older GCC) continues to run on a newer system.

The reverse: running binary linked against glibc-2.10 on a glibc-2.5 system, or running a binary built with GCC-4.4 against libstdc++ which came with GCC-4.2 is a non-goal.

Therefore, you must either ask for binaries built with the oldest GCC you want to support (4.2 in your case), or arrange for a new libstdc++ to be used (by modifying LD_LIBRARY_PATH, or by using LD_PRELOAD) for Matlab and Python.

The backward compatibility of glibc is very good, but libstdc++ has more spotty record; so Matlab's claim that only up to libstdc++.so.6.0.8 is stable and supported may well be true. You just have to try it, since (unless you get libmaxent_k8.so built with GCC-4.2) you don't have any other options (that I can think of).