Dynamic loaded libraries and shared global symbols

Question

Since I observed some strange behavior of global variables in my dynamically loaded libraries, I wrote the following test.

At first we need a statically linked library: The header test.hpp

#ifndef __BASE_HPP
#define __BASE_HPP

#include <iostream>

class test {
private:
  int value;
public:
  test(int value) : value(value) {
    std::cout << "test::test(int) : value = " << value << std::endl;
  }

  ~test() {
    std::cout << "test::~test() : value = " << value << std::endl;
  }

  int get_value() const { return value; }
  void set_value(int new_value) { value = new_value; }
};

extern test global_test;

#endif // __BASE_HPP

and the source test.cpp

#include "base.hpp"

test global_test = test(1);

Then I wrote a dynamically loaded library: library.cpp

#include "base.hpp"

extern "C" {
  test* get_global_test() { return &global_test; }
}

and a client program loading this library: client.cpp

#include <iostream>
#include <dlfcn.h>
#include "base.hpp"

typedef test* get_global_test_t();

int main() {
  global_test.set_value(2); // global_test from libbase.a
  std::cout << "client:        " << global_test.get_value() << std::endl;

  void* handle = dlopen("./liblibrary.so", RTLD_LAZY);
  if (handle == NULL) {
    std::cout << dlerror() << std::endl;
    return 1;
  }

  get_global_test_t* get_global_test = NULL;
  void* func = dlsym(handle, "get_global_test");
  if (func == NULL) {
    std::cout << dlerror() << std::endl;
    return 1;
  } else get_global_test = reinterpret_cast<get_global_test_t*>(func);

  test* t = get_global_test(); // global_test from liblibrary.so
  std::cout << "liblibrary.so: " << t->get_value() << std::endl;
  std::cout << "client:        " << global_test.get_value() << std::endl;

  dlclose(handle);
  return 0;
}

Now I compile the statically loaded library with

g++ -Wall -g -c base.cpp
ar rcs libbase.a base.o

the dynamically loaded library

g++ -Wall -g -fPIC -shared library.cpp libbase.a -o liblibrary.so

and the client

g++ -Wall -g -ldl client.cpp libbase.a -o client 

Now I observe: The client and the dynamically loaded library possess a different version of the variable global_test. But in my project I'm using cmake. The build script looks like this:

CMAKE_MINIMUM_REQUIRED(VERSION 2.6)
PROJECT(globaltest)

ADD_LIBRARY(base STATIC base.cpp)

ADD_LIBRARY(library MODULE library.cpp)
TARGET_LINK_LIBRARIES(library base)

ADD_EXECUTABLE(client client.cpp)
TARGET_LINK_LIBRARIES(client base dl)

analyzing the created makefiles I found that cmake builds the client with

g++ -Wall -g -ldl -rdynamic client.cpp libbase.a -o client

This ends up in a slightly different but fatal behavior: The global_test of the client and the dynamically loaded library are the same but will be destroyed two times at the end of the program.

Am I using cmake in a wrong way? Is it possible that the client and the dynamically loaded library use the same global_test but without this double destruction problem?

Answer 1

My first question is if there is any particular reason for which you both statically and dynamically (via dlopen) link the same code?

For your problem: -rdynamic will export the symbols from your program and what probably is happening is that dynamic linker resolves all references to your global variable to the first symbol it encounters in symbol tables. Which one is that I don't know.

EDIT: given your purpose I would link your program that way:

g++ -Wall -g -ldl client.cpp -llibrary -L. -o client

You may need to fix the order.

Answer 2

If using shared libraries you must define the stuff you want to export with macro like here. See DLL_PUBLIC macro definition in there.

Answer 3

By default, the linker won't combine a global variable (a 'D') in the base executable with one in a shared library. The base executable is special. There might be an obscure way to do this with one of those obscure control files that ld reads, but I sort of doubt it.

--export-dynamic will cause a.out 'D' symbols to be available to shared libs.

However, consider the process. Step 1: you create a DSO from a .o with a 'U' and a .a with a 'D'. So, the linker incorporates the symbol in the DSO. Step 2, you create the executable with a 'U' in one of the .o files, and 'D' in both a .a and the DSO. It will try to resolve using the left-to-right rule.

Variables, as opposed to functions, pose certain difficulties for the linker across modules in any case. A better practice is to avoid global var references across module boundaries, and use function calls. However, that would still fail for you if you put the same function in both the base executable and a shared lib.

Answer 4

g++ -Wall -g -ldl -rdynamic client.cpp libbase.a -o client

CMake adds -rdynamic option allowing loaded library to resolve symbols in the loading executable... So you can see that this is what you don't want. Without this option it just misses this symbol by accident.

But... You should not do any stuff like that there. Your libraries and executable should not share symbols unless they are really should be shared.

Always think of dynamic linking as static linking.

Answer 5

I would advise to use a dlopen(... RTLD_LAZY|RTLD_GLOBAL); to merge global symbol tables.