Using libtool to load a duplicate function name from a shared library

Question

I'm trying to create a 'debug' shared library (i.e., .so or .dll file) that calls another 'real' shared library that has the same C API as the debug library (in this case, to emulate the PKCS#11 API). However, I'm running into trouble where the link map of the debug library is colliding with that of the real library and causing the debug library to call its own functions instead of the corresponding functions in the real library. I found a solution to this problem by using the POSIX dlmopen command, but would like to understand if the same is possible using GNU's libtool.

On my Solaris 10 system, the following code fails the assertion when a test application statically links to the debug library:

#include <dlfcn.h>
int MyFunctionName() {
  int (*function_ptr)();
  void *handle = dlopen("realsharedlibrary.so", RTDL_LAZY);
  *(void **)(&function_ptr) = dlsym(handle, "MyFunctionName");
  ASSERT(function_ptr != MyFunctionName); // Fails
  return (*function_ptr)();
}

In this case, I get a function pointer to the local 'MyFunctionName' (in the debug library) instead of MyFunctionName within the real shared library.

I've discovered that it's possible to get around this problem by using the command 'dlmopen' instead of 'dlopen', and telling dlmopen to create a new link map (with the LM_ID_NEWLM parameter) when loading the real library:

int MyFunctionName() {
  int (*function_ptr)();
  void *handle = dlmopen(LM_ID_NEWLM, "realsharedlibrary.so", RTDL_LAZY);
  *(void **)(&function_ptr) = dlsym(handle, "MyFunctionName");
  ASSERT(function_ptr != MyFunctionName); // succeeds
  return function_ptr(); // call real function
}

Unfortunately, dlmopen does not seem to be included within libtool (i.e., I don't see an lt_dlmopen function in libtool).

Is it possible to do the same thing using libtool commands -- that is, to create a new link map when loading the new library so that it doesn't collide with the link map of the debug library?

Answer 1

I haven't found a good way to use libtool to solve this problem yet, but there's a way to avoid the Solaris-specific 'dlmopen' function by using dlopen with these flags:

void *handle = dlopen("realsharedlibrary.so", RTLD_NOW | RTLD_GROUP | RTLD_LOCAL)

Apparently, the problem of symbol-collisions is solved by using RTLD_NOW instead of RTLD_LAZY and by adding RTLD_GROUP. The RTLD_LOCAL is there because POSIX requires using either RTLD_LOCAL or RTLD_GLOBAL, or the behavior is undefined. For Solaris, the behavior is to default to RTLD_LOCAL.

The open question, though, is whether it's possible to pass these types of flags to lt_dlopen.